JP2020086143A - Information processing system, information processing method, measurement system and program - Google Patents

Abstract

To easily generate information on a transfer characteristic of sound corresponding to a user.SOLUTION: An information processor 100 acquires a collected sound signal on the basis of collected sound by a microphone 103R attached to a right ear of a user and on the basis of collected sound of a plurality of sounds emitted from a plurality of directions at a periphery of the right ear. The information processor 100 acquires a collected sound signal on the basis of collected sound by a microphone 103L attached to a left ear of the user and on the basis of collected sound of the plurality of sounds emitted from the plurality of directions at a periphery of the left ear. The information processor 100 generates an HRTF set being characteristic information on a transfer characteristic of sound corresponding to the user on the basis of the collected sound signal acquired on the right ear and the collected sound signal acquired on the left ear.SELECTED DRAWING: Figure 2

Description

The present invention relates to a technique of processing information regarding sound transmission characteristics according to a user.

When a user hears a sound emitted from headphones or a speaker, there is a technique for localizing a sound image in an arbitrary direction to the user by using an acoustic signal processed by using a head-related transfer function according to the user. is there. As a method of measuring the head-related transfer function, sound for measurement is emitted to the user's head from various directions, and each sound is collected by an earplug type microphone attached to the user's ear. Then, there is a method of measuring the transfer characteristic of sound in each direction. Further, in Patent Document 1, by performing signal processing using transfer characteristics measured in a plurality of directions included in the median plane of the user and signal processing such as delay control and volume control, a small number of directions can be obtained. It is described that sound image localization is realized using the measurement results.

JP, 2002-218598, A

However, in the conventional method, it is not easy to generate information about the transfer characteristic because a large-scale dedicated apparatus is used to measure the head related transfer function representing the transfer characteristic of sound according to the user. For example, equipment for moving the position of the speaker on the median plane of the user in order to emit sound from the anechoic room or the user's head from multiple directions, or a plurality of speakers installed in multiple directions on the median plane. Was needed.

The present invention has been made in view of the above problems, and an object of the present invention is to enable easy generation of information regarding sound transmission characteristics according to a user.

In order to solve the above problems, an information processing system according to the present invention has, for example, the following configuration. That is, a sound pickup signal based on sound pickup at the position of one ear of the user, and a sound pickup signal based on sound pickup of a plurality of sounds emitted from a plurality of directions around the one ear is acquired. 1 acquisition means and a sound collection signal based on sound collection at the position of the other ear of the user, the sound collection signal being based on sound collection of a plurality of sounds emitted from a plurality of directions around the other ear Based on the sound collecting signal acquired by the first acquiring means, the sound collecting signal acquired by the first acquiring means, and the sound collecting signal acquired by the second acquiring means. And generating means for generating information.

According to the present invention, it is possible to easily generate information regarding sound transmission characteristics according to a user.

It is a figure which shows the structural example of an information processing system. It is a figure which shows the structural example of a measurement system. It is a figure which shows the hardware structural example of an information processing apparatus. It is a flow chart which shows an example of operation of an information processor. It is a figure which shows another structural example of a measurement system. It is a figure which shows the modification of an information processing system. It is a flow chart which shows an example of operation of an information processing system.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and all combinations of the features described in the present embodiment are not necessarily essential to the solving means of the present invention. The same configurations will be described with the same reference numerals.

[Configuration of information processing system]
FIG. 1 is a diagram showing a configuration example of an information processing system 10 according to the present embodiment. The information processing system 10 is capable of localizing a sound image in an arbitrary direction for a user by generating characteristic information regarding a sound transmission characteristic according to the user and processing the acoustic signal using the information. It is a system that realizes three-dimensional sound. In the present embodiment, a case will be mainly described in which information indicating a head-related transfer function (HRTF) is used as the characteristic information. However, the information processing system 10 may use other characteristic information such as information indicating a head impulse response.

The information processing system 10 includes a measurement system 20, an information processing device 100, a storage unit 106, and an acoustic signal processing unit 107 (hereinafter referred to as the processing unit 107). Although a single processing unit 107 is connected to the storage unit 106 as an example in FIG. 1, a plurality of processing units 107 may be connected to the storage unit 106. Similarly, a plurality of storage units 106 may be connected to the information processing device 100. Although the storage unit 106 and the processing unit 107 are configured as independent devices in the present embodiment, the present invention is not limited to this. For example, the storage unit 106 and the processing unit 107 may be integrally configured, or at least one of the storage unit 106 and the processing unit 107 may be included in the information processing apparatus 100. The configuration of the information processing device 100 is not limited to that shown in FIG. 1, and the signal output unit 101 may be mounted in a device different from the information processing device 100, for example.

The measurement system 20 is a measurement system for measuring the transfer characteristic of sound according to the auricle of the user, and includes a sounding device 102L, a sounding device 102R, a microphone 103L, and a microphone 103R. As shown in FIG. 2A, the sounding device 102L is mounted by the fixing portion 210 so as to surround the left ear of the user. In the present embodiment, as shown in FIG. 2A, it is assumed that the sounding device 102L has a ring shape and the user's ear is not covered. However, the sound producing device 102L is not limited to this, and may have a shape that hermetically seals the user's ear like an earpiece of a headphone. In this case, the sound producing device 102L may be provided with a member having a high sound absorbing property. The microphone 103L is a microphone that can be attached to the position of the hole of the user's left ear, and is installed in, for example, an earplug type device that closes the entrance of the ear hole so that a small microphone faces the outside of the ear hole.

As shown in FIG. 2B, in the sound producing device 102L, eight sound producing units from speaker 201-1 to speaker 201-8 are installed so as to face the center of the ring. Therefore, when the sound generator 102L is attached to the user by the fixing unit 210, the eight small speakers are located in different directions with respect to the user's left ear. Hereinafter, these small speakers will be referred to as speakers 201 unless otherwise specified. In the present embodiment, the eight speakers 201 are located in eight directions substantially perpendicular to the position of the microphone 103L in the direction connecting the left and right ears of the user. That is, the eight speakers 201 and the microphone 103L are located on substantially the same plane. However, the arrangement of the speaker 201 included in the sound producing device 102L is not limited to this. Further, the number of speakers 201 included in the sound producing device 102L is not limited to this, and the sound producing device 102L may have one or more speakers 201.

The plurality of speakers 201 emit sound to the microphone 103L from a plurality of different directions in response to the input of a signal from the signal output unit 101 of the information processing apparatus 100. Then, the microphone 103L outputs a sound collection signal obtained by collecting the sound emitted by the speaker 201 to the transfer characteristic estimation unit 104 (hereinafter referred to as the estimation unit 104) of the information processing device 100.

Like the sounding device 102L, the sounding device 102R has a plurality of speakers 201 and is mounted so as to surround the right ear of the user. Like the microphone 103L, the microphone 103R is attached to the position of the hole of the user's right ear. In the present embodiment, the number of speakers 201 included in the sounding device 102R is equal to the number of speakers 201 included in the sounding device 102L. However, the sound producing device 102L and the sound producing device 102R do not have to have the same configuration. Note that the sound producing device 102L, the sound producing device 102R, the microphone 103L, and the microphone 103R are not limited to those mounted in the form shown in FIG. The sound generators 102L and 102R may be arranged such that the speakers 201 are located around the user's ears, and the microphones 103L and 103R may be located near the holes of the user's ears. .. For example, each of these devices may be supported by an arm that does not contact the user.

In the present embodiment, the sound source direction with respect to the user's ear, which corresponds to the direction of the speaker 201 with respect to the microphone 103L, is represented by a rising angle and a lateral angle. The rising angles in the direction of the plurality of speakers 201 included in the sounding device 102R with respect to the microphone 103R are substantially equal to the rising angles in the direction of the plurality of speakers 201 included in the sounding device 102L with respect to the microphone 103L. The lateral angle α and the ascending angle β are expressed as follows using the azimuth angle ψ and the elevation angle θ with respect to the user's head.

The information processing device 100 includes a signal output unit 101, an estimation unit 104, and a characteristic information generation unit 105 (hereinafter referred to as the generation unit 105). The signal output unit 101 generates a signal for measuring the transfer characteristic, such as TSP (Time Stretched Pulse) or MLS (Maximum Length Sequence), and outputs the signal to the sound generator 102R and the sound generator 102L.

The estimation unit 104 acquires the sound collection signal output from the microphone 103L, and estimates the sound transfer characteristic based on the sound collection signal and the measurement signal output from the signal output unit 101. The transfer characteristic obtained by this estimation reflects the influence of the user's left auricle or the like until the sound emitted from the speaker 201 of the sound generator 102L is picked up by the microphone 103L. Further, the estimation unit 104 acquires the sound collection signal output from the microphone 103R, and estimates the transfer characteristic of the sound based on the sound collection signal and the measurement signal output from the signal output unit 101. The transfer characteristic obtained by this estimation reflects the influence of the user's left auricle or the like until the sound emitted from the speaker 201 of the sound generator 102L is picked up by the microphone 103L. Unless the shape of the user's right and left ears is the same, the transfer characteristic obtained based on the measurement at the right ear and the transfer characteristic obtained based on the measurement at the left ear are different.

The generation unit 105 generates an HRTF set, which is characteristic information regarding the transfer characteristic of sound corresponding to the head of the user, based on the estimation result of the transfer characteristic corresponding to each of the left and right ears by the estimation unit 104. The HRTF set includes HRTFs corresponding to a plurality of directions for the user. In particular, the HRTF set generated by the estimation unit 104 includes HRTFs corresponding to the directions included in the median plane of the user. By using such an HRTF, the sound image can be localized in the designated direction with the median plane of the user as a reference. Note that HRTFs corresponding to other directions not included in the median plane may be included in the HRTF set. The generated HRTF set is output from the generation unit 105 and stored in the storage unit 106.

For example, the estimation unit 104 determines the rising angle (90°) in the direction of the speaker 201-1 with respect to the microphone 103L based on the sound collection signal corresponding to the sound emitted from the one speaker 201-1 included in the sound generator 102L. Estimate the characteristic information of the corresponding left ear. In addition, the estimation unit 104 estimates the characteristic information of the right ear corresponding to the same rising angle (90°), based on the sound collection signal corresponding to the sound emitted from the corresponding speaker 201 included in the sound generator 102R. .. And the production|generation part 105 produces|generates HRTF corresponding to the same ascending angle (90 degree) in a median surface based on the estimation result of those characteristic information. By performing these processes for each of the eight speakers 201 included in the sound generator 102L, an HRTF set including HRTFs in eight directions on the median plane is generated.

The storage unit 106 stores the HRTF set output from the generation unit 105 in a storage medium such as a hard disk or a memory. When the measurement system 20 and the information processing apparatus 100 generate HRTF sets for each of a plurality of users, the storage unit 106 may store each HRTF set together with user identification information.

The processing unit 107 acquires an acoustic signal from an external device and outputs a processed acoustic signal obtained by processing the acoustic signal based on the HRTF set acquired from the storage unit 106. For example, stereophonic sound can be realized by inputting the output processed acoustic signal to the headphones worn by the user who is the measurement target of the measurement system 20. The acoustic signal to be processed is not limited to signal acquisition from an external device, but may be acquired by being generated by the processing unit 107.

[Configuration of information processing device]
Next, the hardware configuration of the information processing device 100 will be described with reference to FIG. The information processing device 100 includes a CPU 301, a ROM 302, a RAM 303, an auxiliary storage device 304, a display unit 305, an operation unit 306, a communication I/F 307, and a bus 308.

The CPU 301 implements each functional unit of the information processing apparatus 100 shown in FIG. 1 by controlling the entire information processing apparatus 100 using computer programs and data stored in the ROM 302 and the RAM 303. Note that the information processing apparatus 100 may include one or a plurality of dedicated hardware different from the CPU 301 and the dedicated hardware may execute at least a part of the processing by the CPU 301. Examples of dedicated hardware include ASICs (application specific integrated circuits), FPGAs (field programmable gate arrays), and DSPs (digital signal processors). The ROM 302 stores programs and parameters that do not need to be changed. The RAM 303 temporarily stores programs and data supplied from the auxiliary storage device 304, data externally supplied via the communication I/F 307, and the like. The auxiliary storage device 304 is composed of, for example, a hard disk drive, and stores various data such as image data, audio data, and virtual camera path information.

The display unit 305 includes, for example, a liquid crystal display, an LED, and the like, and displays a GUI (Graphical User Interface) for a user to operate the information processing apparatus 100. The operation unit 306 includes, for example, a keyboard, a mouse, a touch panel, etc., and inputs various instructions to the CPU 301 in response to an operation by the user. The communication I/F 307 is used for communication with external devices such as the sounding device 102L and the storage unit 106. For example, when the information processing apparatus 100 is connected to an external device by wire, a communication cable is connected to the communication I/F 307. When the information processing device 100 has a function of wirelessly communicating with an external device, the communication I/F 307 includes an antenna. The bus 308 connects the respective units of the information processing apparatus 100 to transfer information.

In the present embodiment, the display unit 305 and the operation unit 306 are assumed to exist inside the information processing device 100, but the information processing device 100 may not include at least one of the display unit 305 and the operation unit 306. Further, at least one of the display unit 305 and the operation unit 306 exists outside the information processing apparatus 100 as another device, and the CPU 301 controls the display control unit that controls the display unit 305 and the operation unit 306. You may operate as an operation control part.

[Processing Flow of Information Processing Device]
The processing flow of the information processing apparatus 100 will be described with reference to FIG. The process illustrated in FIG. 4 is started, for example, at the timing when the user mounts the measurement system 20 and the instruction to start the measurement is input to the information processing apparatus 100. However, the start timing of the process shown in FIG. 4 is not limited to the above. The processing illustrated in FIG. 4 is realized by the CPU 301 loading a program stored in the ROM 302 into the RAM 303 and executing the program. Note that at least a part of the processing illustrated in FIG. 4 may be realized by one or a plurality of dedicated hardware different from the CPU 301. The same applies to the processing shown in the flowchart of FIG. 7 described later.

When the measurement processing is started, the information processing apparatus 100 performs the processing from S401 to S403 for each of the left and right ears for each speaker 201 in the sound source direction that is the measurement target. Specifically, the processes from S401 to S403 are performed on each of the eight speakers 201 included in the sounding device 102L, and then the processes from S401 to S403 are performed on each of the eight speakers 201 included in the sounding device 102R. Here, an example in which the left ear is measured first is shown, but the present invention is not limited to this, and the right ear may be measured first. Further, for example, the measurement using the speaker 201 included in the sounding device 102L and the measurement using the speaker 201 included in the sounding device 102R may be alternately performed.

In S401, the signal output unit 101 outputs a predetermined measurement signal to the sounding device 102L having the speaker 201 to be measured. The signal output unit 101 also outputs the measurement signal to the estimation unit 104. In S402, the estimation unit 104 acquires a sound collection signal based on sound collected by the microphone 103L according to the output of the measurement signal in S401. In S403, the estimation unit 104 estimates the transfer characteristic of the sound based on the sound collection signal acquired in S402 and the measurement signal output from the signal output unit 101 in S401. The transfer characteristic obtained by this estimation is a transfer characteristic reflecting the influence of the auricle of the left ear or the like until the sound emitted from the direction corresponding to the speaker 201 to be measured reaches the hole of the left ear. The transfer characteristic estimation result is output to the generation unit 105.

When the processes of S401 to S403 are completed for each of the plurality of speakers 201 included in the sounding device 102L, the processes of S401 to S403 are performed for the speaker 201 included in the sounding device 102R. That is, as in the case of the left ear, the estimation unit 104, based on the sound collection signal acquired in the measurement in the right ear and the measurement signal corresponding to the sound collection signal, the sound corresponding to the direction of each speaker 201. Estimate the transfer characteristics of. In this way, the sound collection period using the sounding device 102L and the microphone 103L and the sound collection period using the sounding device 102R and the microphone 103R are set so as not to overlap. As a result, it is possible to suppress the occurrence of crosstalk in which the sound emitted from the speaker 201 is picked up by the microphone attached to the other ear around the one ear. However, in order to perform the measurement in a short time, these sound collection periods may partially overlap.

In step S<b>404, the generation unit 105 determines, based on the estimation result of the sound transfer characteristic of the right ear and the estimation result of the sound transfer characteristic of the left ear acquired from the estimation unit 104, in the plurality of directions included in the median plane. An HRTF set including a plurality of corresponding HRTFs is generated. In step S405, the generation unit 105 outputs the generated HRTF set and performs storage control to store the HRTF set in the storage unit 106.

As described above, the information processing apparatus 100 according to the present embodiment is a sound collection signal based on a sound collected by the microphone 103R attached to the right ear of the user, and a plurality of sound signals generated from a plurality of directions around the right ear. A sound pickup signal based on the sound pickup of the sound is acquired. Further, the information processing apparatus 100 is a sound collection signal based on sound collection by the microphone 103L attached to the left ear of the user, and is based on sound collection of a plurality of sounds emitted from a plurality of directions around the left ear. Obtain a sound signal. Then, the information processing apparatus 100 generates an HRTF set, which is characteristic information regarding the transfer characteristic of the sound according to the user, based on the sound collection signal acquired for the right ear and the sound collection signal acquired for the left ear.

According to the above configuration, since it is not necessary to use a large facility for emitting sounds from a plurality of directions on the median plane with respect to the user's head, it is possible to easily generate the HRTF set. Further, by providing a member having a high sound absorbing property in a portion surrounding the user's ears in the sound producing devices 102L and 102R, it is possible to easily realize the measurement which is hardly affected by noise as in the measurement using the anechoic chamber. ..

In the present embodiment, a case has been mainly described in which the directions of the plurality of speakers 201 included in the sounding device 102L with respect to the microphone 103L and the directions of the plurality of speakers 201 included in the sounding device 102R with respect to the microphone 103R are symmetrical. This facilitates the process of generating the HRTF set based on the measurement result of the right ear and the measurement result of the left ear. However, the sound producing device 102L and the sound producing device 102R do not have to be symmetrical.

When the positional relationship between the sounding device 102L and the microphone 103L is different from the positional relationship between the sounding device 102R and the microphone 103R, the estimation unit 104 uses the transfer characteristics estimated for the left and right ears as the lateral angle and the rising angle of the speaker 201. , And the distance to the microphone. For this correction, it is possible to use the result of detecting the deviation between the two sounding devices with respect to the front-back axis, the left-right axis, and the vertical axis of the user's head during measurement. For example, the mounting state (tilt or center position shift) of each sounding device can be acquired based on an image of a user who has worn the sounding device 102L and the sounding device R. Alternatively, the sound producing device 102L and the sound producing device 102R may be provided with an attitude sensor or the like so that the wearing state of each sound producing device can be acquired. By using the shift information acquired in this way, the transfer characteristic can be corrected.

For example, when the rising angles in the sound source direction are different in the measurement in the left and right ears, when an acoustic signal processed based on the HRTF set generated using the uncorrected transfer characteristic estimation result is reproduced, a blur or a sound image is generated. There is a risk of being localized in the head. Therefore, the estimation unit 104 estimates the transfer characteristics corresponding to the same rising angle for the left and right ears by interpolating the transfer characteristics in the rising angle direction.

Further, when the distance from the sound producing device 102L to the microphone 103L and the distance from the sound producing device 102R to the microphone 103R are different, the sound to be reproduced may be shifted to the right or left. This is because a time difference and a level difference occur in the transfer functions for the left and right ears, and the sound shifts to the left and right due to the preceding sound effect. Therefore, the estimation unit 104 corrects the transfer characteristic with respect to the delay time and the volume. As a correction method related to time, for example, there is a method of detecting a peak at which the absolute value of the amplitude of the time waveform of the transfer characteristic estimated by the estimation unit 104 is maximum and adjusting the time. Alternatively, a method of aligning the left and right delays may be used by using the minimum phase component of the left and right transfer functions. Alternatively, the loudness of the left and right transfer characteristics may be corrected so as to be within a predetermined range. Alternatively, the left/right shift may be adjusted according to a user operation on an operation unit such as an adjustment knob.

Further, even when the lateral angles in the sound source direction are different in the measurement in the left and right ears, the sound image may be blurred or the sound image may be localized in the head. This is because the transfer characteristics are less likely to be affected by the auricle as the lateral angle approaches the left-right axis (the axis passing through both ears), and the notches and peaks of the HRTF are attenuated. Therefore, the estimation unit 104 may correct the estimated left and right transfer characteristics based on the magnitude of spectral cues such as the first peak, the second peak, the first notch, and the second notch. For example, the spectrum may be emphasized so that the average of the heights of the first peak and the second peak is the same on the left and right sides.

In order to reduce the correction processing as described above, it is desirable that the speaker 201 of the sound producing device 102L and the speaker 201 of the sound producing device 102R be arranged at the same rising angle. Further, since the HRTF spectral cue on the median plane is affected by the shape of the auricle, the speakers 201 of the sounding devices 102L and 102R are arranged so as to be equidistant from the ear hole on the sagittal plane including the user's auricle. It is desirable to be done. In order to realize this, the information processing system 10 may display the mounting position of the sounding device 102L and the sounding device 102R worn by the user on the display unit, or may notify the user to correct the mounting position. Good.

In the present embodiment, the description has been centered on the case where the generation unit 105 generates an HRTF corresponding to the direction included in the median plane, but the generation unit 105 performs panning processing or the like on the HRTF on the median plane, An HRTF in a direction not included in the median plane may be generated. Then, the generation unit 105 may store the HRTF set including the HRTFs in the storage unit 106. For example, based on the estimated transfer characteristics of the same rising angle on the left and right, the sound image localization to the arbitrary lateral direction of the same rising angle can be realized by panning.

In the present embodiment, the speaker 201 of the sound producing device 102L and the sound producing device 102R is located outside the auricle, and the microphone 103L and the microphone 103R are located in the ear canal, but the measurement is performed. The positional relationship may be reversed as in the reciprocal method. That is, the measurement may be performed in a state where the speaker is located in the ear hole and the plurality of microphones are located outside the auricle. Further, in the present embodiment, the plurality of speakers 201 in the sound producing device 102L and the sound producing device 102R are arranged in an annular shape. However, it suffices that the plurality of speakers 201 be located around the ear canal, and the plurality of speakers 201 may be arranged in, for example, a U-shape, a square, or a triangle.

Further, in the present embodiment, the case where two sound producing devices and two microphones are used, such as using the sound producing device 102L and the microphone 103L for the left ear and using the sound producing device 102R and the microphone 103R for the right ear, has been described. .. However, at least one of the sound producing device and the microphone may be one. In this case, the transfer characteristic can be measured in each of the left and right ears by replacing at least one of the sound producing device and the microphone on the left and right.

The headphone-shaped measuring device including the sounding device 102L and the sounding device 102R as shown in FIG. 2 is small and excellent in portability, and the transfer characteristic can be easily measured. However, the configuration of the device used for measuring the transfer characteristic is not limited to this. FIG.3(b) shows the structure of the sounding apparatus 500 which can be used instead of the sounding apparatus 102R and the sounding apparatus 102L in the measuring system 20 for measuring the transfer characteristic of the sound according to a user's pinna. .. Two sound generators 500 may be used for the right ear and one for the left ear, or one sound generator 500 may be used for both the measurement in the right ear and the measurement in the left ear. Inside the case 504 of the sounding device 500 having the hole 502, a speaker 501 that emits sound in response to a signal input, and a sound absorbing member 503 that suppresses the reflection of sound in the case 504 are provided. Further, the sounding device 500 has a configuration such as a rail or a motor (not shown) for moving the speaker 501 in a circular path around the hole so that the position of the speaker 501 with respect to the position of the hole 502 of the case 504 changes. Have.

When performing a measurement in the right ear using the sounding device 500, the user mounts the microphone 103R on the right ear and inserts the right ear into the hole 502, as shown in FIG. Then, the speaker 501 included in the sound producing device 500 changes the position with respect to the microphone 103R and sequentially emits sounds from a plurality of directions, and the sounds are collected by the microphone 103R. The process after collecting the sound is the same as that of the above-described embodiment. According to such a method using the sounding device 500, it is possible to perform highly accurate measurement close to that in an anechoic room.

In the example of FIG. 5, it is assumed that the sound is emitted from the plurality of directions to the microphone 103R when one speaker 501 moves. With such a configuration, it is possible to suppress an increase in cost due to mounting a large number of microphones on the sounding device 500. However, not limited to this, for example, the sounding device 500 may include a plurality of speakers 501, and each speaker 501 may emit a sound while moving. Further, for example, similar to the sound producing device 102R described above, the plurality of speakers 501 located in a plurality of directions with respect to the position of the ear may emit sound without moving. According to this configuration, since a rail and a motor for moving the speaker 501 are not required, the sounding device 500 can be downsized.

Note that the inclination of the user's head may change depending on whether the user inserts the right ear into the hole 502 or the left ear. Therefore, the information processing apparatus 100 may perform the above-described correction process on the transfer function estimated based on the measurement. As another method, when the user puts his/her ear into the hole 502, the tilt of the user's head is detected by a camera or a sensor, and the rising angle in the direction of the speaker 501 during left/right measurement is equalized. The device 500 may adjust the position of the speaker 501. According to this method, the correction process after estimating the transfer characteristic can be reduced.

In addition, in this embodiment, in order to generate the HRTF of the median plane, the measurement is performed using the speakers 201 in a plurality of directions on the sagittal plane including the auricle. However, the present invention is not limited to this. For example, in the sound producing device 102L and the sound producing device 102R, a plurality of speakers 201 may be three-dimensionally arranged on a hemispherical surface centered on the ear position. According to such a configuration, the transfer characteristic can be measured with a fine directional resolution. Similarly, in the sounding device 500, the speakers may be positioned not only in one lateral direction but also in a plurality of lateral directions. In these cases, the measurement is not limited to the measurement with the same configuration for both ears, and the measurement may be performed with a fine directional resolution only in the measurement in any one of the ears.

If you want to measure with a high directional resolution for one ear, perform measurement with a close directional resolution in the ear on which you want to localize the sound image during playback of the acoustic signal, and use the data based on that measurement to improve Improves localization. On the other hand, regarding the transfer characteristics of the ear on the side where the sound image is to be localized and on the side opposite to the side, data based on measurement on the sagittal plane including the auricle of the same rising angle or data close thereto may be used. In this case, the sound image can be localized in the target direction by adding a delay or performing processing for lowering the volume (gain). For example, when the sound image is localized at the lateral angle α and the rising angle β (α is from the right side of the median plane), the transfer characteristics of the lateral angle α and the rising angle β of the right ear and the auricle of the left ear are set. The data measured with the rising angle β on the sagittal plane including the delay-corrected data is used. By doing so, it is possible to improve the localization feeling when reproducing the audio signal.

[Adjustment of hearing aid]
The information processing system 10 in which the processing unit 107 processes an acoustic signal by using the HRTF set generated by the information processing apparatus 100 to realize stereophonic sound has been described above. However, the method of using the measurement result by the measurement system 20 is not limited to this. An information processing system 30 as a modified example of the information processing system 10 will be described below. The information processing system 30 uses the measurement result of the measurement system 20 to adjust the hearing aid that realizes the sound image localization by HRTF.

FIG. 6 is a diagram showing a configuration example of the information processing system 30 and the hearing aid 600. The hearing aid 600 has a microphone array 601, a directivity control unit 602, a convolution processing unit 603, a filter storage unit 604, and a speaker 605. The user wears and uses the hearing aid 600 having such a configuration on both the right and left ears. The microphone array 601 outputs a sound collection signal of a plurality of channels obtained by sound collection by a plurality of microphone elements. The directivity control unit 602 processes a plurality of channels of sound pickup signals input from the microphone array 601, thereby generating a direction sound signal that is a sound signal corresponding to a specific direction. For example, the directivity control unit 602 performs array signal processing such as beam forming, so that the main lobe is oriented in a plurality of directions (forward, upward, backward, downward) having different elevation angles with respect to the user. Generates and outputs a sound signal.

The convolution processing unit 603 convolves the filter data stored in the filter storage unit 604 with respect to each of the plurality of direction sound signals input from the directivity control unit 602. Further, the convolution processing unit 603 adds the signals of the four channels obtained by the convolution and outputs one channel signal. The filter storage unit 604 is a memory that stores filter data used for signal processing for sound image localization. The speaker 605 is a small speaker (earphone) attached to the user's ear canal, and emits sound toward the inside of the ear canal according to the input of a signal. By allowing the sound thus filtered to be heard from both ears of the user, the user can clearly identify the direction of the sound even when the hearing aid 600 is worn.

The information processing system 30 includes a measurement system 20, an information processing device 100, a filter generation unit 108, and a correction information setting unit 109 (hereinafter referred to as the setting unit 109). Note that, in the example of FIG. 6, the filter generation unit 108 and the setting unit 109 are configured as independent devices, but the invention is not limited to this. For example, the filter generation unit 108 and the setting unit 109 may be integrally configured, or at least one of the filter generation unit 108 and the setting unit 109 may be included in the information processing apparatus 100.

The configurations of the measurement system 20 and the information processing device 100 are the same as those in the above-described embodiment. However, the estimation unit 104 of the information processing device 100 acquires the direction sound signal based on the sound pickup by the hearing aid 600 attached to the user's ear from the directivity control unit 602 of the hearing aid 600. Then, the estimation unit 104 estimates the transfer characteristic based on the sound collection signal based on the sound collection by the microphone 103L, the sound collection signal based on the sound collection by the microphone 103R, and the direction sound signal acquired from the hearing aid 600. The generation unit 105 generates characteristic information for correcting the sound emitted by the speaker 605 of the hearing aid 600, based on the estimation result of the transfer characteristic by the estimation unit 104. Specifically, the generation unit 105 generates an HRTF set suitable for the user wearing the hearing aid 600.

For example, the estimation unit 104 estimates the transfer characteristic based on the measurement in the left ear of the user, using the sound pickup signal acquired from the microphone 103L, as in the above-described embodiment. In addition, the measurement is performed with the user wearing the hearing aid 600 instead of the microphone 103L, and the sound emitted from the sounding device 102L is collected by the microphone array 601. Then, the estimation unit 104 acquires the direction sound signal generated by the directivity control unit 602 based on the sound collection signal obtained by the microphone array 601, and based on the direction sound signal, the estimation unit 104 detects the direction of the user wearing the hearing aid 600. Estimate transfer characteristics based on measurements in the left ear. These two transfer characteristics are measured at the same elevation angle on the sagittal plane, including the left ear, and equidistant from the ear canal entrance. Note that the direction sound signal used for estimating the transfer characteristic when the hearing aid 600 is worn is subjected to signal processing by the directivity control unit 602 so that the main lobe is directed toward the speaker 201 used for measurement. This is the signal obtained.

The sound picked up by the microphone array 601 is affected by the pinna because the microphone is mounted near the ear. However, the microphone is not mounted at the entrance of the ear canal, and the structure of the hearing aid 600 also affects the sound propagation. Therefore, the sound transmission characteristics differ depending on whether the user wears the microphone 103L or the microphone array 601L. Then, in order to realize accurate sound image localization even when the user wears the hearing aid 600, the HRTF corrected so as to cancel the influence of the hearing aid 600 should be used. Therefore, the estimation unit 104 performs the following calculation to obtain the corrected transfer characteristic.

H_C(f)=H_F(f)/H_HA(f)
Here, H_C(f) is the corrected transfer characteristic (frequency characteristic), H_F(f) is the transfer characteristic estimated based on the measurement using the earplug type microphone 103L, and H_HA(f) is the hearing aid 600. The transfer characteristic estimated based on the measurement of the mounted state is shown. This processing is performed for each of the left and right ears for each sound source direction. Then, the estimation unit 104 outputs the corrected transfer characteristics of the right ear and the left ear as a set for each of the plurality of rising angles. In the frequency band where H_HA(f)<<H_F(f) in this processing, the value of H_C(f) becomes extremely large, and therefore an upper limit may be set for H_C(f). As a result, it is possible to reduce the possibility that the sound to which the HRTF is applied becomes a jarring sound. At the time of measurement with the left and right ears, or with the earplug type microphone 103L and the microphone array 601, at least one of the rising angle, the lateral angle, and the distance in the sound source direction may not be equal. In this case, the correction processing as described above may be further performed on the information processing system 10.

The generation unit 105 acquires the above-described corrected transfer characteristic from the estimation unit 104, generates an HRTF set based on this, and outputs the HRTF set to the filter generation unit 108. The setting unit 109 is a device that sets the correction information according to the hearing ability of the user who wears the hearing aid 600, and outputs the correction parameter of the filter according to the correction information to the filter generation unit 108. The correction information is, for example, a gain for each frequency band, and is set based on the result of the hearing test by a professional engineer or a doctor. The filter generation unit 108 generates an FIR filter used in the hearing aid 600 based on the HRTF set acquired from the generation unit 105 and the parameter acquired from the setting unit 109, and outputs the FIR filter to the filter storage unit 604 of the hearing aid 600.

FIG. 7 shows a processing flow of the information processing system 30. The process of FIG. 7 is started at a timing when an instruction for adjusting the hearing aid 600 is input to the information processing system 30, for example. In step S<b>701, the setting unit 109 sets the correction information and outputs the correction parameter of the filter according to the set correction information to the information processing apparatus 100. In S702, the measurement system 20 performs measurement using the earplug type microphones 103L and 103R. This processing is similar to the measurement processing in the information processing system 10 described above. In S703, the information processing apparatus 100 acquires the result of measurement using the hearing aid 600. The order of the processes of S702 and S703 may be reversed.

In S704, the information processing apparatus 100 estimates and corrects the transfer characteristic based on the measurement results in S702 and S703 to generate characteristic information for reproducing the head-related transfer characteristic of the median plane. In S705, the filter generation unit 108 generates filter data according to the hearing of the user based on the characteristic information generated in S704 and the correction information set in S701. In step S706, the filter generation unit 108 outputs the filter data corresponding to each direction to the hearing aid 600 as a set of filter data. As described above, according to the information processing system 30, it is possible to perform the adjustment process for improving the sense of localization by the hearing aid 600 by using the small measurement system 20.

In the present embodiment, the direction sound signal is generated by processing the sound collection signal obtained by the microphone array 601 by the directivity control unit 602. However, instead of the microphone array 601, a plurality of directional microphones installed so as to face in different directions may be used. In that case, the hearing aid 600 does not have to include the directivity control unit 602.

Further, in the present embodiment, the filter generation unit 108 generates the FIR filter and outputs it to the filter storage unit 604, but the filter generation unit 108 may generate the IIR filter. In that case, the IIR filter parameters are stored in the filter storage unit 604, and the convolution processing unit 603 performs the IIR filter adaptation processing.

In the present embodiment, data transmission between the hearing aid 600 and the information processing device 100 is performed by communication via a cable or wireless communication. However, the present invention is not limited to this. For example, the hearing aid 600 is provided with a removable storage device, in which the output from the directivity control unit 602 is recorded together with a synchronization signal such as time information, and after the measurement, the storage device is removed to remove the information processing device. Data may be transferred to 100.

Although the case where the user wears the hearing aid 600 in both ears has been described above, the user may wear the hearing aid 600 in one ear. In this case, the estimation unit 104 may correct the transfer characteristic of the ear on which the hearing aid 600 is worn, and characteristic information corresponding to the result may be output to the filter generation unit 108. Further, when the hearing aid 600 is worn in one ear, separate adjustment may be performed in order to suppress a delay in signal processing and a deviation in sound image localization due to a difference in left and right hearing. For example, the information processing device 100 may perform the correction by panning adjustment. Since the sound image tends to be biased to the side where the delay does not occur (the side where the hearing aid 600 is not attached), the sound image is corrected so that the sound image is correctly localized by increasing the volume on the side where the delaying hearing aid 600 is attached. It can be performed. In addition, by performing the volume adjustment processing not only for one sound source direction but for all sound source directions, it is possible to prevent the distance feeling of the sound from changing depending on the direction (the sound image in the direction in which the volume is increased is felt close). it can.

The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by the processing. It can also be realized by a circuit that realizes one or more functions (for example, ASIC). The program may be recorded in a computer-readable recording medium and provided.

10 Information Processing System 20 Measurement System 100 Information Processing Device 102L, 102R Sounding Device 103L, 103R Microphone 201 Speaker

Claims (20)

First acquisition for acquiring a sound collection signal based on sound collection at a position of one ear of the user, the sound collection signal being based on sound collection of a plurality of sounds emitted from a plurality of directions around the one ear Means and
Second, a sound collection signal based on sound collection at the position of the other ear of the user, which is based on sound collection of a plurality of sounds emitted from a plurality of directions around the other ear Acquisition means,
Generating means for generating characteristic information relating to the transfer characteristic of sound corresponding to the user, based on the sound collection signal acquired by the first acquisition means and the sound collection signal acquired by the second acquisition means. An information processing system having. The information processing system according to claim 1, wherein the characteristic information generated by the generating unit includes information indicating a transfer characteristic corresponding to each of a plurality of directions. The information processing system according to claim 1, wherein the characteristic information generated by the generating unit includes information indicating a transfer characteristic corresponding to a direction included in a median plane of the user. The information processing system according to any one of claims 1 to 3, wherein the characteristic information generated by the generation means is information representing a head related transfer function or a head related impulse response. The sound collection period corresponding to the sound collection signal acquired by the first acquisition unit and the sound collection period corresponding to the sound collection signal acquired by the second acquisition unit do not overlap. The information processing system according to any one of 1 to 4. The first acquisition means collects sound based on the collection of a plurality of sounds emitted from a plurality of directions substantially perpendicular to the direction connecting the one ear and the other ear to the position of the one ear. The information processing system according to claim 1, wherein a signal is acquired. Rise angles in the direction of a plurality of sounds emitted around the one ear with respect to the position of the one ear, and a plurality of rise angles around the other ear with respect to the position of the other ear. The information processing system according to any one of claims 1 to 6, wherein the rising angles in the direction of sound are substantially equal to each other. The generating means is
First estimating means for estimating a first transfer characteristic based on the sound pickup signal obtained by the first obtaining means,
Second estimating means for estimating a second transfer characteristic based on the sound collection signal acquired by the second acquiring means;
The characteristic information is generated based on an estimation result of the first transfer characteristic by the first estimating means and an estimation result of the second transfer characteristic by the second estimating means. 7. The information processing system according to any one of 7. An input device for inputting a predetermined signal to a sounding device having one or more sound generators that emit a sound in response to the input of a signal;
9. The information processing according to claim 1, wherein the first acquisition unit acquires a sound collection signal based on a sound collection according to the input of the predetermined signal by the input unit. system. The information processing system according to claim 9, wherein the first acquisition unit acquires a sound collection signal based on sound collection in a state where the sounding device surrounds the one ear. The first acquisition unit is a sound collection signal based on sound collected by a microphone located near the one ear, and is emitted from a plurality of directions to the microphone by a plurality of sound producing units of the sound producing device. The information processing system according to claim 9, wherein a sound collection signal based on the sound collection of a plurality of sounds is acquired. The first acquisition unit is a sound collection signal based on sound collection by a microphone located near the one ear, and the sounding unit of the sounding device changes the position with respect to the microphone and emits sound from a plurality of directions. The information processing system according to claim 9, wherein a sound collection signal based on sound collection of a plurality of sounds is acquired. The information processing system according to claim 1, further comprising a storage control unit that stores the characteristic information generated by the generation unit in a storage unit. Signal acquisition means for acquiring an acoustic signal,
The output means for outputting a processed acoustic signal obtained by processing the acoustic signal acquired by the signal acquisition means based on the characteristic information generated by the generation means. 13. The information processing system according to any one of 13. A third acquisition means for acquiring a signal based on the sound collection by the hearing aid attached to the one ear,
The hearing aid is based on the sound pickup signal acquired by the first acquisition means, the sound collection signal generated by the second acquisition means, and the signal acquired by the third acquisition means. The information processing system according to any one of claims 1 to 14, wherein the characteristic information for correcting the sound emitted by the is generated. First acquisition for acquiring a sound collection signal based on sound collection at a position of one ear of the user, the sound collection signal being based on sound collection of a plurality of sounds emitted from a plurality of directions around the one ear Process,
Second, a sound collection signal based on sound collection at the position of the other ear of the user, which is based on sound collection of a plurality of sounds emitted from a plurality of directions around the other ear Acquisition process,
A generation step of generating characteristic information regarding a sound transmission characteristic according to the user, based on the sound collection signal acquired in the first acquisition step and the sound collection signal acquired in the second acquisition step. An information processing method comprising: The sound collection period corresponding to the sound collection signal acquired in the first acquisition step and the sound collection period corresponding to the sound collection signal acquired in the second acquisition step do not overlap. 16. The information processing method described in 16. A measurement system for measuring a sound transfer characteristic according to a user's auricle,
A microphone wearable on the user's ear,
A sounding device having a plurality of sounding units that emit sounds in response to signal input,
A mounting system for mounting the sounding device to the user such that the plurality of sounding units are located in different directions with respect to the user's ear, respectively. A measurement system for measuring a sound transfer characteristic according to a user's auricle,
A microphone wearable on the user's ear,
A case having a hole into which the user's ear can be inserted;
A sounding unit that emits sound in response to the input of a signal, and a sounding unit located inside the case,
And a control unit that moves the position of the sounding unit so that the position of the sounding unit with respect to the position of the hole of the case changes. A program for causing a computer to function as each unit included in the information processing system according to claim 1.

Images