JPWO2020008889A1 - Audio signal processors and methods, impulse response generators and methods, and programs - Google Patents

Abstract

The present art relates to audio signal processing devices and methods, impulse response generators and methods, and programs that enable the desired phase characteristics to be obtained. The audio signal processing device includes an acquisition unit that acquires an impulse response having a flat or substantially flat amplitude characteristic and having a predetermined phase characteristic, and a phase characteristic convolution unit that convolves the impulse response into the input audio signal. This technology can be applied to audio signal processing devices and impulse response generators.

Description

The present technology relates to audio signal processing devices and methods, impulse response generating devices and methods, and programs, and in particular, audio signal processing devices and methods, impulse response generating devices and methods, which have made it possible to obtain desired phase characteristics. And about the program.

For example, in the case of reproducing audio such as music, there is known a technique of applying an effect such as an effect to the music to be reproduced by performing a filter process on the audio signal.

As such a technique, for example, a technique for changing the amplitude characteristic of an audio signal so as to bring about a bass enhancement effect by combining a plurality of filters has been proposed (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2002-171589

By the way, in recent years, the number of users who play and listen to music with headphones instead of speakers is increasing, and audio playback with headphones is becoming mainstream.

On the other hand, most commercial content is basically mastered by speakers. Therefore, there is a complaint that even if the content such as music mastered while being played back on the speaker is played back on the headphones, it is not possible to hear the bass with a sense of volume as when played back on the speaker. That is, when the content is played back with headphones, the low-frequency sound is different from that of the speaker playback, and it may not be possible to realize the playback with the sound quality that the creator originally wanted to convey.

Therefore, when the applicant investigated, it was found that the way the low frequencies were heard was greatly influenced by the phase characteristics. That is, it was found that one of the causes is that the low-frequency phase characteristics of the speaker and the low-frequency phase characteristics of the headphones are significantly different.

However, with the above-mentioned technique, although the amplitude characteristic of the audio signal can be adjusted, it is not possible to make the phase characteristic of the audio signal a desired characteristic.

The present technology has been made in view of such a situation, and makes it possible to obtain a desired phase characteristic.

The audio signal processing device of the first aspect of the present technology includes an acquisition unit that acquires an impulse response having a flat or substantially flat amplitude characteristic and a predetermined phase characteristic, and a phase that convolves the impulse response with the input audio signal. It is equipped with a characteristic convolution section.

The audio signal processing method or program of the first aspect of the present technology acquires an impulse response having a flat or substantially flat amplitude characteristic and a predetermined phase characteristic, and convolves the impulse response with the input audio signal. Including.

In the first aspect of the present technology, an impulse response having a flat or substantially flat amplitude characteristic and a predetermined phase characteristic is acquired, and the impulse response is convoluted in the input audio signal.

The impulse response generator of the second aspect of the present technology generates a target characteristic impulse response having a flat or substantially flat amplitude characteristic and a predetermined phase characteristic.

The impulse response generation method or program of the second aspect of the present technology includes a step of generating a target characteristic impulse response having a flat or substantially flat amplitude characteristic and a predetermined phase characteristic.

In the second aspect of the present technology, a target characteristic impulse response having a flat or substantially flat amplitude characteristic and a predetermined phase characteristic is generated.

According to the first aspect and the second aspect of the present technology, a desired phase characteristic can be obtained.

The effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

It is a figure explaining the relationship between a frequency characteristic and an impulse response. It is a figure explaining the reconstruction of the impulse response. It is a figure which shows the frequency characteristic of the reconstructed impulse response. It is a figure explaining the reconstruction of the impulse response. It is a figure which shows the frequency characteristic of the reconstructed impulse response. It is a figure explaining the reconstruction of the impulse response. It is a figure which shows the frequency characteristic of the reconstructed impulse response. It is a figure which shows the configuration example of the impulse response generator. It is a flowchart explaining the impulse response generation processing. It is a figure which shows the configuration example of the impulse response generator. It is a flowchart explaining the impulse response generation processing. It is a figure explaining the mastering of contents. It is a figure which shows the configuration example of the reproduction apparatus. It is a flowchart explaining the reproduction process. It is a figure which shows the configuration example of the reproduction apparatus. It is a flowchart explaining the reproduction process. It is a figure which shows the configuration example of the reproduction apparatus. It is a flowchart explaining the reproduction process. It is a figure which shows the configuration example of the reproduction apparatus. It is a figure which shows the configuration example of a computer.

Hereinafter, embodiments to which the present technology is applied will be described with reference to the drawings.

<First Embodiment>
<About this technology>
The present technology makes it possible to adjust only the phase characteristics without changing the amplitude characteristics (gain characteristics) of the audio signal.

That is, the present technology adjusts only the phase characteristic while maintaining the amplitude characteristic of the audio signal by generating an impulse response having a flat or substantially flat amplitude characteristic and a desired phase characteristic, and obtains the desired phase characteristic. It is what makes it possible to obtain.

In this technology, by performing FFT (Fast Fourier Transform) and IFFT (Inverse Fast Fourier Transform) on an impulse response (impulse response) having a target phase characteristic, the amplitude characteristic is flat or substantially flat, and is desired. It is possible to obtain an impulse response having the phase characteristics of. Here, the amplitude characteristic being flat or substantially flat means that, for example, the value of the amplitude (gain) at each frequency of the amplitude characteristic is 1 or substantially 1.

Specifically, in the present technology, the target impulse response is generated by the following method A1 or method A2.

That is, in the method A1, first, 0 data (zero data) of an appropriate length is inserted before the impulse response for which the phase is to be simulated, and FFT (Fast Fourier Transform) is performed.

Amplitude characteristics and phase characteristics can be obtained by such an FFT, and the amplitude (gain) value at each frequency of the amplitude characteristics is set to 1 so that the amplitude characteristics become flat, and the flat amplitude characteristics and the FFT IFFT (Fast Fourier Transform) is performed based on the phase characteristics obtained in. Then, the subsequent stage of the impulse response obtained by IFFT is fade-processed with an appropriate time constant to obtain the desired impulse response.

The impulse response obtained in this way functions as an IIR (Infinite Impulse Response) filter that changes only the phase characteristics while maintaining the amplitude characteristics. Therefore, only the phase characteristic can be adjusted by convolving such an impulse response into the audio signal.

Further, in the method A2, FFT is performed without inserting 0 data for the impulse response for which the phase is to be simulated, and 0 data is inserted for the simple impulse and FFT is performed.

Then, the phase characteristic obtained by the FFT for the impulse response and the phase characteristic obtained by the FFT for the simple impulse into which 0 data is inserted are added, and the phase characteristic obtained as a result and the value of the amplitude at each frequency are added. IFFT is performed based on the flat amplitude characteristic of which is 1. Further, the subsequent stage of the impulse response obtained by IFFT is faded with an appropriate time constant to obtain the desired impulse response.

In the method A2, an impulse response having the same characteristics as in the method A1 can be obtained. In other words, it is possible to obtain an IIR filter that changes only the phase characteristics while maintaining the amplitude characteristics.

In addition, in method A2, if the phase characteristic obtained by FFT for the impulse response and the phase characteristic obtained by FFT for the simple impulse with 0 data inserted are subtracted instead of added, the original impulse response can be obtained. An impulse response with a phase characteristic and an inverse characteristic can be obtained.

By using the impulse responses obtained by the above methods A1 and A2, it is possible to reproduce sounds having the same sound quality even if the playback devices are different.

As a specific example, suppose that there is content mastered while reproducing sound through a speaker, and that content is reproduced by headphones.

In such a case, by convolving the impulse response having the opposite characteristic to the phase characteristic of the headphone and the impulse response having the same phase characteristic as the speaker with respect to the audio signal of the content, the phase characteristic of the headphone is canceled and the speaker is used. The phase characteristics of can be simulated. That is, even when the content is reproduced by the headphones, the sound having the same sound quality as that at the time of mastering can be reproduced.

Then, this technology will be described in more detail below.

For example, consider the case where there is content that has been mastered while playing sound through speakers, and that content is played back through headphones.

In this case, if only the low-frequency phase characteristics of the speaker used for mastering can be added to the sound source of the content, that is, the audio signal of the content when playing back with headphones, the creator will create it in the mastering studio. It is assumed that you can experience almost the same sound as the sound you are using.

In general, it is difficult to identify the speaker used in the mastering studio from any content. In the future, it is possible to identify the speaker used in the mastering studio from the metadata of the content, but it is difficult to do so at present.

Therefore, since the characteristics of the IIR type HPF (High Pass Filter) with a cutoff frequency Fc = 50Hz are close to the characteristics of the speaker, we consider such characteristics of the HPF as pseudo speaker characteristics.

For example, the relationship between the frequency characteristics of the IIR type HPF having a cutoff frequency Fc of 50 Hz, that is, the amplitude characteristics (gain characteristics) and the phase characteristics, and the impulse response of the HPF is shown in FIG.

In FIG. 1, the portion indicated by the arrow Q11 shows the amplitude characteristic of the frequency characteristics of the HPF, and the portion indicated by the arrow Q12 shows the phase characteristic of the frequency characteristics of the HPF.

In particular, the vertical axis in the amplitude characteristics shows the gain (amplitude), and the horizontal axis shows the frequency. The vertical axis of the phase characteristic indicates the phase, and the horizontal axis indicates the frequency. From this frequency characteristic, it can be seen that the gain is small and the phase is a positive value on the low frequency side of the HPF.

On the other hand, the impulse response of the HPF is shown in the portion indicated by the arrow Q13. The vertical axis of the impulse response shows the amplitude, and the horizontal axis shows the time, that is, the time sample (sample). Here, the impulse response of the HPF is enlarged in the vicinity of the 0th sample.

Such an impulse response can be used as an IIR type filter, and by convolving the impulse response indicated by the arrow Q13 into an audio signal, the audio signal can be filtered by HPF.

Furthermore, the frequency characteristics of the HPF, that is, the amplitude characteristics and the phase characteristics, and the impulse response of the HPF have a reversible relationship, although there is a conversion error.

Specifically, when IFFT is performed on the frequency characteristic including the amplitude characteristic shown by the arrow Q11 and the phase characteristic shown by the arrow Q12, the impulse response shown by the arrow Q13 is ideally obtained. On the other hand, when FFT is performed on the impulse response indicated by the arrow Q13, ideally, a frequency characteristic having the amplitude characteristic shown by the arrow Q11 and the phase characteristic shown by the arrow Q12 is obtained.

By convolving the impulse response of such an HPF into the audio signal of the mastered content while reproducing the sound with the above-mentioned speaker, it is possible to add the same phase characteristics to the sound of the content as the speaker, but the low frequency range. Gain (amplitude) of is reduced.

Therefore, FFT is actually performed on the impulse response shown by the arrow Q13, and the frequency characteristics obtained as a result are made flat amplitude characteristics by setting the amplitude (gain) of all frequencies of the amplitude characteristics to 1, and further performing IFFT. Consider reconstructing the impulse response.

At this time, the impulse response to be obtained by the reconstruction is an impulse response in which only the phase characteristic shown in FIG. 1 is added to the audio signal without changing the amplitude characteristic, that is, a desired impulse response without changing the amplitude characteristic. It is an impulse response that adds only the phase characteristics.

Then, by convolving the impulse response obtained by the reconstruction into the audio signal of the mastered content while reproducing the sound with the above-mentioned speaker, the phase characteristic of the target speaker can be obtained without changing the amplitude characteristic. It can be added to the sound of the content. As a result, the listener (user) can experience almost the same sound as the sound produced by the creator in the mastering studio.

In the following, an impulse response that functions as a filter that adds a desired phase characteristic without changing the amplitude characteristic will be referred to as a target phase characteristic impulse response in particular.

When the target phase characteristic impulse response is to be obtained by reconstructing the impulse response, it is conceivable to perform the reconstruction as shown in FIG. 2, for example.

In FIG. 2, the portion indicated by the arrow Q21 shows the impulse response indicated by the arrow Q13 in FIG.

This impulse response converges on approximately 1024 samples. However, here, in consideration of the tone after conversion, as shown by arrow Q22, 0-filling processing is performed on the back side in the time direction of the impulse response, that is, the future side, and 4096 processing is performed.

That is, the total length (number of samples) of the impulse response is 4096 by performing a zero-packing process in which 0 data, which is a sample having a sample value of 0, is added to the back side (end) of the impulse response in the time direction. Make it a sample.

When FFT is performed on the impulse response that has been zero-packed in this way as shown by arrow Q23, the same amplitude characteristics and phase characteristics as those shown in FIG. 1 can be obtained.

Here, the target phase characteristic impulse response has a flat or substantially flat amplitude characteristic, and the phase characteristic is the phase characteristic shown by the arrow Q12.

Therefore, the value of the amplitude (gain) of each frequency in the amplitude characteristic obtained by FFT is adjusted to "1". In other words, the amplitude of the amplitude characteristic obtained by FFT is adjusted so that the amplitude characteristic becomes flat.

Further, since the phase characteristic obtained by the FFT should be the phase characteristic shown by the target arrow Q12, no particular phase adjustment is performed on the phase characteristic obtained by the FFT.

Next, as shown by arrow Q24, IFFT is performed on the frequency characteristic including the flat amplitude characteristic obtained by the amplitude adjustment and the phase characteristic obtained by the FFT.

Further, since the impulse response obtained by IFFT does not converge to 0, the impulse response obtained by IFFT is faded out to the rear side (tail side) in the time direction of the impulse response to converge to 0. Is done.

The impulse response is reconstructed by such a fade process, and as a result, the target phase characteristic impulse response shown by the arrow Q25 is obtained. Here, an impulse response having a length of 4096 samples is obtained as the target phase characteristic impulse response.

The target phase characteristic impulse response indicated by arrow Q25 should ideally have flat or substantially flat amplitude characteristics and the same phase characteristics as the original HPF.

However, since conversion distortion actually occurs in conversions such as FFT and IFFT, the frequency characteristics of the target phase characteristic impulse response shown by arrow Q25 are as shown in FIG.

In FIG. 3, the portion indicated by the arrow Q31 shows the amplitude characteristic, and the portion indicated by the arrow Q32 shows the phase characteristic. The vertical axis of the amplitude characteristic indicates the gain (amplitude), and the horizontal axis indicates the frequency. The vertical axis of the phase characteristic indicates the phase, and the horizontal axis indicates the frequency.

In the portion indicated by the arrow Q31, the curve L11 shows the amplitude characteristic of the target phase characteristic impulse response shown by the arrow Q25 in FIG. 2, and the curve L12 shows the amplitude characteristic of the original HPF shown by the arrow Q21 in FIG. There is. From the curve L11, the amplitude characteristic of the target phase characteristic impulse response is not as flat as the original HPF, but the gain of the low frequency portion, that is, the portion indicated by the arrow W11 is reduced, and the amplitude characteristic is not flat. I understand.

Further, in the portion indicated by the arrow Q32, the curve L13 shows the phase characteristic of the target phase characteristic impulse response shown by the arrow Q25 in FIG. 2, and the curve L14 is the phase characteristic of the original HPF shown by the arrow Q21 in FIG. That is, the target phase characteristic is shown.

In this example, the curve L13 is substantially the same as the curve L14, and it can be seen that the target characteristic is obtained with respect to the phase characteristic in the target phase characteristic impulse response.

By the way, in general, when the target phase characteristic impulse response is opposite, that is, when the phase characteristic is flat (straight line) and the amplitude (gain) changes, the impulse response basically has a symmetrical shape. It has been known.

Therefore, the applicant applies zero-packing processing so that the impulse response is substantially symmetrical around the portion where the pulse rises, and makes the section of the same length before and after the portion where the pulse rises. For example, I thought that it would be possible to obtain an impulse response with flat amplitude characteristics.

Here, the impulse response of the original HPF is zero-packed at least on the front side (past side) in the time direction so that the impulse response has a substantially symmetrical shape, and then FFT and IFFT are performed. Consider generating a target phase characteristic impulse response.

In such a case, for example, as shown in FIG. 4, the impulse response is reconstructed and the target phase characteristic impulse response is generated.

In FIG. 4, the portion indicated by the arrow Q41 shows the impulse response of the HPF shown by the arrow Q13 in FIG. 1, and this impulse response converges in approximately 1024 samples.

In this example, the impulse response of the HPF indicated by the arrow Q41 is zero-packed as shown by the arrow Q42.

That is, 0 data is added not only to the rear side (tail side) in the time direction of the impulse response but also to the front side (head side) according to the length of the impulse response.

In particular, here, 0 data is added to the front side of the impulse response in the time direction by 8192 samples, and 0 data is also added to the rear side of the impulse response in the time direction so that the impulse length itself is 8192 samples. It has been added. By such zero-packing processing, the impulse response shown by arrow Q42 has a substantially symmetrical shape, and the total length is 16384 samples.

Next, as shown by arrow Q43, when FFT is performed on the impulse response that has been zero-packed, amplitude characteristics and phase characteristics are obtained as in the case of arrow Q23 in FIG.

In this example as well, the target phase characteristic impulse response has a flat amplitude characteristic, so the amplitude (gain) value of each frequency in the amplitude characteristic obtained by FFT is adjusted to "1" and is flat. It is considered to be an amplitude characteristic.

Further, since the phase characteristic obtained by the FFT should be the target phase characteristic, no particular phase adjustment is performed on the phase characteristic obtained by the FFT.

Subsequently, as shown by arrow Q44, IFFT is performed on the frequency characteristic consisting of the flat amplitude characteristic obtained by the amplitude adjustment and the phase characteristic obtained by the FFT, and the impulse response obtained as a result is subjected to IFFT. The fade process is performed in the same manner as in the case of the arrow Q24 in FIG.

Then, the impulse response obtained by the fade processing is used as the target phase characteristic impulse response. Here, the target phase characteristic impulse response shown by the arrow Q45 is obtained, and the target phase characteristic impulse response has a shape close to left-right symmetry. The length of the target phase characteristic impulse response is 16384 samples.

The frequency characteristics of the target phase characteristic impulse response shown by the arrow Q45 thus obtained are as shown in FIG.

In FIG. 5, the portion indicated by the arrow Q51 shows the amplitude characteristic, and the portion indicated by the arrow Q52 shows the phase characteristic. The vertical axis of the amplitude characteristic indicates the gain (amplitude), and the horizontal axis indicates the frequency. The vertical axis of the phase characteristic indicates the phase, and the horizontal axis indicates the frequency.

In the portion indicated by the arrow Q51, the curve L31 shows the amplitude characteristic of the target phase characteristic impulse response shown by the arrow Q45 in FIG. 4, and the curve L32 shows the amplitude characteristic of the original HPF shown by the arrow Q41 in FIG. There is.

The amplitude characteristic of the target phase characteristic impulse response shown in the curve L31 is within ± 0.2 dB of the amplitude (gain) value at each frequency, and it can be seen that a substantially flat characteristic is obtained. That is, it can be seen that the target amplitude characteristic is obtained.

Further, in the portion indicated by the arrow Q52, the curve L33 shows the phase characteristic of the target phase characteristic impulse response shown by the arrow Q45 in FIG. 4, and the curve L34 is the phase characteristic of the original HPF shown by the arrow Q41 in FIG. That is, the target phase characteristic is shown. Further, the curve L35 shows the phase characteristic of a simple impulse in which only 8192 samples are delayed, that is, the linear phase.

Here, it can be seen that the curve L33 and the curve L34 almost overlap each other, and that a characteristic substantially equivalent to the target characteristic is obtained as the phase characteristic of the target phase characteristic impulse response.

Further, the curve L35 is shown for comparison. Since the curve L35 shows the phase characteristic of a simple impulse having a linear phase, the difference between the curve L33 and the curve L35 at each frequency is the phase value at each frequency of the phase characteristic shown by the arrow Q12 in FIG. Then, the target characteristic is obtained as the phase characteristic of the target phase characteristic impulse response. The phase characteristic of the original HPF shown by the arrow Q41 in FIG. 4 is the same as the phase characteristic shown by the arrow Q12 in FIG.

In this example, when the curve L33 and the curve L35 are compared, the difference in their phases becomes smaller as the frequency becomes higher. Therefore, it can be seen from the curves L33 and the curve L35 that substantially the same characteristics as the phase characteristics shown by the arrow Q12 in FIG. 1 are obtained as the phase characteristics of the target phase characteristic impulse response.

From the above, the impulse response having the target phase characteristic is zero-packed at least in the front side in the time direction, and the zero-packed impulse response is subjected to FFT, IFFT, and fade processing. It can be seen that a target phase characteristic impulse response having a flat or substantially flat amplitude characteristic and a target phase characteristic can be obtained.

The method for generating the target phase characteristic impulse response described with reference to FIG. 4 as described above is the above-mentioned method A1.

In generating the target phase characteristic impulse response, the longer the length of the zero-packed impulse response, that is, the larger the number of samples, the closer the frequency characteristic of the target phase characteristic impulse response becomes to the target characteristic. .. That is, better characteristics can be obtained. In particular, when the length of the impulse response processed to zero is an infinite sample, the error between the frequency characteristic of the target phase characteristic impulse response and the target characteristic becomes infinitely close to zero.

Further, when generating the target phase characteristic impulse response, it may be desired to reduce the processing amount even if an error from the target characteristic is allowed to some extent. For example, if the length of the target phase characteristic impulse response is shortened, the amount of processing is reduced both at the time of generation and at the time of convolution after generation.

In such a case, for example, as shown in FIG. 6, by reducing the number of 0 data added to the impulse response in the zero packing process, the processing amount can be reduced and the target phase characteristic impulse response with sufficient characteristics can be obtained. You may.

In FIG. 6, the portion indicated by the arrow Q61 shows the impulse response of the HPF shown by the arrow Q13 in FIG. 1, and this impulse response converges in approximately 1024 samples.

In this example, the impulse response of the HPF indicated by the arrow Q61 is zero-packed as shown by the arrow Q62.

Here, 0 data is added to the front side of the impulse response in the time direction by 384 samples, and 0 data is also added to the rear side of the impulse response in the time direction so that the total length of the impulse response is 4096 samples. Has been done.

In this zero-packing process, the number of 0-data added to the front side in the time direction in the impulse response is small, so that the impulse response after the zero-packing process shown by arrow Q62 does not have a symmetrical shape.

Next, as shown by arrow Q63, when FFT is performed on the impulse response that has been zero-packed, amplitude characteristics and phase characteristics are obtained as in the case of arrow Q23 in FIG.

In this example as well, as in the case of FIG. 4, the value of the amplitude (gain) of each frequency in the amplitude characteristic obtained by the FFT is adjusted to "1" to obtain a flat amplitude characteristic, and the phase characteristic obtained by the FFT. No particular phase adjustment is performed on the.

Subsequently, as shown by arrow Q64, IFFT is performed on the frequency characteristic consisting of the flat amplitude characteristic obtained by the amplitude adjustment and the phase characteristic obtained by the FFT, and the impulse response obtained as a result is subjected to IFFT. The fade process is performed in the same manner as in the case of the arrow Q24 in FIG.

Then, the impulse response obtained by the fade processing is used as the target phase characteristic impulse response. Here, the target phase characteristic impulse response shown by arrow Q65 is obtained, and the length of the target phase characteristic impulse response is 4096 samples.

In this example, since the number of 0 data added to the front side in the time direction in the impulse response is small, the target phase characteristic impulse response shown by the arrow Q65 does not have a symmetrical shape.

The frequency characteristics of the target phase characteristic impulse response shown by the arrow Q65 thus obtained are as shown in FIG.

In FIG. 7, the portion indicated by the arrow Q71 shows the amplitude characteristic, and the portion indicated by the arrow Q72 shows the phase characteristic. The vertical axis of the amplitude characteristic indicates the gain (amplitude), and the horizontal axis indicates the frequency. The vertical axis of the phase characteristic indicates the phase, and the horizontal axis indicates the frequency.

In the portion indicated by the arrow Q71, the curve L51 shows the amplitude characteristic of the target phase characteristic impulse response shown by the arrow Q65 in FIG. 6, and the curve L52 shows the amplitude characteristic of the original HPF shown by the arrow Q61 in FIG. There is.

The amplitude characteristic of the target phase characteristic impulse response shown in the curve L51 is within ± 1 dB of the amplitude (gain) value at each frequency, and it can be seen that a substantially flat characteristic is obtained. That is, it can be seen that sufficient amplitude characteristics are obtained.

In particular, here, the amplitude characteristic shown in the curve L51 has a larger error from the target characteristic than the amplitude characteristic shown in the curve L31 in FIG. 5, but the error is within a sufficiently small range. You can see that there is.

Further, in the portion shown by the arrow Q72, the curve L53 shows the phase characteristic of the target phase characteristic impulse response shown by the arrow Q65 in FIG. 6, and the curve L54 shows the phase characteristic of the original HPF shown by the arrow Q61 in FIG. That is, the target phase characteristic is shown. Further, the curve L55 shows the phase characteristics of the delayed simple impulse, similarly to the curve L35 of FIG.

Here, the curve L53 and the curve L54 have a larger error than in the case of FIG. 5, but almost overlap each other, and the target phase characteristic has substantially the same characteristic as the target characteristic as the phase characteristic of the impulse response. I understand.

Further, when the curve L53 and the curve L55 are compared, the difference in their phases becomes smaller as the frequency becomes higher, and as in the case of FIG. 5, the phase characteristic of the target phase characteristic impulse response is the arrow Q12 in FIG. It can be seen that substantially the same characteristics as the phase characteristics shown in are obtained.

As described above, even if the number of 0 data added to the front side of the impulse response having the target phase characteristic in the time direction is reduced to some extent, the amplitude characteristic is flat or substantially flat and has the target phase characteristic. The target phase characteristic impulse response can be obtained.

How much 0 data is added to the front side of the impulse response in the time direction is a trade-off between the margin of error with the target characteristic and the processing amount, so the number of 0 data to be added is necessary. You can adjust it.

Further, instead of performing zero-packing processing on the impulse response having the target phase characteristic, for example, zero-packing processing is performed on the simple impulse as shown in the curve L35 of FIG. 5 to generate the target phase characteristic impulse response. You may. The method for generating such a target phase characteristic impulse response is the above-mentioned method A2.

In the method A2, a zero-packing process of adding 0 data to the front side of the simple impulse in the time direction is performed, and an FFT is performed on the simple impulse after the zero-packing process.

In the following, the phase characteristic of the frequency characteristic obtained by FFT with respect to the simple impulse after the zero-packing process will be referred to as the phase characteristic of the simple impulse in particular.

Further, in the method A2, the impulse response having the target phase characteristic is not zero-packed, and the FFT is performed as it is for the impulse response. In the following, the phase characteristic of the frequency characteristic obtained by FFT with respect to the impulse response having the target phase characteristic will be referred to as the target phase characteristic in particular.

When the phase characteristics of the simple impulse and the target phase characteristics are obtained by FFT in this way, the phase characteristics and the target phase characteristics of those simple impulses are added, and the phase characteristics obtained by the addition and the flat amplitude are added. IFFT is performed on the frequency characteristics consisting of the characteristics.

Then, a fade process is performed on the impulse response obtained by IFFT, and the impulse response obtained as a result is used as the target phase characteristic impulse response.

The target phase characteristic impulse response thus obtained is an impulse response having a flat or substantially flat amplitude characteristic and a target phase characteristic.

In the method A2, instead of adding the phase characteristic of the simple impulse and the target phase characteristic, if the target phase characteristic is subtracted from the phase characteristic of the simple impulse, the impulse response having the inverse characteristic of the target phase characteristic is obtained as the target phase. It can be obtained as a characteristic impulse response.

Specifically, for example, the phase characteristic obtained by performing FFT on the impulse response of a predetermined HPF without performing the zero-packing process is subtracted from the phase characteristic of the simple impulse, and the phase characteristic obtained as a result is obtained. , IFFT is performed on the frequency characteristic consisting of flat amplitude characteristic. Then, a fade process is performed on the impulse response obtained by IFFT, and the impulse response obtained as a result is used as the target phase characteristic impulse response.

In this case, the phase characteristic of the obtained target phase characteristic impulse response becomes the inverse characteristic of the phase characteristic of the original HPF.

As described above, in the method A1, the impulse response having the target phase characteristic is zero-packed, and then the FFT, IFFT, and fade processing are performed to generate the target phase characteristic impulse response. On the other hand, in the method A2, the simple impulse is subjected to zero-packing processing, and then FFT, IFFT, and fade processing are performed to generate a target phase characteristic impulse response.

The impulse response used in the method A1 and the simple impulse used in the method A2 are both impulse information, that is, information related to the impulse. Therefore, when the method A1 and the method A2 are generalized, the impulse information is zero-packed, the resulting phase characteristic is subjected to FFT, IFFT, and fade processing, and the target phase characteristic impulse is performed. It can be said that the response has been generated.

By using the target phase characteristic impulse response obtained as described above, it is possible to add a desired phase characteristic to the audio signal without changing the amplitude characteristic.

As a specific example, consider a case where there is content mastered while playing sound on a mastering speaker, and the content is played back on the playback side headphones or speakers.

In this case, by convolving the target phase characteristic impulse response having the opposite characteristic to the phase characteristic of the playback side headphone or speaker into the content audio signal, the phase characteristic of the playback side headphone or speaker is canceled for the content audio signal. Can be done. Here, the audio signal in which the phase characteristic of the headphone or the speaker on the playback side is canceled is referred to as a corrected audio signal.

The target phase characteristic impulse response having the opposite characteristic to the phase characteristic of the headphone or the speaker on the reproduction side may be generated by the above-mentioned method A2 or may be generated by the method A1. For example, when such a target phase characteristic impulse response is generated by the method A1, the impulse response having the phase characteristic opposite to that of the headphone or speaker on the playback side is zero-packed to perform FFT, IFFT, and fade processing. Just do it.

Furthermore, by convolving the target phase characteristic impulse response, which has the same characteristics as the phase characteristics of the mastering speaker, into the corrected audio signal, the phase characteristics of the mastering speaker can be obtained for the corrected audio signal, that is, for the sound of the content. Can be added.

Therefore, if the sound of the content is reproduced based on the corrected audio signal to which the phase characteristic of the speaker for mastering is added in this way, the listener (listener) can hear almost the same sound as the sound produced by the creator in the mastering studio. It can be experienced by the user).

In addition, for example, when the listener reproduces the sound of the content with headphones on the playback side, the head related transfer function (HRTF) can be used to create a mastering studio. It is possible to present a sound that is closer to the sound produced by.

Here, the HRTF is a function indicating the sound transmission characteristics from the sound source to the listener's ear, and more specifically, to the vicinity of the eardrum of the listener or the entrance of the ear canal.

In this example, by further convolving the HRTF with the corrected audio signal to which the phase characteristics of the mastering speaker are added, the listener can experience a sound closer to the sound heard when the creator is producing in the mastering studio. Can be made to.

<Configuration example of impulse response generator>
Subsequently, a specific configuration and operation of the impulse response generator that generates the target phase characteristic impulse response described above will be described.

FIG. 8 is a diagram showing a configuration example of an impulse response generator that generates a target phase characteristic impulse response, that is, an impulse response having a flat or substantially flat amplitude characteristic and having a desired phase characteristic by the above-mentioned method A1. ..

The impulse response generation device 11 shown in FIG. 8 has a zero-packing processing unit 21, an FFT processing unit 22, an IFFT processing unit 23, and a fade processing unit 24.

The zero-packing processing unit 21 is supplied with an impulse response having the target phase characteristic used for generating the target phase characteristic impulse response. Hereinafter, an impulse response having such a target phase characteristic will be referred to as an input impulse response.

The 0-packing processing unit 21 performs 0-packing processing on the supplied input impulse response and supplies it to the FFT processing unit 22.

The FFT processing unit 22 performs FFT on the input impulse response after the 0-packing process supplied from the 0-packing processing unit 21, and supplies the phase characteristic of the frequency characteristics obtained as a result to the IFFT processing unit 23. ..

A flat amplitude characteristic (gain characteristic) in which the gain (amplitude) of each frequency is "1" is supplied to the IFFT processing unit 23 from the outside.

The IFFT processing unit 23 performs IFFT on the frequency characteristics including the flat amplitude characteristics supplied from the outside and the phase characteristics supplied from the FFT processing unit 22, and the impulse response obtained as a result is faded. Supply to 24. In other words, IFFT is performed based on the flat amplitude characteristic and the phase characteristic supplied from the FFT processing unit 22, and an impulse response is generated.

The IFFT processing unit 23 does not use the flat amplitude characteristics supplied from the outside, but flat amplitudes by adjusting the gain with respect to the amplitude characteristics of the frequency characteristics obtained by the FFT in the FFT processing unit 22. A characteristic may be generated and the amplitude characteristic may be used for IFFT.

The fade processing unit 24 performs fade processing on the impulse response supplied from the IFFT processing unit 23, and outputs the impulse response obtained as a result as a target phase characteristic impulse response.

<Explanation of impulse response generation processing>
Next, the operation of the impulse response generator 11 will be described.

That is, the impulse response generation process performed by the impulse response generation device 11 will be described below with reference to the flowchart of FIG.

In step S11, the 0-packing processing unit 21 performs 0-packing processing on the supplied input impulse response and supplies it to the FFT processing unit 22.

For example, in step S11, as described with reference to FIGS. 4 and 6, 0-packing processing is performed in which 0 data is added to the rear side and the front side in the time direction in the input impulse response. In the zero-packing process, 0 data is added at least to the front side in the time direction in the input impulse response.

In step S12, the FFT processing unit 22 performs FFT on the input impulse response after the 0-packing process supplied from the 0-packing processing unit 21, and the phase characteristic of the frequency characteristics obtained as a result is the phase characteristic of the IFFT processing unit 23. Supply to.

In step S13, the IFFT processing unit 23 performs IFFT on the frequency characteristic including the flat amplitude characteristic supplied from the outside and the phase characteristic supplied from the FFT processing unit 22, and obtains the impulse response obtained as a result. It is supplied to the fade processing unit 24.

In step S14, the fade processing unit 24 performs fade processing on the impulse response supplied from the IFFT processing unit 23, and outputs the impulse response obtained as a result as the target phase characteristic impulse response.

For example, in the fade processing, the target phase characteristic impulse response is generated by fading out the rear side (tail side) of the impulse response supplied from the IFFT processing unit 23 in the time direction and converging it to 0. If the impulse response obtained by IFFT converges to 0, no special fade processing is required.

Further, if, for example, an impulse response having the inverse characteristic of the phase characteristic of the headphone is used as the input impulse response, the phase characteristic of the headphone is canceled as the target phase characteristic impulse response, that is, an impulse response having the inverse characteristic of the phase characteristic of the headphone is obtained. be able to.

When the target phase characteristic impulse response is generated in this way, the impulse response generation process ends.

As described above, the impulse response generator 11 performs a zero-packing process of adding 0 data to the front side in the time direction at least in the input impulse response, and FFT, IFFT, and fade to the zero-packed input impulse response. The target phase characteristic impulse response is generated by performing the processing.

By doing so, it is possible to obtain a target phase characteristic impulse response that functions as a filter capable of adding the target phase characteristic without changing the amplitude characteristic. This makes it possible to obtain a desired phase characteristic by using the target phase characteristic impulse response without changing the amplitude characteristic.

<Second Embodiment>
<Configuration example of impulse response generator>
Further, when the target phase characteristic impulse response is generated by the above-mentioned method A2, the impulse response generator is configured as shown in FIG. 10, for example. In FIG. 10, the same reference numerals are given to the parts corresponding to the cases in FIG. 8, and the description thereof will be omitted as appropriate.

The impulse response generation device 51 shown in FIG. 10 includes an FFT processing unit 61, a zero filling processing unit 62, an FFT processing unit 63, an arithmetic processing unit 64, an IFFT processing unit 23, and a fade processing unit 24. The impulse response generation device 51 is configured to include an FFT processing unit 61 to an arithmetic processing unit 64 in place of the zero-packing processing unit 21 and the FFT processing unit 22 in the impulse response generation device 11.

The FFT processing unit 61 is supplied with an impulse response having a target phase characteristic, that is, an input impulse response, which is used for generating the target phase characteristic impulse response.

The FFT processing unit 61 performs FFT on the supplied input impulse response, and supplies the phase characteristic of the frequency characteristics obtained as a result to the arithmetic processing unit 64. If the target phase characteristic itself can be obtained and the target phase characteristic can be supplied to the arithmetic processing unit 64, the FFT processing unit 61 does not need to be provided.

A simple impulse used for generating a target phase characteristic impulse response is supplied to the zero-packing processing unit 62. The 0-packing processing unit 62 performs 0-packing processing on the supplied simple impulse and supplies it to the FFT processing unit 63.

The FFT processing unit 63 performs FFT on the simple impulse after the 0-packing process supplied from the 0-packing processing unit 62, and supplies the phase characteristic of the frequency characteristics obtained as a result to the arithmetic processing unit 64.

The arithmetic processing unit 64 performs arithmetic processing based on the phase characteristics supplied from the FFT processing unit 61 and the phase characteristics supplied from the FFT processing unit 63, and supplies the resulting phase characteristics to the IFFT processing unit 23. To do. Here, addition processing or subtraction processing is performed as arithmetic processing.

<Explanation of impulse response generation processing>
Next, the operation of the impulse response generator 51 will be described.

That is, the impulse response generation process performed by the impulse response generation device 51 will be described below with reference to the flowchart of FIG.

In step S41, the FFT processing unit 61 performs FFT on the supplied input impulse response, and supplies the phase characteristic of the frequency characteristics obtained as a result to the arithmetic processing unit 64.

In step S42, the 0-packing processing unit 62 performs 0-packing processing on the supplied simple impulse and supplies it to the FFT processing unit 63. In the zero-packing process, 0 data is added to the front side of the simple impulse in the time direction, and the simple impulse is appropriately delayed.

In step S43, the FFT processing unit 63 performs FFT on the simple impulse after the 0-packing process supplied from the 0-packing processing unit 62, and transfers the phase characteristic of the frequency characteristics obtained as a result to the arithmetic processing unit 64. Supply.

In step S44, the arithmetic processing unit 64 performs arithmetic processing based on the phase characteristics supplied from the FFT processing unit 61 and the phase characteristics supplied from the FFT processing unit 63, and the phase characteristics obtained as a result are obtained by the IFFT processing unit. Supply to 23.

For example, when the arithmetic processing unit 64 wants to obtain the same phase characteristics as the input impulse response as the phase characteristics of the target phase characteristic impulse response, the phase characteristics of the input impulse response supplied from the FFT processing unit 61. And the phase characteristic of the simple impulse after the zero-packing process supplied from the FFT processing unit 63 are added, and the phase characteristic obtained as a result is supplied to the IFFT processing unit 23.

On the other hand, the arithmetic processing unit 64 receives the input impulse response supplied from the FFT processing unit 61 when trying to obtain the inverse characteristic of the phase characteristic of the input impulse response as the phase characteristic of the target phase characteristic impulse response. Is subtracted from the phase characteristic of the simple impulse after the zero-packing process supplied from the FFT processing unit 63, and the phase characteristic obtained as a result is supplied to the IFFT processing unit 23.

When addition or subtraction is performed as arithmetic processing for the phase characteristic in this way, the processing of step S45 and step S46 is performed thereafter to end the impulse response generation processing, but these processing are performed in step S13 and step S13 of FIG. Since it is the same as the process of S14, the description thereof will be omitted.

As described above, the impulse response generation device 51 performs a zero-packing process of adding 0 data to the front side in the time direction in the simple impulse, and the target phase is based on the zero-packed simple impulse response and the input impulse response. Generates a characteristic impulse response.

By doing so, it is possible to obtain a target phase characteristic impulse response that functions as a filter capable of adding the target phase characteristic without changing the amplitude characteristic. This makes it possible to obtain a desired phase characteristic by using the target phase characteristic impulse response without changing the amplitude characteristic.

<Third embodiment>
<Configuration example of playback device>
Here, a reproduction device that reproduces the content by using the target phase characteristic impulse response generated by the impulse response generation device 11 and the impulse response generation device 51 described above will be described.

In the following, in order to make the explanation concrete, it is assumed that the content to be reproduced is mastered in a predetermined studio as shown in FIG.

In the example shown in FIG. 12, there is a creator M11 who performs mastering in the studio, and the creator M11 adjusts the amplitude of each band of the content while reproducing the sound of the content by the speaker 91 arranged in the studio. As a mastering task.

Further, the audio signal of the content obtained by mastering is reproduced by a reproduction system including a reproduction device possessed by the listener. Any kind of headphone, speaker, earphone, or the like may be used for reproducing the sound of the content, but the description will be continued below assuming that the headphone is used as a specific example.

The reproduction device used for reproducing the content is configured as shown in FIG. 13, for example.

In the example shown in FIG. 13, the reproduction device 121 includes at least a portable player, a smartphone, a personal computer, or the like capable of controlling the reproduction of audio content, and the headphone 122 is connected to the reproduction device 121.

The reproduction device 121 includes an acquisition unit 131, a speaker phase characteristic convolution unit 132, and a reproduction control unit 133.

In the reproduction device 121, the audio signal of the content obtained by mastering by the creator M11 is supplied to the speaker phase characteristic convolution unit 132.

The acquisition unit 131 acquires and holds the target phase characteristic impulse response from an external device such as the impulse response generation device 11 or the impulse response generation device 51 at an arbitrary timing. Further, the acquisition unit 131 supplies the held target phase characteristic impulse response to the speaker phase characteristic convolution unit 132.

The target phase characteristic impulse response acquired by the acquisition unit 131 is generated by the impulse response generator 11 or the impulse response generator 51 using the input impulse response having the phase characteristic of the speaker 91 used for mastering. Is. That is, the target phase characteristic impulse response is an impulse response having the same phase characteristic as the phase characteristic of the speaker 91.

The target phase characteristic impulse response may not be acquired by the acquisition unit 131 at an arbitrary timing, but may be held in advance by the acquisition unit 131.

Further, in the following, a target phase characteristic impulse response having the same phase characteristic as the phase characteristic of the speaker 91 will be referred to as a speaker characteristic impulse response in particular.

The speaker phase characteristic convolution unit 132 convolves the speaker characteristic impulse response supplied from the acquisition unit 131 with respect to the supplied audio signal, and supplies the audio signal obtained as a result to the reproduction control unit 133.

The reproduction control unit 133 supplies the audio signal supplied from the speaker phase characteristic convolution unit 132 to the headphones 122, and reproduces the sound of the content. In other words, the reproduction control unit 133 controls the reproduction of the sound of the content on the headphones 122.

The headphone 122 reproduces the sound of the content based on the audio signal supplied from the reproduction control unit 133.

Although the playback device 121 is not provided with the headphone 122 here, the headphone 122 may be provided in the playback device 121, or the acquisition unit 131 to the playback control inside the headphone 122. The unit 133 may be provided.

<Explanation of playback process>
Subsequently, the operation of the reproduction device 121 will be described. That is, the reproduction process by the reproduction device 121 will be described below with reference to the flowchart of FIG. At the timing when this reproduction process is started, the speaker characteristic impulse response has already been acquired by the acquisition unit 131.

In step S71, the speaker phase characteristic convolution unit 132 convolves the speaker characteristic impulse response supplied from the acquisition unit 131 with respect to the supplied audio signal, and supplies the audio signal obtained as a result to the reproduction control unit 133.

Thereby, the phase characteristic of the speaker characteristic impulse response, that is, the phase characteristic of the speaker 91 can be added to the sound of the content based on the audio signal.

In step S72, the reproduction control unit 133 supplies the audio signal supplied from the speaker phase characteristic convolution unit 132 to the headphones 122 to reproduce the sound of the content, and the reproduction process ends.

Since the same characteristic as the phase characteristic of the speaker 91 is added to the sound of the content reproduced by the headphone 122, the content that the creator M11 was listening to in the studio to the listener listening to the sound of the content. I can hear the sound with almost the same sound quality as the sound of. Moreover, since the speaker characteristic impulse response can add only the desired phase characteristic to the sound of the content without changing the amplitude characteristic, the gain of the sound of the content does not change.

As described above, the reproduction device 121 reproduces the sound of the content after convolving the speaker characteristic impulse response into the audio signal of the content. By doing so, even when the sound of the content is reproduced by the headphones 122, the phase characteristic of the speaker 91 used for mastering can be added to the sound of the content. That is, a desired phase characteristic can be obtained.

<Fourth Embodiment>
<Configuration example of playback device>
It was explained that the playback device 121 adds the same characteristics as the phase characteristics of the speaker 91 to the sound of the content. However, when the sound of the content is reproduced by the headphone 122, the phase characteristic of the headphone 122 is also added to the sound.

Therefore, not only the same characteristic as the phase characteristic of the speaker 91 is added to the sound of the content, but also the phase characteristic of the headphone 122 is canceled (removed), so that the sound of the content that the creator M11 is listening to in the studio is closer. The sound may be made to be heard by the listener.

In such a case, the reproduction device is configured as shown in FIG. 15, for example. In FIG. 15, the same reference numerals are given to the parts corresponding to the cases in FIG. 13, and the description thereof will be omitted as appropriate.

Headphones 122 are connected to the playback device 161 shown in FIG. Further, the reproduction device 161 includes an acquisition unit 131, a headphone reverse characteristic convolution unit 171, a speaker phase characteristic convolution unit 132, and a reproduction control unit 133.

In particular, the playback device 161 is configured such that the headphone reverse characteristic convolution unit 171 is provided in front of the speaker phase characteristic convolution unit 132 in the playback device 121.

In the reproduction device 161, not only the above-mentioned speaker characteristic impulse response but also the target phase characteristic impulse response having the opposite characteristic to the phase characteristic of the headphone 122 is obtained by the acquisition unit 131, such as the impulse response generation device 11 and the impulse response generation device 51. Obtained from an external device and retained. Hereinafter, the target phase characteristic impulse response having the opposite characteristic to the phase characteristic of the headphone 122 will also be referred to as a headphone inverse characteristic impulse response.

For this headphone inverse characteristic impulse response, for example, an input impulse response having the phase characteristic of the headphone 122 is used, and the target phase characteristic impulse response generated by the impulse response generation device 51 by performing subtraction as arithmetic processing in the arithmetic processing unit 64. Is.

The headphone inverse characteristic impulse response may not be acquired by the acquisition unit 131, but may be held in advance by the acquisition unit 131.

The acquisition unit 131 supplies the held headphone reverse characteristic impulse response to the headphone reverse characteristic convolution unit 171.

The headphone inverse characteristic convolution unit 171 convolves the headphone inverse characteristic impulse response supplied from the acquisition unit 131 with respect to the audio signal of the supplied content, and supplies the audio signal obtained as a result to the speaker phase characteristic convolution unit 132. To do.

<Explanation of playback process>
Next, the operation of the reproduction device 161 will be described. That is, the reproduction process by the reproduction device 161 will be described below with reference to the flowchart of FIG. At the timing when this reproduction process is started, the speaker characteristic impulse response and the headphone reverse characteristic impulse response have already been acquired by the acquisition unit 131.

In step S101, the headphone inverse characteristic convolution unit 171 convolves the headphone inverse characteristic impulse response supplied from the acquisition unit 131 with respect to the audio signal of the supplied content, and the audio signal obtained as a result is the speaker phase characteristic convolution unit. Supply to 132.

As a result, it is possible to add the inverse characteristic of the phase characteristic of the headphone 122 to the sound of the content. In other words, the phase characteristic of the headphone 122, which is added when the sound of the content is reproduced by the headphone 122, is canceled. Moreover, in the convolution of the headphone inverse characteristic impulse response, only the phase characteristic can be adjusted without changing the amplitude (gain) of the sound of the content.

When the headphone inverse characteristic impulse response is convoluted with respect to the audio signal, the processes of steps S102 and S103 are performed thereafter to end the reproduction process, but these processes are the processes of steps S71 and S72 of FIG. Since it is the same as the above, the description thereof will be omitted.

In the reproduction of the content sound by the reproduction device 161, the phase characteristic of the headphone 122 is first canceled with respect to the content sound, and then the phase characteristic of the speaker 91, which is a characteristic to be added, is added.

It should be noted that a target phase characteristic impulse response capable of adding the inverse characteristic of the phase characteristic of the headphone 122 and simultaneously adding the phase characteristic of the speaker 91 is generated, and the target phase characteristic impulse response is convoluted into the audio signal of the content. You may.

However, as in the playback device 161 by separately convolving the speaker characteristic impulse response and the headphone inverse characteristic impulse response, the phase characteristic added to the sound of the content can be freely changed. That is, for example, in the reproduction device 161, it is possible to select an arbitrary speaker 91 from a plurality of speakers 91 of different manufacturers and convolve the speaker characteristic impulse response having the phase characteristic of the selected speaker 91. ..

As described above, the reproduction device 161 convolves the headphone inverse characteristic impulse response into the audio signal of the content, further convolves the speaker characteristic impulse response into the audio signal, and then reproduces the sound of the content.

By doing so, even when the sound of the content is reproduced by the headphone 122, the phase characteristic added by the headphone 122 is canceled and the phase characteristic of the speaker 91 used for mastering is added to the sound of the content. can do. That is, a desired phase characteristic can be obtained. In particular, in the reproduction process described with reference to FIG. 16, it is possible to hear a sound closer to the sound of the content heard by the creator M11 in the studio than in the reproduction process described with reference to FIG. ..

<Fifth Embodiment>
<Configuration example of playback device>
When the sound of the content is reproduced by the headphones 122, if the sound source, for example, the HRTF showing the sound transmission characteristics from the speaker 91 to the creator M11 is convoluted, the sound of the content that the creator M11 is listening to in the studio is closer. You can hear the sound. That is, it is possible to reproduce the listening environment of the studio at the time of mastering.

When convolving the HRTF into the audio signal of the content, the playback device is configured, for example, as shown in FIG. In FIG. 17, the same reference numerals are given to the portions corresponding to the cases in FIG. 15, and the description thereof will be omitted as appropriate.

Headphones 122 are connected to the playback device 201 shown in FIG. Further, the reproduction device 201 includes an acquisition unit 131, a headphone reverse characteristic convolution unit 171, a speaker phase characteristic convolution unit 132, an HRTF convolution unit 211, and a reproduction control unit 133.

In particular, the reproduction device 201 is configured such that the HRTF convolution unit 211 is provided after the speaker phase characteristic convolution unit 132 in the reproduction device 161.

In the reproduction device 201, not only the speaker characteristic impulse response and the headphone reverse characteristic impulse response described above, but also the HRTF is acquired from an external device by the acquisition unit 131 and held. The HRTF may also be held in advance by the acquisition unit 131 instead of being acquired by the acquisition unit 131.

The acquisition unit 131 supplies the held HRTF to the HRTF convolution unit 211.

The HRTF convolution unit 211 convolves the HRTF supplied from the acquisition unit 131 with respect to the audio signal supplied from the speaker phase characteristic convolution unit 132, and supplies the audio signal obtained as a result to the reproduction control unit 133.

The reproduction device 121 shown in FIG. 13 may be provided with the HRTF convolution unit 211.

<Explanation of playback process>
Next, the operation of the reproduction device 201 will be described. That is, the reproduction process by the reproduction device 201 will be described below with reference to the flowchart of FIG. At the timing when this reproduction process is started, the speaker characteristic impulse response, the headphone reverse characteristic impulse response, and the HRTF have already been acquired by the acquisition unit 131.

When the reproduction process is started, the processes of steps S131 and S132 are performed, but since these processes are the same as the processes of steps S101 and S102 of FIG. 16, the description thereof will be omitted.

In step S133, the HRTF convolution unit 211 convolves the HRTF supplied from the acquisition unit 131 with respect to the audio signal supplied from the speaker phase characteristic convolution unit 132, and supplies the audio signal obtained as a result to the reproduction control unit 133. ..

In step S134, the reproduction control unit 133 supplies the audio signal supplied from the HRTF convolution unit 211 to the headphones 122 to reproduce the sound of the content, and the reproduction process ends. As a result, when reproducing the sound of the content, the phase characteristic of the headphone 122 is canceled, and the phase characteristic of the speaker 91 and the sound transmission characteristic in the studio are added.

As described above, the reproduction device 201 reproduces the sound of the content after convolving the headphone inverse characteristic impulse response, the speaker characteristic impulse response, and the HRTF into the audio signal.

By doing so, even when the sound of the content is reproduced by the headphones 122, a desired phase characteristic and a transmission characteristic in a desired listening environment such as a studio are added, and the creator M11 listens in the studio. The listener can hear the sound that is almost the same as the sound of the content.

<Modification example>
<Configuration example of playback device>
It should be noted that a generation unit for generating a target phase characteristic impulse response may be provided inside the reproduction device 121, the reproduction device 161 and the reproduction device 201.

For example, when such a generation unit is provided inside the reproduction device 161, the reproduction device 161 is configured as shown in FIG. In FIG. 19, the parts corresponding to the case in FIG. 15 are designated by the same reference numerals, and the description thereof will be omitted as appropriate.

The reproduction device 161 shown in FIG. 19 includes a generation unit 241, an acquisition unit 131, a headphone reverse characteristic convolution unit 171, a speaker phase characteristic convolution unit 132, and a reproduction control unit 133.

The configuration of the reproduction device 161 shown in FIG. 19 is such that the reproduction device 161 shown in FIG. 15 is further provided with a generation unit 241.

The generation unit 241 corresponds to the impulse response generation device 11 and the impulse response generation device 51. That is, the generation unit 241 performs the same processing as the impulse response generation processing described with reference to FIGS. 9 and 11 to generate a headphone inverse characteristic impulse response and a speaker characteristic impulse response, and supplies the impulse response to the acquisition unit 131.

According to the present technology described in each of the above embodiments and modifications, it is possible to obtain a desired phase characteristic by adjusting only the phase characteristic without changing the amplitude characteristic.

For example, the phase characteristics of an arbitrary speaker used for mastering in music production, particularly the phase characteristics of low frequencies, can be added to the sound source while the amplitude characteristics remain flat. As a result, even when the sound is reproduced using headphones, it is possible to obtain an effect equivalent to that obtained in a mastering studio as a low-frequency sound quality effect.

Moreover, even if the target speaker is unknown, if the impulse response of any general IIR filter that imitates the phase characteristic of the speaker is used as the above-mentioned input impulse response, the obtained speaker characteristic impulse response is used. Therefore, it is possible to add a low-frequency phase characteristic equivalent to that of a speaker without changing the amplitude characteristic.

Further, if the phase characteristic of the headphone, particularly the phase characteristic of the low frequency band is added by the impulse response of the headphone reverse characteristic, the phase characteristic of the headphone, particularly the phase characteristic of the low frequency band can be canceled. Then, after canceling the phase characteristic of the headphones, if the phase characteristic of the speaker, particularly the low frequency characteristic, is further added by the speaker characteristic impulse response, an effect closer to the low frequency sound quality effect in the mastering studio can be obtained.

If the speaker used in the mastering studio can be specified by metadata or the like in the future, an impulse response having the phase characteristics of the speaker may be used as the input impulse response. If the speaker used in the mastering studio cannot be specified, an impulse response such as an IIR type HPF that imitates the phase characteristics of the speaker may be used as the input impulse response.

Furthermore, when playing the sound of the content with headphones, in addition to canceling the phase characteristic of the headphones and adding the phase characteristic of the speaker, by convolving the HRTF with the audio signal of the content, the listening environment in the mastering studio The low-frequency phase characteristics can be simulated with headphones.

<Computer configuration example>
By the way, the series of processes described above can be executed by hardware or software. When a series of processes are executed by software, the programs that make up the software are installed on the computer. Here, the computer includes a computer embedded in dedicated hardware and, for example, a general-purpose personal computer capable of executing various functions by installing various programs.

FIG. 20 is a block diagram showing a configuration example of hardware of a computer that executes the above-mentioned series of processes programmatically.

In a computer, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, and a RAM (Random Access Memory) 503 are connected to each other by a bus 504.

An input / output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a recording unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

The input unit 506 includes a keyboard, a mouse, a microphone, an image sensor, and the like. The output unit 507 includes a display, a speaker, and the like. The recording unit 508 includes a hard disk, a non-volatile memory, and the like. The communication unit 509 includes a network interface and the like. The drive 510 drives a removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

In the computer configured as described above, the CPU 501 loads the program recorded in the recording unit 508 into the RAM 503 via the input / output interface 505 and the bus 504 and executes the above-described series. Is processed.

The program executed by the computer (CPU 501) can be recorded and provided on a removable recording medium 511 as a package medium or the like, for example. Programs can also be provided via wired or wireless transmission media such as local area networks, the Internet, and digital satellite broadcasts.

In a computer, the program can be installed in the recording unit 508 via the input / output interface 505 by mounting the removable recording medium 511 in the drive 510. Further, the program can be received by the communication unit 509 and installed in the recording unit 508 via a wired or wireless transmission medium. In addition, the program can be pre-installed in the ROM 502 or the recording unit 508.

The program executed by the computer may be a program that is processed in chronological order according to the order described in this specification, or may be a program that is processed in parallel or at a necessary timing such as when a call is made. It may be a program in which processing is performed.

Further, the embodiment of the present technology is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present technology.

For example, the present technology can have a cloud computing configuration in which one function is shared by a plurality of devices via a network and jointly processed.

Further, each step described in the above-mentioned flowchart can be executed by one device or can be shared and executed by a plurality of devices.

Further, when one step includes a plurality of processes, the plurality of processes included in the one step can be executed by one device or shared by a plurality of devices.

Further, the present technology can also have the following configurations.

(1)
An acquisition unit that acquires an impulse response with flat or substantially flat amplitude characteristics and a predetermined phase characteristic,
An audio signal processing device including a phase characteristic convolution unit that convolves the impulse response with an input audio signal.
(2)
The audio signal processing device according to (1), wherein the predetermined phase characteristic is a phase characteristic possessed by a predetermined speaker.
(3)
The audio signal processing device according to (1) or (2), further comprising a reproduction control unit that controls reproduction of sound based on the audio signal obtained by convolution of the impulse response in headphones.
(4)
The audio signal processing device according to (3), further comprising an inverse characteristic convolution unit that convolves an impulse response having an inverse characteristic of the phase characteristics of the headphones into the input audio signal.
(5)
The audio signal processing device according to any one of (1) to (4), further comprising an HRTF convolution unit that folds the HRTF into the audio signal obtained by convolution by the phase characteristic convolution unit.
(6)
The audio signal processing device according to any one of (1) to (5), further comprising an impulse response generation unit that generates the impulse response.
(7)
The audio signal processor
Obtaining an impulse response with flat or substantially flat amplitude characteristics and predetermined phase characteristics
An audio signal processing method that convolves the impulse response with an input audio signal.
(8)
Obtaining an impulse response with flat or substantially flat amplitude characteristics and predetermined phase characteristics
A program that causes a computer to perform a process that includes a step of convolving the impulse response into an input audio signal.
(9)
An impulse response generator that generates a target characteristic impulse response having a flat or substantially flat amplitude characteristic and a predetermined phase characteristic.
(10)
A 0-packing processing unit that performs 0-packing processing that adds 0 data to predetermined impulse information,
An impulse information FFT processing unit that performs FFT on the impulse information to which the 0 data is added, and an impulse information FFT processing unit.
The impulse response generator according to (9), further comprising an IFFT processing unit that generates the target characteristic impulse response by performing IFFT based on the phase characteristic obtained by the FFT and the flat amplitude characteristic.
(11)
The impulse response generation device according to (10), wherein the zero-packing processing unit adds the zero data to at least the front side of the impulse information in the time direction.
(12)
The impulse response generator according to (10) or (11), further comprising a fade processing unit that performs a fade process on the impulse response obtained by the IFFT to obtain the target characteristic impulse response.
(13)
The impulse response generator according to any one of (10) to (12), wherein the impulse information is an impulse response having the predetermined phase characteristic.
(14)
The impulse information is a simple impulse,
An impulse response FFT processing unit that performs FFT on an impulse response having a predetermined phase characteristic, and an impulse response FFT processing unit.
Further, it is provided with an arithmetic processing unit that performs an operation based on the phase characteristic obtained by the FFT by the impulse information FFT processing unit and the phase characteristic obtained by the FFT by the impulse response FFT processing unit.
The impulse response generator according to any one of (10) to (12), wherein the IFFT processing unit performs the IFFT based on the phase characteristic obtained by the calculation and the flat amplitude characteristic.
(15)
As the calculation, the arithmetic processing unit adds the phase characteristics obtained by the FFT by the impulse information FFT processing unit and the phase characteristics obtained by the FFT by the impulse response FFT processing unit (14). The impulse response generator according to.
(16)
As the calculation, the arithmetic processing unit subtracts the phase characteristic obtained by the FFT by the impulse response FFT processing unit from the phase characteristic obtained by the FFT by the impulse information FFT processing unit (14). The impulse response generator according to.
(17)
Impulse response generator
A method for generating an impulse response that generates a target characteristic impulse response having a flat or substantially flat amplitude characteristic and a predetermined phase characteristic.
(18)
A program that causes a computer to perform a process including a step of generating a target characteristic impulse response having a flat or substantially flat amplitude characteristic and a predetermined phase characteristic.

11 Impulse response generator, 210 packing processing unit, 22 FFT processing unit, 23 IFFT processing unit, 24 fade processing unit, 61 FFT processing unit, 620 filling processing unit, 63 FFT processing unit, 64 arithmetic processing unit, 121 playback Equipment, 131 acquisition unit, 132 speaker phase characteristic convolution unit, 133 playback control unit, 171 headphone reverse characteristic convolution unit, 211 HRTF convolution unit, 241 generator unit

Claims (18)

An acquisition unit that acquires an impulse response with flat or substantially flat amplitude characteristics and a predetermined phase characteristic,
An audio signal processing device including a phase characteristic convolution unit that convolves the impulse response with an input audio signal. The audio signal processing device according to claim 1, wherein the predetermined phase characteristic is a phase characteristic possessed by a predetermined speaker. The audio signal processing apparatus according to claim 1, further comprising a reproduction control unit that controls reproduction of sound based on the audio signal obtained by convolution of the impulse response with headphones. The audio signal processing device according to claim 3, further comprising an inverse characteristic convolution unit that convolves an impulse response having an inverse characteristic of the phase characteristics of the headphones into the input audio signal. The audio signal processing device according to claim 1, further comprising an HRTF convolution unit that folds the HRTF into the audio signal obtained by convolution by the phase characteristic convolution unit. The audio signal processing device according to claim 1, further comprising an impulse response generation unit that generates the impulse response. The audio signal processor
Obtaining an impulse response with flat or substantially flat amplitude characteristics and predetermined phase characteristics
An audio signal processing method that convolves the impulse response with an input audio signal. Obtaining an impulse response with flat or substantially flat amplitude characteristics and predetermined phase characteristics
A program that causes a computer to perform a process that includes a step of convolving the impulse response into an input audio signal. An impulse response generator that generates a target characteristic impulse response having a flat or substantially flat amplitude characteristic and a predetermined phase characteristic. A 0-packing processing unit that performs 0-packing processing that adds 0 data to predetermined impulse information,
An impulse information FFT processing unit that performs FFT on the impulse information to which the 0 data is added, and an impulse information FFT processing unit.
The impulse response generator according to claim 9, further comprising an IFFT processing unit that generates the target characteristic impulse response by performing IFFT based on the phase characteristic obtained by the FFT and the flat amplitude characteristic. The impulse response generation device according to claim 10, wherein the 0-packing processing unit adds the 0 data to at least the front side of the impulse information in the time direction. The impulse response generator according to claim 10, further comprising a fade processing unit that performs a fade process on the impulse response obtained by the IFFT to obtain the target characteristic impulse response. The impulse response generator according to claim 10, wherein the impulse information is an impulse response having the predetermined phase characteristics. The impulse information is a simple impulse,
An impulse response FFT processing unit that performs FFT on an impulse response having a predetermined phase characteristic, and an impulse response FFT processing unit.
Further, it is provided with an arithmetic processing unit that performs an operation based on the phase characteristic obtained by the FFT by the impulse information FFT processing unit and the phase characteristic obtained by the FFT by the impulse response FFT processing unit.
The impulse response generation device according to claim 10, wherein the IFFT processing unit performs the IFFT based on the phase characteristic obtained by the calculation and the flat amplitude characteristic. Claim 14 that the arithmetic processing unit adds, as the arithmetic, the phase characteristic obtained by the FFT by the impulse information FFT processing unit and the phase characteristic obtained by the FFT by the impulse response FFT processing unit. The impulse response generator according to. Claim 14 that the arithmetic processing unit subtracts the phase characteristic obtained by the FFT by the impulse response FFT processing unit from the phase characteristic obtained by the FFT by the impulse information FFT processing unit as the operation. The impulse response generator according to. Impulse response generator
A method for generating an impulse response that generates a target characteristic impulse response having a flat or substantially flat amplitude characteristic and a predetermined phase characteristic. A program that causes a computer to perform a process including a step of generating a target characteristic impulse response having a flat or substantially flat amplitude characteristic and a predetermined phase characteristic.

Images