JP2016134768A - Audio signal processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an audio signal processor for estimating position information of an object included in a content.SOLUTION: A correlation calculation part 912 calculates mutual correlations between channels in respective divided bandwidths. The calculated mutual correlations are inputted to an object information acquisition unit 172 of a CPU 17. The correlation calculation part 912 also functions as a level detection part for detecting a level of an audio signal of each of the channels. Level information of the audio signal of each channel is inputted to the object information acquisition unit 172 as well. The object information acquisition unit 172 estimates a position of an object on the basis of inputted correlation value and level information of the audio signal of each channel. For example, when a correlation value between an L channel and an SL channel is high (over a prescribed threshold) and the level of the L channel and the level of the SL channel are high, it is estimated that an object exists between a speaker 21L and a speaker 21SL.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an audio signal processing apparatus that performs various processing on an audio signal.

  2. Description of the Related Art Conventionally, a sound field support device that forms a desired sound field in a listening environment is known (see, for example, Patent Document 1). The sound field support device generates a pseudo reflected sound (sound field effect sound) by combining a plurality of channels of audio signals and convolving a predetermined parameter with the combined audio signal.

  On the other hand, in recent years, a sound image localization method using object information added to content has become widespread. The object information includes information indicating the position of each object (sound source).

JP 2001-186599 A

  In the method of sound image localization based on object information, there are cases where the position information itself of the original object cannot be acquired simply by inputting an audio signal after channel distribution based on the listening environment (arrangement of speakers).

  Therefore, an object of the present invention is to provide an audio signal processing apparatus that estimates position information of an object included in content.

  The audio signal processing apparatus according to the present invention is based on an audio signal input unit that inputs audio signals of a plurality of channels, a correlation detection unit that detects a correlation component between channels, and the correlation component detected by the correlation detection unit, Obtaining means for obtaining position information of an object included in the content corresponding to the audio signal.

  For example, when the correlation value between the L channel and the SL channel is high (exceeding a predetermined threshold) and the level of the L channel and the level of the SL channel are high (exceeding the predetermined threshold), the L channel speaker and the SL channel speaker It can be estimated that an object exists between the two. The estimated position of the object can be displayed, for example, on a display unit, or a sound field effect suitable for the position of the object can be given.

  In addition, it is preferable that a band dividing unit that divides the audio signals of the plurality of channels into predetermined bands is provided, and the correlation detection unit detects a correlation component for each band.

  In addition, it is preferable that a level detection unit that detects the level of each divided band is provided, and the acquisition unit acquires the type information of the object based on the level of each divided band.

  According to the present invention, it is possible to estimate position information of an object included in content.

It is a schematic diagram of listening environment. 1 is a block diagram of an audio signal processing device according to a first embodiment. It is the block diagram which showed the functional structure of DSP and CPU. It is a block diagram which shows the functional structure of DSP which concerns on the modification of 1st Embodiment. It is a block diagram which shows the functional structure of DSP which concerns on the modification of 2nd Embodiment. It is a block diagram which shows the functional structure of an analysis part. It is a block diagram which shows the functional structure of the audio signal processing part 14 which concerns on the modification 1 of 1st Embodiment (or 2nd Embodiment). It is a schematic diagram of the listening environment which concerns on 3rd Embodiment. It is a block diagram of the audio signal processing apparatus in 3rd Embodiment.

(First embodiment)
FIG. 1 is a schematic diagram of a listening environment in the first embodiment, and FIG. 2 is a block diagram of an audio signal processing apparatus 1 in the first embodiment. In the present embodiment, as an example, a listening environment in which the center position is the listening position in a square room in plan view is shown. A plurality of speakers (in this example, five speakers 21L, speakers 21R, speakers 21C, speakers 21SL, and speakers 21SR) are installed around the listening position. The speaker 21L is the front left side of the listening position, the speaker 21R is the front right side of the listening position, the speaker 21C is the front center of the listening position, the speaker 21SL is the rear left side of the listening position, and the speaker 21SR is the rear right side of the listening position. is set up. The speaker 21L, the speaker 21R, the speaker 21C, the speaker 21SL, and the speaker 21SR are each connected to the audio signal processing device 1.

  The audio signal processing apparatus 1 includes an input unit 11, a decoder 12, a renderer 13, an audio signal processing unit 14, a D / A converter 15, an amplifier (AMP) 16, a CPU 17, a ROM 18, and a RAM 19.

  The CPU 17 reads an operation program (firmware) stored in the ROM 18 to the RAM 19 and controls the audio signal processing apparatus 1 in an integrated manner.

  The input unit 11 has an interface such as HDMI (registered trademark). The input unit 11 receives content data from a player or the like and outputs the content data to the decoder 12.

  The decoder 12 is composed of a DSP, for example, and decodes content data and extracts an audio signal. In the present embodiment, audio signals are all described as digital audio signals unless otherwise specified.

  The decoder 12 extracts the object information when the input content data corresponds to the object base method. In the object-based method, an object (sound source) included in content is stored as an independent audio signal. The object-based method performs sound image localization (in object units) by distributing the audio signal of the object to the audio signal of each channel by the renderer 13 at the subsequent stage. Therefore, the object information includes information such as position information and level of each object.

  The renderer 13 is made of, for example, a DSP, and performs sound image localization processing based on the position information of each object included in the object information. That is, the renderer 13 distributes the audio signal of each object output from the decoder 12 to the audio signal of each channel with a predetermined gain so that the sound image is localized at a position corresponding to the position information of each object. In this way, a channel-based audio signal is generated. The generated audio signal of each channel is output to the audio signal processing unit 14.

  The audio signal processing unit 14 is formed of a DSP, for example, and performs a process of giving a predetermined sound field effect to the input audio signal of each channel according to the setting of the CPU 17.

  The sound field effect is composed of, for example, a pseudo reflected sound generated from an input audio signal. The generated pseudo reflected sound is added to the original audio signal and output.

  FIG. 3 is a block diagram showing functional configurations of the audio signal processing unit 14 and the CPU 17. The audio signal processing unit 14 functionally includes an addition processing unit 141, a sound field effect sound generation unit 142, and an addition processing unit 143.

  The addition processing unit 141 synthesizes the audio signals of the respective channels with a predetermined gain and mixes them down to a monaural signal. The gain of each channel is set by the control unit 171 in the CPU 17. Generally, when the sound source is speech such as speech, it is preferable to suppress the sound field effect. Therefore, the gain of the front channel and surround channel, which often contains a lot of components such as music, is high. The gain of the center channel that is often included is set low.

  The sound field effect sound generation unit 142 includes, for example, an FIR filter, and generates a pseudo reflection sound by convolving a parameter (filter coefficient) indicating a predetermined impulse response with the input audio signal. The sound field effect sound generation unit 142 performs a process of distributing the generated pseudo reflected sound to each channel. The filter coefficient and the distribution ratio are set by the control unit 171 in the CPU 17.

  The CPU 17 functionally includes a control unit 171 and an object information acquisition unit 172. Based on the sound field effect information stored in the ROM 18, the control unit 171 sets the filter coefficient, the distribution ratio to each channel, and the like in the sound field effect sound generation unit 142.

  The sound field effect information includes information indicating an impulse response of a reflected sound group generated in a certain acoustic space and a sound source position of the reflected sound group. For example, when an audio signal is supplied to the speaker 21L and the speaker 21SL with a predetermined delay amount and a predetermined gain ratio (for example, 1: 1), a pseudo reflected sound can be generated on the left side of the listening position. The sound field effect information includes, for example, a setting for a presence sound field that produces a sound field on the upper front side and a setting for a surround sound field that produces a sound field on the surround side. The sound field effect information to be selected may be fixed to one in the audio signal processing apparatus 1, but it accepts designation of the acoustic space desired by the user, such as a movie theater or a concert hall, and corresponds to the accepted acoustic space. Sound field effect information to be selected may be selected.

  As described above, the sound field effect sound is generated and added to each channel in the addition processing unit 141. Thereafter, the audio signal of each channel is converted into an analog signal by the D / A converter 15, amplified by the amplifier 16, and then output to each speaker. Thereby, a sound field imitating a predetermined acoustic space such as a concert hall is formed around the listening position.

  Then, in the audio signal processing apparatus 1 of the present embodiment, the object information acquisition unit 172 acquires the object information extracted by the decoder 12, and forms an optimal sound field for each object. The control unit 171 sets the gain of each channel of the addition processing unit 141 based on the position information included in the object information acquired by the object information acquisition unit 172. Thereby, the control unit 171 controls the gain of each channel in the sound field effect sound generation unit 142.

  For example, if an object exists in front of the listening position at time t = 1, the object moves to the vicinity of the listening position at time t = 2, and moves to the rear of the listening position at time t = 3. Assume. At time t = 1, the control unit 171 sets the gain of the front channel to the maximum, and sets the gain of the surround channel of the addition processing unit 141 to the minimum. At time t = 2, the control unit 171 sets the gain of the front channel and the surround channel of the addition processing unit 141 to the same level. Thereafter, at time t = 3, the control unit 171 sets the surround channel gain of the addition processing unit 141 to the maximum, and sets the front channel gain to the minimum.

  As described above, the audio signal processing apparatus 1 can dynamically change the formed sound field by dynamically changing the gain of each channel of the addition processing unit 141 corresponding to the moving object. . Therefore, the listener can obtain a more effective sound field effect.

  In the present embodiment, for ease of explanation, an example in which five speakers 21L, 21R, 21C, 21SL, and 21SR are installed and a 5-channel audio signal is processed is shown. The number of channels and the number of channels are not limited to this example. Actually, in order to realize a three-dimensional sound image localization and a sound field effect, it is preferable to install a larger number of speakers at different heights.

  In the above-described example, the audio signal of each channel is synthesized with the gain based on the acquired position information, and a process of generating a pseudo reflected sound is performed by convolving a parameter (filter coefficient) indicating a predetermined impulse response. However, a process of adding a sound field effect may be performed by convolving an individual filter coefficient with the audio signal of each channel. In this case, the ROM 18 stores a plurality of filter coefficients corresponding to the position of the object, and the control unit 171 reads out the corresponding filter coefficients from the ROM 18 based on the acquired position information to generate sound field sound effects. Section 142. In addition, the control unit 171 combines the audio signals of the respective channels with a gain based on the acquired position information, reads out the corresponding filter coefficient from the ROM 18 based on the acquired position information, and sends it to the sound field effect sound generating unit 142. You may perform the process to set.

(Second Embodiment)
Next, FIG. 4 is a block diagram showing a configuration of an audio signal processing device 1B according to the second embodiment. The components common to the audio signal processing apparatus 1 according to the first embodiment shown in FIG. The listening environment according to the second embodiment is the same as the listening environment according to the first embodiment shown in FIG.

  The audio signal processing unit 14 in the audio signal processing device 1B has the function of the analysis unit 91 in addition to the functions shown in FIG. Actually, the analysis unit 91 is realized as another hardware (DSP), but in the second embodiment, it is assumed to be realized as a function of the audio signal processing unit 14 for the sake of explanation. The analysis unit 91 can also be realized by software by the CPU 17.

  The analysis unit 91 extracts object information included in the content by analyzing the audio signal of each channel. That is, in the audio signal processing device 1B of the second embodiment, when the CPU 17 does not acquire (cannot acquire) object information from the decoder 12, the object information is estimated by analyzing the audio signal of each channel.

  FIG. 5 is a block diagram illustrating a functional configuration of the analysis unit 91. The analysis unit 91 includes a band division unit 911 and a calculation unit 912. The band dividing unit 911 divides the audio signal of each channel into a predetermined frequency band. In this example, an example of dividing into three bands of a low frequency (LPF), a mid frequency (BPF), and a high frequency (HPF) is shown. However, the bandwidth to be divided is not limited to three. The band-divided audio signal of each channel is input to the calculation unit 912.

  The calculation unit 912 calculates the cross-correlation between channels in each divided band. The calculated cross-correlation is input to the object information acquisition unit 172 of the CPU 17. The calculation unit 912 also functions as a level detection unit that detects the level of the audio signal of each channel. The level information of the audio signal of each channel is also input to the object information acquisition unit 172.

  The object information acquisition unit 172 estimates the position of the object based on the input correlation value and the level information of the audio signal of each channel.

  For example, as shown in FIG. 6 (A), the correlation value between the L channel and the SL channel in the low frequency (Low) is high (exceeds a predetermined threshold), and as shown in FIG. 6 (B), the low frequency (Low) ), The level of the L channel and the level of the SL channel are high (exceeding a predetermined threshold value), as shown in FIG. 6C, it is assumed that an object exists between the speaker 21L and the speaker 21SL.

  In the high range (High), there is no highly correlated channel, but a high-level audio signal is input in the mid-range (Mid) C channel. Therefore, as shown in FIG. 6C, it is assumed that an object is also present near the speaker 21C.

  In this case, the control unit 171 sets the gain of the L channel and the gain of the SL channel to the same level (0.5: 0.5) for the gain set in the addition processing unit 141 in FIG. Set the gain to maximum (1). The gain of other channels is set to the minimum. Thereby, a sound field effect sound in which an optimum contribution rate according to the position of each object is set is generated.

  However, since the high-level signal in the C channel may be related to speech such as speech, the control unit 171 preferably sets the gain with reference to information on the type of object. Information regarding the type of object will be described later.

  At this time, it is preferable that the control unit 171 reads the sound field effect information set for each band from the ROM 18 and sets individual parameters (filter coefficients) for each band in the sound field effect sound generation unit 142. For example, the reverberation time is set short for the low frequency range, and the reverberation time is set long for the high frequency range.

  Note that the more the number of channels, the more accurately the position of the object can be estimated. In this example, all the speakers are arranged at the same height, and the correlation value of the 5-channel audio signal is calculated. However, in order to actually realize a three-dimensional sound image localization and a sound field effect. Since more speakers are installed at different heights and correlation values between more channels are calculated, the position of the sound source can be determined almost uniquely.

  In this embodiment, the audio signal of each channel is divided for each band and the object position information is acquired for each band. However, the configuration for acquiring the object position information for each band is as follows. This is not an essential configuration in the present invention.

(Modification 1)
Next, FIG. 7 is a block diagram illustrating a functional configuration of the audio signal processing unit 14 according to the first modification of the first embodiment (or the second embodiment). The audio signal processing unit 14 according to Modification 1 includes an addition processing unit 141A, a first sound field effect sound generation unit 142A, an addition processing unit 141B, a second sound field effect sound generation unit 142B, and an addition processing unit 143. Yes. Note that the addition processing unit 141B and the second sound field effect sound generation unit 142B are actually configured as different hardware (DSP), but in this example, for the sake of explanation, the functions of the audio signal processing unit 14 are respectively provided. It shall be realized as

  The addition processing unit 141A combines the audio signals of the respective channels with a predetermined gain and mixes them down to a monaural signal. The gain of each channel is fixed. For example, as described above, the gain of the front channel and the surround channel is set high, and the gain of the center channel is set low.

  The first sound field effect sound generation unit 142A generates a pseudo reflected sound by convolving a parameter (filter coefficient) indicating a predetermined impulse response with the input audio signal. The first sound field effect sound generation unit 142A performs a process of distributing the generated pseudo reflected sound to each channel. The filter coefficient and the distribution ratio are set by the control unit 171. Similarly to the example of FIG. 3, it is also possible to accept designation of an acoustic space desired by the user, such as a movie theater or a concert hall, and select sound field effect information corresponding to the accepted acoustic space.

  On the other hand, the control unit 171 sets the gain of each channel of the addition processing unit 141B based on the position information included in the object information acquired by the object information acquisition unit 172. Thereby, the control part 171 controls the gain of each channel in the 2nd sound field effect sound generation part 142B.

  The sound field effect sound generated by the first sound field effect sound generation unit 142A and the sound field effect sound generated by the second sound field effect sound generation unit 142B are respectively added to the audio of each channel by the addition processing unit 143. Added to the signal.

  Therefore, the audio signal processing unit 14 according to the modified example generates a sound field effect sound in which the contribution rate of each channel is fixed as in the related art, while setting the optimum contribution rate according to the position of each object. A field effect sound is generated.

(Modification 2)
Next, an audio signal processing device according to Modification 2 of the first embodiment (or the second embodiment) will be described. The audio signal processing unit 14 and the CPU 17 according to Modification 2 have the same functional configuration as the configuration illustrated in FIG. 3 (or the configuration illustrated in FIG. 7). However, the object information acquisition unit 172 according to the modification 2 acquires information indicating the type of the object in addition to the position information as the object information.

  The information indicating the type of object is information indicating the type of sound source such as speech, musical instrument, sound effect, and the like. The information indicating the type of the object is extracted by the decoder 12 when included in the content data, but can be estimated by the calculation unit 912 in the analysis unit 91.

  For example, the band dividing unit 911 in the analysis unit 91 extracts the first formant (200 Hz to 500 Hz) and second formant (2 kHz to 3 kHz) bands from the input audio signal. If the input signal component includes many components related to serifs or only includes components related to serifs, these first formant and second formant components are included more than other bands.

  Therefore, the object information acquisition unit 172 determines that the type of the object is a serif when the level of the component of the first formant or the second formant is higher than the average level of the entire frequency band.

  The control unit 171 sets the gain of the addition processing unit 141 (or the addition processing unit 141B) based on the type of object. For example, as shown in FIG. 6C, when an object exists on the left side of the listening position and the type of the object is a line, the gains of the L channel and the SL channel are set low. Further, as shown in FIG. 6C, when an object is present in front of the listening position and the type of the object is a line, the gain of the C channel is set low.

(Modification 3)
As a third modification of the second embodiment, the audio signal processing device 1B can display the position of the object on a display unit (not shown) using the estimated position information of the object. Thereby, the user can visually grasp the movement of the sound source. In the case of content such as a movie, in many cases, a video corresponding to a sound source is already displayed on the display unit, but the displayed video has a subjective field of view. Therefore, the audio signal processing device 1B can also display the position of the object as an overhead view centering on its own position, for example.

(Third embodiment)
Next, FIGS. 8A and 8B are schematic views of a listening environment according to the third embodiment, and FIG. 9 is a block diagram of an audio signal processing device 1C according to the third embodiment. The audio signal processing device 1C according to the third embodiment has the same hardware configuration as the audio signal processing device 1 shown in FIG. 2, but further includes a user interface (I / F) 81.

  The user I / F 81 is an interface that accepts user operations, and includes, for example, a switch, a touch panel, a remote controller, or the like provided in a housing of the audio signal processing device. The user designates a desired acoustic space as a listening environment change instruction via the user I / F 81.

  The control unit 171 of the CPU 17 receives designation of the acoustic space and reads out sound field effect information corresponding to the designated acoustic space from the ROM 18. Then, the control unit 171 sets a filter coefficient based on the sound field effect information, a distribution ratio to each channel, and the like in the audio signal processing unit 14.

  Further, the control unit 171 converts the position information of the object acquired by the object information acquisition unit 172 into a position corresponding to the read sound field effect information, and outputs the converted position information to the renderer 13, thereby Rearrange.

  That is, for example, when the designation of the acoustic space of the large concert hall is received, the control unit 171 rearranges the position of the object at a position far from the listening position, so that each object is positioned at a position corresponding to the scale of the large concert hall. Rearrange. The renderer 13 performs sound image localization processing based on the position information input from the control unit 171.

  For example, as shown in FIG. 8A, when the object 51R is arranged on the right front side of the listening position and the object 51L is arranged on the left front side of the listening position, the control unit 171 controls the control unit 171 in FIG. As shown in FIG. 5, when designation of the acoustic space of the large concert hall is received, the object 51R and the object 51L are rearranged at a position away from the listening position. Thereby, not only the sound field environment of the selected acoustic space but also the position of the sound source corresponding to the direct sound can be brought close to the actual acoustic space.

  The control unit 171 also converts the movement of the object into a movement amount corresponding to the selected acoustic space. For example, in a theater or the like, a performer utters a speech while moving dynamically. For example, when the designation of the acoustic space of the large hall is received, the control unit 171 increases the movement amount of the object extracted by the decoder 12 and rearranges the position of the object corresponding to the performer. As a result, it is possible to give a sense of presence as if the performer is performing at the place.

  Further, the user I / F 81 can also accept the designation of the listening position as a listening environment change instruction. For example, after the user selects an acoustic space in a large hall, the user is in a position immediately in front of the stage in the hall, a second-floor seat (a position where the stage is looked down obliquely), a position far from the stage near the exit, etc. Select the listening position.

  The control unit 171 rearranges each object according to the designated listening position. For example, when the listening position is specified immediately before the stage, the object position is rearranged to a position close to the listening position, and when the listening position is specified far from the stage, the object position is changed. Relocate to a position far from the listening position. Further, for example, when the position of the second-floor seat is designated as the listening position (position where the stage is looked down from above), the position of the object is rearranged obliquely as viewed from the listener.

  In addition, when receiving the designation of the listening position, it is preferable to measure the actual sound field (indirect sound arrival timing and direction) at each position and store it in the ROM 18 as sound field effect information. The control unit 171 reads out the sound field effect information corresponding to the designated listening position from the ROM 18. Thereby, a sound field at a position immediately in front of the stage, a sound field at a position far from the stage, and the like can be reproduced.

  The sound field effect information need not be measured at all positions in the actual acoustic space. For example, the direct sound increases at a position immediately in front of the stage, and the indirect sound increases at a position far from the stage. Therefore, for example, when the listening position at the center of the hall is selected, the sound field effect information corresponding to the measurement result at the position immediately before the stage and the sound field effect information corresponding to the measurement result placed at a position far from the stage are obtained. By averaging, the sound field effect information corresponding to the listening position at the center of the hall can be interpolated.

(Application examples)
The audio signal processing device 1B according to the application example acquires information regarding the direction in which the user is facing, using a gyro sensor or the like provided in a terminal worn by the user. The control unit 171 rearranges each object according to the direction in which the user is facing.

  For example, when the listener is facing the right side, the control unit 171 rearranges the position of the object at a position on the left side as viewed from the listener.

  Further, the ROM 18 of the audio signal processing device 1B according to the application example stores sound field effect information for each direction. The control unit 171 reads out the sound field effect information from the ROM 18 according to the direction in which the listener is facing, and sets it in the audio signal processing unit 14. As a result, the user can obtain a sense of reality as if he were in the place.

DESCRIPTION OF SYMBOLS 1, 1B, 1C ... Audio signal processing apparatus 11 ... Input part 12 ... Decoder 13 ... Renderer 14 ... Audio signal processing part 15 ... D / A converter 17 ... CPU
18 ... ROM
19 ... RAM
21C, 21L, 21R, 21SL, 21SR ... Speakers 51L, 51R ... Object 91 ... Analysis units 141, 141A, 141B ... Addition processing unit 142 ... Sound field effect sound generation unit 142A ... First sound field effect sound generation unit 142B ... No. 2 sound field effect sound generation unit 143 ... addition processing unit 171 ... control unit 172 ... object information acquisition unit 911 ... band division unit 912 ... correlation calculation unit

Claims (3)

Audio signal input means for inputting audio signals of a plurality of channels;
A correlation detector for detecting a correlation component between channels;
Acquisition means for acquiring position information of an object included in content corresponding to the audio signal based on the correlation component detected by the correlation detection unit;
An audio signal processing apparatus comprising: A band dividing unit for dividing the plurality of channels of audio signals into predetermined bands,
The audio signal processing apparatus according to claim 1, wherein the correlation detection unit detects a correlation component for each band. A level detection unit that detects the level of each divided band,
The audio signal processing apparatus according to claim 2, wherein the acquisition unit acquires the type information of the object based on the level of each divided band.

Images