JP6694755B2 - Channel number converter and its program - Google Patents

Description

  The present invention relates to a channel number conversion device and a program for generating a reproduction audio signal according to a reproduction environment from a multi-channel audio signal.

  Currently, practical application of multi-channel audio broadcasting (Non-Patent Document 1) such as 22.2 ch is in progress. In addition, in recent years, 5.1-channel multi-channel audio systems are spreading in homes and the like. However, it is assumed that home acoustic systems are often systems that can reproduce only with a channel number less than 22.2 ch. Generally, when an audio signal of a program produced with a predetermined number of channels is reproduced with a smaller number of channels than at the time of production, a channel number conversion process called downmix is performed. Downmix is a process of multiplying the signal of each channel of the acoustic signal at the time of production by a downmix coefficient and adding the signals to calculate an acoustic signal according to the number of channels at the time of reproduction. The downmix coefficient may be predetermined in the standard (for example, Non-Patent Document 2). For example, the downmix coefficient from 5.1ch surround (L, R, C, LFE, Ls, Rs 6 channels) to stereo 2ch (Lt, Rt 2 channels) is specified in ARIB STD-B32 as follows. Has been done.

  Here, the following values are used for the coefficient k that defines the levels of the surround channels (Ls, Rs).

  When the metadata is not transmitted, the following value is used for the coefficient k in the digital television receiver.

  Also, in broadcasting, by attaching metadata for each program, it is possible to specify different downmix coefficients according to the content of the program.

  It should be noted that in recent years, an object-based audio system is often adopted in an audio system used at home, a theater, or the like. In the object-based audio system, various speaker arrangements are adopted, and the reproduction environment is diversified.

"Video CODING, AUDIO CODING AND MULTIPLEXING SPECIFICATIONS FOR DIGITAL BROADCASTING ARIB STANDARD ARIB STD-B32 3.6 version", March 25, 2016, "Video coding, audio coding and multiplexing standard in digital broadcasting" General Association of Radio Industries and Businesses Recommendation ITU-R BS.775-3, "Multichannel stereophonic sound system with and without accompanying picture", Internet <URL: https: //www.itu.int/rec/R-REC-BS.775-3-201208- I / en ＞

  As described above, multi-channel audio broadcasting having a number of channels exceeding 5.1 ch, such as 22.2 ch for 4K / 8K broadcasting, has been proposed. However, at home, the number of speakers corresponding to 22.2 ch cannot be set at a predetermined speaker position, and in many cases, reproduction is performed with a smaller number of speakers than the number assumed at the time of program production. In that case, downmixing is required, but the number of speakers and their arrangement depend on the manufacturer's product specifications, and it is not realistic to specify downmix coefficients for all speaker arrangements. Also, the optimal downmix coefficient may differ depending on the program such as documentary or music program, and even with the same speaker arrangement, it is possible to specify the number of speakers for each program and an appropriate downmix coefficient according to the speaker arrangement. desirable. Under such circumstances, there is a need for a technique for optimally downmixing a multi-channel audio signal according to a reproduction environment (number of speakers, arrangement) and program content.

  Therefore, an object of the present invention is to provide a channel number conversion device and a program thereof that can solve the above problems.

  According to one aspect of the present invention, the channel number conversion device calculates a desired first downmix signal from the multi-channel acoustic signal using the first downmix coefficient, and uses the second downmix coefficient to calculate the desired first downmix signal. A downmix signal calculator that calculates a second downmix signal having the same number of channels as the reference signal from the downmix signal, a difference signal calculator that calculates the difference between the second downmix signal and the reference signal, and the difference signal. A downmix coefficient updating unit that updates the first downmix coefficient and the second downmix coefficient so that the difference calculated by the calculating unit becomes a minimum value or a predetermined threshold value or less.

  According to an aspect of the present invention, the channel number conversion device may further include a reference signal calculation unit that downmixes the multi-channel acoustic signal using a predetermined downmix coefficient and calculates the reference signal. Good.

  According to an aspect of the present invention, the downmix coefficient updating unit may fix the second downmix coefficient and update only the first downmix coefficient.

  According to an aspect of the present invention, the channel number conversion device includes a downmix coefficient storage unit that stores at least one of an initial value of the first downmix coefficient and an initial value of the second downmix coefficient. It may be further provided.

  According to an aspect of the present invention, the downmix coefficient storage unit reproduces an audio signal of each channel included in the multi-channel audio signal and an audio signal of each channel included in the first downmix signal. The first downmix coefficient having an initial value determined based on the positional relationship with the position may be stored.

  According to an aspect of the present invention, the downmix coefficient storage unit includes a reproduction position of an acoustic signal of each channel included in the first downmix signal and an acoustic signal of each channel included in the second downmix signal. The second downmix coefficient having an initial value determined based on the positional relationship with the reproduction position may be stored.

  According to an aspect of the present invention, the downmix coefficient updating unit reproduces an acoustic signal of each channel included in the multi-channel acoustic signal and an acoustic signal of each channel included in the first downmix signal. The first downmix coefficient may be updated based on a constraint condition defined by the similarity of the position to the position.

  According to an aspect of the present invention, the downmix coefficient updating unit includes a reproduction position of an acoustic signal of each channel included in the first downmix signal and an acoustic signal of each channel included in the second downmix signal. The second downmix coefficient may be updated based on a constraint condition determined by the similarity of the position with the reproduction position.

    According to an aspect of the present invention, the downmix coefficient updating unit sets the first downmix coefficient and the first downmix coefficient as a constraint condition that a value of a downmix coefficient between channels having the highest position similarity becomes maximum. The second downmix coefficient may be updated.

  According to one aspect of the present invention, there is provided a program for causing a computer to function as the channel number conversion device according to any one of the above.

  According to the channel number conversion device of the present invention, the downmix coefficient according to the reproduction environment (the number of speakers and the speaker arrangement) can be calculated for each program.

It is a block diagram showing an example of a channel number conversion device in a first embodiment concerning the present invention. It is a 1st figure which shows an example of the downmix coefficient in 1st embodiment which concerns on this invention. It is a 1st figure which shows an example of the channel arrangement of the same plane in 1st embodiment which concerns on this invention. It is a 2nd figure which shows an example of the channel arrangement of the same plane in 1st embodiment which concerns on this invention. It is a figure showing an example of channel arrangement with an upper layer in a first embodiment concerning the present invention. It is a 2nd figure which shows an example of the downmix coefficient in 1st embodiment which concerns on this invention. It is a flow chart which shows an example of channel number conversion processing in a first embodiment concerning the present invention. It is a block diagram which shows an example of the channel number converter in 2nd embodiment which concerns on this invention. It is a flow chart which shows an example of channel number conversion processing in a second embodiment concerning the present invention.

<First embodiment>
Hereinafter, a channel number conversion device according to a first exemplary embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a block diagram showing an example of a channel number conversion device according to the first embodiment of the present invention.
As shown in FIG. 1, the channel number conversion device 10 includes a multi-channel acoustic signal input unit 11, a reference signal input unit 12, a speaker position information input unit 13, a downmix signal calculation unit 14, and a difference signal calculation unit 15. And a downmix coefficient updating unit 16 and a downmix coefficient storage unit 17.
The channel number conversion device 10 converts a predetermined multi-channel audio signal (N channel) into a multi-channel audio signal (hereinafter referred to as a reference signal) (S channel) having a smaller number of channels than the multi-channel audio signal. This is a device for converting into a downmix audio signal for reproduction (M channel) of a desired number of channels based on the information on the speaker position of the channel audio signal and the information on the speaker position of the downmix destination.

An example will be described below in which the predetermined multi-channel audio signal is a 22.2ch audio system of 4k / 8K, the stereo signal is 2ch as a reference signal, and the reproduced audio signal is 7.1ch. However, the number of channels of the multi-channel audio signal, the reference signal, and the downmix signal is not limited to the number in this example. The channel number conversion device 10 is composed of a computer, and may be incorporated in a broadcast receiver such as a television or a media reproducing device such as a home theater, for example.
FIG. 1 shows a configuration in which a multi-channel acoustic signal and a reference signal are input to a channel number conversion device 10 and a downmix signal is output. The downmix signal output from the channel number conversion device 10 is output from, for example, a speaker connected to the reproduction device. At this time, the channel number conversion device 10 can reproduce the impression of the sound that the listener feels when it is output according to the number of speakers and the arrangement of the speakers when the input multi-channel acoustic signal is created, as much as possible. To generate. In order to generate such a downmix signal, the channel number conversion device 10 calculates an appropriate downmix coefficient according to the number of speakers and the speaker arrangement in the reproduction environment.

The multi-channel audio signal input unit 11 inputs a multi-channel audio signal. Here, the multi-channel audio signal is, for example, a 22.2 ch multi-channel audio signal transmitted from a broadcasting station.
The reference signal input unit 12 inputs a reference signal. The reference signal is an acoustic signal having the same content as the multi-channel acoustic signal produced with a smaller number of channels than the desired downmix signal, and is an acoustic signal separately input from the original multi-channel acoustic signal. Here, the reference signal is, for example, a stereo 2ch audio signal broadcast simultaneously with the multi-channel audio signal. Further, the reference signal may be a reference signal generated by downmixing a multi-channel audio signal based on a predetermined downmix coefficient.

  The speaker position information input unit 13 acquires the position information of each of a plurality of speakers defined for the multi-channel audio signal when creating the multi-channel audio signal. Further, the speaker position information input unit 13 acquires position information of each of one or a plurality of speakers in the reproduction environment of the downmix signal. Here, the speaker position information is given as, for example, coordinate information or polar coordinate information.

The downmix signal calculation unit 14 includes a first downmix signal calculation unit 141 and a second downmix signal calculation unit 142. The first downmix signal calculation unit 141 uses a first downmix coefficient (M × N) stored in a downmix coefficient storage unit 17 to be described later to convert a desired number of channels from a multi-channel audio signal (N) having N channels. Calculate a number M of first downmix signals (M).
The second downmix signal calculation unit 142 uses the second downmix coefficient (S × M) stored in the downmix coefficient storage unit 17 to calculate the reference signal from the first downmix signal (M) having the number of channels M calculated. And a second downmix signal (S) having the same number of channels S as.

The difference signal calculation unit 15 calculates the difference between the reference signal and the second downmix signal (S). Further, the difference signal calculation unit 15 determines whether the calculated difference is the minimum or whether the calculated difference is less than or equal to a predetermined threshold. Even if the difference signal calculation unit 15 does not repeat the calculation until the value becomes the minimum value completely, the difference signal calculation unit 15 updates the difference between the first downmix coefficient (M × N) and the second downmix coefficient (S × M). It may be determined that the difference has become the minimum based on that is less than or equal to the threshold.
The downmix coefficient updating unit 16 sets the first downmix coefficient (M × N) and the second downmix coefficient (S × M) so that the difference calculated by the difference signal calculating unit 15 becomes a minimum value or a threshold value or less, or Only the first downmix coefficient (M × N) is corrected. The downmix coefficient updating unit 16 updates the first downmix coefficient (M × N) and the like stored in the downmix coefficient storage unit 17 with the corrected first downmix coefficient (M × N) and the like. As will be described later, the downmix coefficient updating unit 16 may, for example, reproduce the acoustic signal of each channel included in the multi-channel acoustic signal and the acoustic signal of each channel included in the first downmix signal (M). The first downmix coefficient (M × N) is updated based on the positional relationship with the reproduction position of.

  The downmix coefficient storage unit 17 stores various data necessary for the channel number conversion processing. First, the downmix coefficient storage unit 17 stores the initial value of the first downmix coefficient (M × N) and the initial value of the second downmix coefficient (S × M). Then, the downmix signal calculation unit 14 starts calculation of the first downmix coefficient (M × N) and the second downmix coefficient (S × M) using these initial values. The first downmix coefficient (M × N) initial value and the second downmix coefficient (S × M) initial value may be set by generating a random number, for example. Alternatively, a numerical value may be set according to the angle difference or distance difference between the positions of the channels before and after downmixing. Further, when the downmix coefficient is determined by the standard and the like, and when the channel number conversion device 10 corrects each program, the downmix coefficient defined by the standard may be used as the initial value.

FIG. 2 is a first diagram showing an example of the downmix coefficient in the first embodiment according to the present invention. An outline of the channel number conversion processing of this embodiment will be described with reference to FIG.
(M-Ch ₁ , ..., M-Ch _N ) shown in FIG. 2 is an N-channel multi-channel acoustic signal.
Also, the second item of the matrix shown in FIG.

Is a first downmix coefficient (M × N) of M rows and N columns. The first downmix signal calculation unit 141 uses the first downmix coefficient (M × N) to calculate the first downmix signal (M) of the desired number M of channels from the multi-channel acoustic signal.
Also, the first item of the matrix shown in FIG.

Is a second downmix coefficient (S × M) of S rows and M columns. The second downmix signal calculation unit 142 uses the second downmix coefficient (S × M) to output the second downmix signal (S) having the same number S of channels as the reference signal from the first downmix signal (M). To calculate. In FIG. 2, (Lt, Rt) is the second downmix signal (S). The difference signal calculation unit 15 calculates the difference between the second downmix signal (S) “(Lt, Rt)” and the reference signal “(L, R)”, the energy difference between the respective signals, the root mean square error, 1− It is calculated by a method such as a normalized cross-correlation coefficient (subtracting the normalized cross-correlation coefficient of two signals from 1). Note that the difference calculation method is not limited to these methods.

The downmix coefficient updating unit 16 is based on the difference between the second downmix signal (S) calculated by the difference signal calculating unit 15 and the reference signal, and the first downmix coefficient (M × N) and the second downmix coefficient ( S × M) is updated. A genetic algorithm, a steepest descent method, a stochastic gradient descent method, or the like can be used as an algorithm for the downmix coefficient updating unit 16 to update the downmix coefficient so that the difference becomes small. If so, another method may be used.
Further, when the downmix coefficient updating unit 16 updates the downmix coefficient, the first downmix coefficient (M × N) and the second downmix coefficient (S × M) may be changed at the same time. Alternatively, the downmix coefficient updating unit 16 may change only the first downmix coefficient (M × N) while fixing the second downmix coefficient (S × M) to a predetermined value. Further, the downmix coefficient updating unit 16 may change the downmix coefficient in order from the audio signal of the channel having the higher importance according to the importance of the channel. The importance of the channel is defined by the playback position, for example, the channel corresponding to the speaker located in the front is set to have high importance, and the channel including the dialog audio signal is set to have high importance. It may be set according to the content of the audio signal. The importance level is added to the multi-channel audio signal as metadata, and may be input separately, stored in advance, or designated by the user. The voice included in the dialog voice signal is not necessarily limited to the voice of the dialog (dialogue). An audio signal mainly composed of human voice such as narration may be used as the dialog audio signal.

  The downmix coefficient updating unit 16 outputs the second downmix signal (S) again using the first downmix coefficient (M × N) and the second downmix coefficient (S × M). The downmix coefficient updating unit 16 repeats the process of updating the downmix coefficient until the difference becomes minimum or the difference becomes equal to or less than a predetermined threshold value. When the difference signal calculation unit 15 determines that the difference is less than or equal to the threshold value or the minimum value, the downmix signal calculation unit 14 (second downmix signal calculation unit 142) causes the finally updated first downmix coefficient (M ×). N) is used to calculate the desired first downmix signal (M) from the original multi-channel audio signal and output to the playback device.

FIG. 3 is a first diagram showing an example of coplanar channel arrangement in the first embodiment according to the present invention.
FIG. 3A shows an example of the arrangement of the channels included in the multi-channel acoustic signal before conversion. FIG. 3A shows an example of channel arrangement in the middle layer of 22.2 ch. In FIG. 3A, for example, the channel “FC” is located in front of the user, and the channels “FL” “FLc”, “FC”, “FRc”, and “FR” are located in front of the user. Further, for example, the channel “SiL” is located on the left side of the user, the channel “SiR” is located on the right side of the user, and the channel “BC” is located on the rear side of the user.
FIG. 3B shows an example of the arrangement of the channels included in the desired first downmix signal (M) after conversion. FIG. 3B shows, as an example, an example of channel arrangement of 7.1ch. For example, the channel “Cm” is located in front of the user, and the channels “Lm”, “Cm”, and “Rm” are located in front of the user. A high degree of importance may be set for these front or front channels.
FIG. 3C shows an example of the arrangement of the channels included in the reference signal. As an example, FIG. 3C shows an example of channel arrangement of 2ch.
The channels illustrated in FIGS. 3B and 3C are channels corresponding to the height of the middle layer of 22.2 ch.

  The channel number conversion device 10 of the present embodiment calculates the first downmix signal (M) while maintaining the acoustic impression of the original multi-channel audio signal as much as possible. Specifically, the downmix coefficient updating unit 16 repeats the above-described update processing of the first downmix coefficient (M × N) and the like so that the acoustic impression of the multi-channel audio signal can be retained as much as possible. One downmix coefficient (M × N) is calculated, and the first downmix coefficient (M × N) and the like are updated so that the difference between the second downmix signal (S) and the reference signal converges. Therefore, in the number-of-channels conversion device 10, in the calculation of the first downmix coefficient (M × N), a constraint condition for holding the characteristics of the multi-channel audio signal is required. For example, when the reference signal is a mono signal (1 ch) or a stereo signal (2 ch) and the desired number of channels is 5.1 ch or 7.1 ch, the channel position of the reference signal is the channel arrangement of the desired number of channels. Is completely included in. In this case, if the constraint condition is not set, the first downmix coefficient (M × N) for converting the original multi-channel sound to the desired number of channels has the same number of channels as the reference signal from the original multi-channel sound signal. May be the same as the downmix coefficient of. This is because the first downmix signal (M) converted by such a first downmix coefficient (M × N) has a speaker group installed according to the arrangement of the channels illustrated in FIG. 3B. , The sound reproduced at that time is output only from the speakers corresponding to the channels “Lm” and “Rm” arranged at the same positions as “Lt” and “Rt” in FIG. 3C. This is because the target sound does not have the "maintaining the acoustic impression of the multi-channel audio signal". From this, too, in order to keep the characteristics of the original multi-channel sound as much as possible, a constraint condition is necessary for the calculation of the first downmix coefficient (M × N).

  Next, some of the constraint conditions will be described using the channel arrangement illustrated in FIG. The channel arrangement in each number of channels illustrated in FIG. 3 is intended for a group of channels arranged on substantially the same plane in each number of channels before and after conversion. For example, in a 22.2 ch multi-channel audio signal, in addition to the middle layer illustrated in FIG. 3A, there are upper and lower layers, and when these are included, conversion from the upper and lower layers having different heights is considered. Must. In FIG. 3, these conversions are not taken into consideration, and constraint conditions for conversions between channels arranged in substantially the same plane will be described.

(Restriction condition by position similarity)
The constraint condition may be defined by, for example, the degree of similarity between the original multi-channel sound channel position and the desired down-mix destination channel position. The position similarity may be defined by, for example, a change in distance and angle between each channel position before and after conversion with respect to the listening position of the user (in the case of FIG. 3, the center of the circle). For example, the similarity between the position of “FC” in FIG. 3A and the position of “Cm” in FIG. 3B after conversion is high (both are in front of the user and the distance and angle before and after conversion are the same). In such a case, the downmix coefficient for converting the audio signal assigned to the channel “FC” to the audio signal assigned to the channel “Cm” is restricted to be set to “1.0”, for example. It may be set as a condition.
Alternatively, the constraint condition may be set so that the downmix coefficient in this case becomes maximum. When the constraint condition is set to "1.0", the amount of downmix coefficient calculation can be reduced.

(When the distance from the user does not change before and after downmix)
In the case of this example, first, "FC", which is one of the reproduction positions of the audio signals of the channels included in the multi-channel audio signal, and one of the reproduction positions of the audio signals of the channels included in the downmix signal, are selected. A certain “Cm” is associated in advance. Then, the constraint condition is that the value of the downmix coefficient from "FC" to "Cm" is set to "1.0", and the value of the downmix coefficient from "FC" to another channel is "0". May be

  Further, for example, the similarity between the positions of “FC” in FIG. 3A and “Lm” and “Rm” in FIG. 3B after conversion is slightly high (both in front of the user, The distance from the user does not change before and after the conversion, and the angle changes, for example, slightly more than 20 degrees). In such a case, the downmix coefficient for converting the audio signal assigned to the channel “FC” to the audio signal assigned to the channels “Lm” and “Rm” according to the position similarity is, for example, , “Same value less than or equal to 1.0” may be set as a constraint condition.

  Similarly, for example, the position similarity between “FC” in FIG. 3A and “Lssm” and “Rssm” in FIG. 3B after conversion is low (the distance from the user changes before and after conversion. No, but the angle differs by 110 degrees or more). In such a case, a downmix coefficient for converting the audio signal assigned to the channel “FC” to the audio signals assigned to the channels “Lssm” and “Rssm” is based on the fact that the position similarity is low. May be set as a constraint condition that "0" is set, for example.

  Further, for example, the position similarity between “BL” in FIG. 3A and “Lrsm” in FIG. 3B after conversion, and “BR” in FIG. The degree of similarity with the position of “Rrsm” in b) is approximately the same (both conversion from a channel on the left or right side of the user to a channel on the same side and diagonally rearward). In this case, since the position similarity is similar, the downmix coefficient for converting the audio signal assigned to the channel “BL” to the audio signal assigned to the channel “Lrsm” and the channel “BR”. The constraint condition may be that the same value is set for the downmix coefficient for converting the audio signal assigned to “” to the audio signal assigned to the channel “Rrsm”.

  Also, for example, a downmix coefficient for converting the audio signal assigned to “BL” in FIG. 3A to the converted audio signal assigned to “Lssm” and “Lrsm” in FIG. 3B. For, the distance from the user is the same for all channels. However, the position of “BL” before conversion (angle from the user) is closer to “Lrsm”, and “Lssm” is farther than “Lrsm”. The constraint condition may be set so that the value becomes maximum. Alternatively, the constraint condition may be set such that the value of the downmix coefficient to “Lssm” is the second largest. Further, based on the opening angle formed by the line connecting the center of the circle and each of “Lssm” and “Lssm” with reference to the line connecting the center of the circle and the position of “BL”, the value of each downmix coefficient is calculated. The constraint condition may be to set the downmix coefficient so that the ratio is the reciprocal of the ratio of the opening angle.

(When the speaker cannot be placed in the ideal position)
FIG. 3 illustrates a case where the distance from the user does not change before and after the downmix. Next, the constraint conditions will be described with reference to FIG. 4 by taking an example in which the speaker corresponding to a certain channel cannot be placed in an ideal position due to the layout of the room where the user installs the speaker and the like. I do.

FIG. 4 is a second diagram showing an example of coplanar channel arrangement in the first embodiment according to the present invention.
FIG. 4A shows an example of channel arrangement in the multi-channel acoustic signal before conversion. FIG. 4B shows an example of channel arrangement in the desired first downmix signal (M) after conversion. In FIG. 4B, the positions of the speakers corresponding to the channels “Lssm” and “Lrsm” are deviated from the predetermined “Lssm” and “Lrsm” positions. In the case of this example, the speaker of “Lssm” is installed at a position of a distance v from the position of the channel “BL” in the multi-channel audio signal before conversion, and the speaker of “Lrsm” in the multi-channel audio signal before conversion. It is assumed that it is installed at a position at a distance u from the position of the channel “BL” (v> u). Further, as shown in the figure, it is assumed that the speaker position of “Lssm” from another channel is farther than the distance y from the channel “SiL” to “Lssm”. In this case, the constraint condition may be set such that the value of the downmix coefficient from “SiL” to “Lssm” is maximized. Further, the constraint condition may be set such that the value of the downmix coefficient from “BL” to “Lrsm” is the first largest and the value of the downmix coefficient to “Lssm” is the second largest. Also, based on the distances u and v, the ratio of the value of the downmix coefficient from “BL” to “Lssm” and the value of the downmix coefficient from “BL” to “Lrsm” becomes v: u. Setting the downmix coefficient may be set as a constraint condition.

  Further, in FIG. 4B, the position of the speaker corresponding to the channel “Cm” is laterally displaced from the position of the predetermined “Cm” by the distance w. “FC” and “Cm” are associated in advance, the value of the downmix coefficient from “FC” to “Cm” is “1.0”, and the value of the downmix coefficient to other channels is “0”. An example of the constraint condition that is “. In this case, only when the position similarity between the position of “FC” before conversion and the position of “Cm” including the shift of w after conversion (distance difference in this case) is within a predetermined range, Constraints may be applied.

  The user may input the coordinate information of the speaker position at home into the channel number conversion device 10. Alternatively, the sound reproducing device may input the coordinate information obtained by using the function of detecting the position of each speaker provided in the sound reproducing device to the channel number converting device 10.

(Example of channel arrangement in multi-channel audio signal before conversion)
Next, the constraint condition of the downmix coefficient in the conversion from the upper layer to the middle layer will be described with reference to FIG.
FIG. 5 is a diagram showing an example of a channel arrangement with an upper layer according to the first embodiment of the present invention.
FIG. 5A shows an example of channel arrangement in the multi-channel acoustic signal before conversion. FIG. 5A shows an example of a channel arrangement including two layers, an upper layer and a middle layer. In FIG. 5A, the channel “TpFL” is diagonally to the left of the upper layer user, the channel “TpFR” is diagonally to the right of the upper layer user, the channel “TpBL” is diagonally to the left of the upper layer user, and the channel “TpBR”. Is located diagonally to the right of the user in the upper layer. FIG. 5B shows an example of channel arrangement in the desired first downmix signal (M) after conversion. FIG. 5C shows an example of channel arrangement in the reference signal.

  For example, the similarity between the positions of “TpFL” in FIG. 5A and “L” in FIG. 5B after conversion is both diagonally left front of the user, but the opening angle from the user is slightly different. Change. Further, there is a difference that "TpFL" is located in the upper layer and "L" is located in the middle layer. Therefore, the downmix coefficient for converting the audio signal assigned to the channel "TpFL" to the audio signal assigned to the channel "L" has a value of, for example, "1.0" or "1.0 or less". The setting may be defined as a constraint condition. When the constraint condition is set to "1.0", the amount of downmix coefficient calculation can be reduced.

  In the case of this example, the “TpFL” in the upper layer and the “L” existing in the vicinity of the position corresponding to the “TpFL” in the middle layer are previously associated with each other, and the downmix coefficient from “TpFL” to “L” is set. The constraint condition may be that the value of is, for example, “1.0”, and the value of the downmix coefficient from “TpFL” to another channel is “0”.

  Further, for example, the degree of similarity between the positions of “TpFL” in FIG. 5A and “Ls” and “Rs” in FIG. 5B after conversion is low (the distance from the user increases before and after conversion). In, the angle is also very different, there is a difference between the upper and middle layers). In such a case, the downmix coefficient for converting the audio signal assigned to the channel "TpFL" to the audio signal assigned to the channels "Ls" and "Rs" should be set to "0", for example. May be defined as a constraint condition.

  Further, for example, the similarity of the position between “TpFL” in FIG. 5A and “L” in FIG. 5B after conversion, and “TpFR” in FIG. The degree of similarity with the position of “R” in b) is about the same. In such a case, as with the downmix coefficient on the same plane, the downmix coefficient for converting the audio signal assigned to the channel "TpFL" to the audio signal assigned to the channel "L" and the channel "TpFR". The constraint condition may be that the same value is set to the downmix coefficient for converting the audio signal assigned to “” to the audio signal assigned to the channel “R”.

  Further, for example, based on the distance between “TpFL” in FIG. 5A and “L” in FIG. 5B, and the distance between “TpFL” in FIG. 5A and “C” in FIG. 5B. , "TpFL" is closer to "L", so the value of the downmix coefficient for converting the audio signal assigned to "TpFL" to the audio signal assigned to "L" should be set to the maximum value. May be defined as a constraint condition.

  In this way, the constraint condition is that the distance or opening angle between the channel position of the conversion source multi-channel audio signal and the channel position of the downmix destination is calculated, and the downmix coefficient to the nearest channel position is maximized. By doing so, it becomes possible to calculate a downmix coefficient that can retain the characteristics of the original multi-channel audio signal.

(Restriction condition based on ease of listening)
Returning to FIGS. 3 and 5, another example of another constraint condition will be described. The restraint condition may be defined in terms of the audibility of sound. For example, when a rear channel or a side channel is downmixed with a front channel, it may be difficult to hear a voice such as a dialog voice signal assigned to the front channel. In such a case, in order to ensure audibility, the downmix coefficient from the audio signal assigned to the channel located at least one of the rear side and the side to the audio signal assigned to the front channel is set to "1. The constraint condition may be defined as multiplication by a correction value smaller than "0".

  Similarly, when down-mixing channels from the upper layer, the lower layer, or both to the channels in the middle layer, the sounds in the middle layer may be difficult to hear due to the sounds in the upper and lower layers. Therefore, the use of a correction value smaller than 1.0 as the downmix coefficient from the channels in the upper layer, the lower layer, or both to the channel in the middle layer may be added as a constraint condition.

  Further, although the channel arrangement is usually bilaterally symmetric, if the balance of the volume of sounds heard from the left and right changes, the impression when reproducing a multi-channel audio signal may change greatly. Therefore, when calculating the downmix coefficient from the channel arranged in the symmetrical position to the corresponding channel arranged in the symmetrical position, the downmix coefficient is arranged in the symmetrical position included in the original multi-channel audio signal. It may be added as a constraint condition that the same numerical value is used for the downmix coefficient from the channel to the corresponding channel arranged at the symmetrical position.

  It may be desirable that the characteristics of the downmix signal do not exactly match the characteristics of the original multi-channel audio, as in the case where it is easier to hear if the dialog speech signal from the front is emphasized. The ease of listening for the user can be secured by applying the constraint condition based on the ease of listening.

(Restriction by importance)
Next, the constraint condition based on the degree of importance will be described. The constraint condition may be defined in terms of importance. For example, in the case of a news program, a dialog voice signal by an announcer or the like becomes the most important. In such a case, for example, with respect to the channels located on the front side (for example, “L”, “C”, and “R” in FIG. 5), it is considered that these are channels corresponding to the dialog voice signal, and therefore, the importance is high. It may be set. The importance level is input, for example, as metadata of a multi-channel audio signal. Alternatively, it may be set in the downmix coefficient updating unit 16 by being input by the user.
In this case, for example, setting a maximum value to the downmix coefficient from “FL” to “L”, “FC” to “C”, and “FR” to “R” may be added as a constraint condition.
By applying an importance constraint, it is possible to downmix to emphasize the impression of a particular acoustic signal.

  The constraint condition described above may be applied not only when calculating the first downmix coefficient (M × N) but also when calculating the second downmix coefficient (S × M). In addition, regarding the value of each element of the initial value of the first downmix coefficient (M × N) and the initial value of the second downmix coefficient (S × M), the value of the downmix coefficient is “1.0” depending on the constraint condition. Then, in the case where it is predetermined, "1.0" may be set at the stage of the initial value for the element corresponding to the downmix coefficient between the channels. Further, even when the value is not set to “1.0” or the like due to the constraint condition, each of the initial value of the first downmix coefficient (M × N) and the initial value of the second downmix coefficient (S × M) is obtained. The value of the element may be set to a predetermined value in consideration of the constraint condition based on the above-described position similarity and the like.

Next, with reference to FIG. 6, an example of the first downmix coefficient (M × N) and the second downmix coefficient (S × M) calculated by imposing these constraint conditions will be described.
FIG. 6 is a second diagram showing an example of the downmix coefficient in the first embodiment according to the present invention.
The first downmix coefficient (M × N) and the second downmix coefficient (S × M) illustrated in FIG. 6 are calculated by the downmix coefficient updating unit 16 so as to satisfy the constraint condition.
The upper diagram of FIG. 6 shows each channel included in the desired first downmix signal (M) illustrated in FIG. 5B from each channel signal included in the multi-channel acoustic signal illustrated in FIG. 5A. The example of the 1st downmix coefficient (MxN) to a signal is shown. For example, “L” and “R”, “FL” and “FR” are arranged at symmetrical positions. Therefore, a downmix coefficient for converting an audio signal assigned to "FL" to an audio signal assigned to channel "L", and an audio signal assigned to channel "R" from an audio signal assigned to "FR". The same value "C1" is set as the downmix coefficient for performing the conversion with. Moreover, “FL” and “L” are arranged at substantially the same position. Similarly, “FR” and “R” are arranged at almost the same position. Therefore, the size of “C1” may be “1.0”, for example. Further, for example, the downmix coefficient “k1” of the downmix coefficient “k1C1” for converting the audio signal assigned to “TpFL” to the audio signal assigned to the channel “L” is downmixed from the upper layer to the middle layer. It is an example of a correction value by which a coefficient is multiplied. Here, k1 is a value smaller than 1. Further, comparing the distance between “SiL” and “L” and the distance between “SiL” and “Ls”, the distance between “SiL” and “Ls” is shorter. Therefore, a large value is set for the downmix coefficient from “SiL” to “Ls” (C4 ≧ C3). Similarly, comparing the distance between “BL” and “Ls” and the distance between “BR” and “Ls”, the distance between “BL” and “Ls” is shorter. Therefore, for “BL” and “BR”, a larger value is set for the downmix coefficient from “BL” to “Ls” (C5 ≧ C6). In addition, the downmix coefficient for converting the audio signals assigned to the upper layers “TpBL” and “TpBR” to the audio signals assigned to the channel “Ls” is the downmix coefficient from the upper layer to the middle layer. The correction value “k1” to be used is included.

  The lower part of FIG. 6 shows the second downmix coefficient (S × M) from each channel signal in the desired first downmix signal (M) illustrated in FIG. 5B to the reference signal illustrated in FIG. 5C. ) Is shown. The above constraint condition can also be applied to the second downmix coefficient (S × M). For example, k2 is a correction value used for the downmix coefficient from the rear channel to the front channel. Since "Lt" is closer to "L" than "C", a larger value is set for the downmix coefficient from "L" to "Lt" (Ct1> Ct2).

FIG. 7 is a flowchart showing an example of the channel number conversion processing in the first embodiment according to the present invention.
The flow of the channel number conversion processing of this embodiment will be described with reference to FIG.
As a premise, the position information (coordinate information) of each channel of the multimix audio signal, the number of speakers in the reproduction environment of the desired downmix signal, and the position information of each speaker are input to the channel number conversion device 10 in advance, and the speaker position is adjusted. The information input unit 13 receives input of these pieces of information. Further, the speaker position information input unit 13 outputs the position information of each channel of the multimix acoustic signal and the position information of each speaker in the reproduction environment to the downmix coefficient updating unit 16. In the downmix coefficient updating unit 16, various constraint conditions for downmix coefficient calculation are set. Further, the downmix coefficient storage unit 17 stores the initial value of the first downmix coefficient (M × N) and the initial value of the second downmix coefficient (S × M).

First, in step S11, the reference signal input unit 12 inputs a reference signal. The reference signal input unit 12 outputs the input reference signal to the difference signal calculation unit 15. In addition, in parallel with step S11, the multi-channel acoustic signal input unit 11 inputs the multi-channel acoustic signal in step S12. Then, the multi-channel acoustic signal input unit 11 outputs the multi-channel acoustic signal to the downmix signal calculation unit 14.
Next, in step S13, in the downmix signal calculation unit 14, the first downmix signal calculation unit 141 calculates the first downmix signal (M). Specifically, the first downmix signal calculation unit 141 reads and acquires the initial value of the first downmix coefficient (M × N) from the downmix coefficient storage unit 17, and the multichannel audio signal is obtained with this initial value. Downmixing is performed to calculate a first downmix signal (M). Then, the first downmix signal calculation unit 141 outputs the first downmix signal (M) to the second downmix signal calculation unit 142.
Next, in step S14, the second downmix signal calculation unit 142 calculates the second downmix signal (S). Specifically, the second downmix signal calculation unit 142 reads and acquires the initial value of the second downmix coefficient (S × M) from the downmix coefficient storage unit 17, and the first downmix signal is obtained with this initial value. The second downmix signal (S) is calculated by downmixing (M). The second downmix signal calculation unit 142 outputs the downmix signal (S) to the difference signal calculation unit 15.

Next, in step S15, the difference signal calculation unit 15 calculates the difference between the second downmix signal (S) and the reference signal. A method such as energy difference between two signals, mean square error, and 1-normalized cross-correlation coefficient may be used to calculate the difference.
Next, in step S16, the difference signal calculation unit 15 determines whether the difference is less than or equal to a predetermined threshold value. Alternatively, the difference signal calculation unit 15 may determine whether the difference has become the minimum. When the difference is less than or equal to the threshold value (when the difference is the minimum), the first downmix signal calculation unit 141 downmixes the multi-channel audio signal with the first downmix coefficient (M × N) in step S13. The generated first downmix signal (M) is output to the reproducing device.

When the difference is larger than the threshold value (when the difference is not the minimum), the difference signal calculation unit 15 outputs the calculated difference to the downmix coefficient updating unit 16.
Next, in step S17, the downmix coefficient updating unit 16 updates the downmix coefficient. The downmix coefficient updating unit 16 sets the first downmix coefficient (M × N) and the second downmix coefficient (S × M) that reduce the difference while satisfying the constraint conditions described with reference to FIGS. 3 to 6. calculate.
Alternatively, the downmix coefficient updating unit 16 calculates only the first downmix coefficient (M × N) when the second downmix coefficient (S × M) is fixed. A genetic algorithm, a steepest descent method, a stochastic gradient descent method, or the like may be used to calculate the first downmix coefficient (M × N) and the like. After calculating the first downmix coefficient (M × N) and the like, the downmix coefficient updating unit 16 records the calculated new first downmix coefficient (M × N) and the like in the downmix coefficient storage unit 17. Then, the processing from step S13 is repeated until the difference becomes equal to or less than the threshold value.
In the processing of step S13 and step S14 after the second time, the initial value of the first downmix coefficient (M × N) stored in the downmix coefficient storage unit 17 and the second downmix coefficient (S × M) are stored. Instead of using the initial value, the downmix coefficient updating unit 16 calculates the first downmix coefficient (M × N) and the second downmix coefficient (S × M) stored in the downmix coefficient storage unit 17 in step S17. ) Is used.

The basic audio format in terrestrial digital broadcasting is stereo 2ch, while the BS digital broadcasting is stereo 2ch or 5.1ch surround broadcasting. In 4K / 8K broadcasting, so-called simulcasting of audio signals (reference signals) for stereo 2ch at the same time as 22.2ch multi-channel sound is being considered.
Therefore, in the present embodiment, the first downmix signal (M) corresponding to the speaker arrangement in the reproduction environment and the second downmix signal having the same number of channels as the reference signal corresponding to the speaker arrangement in the reproduction environment are generated from the multi-channel audio signal. (S) and are created using the initial value of the downmix coefficient. Then, the downmix coefficient is optimized so that the difference between the audio signal for stereo 2ch and the downmix signal (S) is minimized.
At this time, regarding the conversion of the original multi-channel audio signal and the desired first downmix signal (M), by adding a constraint condition according to the position of the speaker, the first impression of the original multi-channel audio is retained as much as possible. The first downmix coefficient (M × N) that realizes the downmix signal (M) can be calculated. Further, the first downmix coefficient that realizes the first downmix signal (M) more in line with the program producer's intention by referring to the broadcasted audio signal from stereo 2ch in which the intention of the program producer is reflected. (M × N) can be calculated.

<Second embodiment>
Hereinafter, a channel number conversion device according to a second exemplary embodiment of the present invention will be described with reference to FIGS.
FIG. 8 is a block diagram showing an example of a channel number conversion device according to the second embodiment of the present invention.
As shown in FIG. 8, the channel number conversion device 10a includes a multi-channel acoustic signal input unit 11, a speaker position information input unit 13, a downmix signal calculation unit 14, a difference signal calculation unit 15, and a downmix coefficient update unit. 16, a downmix coefficient storage unit 17, and a reference signal calculation unit 18. That is, the channel number conversion device 10a according to the second embodiment includes the reference signal calculation unit 18 instead of the reference signal input unit 12 of the first embodiment. Other configurations are similar to those of the first embodiment.

The reference signal calculation unit 18 downmixes the multi-channel audio signal using a predetermined downmix coefficient to calculate, for example, a 2ch stereo reference signal.
The first embodiment is based on the premise that a stereo 2ch audio signal (reference signal) is broadcast at the same time as the multi-channel audio signal. However, the reference signal corresponding to the multi-channel audio signal is not always obtained. For example, a downmix coefficient for a reference signal may be transmitted as metadata added to a multi-channel audio signal. Therefore, in the second embodiment, the reference signal calculation unit 18 calculates the reference signal from the input multi-channel acoustic signal.

Next, the flow of the channel number conversion processing of this embodiment will be described with reference to FIG.
FIG. 9 is a flowchart showing an example of the channel number conversion processing in the second embodiment according to the present invention.
As a premise, it is assumed that a downmix coefficient (referred to as a reference downmix coefficient) for downmixing a multi-channel audio signal into a stereo 2ch audio signal (reference signal) is set in the reference signal calculation unit 18 in advance. Other prerequisites are the same as those in the first embodiment. Moreover, a process similar to that of FIG. 7 will be briefly described.
First, in step S12, the multi-channel acoustic signal input unit 11 inputs a multi-channel acoustic signal. The multi-channel acoustic signal input unit 11 outputs the multi-channel acoustic signal to the reference signal calculation unit 18 and the downmix signal calculation unit 14.
In step S121, the reference signal calculator 18 downmixes the multi-channel audio signal with the reference downmix coefficient to calculate the reference signal. The reference signal calculation unit 18 outputs the calculated reference signal to the difference signal calculation unit 15. The subsequent processing is the same as in the first embodiment.
That is, next, in step S13, the first downmix signal calculation unit 141 calculates the first downmix signal (M), and in step S14, the second downmix signal calculation unit 142 causes the second downmix signal (S). ) Is calculated.

  Next, in step S15, the difference signal calculation unit 15 calculates the difference between the second downmix signal (S) calculated by the second downmix signal calculation unit 142 and the reference signal calculated by the reference signal calculation unit 18. Next, in step S16, the difference signal calculation unit 15 determines whether the difference is less than or equal to a predetermined threshold value, and if the difference is less than or equal to the threshold value, the first downmix signal calculation unit 141 causes the first downmix signal (M) to be detected. To the playback device.

  If the difference is larger than the threshold, the downmix coefficient updating unit 16 updates the downmix coefficient in step S17, and repeats the processing from step S13 until the difference becomes smaller than the threshold.

  According to the present embodiment, even if the stereo 2ch is not broadcast at the same time, the optimized downmix coefficient from the multi-channel audio system to the existing number of channels such as the stereo 2ch (for example, the optimized downmix coefficient is multi-valued). The signal down-mixed by the channel audio signal (which may be added as metadata) is used instead of the reference signal. With this, by the same procedure as in the first embodiment, the number of speakers and the speaker arrangement of the reproducing apparatus are adjusted while maintaining the number of speakers when producing the multi-channel audio signal and the impression of the sound when reproduced with the speaker arrangement as much as possible. The corresponding first downmix coefficient (M × N) can be calculated for each program.

  The above-described channel number conversion devices 10 and 10a have a computer system inside. The process of the operation of the channel number conversion device 10 and the like is stored in a computer-readable recording medium in the form of a program, and the above process is performed by the computer system reading and executing this program. The computer system mentioned here includes a CPU, various memories, an OS, and hardware such as peripheral devices.

Further, the “computer system” also includes a homepage providing environment (or display environment) if a WWW system is used.
Further, the “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in a computer system. Further, the "computer-readable recording medium" means to hold a program dynamically for a short time like a communication line when transmitting the program through a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system that serves as a server or a client in that case holds a program for a certain period of time. Further, the program may be for realizing a part of the functions described above, or may be a program for realizing the functions described above in combination with a program already recorded in the computer system.

In addition, it is possible to appropriately replace the components in the above-described embodiments with known components without departing from the spirit of the present invention. Further, the technical scope of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention.
The initial values of the first downmix coefficient (M × N) and the first downmix coefficient (M × N) are examples of the first downmix coefficient. The initial values of the second downmix coefficient (S × M) and the second downmix coefficient (S × M) are examples of the second downmix coefficient. The first downmix signal (M) is an example of the first downmix signal, and the second downmix signal (S) is an example of the second downmix signal.

10, 10a ... Channel number converter 11 ... Multi-channel acoustic signal input unit 12 ... Reference signal input unit 13 ... Speaker position information input unit 14 ... Downmix signal calculation unit 141 ... First downmix signal calculation unit 142 ... Second downmix signal calculation unit 15 ... Difference signal calculation unit 16 ... Downmix coefficient update unit 17 ... Downmix coefficient storage unit 18 ... Reference signal Calculator

Claims (11)

A desired first downmix signal is calculated from the multi-channel acoustic signal using the first downmix coefficient, and a second downmix having the same number of channels as the reference signal from the first downmix signal is calculated using the second downmix coefficient. A downmix signal calculation unit that calculates a signal,
A difference signal calculation unit that calculates a difference between the second downmix signal and a reference signal,
The difference calculated by the difference signal calculation unit is a minimum or a predetermined threshold value or less, a downmix coefficient updating unit for updating the first downmix coefficient and the second downmix coefficient,
An apparatus for converting the number of channels, comprising: A reference signal calculation unit that down-mixes the multi-channel acoustic signal using a predetermined down-mix coefficient, and calculates the reference signal,
The channel number conversion device according to claim 1, further comprising: The channel number conversion device according to claim 1 or 2,
The downmix coefficient updating unit fixes the second downmix coefficient and updates only the first downmix coefficient,
A channel number conversion device characterized by the above. The channel number conversion device according to any one of claims 1 to 3,
A downmix coefficient storage unit that stores at least one of the initial value of the first downmix coefficient and the initial value of the second downmix coefficient,
An apparatus for converting the number of channels, further comprising: The channel number conversion device according to claim 4,
The downmix coefficient storage unit is determined based on a positional relationship between a reproduction position of an audio signal of each channel included in the multi-channel audio signal and a reproduction position of an audio signal of each channel included in the first downmix signal. Storing the first downmix coefficient with an initial value set to
A channel number conversion device characterized by the above. The channel number conversion device according to claim 4 or 5,
The downmix coefficient storage unit is based on the positional relationship between the reproduction position of the acoustic signal of each channel included in the first downmix signal and the reproduction position of the acoustic signal of each channel included in the second downmix signal. Storing said second downmix coefficient having a defined initial value,
A channel number conversion device characterized by the above. The channel number conversion device according to any one of claims 1 to 6,
The downmix coefficient updating unit is determined by the similarity of the position between the reproduction position of the audio signal of each channel included in the multi-channel audio signal and the reproduction position of the audio signal of each channel included in the first downmix signal. Updating the first downmix coefficient based on the restrained conditions,
A channel number conversion device characterized by the above. The channel number conversion device according to claim 7,
The downmix coefficient updating unit updates the first downmix coefficient with a constraint that the value of the downmix coefficient between the channels having the highest position similarity is maximum.
A channel number conversion device characterized by the above. The channel number converter according to any one of claims 4 to 8, which does not cite claim 1, claim 2, or claim 3 ,
The downmix coefficient updating unit determines the position similarity between the reproduction position of the acoustic signal of each channel included in the first downmix signal and the reproduction position of the acoustic signal of each channel included in the second downmix signal. Updating the second downmix coefficient based on a defined constraint condition,
A channel number conversion device characterized by the above. The channel number conversion device according to claim 9,
The downmix coefficient updating unit updates the second downmix coefficient with a constraint that the value of the downmix coefficient between the channels having the highest position similarity is maximum.
A channel number conversion device characterized by the above.   A program for causing a computer to function as the channel number conversion device according to any one of claims 1 to 10.

Images