JP2020101837A - Voice signal processor - Google Patents

Abstract

To enable a viewer to control a dialog using a receiver, etc., within the framework of creation technique-encoding technique of a channel based scheme.SOLUTION: A voice signal processor 3 comprises: a dialog control allowability determination unit 31 for determining from a flag of a program or not that corresponds to a dialog control function whether or not dialog control is allowed; a dialog-exclusive channel signal specification unit 32 for specifying a dialog-exclusive channel signal when dialog control is allowed; a voice signal separation unit 33 for separating the voice signal into a dialog-exclusive channel signal and other signals; and a control unit 34 for acquiring dialog control information (upper- and lower-limit values of gain control amount of the dialog-exclusive channel signal) and performing different signal processing on the dialog-exclusive channel signal and other signals. The control unit 34 reduces the gain of other signals when adjustment information for increasing a dialog sound volume is acquired, and reduces only the gain of the dialog-exclusive channel signal when adjustment information for reducing a dialog sound volume is acquired.SELECTED DRAWING: Figure 4

Description

Cross-reference to related application

This application claims the priority of Japanese Patent Application No. 2014-182695 (filed on Sep. 8, 2014), and the entire disclosure of the application is incorporated herein by reference.

The present invention relates to an audio signal processing device.

Most of viewers' opinions about broadcast sound are related to the easiness of hearing dialogues (narration, speech, dialogue, etc.). Conventional Japanese broadcast audio employs a channel-based method in which the audio engineer uniquely adjusts the volume balance between the dialog and the background on the broadcast station side and then sends the audio (for example, Non-Patent Document 1). The channel-based method is, for example, MPEG-4 AAC (for example, Non-Patent Document 2). Many viewers are interested in the ease of listening to dialogs.

In order to make the dialog easier to hear, the next-generation broadcast audio systems in Europe and the United States are under study in the direction of adopting an object-based system (for example, Patent Document 1). The object-based method is a method of transmitting by an encoding method such as MPEG-H 3D Audio (for example, Non-Patent Document 3) or AC-4 of Dolby, and an important audio object such as a dialog is controlled by a receiver. It will be possible.

ITU-R, "Advanced sound system for program production", [Online], February 2014, [September 7, 2014 search], Internet <http://www.itu.int/dms_pubrec/itu- r/rec/bs/R-REC-BS.2051-0-201402-I!!PDF-E.pdf> ARIB STD-B 3.0 version, General incorporated association, "Video coding, audio coding and multiplexing method in digital broadcasting", [Online], July 31, 2014, [September 2014 7 days search], Internet <http://www.arib.or.jp/english/html/overview/doc/2-STD-B32v3_0.pdf> ISO/IEC DIS 23008-3, "Information technology-High efficiency coding and media delivery in heterogeneous environment-Part 3:3D audio"

In the above-mentioned channel-based system adopted by Japan, the viewer operating the receiver cannot adjust the volume of the dialogue. However, in consideration of viewer preferences, age, and diversity of reproduction environment, it is considered that there are situations in which the diversity cannot be fully covered by the volume balance adjusted at the broadcasting station. This is considered to be one of the factors that make the dialog difficult to hear.

The audio encoding method of 8K SHV 22.2ch broadcasting in Japan is the above-mentioned MPEG-4 AAC, which is a channel-based method in which audio signals and speakers correspond one-to-one. In addition, the audio encoding method for terrestrial digital broadcasting in Japan is MPEG-2 AAC, which is a channel-based method. Therefore, at present, it is impossible to control a voice object such as a dialog.

In view of such circumstances, an object of the present invention is to provide a mechanism that allows a viewer to control a dialog using a receiver or the like within the framework of a channel-based production method and a channel-based encoding method. An object is to provide an encoding device, a decoding device, and an audio signal processing device that are realized.

An audio signal processing device according to the present invention is an audio signal processing device for performing an audio signal processing on an audio signal corresponding to each channel based on a channel-based system, and whether or not the program corresponds to a dialog control function. A dialog control availability determination unit that determines whether the dialog control is possible based on the flag, and a dialog dedicated channel signal identification unit that identifies the dialog dedicated channel signal when the dialog control availability determination unit determines that the dialog control is possible. An audio signal separation unit that separates the audio signal into the dialog dedicated channel signal and a channel signal other than the dialog dedicated channel signal based on the specification of the dialog dedicated channel signal specifying unit; A control unit that obtains the upper limit value and the lower limit value of the gain control amount as the control information of the dialog and performs different signal processing on the dialog dedicated channel signal and the channel signals other than the dialog dedicated channel signal, respectively. When the control unit obtains the adjustment information for increasing the dialog volume, the control unit reduces the gain of a channel signal other than the dialog dedicated channel signal, and obtains the adjustment information for reducing the dialog volume, the dialog dedicated channel. Only reduce the gain of the signal.

Further, the control unit may convert the number of channels by a conversion unit including downmix after controlling the dialog.

Further, the control unit may perform signal processing including frequency correction processing on both or either of the dialog dedicated channel signal and the channel signal other than the dialog dedicated channel signal.

Further, when the audio signal is a compressed audio signal separated from a bitstream, the control unit may directly perform the signal processing without decoding the compressed audio signal.

According to the audio signal processing device of the present invention, within the framework of the channel-based production method and the channel-based encoding method, the viewer uses the receiver or the reproduction device connected to the receiver to display a dialog. A mechanism that can be controlled can be realized.

It is a figure which shows the three-dimensional (3D) acoustic system which concerns on one Embodiment of this invention. FIG. 3 is a functional block diagram of an encoding device according to an embodiment of the present invention. It is a functional block diagram of the decoding device which concerns on one Embodiment of this invention. It is a functional block diagram of an audio signal processing device and a control information input device concerning one embodiment of the present invention. It is a figure which shows the operation|movement flow of the audio|voice signal processing system which concerns on one Embodiment of this invention.

Hereinafter, a mechanism for enabling a dialog to be controlled in a receiver or a reproducing device (external reproducing device such as a speaker or a recording device) connected to the receiver (hereinafter, referred to as a receiver) will be described. In the present embodiment, a 22.2 ch sound system for 8K SHV will be described as an example of a sound system having a plurality of audio channels and a channel dedicated to dialogue.

The audio signal processing system of this embodiment includes an encoding device 1, a decoding device 2, an audio signal processing device 3 and a control information input device 4, which communicate with each other by wire or wirelessly via a network. It should be noted that the following description describes each function of the voice processing system according to the present invention, but is not intended to exclude other functions included in them.

FIG. 1 is a diagram showing a three-dimensional (three-dimensional) sound system at the time of production by a production method corresponding to a dialog control function in a 22.2 ch sound system. As shown in FIG. 1, program production of an ultra-high definition and highly realistic audiovisual system is performed under standard production conditions in which a large screen image display 1a (for example, 7680×4320 pixels) and a speaker are arranged. Under this standard production condition, the large-screen image display 1a is placed in front of the listening position, and the low-frequency sound effect speakers LFE1 and LFE2 are excluded, and the upper layer of 9 channels is composed of the middle layer of 10 channels and 3 channels. A sound signal is produced by a total of 22 channels of speakers in the lower layer. The position where the 22-channel speaker is arranged is defined in the standard SMPTE ST2036-2-2008.

In order to realize the dialog control function by the channel-based method, a dedicated channel for dialog that does not overlap background sound is required. In the present embodiment, as an example, the FC shown in FIG. 1 will be described as a dialog dedicated channel. There may be a plurality of dialog dedicated channels. When there are a plurality of dialog-dedicated channels, the dialog-dedicated channels may reproduce the same audio signal or different audio signals.

FIG. 2 is a functional block diagram of the encoding device 1. The encoding device 1 includes a compression encoding unit 11 and a multiplexing unit 12. Various operations performed by the compression encoding unit 11 and the multiplexing unit 12 are processed by an arbitrary processing device such as a processor or a microcomputer not shown.

The compression encoding unit 11 acquires the input audio signal and compression-encodes it by a digital method. The compression encoding unit 11 converts the compression encoded audio signal into a compressed audio signal of 22.2 ch and outputs it to the multiplexing unit 12.

The multiplexing unit 12 acquires the compressed audio signal acquired from the compression encoding unit 11 and the input dialog control metadata and the metadata indicating the audio system (for example, channel configuration in MPEG Audio).

Next, the multiplexing unit 12 encodes the dialog control metadata and the metadata indicating the audio method, and multiplexes them with the acquired compressed audio signal. The dialog control metadata is, for example, data such as a flag indicating whether the program is compatible with the dialog control function, an upper limit value and a lower limit value of gain control in a receiver or the like. When transmitting by MPEG-4 AAC, the multiplexing unit 12 stores the metadata in, for example, DSE (Data Stream Element) of the user extension area. The multiplexing unit 12 outputs the multiplexed data as a bitstream.

FIG. 3 is a functional block diagram of the decoding device 2. The decoding device 2 includes a separation unit 21, a metadata separation unit 22 and a decoding unit 23. Various operations performed by the separation unit 21, the metadata separation unit 22, and the decoding unit 23 are processed by an arbitrary processing device such as a processor or a microcomputer (not shown).

The separation unit 21 separates the bitstream acquired from the encoding device 1. Specifically, the separation unit 21 separates the bit stream (input signal) into metadata and a compressed audio signal, and outputs the metadata and the compressed audio signal to the metadata separation unit 22 and the decoding unit 23, respectively.

The metadata separation unit 22 separates the acquired metadata into dialog control metadata and audio method metadata.

The decoding unit 23 decodes the acquired compressed audio signal into an audio signal. The decoding unit 23 does not have to decode the acquired compressed audio signal. In this case, the control unit 34 of the audio signal processing device 3 performs the audio signal processing described below without decoding the compressed audio signal and then decodes the compressed audio signal to output it as an audio signal. The control unit 34 may perform the audio signal processing described below as it is without decoding the compressed audio signal, and then output the compressed audio signal as a compressed audio signal without decoding.

FIG. 4 is a functional block diagram of the audio signal processing device 3 and the control information input device 4. The audio signal processing device 3 is arranged, for example, in the latter stage of the decoding device 2, and acquires the dialog control metadata, the audio method metadata, and the audio signal from the decoding device 2. The audio signal processing device 3 includes a dialog control availability determination unit 31, a dialog dedicated channel signal identification unit 32, an audio signal separation unit 33, a control unit 34, a control information acquisition unit 35, and a storage unit 36. Various operations (audio signal processing) performed by the dialog dedicated channel signal specifying unit 32, the audio signal separating unit 33, the control unit 34, and the control information acquiring unit 35 are processed by an arbitrary processing device such as a processor or a microcomputer not shown.

Based on the dialog control metadata (flag indicating whether or not the program corresponds to the dialog control function) acquired from the decoding device 2, the dialog control availability determination unit 31 determines that the audio signal acquired from the decoding device 2 is a dialog. Whether or not the program is compatible with the control function (whether or not the dialog control is possible) is determined. When the dialog control availability determination unit 31 determines that the audio signal is not a program compatible with the dialog control function, the audio signal processing device 3 outputs the audio signal to the receiver or the like without performing the audio signal processing.

The dialog dedicated channel signal identifying unit 32 identifies the signal of the dialog dedicated channel based on the audio method metadata acquired from the decoding device 2. Note that the dialog dedicated channel signal specifying unit 32 may specify the dialog dedicated channel signal using information acquired from an external device other than the decoding device 2.

The audio signal separation unit 33 separates the audio signal into a dialog dedicated channel signal and other background sound channel signals based on the identification by the dialogue dedicated channel signal identification unit 32.

The control unit 34 acquires the dialog dedicated channel signal and the background sound channel signal from the audio signal separation unit 33.

Next, the control unit 34 sets the upper limit value and the lower limit value (for example, the upper limit value is +18 dB and the lower limit value is −∞) of the gain control in the receiver or the like based on the dialog control metadata acquired from the decoding device 2. get.

Further, since the audio system is 22.2 ch, the control unit 34 refers to the storage unit 36 to specify the dialog dedicated channel (FC in FIG. 1 in this embodiment). The control unit 34 may specify the dialog dedicated channel from other information (for example, program information).

Further, the control unit 34 controls the control information input from the control information input device 4 outside the audio signal processing device 3 to the control information input device 4 according to the received viewing environment by the remote control operation by the viewer or the like (for example, the volume). The adjustment information) is acquired via the control information acquisition unit 35. The control unit 34 controls the dialog dedicated channel signal and the background sound channel signal by using the dialog control metadata and the control information provided by the viewer.

In the control, the control unit 34 may perform the speech speed conversion process on the dialog. Further, in the control, when the control unit 34 acquires the adjustment information of the dialog volume higher than the upper limit value of the gain control amount or lower than the lower limit value, the control unit 34 limits the adjustment by the upper limit value or the lower limit value of the gain control amount. May be.

In the control, the control unit 34 may perform different signal processing on the dialog dedicated channel signal and the background sound channel signal, respectively. For example, the control unit 34, when acquiring the adjustment information for increasing the dialog volume, reduces the gain of channel signals other than the dialog dedicated channel signal, and acquires the adjustment information for decreasing the dialog volume, the dialog dedicated channel signal. May be reduced only. Further, the control unit 34 may increase or decrease the volume of the dialog dedicated channel signal and the background sound channel signal at the same time after adjusting the dialog volume. Further, the control unit 34 may perform signal processing including frequency correction processing on both or either of the dialog-dedicated channel signal and an arbitrary number of other channel signals.

Further, the control unit 34 converts the number of channels by a conversion unit including downmix as necessary, and then outputs a 22.2 ch audio signal, which is a combination of the dialog dedicated channel signal and the background sound channel signal, to the receiver. .. The receiver outputs the audio signal from the reproduction device connected to the receiver, and as a result, the viewer can view the desired audio as shown in the control information. When the above-described audio signal processing is performed on the compressed audio signal, the control unit 34 multiplexes the compressed audio signal with the dialog control metadata and/or the audio method metadata, and receives the multiplexed audio signal as a bit stream. A compressed audio signal may be output without multiplexing the metadata.

FIG. 5 is a diagram showing an operation flow according to the embodiment of the present invention.

The encoding device 1 acquires the input audio signal (step S1) and performs compression encoding (step S2). Next, the encoding device 1 multiplexes the compression-encoded compressed audio signal, the dialog control metadata, and the metadata indicating the audio system (step S3). The encoding device 1 outputs the multiplexed data as a bit stream to the decoding device 2 (step S4).

The decoding device 2 separates the bit stream acquired from the encoding device 1 into metadata and a compressed audio signal (step S5). The decryption device 2 also separates the metadata into dialog control metadata and audio method metadata (step S6). Next, the decoding device 2 decodes the acquired compressed audio signal into an audio signal (step S7), and outputs the dialog control metadata, audio system metadata, and audio signal to the audio signal processing device 3 (step S8).

The audio signal processing device 3 determines whether or not the audio signal acquired from the decoding device 2 is a program compatible with the dialog control function (step S9). If the audio signal processing device 3 determines that the audio signal is not a program compatible with the dialog control function (No in step S9), steps S10 to S14 are not performed.

On the other hand, when the audio signal processing device 3 determines that the audio signal is a program compatible with the dialog control function (Yes in step S9), the upper limit value and the lower limit value of the gain control in the receiver or the like are determined from the dialog control metadata. Value information is acquired (step S10). Next, the audio signal processing device 3 identifies the signal of the channel dedicated to the dialog (step S11). The audio signal processing device 3 separates the audio signal into a dialog dedicated channel signal and other background sound channel signals based on the identification (step S12).

The audio signal processing device 3 acquires control information (for example, volume adjustment information) from the control information input device 4 outside the audio signal processing device 3 via the control information acquisition unit 35 (step S13). The audio signal processing device 3 adjusts the audio signal based on the control information (step S14).

Next, the audio signal processing device 3 outputs the audio signal to the receiver or the like (step S15).

Therefore, according to the encoding device 1, the decoding device 2, the audio signal processing device 3, and the control information input device 4 according to the present embodiment, within the framework of the channel-based production method and the channel-based encoding method. It is possible to realize a mechanism that allows the viewer to control the dialog using a receiver or the like.

Although the present invention has been described based on the drawings and the embodiments, those skilled in the art can easily make various variations and modifications based on the present disclosure. Therefore, it should be noted that these variations and modifications are included in the scope of the present invention. For example, the functions and the like included in each functional unit, each unit, and each step can be rearranged so as not to logically contradict, and a plurality of functional units and steps can be combined or divided. Is. In addition, the above-described embodiments of the present invention are not limited to being carried out faithfully to the respective described embodiments, and may be carried out by appropriately combining the respective features or omitting a part thereof. it can.

Further, it goes without saying that the present invention can be applied to audio systems other than 22.2 ch. The present invention is not limited to MPEG-4 AAC, but can be applied to any audio encoding method having a metadata area capable of storing dialog control information. Further, it goes without saying that the present invention is not necessarily applied only to the dialog, but can be applied to control for the purpose of individually controlling by providing a dedicated channel for some audio signal.

DESCRIPTION OF SYMBOLS 1 Coding device 11 Compression coding part 12 Multiplexing part 2 Decoding device 21 Separation part 22 Metadata separation part 23 Decoding part 3 Audio signal processing device 31 Dialog control availability judgment part 32 Dialog dedicated channel signal specifying part 33 Audio signal Separation unit 34 Control unit 35 Control information acquisition unit 36 Storage unit 4 Control information input device

Claims (4)

An audio signal processing device for performing audio signal processing on the basis of a channel-based system, for an audio signal corresponding to each channel,
A dialog control availability determination unit that determines availability of dialog control based on a flag indicating whether or not the program corresponds to the dialog control function,
A dialog dedicated channel signal specifying unit that specifies a dialog dedicated channel signal when the dialog control availability determination unit determines that the dialog control is possible;
An audio signal separation unit that separates the audio signal into the dialog dedicated channel signal and a channel signal other than the dialog dedicated channel signal based on the specification of the dialog dedicated channel signal specifying unit,
The upper limit value and the lower limit value of the gain control amount of the dialog dedicated channel signal are acquired as dialog control information, and different signal processing is performed on the dialog dedicated channel signal and the channel signals other than the dialog dedicated channel signal. A control unit,
Equipped with
When the control unit obtains the adjustment information for increasing the dialog volume, the gain of channel signals other than the dialog dedicated channel signal is reduced, and when the adjustment information for reducing the dialog volume is obtained, the control unit detects the dialog dedicated channel signal. An audio signal processing device that reduces only gain. The audio signal processing device according to claim 1, wherein the control unit converts the number of channels by a conversion unit including downmix after controlling the dialog. The audio signal according to claim 1, wherein the control unit performs signal processing including frequency correction processing on both or either of the dialog dedicated channel signal and the channel signal other than the dialog dedicated channel signal. Processing equipment. The control unit, if the audio signal is a compressed audio signal separated from a bitstream, directly performs the signal processing without decoding the compressed audio signal. The audio signal processing device described.

Images