JP2017009743A - Voice data processing device, and electronic apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reproduce a voice that is closer to an actual voice.SOLUTION: A voice data processing device 1 has a high-frequency interpolation unit 30 that converts the time domain signal of an original sound decoded on the basis of voice data into a frequency domain signal, extracts information on the amplitude of the original voice to a frequency fa/3 in the frequency domain signal when interpolating the frequency fa component of a high frequency domain that was cut in the original voice, generates a first interpolation-use frequency domain signal in which the amplitude information multiplied by 1/3 is made the amplitude information Ae of a frequency fa, extracts information on the amplitude of the original voice to a frequency fa/2 in the frequency domain signal, generates a second interpolation-use frequency domain signal in which the amplitude information multiplied by 1/2 is made the amplitude information Ae of a frequency fa, and synthesizes a signal derived from the first interpolation-use frequency domain signal by conversion to a time domain signal and a signal derived from the second interpolation-use frequency domain signal by conversion to a time domain signal, the synthesized time domain signal and the time domain signal of the original voice being synthesized.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an audio data processing device that interpolates a high-frequency component of an original sound that is decoded based on audio data, and an electronic device that includes the audio data processing device.

  In recent years, there has been a demand for higher sound quality as well as higher image quality and higher resolution of televisions, and televisions compatible with so-called high resolution sound sources (hereinafter referred to as high resolution sound sources) have come out. The high-resolution sound source includes frequency components exceeding 20 kHz that have been considered to be human audible limits and have been cut during compression. This is because in recent years it has been found that humans are sensitive to frequency components exceeding the audible limit.

  As described above, AV (Audio Visual) devices such as TVs that support high-resolution sound sources and reproduce high-frequency components are emerging, and broadcast audio data and CD audio data input to the AV devices are high-resolution. It is not data for a sound source, and in many cases, the high frequency region is cut by data compression. Even if the AV device is compatible with a high-resolution sound source, if the input data is not data compatible with the high-resolution sound source, the reproduced sound does not include a high-frequency component. Even in such a case, it is necessary to interpolate the high frequency component in order to include the high frequency component.

  In the high-frequency component interpolation method disclosed in Patent Document 1, first, audio data irreversibly compressed according to the MP3 (MPEG-1 Audio Layer 3) standard is reproduced in a sound range close to an actual sound. Extracts the fundamental tone, generates frequency components that become harmonics from this fundamental tone, leaves only the frequency components of the region that was cut during compression, among the generated harmonic components, cuts the others, and amplifies the frequency components . Then, the temporal change characteristic of the timbre is extracted from the MP3 music data, and the amplified frequency component and the frequency component of the original MP3 audio data are synthesized and reproduced based on the temporal change characteristic.

JP 2001-324996 A

  However, since the actual sound includes components other than the fundamental tone and its overtones, the method disclosed in Patent Document 1 is not sufficient to reproduce the actual sound.

  The present invention has been made in view of the above circumstances, and provides an audio data processing device capable of reproducing a sound closer to an actual sound and an electronic device including the audio data processing device. For that purpose.

  In order to solve the above-described problem, the first technical means of the present invention is a high-frequency interpolation that interpolates a frequency component of a high-frequency region cut in the original sound with respect to a time-domain signal of the original sound decoded based on audio data. A high-frequency interpolating unit for converting the time domain signal of the decoded original sound into a frequency domain signal, and interpolation of the frequency fa component of the cut high-frequency domain At this time, amplitude information for the frequency fa / (2n + 1) (n is a positive integer) in the frequency domain signal of the original sound is extracted, and the extracted amplitude information multiplied by 1 / (2n + 1) is the frequency fa. A first interpolation signal generation unit that generates a first interpolation frequency domain signal as amplitude information Aa, and amplitude information for the frequency fa / 2n in the frequency domain signal of the original sound are extracted. A second interpolation signal generating unit for generating a second interpolation frequency domain signal, which is obtained by multiplying the extracted amplitude information by ½n into the amplitude information Ae of the frequency fa, and the first interpolation frequency. It has a combining unit that combines a signal obtained by converting a domain signal into a time domain signal and a signal obtained by converting the second interpolation frequency domain signal into a time domain signal.

  According to a second technical means of the present invention, in the first technical means, the high frequency interpolation unit applies a filter that passes the frequency fa / 2n to the time domain signal of the original sound, and the original sound that has been subjected to the filter is applied. A determination unit for determining whether a time interval Tin from the maximum value to the minimum value of the time domain signal is 4n / fa, wherein the second interpolation signal generation unit is a filter that passes the frequency fa / 2n When the time interval Tin of the original sound multiplied by 4 is 4n / fa, the amplitude information Ae of the frequency fa is set to 0.

  According to a third technical means of the present invention, in the first or second technical means, the combining unit converts the first interpolation frequency domain signal into a time domain signal, and the second interpolation means. A signal obtained by converting a frequency domain signal into a time domain signal is synthesized so that the mixing ratio can be adjusted.

  According to a fourth technical means of the present invention, there is provided an electronic apparatus comprising the audio data processing apparatus according to any one of the first to third technical means.

  The fifth technical means of the present invention is characterized in that the audio data processing apparatus of the fourth technical means is a television receiver.

  According to the present invention, a sound closer to an actual sound can be reproduced.

It is a block diagram for demonstrating the audio | voice data processing apparatus which concerns on 1st Example of this invention. It is a figure which shows the time-domain signal of an original sound. It is a figure which shows the time domain signal by which the high frequency component of the original sound was interpolated. It is a figure for demonstrating the detection method of the cutoff frequency of an original sound. It is a figure which shows an example of the frequency domain signal for every unit time of an original sound. It is a figure for demonstrating the production | generation process of the frequency domain signal for 1st interpolation used for the interpolation of a high frequency component. It is a figure for demonstrating the production | generation process of the 2nd frequency domain signal for interpolation used for the interpolation of a high frequency component. It is a figure for demonstrating the determination process in the determination part of the audio | voice data processing apparatus of FIG.

  Hereinafter, preferred embodiments of an audio data processing device and an electronic apparatus according to the present invention will be described with reference to the drawings. In the following inventions, the same reference numerals in different drawings are the same, and the description thereof may be omitted.

(First embodiment)
FIG. 1 is a block diagram for explaining an audio data processing apparatus according to a first embodiment of the present invention. FIG. 2 is a diagram showing a time domain signal of the original sound decoded by the decoding unit of the audio data processing apparatus of FIG. FIG. 3 is a diagram showing a time domain signal obtained by interpolating the high frequency component of the original sound by the audio data processing apparatus of FIG. FIG. 4 is a diagram for explaining a cut-off frequency detection process in the band detection unit of the audio data processing apparatus of FIG. FIG. 5 is a diagram illustrating an example of a frequency domain signal for each unit time of the original sound. FIG. 6 is a diagram for explaining a generation process of the first interpolation frequency domain signal used for the interpolation of the high-frequency component. FIG. 7 is a diagram for explaining the generation process of the second interpolation frequency domain signal used for the interpolation of the high-frequency component. FIG. 8 is a diagram for explaining the determination process in the determination unit of the audio data processing apparatus of FIG.

  The audio data processing apparatus 1 shown in the figure includes a decoding unit 10, an upsampling unit 20, a high frequency interpolation unit 30, and a synthesis unit 50.

The decoding unit 10 decodes the irreversibly compressed audio data input to the audio data processing device 1 to generate a time domain signal of the original sound as shown in FIG. Note that irreversible compression methods include MP3, AAC (Advanced Audio Coding), and the like.
The up-sampling unit 20 up-samples the time domain signal of the original sound decoded by the decoding unit 10 at a sampling rate twice that at the time of irreversible compression.

The high frequency interpolation unit 30 generates data for interpolating high frequency components not included in the time domain signal of the original sound, and is based on the time domain signal of the original sound upsampled by the upsampling unit 20. Data for interpolating high frequency components is generated. The data output from the high frequency interpolation unit 30 is time domain data. A specific processing example in the high frequency interpolation unit 30 will be described later.
The synthesizing unit 50 synthesizes and outputs the time domain signal of the original sound decoded by the decoding unit 10 and upsampled by the upsampling unit and the time domain data for high frequency interpolation output from the high frequency interpolation unit 30 It is.

  With such a configuration, the audio data processing apparatus 1 has the time domain signal audio as shown in FIG. 3, that is, the audio of the time domain signal including the high frequency components not included in the original time domain signal of FIG. Can be output.

  The high frequency interpolation unit 30 according to the characteristic part of the audio data processing device 1 includes a band detection unit 31, a frequency domain conversion unit 32, an odd multiple harmonic generation unit 33, a first gain multiplication unit 34, a first reconversion unit 35, and a determination. A unit 36, an even harmonic generation unit 37, a second gain multiplication unit 38, a second reconversion unit 39, and a synthesis unit 40.

  The band detection unit 31 detects a cut-off frequency described later in order to detect a frequency band included in the time domain signal of the original sound. For example, the band detection unit 31 creates a frequency histogram as shown in FIG. 4 using the Fourier transform or the like for the first several seconds of the original sound time domain signal input from the decoding unit 10, and the frequency is A frequency detected to be equal to or lower than the threshold is detected as a cut-off frequency fc. When there are a plurality of frequencies whose frequency is equal to or less than the threshold value, the maximum frequency is set as the cut-off frequency fc. Further, the determination method of the cutoff frequency fc is not limited to the above example. For example, information related to the cut-off frequency is given to the irreversibly compressed audio data so that the information is input to the band detecting unit 31, and the band detecting unit 31 determines the cut-off frequency based on the information. May be. The cut-off frequency fc detected by the band detector 31 is output to the odd harmonic generator 33 and the even harmonic generator 37. In the following description, it is assumed that the cut-off frequency fc is 24 kHz.

  The frequency domain transform unit 32 uses a general technique such as that used for FFT (Fast Fourier Transform) analyzers and equalizers for the time domain signal for each unit time of the original sound upsampled by the upsampling unit 20. 5 is converted into a frequency domain signal for each unit time of the original sound as shown in FIG. The converted frequency domain signal of the original sound for each unit time is output to the odd harmonic generation unit 33 and the even harmonic generation unit 37.

The odd harmonic generation unit 33 performs an interpolation region of the frequency region signal for each unit time of the original sound, that is, a region larger than the cut-off frequency fc and less than or equal to twice the cut-off frequency fc with the amplitude of the frequency equal to or lower than the cut-off frequency fc Interpolation based on (level) information.
The odd-numbered harmonic generation unit 33 is, for example, greater than 24 kHz and less than 48 kHz based on amplitude information of a frequency greater than 8 kHz and less than or equal to 16 kHz included in the frequency domain signal per unit time as shown in FIG. Interpolates frequency amplitude information. More specifically, the odd multiple harmonic generation unit 33 adds the amplitude information of the frequency fa / 3 as the amplitude information of the frequency fa of the interpolation region where 24 kHz <fa ≦ 48 kHz, as shown in FIG. Creates data for a simple frequency domain signal. For example, amplitude information of frequency 16 kHz is added as amplitude information of 48 kHz. The interval of the frequency fa is, for example, 0.3 kHz.

  The first gain multiplication unit 34 applies 1/3 as the first gain to the amplitude information of each frequency fa for the data of the frequency domain signal of FIG. 6B output from the odd multiple harmonic generation unit 33. A first interpolation frequency domain signal as shown in FIG. 6C is generated by multiplying and multiplying the amplitude information Aa of each frequency fa. The “first interpolation signal generation unit” of the present invention can be configured by an odd harmonic generation unit 33 and a first gain multiplication unit 34.

  The first reconversion unit 35 performs, for example, inverse Fourier transform on the first interpolation frequency domain signal output from the first gain multiplication unit 34, converts it into a time domain signal, and outputs the time domain signal to the synthesis unit 40. When phase information is necessary for the frequency fa of the interpolation region during the conversion, the phase information for the frequency fa / 3 is used.

  The determination unit 36 applies a filter that passes one frequency fa / 2 among a plurality of frequencies fa / 2 to the time domain signal decoded based on the irreversibly compressed audio data, and the original sound that has been subjected to the filter is filtered. It is determined whether or not the time domain signal satisfies a predetermined condition described later.

  Similarly to the odd harmonic generation unit 33, the even harmonic generation unit 37 generates amplitude information of the frequency in the interpolation region based on the amplitude information of the frequency equal to or lower than the cutoff frequency fc included in the frequency region signal of the original sound. Interpolate. However, unlike the odd harmonic generation unit 33, the even harmonic generation unit 37 adds the amplitude value or 0 for the frequency fa / 2 in the frequency domain signal as the amplitude value of the frequency fa of the interpolation domain.

  The even harmonic generation unit 37, for example, based on amplitude information of a frequency greater than 12 kHz and less than or equal to 24 kHz included in a frequency domain signal for each unit time of the original sound as shown in FIG. 7A, greater than 24 kHz and 48 kHz. Interpolate the amplitude information of the following frequencies. At that time, the even-numbered harmonic generation unit 37 adds the amplitude information of the frequency fa / 2 or 0 as shown in FIG. 7B as the amplitude information of the frequency fa in the interpolation region of 24 kHZ <fa ≦ 48 kHZ. The case where 0 is added is a case where the time domain signal of the original sound filtered through the frequency fa / 2 satisfies a predetermined condition as a result of determination by the determination unit 36. The interval of the frequency fa in the even-numbered harmonic generation unit 37 may be the same as that of the odd-numbered harmonic generation unit 33, for example, 0.3 kHz, or may be different.

  The second gain multiplication unit 38 halves the amplitude information of each frequency fa as a second gain for the frequency domain signal data in FIG. 7B output from the even-numbered harmonic generation unit 37. A second interpolation frequency domain signal as shown in FIG. 7C is generated by multiplying and multiplying the amplitude information Ae of each frequency fa. The “second interpolation signal generation unit” of the present invention can be configured by an even harmonic generation unit 37 and a second gain multiplication unit 38.

  The second re-conversion unit 39 performs, for example, inverse Fourier transform on the second interpolation frequency domain signal output from the second gain multiplication unit 38, converts it into a time-domain signal, and outputs it to the synthesis unit 40. When phase information is necessary for the frequency fa of the interpolation region during the conversion, the phase information for the frequency fa / 2 is used.

  The combining unit 40 converts the time domain signal obtained by converting the first interpolation frequency domain signal from the first reconversion unit 35 and the time domain obtained by converting the first interpolation frequency domain signal from the second reconversion unit 39. The signals are combined and output to the combining unit 50.

  As described above, since the sound data processing apparatus 1 can include harmonics of sounds other than the fundamental sound, it is possible to reproduce sounds closer to actual sounds.

Note that when there is no amplitude information of the frequency fa / 3 or the frequency fa / 2 in the frequency domain signal of the fundamental tone, the amplitude information for the frequency fa or the frequency fa of the interpolation domain is zero.
Moreover, instead of adding the amplitude information of the frequency fa / 2 or 0 as the amplitude information of the frequency fa in the high frequency region in the even harmonic generation unit 37 according to whether or not the predetermined condition is satisfied as described above. The even harmonic generation unit 37 adds the amplitude information of the frequency fa / 2 as the amplitude information of the frequency fa in the high frequency region regardless of whether or not the predetermined condition is satisfied. The second gain multiplication unit 38 may multiply the amplitude information of the corresponding frequency fa output from the even-numbered harmonic generation unit 37 by 0.

FIG. 8 is a diagram illustrating the determination process in the determination unit 36.
The determination unit 36 includes a band pass filter 36a. The band-pass filter 36a is configured so that only one frequency among a plurality of frequencies fa / 2 included in the time domain signal of the original sound can be selectively passed.

The determination unit 36 applies a filter that passes one frequency fa / 2 for each frequency fa / 2 using the band-pass filter 36a, and whether the time domain signal of the filtered original sound satisfies a predetermined condition. That is, it is determined whether or not the time interval Tin from the maximum value to the minimum value of the time domain signal of the filtered original sound is 4 / fa. As shown in FIG. 8A, the determination unit 36 applies a sine wave as shown in FIG. 8A, so the time interval Tin is 4 / fa (if fa is 1 kHz, 4 × 10 ⁻⁴ It is determined that “predetermined condition is satisfied”. If a time domain signal that is not a sine wave is obtained as a result of applying the filter as shown in FIG. 8B, the time interval Tin does not become 4 / fa, so it is determined that “the predetermined condition is not satisfied”.

In the even harmonic generation unit 37, as described above, when interpolating the amplitude information of the frequency fa in the high frequency region, the time domain signal of the original sound that has been filtered through the frequency fa / 2 is the result of the determination in the determination unit 36. When the predetermined condition is satisfied, the amplitude information of the frequency fa / 2 is added as the amplitude information of the frequency fa. However, when the predetermined condition is not satisfied, 0 is added as the amplitude information of the frequency fa.
By doing in this way, it becomes possible to reproduce the sound more different from the actual sound.

(Second Embodiment)
In the first embodiment, after the determination unit 36 is provided, the even harmonic generation unit 37 sets the amplitude value or 0 for the frequency fa / 2 in the frequency domain signal as the amplitude value of the frequency fa in the interpolation domain. I was adding. On the other hand, in this embodiment, the determination unit 36 is not provided, and the even harmonic generation unit 37 always uses the amplitude value for the frequency fa / 2 in the frequency domain signal of the original sound as the amplitude value of the frequency fa in the interpolation domain. Add values.
Also with this configuration, it becomes possible to interpolate high frequency components and reproduce the sound more different from the actual sound.

(Third embodiment)
In the first embodiment, the synthesis unit 40 synthesizes the output from the first reconversion unit 35 and the output from the second reconversion unit 39 at a one-to-one mixing ratio. On the other hand, in the third embodiment, the combining unit 40 can combine the output from the first reconversion unit 35 and the output from the second reconversion unit 39 by changing the mixing ratio. If the mixing ratio is variable according to the user input, it is possible to reproduce audio that suits the user's preference.

(Fourth embodiment)
In the first embodiment, when interpolating the amplitude information of the frequency fa in the interpolation region, the odd harmonic generation unit 33 extracts the amplitude information for the frequency fa / 3 in the frequency domain signal of the original sound, and even harmonics are generated. The generation unit 37 extracts amplitude information for the frequency fa / 2 in the frequency domain signal of the original sound.

On the other hand, in the fourth embodiment, when interpolating the amplitude information of the frequency fa in the interpolation region, the odd harmonic generation unit 33 performs the frequency information in addition to the amplitude information for the frequency fa / 3 in the frequency region signal of the original sound. The amplitude information for the frequency fa / (2m + 1) (m is a positive integer greater than or equal to 2), such as the amplitude information for fa / 5, is extracted, and the even-numbered harmonic generation unit 37 uses the frequency in the frequency domain signal of the original sound. In addition to amplitude information for fa / 2, amplitude information for frequency fa / 2m, such as amplitude information for frequency fa / 4, is extracted.
In this case, the first gain multiplied by the amplitude information for the frequency fa / (2m + 1) by the first gain multiplier 34 is 1 / (2m + 1), and the second gain multiplier 38 multiplies the amplitude information for the frequency fa / 2m. The second gain is 1 / 2m.

  By doing so, it is possible to reproduce a sound that is different from the actual sound.

  In addition, the present invention may be an electronic device including the audio data processing device according to any one of the first to fourth embodiments described above. The electronic device is, for example, a television receiver or various audio devices.

DESCRIPTION OF SYMBOLS 1 ... Audio | voice data processing apparatus, 10 ... Decoding part, 20 ... Upsampling part, 30 ... High frequency interpolation part, 31 ... Band detection part, 32 ... Frequency domain conversion part, 33 ... Odd harmonics generation part, 34 ... 1st Gain multiplying unit, 35 ... first reconversion unit, 36 ... determination unit, 36a ... bandpass filter, 37 ... even multiple harmonic generation unit, 38 ... second gain multiplication unit, 39 ... second reconversion unit, 40 ... Synthesis unit, 50... Synthesis unit.

Claims (5)

An audio data processing device including a high frequency interpolation unit that interpolates a frequency component of a high frequency region cut in the original sound with respect to a time domain signal of the original sound decoded based on audio data,
The high-frequency interpolation unit
A converter that converts the decoded time domain signal of the original sound into a frequency domain signal;
When the frequency fa component of the cut high frequency region is interpolated, amplitude information for the frequency fa / (2n + 1) (n is a positive integer) in the frequency region signal of the original sound is extracted, and the extracted amplitude information is 1 / A first interpolation signal generation unit for generating a first interpolation frequency domain signal obtained by multiplying (2n + 1) by the amplitude information Aa of the frequency fa;
A second frequency domain signal for interpolation is obtained by extracting amplitude information for the frequency fa / 2n in the frequency domain signal of the original sound and multiplying the extracted amplitude information by 1 / 2n as amplitude information Ae of the frequency fa. A second interpolation signal generator for generating
And a synthesis unit that synthesizes a signal obtained by converting the first interpolation frequency domain signal into a time domain signal and a signal obtained by converting the second interpolation frequency domain signal into a time domain signal. Audio data processing device. The high-frequency interpolator applies a filter that passes the frequency fa / 2n to the time domain signal of the original sound, and the time interval Tin from the maximum value to the minimum value of the filtered time domain signal is 4n. A determination unit for determining whether or not
The second interpolation signal generation unit sets the amplitude information Ae of the frequency fa to 0 when the time interval Tin of the original sound that has been filtered through the frequency fa / 2n is 4n / fa. The audio data processing apparatus according to claim 1.   The synthesizer can adjust a mixing ratio between a signal obtained by converting the first frequency domain signal for interpolation into a time domain signal and a signal obtained by converting the second frequency domain signal for interpolation into a time domain signal. The audio data processing apparatus according to claim 1, wherein the audio data processing apparatus is synthesized.   An electronic apparatus comprising the audio data processing device according to claim 1. The electronic apparatus according to claim 4, wherein the electronic apparatus is a television receiver.

Images