JP6003107B2 - Second sound signal embedding device for acoustic signal and interfering sound embedding device for acoustic signal - Google Patents

Description

  The present invention relates to the field of packaged music for viewing in consumer and business applications using CDs, DVDs, BDs, and the like, and the network music distribution field distributed by music content providers for commercial purposes, and in particular, prevents copying of music content. Related to technology.

  Conventionally, various techniques have been developed to prevent duplication of music content. For example, in a method called DRM (see Patent Document 1), duplication of music content is prevented by encrypting digital music content. Specifically, recording from a commercial DVD or BD media to another DVD-R or other recording media with a digital cable connection with a recorder device or inserting into a personal computer drive and ripping as a video file on a personal computer is not possible. It is possible. However, DRM has been released for music CDs due to various circumstances, so that digital pirated music files created by ripping from rental CDs can be uploaded to the site and distributed. It is a problem. In this DRM method, copying of digital content can be prevented, but copying of analog content cannot be prevented. In other words, the reproduction can be performed by photographing the display screen being reproduced with a video camera or recording the reproduction signal with a line or a microphone from the speaker output. At present, the biggest problem is that a small video camera is brought into a movie theater or hall, and the soundtrack flowing from the speaker is recorded along with the image projected on the screen, and the DVD is created inexhaustibly and shipped as a product (pirated version). There are examples in which illegally copied video files are distributed via video sites and P2P. Since consumer video cameras in recent years are HDTV compatible and can be recorded with an image quality comparable to that of a BD, it is easy to ensure commercial quality for a DVD that is duplicated using it as a master.

  Fortunately, in Japan, due to the effect of the “Act on the Prevention of Camera Voyeurism” enforced since 2007, the screen voyeurism incident ended the arrest of current offenders in Aichi Prefecture in 2009, but a new problem emerged doing. Since voyeurism continues to take place overseas, voyeur movies have landed in Japan via the Internet, but cannot be sold in Japan as they are. In other words, in order to distribute it in Japan, it is necessary to put a caption supervision into the voyeur movie or to dubb it in Japanese, but the latter is overwhelmingly in demand. Therefore, only the sound is recorded in a movie theater where a Japanese dubbed version of a pirated Western movie imported from overseas is screened, and it is dubbed into an overseas voyeur video and shipped. An example of a foreign film that was exported overseas and illegally copied from a foreign language version of a Japanese film was illegally transferred to a Japanese film, and only the voice was hidden in Japan. Also found. It is almost impossible to find a crime in a dark movie theater because it does not require a tripod like a screen voyeur and can be hidden in a pocket. In other words, voyeurism for video has been eradicated by legal development, but there is no edge for acoustic voyeurism. At present, the demand for audio content protection, including the illegal ripping problem of rental music CDs, is video content. It is much larger than protection.

Special table 2003-517767 gazette WO2011 / 002059 WO2011 / 105164 US2007 / 0192091 US2004 / 025261

  As a technique for preventing duplication of analog contents, a technique for adding an invisible copy disturbing signal to a video signal has been proposed as a countermeasure against the aforementioned pirated DVD manufacturing (see Patent Document 2). In the method disclosed in Patent Document 2, since infrared rays are used as a copy interference signal, it is invisible to humans, but is reflected in a video camera, and illegal copying can be suppressed. However, copy disturbing signals cannot be embedded in the content itself, and only work with screens and displays equipped with special modules that emit copy disturbing signals. Professional video cameras and cameras equipped with infrared cut filters When using, there is a problem that reflection of a copy disturbing signal can be avoided. Also, they are completely vulnerable to illegal copying of video soundtracks.

  On the other hand, as a method of preventing duplication in the digital state, a method of utilizing aliasing (folding distortion) at the time of sampling space (video / image) or time (sound) has been proposed (see Patent Document 3). In the past, an infinite frequency component could be recorded in a video signal or audio signal in an analog form, but in the process of converting to a digital form, 1/1 of the sampling frequency based on Shannon-Nyquist information theory. There is a restriction that a signal component exceeding the frequency of 2 (Nyquist frequency) cannot be recorded, and a high-band signal component exceeding the Nyquist frequency is folded back to a low frequency centering on the Nyquist frequency, and an erroneous frequency as a low-band signal component. It focuses on the phenomenon (aliasing) that is superimposed on the band. In Patent Document 3, a binary image for copy check is superimposed on a high band of each frame of a moving image to be protected. In normal playback, slight noise is recognized at the edge of the image. However, if a reduction operation is performed on the moving image so that the sampling frequency is ½ or less, the superimposed binary image is displayed. The signal component is folded back to a low band and superimposed on the moving image to visualize the binary image. In this method, since the frequency band in which the signal component of the binary image is embedded is not invisible, a slight quality degradation cannot be avoided during normal video playback, and the sampling frequency can be changed without changing the image size. There is a problem in that a superimposed binary image cannot be visualized if it is copied so as not to be changed, or once converted to an analog system and resampled.

As a prior art in the acoustic field similar to Patent Document 2, there is Patent Document 4. Similar to superimposing interference noise due to invisible infrared rays on the image in Patent Document 2, the latter is an ultrasonic speaker arranged together with a normal acoustic speaker to add interference noise due to ultrasound. is there. This interference noise is inaudible. However, when sampling is performed through a microphone, the noise is shifted to the audible band by aliasing utilized in Patent Document 3, and the characteristic that is made audible is used. In Patent Document 4, there is a problem that it is necessary to add an amplifier / speaker for ultrasonic output to the hardware, and when an additional test is performed based on Patent Document 4, the sensitivity of the ultrasonic band is almost the case with a normal recorder microphone. However, as described in this patent, it has been difficult to shift the interference noise into the audible band by aliasing.
Also, as a technique for suppressing analog copying of an acoustic signal for the same purpose as in Patent Document 4, in Patent Document 5, interference noise is embedded inaudibly in the high frequency part of the acoustic signal to be protected, and analog copying is performed. Non-linear distortion based on the tape recorder is generated in the spectrum, and the overtone and intermodulation components of the embedded interference noise are generated on the low frequency side, thereby making the interference noise audible. When a follow-up test was conducted based on Patent Document 5, nonlinear distortion as described in Patent Document 5 did not occur in a recent recorder, and it was difficult to make the interference noise audible.

  Therefore, the present invention makes use of aliasing characteristics at the time of sampling, samples an audio signal at a predetermined sampling frequency or less, performs duplication, and reproduces the digitized audio signal after duplication. Provided is a second acoustic signal embedding device for an acoustic signal and an interfering sound embedding device for an acoustic signal that can prevent duplication in the original state by reproducing an inaudible interfering sound. The task is to do.

In order to solve the above problems, in the first aspect of the present invention,
A device for embedding a second acoustic signal composed of a time-series sample sequence in an inaudible state with respect to an original acoustic signal of a sampling frequency Fs composed of a time-series sample sequence,
An acoustic frame reading means for reading a first acoustic frame composed of a predetermined number of samples from the original acoustic signal and for reading a second acoustic frame composed of a predetermined number of samples from the second acoustic signal;
Frequency conversion is performed on the first sound frame to obtain a first spectrum that is a complex frequency component, and frequency conversion is performed on the second sound frame to obtain a second spectrum that is a complex frequency component. Means,
Among the signal components of the first spectrum, the signal components of the frequency Ft or higher are removed, the signal components of the frequency Ft or lower are folded back in the high frequency direction around the frequency Ft, and the folded frequency Ft is used. while multiplying the inverting and predetermined coefficient value the sign for the first spectral signal component of the range of frequencies 2 Ft, by adding the signal component of the corresponding frequency before Symbol first spectrum, said first First frequency component modifying means for modifying the frequency component of the spectrum;
Among the signal components of the second spectrum, the signal components of the frequency Ft or higher are removed, the signal components of the frequency Ft or lower are folded around the frequency Ft in the high frequency direction, and the folded frequency Ft is used. By multiplying the signal component of the second spectrum in the range of frequency 2Ft by a predetermined coefficient value and adding to the signal component of the corresponding frequency of the modified first spectrum, the frequency component of the first spectrum A second frequency component modifying means for modifying
Frequency inverse transforming means for performing frequency inverse transform on the first spectrum in which the frequency component is modified to generate a modified acoustic frame;
There is provided a second acoustic signal embedding device for an acoustic signal having modified acoustic frame output means for sequentially outputting the generated modified acoustic frames.

  According to the first aspect of the present invention, among the signal components of the first spectrum of the original sound signal, the signal component of the predetermined frequency Ft or more is set to 0, and the signal component of the frequency Ft or less is centered on the frequency Ft. The first spectrum signal component in the range from the folded frequency Ft to the frequency 2Ft (twice Ft) is inverted with the sign of the first spectrum and multiplied by a predetermined coefficient value. After modifying the frequency component of the first spectrum by adding to the signal component of the corresponding frequency of one spectrum, the signal component having the frequency Ft or higher is removed from the signal component of the second spectrum, and the frequency Ft is changed to A signal component having a frequency less than or equal to the frequency Ft is folded at the center in the high frequency direction, and a predetermined spectrum signal component in the range of the folded frequency Ft to the frequency 2Ft is determined. Since it is added to the signal component of the corresponding frequency of the modified first spectrum while multiplying by a numerical value, in normal reproduction, the phase-inverted original sound signal embedded in the high frequency range of the original sound signal is reduced. The band component and the second acoustic signal component are inaudible, but when the reproduced sound signal is reproduced by re-sampling the normally reproduced sound at the sampling frequency 2Ft, the first in the range of the frequency Ft to the frequency 2Ft is used. On the contrary, the signal component having the inverted phase of the spectrum is folded back in the low frequency direction below the frequency Fk around the frequency Ft and superimposed (subtracted) on the signal component below the frequency Ft of the original acoustic signal. The sound based on the acoustic signal is canceled by a ratio of a predetermined coefficient value, and the signal component of the second spectrum in the range from the frequency Ft to the frequency 2Ft is conversely centered on the frequency Ft. It is folded in the direction of the low frequency below the wave number Fk, superimposed on the signal component below the frequency Ft of the original sound signal, and the sound based on the second sound signal is reproduced instead of the sound based on the suppressed original sound signal. Thus, it is possible to prevent reproduction of only sound based on the original original sound signal. In addition, the sampling frequency at the time of duplication need not be exactly 2Ft, and if it is less than or equal to 2Ft (3Ft or less), folding will occur centering around a half of the set sampling frequency, Although the signal component and the low frequency band to be folded are shifted from the desired position, the original sound signal is canceled halfway, and the second sound signal to be reproduced is distorted by adding distortion, but at least originally The purpose of preventing only reproduction of sound based on the original sound signal can be achieved.

In the second aspect of the present invention,
An apparatus that embeds an interfering sound in an inaudible state with respect to an original sound signal having a sampling frequency Fs composed of a time-series sample sequence,
Sound frame reading means for reading a first sound frame composed of a predetermined number of samples from the original sound signal;
Frequency conversion means for performing frequency conversion on the first acoustic frame to obtain a first spectrum which is a complex frequency component;
Among the signal components of the first spectrum, the signal components of the frequency Ft or higher are removed, the signal components of the frequency Ft or lower are folded back in the high frequency direction around the frequency Ft, and the folded frequency Ft is used. while multiplying the inverting and predetermined coefficient value the sign for the first spectral signal component of the range of frequencies 2 Ft, by adding the signal component of the corresponding frequency before Symbol first spectrum, said first First frequency component modifying means for modifying the frequency component of the spectrum;
The frequency component of the first spectrum is modified so that the absolute value of the signal increases by a predetermined value with respect to the signal component of the frequency Ft or more and the frequency 2 Ft or less among the signal components of the first spectrum. Two frequency component modification means;
Frequency inverse transforming means for performing frequency inverse transform on the first spectrum in which the frequency component is modified to generate a modified acoustic frame;
There is provided a device for embedding a disturbing sound in an acoustic signal having modified acoustic frame output means for sequentially outputting the generated modified acoustic frames.

According to the second aspect of the present invention, among the signal components of the first spectrum of the original sound signal, the signal component of the predetermined frequency Ft or more is set to 0, and the signal component of the frequency Ft or less is centered on the frequency Ft. The corresponding frequency of the first spectrum is folded in the high-frequency direction, while inverting the sign and multiplying the signal component of the first spectrum in the range of the folded frequency Ft to the frequency 2Ft by a predetermined coefficient value. After the frequency component of the first spectrum is altered by adding to the signal component of the first spectrum, the absolute value of the signal is greater than the signal component of the first spectrum with respect to the signal component having the frequency of Ft or higher and the frequency of 2 Ft or lower. Since the frequency component of the first spectrum is modified so as to increase by a predetermined value, in normal reproduction, the phase-inverted original acoustic signal embedded in the high frequency range of the original acoustic signal is used. The low frequency component and white noise component of the signal are inaudible, but when the reproduced sound signal is reproduced by re-sampling the normally reproduced sound at the sampling frequency 2Ft, the first in the range from the frequency Ft to the frequency 2Ft Conversely, the signal component of which the phase of the spectrum is inverted is folded back in the low frequency direction below the frequency Ft around the frequency Ft and superimposed (subtracted) on the signal component below the frequency Ft of the original acoustic signal. The sound based on the acoustic signal is canceled by a ratio of a predetermined coefficient value, and the component corresponding to the white noise in the range of Ft to frequency 2Ft is reversed in the frequency direction of the low band below the frequency Ft centering on the frequency Ft. Is reproduced instead of the sound based on the original sound signal with the white noise suppressed, superimposed on the signal component of the original sound signal having the frequency Ft or less, and based on the original original sound signal. It makes it possible to prevent the playback of sound only. In addition, the sampling frequency at the time of duplication need not be exactly 2Ft, and if it is less than or equal to 2Ft (3Ft or less), folding will occur centering around a half of the set sampling frequency, The signal component and the low frequency band that is folded back deviate from the desired position, the original acoustic signal is canceled halfway, and the frequency characteristics of the white noise that is reproduced instead change, but at least the original original acoustic signal The objective of preventing only the sound based on it can be achieved.

  According to a third aspect of the present invention, in the apparatus for embedding interference sound in the acoustic signal according to the second aspect of the present invention, the first frequency component modifying means and the second frequency component modifying means are read by the acoustic frame reading means. Among the acoustic frames, the processing is performed on the acoustic frames at predetermined intervals.

  According to the third aspect of the present invention, only one of the odd-numbered sound frames and the even-numbered sound frames is modified with respect to the sound frames at predetermined intervals. The amount of calculation processing for signal modification can be reduced, and the noise reproduction sound can be clarified. In particular, if the modification is performed only for every two acoustic frames, that is, only the odd-numbered acoustic frame or the even-numbered acoustic frame as the predetermined interval, the efficiency of embedding the interference sound with respect to the calculation processing amount is good.

According to a fourth aspect of the present invention, in the second acoustic signal embedding device or the disturbing sound embedding device for the acoustic signal according to any one of the first to third aspects of the present invention, , Further comprising upsampling means for performing upsampling at a sampling frequency Fs ′ (Fs ′> Fs) to create a wideband original sound signal, wherein Ft = Fs / 2, the sound frame reading means, and the frequency conversion means, said first frequency component changing means, the second frequency component changing means, said inverse frequency transform means, said modified acoustic frame output unit performs processing on the original acoustic signal after the upsampling It is characterized by being.

  According to the fourth aspect of the present invention, the frequency Ft that is the lower limit of the signal component removal of the original sound signal and is the center of folding is set to ½ of the sampling frequency Fs of the original sound signal, and the number of samples of the modified sound signal Is modified so as to be larger than the number of samples of the original sound signal, the lower limit of the low-frequency component of the embedded phase-inverted original sound signal, the second sound signal, or the component of the disturbing sound is Fs / 2. By setting the frequency Fs / 2 near the upper limit of the human audible range, the low frequency component of the original acoustic signal whose phase has been inverted, the second acoustic signal, or the signal component of the interfering sound is completely inaudible. The signal components are not heard at all. However, when the modified acoustic signal is converted into an analog system or digitally sampled and duplicated again at the sampling frequency Fs, when the reproduced acoustic signal is reproduced, aliasing occurs and the frequency Fs / 2 or higher is generated. The signal component of the disturbing sound is folded back to an audible band of frequency Fs / 2 or less around the frequency Fs / 2, the sound based on the original sound signal is suppressed, and the sound based on the second acoustic signal or the disturbing sound is audible instead. Is output in the correct state. For this reason, it is possible to prevent only the sound recorded in the original sound signal from being duplicated at the same sampling frequency as that of the original sound signal. In addition, the sampling frequency at the time of duplication need not be exactly Fs, and if it is less than or equal to Fs (3Fs / 2 (Fs multiplied by 3/2) or less), it is 1/2 of the set sampling frequency. The second acoustic signal is generated in which the signal component to be folded or the low frequency band to be folded is shifted from a desired position, the original acoustic signal is canceled halfway, and reproduced instead. Although the noise is added to the noise and the frequency characteristics of the disturbing sound are changed, at least the purpose of preventing the reproduction of only the sound based on the original original sound signal can be achieved.

  According to a fifth aspect of the present invention, in the second acoustic signal embedding device for an acoustic signal according to any one of the first to third aspects of the present invention or an interference sound embedding device for an acoustic signal, Ft = Fs / 4 It is characterized by being.

  According to the fifth aspect of the present invention, since the frequency Ft that is the center of the folding is set to ¼ of the sampling frequency Fs of the original sound signal, the low frequency component of the phase-inverted original sound signal and the second sound The upper limit of the signal component of the signal or the interference sound is Fs / 2, and the low-frequency component of the original sound signal and the second sound signal or When the signal component of the interfering sound is superimposed and replicated at the sampling frequency Fs / 2, when the reproduced acoustic signal is reproduced, aliasing occurs and the phase inversion from the frequency Fs / 4 to Fs / 2 occurs. The low-frequency component of the original sound signal and the signal component of the second sound signal or the disturbing sound are folded back to the frequency Fs / 4 or less around the frequency Fs / 4, and the sound based on the original sound signal is suppressed, Sound or interference sound based on the second audio signal is output at audible state. For this reason, it is possible to prevent only the sound recorded in the original sound signal from being duplicated at a sampling frequency that is half the sampling frequency of the original sound signal. Further, the sampling frequency at the time of duplication need not be exactly Fs / 2, and if it is around Fs / 2 or less (3Fs / 4 (Fs multiplied by 3/4) or less), 1 of the set sampling frequency. / 2 occurs, the signal component to be folded and the low frequency band to be folded are shifted from the desired position, and the original sound signal is canceled halfway and is reproduced instead. Although the second acoustic signal is distorted by adding distortion or the frequency characteristics of the disturbing sound are changed, at least the object of preventing the reproduction of only the sound based on the original original acoustic signal can be achieved.

  According to a sixth aspect of the present invention, in the second acoustic signal embedding device for an acoustic signal or the disturbing sound embedding device for an acoustic signal according to any one of the first to fifth aspects of the present invention, And one of the even-numbered sample values, a value obtained by multiplying the sample value immediately before the sample value by a weight G (Gl, Gr) that is a real value of 1 or less, and the sample value (1- G) further includes a modified acoustic frame correcting unit that corrects the modified acoustic frame by converting the sum into a sum of the multiplied value and the modified acoustic frame output unit sequentially outputs the modified modified acoustic frame. It is characterized by being.

  According to the sixth aspect of the present invention, every other sample value among the odd-numbered and even-numbered sample values of the modified acoustic frame is converted by multiplying by a predetermined weight, so that the sampling frequency is partially The same effect as when re-sampling is performed at a frequency of 1/2 of the low frequency component of the phase-inverted original sound signal embedded in the high frequency of 1/2 or more of the sampling frequency and the second The sound signal or the signal component of the interference sound is slightly folded in advance by a ratio based on the predetermined weight G. However, due to the effect of auditory masking based on the low-frequency component of the original acoustic signal, the low-frequency component of the original acoustic signal whose phase has been inverted and the second acoustic signal or the signal component of the disturbing sound cannot be heard. Since the weight G is adjusted to the level, only the original original sound signal is heard during normal reproduction. However, if re-sampling is performed at a predetermined sampling frequency Ft and duplication is performed, the signal may be re-sampled via an LPF (Low Pass Filter) circuit that attenuates a high-frequency signal component having a frequency Ft / 2 or higher. In general, when copied in such a manner, the low-frequency component of the phase-reversed original sound signal and the signal component based on the second sound signal or the disturbing sound are folded while being greatly attenuated. Since the same signal components are folded by a ratio based on the predetermined weight G, these signal components are strengthened. Then, the suppression of the original sound signal works, and the level of the second sound signal or the disturbing sound reproduced instead deviates from the range of the auditory masking by the original sound signal and can be heard.

  According to a seventh aspect of the present invention, in the second acoustic signal embedding device for the acoustic signal or the disturbing sound embedding device for the acoustic signal according to the sixth aspect of the present invention, the modified acoustic frame correcting means includes the modified acoustic frame in the modified acoustic frame. The weight G is calculated based on an average value of included samples.

  According to the seventh aspect of the present invention, the weight G by which the sample value is multiplied in order to add the low frequency component is the average value of the samples of each modified acoustic frame, so that it corresponds to each modified acoustic frame changing in time series. Can be modified.

  According to an eighth aspect of the present invention, in the second acoustic signal embedding device or the disturbing sound embedding device for the acoustic signal according to any one of the first to seventh aspects of the present invention, the frequency converting means comprises: As the window width N samples, the weight W (i) (0 ≦ W (i) ≦ 1) at the sample position i (0 ≦ i ≦ N−1) is W (i) = 0.5−0.5 cos (2πi). The frequency conversion is performed using a Hanning window function defined by / N).

  According to the eighth aspect of the present invention, since the Hanning window function is used when performing the frequency analysis, the frequency conversion is performed while reducing the distortion applied to the original acoustic signal without generating a pseudo-harmonic component. It becomes possible to do.

  According to the present invention, when a sound signal is sampled at a predetermined sampling frequency or less and copied, and the reproduced digital sound signal is reproduced, the sound based on the original sound signal is suppressed and instead inaudible. By reproducing the sound or the disturbing sound based on the second acoustic signal embedded in the recording medium, it is possible to prevent duplication in a state where only the original original acoustic signal is recorded.

It is a figure which shows the principle of aliasing. It is a figure which shows the basic principle of this invention. It is a hardware block diagram of the embedding apparatus of the 2nd acoustic signal with respect to the acoustic signal which concerns on this invention, or disturbance sound. It is a functional block diagram which shows the structure of the embedding apparatus of the 2nd acoustic signal with respect to the acoustic signal which concerns on this invention, or disturbance sound. It is a flowchart which shows the processing operation of the 2nd acoustic signal embedding apparatus with respect to the acoustic signal which concerns on 1st (3rd) embodiment. It is a figure which shows the concept in case the modified acoustic signal obtained by the process to step S8 is replicated with the realistic apparatus provided with the LPF circuit. It is a figure which shows the low frequency component correction process of step S9 and step S10 in 1st Embodiment. It is a figure which shows the concept in case the modified acoustic signal obtained by the process to step S10 is replicated with the realistic apparatus provided with the LPF circuit. It is a flowchart which shows the processing operation of the embedding device of the disturbance sound with respect to the acoustic signal which concerns on 2nd (4th) embodiment. It is a figure which shows the concept of the embedding process of the 2nd acoustic signal in 3rd Embodiment. It is a figure which shows the low frequency component correction process of step S9 and step S10 in 3rd Embodiment. It is a figure which shows the concept in case the modified acoustic signal obtained by the process to step S10 of 3rd Embodiment is replicated with the realistic apparatus provided with the LPF circuit.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(1. Basic concept of the present invention)
First, the basic concept of the present invention will be described. In the present invention, by utilizing aliasing distortion called aliasing, the signal component obtained by inverting the phase of the original sound signal for suppressing the level of the original sound signal and a new warning message for the original sound signal to be prevented from being duplicated. A modification for embedding a second sound signal or a disturbing sound such as white noise is performed, and processing for obtaining a modified sound signal is performed. The signal component of the embedded original phase-inverted signal and the signal component of the second or interfering sound cannot be heard during normal reproduction of the modified acoustic signal, but the reproduced modified acoustic signal is recorded. Duplicated audio signal resampled and duplicated (regardless of whether the copy method is digital copy or analog copy, it is limited to analog copy in order to prevent duplication with the same sampling frequency as the original sound signal) ) Is suppressed, the second acoustic signal such as a warning message and a disturbing sound such as white noise are reproduced in an audible state instead. However, since the phenomenon of aliasing is a well-known matter in the industry, anti-aliasing processing is installed in the recording device or sampler, specifically, an LPF circuit is installed in front of the sampler so that high-frequency noise is not mixed during recording. Is common. As a result, the high frequency component folded back by aliasing is attenuated by the LPF circuit, so that it is substantially equivalent to almost no folding, and the above-mentioned object cannot be realized. The present invention also proposes a solution for this. Specifically, a state in which aliasing slightly occurs in advance is set to a level that falls within the range of auditory masking. When resampling is performed through the LPF circuit, the folded signal component attenuated by the LPF is added to the slightly folded signal component in advance, strengthening it, deviating from the range of auditory masking, and the folded signal component can be heard. The above-mentioned purpose can be realized.

  FIG. 1 is a diagram illustrating the principle of aliasing. 1A shows the signal spectrum of the original acoustic signal, and FIG. 1B shows the signal spectrum of the sampling data obtained by sampling the original acoustic signal of FIG. 1A at the sampling frequency Fs. When the original sound signal is sampled at the sampling frequency Fs, a high-frequency signal component exceeding the Nyquist frequency Fs / 2 is folded back to the low frequency side centering on Fs / 2, and the low frequency signal of Fs / 2 or less. It is misanalyzed as a component and superimposed. This is aliasing. In the example of FIG. 1, by sampling at the sampling frequency Fs, the signal components of the frequencies Fs / 2 to Fs are inverted and superimposed in the frequency direction, and the signal components of the frequencies Fs to 3Fs / 2 are twice in the frequency direction. Inverted and superimposed in the original orientation. In the example of FIG. 1B, three types of spectral components are displayed in an overlapping manner, but in reality they are added as complex vectors.

  FIG. 2 is a diagram showing the basic principle of the present invention. 2A is a signal spectrum of an original acoustic signal sampled at the sampling frequency Fs, and FIG. 2B is a second acoustic signal that is a second acoustic signal sampled at the sampling frequency Fs. 2 (c) is a signal spectrum of the modified acoustic signal obtained as a result of the embedding process according to the present invention, and FIG. 2 (d) is obtained by re-sampling the modified acoustic signal by some replication means. It is a signal spectrum of a duplication acoustic signal.

  Both the original acoustic signal shown in FIG. 2 (a) and the second acoustic signal shown in FIG. 2 (b) are subjected to high-cut filter processing before sampling to remove components in the higher frequency range than the sampling frequency Fs / 2. Use something that has not occurred. Here, the same sampling frequency is expanded to 2 Fs (twice Fs) in advance with respect to the original sound signal shown in FIG. 1A, and the different sound signal shown in FIG. 2 is synthesized with the original acoustic signal whose sampling frequency is expanded, and the modified acoustic signal shown in FIG. 2C is obtained. At this time, in the present invention, the component of Fs / 2 or less of the original sound signal is folded in the frequency direction around Fs / 2 while being amplitude-inverted and synthesized.

  When the modified acoustic signal shown in FIG. 2C obtained in this way is reproduced, by setting Fs / 2 to a value higher than the human audible range (for example, 22.05 kHz), Fs / Sounds based on signal components derived from different acoustic signals higher than 2 are hardly audible to humans. When the modified acoustic signal shown in FIG. 2C is duplicated and resampled at the sampling frequency Fs, the duplicate acoustic signal shown in FIG. 2D is obtained. The duplicated sound signal shown in FIG. 2 (d) is aliased, and the original sound signal component is inverted in amplitude and turned back to an original sound signal of Fs / 2 or less, which is the human audible range, and the original sound signal is canceled out. Instead, the signal component derived from the second acoustic signal is folded back below Fs / 2, which is the human audible range. As a result, since the signal component derived from the second sound signal is mainly recorded in the duplicate sound signal, only the second sound signal such as a warning message recorded as the second sound signal can be heard by humans. . For this reason, duplication of the original acoustic signal can be prevented.

(2.1. Configuration of embedded device)
Next, a second acoustic signal or disturbance sound embedding device for an acoustic signal according to the present invention will be described. FIG. 3 is a hardware configuration diagram of the second acoustic signal or disturbance sound embedding device for the acoustic signal according to the present invention. The device for embedding the second sound signal or the disturbing sound with respect to the sound signal can be realized by a general-purpose computer. As shown in FIG. 3, a CPU 1 (CPU: Central Processing Unit) and a RAM 2 which is a main memory of the computer. (RAM: Random Access Memory), a large-capacity storage device 3 (for example, a hard disk, a flash memory, etc.) for storing programs and data executed by the CPU 1, and a key input I / F (interface) such as a keyboard and a mouse ) 4, a data input / output I / F (interface) 5 for data communication with an external device (such as a data storage medium), and a display output I / F (interface) for sending information to a display device (display) 6 are connected to each other via a bus.

  FIG. 4 is a functional block diagram showing the configuration of the second acoustic signal or disturbance sound embedding device for the acoustic signal according to the present invention. In FIG. 4, 8 is an upsampling means, 10 is an acoustic frame reading means, 20 is a frequency converting means, 30 is a frequency component modifying means, 40 is a frequency inverse transforming means, 45 is a modified acoustic frame correcting means, and 50 is a modified acoustic frame. An output means, 60 is a storage means, 61 is an acoustic signal storage section, and 62 is a modified acoustic signal storage section. Note that the apparatus shown in FIG. 4 can handle any of a stereo sound signal, a monaural sound signal, or a surround sound signal of three or more channels, but in the present embodiment, processing is performed on a stereo sound signal of two channels. The case of performing will be described.

  The up-sampling means 8 samples the first acoustic signal, which is the original acoustic signal stored in the acoustic signal storage unit 61, at a frequency higher than the sampling frequency of the first acoustic signal to obtain a broadband acoustic signal. It has a function. The acoustic frame reading means 10 has a function of reading a predetermined number of samples as one acoustic frame from each channel of the broadband acoustic signal. When embedding the second acoustic signal, the acoustic frame reading means 10 also has a function of reading a predetermined number of samples as one acoustic frame from each channel of the second acoustic signal. The frequency conversion unit 20 has a function of generating a frequency-dimensional complex number spectrum by frequency-converting an acoustic frame (hereinafter referred to as a first acoustic frame) read from the wideband acoustic signal by the acoustic frame reading unit 10 by Fourier transformation or the like. Have. When embedding the second acoustic signal, the frequency converting means 20 converts the frequency of the acoustic frame read from the second acoustic signal by the acoustic frame reading means 10 (hereinafter referred to as the second acoustic frame) by Fourier transformation or the like. It also has the function of generating a dimensional complex spectrum. The frequency component modifying means 30 has a function of modifying the spectrum obtained from the first acoustic frame. When embedding the second acoustic signal, the frequency component modifying means 30 has a function of modifying the spectrum obtained from the first acoustic frame while referring to the spectrum obtained from the second acoustic frame. The frequency inverse transform means 40 has a function of generating a time-order modified acoustic frame by performing frequency inverse transform on a plurality of complex spectra including the modified spectrum set. The modified acoustic frame correcting unit 45 has a function of correcting the modified acoustic frame obtained by the frequency inverse converting unit 40. The modified sound frame output means 50 has a function of sequentially outputting the generated modified sound frames.

  The storage unit 60 includes an acoustic signal storage unit 61 that stores the first acoustic signal including the second acoustic signal when the second acoustic signal is embedded, and a modified acoustic signal storage unit 62 that stores the modified acoustic signal. In addition, various other information necessary for processing is stored.

  Each component shown in FIG. 4 is actually realized by installing a dedicated program in hardware such as a computer and its peripheral devices as shown in FIG. That is, the computer executes the contents of each means according to a dedicated program.

  In the storage device 3 of FIG. 3, a dedicated program for operating the CPU 1 and causing the computer to function as a device for embedding different acoustic signals with respect to the acoustic signals is installed. By executing this dedicated program, the CPU 1 performs upsampling means 8, acoustic frame reading means 10, frequency conversion means 20, frequency component modification means 30, frequency inverse conversion means 40, modified sound frame correction means 45, modified sound. The function as the frame output means 50 is realized. In addition to realizing the function as the storage unit 60, the storage device 3 stores various data necessary for processing.

(2.2. Processing operation of embedded device)
(2.2.1. First embodiment)
The processing operation of the second acoustic signal embedding device and the disturbing sound embedding device for the acoustic signal according to the present invention will be described below. FIG. 5 is a flowchart showing the processing operation of the second acoustic signal embedding device for the acoustic signal according to the first embodiment of the present invention. As the first acoustic signal and the second acoustic signal, those sampled at 96 kHz, 48 kHz (for video / broadcasting business such as movies) or 44.1 kHz (for consumer use such as CD) as the sampling frequency Fs can be used. In the embodiment, one sampled at 48 kHz is used. In the present embodiment, the frequency Ft = Fs / 2, which is the center of folding, is used.

  First, the upsampling means 8 performs sampling by raising the sampling frequency higher than the original sampling frequency for the left and right channels of the stereo first acoustic signal stored in the acoustic signal storage unit 61, that is, Upsampling is performed (step S1). Although how much the sampling frequency is increased can be set as appropriate, in the present embodiment, the sampling frequency of 48 kHz of the original first acoustic signal is up-sampled to the sampling frequency of 96 kHz. Specifically, the up-sampling means 8 performs processing according to the following [Formula 1] on the first acoustic signals xl and xr of the total number of samples S of each channel stored in the acoustic signal storage unit 61. Execute.

[Formula 1]
xl ′ (s) = xl (s · 48/96)
xr ′ (s) = xr (s · 48/96)

  In the above [Expression 1], s = 0,..., S′−1, and S ′ = S · 96/48. As a result of executing the processing according to the above [Equation 1], the acoustic signal with the number of samples S for each channel is up-sampled into a broadband acoustic signal with the number of samples S 'for each channel. For samples that do not exist in the original acoustic signals xl and xr, interpolation is performed using the values of neighboring samples.

  The acoustic frame reading means 10 reads a predetermined number N of samples as one acoustic frame from each of the left and right channels of the upsampled stereo broadband acoustic signal (step S2). Similarly, the acoustic frame reading means 10 reads a predetermined number M of samples as one second acoustic frame from the left and right channels of the stereo second acoustic signal stored in the acoustic signal storage unit 61 (step S3). ).

  The number of samples N of one first sound frame and the number of samples M of a second sound frame read by the sound frame reading means 10 can be set as appropriate, but in this embodiment, hereinafter, N = 8192, M = 4096. The case will be described. Therefore, the sound frame reading means 10 sequentially reads 8192 samples for each of the left channel and the right channel from the wideband sound signal as the first sound frame, and sequentially stores 4096 samples for the left channel and the right channel from the second sound signal. It will be read as 2 sound frames.

  In the present embodiment, both the first and second sound frames have a predetermined number of odd-numbered sound frames and even-numbered sound frames (N / 2 = 4096 and M / 2 = 2048 in the present embodiment). Duplicate samples are set. Therefore, in the case of the second sound frame, if the odd-numbered sound frames are A1, A2, A3... From the top, and the even-numbered sound frames are B1, B2, B3. Are samples 4097 to 8192, A3 is samples 8193 to 12288, B1 is samples 2049 to 6144, B2 is samples 6145 to 10240, and B3 is samples 10241 to 14336. Note that the number of samples to be overlapped can be set as appropriate.

  Next, the frequency conversion means 20 performs frequency conversion on the first sound frame to obtain a complex spectrum of the first sound frame (step S4). Similarly, the frequency conversion means 20 performs frequency conversion on the second sound frame to obtain a complex spectrum of the second sound frame (step S5). In steps S4 and S5, specifically, frequency conversion is performed using a window function. As the frequency transform, various known methods such as Fourier transform, wavelet transform and the like can be used, but it is necessary to be a method capable of obtaining a complex spectrum. In the present embodiment, a case where Fourier transform is used will be described as an example.

  In general, when Fourier transform is performed on a predetermined signal, it is necessary to divide the signal into predetermined lengths. In this case, if Fourier transform is performed on the divided signal as it is in a rectangular window, the division is performed. A pseudo-harmonic component based on the boundary is generated. Therefore, in general, when Fourier transform is performed, the value of the window boundary is attenuated and changed using a window function called a Hanning window, and then the Fourier transform is performed on the changed value.

  Also in this embodiment, Hanning window functions W (i) and W2 (i) are used. The Hanning window functions W (i) and W2 (i) are set to take a maximum value 1 at the position of the predetermined sample number i in the center and take a minimum value 0 at the positions of the sample numbers i near both ends. . In this embodiment, the Fourier transform for each first acoustic frame is performed on the product multiplied by the Hanning window function W (i) defined by the following [Equation 2]. Further, the Fourier transform for each second acoustic frame is performed on a product multiplied by the Hanning window function W2 (i) defined by the following [Equation 3].

[Formula 2]
W (i) = 0.5−0.5 cos (2πi / N)

[Formula 3]
W2 (i) = 0.5−0.5 cos (2πi / M)

  In the present embodiment, the odd-numbered sound frame and the even-numbered sound frame are read in duplicate by predetermined samples for both the first sound frame and the second sound frame. Therefore, when the frequency component is modified and then restored to the state of the acoustic signal, when an odd-numbered acoustic frame multiplied by the window function and an overlapping sample of the even-numbered acoustic frame multiplied by the window function are added Should return to almost the original value. For this reason, when the window functions W (i) and W2 (i) are added in the overlapping portion between the odd-numbered acoustic frames and the even-numbered acoustic frames, it is necessary to define the whole section fixed value 1.

  When the frequency conversion unit 20 performs Fourier transform on the first sound frame, for the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1), The window function W (i) is used to perform processing according to the following [Equation 4] and correspond to the real part Al (j), the imaginary part Bl (j), and the right channel of the conversion data corresponding to the left channel. Real part Ar (j) and imaginary part Br (j) of the conversion data are obtained.

[Formula 4]
Al (j) = Σ _{i = 0,..., N-1} W (i) · Xl (i) · cos (2πij / N)
Bl (j) = Σi _{= 0,..., N-1} W (i) · Xl (i) · sin (2πij / N)
Ar (j) = Σi _{= 0,..., N-1} W (i) · Xr (i) · cos (2πij / N)
Br (j) = Σi _{= 0,..., N-1} W (i) · Xr (i) · sin (2πij / N)

  In the above [Expression 4], i is a serial number assigned to N samples in each acoustic frame, and takes an integer value of i = 0, 1, 2,... N−1. Further, j is a serial number assigned in order from the smallest value of the frequency value, and takes an integer value of j = 0, 1, 2,... N / 2-1 like i. When the sampling frequency is 96 kHz and N = 8192, if the value of j is different by one, the frequency will be different by about 11.7 Hz.

  When the frequency conversion means 20 performs the Fourier transform on the second sound frame, for the left channel signal X2l (i) and the right channel signal X2r (i) (i = 0,..., M−1) The window function W (i) is used to perform processing according to the following [Formula 5], and corresponding to the real part A2l (j), imaginary part Bl (j), and right channel of the conversion data corresponding to the left channel. Real part A2r (j) and imaginary part Br (j) of the conversion data are obtained.

[Formula 5]
A2l (j) = Σi _{= 0,..., N-1} W (i) .X2l (i) .cos (2πij / M)
B2l (j) = Σi _{= 0,..., N-1} W (i) .X2l (i) .sin (2πij / M)
A2r (j) = Σi _{= 0,..., N-1} W (i) .X2r (i) .cos (2πij / M)
B2r (j) = Σi _{= 0,..., N-1} W (i) .X2r (i) .sin (2πij / M)

  In the above [Expression 4] and [Expression 5], i is a serial number assigned to M samples in each acoustic frame, and takes an integer value of i = 0, 1, 2,... M−1. Further, j is a serial number assigned to the frequency values in ascending order, and takes an integer value of j = 0, 1, 2,... M / 2-1 as with i. When the sampling frequency is 48 kHz and M = 4096, if the value of j is different by one, the frequency will be different by about 11.7 Hz.

  In steps S4 and S5, the processes according to the above [Equation 4] and [Equation 5] are executed, whereby complex spectra corresponding to the first and second acoustic frames are obtained. Subsequently, the frequency component modifying means 30 modifies the frequency component in steps S6 and S7. The frequency component modification unit 30 can be classified as a first frequency component modification unit and a second frequency component modification unit. The first frequency component modification unit performs the process of step S6, and the second frequency component modification unit includes the second frequency component modification unit. The process of step S7 is performed. First, the frequency component modification means 30 performs a process of adding a low frequency cancellation component to a high frequency using the first spectrum obtained from the first acoustic frame (step S6). Specifically, the first spectrum component is inverted between positive and negative, and a process of turning around the frequency Fs / 2 is performed.

  Once the low frequency cancellation component addition processing in step S6 has been performed, the frequency component modification means 30 then combines the first spectrum and the second spectrum after the low frequency cancellation component addition processing (step S7). . The second spectrum is added while folding around the frequency Fs / 2. As a result, the first spectrum (synthetic spectrum) is in a state as shown in FIG. In the first spectrum after synthesis (the spectrum of the modified acoustic signal), a signal component obtained by inverting the amplitude of the signal component existing in the original first spectrum and a signal component of the second spectrum in a range larger than the frequency Fs / 2, It exists in a folded state.

  In the example of FIG. 2C, two types of spectral components are superimposed and displayed at frequencies Fs / 2 to Fs, but in reality, they are added as complex vectors. As shown in FIG.2 (c), in the spectrum after a synthesis | combination, in frequency Fs / 2-Fs, the amplitude inversion component (low frequency cancellation component) of a 1st acoustic signal and the component derived from a 2nd acoustic signal are both. Since both are made noise and masked by the low frequency component of the first acoustic signal having high energy, even if the modified acoustic signal is reproduced, the amplitude inversion component of the first acoustic signal and the component of the second acoustic signal are , Not heard by humans.

  The processing performed by the frequency component modifying means 30 in steps S6 and S7 can be summarized as the following [Equation 6].

[Formula 6]
Al ′ (j) ← −Al (N / 2−j) · α + A2l (M−j) · β
Bl ′ (j) ← −Bl (N / 2−j) · α + B2l (M−j) · β
Ar ′ (j) ← −Ar (N / 2−j) · α + A2r (M−j) · β
Br ′ (j) ← −Br (N / 2−j) · α + B2r (M−j) · β

  The frequency component modifying means 30 performs the processing according to the above [Equation 6] on the frequency components Al (j), Bl (j), A2l (j), B2l (j), j = N / 4 + 1,. .., for each j of N / 2-1. In the case where the number of samples of one acoustic frame N = 8192 and the sampling frequency Fus = 96 kHz (= 2Fs), j = N / 4 corresponds to the frequency Fs / 2 (Fus / 4), and j = N / 2 is the frequency Corresponds to Fs (Fus / 2). Therefore, in [Formula 6], j = 2049,..., 4096 is a processing target, and the frequency component of about 24.0 kHz (approximately the upper limit of the human audible range) to about 47.9 kHz is changed.

  In the above [Expression 6], α and β are real scaling values set in the range of 0.0 <α and β ≦ 1.0. In the present embodiment, α = β = 1.0 is set. The first term (−Al (N / 2−j) · α, etc.) on the right side of each equation shown in [Formula 6] is a low-frequency canceling component of the first spectrum, and the second term ( A2l (M−j) · β, etc.) is a component obtained by folding the second spectrum in the frequency direction.

  When the correspondence between the flowchart of FIG. 5 and the above [Equation 6] is shown, the process of adding the low frequency cancellation component to the high frequency in Step S6 corresponds to the process of adding the first term on the right side of the above [Equation 6]. Then, the synthesis process of the first spectrum and the second spectrum in step S7 corresponds to the process of adding the second term on the right side of the above [Equation 6].

  After the frequency component modifying unit 30 finishes the synthesis process in step S7, the frequency inverse transform unit 40 performs a process of performing frequency inverse transform on the modified first spectrum to obtain a modified acoustic frame (step S8). . Naturally, the inverse frequency conversion needs to correspond to the method executed by the frequency conversion means 20. In the present embodiment, since the frequency transform unit 20 performs the Fourier transform, the frequency inverse transform unit 40 executes the Fourier inverse transform.

  Specifically, the frequency inverse transforming means 40 has a real part Al ′ (j), an imaginary part Bl ′ (j) of the left channel of the first spectrum obtained by the frequency component modifying means 30, and a real part Ar of the right channel. Using ′ (j) and imaginary part Br ′ (j), processing according to the following [Equation 7] is performed to calculate Xl ′ (i) and Xr ′ (i). The frequency components that have not been modified by the frequency component modifying means 30 are Al ′ (j), Bl ′ (j), Ar ′ (j), and Br ′ (j), which are the original frequency components, Al. (J), Bl (j), Ar (j), and Br (j) are used.

[Formula 7]
Xl' (i) = 1 / N · {Σ j Al' (j) · cos (2πij / N) -Σ j Bl' (j) · sin (2πij / N)} + Xlp (i + N / 2)
Xr' (i) = 1 / N · {Σ j Ar' (j) · cos (2πij / N) -Σ j Br' (j) · sin (2πij / N)} + Xrp (i + N / 2)

In the above [Expression 7], Σ _{j = 0,..., N−1} is shown as Σ _{j in} order to prevent the expression from becoming complicated. The terms “+ Xlp (i + N / 2)” in the first expression and “+ Xrp (i + N / 2)” in the second expression in the above [Expression 7] are the data Xlp (i) of the modified acoustic frame modified immediately before, When Xrp (i) exists, the addition is performed in consideration of the overlap of N / 2 samples on the time axis. By the above [Equation 7], each sample Xl ′ (i) of the left channel and each sample Xr ′ (i) of the right channel of the modified acoustic frame are obtained.

  After the frequency inversion in step S8, the modified acoustic frames obtained can be sequentially output to obtain the modified acoustic signal. Even in the processing up to step S8, in the experiment, as described with reference to FIG. 2, when the digital sound is duplicated by re-sampling, the original sound signal is suppressed, and the second sound is used instead. It can be confirmed that the component of the signal is audible. However, if digitally re-sampling is performed using a widely used acoustic signal processing tool, or if replication is performed using an A / D converter or sampler generally used for replication, the above-mentioned The desired effect can hardly be realized. The reason for this is that commercial acoustic signal processing tools, samplers, recording devices, etc. usually have an anti-aliasing LPF circuit in front, which is higher than half the sampling frequency specified at the time of sampling. This is because sampling is performed after the frequency component of the frequency band is attenuated, and measures are taken to prevent quality degradation due to the high frequency aliasing signal component. Therefore, the signal component for canceling the original sound signal embedded in the processing up to step 8 and the signal component for generating the second sound signal are usually deleted.

  FIG. 6 is a diagram showing a concept in a case where the modified acoustic signal obtained by the processing up to step S8 is duplicated by a realistic device including an LPF circuit. In FIG. 6, FIG. 6 (a) is the same as FIG. 2 (c), and shows the signal spectrum of the modified acoustic signal obtained as a result of the processing up to step S8. FIG. 6B shows a filter gain obtained by LPF (Low Pass Filter) processing. FIG. 6C shows a signal spectrum of a signal obtained by performing LPF processing on the modified acoustic signal. FIG. 6D shows a signal spectrum of a duplicated sound signal obtained by re-sampling the modified sound signal after LPF processing.

  As shown in FIG. 6B, the filter gain by the LPF processing is constant at an attenuation value of 0 [dB] up to the frequency Fs / 2, decreases sharply around the frequency Fs / 2, and reaches a maximum when it decreases to some extent. The attenuation value is constant at a value (such as −30 dB). Of course, when analog duplication processing is performed, even when digital duplication processing is performed, if the sampling frequency is changed, normal LPF processing is performed as sampling preprocessing. The modified acoustic signal shown in Fig. 6 is subjected to LPF processing having the characteristics shown in Fig. 6B. A signal obtained by performing the LPF process on the modified acoustic signal is as shown in FIG. As can be seen by comparing FIG. 6 (a) and FIG. 6 (c), the part corresponding to the original acoustic signal having the frequency Fs / 2 or less remains as it is, and it corresponds to the cancellation component and the interfering sound exceeding the frequency Fs / 2. The component to be attenuated is affected by the LPF.

  Furthermore, when the signal shown in FIG. 6C is resampled at the sampling frequency Fs, the duplicate acoustic signal shown in FIG. 6D is obtained. In the duplicated sound signal, aliasing occurs, and the attenuated canceling component and the component corresponding to the interfering sound are folded back to the frequency Fs / 2 or less which is a human audible range. As a result, as shown in FIG. 6D, in the range of the frequency Fs / 2 or less of the duplicated acoustic signal, the original acoustic signal component attenuated slightly due to the influence of the attenuated cancellation component. A component corresponding to the second acoustic signal is superimposed. The attenuated second acoustic signal is masked by the remaining original acoustic signal component that has been slightly attenuated, but is almost inaudible to humans, and the anti-duplication effect is lost. In the present embodiment, the following steps S9 and S10 are performed so that the second acoustic signal can be heard even when the LPF process is performed at the time of replication.

  FIG. 7 is a diagram illustrating the processes in steps S9 and S10. 7A is the same as FIG. 2C and FIG. 6A, and shows the signal spectrum of the modified acoustic signal obtained as a result of the processing up to step S8. FIG. 7B shows a signal spectrum of a signal obtained by thinning out the modified acoustic signal of FIG. FIG. 7C shows a signal spectrum of a signal obtained by up-sampling FIG. 7B again to restore the sampling frequency and attenuate it by a predetermined ratio. FIG. 7D shows the signal spectrum of the corrected modified acoustic signal obtained by synthesizing the signal of FIG. 7C with the modified acoustic signal of FIG.

  Thinning / enlarging processing is performed on the modified acoustic signal shown in FIG. 7A obtained by the processing up to step S8 (step S9). Specifically, first, as a thinning process, a modified acoustic signal having a sampling frequency of 2Fs is down-sampled at the sampling frequency Fs. An image of the spectrum of the signal down-sampled at the sampling frequency Fs is as shown in FIG. As the down-sampling method, a method of simply thinning out every other sample is taken, and a down-sampling method (because the LPF action works) that takes an average value is not used.

  Further, an expansion process in the frequency direction of the thinned signal is performed. Specifically, the sampling frequency Fs signal subjected to the thinning process is up-sampled at the sampling frequency 2Fs. As an up-sampling method, an interpolation method using adjacent samples as they are is taken, and an up-sampling method (because of the LPF action) that interpolates with an average value between adjacent samples is not used. After up-sampling at the sampling frequency 2Fs, the image of the spectrum of the signal from which components exceeding the frequency Fs / 2 are removed is as shown in FIG.

  Subsequently, the signal after the thinning / enlarging process is weighted and synthesized with the modified acoustic signal (step S10). That is, a signal attenuated by a predetermined ratio as shown in FIG. 7C is added to the modified acoustic signal shown in FIG. As a result, a corrected modified acoustic signal as shown in FIG. 7D is obtained. Specifically, the signal component after the thinning / enlargement process / amplitude attenuation process is added to the component of the modified acoustic signal having a frequency of Fs / 2 or less.

  The processing performed by the modified acoustic signal correcting means 55 in steps S9 and S10 can be summarized as the following [Equation 8].

[Formula 8]
Xl ′ (k · 2 + 1) ← Xl ′ (k · 2) · Gl + Xl ′ (k · 2 + 1) · (1-Gl)
Xr ′ (k · 2 + 1) ← Xr ′ (k · 2) · Gr + Xr ′ (k · 2 + 1) · (1-Gr)

  In the above [Equation 8], only the odd-numbered sample values of the modified acoustic signal are corrected to slightly add the previous even-numbered sample values by a predetermined ratio. Specifically, the left side is the odd-numbered sample value to be corrected for the modified acoustic signal, and the first term on the right side is the weighted G1 or Gr multiplied by the even-numbered sample value immediately before the modified acoustic signal to be corrected. The second term on the right-hand side is obtained by multiplying the odd-numbered sample value to be corrected by the modified acoustic signal by the weight (1-Gl) or (1-Gr). The weights Gl and Gr are calculated by processing according to the following [Equation 9].

[Formula 9]
Gl = 0.2 · Σi _{= 0,..., N−1} | Xl ′ (i) | / N / 32768
Gr = 0.2 · Σ _{i = 0,..., N−1} | Xr ′ (i) | / N / 32768

  In the above [Equation 9], the multiplication by 0.2 is to leave 80% of the sample on average and to obtain 20% from the adjacent sample, which is a value that can be changed as appropriate. About 2 is desirable. However, in order to prevent hearing using the auditory masking effect even if such modification is applied, the weights Gl and Gr are increased or decreased around 0.2 based on the average level of the modified acoustic signal. The reason why 32768 is divided at the end of [Formula 9] is to normalize Gl and Gr to the range of 0-1.

  The modified sound frame output unit 50 sequentially outputs the modified sound frames after correction obtained by the processing of the modified sound signal correction unit 55 to the output file. The process shown in the flowchart of FIG. 5 is executed for all the first sound frames of the wideband sound signal. When the number of the first sound frames is larger than the number of the second sound frames, the process returns to the first second sound frame and is repeated. As a result of performing processing on all the first sound frames in this way, a modified sound signal that is a set of modified sound frames is stored in the modified sound signal storage unit 62. In the present embodiment, the second acoustic signal is embedded from the beginning to the end of the first acoustic signal. However, it is also possible to set an embedding position and embed only in that range.

  FIG. 8 is a diagram showing a concept in a case where the modified acoustic signal obtained by the processing up to step S10 is duplicated by a realistic device including an LPF circuit. In FIG. 8, FIG. 8A shows a signal spectrum of the modified acoustic signal after correction obtained as a result of the processing up to step S10. FIG. 8B is the same as FIG. 6B and shows the filter gain by LPF (Low Pass Filter) processing. FIG.8 (c) shows the signal spectrum of the signal which performed the LPF process with respect to the modified acoustic signal. FIG. 8D shows a signal spectrum of a duplicated acoustic signal obtained by resampling the modified acoustic signal after LPF processing.

  A signal obtained by performing the LPF process on the modified acoustic signal is as shown in FIG. As can be seen by comparing FIG. 8 (a) and FIG. 8 (c), the portion corresponding to the original acoustic signal having a frequency of Fs / 2 or lower, and the thinned / enlarged component remain as they are, and the original exceeding the frequency Fs / 2 remains. The canceling component of the acoustic signal and the component corresponding to the second acoustic signal are attenuated by the influence of the LPF.

  Further, when the signal shown in FIG. 8C is resampled at the sampling frequency Fs, a duplicate acoustic signal shown in FIG. 8D is obtained. In the duplicate sound signal, aliasing occurs, and the canceling component of the attenuated original sound signal and the component corresponding to the second sound signal are folded back to a frequency Fs / 2 or less that is a human audible range. As a result, as shown in FIG. 8 (d), in the range below the frequency Fs / 2 of the duplicate acoustic signal, the cancellation component of the original acoustic signal attenuated to the thinned and enlarged component and the second acoustic signal are supported. Since these components are in phase with each other, they are strengthened, and as a result, the component that corresponds to the cancellation component of the attenuated original sound signal and the second sound signal is enhanced. That is, the suppression effect of the original acoustic signal that is a masker increases, the level of the second acoustic signal that is a masky increases, the auditory masking effect does not work, and the second acoustic signal can be heard by humans. If the second acoustic signal such as a warning message is heard over the music, it is objectively found to be an illegal copy, and the music cannot be appreciated in its original state and cannot be commercialized. For this reason, deterrence against replication works.

  Further, even when the sampling frequency when replicating the modified acoustic signal is Fs ′ slightly smaller than Fs and larger than Fs / 2, aliasing similarly occurs, and the signal components of the frequencies Fs ′ / 2 to Fs ′ are inverted. And superimposed on a frequency band of Fs ′ / 2 or less. That is, the left end of the right half of FIG. 8A is shifted to the left, and the signal component having a narrower bandwidth than the right half of FIG. 8A is shifted to the left of the right end of the left half of FIG. Wrapped to When this duplicated sound signal is reproduced, the original sound signal is canceled halfway, and it cannot be expected that a sound equivalent to the sound recorded in the second sound signal will be emitted, but the original sound signal is accompanied by distortion and considerable noise. The sound based on the canceling component of the acoustic signal or the second acoustic signal is clearly heard by the human ear. Even in this case, the purpose of not being able to obtain reproduced sound that can be enjoyed for viewing can be achieved, so that duplication can be suppressed. Furthermore, when the sampling frequency when replicating the modified sound signal is smaller than Fs / 2, significant aliasing occurs, and the main signal component of the original sound signal itself is destroyed to obtain a reproduced sound that can be appreciated. Can not be. Conversely, even when the sampling frequency when replicating the modified acoustic signal is Fs ′ slightly larger than Fs and smaller than 3 Fs / 2, aliasing similarly occurs, and the signal components of the frequencies Fs ′ / 2 to Fs ′ are Inverted and superimposed on the frequency band below Fs ′ / 2.

  That is, the right end of the right half of FIG. 8A is shifted to the right, and the signal component having a wider bandwidth than the right half of FIG. 8A is shifted to the right of the right end of the left half of FIG. Wrapped to When this duplicated sound signal is reproduced, the original sound signal is canceled halfway, and it cannot be expected that a sound equivalent to the sound recorded in the second sound signal will be emitted, but the original sound signal is accompanied by distortion and considerable noise. The sound based on the canceling component of the acoustic signal or the second acoustic signal is clearly heard by the human ear. Even in this case, the purpose of not being able to obtain reproduced sound that can be enjoyed for viewing can be achieved, so that duplication can be suppressed. However, if it is larger than 3Fs / 2, aliasing hardly occurs, the quality of the original sound signal and the second sound signal is maintained, the second sound signal is not made audible, and duplication cannot be suppressed. From the above, aliasing occurs when the sampling frequency when replicating the modified acoustic signal is smaller than 3Fs / 2. It becomes possible to suppress duplication.

(2.2.2 Second Embodiment)
Next, a description will be given of a disturbance sound embedding device for an acoustic signal according to a second embodiment of the present invention. FIG. 9 is a flowchart showing the processing operation of the interference sound embedding device for the acoustic signal according to the second embodiment. In the first embodiment, the second acoustic signal prepared in advance is used. However, in the second embodiment, processing for embedding artificially generated noise such as white noise without preparing a sound source in advance is performed. Do. Also in the present embodiment, as in the first embodiment, the first acoustic signal is sampled at 48 kHz as the sampling frequency Fs. In the present embodiment, the frequency Ft = Fs / 2, which is the center of folding, is used.

  First, as in step S1 in the first embodiment, the upsampling means 8 samples the left and right channels of the stereo first acoustic signal stored in the acoustic signal storage unit 61 more than the original sampling frequency. A sampling process is performed by increasing the frequency (step S11). Also in the present embodiment, as in the first embodiment, the processing according to the above [Equation 1] is executed, and the sampling frequency 48 kHz of the original first acoustic signal is upsampled to the sampling frequency 96 kHz.

  The acoustic frame reading means 10 reads a predetermined number N of samples as one acoustic frame from each of the left and right channels of the upsampled stereo broadband acoustic signal (step S12).

  Although the number N of samples of one first sound frame read by the sound frame reading means 10 can be set as appropriate, in the present embodiment, a case where N = 4096 will be described below. Therefore, the acoustic frame reading means 10 sequentially reads 4096 samples for each of the left channel and the right channel from the broadband acoustic signal as the first acoustic frame.

  Also in the present embodiment, as in the first embodiment, the odd-numbered sound frames and the even-numbered sound frames are set by overlapping a predetermined number of samples (N / 2 = 2048 in the present embodiment). Therefore, if the odd-numbered acoustic frames are A1, A2, A3... From the top, and the even-numbered acoustic frames are B1, B2, B3... From the top, A1 is samples 1 to 4096, A2 is samples 4097 to 8192, A3. Is samples 8193-12288, B1 is samples 2049-6144, B2 is samples 6145-10240, and B3 is samples 10241-14336. Note that the number of samples to be overlapped can be set as appropriate.

  Next, the frequency conversion means 20 performs frequency conversion on the first sound frame to obtain a complex spectrum of the first sound frame (step S13). In step S13, specifically, frequency conversion is performed using a window function. As the frequency conversion, various known methods for obtaining a complex spectrum, such as Fourier transform, wavelet transform, and the like, can be used. In the present embodiment, as in the first embodiment, a case where Fourier transform is used will be described as an example.

  In the present embodiment, the Fourier transform for each first acoustic frame is performed on the product multiplied by the Hanning window function W (i) defined by the above [Equation 2].

  When the frequency conversion unit 20 performs Fourier transform on the first sound frame, for the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1), The window function W (i) is used to perform the processing according to the above [Equation 4] and the conversion corresponding to the real part Al (j), the imaginary part Bl (j), and the right channel of the conversion data corresponding to the left channel. Real part Ar (j) and imaginary part Br (j) of data are obtained. As a result of the above processing of [Equation 4], when the sampling frequency is 96 kHz and N = 4096, if the value of j is different by one, the frequency is different by about 23.4 Hz.

  By executing the processing according to [Formula 4] in step S13, a complex spectrum corresponding to each first acoustic frame is obtained. Subsequently, the frequency component modifying means 30 performs a process of adding a low frequency cancellation component to the high frequency using the first spectrum obtained from the first acoustic frame (step S14). Specifically, similarly to step S6 in the first embodiment, the first spectrum component is inverted in the positive and negative directions, and the process of folding around the frequency Fs / 2 is performed. The low frequency cancellation component addition processing in step S14 is performed only for odd-numbered sound frames, and is not performed for even-numbered sound frames. By turning On / Off the low-frequency canceling component at odd and even numbers, the ratio of modification to the original sound signal is halved, and noise is generated by adding low frequency alternating fluctuations of On / Off. The reproduced sound can be made clear.

  Once the low frequency cancellation component addition processing in step S14 has been performed, the frequency component modification means 30 then performs processing for adding white noise to the first spectrum after the low frequency cancellation component addition processing (step S15). ). Specifically, processing according to the following [Equation 10] is executed, and processing for adding white noise, which is noise that equally includes each frequency component, to a predetermined high frequency range is performed.

[Formula 10]
When Al (j) ≧ 0 Al ′ (j) ← −Al (N / 2−j) · α + γ
When Al (j) <0 Al ′ (j) ← −Al (N / 2−j) · α−γ
When B1 (j) ≧ 0 B1 ′ (j) ← −B1 (N / 2−j) · α + γ
When B1 (j) <0 B1 ′ (j) ← −B1 (N / 2−j) · α−γ
When Ar (j) ≧ 0 Ar ′ (j) ← −Ar (N / 2−j) · α + γ
When Ar (j) <0 Ar ′ (j) ← −Ar (N / 2−j) · α−γ
When Br (j) ≧ 0 Br ′ (j) ← −Br (N / 2−j) · α + γ
When Br (j) <0 Br ′ (j) ← −Br (N / 2−j) · α−γ

  The frequency component modifying means 30 performs the processing according to the above [Formula 10] on the frequency components Al (j), Bl (j), Ar (j), Br (j), j = 1,. , N / 2 for each j. When the number of samples of one acoustic frame is N = 4096 and the sampling frequency after upsampling is Fs ′ = 96 kHz, j = N / 4 corresponds to the frequency Fs ′ / 4, and j = N / 2 is the frequency Fs / 2. Corresponding to It should be noted that the high frequency component exceeding j = N / 2 is not modified. In the above [Expression 10], α is a scaling real value set in a range of 0.0 <α ≦ 1.0. In this embodiment, α = 1.0 is set. In addition, γ is a real value of the signal level. In this embodiment, Al (j), Bl (j), Ar (j), and Br (j) are defined in a 16-bit range (−32768 to +32767). Γ = 10240.0 is set. The first term (−Al (N / 2−j) · α, etc.) on the right side of each equation shown in [Formula 10] is a low-frequency canceling component, and the second term γ on the right side of each equation is white noise. It is.

  As shown in [Equation 10] above, a predetermined intensity γ is given as white noise so as to increase the absolute values of the real and imaginary components of the complex number. The white noise addition processing in step S15 is performed only for odd-numbered sound frames, and is not performed for even-numbered sound frames. By turning on / off white noise at odd and even numbers, the ratio of modification to the original sound signal is halved and noise reproduction sound is reduced by adding low frequency alternating fluctuations of On / Off. Can be clear.

  In the present embodiment, the frequency component modification processing in steps S14 and S15 is performed on the odd-numbered sound frames and is not performed on the even-numbered sound frames, but is modified on every other sound frame. It is not always necessary to perform the processing. If the effect of the interfering sound is enhanced, the modification processing may be performed on every second or third sound frame, and conversely, it is performed on all sound frames. You may do it. In the above example, since the odd number and the even number are relative, the process may be performed on the even-numbered sound frame and may not be performed on the odd-numbered sound frame. .

  After the frequency component modification unit 30 performs the white noise addition in step S15 and completes the modification process, the frequency inverse transform unit 40 performs a process of performing frequency inverse transform on the modified spectrum to obtain a modified acoustic frame. (Step S16). Naturally, the inverse frequency conversion needs to correspond to the method executed by the frequency conversion means 20. In the present embodiment, since the frequency transform unit 20 performs the Fourier transform, the frequency inverse transform unit 40 performs the Fourier inverse transform by the process according to the above [Equation 7].

  After the frequency inversion in step S16, the modified acoustic frames obtained can be sequentially output to obtain the modified acoustic signal. However, even if duplication is performed at this stage, as described above, the anti-aliasing process that precedes the A / D converter and sampler used during duplication cancels out and disturbs the embedded original sound signal. Usually, signal components that generate sound are attenuated. Therefore, in the present embodiment as well, in the same way as in the first embodiment, the following steps S17 and S18 are performed so that a desired disturbing sound can be heard even when the LPF process is performed during duplication. Do.

  First, thinning / enlarging processing is performed on the modified acoustic signal obtained by the processing up to step S16 (step S17). Specifically, first, as the thinning-out process, the modified acoustic signal having the sampling frequency 2Fs is down-sampled at the sampling frequency Fs as in the first embodiment.

  Further, an expansion process in the frequency direction of the thinned signal is performed. Specifically, the sampling frequency Fs signal subjected to the thinning process is up-sampled at the sampling frequency 2Fs.

  Subsequently, the signal after the thinning / enlarging process is weighted and synthesized with the modified acoustic signal (step S18). Specifically, the signal component after the thinning / enlarging process is added to the component of the modified acoustic signal having the frequency Fs / 2 or less.

  The processing performed by the modified acoustic signal correcting unit 55 in steps S17 and S18 can be summarized as the above [Equation 8] as in the first embodiment.

  The modified sound frame output unit 50 sequentially outputs the modified sound frames after correction obtained by the processing of the modified sound signal correction unit 55 to the output file. The process shown in the flowchart of FIG. 9 is executed for all the first sound frames of the wideband sound signal. As a result of performing processing on all the first sound frames in this way, a modified sound signal that is a set of modified sound frames is stored in the modified sound signal storage unit 62. In the present embodiment, the disturbing sound is embedded from the beginning to the end of the first acoustic signal. However, it is also possible to set an embedding position and embed only in that range.

(2.2.3. Third embodiment)
Next, a second acoustic signal embedding device for an acoustic signal according to a third embodiment of the present invention will be described. In the first embodiment, the original sound signal is upsampled to create a wideband sound signal, and the second sound signal is embedded in the wideband sound signal. In the third embodiment, upsampling is performed. First, the process of embedding the second acoustic signal directly into the original acoustic signal is performed. Since there are many processes similar to those of the first embodiment, description will be made with reference to the flowchart of FIG. In the present embodiment, the first sound signal sampled at the sampling frequency Fs = 44.1 kHz is used. In the present embodiment, the frequency Ft = Fs / 4, which is the center of folding, is used.

  In the present embodiment, the acoustic frame reading unit 10 directly reads the acoustic signal stored in the acoustic signal storage unit 61 without using the upsampling unit 8. The acoustic frame reading means 10 reads a predetermined number N of samples as one first acoustic frame from each of the left and right channels of the stereo acoustic signal stored in the acoustic signal storage unit 61 (step S2). Similarly, the acoustic frame reading means 10 reads a predetermined number N of samples as one second acoustic frame from each of the left and right channels of the stereo second acoustic signal stored in the acoustic signal storage unit 61 (step S3). ).

  The number N of samples of one sound frame read by the sound frame reading means 10 can be set as appropriate. However, when the sampling frequency is 44.1 kHz, it is understood that the damage to the original sound can be reduced most when the number is about 4096 samples. This setting value will be described below. Therefore, the acoustic frame reading means 10 sequentially reads 4096 samples for the left channel and the right channel as acoustic frames from the acoustic signal.

  Also in this embodiment, as in the first embodiment, the odd-numbered acoustic frames and the even-numbered acoustic frames are set by overlapping a predetermined number of samples (2048 in this embodiment). Therefore, if the odd-numbered acoustic frames are A1, A2, A3... From the top, and the even-numbered acoustic frames are B1, B2, B3... From the top, A1 is samples 1 to 4096, A2 is samples 4097 to 8192, A3. Is samples 8193-12288, B1 is samples 2049-6144, B2 is samples 6145-10240, and B3 is samples 10241-14336.

  Next, the frequency conversion means 20 performs frequency conversion on the first sound frame to obtain a complex spectrum of the first sound frame (step S4). Similarly, the frequency conversion means 20 performs frequency conversion on the second sound frame to obtain a complex spectrum of the second sound frame (step S5). In steps S4 and S5, specifically, frequency conversion is performed using a window function. As the frequency transform, various known methods such as Fourier transform, wavelet transform and the like can be used, but it is necessary to be a method capable of obtaining a complex spectrum. In the present embodiment, a case where Fourier transform is used will be described as an example. In steps S4 and S5, as in the first embodiment, frequency conversion is performed using the window function W (i) shown in the above [Equation 2].

  When the frequency conversion unit 20 performs Fourier transform on the first sound frame, for the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1), The window function W (i) is used to perform the processing according to the above [Equation 4] and the conversion corresponding to the real part Al (j), the imaginary part Bl (j), and the right channel of the conversion data corresponding to the left channel. Real part Ar (j) and imaginary part Br (j) of data are obtained. As a result of the processing according to the above [Equation 4], when the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by about 10.8 Hz. The signal spectrum of the original sound signal obtained in step S4 is as shown in FIG.

  When the frequency conversion means 20 performs the Fourier transform on the second sound frame, for the left channel signal X2l (i) and the right channel signal X2r (i) (i = 0,..., M−1) Using the window function W (i), the processing according to the above [Formula 5] is performed, and the conversion corresponding to the real part A2l (j), the imaginary part Bl (j), and the right channel of the conversion data corresponding to the left channel is performed. Real part A2r (j) and imaginary part Br (j) of data are obtained. As a result of the processing according to the above [Equation 4], when the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by about 10.8 Hz. The signal spectrum of the second acoustic signal obtained in step S5 is as shown in FIG.

  In steps S4 and S5, the processes according to the above [Equation 4] and [Equation 5] are executed, whereby complex spectra corresponding to the first and second acoustic frames are obtained. Subsequently, the frequency component modifying means 30 performs a process of adding a low frequency cancellation component to the high frequency using the first spectrum obtained from the first acoustic frame (step S6). Specifically, the first spectrum component is inverted between positive and negative, and a process of turning around the frequency Fs / 2 is performed.

  Once the low frequency cancellation component addition processing in step S6 has been performed, the frequency component modification means 30 then combines the first spectrum and the second spectrum after the low frequency cancellation component addition processing (step S7). . The second spectrum is added while being folded around the frequency Fs / 4. As a result, the first spectrum (synthetic spectrum) is in a state as shown in FIG. In the first spectrum after synthesis (the spectrum of the modified acoustic signal), a signal component obtained by inverting the amplitude of the signal component existing in the original first spectrum and a signal component of the second spectrum in a range larger than the frequency Fs / 4, It exists in a folded state.

  In the example of FIG. 10C, two types of spectral components are superimposed and displayed at frequencies Fs / 4 to Fs / 2, but in reality they are added as complex vectors. As shown in FIG. 10C, in the synthesized spectrum, the amplitude inversion component of the first acoustic signal and the component derived from the second acoustic signal are converted into noise at frequencies Fs / 4 to Fs / 2. Since the low-frequency component of one acoustic signal is masked, even if the modified acoustic signal is reproduced, the amplitude inversion component of the first acoustic signal and the component of the second acoustic signal are not heard by humans.

  The processing performed by the frequency component modifying means 30 in the above steps S6 and S7 can be summarized as the following [Equation 11].

[Formula 11]
Al ′ (j) ← −Al (N / 2−j) · α + A2l (N / 2−j) · β
Bl ′ (j) ← −Bl (N / 2−j) · α + B2l (N / 2−j) · β
Ar ′ (j) ← −Ar (N / 2−j) · α + A2r (N / 2−j) · β
Br ′ (j) ← −Br (N / 2−j) · α + B2r (N / 2−j) · β

  The frequency component modifying means 30 performs the processing according to the above [Formula 11] on the frequency components Al (j), B1 (j), A2l (j), B2l (j), j = N / 4 + 1,. .., for each j of N / 2-1. When the number of samples of one acoustic frame is N = 4096 and the sampling frequency Fs = 44.1 kHz (= 2Fs), j = N / 4 corresponds to the frequency Fs / 4, and j = N / 2 is the frequency Fs / 2. Corresponding to Therefore, in [Formula 11], j = 1025,..., 2048 is a processing target, and the frequency component of about 11.1 kHz to about 22.1 kHz (approximately the upper limit of the human audible range) is changed.

  In the above [Expression 11], α and β are scaling real values set in the range of 0.0 <α and β ≦ 1.0. In the present embodiment, α = β = 1.0 is set. The first term (−Al (N / 2−j) · α, etc.) on the right side of each formula shown in [Formula 11] is a low-frequency canceling component, and the second term (A2l (N / 2-j), β, etc. are components obtained by folding the second spectrum in the frequency direction.

  When the correspondence relationship between the flowchart of FIG. 5 and the above [Equation 11] is shown, the process of adding the low frequency cancellation component to the high frequency in Step S6 corresponds to the process of adding the first term on the right side of the above [Equation 11]. Then, the synthesis process of the first spectrum and the second spectrum in step S7 corresponds to the process of adding the second term on the right side of the above [Formula 11].

  After the frequency component modifying unit 30 finishes the synthesis process in step S7, the frequency inverse transform unit 40 performs a process of performing frequency inverse transform on the modified first spectrum to obtain a modified acoustic frame (step S8). . Naturally, the inverse frequency conversion needs to correspond to the method executed by the frequency conversion means 20. In the present embodiment, since the frequency transform unit 20 performs the Fourier transform, the frequency inverse transform unit 40 executes the Fourier inverse transform.

  More specifically, as in the first embodiment, the frequency inverse transform unit 40 includes the real part Al ′ (j) and the imaginary part Bl ′ (j) of the left channel of the first spectrum obtained by the frequency component modification unit 30. ), Using the real part Ar ′ (j) and the imaginary part Br ′ (j) of the right channel, the process according to the above [Equation 7] is performed to calculate Xl ′ (i) and Xr ′ (i). . The frequency components that have not been modified by the frequency component modifying means 30 are Al ′ (j), Bl ′ (j), Ar ′ (j), and Br ′ (j), which are the original frequency components, Al. (J), Bl (j), Ar (j), and Br (j) are used.

  Similar to the first embodiment, after the inverse frequency conversion in step S8, the modified acoustic frames obtained can be sequentially output to obtain the modified acoustic signal. When the processing up to step S8 is duplicated as described above, the anti-aliasing component is preliminarily placed in the A / D converter or sampler in which the cancellation component of the original sound signal and the component of the second acoustic signal are used in the duplication. Usually, it is attenuated by the processing. Also in the present embodiment, the following steps S9 and S10 are performed in order to exert a desired effect even when the LPF process is performed at the time of replication.

  FIG. 11 is a diagram illustrating the processes in steps S9 and S10. 11, FIG. 11 (a) is the same as FIG. 10 (c), and shows the signal spectrum of the modified acoustic signal obtained as a result of the processing up to step S8. FIG. 11B shows a signal spectrum of a signal obtained by thinning out the modified acoustic signal of FIG. FIG. 11C shows a signal spectrum of a signal obtained by upsampling and expanding the sampling frequency. FIG.11 (d) shows the signal spectrum of the correction | amendment modified acoustic signal obtained by synthesize | combining the signal of FIG.11 (c) with the modified acoustic signal of Fig.11 (a).

  Thinning / enlarging processing is performed on the modified acoustic signal shown in FIG. 11A obtained by the processing up to step S8 (step S9). Specifically, first, as a thinning-out process, the modified acoustic signal having the sampling frequency Fs is down-sampled at the sampling frequency Fs / 2. An image of the spectrum of the signal down-sampled at the sampling frequency Fs / 2 is as shown in FIG.

  Further, an expansion process in the frequency direction of the thinned signal is performed. Specifically, the sampling frequency Fs / 2 signal subjected to the thinning process is up-sampled at the sampling frequency Fs. After up-sampling at the sampling frequency Fs, an image of the spectrum of a signal from which components exceeding the frequency Fs / 4 are removed is as shown in FIG.

  Subsequently, the signal after the thinning / enlarging process is weighted and synthesized with the modified acoustic signal (step S10). That is, a signal as shown in FIG. 11C is added to the modified acoustic signal shown in FIG. As a result, a corrected modified acoustic signal as shown in FIG. 11D is obtained. Specifically, the signal component after the thinning / enlarging process is added to the component of the modified acoustic signal having a frequency of Fs / 4 or less.

  The processing performed by the modified acoustic signal correcting unit 55 in steps S9 and S10 can be summarized as the above [Equation 8] as in the first embodiment.

  The modified sound frame output unit 50 sequentially outputs the modified sound frames after correction obtained by the processing of the modified sound signal correction unit 55 to the output file. The process shown in the flowchart of FIG. 5 is executed for all the first sound frames of the first sound signal. When the number of the first sound frames is larger than the number of the second sound frames, the process returns to the first second sound frame and is repeated. As a result of performing processing on all the first sound frames in this way, a modified sound signal that is a set of modified sound frames is stored in the modified sound signal storage unit 62. In the present embodiment, the second acoustic signal is embedded from the beginning to the end of the first acoustic signal. However, it is also possible to set an embedding position and embed only in that range.

  FIG. 12 is a diagram illustrating a concept in a case where the modified acoustic signal obtained by the processing up to step S10 of the third embodiment is replicated by a realistic device including an LPF circuit. In FIG. 12, FIG. 12 (a) shows the signal spectrum of the modified acoustic signal after correction obtained as a result of the processing up to step S10. FIG. 12B shows a filter gain by LPF (Low Pass Filter) processing. FIG. 12C shows a signal spectrum of a signal obtained by performing LPF processing on the modified acoustic signal. FIG. 12D shows a signal spectrum of a duplicated sound signal obtained by resampling the modified sound signal after LPF processing.

  A signal obtained by performing the LPF process on the modified acoustic signal is as shown in FIG. As can be seen by comparing FIG. 12 (a) and FIG. 12 (c), the portion corresponding to the original acoustic signal having a frequency of Fs / 4 or lower and the thinned / enlarged component remain as they are, and the cancellation exceeds the frequency Fs / 4. The component corresponding to the second component and the second acoustic signal is attenuated by the influence of the LPF.

  Further, when the signal shown in FIG. 12C is resampled at the sampling frequency Fs / 2, a duplicate acoustic signal shown in FIG. 12D is obtained. In the duplicated sound signal, aliasing occurs, the attenuated canceling component, and the component corresponding to the second sound signal are combined in-phase with the component that has been decimated and expanded to a frequency below the human audible frequency Fs / 4. Is done. Thereby, the attenuated canceling component and the component corresponding to the second acoustic signal are emphasized and can be heard by humans. If the second acoustic signal such as a warning message is heard over the music, the music cannot be appreciated in its original state. For this reason, deterrence against replication works.

  Further, even when the sampling frequency for duplicating the modified acoustic signal is Fs ′ that is slightly smaller than Fs / 2 and larger than Fs / 4, aliasing occurs in the same manner, and signal components of frequencies Fs ′ / 2 to Fs ′. Is inverted and superimposed on a frequency band of Fs ′ / 2 or less. That is, the left end of the right half of FIG. 12A is shifted to the left, and the signal component having a narrower bandwidth than the right half of FIG. 12A is shifted to the left of the right end of the left half of FIG. Wrapped to When this duplicated sound signal is reproduced, the original sound signal is canceled halfway and it cannot be expected that a sound equivalent to the sound recorded in the second sound signal will be emitted, but the original sound signal is accompanied by distortion and considerable noise. The sound based on the signal cancellation component and the second acoustic signal is clearly heard by the human ear. Even in this case, the purpose of not being able to obtain reproduced sound that can be enjoyed for viewing can be achieved, so that duplication can be suppressed. Furthermore, when the sampling frequency when replicating the modified sound signal is smaller than Fs / 4, significant aliasing occurs, and the main signal component of the original sound signal itself is destroyed to obtain a reproduced sound that can be appreciated. Can not be. Conversely, even when the sampling frequency when replicating the modified acoustic signal is Fs ′ that is slightly larger than Fs / 2 and smaller than 3Fs / 4, aliasing occurs in the same manner, and signals with frequencies Fs ′ / 2 to Fs ′. The component is inverted and superimposed on the frequency band below Fs ′ / 2. That is, the right end of the right half of FIG. 12A is shifted to the right, and the signal component having a wider bandwidth than the right half of FIG. 12A is shifted to the right of the right half of the left half of FIG. Wrapped to When this duplicated sound signal is reproduced, the original sound signal is canceled halfway and it cannot be expected that a sound equivalent to the sound recorded in the second sound signal will be emitted, but the original sound signal is accompanied by distortion and considerable noise. The sound based on the signal cancellation component and the second acoustic signal is clearly heard by the human ear. Even in this case, the purpose of not being able to obtain reproduced sound that can be enjoyed for viewing can be achieved, so that duplication can be suppressed. However, if it is greater than 3Fs / 4, aliasing hardly occurs, the quality of the original sound signal and the second sound signal is maintained, the second sound signal is not made audible, and duplication cannot be suppressed. From the above, aliasing occurs when the sampling frequency when replicating the modified acoustic signal is smaller than 3Fs / 4. It becomes possible to suppress duplication.

(2.2.4. Fourth embodiment)
Next, an interference sound embedding device for an acoustic signal according to a fourth embodiment of the present invention will be described. In the second embodiment, the original sound signal is upsampled to create a wideband sound signal, and white noise is embedded. However, in the fourth embodiment, upsampling is not performed and the original sound signal is embedded. A process of directly embedding white noise is performed. Since there are many processes similar to those of the second embodiment, description will be made with reference to the flowchart of FIG. In the present embodiment, the first sound signal sampled at the sampling frequency Fs = 44.1 kHz is used. In the present embodiment, the frequency Ft = Fs / 4, which is the center of folding, is used.

  In the present embodiment, the acoustic frame reading unit 10 directly reads the acoustic signal stored in the acoustic signal storage unit 61 without using the upsampling unit 8. The acoustic frame reading means 10 reads a predetermined number N of samples as one first acoustic frame from each of the left and right channels of the stereo acoustic signal stored in the acoustic signal storage unit 61 (step S12).

  Although the number N of samples of one first sound frame read by the sound frame reading means 10 can be set as appropriate, in the present embodiment, a case where N = 4096 will be described below. Therefore, the acoustic frame reading means 10 sequentially reads 4096 samples for each of the left channel and the right channel from the broadband acoustic signal as the first acoustic frame.

  Also in this embodiment, as in the first to third embodiments, odd-numbered acoustic frames and even-numbered acoustic frames are set by overlapping a predetermined number of samples (N / 2 = 2048 in this embodiment). Is done. Therefore, if the odd-numbered acoustic frames are A1, A2, A3... From the top, and the even-numbered acoustic frames are B1, B2, B3... From the top, A1 is samples 1 to 4096, A2 is samples 4097 to 8192, A3. Is samples 8193-12288, B1 is samples 2049-6144, B2 is samples 6145-10240, and B3 is samples 10241-14336. Note that the number of samples to be overlapped can be set as appropriate.

  Next, the frequency conversion means 20 performs frequency conversion on the first sound frame to obtain a complex spectrum of the first sound frame (step S13). In step S13, specifically, frequency conversion is performed using a window function. As the frequency conversion, various known methods for obtaining a complex spectrum, such as Fourier transform, wavelet transform, and the like, can be used. In the present embodiment, as in the first embodiment, a case where Fourier transform is used will be described as an example.

  In the present embodiment, the Fourier transform for each first acoustic frame is performed on the product multiplied by the Hanning window function W (i) defined by the above [Equation 1].

  When the frequency conversion unit 20 performs Fourier transform on the first sound frame, for the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1), The window function W (i) is used to perform the processing according to the above [Equation 4] and the conversion corresponding to the real part Al (j), the imaginary part Bl (j), and the right channel of the conversion data corresponding to the left channel. Real part Ar (j) and imaginary part Br (j) of data are obtained. As a result of the above processing of [Equation 4], when the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by about 10.8 Hz.

  By executing the processing according to [Formula 4] in step S13, a complex spectrum corresponding to each first acoustic frame is obtained. Subsequently, as in the second embodiment, the frequency component modifying unit 30 performs a process of adding a low frequency cancellation component to the high frequency using the first spectrum obtained from the first acoustic frame (step S14). ).

  After performing the low frequency cancellation component addition processing in step S14, next, the frequency component modification means 30 adds white noise to the first spectrum after the low frequency cancellation component addition processing, as in the second embodiment. Processing to add is performed (step S15). Specifically, a process according to the following [Equation 12] is executed, and a process of adding white noise, which is a noise equivalently including each frequency component, to a predetermined high range is performed.

[Formula 12]
When Al (j) ≧ 0 Al ′ (j) ← −Al (N / 4−j) · α + γ
When Al (j) <0 Al ′ (j) ← −Al (N / 4−j) · α−γ
When B1 (j) ≧ 0 B1 ′ (j) ← −B1 (N / 4−j) · α + γ
When B1 (j) <0 B1 ′ (j) ← −B1 (N / 4−j) · α−γ
When Ar (j) ≧ 0 Ar ′ (j) ← −Ar (N / 4−j) · α + γ
When Ar (j) <0 Ar ′ (j) ← −Ar (N / 4−j) · α−γ
When Br (j) ≧ 0 Br ′ (j) ← −Br (N / 4−j) · α + γ
When Br (j) <0 Br ′ (j) ← −Br (N / 4−j) · α−γ

  The frequency component modifying means 30 performs the processing according to the above [Equation 12] on the frequency components Al (j), Bl (j), Ar (j), Br (j), j = N / 4 + 1,. .., for each j of N / 2-1. When the number of samples of one acoustic frame is N = 4096 and the sampling frequency Fs is 44.1 kHz, j = N / 4 corresponds to the frequency Fs / 4, and j = N / 2 corresponds to the frequency Fs / 2. Note that no modification is made to each component of j = 1,... N / 4. In the above [Equation 12], α is a scaling real value set in a range of 0.0 <α ≦ 1.0. In this embodiment, α = 1.0 is set. In addition, γ is a real value of the signal level. In this embodiment, Al (j), Bl (j), Ar (j), and Br (j) are defined in a 16-bit range (−32768 to +32767). Γ = 1024.0 is set. The first term (−Al (N / 4-j) · α, etc.) on the right side of each equation shown in [Equation 12] is a low-frequency canceling component, and the second term γ on the right side of each equation is white noise. It is.

  As shown in the above [Equation 12], a predetermined intensity γ is given as white noise so as to increase the absolute values of the real and imaginary components of the complex number. The white noise addition processing in step S15 is performed only for odd-numbered sound frames, and not performed for even-numbered sound frames, as in the second embodiment. By turning on / off white noise at odd and even numbers, the ratio of modification to the original sound signal is halved and noise reproduction sound is reduced by adding low frequency alternating fluctuations of On / Off. Can be clear.

  In the present embodiment, the frequency component modification process in steps S14 and S15 is performed on the odd-numbered sound frames and not performed on the even-numbered sound frames. There is no need to perform modification processing on the above, and a method with a high effect of interfering sound may be appropriately selected.

  After the frequency component modification unit 30 performs the white noise addition in step S15 and completes the modification process, the frequency inverse transform unit 40 performs a process of performing frequency inverse transform on the modified spectrum to obtain a modified acoustic frame. (Step S16). Naturally, the inverse frequency conversion needs to correspond to the method executed by the frequency conversion means 20. In the present embodiment, since the frequency transform unit 20 performs the Fourier transform, the frequency inverse transform unit 40 performs the Fourier inverse transform by the process according to the above [Equation 7].

  After the frequency inversion in step S16, the modified acoustic frames obtained can be sequentially output to obtain the modified acoustic signal. However, as described above, there is no copy prevention effect at the stage of processing up to step S16. Therefore, also in the present embodiment, as in the first to third embodiments, in order to reproduce a desired disturbing sound even when the LPF process is performed at the time of copying, the following steps S17 and S18 are performed. Perform the process.

  First, thinning / enlarging processing is performed on the modified acoustic signal obtained by the processing up to step S16 (step S17). Specifically, first, as a thinning-out process, the modified acoustic signal having the sampling frequency Fs is down-sampled at the sampling frequency Fs / 2, as in the third embodiment.

  Further, an expansion process in the frequency direction of the thinned signal is performed. Specifically, the sampling frequency Fs / 2 signal subjected to the thinning process is up-sampled at the sampling frequency Fs.

  Subsequently, the signal after the thinning / enlarging process is weighted and synthesized with the modified acoustic signal (step S18). Specifically, the signal component after the thinning / enlarging process is added to the component of the modified acoustic signal having a frequency of Fs / 4 or less.

  The processing performed by the modified acoustic signal correcting unit 55 in steps S17 and S18 can be summarized as the above [Equation 8] as in the first embodiment.

  The modified sound frame output unit 50 sequentially outputs the modified sound frames after correction obtained by the processing of the modified sound signal correction unit 55 to the output file. The process shown in the flowchart of FIG. 9 is executed for all the first sound frames of the first sound signal. As a result of performing processing on all the first sound frames in this way, a modified sound signal that is a set of modified sound frames is stored in the modified sound signal storage unit 62. In the present embodiment, the disturbing sound is embedded from the beginning to the end of the first acoustic signal. However, it is also possible to set an embedding position and embed only in that range.

(3. Modified examples)
As mentioned above, although it limited about the suitable embodiment of the present invention, the present invention is not limited to the above-mentioned embodiment, and various modifications are possible. For example, in the above-described embodiment, in the first acoustic signal, the signal component having the frequency Ft or higher is removed, and the signal component having the frequency Ft or lower is folded around the frequency Ft in the high frequency direction. The value of is not limited to the above embodiment, and various values can be set.

  In the above embodiment, a case where a two-channel stereo sound signal generally distributed as a product is used has been described as an example, but each of five channels except for LFC is also applied to a 5.1 channel surround sound signal. A similar process may be applied to the acoustic signal, and conversely, a mono-channel monaural acoustic signal may be used. In this case, the process performed on either the L channel or the R channel may be executed.

  In the second and fourth embodiments, the low-frequency cancellation component and white noise addition processing in steps S14 and S15 are performed on the odd-numbered acoustic frame or the even-numbered acoustic frame. The additional processing may be executed with the above frame interval, or may be executed for all sound frames.

1 ... CPU
2 ... RAM
3 ... Storage device 4 ... Key input I / F
5. Data input / output I / F
6 ... Display output I / F
8 ... Upsampling means 10 ... Acoustic frame reading means 20 ... Frequency converting means 30 ... Frequency component modifying means 40 ... Inverse frequency converting means 45 ... Modified acoustic frame correcting means 50 ... -Modified sound frame output means 60 ... storage means 61 ... sound signal storage section 62 ... modified sound signal storage section

Claims (9)

A device for embedding a second acoustic signal composed of a time-series sample sequence in an inaudible state with respect to an original acoustic signal of a sampling frequency Fs composed of a time-series sample sequence,
An acoustic frame reading means for reading a first acoustic frame composed of a predetermined number of samples from the original acoustic signal and for reading a second acoustic frame composed of a predetermined number of samples from the second acoustic signal;
Frequency conversion is performed on the first sound frame to obtain a first spectrum that is a complex frequency component, and frequency conversion is performed on the second sound frame to obtain a second spectrum that is a complex frequency component. Means,
Among the signal components of the first spectrum, the signal components of the frequency Ft or higher are removed, the signal components of the frequency Ft or lower are folded back in the high frequency direction around the frequency Ft, and the folded frequency Ft is used. while multiplying the inverting and predetermined coefficient value the sign for the first spectral signal component of the range of frequencies 2 Ft, by adding the signal component of the corresponding frequency before Symbol first spectrum, said first First frequency component modifying means for modifying the frequency component of the spectrum;
Among the signal components of the second spectrum, the signal components of the frequency Ft or higher are removed, the signal components of the frequency Ft or lower are folded around the frequency Ft in the high frequency direction, and the folded frequency Ft is used. By multiplying the signal component of the second spectrum in the range of frequency 2Ft by a predetermined coefficient value and adding to the signal component of the corresponding frequency of the modified first spectrum, the frequency component of the first spectrum A second frequency component modifying means for modifying
Frequency inverse transforming means for performing frequency inverse transform on the first spectrum in which the frequency component is modified to generate a modified acoustic frame;
Modified acoustic frame output means for sequentially outputting the generated modified acoustic frames;
A device for embedding a second acoustic signal with respect to the acoustic signal. An apparatus that embeds an interfering sound in an inaudible state with respect to an original sound signal having a sampling frequency Fs composed of a time-series sample sequence,
Sound frame reading means for reading a first sound frame composed of a predetermined number of samples from the original sound signal;
Frequency conversion means for performing frequency conversion on the first acoustic frame to obtain a first spectrum which is a complex frequency component;
Among the signal components of the first spectrum, the signal components of the frequency Ft or higher are removed, the signal components of the frequency Ft or lower are folded back in the high frequency direction around the frequency Ft, and the folded frequency Ft is used. while multiplying the inverting and predetermined coefficient value the sign for the first spectral signal component of the range of frequencies 2 Ft, by adding the signal component of the corresponding frequency before Symbol first spectrum, said first First frequency component modifying means for modifying the frequency component of the spectrum;
The frequency component of the first spectrum is modified so that the absolute value of the signal increases by a predetermined value with respect to the signal component of the frequency Ft or more and the frequency 2 Ft or less among the signal components of the first spectrum. Two frequency component modification means;
Frequency inverse transforming means for performing frequency inverse transform on the first spectrum in which the frequency component is modified to generate a modified acoustic frame;
Modified acoustic frame output means for sequentially outputting the generated modified acoustic frames;
A device for embedding a disturbing sound for an acoustic signal, comprising: In claim 2,
The first frequency component modifying means and the second frequency component modifying means perform processing on acoustic frames at predetermined intervals among the acoustic frames read by the acoustic frame reading means. A device for embedding interfering sound with respect to a characteristic acoustic signal. In any one of Claims 1-3,
Upsampling means for performing upsampling on the original sound signal at a sampling frequency Fs ′ (Fs ′> Fs) to create a wideband original sound signal;
Ft = Fs / 2,
The acoustic frame reading unit, said frequency converting means, before Symbol first frequency component changing means, the second frequency component changing means, said inverse frequency transform means, said modified acoustic frame output means, the original after the upsampling A device for embedding a second acoustic signal for an acoustic signal or an embedding device for an interfering sound for an acoustic signal, characterized by performing processing on the acoustic signal. In any one of Claims 1-3,
The second acoustic signal embedding device for an acoustic signal or the disturbing sound embedding device for an acoustic signal, wherein Ft = Fs / 4. In any one of Claims 1-5,
One sample value of the odd-numbered and even-numbered sample values of the modified acoustic frame is multiplied by a sample value immediately before the sample value by a weight G that is a real value of 1 or less, and the sample value is ( 1-G) is further provided with modified acoustic frame correction means for correcting the modified acoustic frame by converting it into a sum with a value multiplied by 1).
The modified acoustic frame output means sequentially outputs the corrected modified acoustic frames, and a second acoustic signal embedding device for an acoustic signal or an interfering sound embedding device for an acoustic signal. In claim 6,
The modified acoustic frame correcting means calculates the weight G based on an average value of samples included in the modified acoustic frame, or a second acoustic signal embedding device or an acoustic signal for an acoustic signal Interference sound embedding device against. In any one of Claims 1-7,
The frequency converting means uses a weight W (i) (0 ≦ W (i) ≦ 1) at a sample position i (0 ≦ i ≦ N−1) as a window width N samples, and W (i) = 0.5. A device for embedding a second acoustic signal in an acoustic signal or embedding an interfering sound in an acoustic signal, wherein frequency conversion is performed using a Hanning window function defined by −0.5 cos (2πi / N) apparatus.   A program for causing a computer to function as the second acoustic signal embedding device for the acoustic signal or the disturbing sound embedding device for the acoustic signal according to any one of claims 1 to 8.

Images