JP2013257378A - Device of embedding disturbing sound to acoustic signal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device of embedding disturbance sound to acoustic signals, capable of preventing duplication in an original state when duplicate acoustic signals after recording are played back, when recording is performed without a permission by a sound recorder.SOLUTION: A first acoustic frame of N samples is read from broadband acoustic signals, a second acoustic frame of M samples is read from second acoustic signals (S2, 3), and a first spectrum and a second spectrum are obtained (S4, 5). A frequency Ft and higher frequencies are removed in signal components of the first spectrum, the frequencies lower than Ft are positive/negative inverted, folded and added to a high band (S6). The frequency Ft and the higher frequencies of the second spectrum are removed, and the frequencies lower than Ft that are folded are combined with the signal components of corresponding frequency of the first spectrum (S7). After frequency-time conversion (S8), to modified acoustic frames for two channels, correction is performed to change values of two samples in the total of four samples which are adjacent two samples for the two channels so that the adjacent two samples match when adding corresponding samples of both channels (S9).

Description

  The present invention relates to the field of packaged music for viewing in consumer and business applications using CDs, DVDs, BDs, and the like, and the network music distribution field distributed by music content providers for commercial purposes, and in particular, prevents copying of music content. Related to technology.

  Conventionally, various techniques have been developed to prevent duplication of music content. For example, in a method called DRM (Digital Rights Management: Patent Document 1), duplication of music content is prevented by encrypting digital music content. Specifically, recording from a commercial DVD or BD media to another DVD-R or other recording media using a digital cable connection with a recorder device, or inserting it into a PC drive and ripping it as a video file on the PC Making it impossible. However, DRM has been released for music CDs due to various circumstances, so that digital pirated music files created by ripping from rental CDs can be uploaded to the site and distributed. It is a problem. In this DRM method, copying of digital content can be prevented, but copying of analog content cannot be prevented. In other words, it is possible to copy the image by photographing the display screen being reproduced with a video camera or recording the reproduction signal with a line or a microphone from the speaker output. At present, the biggest problem is that a small video camera is brought into a movie theater or hall, and the soundtrack flowing from the speaker is recorded along with the image projected on the screen, and the DVD is created inexhaustibly and shipped as a product (pirated version). There are examples where illegally copied video files are distributed via video sites and P2P. Since consumer video cameras in recent years are HDTV compatible and can be recorded with an image quality comparable to that of a BD, it is easy to ensure commercial quality for a DVD that is duplicated using it as a master.

  Fortunately, in Japan, due to the effect of the “Act on the Prevention of Camera Voyeurism” enforced since 2007, the screen voyeurism incident ended the arrest of current offenders in Aichi Prefecture in 2009, but a new problem emerged doing. Since voyeurism continues to take place overseas, voyeur movies have landed in Japan via the Internet, but cannot be sold in Japan as they are. In other words, in order to distribute it in Japan, it is necessary to put a caption supervision into the voyeur movie or to dubb it into Japanese, but the latter is overwhelmingly in demand. Therefore, only the audio is concealed in a movie theater where a Japanese dubbed version of a foreign film imported from overseas is being screened, and it is dubbed into an overseas voyeur video and shipped. An example of a foreign film that was exported overseas and illegally copied to a Japanese film was illegally copied, and only the voice was hidden in Japan. Also found. It is almost impossible to find a crime in a dark movie theater because it does not require a tripod like a screen voyeur and can be hidden in a pocket. In other words, voyeurism for video has been eradicated by legal development, but there is no edge for acoustic voyeurism. At present, the demand for audio content protection, including the illegal ripping problem of rental music CDs, is video content. It is much larger than protection.

Special table 2003-517767 gazette WO2011 / 002059

  As a technique for preventing duplication of analog contents, a technique for adding an invisible copy disturbing signal to a video signal has been proposed as a countermeasure against the aforementioned pirated DVD manufacturing (see Patent Document 2). In the method disclosed in Patent Document 2, since infrared rays are used as a copy interference signal, it is invisible to humans, but is reflected in a video camera, and illegal copying can be suppressed. However, copy disturbing signals cannot be embedded in the content itself, and only work with screens and displays equipped with special modules that emit copy disturbing signals. Professional video cameras and cameras equipped with infrared cut filters When using, there is a problem that reflection of a copy disturbing signal can be avoided. Also, they are completely vulnerable to illegal copying of video soundtracks.

  On the other hand, recording equipment has also evolved, and recent voice recorders are portable and can be recorded in stereo. However, if illegal recording is to be performed, recording is concealed and it is difficult to set up two microphones, so recording is performed in a state close to a monaural signal in which a plurality of channel signals are mixed. In Japan, illegal recordings are rarely made for the purpose of creating pirated versions (commercial), and illegal recordings are voluntarily distributed on online video sites. (Because it is not for profit, high-risk screen sneak shots are no longer performed.) Therefore, there is no problem with the image quality and sound quality, and if the sound can be heard even if the music is somewhat distorted, Of course, mono is sufficient.

  Therefore, the present invention is embedded in the original sound signal inaudibly when the digitally reproduced duplicate sound signal is reproduced even when the sound is recorded without permission by a recording device such as a voice recorder. It is an object of the present invention to provide a device for embedding a disturbing sound with respect to an acoustic signal that can prevent duplication in an original state by reproducing the disturbing sound that is audible.

  In order to solve the above-described problem, in the first aspect of the present invention, a second acoustic signal composed of a time-series sample sequence is compared to an original acoustic signal composed of a time-series sample sequence of at least two channels. In an inaudible state as an interfering sound, wherein two channels are selected from the original sound signal, a sound frame composed of a predetermined number of samples is read for each channel, and the first sound frame is read. A sound frame reading means for reading a sound frame composed of a predetermined number of samples from two sound signals as a second sound frame, and performing time-frequency conversion for each channel on the first sound frame, and a complex frequency component A certain first spectrum is obtained, and time-frequency conversion is performed on the second sound frame to obtain a complex frequency component. A time-frequency converting means for obtaining two spectra, and removing signal components above the frequency Ft from the signal components of the second spectrum, and converting signal components below the frequency Ft around the frequency Ft to high frequencies A signal of a frequency corresponding to the first spectrum is obtained by folding in a frequency direction and multiplying a predetermined range of signal components included in the frequency 2Ft from the folded frequency Ft by a predetermined coefficient value (β, βl, βr). A frequency component modifying unit that modifies the frequency component of the first spectrum for each channel by adding to the component; and frequency-time conversion is performed on the first spectrum in which the frequency component is modified, and the modified acoustic frame The frequency-time conversion means for generating the two-channel and the two modified channels of the generated sound frames are paired with each other. Modification that corrects the value of 2 samples in each set by making a total of 4 samples of 2 adjacent samples for 2 channels so that 2 adjacent samples match when added for each sample. There is provided an apparatus for embedding an interference sound in an acoustic signal, comprising acoustic frame correcting means and modified acoustic frame output means for sequentially outputting modified acoustic frames corrected by the modified acoustic frame correcting means.

  According to the first aspect of the present invention, the signal component having the frequency Ft or higher is removed from the signal component of the second spectrum obtained from the second acoustic signal to be embedded, and the frequency Ft or lower is centered on the frequency Ft. The signal component is folded back in the high frequency direction, and the second spectrum signal component in the range from the folded frequency Ft to the frequency 2Ft (twice Ft) is multiplied by a predetermined coefficient value (β, βl, βr). However, when the modified acoustic frame is obtained by adding to the signal component of the corresponding frequency of the first spectrum for each channel after modification, both channels are added to the modified acoustic frame for each corresponding sample. Since the correction was made to change the value of two samples of one channel so as to match two adjacent samples, the monophonic signal via a single microphone was used. By recording, both channels are mixed, and the mixed recording signal is equivalent to a state in which the mixed recording signal is artificially decimated by half and resampled, and is affected by LPF (Low Pass Filter) during duplication. Aliasing occurs almost without being received, and when a duplicate sound signal recorded and reproduced in monaural by a recording device in a stereo or surround sound space of two or more channels is reproduced, the disturbing sound can be made audible. Here, the meaning of “pseudo ½ resampling” does not physically change the sampling frequency and the total number of samples, but the fact that the adjacent samples are the same means that if the Fourier transform is performed, the sampling frequency and This is equivalent to changing the total number of samples to ½. As a result, when the duplicate sound signal is reproduced, music or a message different from the sound recorded in the original sound signal is emitted as an interfering sound simultaneously with the sound recorded in the original sound signal. Can be suppressed.

  Further, in the second aspect of the present invention, with respect to an original sound signal composed of a time-series sample sequence of at least two channels, the noise composed of the time-series sample sequence is inaudible as an interference sound. An apparatus for embedding, wherein two channels are selected from the original sound signal, and a sound frame reading means for reading a sound frame composed of a predetermined number of samples for each channel as a first sound frame; A time-frequency conversion means for performing time-frequency conversion for each channel to obtain a first spectrum which is a complex frequency component, and a predetermined range included in the frequency 2Ft from the frequency Ft among the signal components of the first spectrum. The frequency component of the first spectrum is modified so that the absolute value of the signal increases by a predetermined value (γ, γl, γr) with respect to the signal component. Frequency component modifying means for performing frequency-time conversion on the first spectrum in which the frequency component is modified to generate a modified acoustic frame, and modification for the generated two channels When adding both channels for each sample corresponding to the sound frame, a total of four samples of two adjacent samples for two channels are set as one set so that the two adjacent samples match. Interference with an acoustic signal, comprising: modified acoustic frame correcting means for performing correction to change the value of two samples; and modified acoustic frame output means for sequentially outputting modified acoustic frames corrected by the modified acoustic frame correcting means A sound embedding device is provided.

  According to the second aspect of the present invention, among the signal components of the first spectrum obtained from the original sound signal, the absolute value of the signal with respect to the signal component of the frequency Ft or more and the frequency 2Ft (twice Ft) or less. Is modified by a predetermined value (γ, γl, γr) to obtain a modified acoustic frame by modifying the frequency components of the first spectrum, and for each sample corresponding to both channels for the modified acoustic frame. When adding, correction was made to change the value of two samples in one channel so that two adjacent samples in one channel would match, so monaural recording via a single microphone should be performed As a result, both channels are mixed, and the mixed recording signal is equivalent to a state in which the mixed recording signal is artificially decimated by half and resampled. Aliasing occurs almost without being affected, and when reproducing a duplicate sound signal recorded in monaural by a recording device in a stereo or surround sound space of two or more channels, the disturbance sound can be made audible. . As a result, when the duplicated sound signal is reproduced, noise that does not make sense as a disturbing sound is emitted simultaneously with the sound recorded in the original sound signal, so that the meaning of duplication is lost and duplication can be suppressed.

  According to a third aspect of the present invention, in the apparatus for embedding a disturbing sound with respect to the acoustic signal according to the second aspect of the present invention, the frequency component modifying means is arranged at predetermined intervals among the acoustic frames read by the acoustic frame reading means. The processing is performed on the acoustic frame.

  According to the third aspect of the present invention, since the sound frames at predetermined intervals are modified, the amount of calculation processing for modifying the original sound signal can be reduced, and the noise reproduced sound is clearly displayed. Can be. In particular, if the modification is performed only for every two acoustic frames, that is, only the odd-numbered acoustic frame or the even-numbered acoustic frame as the predetermined interval, the efficiency of embedding the interference sound with respect to the calculation processing amount is good.

  According to a fourth aspect of the present invention, in the apparatus for embedding interference sound in the acoustic signal according to any one of the first to third aspects of the present invention, the frequency component modifying means is the first obtained by the time-frequency conversion means. Among signal components of the spectrum, the signal components higher than the frequency Ft are removed, the signal components lower than the frequency Ft are folded around the frequency Ft in the high-frequency direction, and the frequency Ft to the frequency 2Ft is folded. The frequency component of the first spectrum is added to the signal component of the corresponding frequency of the first spectrum while inverting the sign of the signal component of the first spectrum of the range and multiplying by a predetermined coefficient value. It is characterized in that a process for modifying each channel is performed in advance.

  According to the fourth aspect of the present invention, the signal component having the frequency Ft or higher is removed from the signal component of the first spectrum of the original sound signal, and the signal component having the frequency Ft or lower is increased with the frequency Ft as the center. The first spectral signal component in the range from the folded frequency Ft to the frequency 2Ft (twice Ft) is inverted while the sign is inverted and multiplied by a predetermined coefficient value. Since the frequency component of the first spectrum is modified by adding to the signal component of the corresponding frequency of the spectrum, the modification of the frequency component shown in the first and second modes is performed. When a reproduced sound signal equivalent to a duplicated signal obtained by re-sampling the pseudo-normally reproduced sound at the sampling frequency 2Ft by performing the monaural recording shown in the aspect 2 is reproduced. Is a signal component obtained by inverting the phase of the first spectrum in the range from the frequency Ft to the frequency 2Ft in the low-frequency direction below the frequency Ft centered on the frequency Ft and lower than the frequency Ft of the original sound signal. Since the signal component is superimposed (or subtracted), the sound based on the original sound signal is canceled by a ratio of a predetermined coefficient value.

  According to a fifth aspect of the present invention, in the apparatus for embedding a disturbing sound with respect to the acoustic signal according to any one of the first to fourth aspects of the present invention, the original acoustic signal is sampled at a sampling frequency Fs, Upsampling means for performing upsampling on the acoustic signal at a sampling frequency Fus (Fus> Fs) to create a broadband acoustic signal that is a broadband original acoustic signal, and Ft = Fs / 2, The time-frequency conversion means, the frequency component modification means, the frequency-time conversion means, the modified sound frame correction means, and the modified sound frame output means perform processing on the wideband sound signal. .

  According to the fifth aspect of the present invention, the frequency Ft that is the lower limit of the signal component removal of the original sound signal and is the center of folding is set to ½ of the sampling frequency Fs of the original sound signal, and the number of samples of the modified sound signal Is modified so as to be larger than the number of samples of the original sound signal, so that the lower limit of the low-frequency component and the disturbing sound component of the embedded phase-inverted original sound signal is Fs / 2, and the frequency Fs / 2 Is set near the upper limit of the human audible range, the low-frequency component of the original acoustic signal whose phase is inverted and the signal component of the disturbing sound are completely inaudible, and these signal components are not heard at all. . However, when the modified acoustic signal is converted into an analog system or digitally and the monaural recording shown in the first and second modes is performed, it is artificially sampled again at the sampling frequency Fs and duplicated. When the reproduced acoustic signal is reproduced, aliasing occurs, and the signal component of the disturbing sound having the frequency Fs / 2 or higher is folded back to the audible band having the frequency Fs / 2 or lower, and based on the original sound signal. The sound is suppressed, and instead, a sound based on the second acoustic signal or a disturbing sound is output in an audible state. For this reason, it is possible to prevent only the sound recorded in the original sound signal from being duplicated at the same sampling frequency as that of the original sound signal.

  According to a sixth aspect of the present invention, in the apparatus for embedding a disturbing sound with respect to the acoustic signal according to any one of the first to fourth aspects of the present invention, the original sound signal is sampled at a sampling frequency Fs, and the Ft = Fs / 4.

  According to the sixth aspect of the present invention, since the frequency Ft that is the center of the folding is set to ¼ of the sampling frequency Fs of the original sound signal, the low-frequency component of the phase-inverted original sound signal and the second sound The upper limit of the signal component of the signal or interfering sound is Fs / 2, and the low-frequency component of the original acoustic signal and the second acoustic signal or the second acoustic signal, the phase of which is inverted to the signal component of the original acoustic signal from the frequency Fs / 4 to Fs / 2. The signal component of the interfering sound is superimposed, and by reproducing the monaural recording shown in the first and second modes, the duplicated sound signal is reproduced when it is duplicated at the sampling frequency Fs / 2. Then, aliasing occurs, and the low frequency component of the original acoustic signal whose phase is inverted from the frequency Fs / 4 to Fs / 2 and the second acoustic signal or the noise signal component are centered on the frequency Fs / 4. / 4 folded back below, the sound based on the original acoustic signal is suppressed, the sound or noise based on the second audio signal is output at audible state instead. For this reason, it is possible to prevent only the sound recorded in the original sound signal from being duplicated at a sampling frequency that is half the sampling frequency of the original sound signal.

  According to a seventh aspect of the present invention, in the apparatus for embedding a disturbing sound for an acoustic signal according to the sixth aspect of the present invention, predetermined coefficient values (β, βl, βr) to be multiplied in the frequency component modifying means or predetermined values to be increased ( [gamma], [gamma] l, [gamma] r) are changed based on the average value of the signal components in the predetermined range included in the frequency 2Ft from the frequency Ft among the signal components of the first spectrum obtained by the time-frequency conversion means. It is characterized by being.

  According to the seventh aspect of the present invention, when modifying the original sound signal, the degree to which the original sound signal acts as the component of the second sound signal or the disturbing sound is determined from the frequency Ft to the frequency 2Ft of the original sound signal. Since it is changed based on the average intensity value of the signal components in the range, it is increased / decreased in conjunction with the signal intensity of the original acoustic signal, and the deterioration of the quality of the original acoustic signal due to embedding can be suppressed.

  According to an eighth aspect of the present invention, in the apparatus for embedding a disturbing sound with respect to the acoustic signal according to any one of the first to seventh aspects of the present invention, the time-frequency conversion means uses a sample position i ( Hanning window function in which weight W (i) (0 ≦ W (i) ≦ 1) in 0 ≦ i ≦ N−1) is defined by W (i) = 0.5−0.5 cos (2πi / N) Is used to perform time-frequency conversion.

  According to the eighth aspect of the present invention, since the Hanning window function is used when performing the frequency analysis, the time-frequency is reduced while reducing the distortion applied to the original acoustic signal without generating a pseudo-harmonic component. Conversion can be performed.

  According to the present invention, even when recording is performed without permission by a recording device such as a voice recorder, aliasing occurs when the digitally reproduced duplicate audio signal after recording is reproduced, and the original audio signal In this way, it is possible to prevent duplication in the original state by reproducing the non-audible disturbing sound as audible.

It is a figure which shows the principle of aliasing. It is a figure which shows the basic principle of one Embodiment of this invention. It is a hardware block diagram of the embedding device of the disturbance sound with respect to the acoustic signal which concerns on this invention. It is a functional block diagram which shows the structure of the embedding device of the disturbance sound with respect to the acoustic signal which concerns on this invention. It is a flowchart which shows the processing operation of the embedding device of the disturbance sound with respect to the acoustic signal which concerns on 1st (3rd) embodiment. It is a figure which shows the concept in case the modified acoustic signal obtained by the process to step S8 is replicated with the realistic apparatus provided with the LPF circuit. It is a figure which shows the correction process of step S9 in 1st Embodiment. It is a figure which shows the concept in case the modified acoustic signal obtained by the process to step S9 is replicated with the realistic apparatus provided with the LPF circuit. It is a flowchart which shows the processing operation of the embedding device of the disturbance sound with respect to the acoustic signal which concerns on 2nd (4th) embodiment. It is a figure which shows the concept of the embedding process of the disturbance sound in 3rd Embodiment. It is a figure which shows the correction process of step S9 in 3rd Embodiment. It is a figure which shows the concept in case the modified acoustic signal obtained by the process to step S9 of 3rd Embodiment is replicated with the realistic apparatus provided with the LPF circuit. It is a figure which shows the relationship between a speaker and a process target channel in the case of applying to a 5.1 channel surround sound signal.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
<1. Basic concept of the present invention>
First, the basic concept of the present invention will be described. In the present invention, by utilizing aliasing distortion called aliasing, the signal component obtained by inverting the phase of the original sound signal for suppressing the level of the original sound signal and a new warning message for the original sound signal to be prevented from being duplicated. Modification is performed to embed the second acoustic signal or noise such as white noise as an interfering sound, and processing for obtaining the modified acoustic signal is performed. The signal component of the embedded original phase-inverted acoustic signal and the second acoustic signal or noise signal component cannot be heard during normal playback of the modified acoustic signal, but the reproduced modified acoustic signal is recorded. Re-sampled and duplicated audio signal (replication method is digital copy or analog copy, but it is limited to analog copy to prevent duplication with the same sampling frequency as the original audio signal) Is reproduced, the second acoustic signal such as a warning message and noise such as white noise are reproduced in an audible state instead. However, since the phenomenon of aliasing is well known in the industry, anti-aliasing processing, specifically, an LPF circuit is installed in front of the sampler so that high-frequency noise is not mixed during recording. It is common. As a result, the high frequency component folded back by aliasing is attenuated by the LPF circuit, so that it is substantially equivalent to almost no folding, and the above-mentioned object cannot be realized. The present invention also proposes a solution for this. Specifically, the original sound signal is played in stereo or surround (movies, theaters, etc.) with two or more channels, but those who make illegal recordings are monaural using a portable voice recorder or smartphone that can be hidden in a pocket. Paying attention to the point of recording, when the signals of multiple channels are mixed, the adjacent samples are controlled to have the same value and recorded at a sampling frequency of ½, and the voice recorder The aliasing can be reliably generated without depending on the sampling frequency. If the person who performs illegal recording records at a sampling frequency equal to or higher than that of the original sound signal, the sampling is performed again at the sampling frequency of ½ without being affected by the LPF circuit even through the LPF circuit. The above-mentioned purpose can be realized.

  FIG. 1 is a diagram illustrating the principle of aliasing. 1A shows the signal spectrum of the original acoustic signal, and FIG. 1B shows the signal spectrum of the sampling data obtained by sampling the original acoustic signal of FIG. 1A at the sampling frequency Fs. When the original sound signal is sampled at the sampling frequency Fs, a high-frequency signal component exceeding the Nyquist frequency Fs / 2 is folded back to the low frequency side centering on Fs / 2, and the low frequency signal of Fs / 2 or less. It is misanalyzed as a component and superimposed. This is aliasing. In the example of FIG. 1, by sampling at the sampling frequency Fs, the signal components of the frequencies Fs / 2 to Fs are inverted and superimposed in the frequency direction, and the signal components of the frequencies Fs to 3Fs / 2 are twice in the frequency direction. Inverted and superimposed in the original orientation. In the example of FIG. 1B, three types of spectral components are displayed in an overlapping manner, but in reality they are added as complex vectors.

  FIG. 2 is a diagram showing the principle of one embodiment of the present invention to be described later. 2A is a signal spectrum of an original acoustic signal sampled at the sampling frequency Fs, and FIG. 2B is a second acoustic signal that is a second acoustic signal sampled at the sampling frequency Fs. 2 (c) shows the signal spectrum of the modified acoustic signal obtained as a result of the embedding process according to the present invention, and FIG. 2 (d) shows the duplicate obtained by re-sampling the modified acoustic signal by the duplicating means. It is a signal spectrum of an acoustic signal.

  Both the original acoustic signal shown in FIG. 2 (a) and the second acoustic signal shown in FIG. 2 (b) are subjected to high-cut filter processing before sampling to remove components in the higher frequency range than the sampling frequency Fs / 2. Use something that has not occurred. Here, the same sampling frequency is expanded to 2 Fs (twice Fs) in advance with respect to the original sound signal shown in FIG. 1A, and the different sound signal shown in FIG. 2 is synthesized with the original acoustic signal whose sampling frequency is expanded, and the modified acoustic signal shown in FIG. 2C is obtained. At this time, in the present invention, the component of Fs / 2 or less of the original sound signal is folded in the frequency direction around Fs / 2 while being amplitude-inverted and synthesized.

  When the modified acoustic signal shown in FIG. 2C obtained in this way is reproduced, by setting Fs / 2 to a value higher than the human audible range (for example, 22.05 kHz), Fs / Sounds based on signal components derived from different acoustic signals higher than 2 are hardly audible to humans. When the modified acoustic signal shown in FIG. 2C is duplicated and resampled at the sampling frequency Fs, the duplicate acoustic signal shown in FIG. 2D is obtained. The duplicated sound signal shown in FIG. 2 (d) is aliased, and the original sound signal component is inverted in amplitude and turned back to an original sound signal of Fs / 2 or less, which is the human audible range, and the original sound signal is canceled out. Instead, the signal component derived from the second acoustic signal is folded back below Fs / 2, which is the human audible range. As a result, since the signal component derived from the second sound signal is mainly recorded in the duplicate sound signal, only the second sound signal such as a warning message recorded as the second sound signal can be heard by humans. . For this reason, duplication of the original acoustic signal can be prevented. However, normally, when re-sampling at the sampling frequency Fs, the signal component of Fs / 2 or higher is cut by the LPF of the sampler preprocessing, so that the phenomenon of folding back to Fs / 2 or lower does not occur, and the desired second acoustic signal Is not heard. In view of this, the present application proposes a technique that is not affected by LPF by controlling so that the effect equivalent to resampling at a desired frequency can be obtained in a pseudo manner by monaural recording.

<2.1. Configuration of embedded device>
Next, a description will be given of an interference sound embedding device for an acoustic signal according to the present invention. FIG. 3 is a hardware configuration diagram of an apparatus for embedding a disturbance sound for an acoustic signal according to the present invention. A device for embedding an interference sound with respect to an acoustic signal can be realized by a general-purpose computer. As shown in FIG. 3, a CPU 1 (CPU: Central Processing Unit) and a RAM 2 (RAM: Random Access Memory) which is a main memory of the computer. ), A large-capacity storage device 3 (for example, a hard disk, a flash memory, etc.) for storing programs and data executed by the CPU 1, a key input I / F (interface) 4 such as a keyboard and a mouse, and an external device A data input / output I / F (interface) 5 for data communication with (a data storage medium or the like), and a display output I / F (interface) 6 for sending information to a display device (display); They are connected to each other via a bus.

  FIG. 4 is a functional block diagram showing the configuration of the interference sound embedding device for the acoustic signal according to the present invention. In FIG. 4, 8 is an upsampling means, 10 is an acoustic frame reading means, 20 is a time-frequency conversion means, 30 is a frequency component modification means, 40 is a frequency-time conversion means, 45 is a modified acoustic frame correction means, and 50 is The modified acoustic frame output means, 60 is a storage means, 61 is an acoustic signal storage section, and 62 is a modified acoustic signal storage section. Note that the apparatus shown in FIG. 4 can cope with a stereo sound signal of two or more channels such as a 5.1 channel surround sound signal, but in the present embodiment, a process is performed on a stereo sound signal of two channels. The case of performing will be described.

  The up-sampling means 8 samples the first acoustic signal, which is the original acoustic signal stored in the acoustic signal storage unit 61, at a frequency higher than the sampling frequency of the first acoustic signal to obtain a broadband acoustic signal. It has a function. The acoustic frame reading means 10 has a function of reading a predetermined number of samples as one acoustic frame from each channel of the broadband acoustic signal. When embedding the second acoustic signal, which is the second acoustic signal, the acoustic frame reading means 10 also has a function of reading a predetermined number of samples as one acoustic frame from each channel of the second acoustic signal. The time-frequency conversion means 20 performs time-frequency conversion on the sound frame (hereinafter referred to as the first sound frame) read from the wide-band sound signal by the sound frame reading means 10 by Fourier transform or the like to generate a complex spectrum in the frequency dimension. It has a function to generate. In the case of embedding the second acoustic signal, the time-frequency conversion means 20 also uses an acoustic frame reading means 10 to read an acoustic frame (hereinafter referred to as a second acoustic frame) from the second acoustic signal by Fourier transformation or the like. It also has the function of generating a complex spectrum in the frequency dimension by conversion. The frequency component modifying means 30 has a function of modifying the spectrum obtained from the first acoustic frame. When embedding the second acoustic signal, the frequency component modifying means 30 has a function of modifying the spectrum obtained from the first acoustic frame while referring to the spectrum obtained from the second acoustic frame. The frequency-time conversion means 40 has a function of generating a time-ordered modified acoustic frame by performing frequency-time conversion on a plurality of complex spectra including a modified spectrum set. The modified sound frame correcting unit 45 has a function of correcting the modified sound frame obtained by the frequency-time converting unit 40. The modified acoustic frame output means 50 has a function of sequentially outputting the modified modified acoustic frames.

  The storage unit 60 includes an acoustic signal storage unit 61 that stores the first acoustic signal including the second acoustic signal when the second acoustic signal is embedded, and a modified acoustic signal storage unit 62 that stores the modified acoustic signal. In addition, various other information necessary for processing is stored.

  Each component shown in FIG. 4 is actually realized by installing a dedicated program in hardware such as a computer and its peripheral devices as shown in FIG. That is, the computer executes the contents of each means according to a dedicated program.

  In the storage device 3 of FIG. 3, a dedicated program for operating the CPU 1 and causing the computer to function as a device for embedding a disturbance sound for an acoustic signal is installed. By executing this dedicated program, the CPU 1 performs upsampling means 8, sound frame reading means 10, time-frequency conversion means 20, frequency component modification means 30, frequency-time conversion means 40, and modified sound frame correction means 45. Thus, the function as the modified sound frame output means 50 is realized. In addition to realizing the function as the storage unit 60, the storage device 3 stores various data necessary for processing.

<2.2. Processing operation of embedded device>
<2.2.1. First Embodiment>
The processing operation of the interference sound embedding device for an acoustic signal according to the present invention will be described below. FIG. 5 is a flowchart showing the processing operation of the interference sound embedding device for the acoustic signal according to the first embodiment of the present invention. As the first acoustic signal and the second acoustic signal, those sampled at 96 kHz, 48 kHz (for video / broadcasting business such as movies) or 44.1 kHz (for consumer use such as CD) as the sampling frequency Fs can be used. In the embodiment, one sampled at 48 kHz is used. In the present embodiment, the frequency Ft = Fs / 2, which is the center of folding, is used.

  First, the upsampling means 8 performs sampling by raising the sampling frequency higher than the original sampling frequency for the left and right channels of the stereo first acoustic signal stored in the acoustic signal storage unit 61, that is, Upsampling is performed (step S1). Although how much the sampling frequency is increased can be set as appropriate, in the present embodiment, the sampling frequency of 48 kHz of the original first acoustic signal is up-sampled to the sampling frequency of 96 kHz. Specifically, the up-sampling means 8 performs processing according to the following [Formula 1] on the first acoustic signals xl and xr of the total number of samples S of each channel stored in the acoustic signal storage unit 61. Execute.

[Formula 1]
xl ′ (s) = xl (s × 48/96)
xr ′ (s) = xr (s × 48/96)

  In the above [Expression 1], s = 0,..., S′−1, and S ′ = S × 96/48. As a result of executing the processing according to the above [Equation 1], the acoustic signal with the number of samples S for each channel is up-sampled into a broadband acoustic signal with the number of samples S 'for each channel. For samples that do not exist in the original acoustic signals xl and xr, interpolation is performed using the values of neighboring samples.

  The acoustic frame reading means 10 reads a predetermined number N of samples as one acoustic frame from each of the left and right channels of the upsampled stereo broadband acoustic signal (step S2). Similarly, the acoustic frame reading means 10 reads a predetermined number M of samples as one second acoustic frame from the left and right channels of the stereo second acoustic signal stored in the acoustic signal storage unit 61 (step S3). ).

  The number of samples N of one first sound frame and the number of samples M of a second sound frame read by the sound frame reading means 10 can be set as appropriate, but in this embodiment, hereinafter, N = 8192, M = 4096. The case will be described. Therefore, the sound frame reading means 10 sequentially reads 8192 samples for each of the left channel and the right channel from the wideband sound signal as the first sound frame, and sequentially stores 4096 samples for the left channel and the right channel from the second sound signal. It will be read as 2 sound frames.

  In the present embodiment, both the first and second sound frames have a predetermined number of odd-numbered sound frames and even-numbered sound frames (N / 2 = 4096 and M / 2 = 2048 in the present embodiment). Duplicate samples are set. Therefore, in the case of the second sound frame, if the odd-numbered sound frames are A1, A2, A3... From the top, and the even-numbered sound frames are B1, B2, B3. Are samples 4097 to 8192, A3 is samples 8193 to 12288, B1 is samples 2049 to 6144, B2 is samples 6145 to 10240, and B3 is samples 10241 to 14336. Note that the number of samples to be overlapped can be set as appropriate.

  Next, the time-frequency conversion means 20 performs time-frequency conversion on the first sound frame to obtain a complex spectrum of the first sound frame (step S4). Similarly, the time-frequency conversion means 20 performs time-frequency conversion on the second sound frame to obtain a complex spectrum of the second sound frame (step S5). In steps S4 and S5, specifically, time-frequency conversion is performed using a window function. As the time-frequency conversion, various known methods such as Fourier transform, wavelet transform, and the like can be used, but the method needs to obtain a complex spectrum. In the present embodiment, a case where Fourier transform is used will be described as an example.

  In general, when Fourier transform is performed on a predetermined signal, it is necessary to divide the signal into predetermined lengths. In this case, if Fourier transform is performed on the divided signal as it is in a rectangular window, the division is performed. A pseudo-harmonic component based on the boundary is generated. Therefore, in general, when Fourier transform is performed, the value of the window boundary is attenuated and changed using a window function called a Hanning window, and then the Fourier transform is performed on the changed value.

  Also in this embodiment, Hanning window functions W (i) and W2 (i) are used. The Hanning window functions W (i) and W2 (i) are set to take a maximum value 1 at the position of the predetermined sample number i in the center and take a minimum value 0 at the positions of the sample numbers i near both ends. . In this embodiment, the Fourier transform for each first acoustic frame is performed on the product multiplied by the Hanning window function W (i) defined by the following [Equation 2]. Further, the Fourier transform for each second acoustic frame is performed on a product multiplied by the Hanning window function W2 (i) defined by the following [Equation 3].

[Formula 2]
W (i) = 0.5−0.5 cos (2πi / N)

[Formula 3]
W2 (i) = 0.5−0.5 cos (2πi / M)

  In the present embodiment, the odd-numbered sound frame and the even-numbered sound frame are read in duplicate by predetermined samples for both the first sound frame and the second sound frame. Therefore, when the frequency component is modified and then restored to the state of the acoustic signal, when an odd-numbered acoustic frame multiplied by the window function and an overlapping sample of the even-numbered acoustic frame multiplied by the window function are added Should return to almost the original value. For this reason, when the window functions W (i) and W2 (i) are added in the overlapping portion between the odd-numbered acoustic frames and the even-numbered acoustic frames, it is necessary to define the whole section fixed value 1.

  When the time-frequency conversion means 20 performs a Fourier transform on the first sound frame, the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1). Then, the window function W (i) is used to perform the processing according to the following [Equation 4], and the real part Al (j), the imaginary part Bl (j) of the conversion data corresponding to the left channel is applied to the right channel. The real part Ar (j) and imaginary part Br (j) of the corresponding conversion data are obtained.

[Formula 4]
Al (j) = Σ _{i = 0,..., N-1} W (i) · Xl (i) · cos (2πij / N)
Bl (j) = Σi _{= 0,..., N-1} W (i) · Xl (i) · sin (2πij / N)
Ar (j) = Σi _{= 0,..., N-1} W (i) · Xr (i) · cos (2πij / N)
Br (j) = Σi _{= 0,..., N-1} W (i) · Xr (i) · sin (2πij / N)

  In the above [Expression 4], i is a serial number assigned to N samples in each acoustic frame, and takes an integer value of i = 0, 1, 2,... N−1. Further, j is a serial number assigned in order from the smallest value of the frequency value, and takes an integer value of j = 0, 1, 2,... N / 2-1 like i. When the sampling frequency is 96 kHz and N = 8192, if the value of j is different by one, the frequency will be different by about 11.7 Hz.

  When the time-frequency conversion means 20 performs a Fourier transform on the second acoustic frame, the left channel signal X2l (i) and the right channel signal X2r (i) (i = 0,..., M-1). Then, the window function W (i) is used to perform processing according to the following [Equation 5] to convert the real part A2l (j), imaginary part Bl (j), and right channel of the conversion data corresponding to the left channel. The real part A2r (j) and imaginary part Br (j) of the corresponding conversion data are obtained.

[Formula 5]
A2l (j) = Σi _{= 0,..., N-1} W (i) .X2l (i) .cos (2πij / M)
B2l (j) = Σi _{= 0,..., N-1} W (i) .X2l (i) .sin (2πij / M)
A2r (j) = Σi _{= 0,..., N-1} W (i) .X2r (i) .cos (2πij / M)
B2r (j) = Σi _{= 0,..., N-1} W (i) .X2r (i) .sin (2πij / M)

  In the above [Expression 4] and [Expression 5], i is a serial number assigned to M samples in each acoustic frame, and takes an integer value of i = 0, 1, 2,... M−1. Further, j is a serial number assigned to the frequency values in ascending order, and takes an integer value of j = 0, 1, 2,... M / 2-1 as with i. When the sampling frequency is 48 kHz and M = 4096, if the value of j is different by one, the frequency will be different by about 11.7 Hz.

  In steps S4 and S5, the processes according to the above [Equation 4] and [Equation 5] are executed, whereby complex spectra corresponding to the first and second acoustic frames are obtained. Subsequently, the frequency component modifying means 30 modifies the frequency component in steps S6 and S7. First, the frequency component modification means 30 performs a process of adding a low frequency cancellation component to a high frequency using the first spectrum obtained from the first acoustic frame (step S6). Specifically, the first spectrum component is inverted between positive and negative, and a process of folding around the frequency Ft (Ft = Fs / 2 in this embodiment) is performed.

  Once the low frequency cancellation component addition processing in step S6 has been performed, the frequency component modification means 30 then combines the first spectrum and the second spectrum after the low frequency cancellation component addition processing (step S7). . The second spectrum is added while folding around the frequency Fs / 2. As a result, the first spectrum (synthetic spectrum) is in a state as shown in FIG. In the first spectrum after synthesis (the spectrum of the modified acoustic signal), a signal component obtained by inverting the amplitude of the signal component existing in the original first spectrum and a signal component of the second spectrum in a range larger than the frequency Fs / 2, It exists in a folded state.

  In the example of FIG. 2C, two types of spectral components are superimposed and displayed at frequencies Fs / 2 to Fs, but in reality, they are added as complex vectors. As shown in FIG.2 (c), in the spectrum after a synthesis | combination, in frequency Fs / 2-Fs, the amplitude inversion component (low frequency cancellation component) of a 1st acoustic signal and the component derived from a 2nd acoustic signal are both. Since both are made noise and masked by the low frequency component of the first acoustic signal having high energy, even if the modified acoustic signal is reproduced, the amplitude inversion component of the first acoustic signal and the component of the second acoustic signal are , Not heard by humans.

  The processing performed by the frequency component modifying means 30 in steps S6 and S7 can be summarized as the following [Equation 6].

[Formula 6]
Al ′ (j) ← −Al (N / 2−j) × α + A2l (M−j) × β
Bl ′ (j) ← −Bl (N / 2−j) × α + B2l (M−j) × β
Ar ′ (j) ← −Ar (N / 2−j) × α + A2r (M−j) × β
Br ′ (j) ← −Br (N / 2−j) × α + B2r (M−j) × β

  The frequency component modifying means 30 performs the processing according to the above [Equation 6] on the frequency components Al (j), Bl (j), A2l (j), B2l (j), j = N / 4 + 1,. .., for each j of N / 2-1. When upsampling is performed with the number of samples of one acoustic frame N = 8192 and the sampling frequency Fus = 96 kHz (= 2Fs), j = N / 4 corresponds to the frequency Fs / 2 (= Fus / 4), and j = N / 2 corresponds to the frequency Fs (= Fus / 2). Therefore, in [Formula 6], j = 2049,..., 4096 is a processing target, and the frequency component of about 24.0 kHz (approximately the upper limit of the human audible range) to about 47.9 kHz is changed.

  In the above [Expression 6], α and β are real scaling values set in the range of 0.0 <α and β ≦ 1.0. In the present embodiment, α = β = 1.0 is set. The first term (−Al (N / 2−j) × α, etc.) on the right side of each formula shown in [Formula 6] is a low-frequency canceling component of the first spectrum, and the second term ( A2l (M−j) × β etc.) is a component obtained by folding the second spectrum in the frequency direction.

  When the correspondence between the flowchart of FIG. 5 and the above [Equation 6] is shown, the process of adding the low frequency cancellation component to the high frequency in Step S6 corresponds to the process of adding the first term on the right side of the above [Equation 6]. Then, the synthesis process of the first spectrum and the second spectrum in step S7 corresponds to the process of adding the second term on the right side of the above [Equation 6].

  After the frequency component modification unit 30 finishes the synthesis process in step S7, the frequency-time conversion unit 40 performs a process of obtaining a modified acoustic frame by performing frequency-time conversion on the first spectrum after modification (step). S8). Naturally, this frequency-time conversion needs to correspond to the technique executed by the time-frequency conversion means 20. In the present embodiment, since the time-frequency conversion means 20 performs the Fourier transform, the frequency-time conversion means 40 executes the inverse Fourier transform.

  Specifically, the frequency-time conversion means 40 has a real part Al ′ (j), an imaginary part Bl ′ (j) of the left channel of the first spectrum obtained by the frequency component modification means 30, and a real part of the right channel. Using Ar ′ (j) and imaginary part Br ′ (j), processing according to the following [Equation 7] is performed to calculate Xl ′ (i) and Xr ′ (i). The frequency components that have not been modified by the frequency component modifying means 30 are Al ′ (j), Bl ′ (j), Ar ′ (j), and Br ′ (j), which are the original frequency components, Al. (J), Bl (j), Ar (j), and Br (j) are used.

[Formula 7]
Xl' (i) = 1 / N × {Σ j Al' (j) × cos (2πij / N) -Σ j Bl' (j) × sin (2πij / N)} + Xlp (i + N / 2)
Xr' (i) = 1 / N × {Σ j Ar' (j) × cos (2πij / N) -Σ j Br' (j) × sin (2πij / N)} + Xrp (i + N / 2)

In the above [Expression 7], Σ _{j = 0,} ... _{, N−1} is shown as Σ _{j in} order to prevent the expression from becoming complicated. The terms “+ Xlp (i + N / 2)” in the first expression and “+ Xrp (i + N / 2)” in the second expression in the above [Expression 7] are the data Xlp (i) of the modified acoustic frame modified immediately before, When Xrp (i) exists, the addition is performed in consideration of the overlap of N / 2 samples on the time axis. By the above [Equation 7], each sample Xl ′ (i) of the left channel and each sample Xr ′ (i) of the right channel of the modified acoustic frame are obtained.

  After the frequency-time conversion in step S8, the modified acoustic frames obtained can be sequentially output to obtain a modified acoustic signal. Even in the processing up to step S8, in the experiment, as described with reference to FIG. 2, when the digital sound is duplicated by re-sampling, the original sound signal is suppressed, and the second sound is used instead. It can be confirmed that the component of the signal is audible. However, if resampling is performed digitally using a widely used acoustic signal processing tool, or if replication is performed using an A / D converter or sampler generally used for replication, The desired effect of can hardly be realized. The reason for this is that commercial acoustic signal processing tools, samplers, recording devices, etc. usually have an anti-aliasing LPF circuit in front, which is higher than half the sampling frequency specified at the time of sampling. This is because sampling is performed after the frequency component of the frequency band is attenuated, and measures are taken to prevent quality degradation due to the high frequency aliasing signal component. Therefore, the signal component for canceling the original sound signal embedded in the processing up to step 8 and the signal component for generating the second sound signal are usually deleted.

  FIG. 6 is a diagram showing a concept in a case where the modified acoustic signal obtained by the processing up to step S8 is duplicated by a realistic device including an LPF circuit. In FIG. 6, FIG. 6 (a) is the same as FIG. 2 (c), and shows the signal spectrum of the modified acoustic signal obtained as a result of the processing up to step S8. FIG. 6B shows a filter gain obtained by LPF (Low Pass Filter) processing. FIG. 6C shows a signal spectrum of a signal obtained by performing LPF processing on the modified acoustic signal. FIG. 6D shows a signal spectrum of a duplicated sound signal obtained by re-sampling the modified sound signal after LPF processing.

  As shown in FIG. 6B, the filter gain by the LPF processing is constant at an attenuation value of 0 [dB] up to the frequency Fs / 2, decreases sharply around the frequency Fs / 2, and reaches a maximum when it decreases to some extent. The attenuation value is constant at a value (such as −30 dB). Of course, when analog duplication processing is performed, even when digital duplication processing is performed, if the sampling frequency is changed, normal LPF processing is performed as sampling preprocessing. The modified acoustic signal shown in Fig. 6 is subjected to LPF processing having the characteristics shown in Fig. 6B. A signal obtained by performing the LPF process on the modified acoustic signal is as shown in FIG. As can be seen by comparing FIG. 6 (a) and FIG. 6 (c), the part corresponding to the original acoustic signal having the frequency Fs / 2 or less remains as it is, and it corresponds to the cancellation component and the interfering sound exceeding the frequency Fs / 2. The component to be attenuated is affected by the LPF.

  Furthermore, when the signal shown in FIG. 6C is resampled at the sampling frequency Fs, the duplicate acoustic signal shown in FIG. 6D is obtained. In the duplicated sound signal, aliasing occurs, and the attenuated canceling component and the component corresponding to the interfering sound are folded back to the frequency Fs / 2 or less which is a human audible range. As a result, as shown in FIG. 6D, in the range of the frequency Fs / 2 or less of the duplicated acoustic signal, the original acoustic signal component attenuated slightly due to the influence of the attenuated cancellation component. A component corresponding to the second acoustic signal is superimposed. The attenuated second acoustic signal is masked by the remaining original acoustic signal component that has been slightly attenuated, but is almost inaudible to humans, and the anti-duplication effect is lost. In the present embodiment, correction processing is performed in the following step S9 so that the second acoustic signal can be heard even when the LPF processing is performed at the time of replication.

  FIG. 7 is a diagram showing how the signal spectrum changes due to the correction processing in step S9. 7 (a) and 7 (b) are the same as FIG. 6 (a), and show the L channel and R channel signal spectra of the modified acoustic signal obtained as a result of the processing up to step S8, respectively. . In reality, in the case of a stereo sound signal, the signal spectrum of the L channel and the R channel is often not the same, but here, for convenience of explanation, the L channel and the R channel are indicated by the same signal spectrum. FIGS. 7C and 7D show the signal spectra of the L channel and the R channel after processing in step S9, respectively. In step S9, since the L channel signal is not modified, but only the R channel signal is modified, FIG. 7C is the same as FIG. 7A.

  Returning to the flowchart of FIG. For the modified acoustic signal shown in FIGS. 7A and 7B obtained by the processing up to step S8, arithmetic processing between channels and between samples is performed (step S9). Specifically, when the corresponding components are added to monaural in two channels, the values of two adjacent samples become the same value (average value of two samples in two channels × 2). The value of each sample in the R channel is corrected. In the present embodiment, processing according to the following [Equation 8] is executed to correct the value Xr ′ of each sample of the R channel.

[Formula 8]
Xr ′ (k × 2) ← {Xl ′ (k × 2) + Xl ′ (k × 2 + 1) + Xr ′ (k × 2) + Xr ′ (k × 2 + 1)} / 2−Xl ′ (k × 2)
Xr ′ (k × 2 + 1) ← {Xl ′ (k × 2) + Xl ′ (k × 2 + 1) + Xr ′ (k × 2) + Xr ′ (k × 2 + 1)} / 2−Xl ′ (k × 2 + 1)

  In the above [Equation 8], k is an integer taking a value in the range of k = 0,..., N / 2-1, and Xl ′ (k × 2) is an even-numbered sample value of the L channel signal. , Xl ′ (k × 2 + 1) is an odd-numbered sample value of the L channel signal, Xr ′ (k × 2) is an even-numbered sample value of the R channel signal, and Xr ′ (k × 2 + 1) is an odd number of the R channel signal. The second sample value is shown. In [Formula 8], the first term common to the first and second formulas is an average value × 2 of values of two adjacent samples (total of four samples) of two channels. The first equation shows processing in which the value obtained by subtracting the even-numbered sample value of the corresponding L channel from the average value of 4 samples × 2 is used as the even-numbered sample value of the new R channel. Further, the second equation shows processing in which a value obtained by subtracting the odd-numbered sample value of the corresponding L channel from the average value of 4 samples × 2 is used as the odd-numbered sample value of the new R channel. The results of such correction are shown in FIGS. 7 (c) and 7 (d). Since no correction is performed on the L channel side, FIG. 7C is exactly the same as FIG. 7D. The R channel side is greatly modified in the time dimension, but no significant change is observed in the frequency dimension shown in FIG. 7D compared to FIG. 7B. However, since the values of two adjacent samples are closer than before the modification, a slight fold occurs as shown by the dotted line on the left side of FIG.

  As described above, in step S9, when the corresponding components are added to monaural in two channels, the values of two adjacent samples are the same value (average value of two samples in two channels × 2). The value of each sample of the R channel is corrected so that In step S9, the degree of change can be increased or decreased so that the two adjacent samples are the same in the monaural state without being limited to the above [Formula 8]. The correction target is not limited to the R channel sample, and two samples on the L channel side may be corrected, or the odd first sample of the R channel and the even first sample of the L channel may be corrected. Anyway.

  The modified sound frame output means 50 sequentially outputs the modified sound frames after correction obtained by the process of the modified sound frame correction means 45 to the output file. The process shown in the flowchart of FIG. 5 is executed for all the first sound frames of the wideband sound signal. When the number of the first sound frames is larger than the number of the second sound frames, the process returns to the first second sound frame and is repeated. As a result of performing processing on all the first sound frames in this way, a modified sound signal that is a set of modified sound frames is stored in the modified sound signal storage unit 62. In the present embodiment, the second acoustic signal is embedded from the beginning to the end of the first acoustic signal. However, it is also possible to set an embedding position and embed only in that range.

  FIG. 8 is a diagram showing a concept in a case where the modified acoustic signal obtained by the processing up to step S9 is duplicated by a realistic device equipped with an LPF circuit. 8, FIGS. 8A and 8B are the same as FIGS. 7C and 7D, respectively, and the L channel of the modified acoustic signal after correction obtained as a result of the processing up to step S9, The signal spectrum of R channel is shown. FIG.8 (c) shows the signal spectrum of the duplication acoustic signal obtained by resampling after LPF processing.

  As shown in FIG. 8, the L channel modified acoustic signal and the R channel modified acoustic signal respectively emitted from the L channel speaker and the R channel speaker are mixed and recorded by a stereo or monaural microphone, and resampled at the sampling frequency Fs. A duplicate acoustic signal is obtained. (Even when recording with a stereo microphone, substantially the same left and right LR mixed signals are recorded, which is not much different from recording with a monaural microphone). Then, due to the mixing of the stereo sound signal, two adjacent samples have the same value, which is equivalent to a state where the sample is thinned out to ½, and aliasing occurs. As shown in FIG. 8C, the frequency Fs of the duplicate sound signal In the range of / 2 or less, the second acoustic signal as shown in FIG. 2B is restored, and a sound based on the second acoustic signal can be heard by humans. If the second acoustic signal such as a warning message is heard over the music, it is objectively found to be an illegal copy, and the music cannot be appreciated in its original state and cannot be commercialized. For this reason, deterrence against replication works.

<2.2.2. Second Embodiment>
Next, a description will be given of a disturbance sound embedding device for an acoustic signal according to a second embodiment of the present invention. FIG. 9 is a flowchart showing the processing operation of the interference sound embedding device for the acoustic signal according to the second embodiment. In the first embodiment, the second acoustic signal prepared in advance is used. However, in the second embodiment, processing for embedding artificially generated noise such as white noise without preparing a sound source in advance is performed. Do. Also in the present embodiment, as in the first embodiment, the first acoustic signal is sampled at 48 kHz as the sampling frequency Fs. In the present embodiment, the frequency Ft = Fs / 2, which is the center of folding, is used.

  First, as in step S1 in the first embodiment, the upsampling means 8 samples the left and right channels of the stereo first acoustic signal stored in the acoustic signal storage unit 61 more than the original sampling frequency. A sampling process is performed by increasing the frequency (step S11). Also in the present embodiment, as in the first embodiment, the processing according to the above [Equation 1] is executed, and the sampling frequency 48 kHz of the original first acoustic signal is upsampled to the sampling frequency 96 kHz.

  The acoustic frame reading means 10 reads a predetermined number N of samples as one acoustic frame from each of the left and right channels of the upsampled stereo broadband acoustic signal (step S12).

  Although the number N of samples of one first sound frame read by the sound frame reading means 10 can be set as appropriate, in the present embodiment, a case where N = 4096 will be described below. Therefore, the acoustic frame reading means 10 sequentially reads 4096 samples for each of the left channel and the right channel from the broadband acoustic signal as the first acoustic frame.

  Also in the present embodiment, as in the first embodiment, the odd-numbered sound frames and the even-numbered sound frames are set by overlapping a predetermined number of samples (N / 2 = 2048 in the present embodiment). Therefore, if the odd-numbered acoustic frames are A1, A2, A3... From the top, and the even-numbered acoustic frames are B1, B2, B3... From the top, A1 is samples 1 to 4096, A2 is samples 4097 to 8192, A3. Is samples 8193-12288, B1 is samples 2049-6144, B2 is samples 6145-10240, and B3 is samples 10241-14336. Note that the number of samples to be overlapped can be set as appropriate.

  Next, the time-frequency conversion means 20 performs time-frequency conversion on the first sound frame to obtain a complex spectrum of the first sound frame (step S13). In step S13, specifically, time-frequency conversion is performed using a window function. As the time-frequency conversion, various known methods for obtaining a complex spectrum, such as Fourier transform, wavelet transform, and the like, can be used. In the present embodiment, as in the first embodiment, a case where Fourier transform is used will be described as an example.

  In the present embodiment, the Fourier transform for each first acoustic frame is performed on the product multiplied by the Hanning window function W (i) defined by the above [Equation 2].

  When the time-frequency conversion means 20 performs a Fourier transform on the first sound frame, the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1). Then, using the window function W (i), the processing according to the above [Equation 4] is performed, and the real part Al (j), the imaginary part Bl (j), and the right channel of the conversion data corresponding to the left channel are handled. The real part Ar (j) and imaginary part Br (j) of the conversion data to be obtained are obtained. As a result of the above processing of [Equation 4], when the sampling frequency is 96 kHz and N = 4096, if the value of j is different by one, the frequency is different by about 23.4 Hz.

  By executing the processing according to [Formula 4] in step S13, a complex spectrum corresponding to each first acoustic frame is obtained. Subsequently, the frequency component modifying means 30 performs a process of adding a low frequency cancellation component to the high frequency using the first spectrum obtained from the first acoustic frame (step S14). Specifically, similarly to step S6 in the first embodiment, the first spectrum component is inverted in the positive and negative directions, and the process of folding around the frequency Fs / 2 is performed. The low frequency cancellation component addition processing in step S14 is performed only for odd-numbered sound frames, and is not performed for even-numbered sound frames. By turning On / Off the low-frequency canceling component at odd and even numbers, the ratio of modification to the original sound signal is halved, and noise is generated by adding low frequency alternating fluctuations of On / Off. The reproduced sound can be made clear.

  Once the low frequency cancellation component addition processing in step S14 has been performed, the frequency component modification means 30 then performs processing for adding white noise to the first spectrum after the low frequency cancellation component addition processing (step S15). ). White noise refers to noise in which energy is uniformly mixed in all frequency bands. Specifically, processing according to the following [Equation 9] is executed, and processing for adding white noise, which is noise that equally includes each frequency component, to a predetermined high frequency range is performed.

[Formula 9]
When Al (j) ≧ 0 Al ′ (j) ← −Al (N / 2−j) × α + γ
When Al (j) <0 Al ′ (j) ← −Al (N / 2−j) × α−γ
When B1 (j) ≧ 0 B1 ′ (j) ← −B1 (N / 2−j) × α + γ
When B1 (j) <0 B1 ′ (j) ← −B1 (N / 2−j) × α−γ
When Ar (j) ≧ 0 Ar ′ (j) ← −Ar (N / 2−j) × α + γ
When Ar (j) <0 Ar ′ (j) ← −Ar (N / 2−j) × α−γ
When Br (j) ≧ 0 Br ′ (j) ← −Br (N / 2−j) × α + γ
When Br (j) <0 Br ′ (j) ← −Br (N / 2−j) × α−γ

  The frequency component modifying means 30 performs the processing according to the above [Equation 9] on the frequency components Al (j), Bl (j), Ar (j), Br (j), j = 1,. , N / 2 for each j. When the number of samples of one acoustic frame is N = 4096 and the sampling frequency after upsampling is Fus = 96 kHz (= 2Fs), j = N / 4 corresponds to the frequency Fs / 2 (= Fus / 4), and j = N / 2 corresponds to the frequency Fs (= Fus / 2). It should be noted that the high frequency component exceeding j = N / 2 is not modified. In the above [Expression 9], α is a scaling real value set in a range of 0.0 <α ≦ 1.0. In this embodiment, α = 1.0 is set. In addition, γ is a real value of the signal level. In this embodiment, Al (j), Bl (j), Ar (j), and Br (j) are defined in a 16-bit range (−32768 to +32767). Γ = 10240.0 is set. The first term (−Al (N / 2−j) × α, etc.) on the right side of each equation shown in [Formula 9] is a low-frequency canceling component, and the second term γ on the right side of each equation is white noise. It is.

  As shown in the above [Equation 9], a predetermined intensity γ is given as white noise so as to increase the absolute values of the real and imaginary components of the complex number. The white noise addition processing in step S15 is performed only for odd-numbered sound frames, and is not performed for even-numbered sound frames. By turning on / off white noise at odd and even numbers, the ratio of modification to the original sound signal is halved and noise reproduction sound is reduced by adding low frequency alternating fluctuations of On / Off. Can be clear.

  In the present embodiment, the frequency component modification processing in steps S14 and S15 is performed on the odd-numbered sound frames and is not performed on the even-numbered sound frames, but is modified on every other sound frame. It is not always necessary to perform the processing. If the effect of the interfering sound is enhanced, the modification processing may be performed on every second or third sound frame, and conversely, it is performed on all sound frames. You may do it. In the above example, since the odd number and the even number are relative, the process may be performed on the even-numbered sound frame and may not be performed on the odd-numbered sound frame. .

  After the frequency component modification unit 30 performs the white noise addition in step S15 and completes the modification process, the frequency-time conversion unit 40 performs frequency-time conversion of the modified spectrum to obtain a modified acoustic frame. Is performed (step S16). Naturally, this frequency-time conversion needs to correspond to the technique executed by the time-frequency conversion means 20. In the present embodiment, since the time-frequency conversion means 20 performs the Fourier transform, the frequency-time conversion means 40 performs the inverse Fourier transform by the process according to the above [Equation 7].

  After the frequency-time conversion in step S16, the modified acoustic frames obtained can be sequentially output to obtain the modified acoustic signal. However, even if duplication is performed at this stage, as described above, the anti-aliasing process that precedes the A / D converter and sampler used during duplication cancels out and disturbs the embedded original sound signal. Usually, signal components that generate sound are attenuated. Therefore, in the present embodiment as well, in the same way as in the first embodiment, the following processing in step S17 is performed so that a desired disturbing sound can be heard even when the LPF processing is performed during duplication.

  That is, as in the first embodiment, arithmetic processing between channels and between samples is performed on the modified acoustic signal obtained by the processing up to step S16 (step S17). Specifically, as in the first embodiment, when both channels are added for each corresponding sample, the value of each sample of the R channel is corrected so that two adjacent samples are matched. In the present embodiment, as in the first embodiment, the process according to the above [Equation 8] is executed to correct the value Xr ′ of each sample of the R channel.

  The modified sound frame output means 50 sequentially outputs the modified sound frames after correction obtained by the process of the modified sound frame correction means 45 to the output file. The process shown in the flowchart of FIG. 9 is executed for all the first sound frames of the wideband sound signal. As a result of performing processing on all the first sound frames in this way, a modified sound signal that is a set of modified sound frames is stored in the modified sound signal storage unit 62. In the present embodiment, the disturbing sound is embedded from the beginning to the end of the first acoustic signal. However, it is also possible to set an embedding position and embed only in that range.

<2.2.3. Third Embodiment>
Next, an interference sound embedding apparatus for an acoustic signal according to a third embodiment of the present invention will be described. In the first embodiment, the original sound signal is upsampled to create a wideband sound signal, and the second sound signal is embedded in the wideband sound signal. In the third embodiment, upsampling is performed. First, the process of embedding the second acoustic signal directly into the original acoustic signal is performed. Since there are many processes similar to those of the first embodiment, description will be made with reference to the flowchart of FIG. In the present embodiment, the first sound signal sampled at the sampling frequency Fs = 44.1 kHz is used. In the present embodiment, the frequency Ft = Fs / 4, which is the center of folding, is used.

  In the present embodiment, the acoustic frame reading means 10 directly reads the original acoustic signal stored in the acoustic signal storage unit 61 without using the upsampling means 8. The acoustic frame reading means 10 reads a predetermined number N of samples as one first acoustic frame from each of the left and right channels of the stereo original acoustic signal stored in the acoustic signal storage unit 61 (step S2). Similarly, the acoustic frame reading means 10 reads a predetermined number N of samples as one second acoustic frame from each of the left and right channels of the stereo second acoustic signal stored in the acoustic signal storage unit 61 (step S3). ).

  The number N of samples of one sound frame read by the sound frame reading means 10 can be set as appropriate. However, when the sampling frequency is 44.1 kHz, it is understood that the damage to the original sound can be reduced most when the number is about 4096 samples. This setting value will be described below. Therefore, the acoustic frame reading means 10 sequentially reads 4096 samples for the left channel and the right channel as acoustic frames from the acoustic signal.

  Also in this embodiment, as in the first embodiment, the odd-numbered acoustic frames and the even-numbered acoustic frames are set by overlapping a predetermined number of samples (2048 in this embodiment). Therefore, if the odd-numbered acoustic frames are A1, A2, A3... From the top, and the even-numbered acoustic frames are B1, B2, B3... From the top, A1 is samples 1 to 4096, A2 is samples 4097 to 8192, A3. Is samples 8193-12288, B1 is samples 2049-6144, B2 is samples 6145-10240, and B3 is samples 10241-14336.

  Next, the time-frequency conversion means 20 performs time-frequency conversion on the first sound frame to obtain a complex spectrum of the first sound frame (step S4). Similarly, the time-frequency conversion means 20 performs time-frequency conversion on the second sound frame to obtain a complex spectrum of the second sound frame (step S5). In steps S4 and S5, specifically, time-frequency conversion is performed using a window function. As the time-frequency conversion, various known methods such as Fourier transform, wavelet transform, and the like can be used, but the method needs to obtain a complex spectrum. In the present embodiment, a case where Fourier transform is used will be described as an example. In steps S4 and S5, as in the first embodiment, time-frequency conversion is performed using the window function W (i) shown in the above [Equation 2].

  When the time-frequency conversion means 20 performs a Fourier transform on the first sound frame, the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1). Then, using the window function W (i), the processing according to the above [Equation 4] is performed, and the real part Al (j), the imaginary part Bl (j), and the right channel of the conversion data corresponding to the left channel are handled. The real part Ar (j) and imaginary part Br (j) of the conversion data to be obtained are obtained. As a result of the processing according to the above [Equation 4], when the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by about 10.8 Hz. The signal spectrum of the original sound signal obtained in step S4 is as shown in FIG.

  When the time-frequency conversion means 20 performs a Fourier transform on the second acoustic frame, the left channel signal X2l (i) and the right channel signal X2r (i) (i = 0,..., M-1). Then, using the window function W (i), the processing according to the above [Formula 5] is performed, and the real part A2l (j), the imaginary part Bl (j), and the right channel of the conversion data corresponding to the left channel are handled. The real part A2r (j) and imaginary part Br (j) of the conversion data to be obtained are obtained. As a result of the processing according to the above [Equation 4], when the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by about 10.8 Hz. The signal spectrum of the second acoustic signal obtained in step S5 is as shown in FIG.

  In steps S4 and S5, the processes according to the above [Equation 4] and [Equation 5] are executed, whereby complex spectra corresponding to the first and second acoustic frames are obtained. Subsequently, the frequency component modifying means 30 performs a process of adding a low frequency cancellation component to the high frequency using the first spectrum obtained from the first acoustic frame (step S6). Specifically, the first spectrum component is inverted between positive and negative, and a process of turning around the frequency Fs / 2 is performed.

  Once the low frequency cancellation component addition processing in step S6 has been performed, the frequency component modification means 30 then combines the first spectrum and the second spectrum after the low frequency cancellation component addition processing (step S7). . The second spectrum is added while being folded around the frequency Fs / 4. As a result, the first spectrum (synthetic spectrum) is in a state as shown in FIG. In the first spectrum after synthesis (the spectrum of the modified acoustic signal), a signal component obtained by inverting the amplitude of the signal component existing in the original first spectrum and a signal component of the second spectrum in a range larger than the frequency Fs / 4, It exists in a folded state.

  In the example of FIG. 10C, two types of spectral components are superimposed and displayed at frequencies Fs / 4 to Fs / 2, but in reality they are added as complex vectors. As shown in FIG. 10C, in the synthesized spectrum, the amplitude inversion component of the first acoustic signal and the component derived from the second acoustic signal are converted into noise at frequencies Fs / 4 to Fs / 2. Since the low-frequency component of one acoustic signal is masked, even if the modified acoustic signal is reproduced, the amplitude inversion component of the first acoustic signal and the component of the second acoustic signal are not heard by humans.

  The processing performed by the frequency component modifying means 30 in steps S6 and S7 can be summarized as the following [Equation 10].

[Formula 10]
Al ′ (j) ← −Al (N / 2−j) × α + A2l (N / 2−j) × βl
Bl ′ (j) ← −Bl (N / 2−j) × α + B2l (N / 2−j) × βl
Ar ′ (j) ← −Ar (N / 2−j) × α + A2r (N / 2−j) × βr
Br ′ (j) ← −Br (N / 2−j) × α + B2r (N / 2−j) × βr

  The frequency component modifying means 30 performs the processing according to the above [Equation 10] on the frequency components Al (j), Bl (j), A2l (j), B2l (j), j = N / 4 + 1,. .., for each j of N / 2-1. When the number of samples of one acoustic frame is N = 4096 and the sampling frequency Fs is 44.1 kHz, j = N / 4 corresponds to the frequency Fs / 4, and j = N / 2 corresponds to the frequency Fs / 2. Therefore, in [Equation 10], j = 1025,..., 2048 is a processing target, and the frequency component of about 11.1 kHz to about 22.1 kHz (approximately the human audible range upper limit) is changed.

  In the above [Equation 10], α, β1, and βr are scaling real values set in the range of 0.0 <α, β1, and βr ≦ 1.0. In the present embodiment, α = βl = βr = 1.0 is set. The first term (−Al (N / 2−j) × α, etc.) on the right side of each equation shown in [Formula 10] is a low-frequency canceling component, and the second term (A2l (N / 2-j) × βl etc. is a component obtained by folding the second spectrum in the frequency direction.

  When embedding the component of the second acoustic signal in the audible range, it is necessary to pay attention to sound quality degradation. Therefore, as in this embodiment, when embedding in the human audible range of about 11.1 kHz to about 22.1 kHz, βl and βr are increased or decreased in conjunction with the signal intensity of the original original sound signal. , Quality deterioration due to embedding can be suppressed. Specifically, it can be calculated by the following [Equation 11].

[Formula 11]
βl = βo [βlp + 50 × Avel] / 2
βr = βo [βrp + 50 × Aver] / 2
Avel = [Σ _{i = 1,..., N / 4-1} {Al (j + N / 4) ² + Bl (j + N / 4) ² } × 16j / N ² ] ^1/2
Aver = [Σ _{i = 1,..., N / 4-1} {Ar (j + N / 4) ² + Br (j + N / 4) ² } × 16j / N ² ] ^1/2

  In the above [Formula 11], βo = 1.0, and βlp and βrp are the values of βl and βr determined in the immediately preceding acoustic frame, respectively. In the case of the first acoustic frame, since there is no previous acoustic frame, βlp and βrp = βo are set. The coefficient “50” appearing in the first and second formulas is a predetermined constant and can be set to an integer value of 0 to 100. However, as the present embodiment is closer to “50”, preferable. Avel and Aver are average values of signal components in a predetermined range included from the frequency Ft to the frequency 2Ft. If βl and βr calculated in the above [Equation 11] are used as they are, spike noise may occur due to a sudden change in βl and βr. Therefore, in this embodiment, βl and βr calculated in the above [Equation 11]. And the average values of βlp and βrp obtained from the immediately preceding sound frame are calculated as new βl and βr and used in [Equation 10].

  When the correspondence between the flowchart of FIG. 5 and the above [Equation 10] is shown, the process of adding a low frequency cancellation component to the high frequency in Step S6 corresponds to the process of adding the first term on the right side of the above [Equation 10]. Then, the synthesis process of the first spectrum and the second spectrum in step S7 corresponds to the process of adding the second term on the right side of the above [Equation 10].

  After the frequency component modification unit 30 finishes the synthesis process in step S7, the frequency-time conversion unit 40 performs a process of obtaining a modified acoustic frame by performing frequency-time conversion on the first spectrum after modification (step). S8). Naturally, this frequency-time conversion needs to correspond to the technique executed by the time-frequency conversion means 20. In the present embodiment, since the time-frequency conversion means 20 performs the Fourier transform, the frequency-time conversion means 40 executes the inverse Fourier transform.

  Specifically, as in the first embodiment, the frequency-to-time conversion means 40 has a real part Al ′ (j), an imaginary part Bl ′ ( j) Using the real part Ar ′ (j) and imaginary part Br ′ (j) of the right channel, the processing according to the above [Equation 7] is performed to calculate Xl ′ (i) and Xr ′ (i). To do. The frequency components that have not been modified by the frequency component modifying means 30 are Al ′ (j), Bl ′ (j), Ar ′ (j), and Br ′ (j), which are the original frequency components, Al. (J), Bl (j), Ar (j), and Br (j) are used.

  Similarly to the first embodiment, after the frequency-time conversion in step S8, the modified acoustic frames obtained can be sequentially output to obtain the modified acoustic signal. When the processing up to step S8 is duplicated as described above, the anti-aliasing component is preliminarily placed in the A / D converter or sampler in which the cancellation component of the original sound signal and the component of the second acoustic signal are used in the duplication. Usually, it is attenuated by the processing. Also in this embodiment, the following step S9 is performed in order to exert a desired effect even when the LPF process is performed at the time of replication.

  FIG. 11 is a diagram showing how the signal spectrum changes due to the processing in step S9. 11 (a) and 11 (b) are the same as FIG. 10 (a) and show the L channel and R channel signal spectra of the modified acoustic signal obtained as a result of the processing up to step S8, respectively. . In reality, in the case of a stereo sound signal, the signal spectrum of the L channel and the R channel is often not the same, but here, as in FIG. 7, for convenience of explanation, the L channel and the R channel are shown by the same signal spectrum. ing. FIGS. 11C and 11D show the signal spectra of the L channel and the R channel after processing in step S9, respectively. In step S9, since the L channel signal is not modified, but only the R channel signal is modified, FIG. 11 (c) is the same as FIG. 11 (a).

  Returning to the flowchart of FIG. For the modified acoustic signals shown in FIGS. 11A and 11B obtained by the processing up to step S8, arithmetic processing between channels and between samples is performed (step S9). Specifically, when both channels are added for each corresponding sample, the value of each sample of the R channel is corrected so that two adjacent samples are matched. In the present embodiment, as in the first embodiment, the process according to the above [Equation 8] is executed to correct the value Xr ′ of each sample of the R channel. The results of such correction are shown in FIGS. 11 (c) and 11 (d). Since no correction is performed on the L channel side, FIG. 11C is exactly the same as FIG. The R channel side is greatly modified in the time dimension, but no significant change is seen in the frequency dimension shown in FIG. 11D compared to FIG. However, since the values of two adjacent samples are closer than before the modification, a slight fold occurs as shown by the dotted line on the left side of FIG.

  The modified sound frame output means 50 sequentially outputs the modified sound frames after correction obtained by the process of the modified sound frame correction means 45 to the output file. The process (except for step S1) shown in the flowchart of FIG. 5 is executed for all the first acoustic frames of the first acoustic signal. When the number of the first sound frames is larger than the number of the second sound frames, the process returns to the first second sound frame and is repeated. As a result of performing processing on all the first sound frames in this way, a modified sound signal that is a set of modified sound frames is stored in the modified sound signal storage unit 62. In the present embodiment, the second acoustic signal is embedded from the beginning to the end of the first acoustic signal. However, it is also possible to set an embedding position and embed only in that range.

  FIG. 12 is a diagram showing a concept in a case where the modified acoustic signal obtained by the processing up to step S9 is duplicated by a realistic device including an LPF circuit. 12, FIGS. 12 (a) and 12 (b) are the same as FIGS. 11 (c) and 11 (d), respectively, and the L channel of the modified acoustic signal after correction obtained as a result of the processing up to step S9, The signal spectrum of R channel is shown. FIG. 12C shows a signal spectrum of a duplicate acoustic signal obtained by re-sampling after LPF processing.

  As shown in FIG. 12, the L channel modified acoustic signal and the R channel modified acoustic signal respectively emitted from the L channel speaker and the R channel speaker are mixed and recorded with a stereo or monaural microphone, and resampled at the sampling frequency Fs. A duplicate acoustic signal is obtained. Then, due to the mixing of the stereo sound signal, two adjacent samples have the same value and are equivalent to a state where the sample is thinned out to ½, and aliasing occurs. As shown in FIG. In the range of / 2 or less, the second acoustic signal as shown in FIG. 2B is restored, and a sound based on the second acoustic signal can be heard by humans. If the second acoustic signal such as a warning message is heard over the music, it is objectively found to be an illegal copy, and the music cannot be appreciated in its original state and cannot be commercialized. For this reason, deterrence against replication works.

<2.2.4. Fourth Embodiment>
Next, an interference sound embedding device for an acoustic signal according to a fourth embodiment of the present invention will be described. In the second embodiment, the original sound signal is upsampled to create a wideband sound signal, and white noise is embedded. However, in the fourth embodiment, upsampling is not performed and the original sound signal is embedded. A process of directly embedding white noise is performed. Since there are many processes similar to those of the second embodiment, description will be made with reference to the flowchart of FIG. In the present embodiment, the first sound signal sampled at the sampling frequency Fs = 44.1 kHz is used. In the present embodiment, the frequency Ft = Fs / 4, which is the center of folding, is used.

  In the present embodiment, the acoustic frame reading unit 10 directly reads the acoustic signal stored in the acoustic signal storage unit 61 without using the upsampling unit 8. The acoustic frame reading means 10 reads a predetermined number N of samples as one first acoustic frame from each of the left and right channels of the stereo acoustic signal stored in the acoustic signal storage unit 61 (step S12).

  Although the number N of samples of one first sound frame read by the sound frame reading means 10 can be set as appropriate, in the present embodiment, a case where N = 4096 will be described below. Therefore, the acoustic frame reading means 10 sequentially reads 4096 samples for each of the left channel and the right channel from the broadband acoustic signal as the first acoustic frame.

  Also in this embodiment, as in the first to third embodiments, odd-numbered acoustic frames and even-numbered acoustic frames are set by overlapping a predetermined number of samples (N / 2 = 2048 in this embodiment). Is done. Therefore, if the odd-numbered acoustic frames are A1, A2, A3... From the top, and the even-numbered acoustic frames are B1, B2, B3... From the top, A1 is samples 1 to 4096, A2 is samples 4097 to 8192, A3. Is samples 8193-12288, B1 is samples 2049-6144, B2 is samples 6145-10240, and B3 is samples 10241-14336. Note that the number of samples to be overlapped can be set as appropriate.

  Next, the time-frequency conversion means 20 performs time-frequency conversion on the first sound frame to obtain a complex spectrum of the first sound frame (step S13). In step S13, specifically, time-frequency conversion is performed using a window function. As the time-frequency conversion, various known methods for obtaining a complex spectrum, such as Fourier transform, wavelet transform, and the like, can be used. In the present embodiment, as in the first embodiment, a case where Fourier transform is used will be described as an example.

  In the present embodiment, the Fourier transform for each first acoustic frame is performed on the product multiplied by the Hanning window function W (i) defined by the above [Equation 1].

  When the time-frequency conversion means 20 performs a Fourier transform on the first sound frame, the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1). Then, using the window function W (i), the processing according to the above [Equation 4] is performed, and the real part Al (j), the imaginary part Bl (j), and the right channel of the conversion data corresponding to the left channel are handled. The real part Ar (j) and imaginary part Br (j) of the conversion data to be obtained are obtained. As a result of the above processing of [Equation 4], when the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by about 10.8 Hz.

  By executing the processing according to [Formula 4] in step S13, a complex spectrum corresponding to each first acoustic frame is obtained. Subsequently, as in the second embodiment, the frequency component modifying unit 30 performs a process of adding a low frequency cancellation component to the high frequency using the first spectrum obtained from the first acoustic frame (step S14). ).

  After performing the low frequency cancellation component addition processing in step S14, next, the frequency component modification means 30 adds white noise to the first spectrum after the low frequency cancellation component addition processing, as in the second embodiment. Processing to add is performed (step S15). Specifically, a process according to the following [Equation 12] is executed, and a process of adding white noise, which is a noise equivalently including each frequency component, to a predetermined high range is performed.

[Formula 12]
When Al (j) ≧ 0 Al ′ (j) ← −Al (N / 4−j) × α + γl
When Al (j) <0 Al ′ (j) ← −Al (N / 4−j) × α−γl
When B1 (j) ≧ 0 B1 ′ (j) ← −B1 (N / 4−j) × α + γ1
When B1 (j) <0 B1 ′ (j) ← −B1 (N / 4−j) × α−γ1
When Ar (j) ≧ 0 Ar ′ (j) ← −Ar (N / 4−j) × α + γr
When Ar (j) <0 Ar ′ (j) ← −Ar (N / 4−j) × α−γr
When Br (j) ≧ 0 Br ′ (j) ← −Br (N / 4−j) × α + γr
When Br (j) <0 Br ′ (j) ← −Br (N / 4−j) × α−γr

  The frequency component modifying means 30 performs the processing according to the above [Equation 12] on the frequency components Al (j), Bl (j), Ar (j), Br (j), j = N / 4 + 1,. .., for each j of N / 2-1. When the number of samples of one acoustic frame is N = 4096 and the sampling frequency Fs is 44.1 kHz, j = N / 4 corresponds to the frequency Fs / 4, and j = N / 2 corresponds to the frequency Fs / 2. Note that no modification is made to each component of j = 1,... N / 4. In the above [Equation 12], α is a scaling real value set in a range of 0.0 <α ≦ 1.0. In this embodiment, α = 1.0 is set. Also, γl and γr are signal level real values corresponding to white noise components. In this embodiment, Al (j), Bl (j), Ar (j), and Br (j) are in a 16-bit range (− 32768 to +32767), γl = γr = 1024.0 is set. The first term (−Al (N / 4−j) × α, etc.) on the right side of each equation shown in [Equation 12] is a low-frequency canceling component, and the second terms γl and γr on the right side of each equation are It is a white noise component.

  When embedding a white noise component in the audible range, it is necessary to pay attention to deterioration in sound quality. Therefore, as in this embodiment, when embedding in the human audible range of about 11.1 kHz to about 22.1 kHz, γl and γr are increased or decreased in conjunction with the signal intensity of the original original sound signal. , Quality deterioration due to embedding can be suppressed. Specifically, it can be calculated by the following [Equation 13].

[Formula 13]
γl = γo [γlp + 50 × Avel] / 2
γr = γo [γrp + 50 × Aver] / 2
Avel = [Σ _{i = 1,..., N / 4-1} {Al (j + N / 4) ² + Bl (j + N / 4) ² } × 16j / N ² ] ^1/2
Aver = [Σ _{i = 1,..., N / 4-1} {Ar (j + N / 4) ² + Br (j + N / 4) ² } × 16j / N ² ] ^1/2

  In the above [Equation 13], γo = 1024.0, and γlp and γrp are the values of γl and γr determined in the immediately preceding acoustic frame, respectively. In the case of the first acoustic frame, since there is no previous acoustic frame, γlp and γrp = γo are set. The coefficient “50” appearing in the first and second formulas is a predetermined constant and can be set to an integer value of 0 to 100. However, as the present embodiment is closer to “50”, preferable. Avel and Aver are average values of signal components in a predetermined range included from frequency Ft to frequency 2Ft, as in the third embodiment. If γl and γr calculated in the above [Equation 13] are used as they are, spike noise may occur due to a sudden change in γl and γr. And the average values of γlp and γrp obtained from the immediately preceding acoustic frame are calculated as new γl and γr and used in [Equation 12].

  As shown in [Expression 12] above, predetermined intensities γl and γr are given as white noise components so as to increase the absolute values of the real and imaginary components of the complex number. The white noise addition processing in step S15 is performed only for odd-numbered sound frames, and not performed for even-numbered sound frames, as in the second embodiment. By turning on / off white noise at odd and even numbers, the ratio of modification to the original sound signal is halved and noise reproduction sound is reduced by adding low frequency alternating fluctuations of On / Off. Can be clear.

  In the present embodiment, the frequency component modification process in steps S14 and S15 is performed on the odd-numbered sound frames and not performed on the even-numbered sound frames. There is no need to perform modification processing on the above, and a method with a high effect of interfering sound may be selected as appropriate.

  After the frequency component modification unit 30 performs the white noise addition in step S15 and completes the modification process, the frequency-time conversion unit 40 performs frequency-time conversion of the modified spectrum to obtain a modified acoustic frame. Is performed (step S16). Naturally, this frequency-time conversion needs to correspond to the technique executed by the time-frequency conversion means 20. In the present embodiment, since the time-frequency conversion means 20 performs the Fourier transform, the frequency-time conversion means 40 performs the inverse Fourier transform by the process according to the above [Equation 7].

  After the frequency-time conversion in step S16, the modified acoustic frames obtained can be sequentially output to obtain the modified acoustic signal. However, as described above, there is no copy prevention effect at the stage of processing up to step S16. Therefore, in this embodiment as well, in the same way as in the first to third embodiments, in order to reproduce a desired disturbing sound even when the LPF processing is performed at the time of duplication, the processing in the following step S17 is performed. Do.

  That is, similarly to the second embodiment, the inter-channel and inter-sample arithmetic processing is performed on the modified acoustic signal obtained by the processing up to step S16 (step S17). Specifically, as in the second embodiment, when the corresponding components are added to monaural in two channels, the values of two adjacent samples are the same value (the average value of a total of four samples in two channels). The value of each sample of the R channel is corrected so that × 2). In the present embodiment, as in the first embodiment, the process according to the above [Equation 8] is executed to correct the value Xr ′ of each sample of the R channel.

  The modified sound frame output means 50 sequentially outputs the modified sound frames after correction obtained by the process of the modified sound frame correction means 45 to the output file. The process shown in the flowchart of FIG. 9 is executed for all the first sound frames of the first sound signal. As a result of performing processing on all the first sound frames in this way, a modified sound signal that is a set of modified sound frames is stored in the modified sound signal storage unit 62. In the present embodiment, the disturbing sound is embedded from the beginning to the end of the first acoustic signal. However, it is also possible to set an embedding position and embed only in that range.

<Support for 3.5.1 channel surround sound>
In the above embodiment, the case of LR two-channel stereo sound has been described, but the present invention can also be adapted to 5.1 channel surround sound. As shown in FIG. 13, the 5.1-channel surround sound signal is composed of a total of 6 channels: front speaker channels Fl, Fc, Fr, rear speaker channels Bl, Br, and bass speaker channel Lf. .

  When the present invention is applied to a 5.1 channel surround sound signal, a combination of Fl and Fc for front speakers, L channel ("L1" in FIG. 13) when Fc is a stereo sound signal, and Fl is a stereo sound signal. For the R channel ("R1" in FIG. 13), the same processing as in the above embodiment is performed, and the modified acoustic frame correction means 45 corrects the Fl signal. Subsequently, with the combination of Fr and Fc for the front speakers, F channel is an L channel in the case of a stereo sound signal ("L2" in FIG. 13), and Fr is an R channel in the case of a stereo sound signal ("R2" in FIG. 13). ), The same processing as in the above embodiment is performed, and the modified acoustic frame correction means 45 corrects the Fr signal. Further, in the combination of Bl and Br for the rear speakers, Bl is an L channel in the case of a stereo sound signal ("L3" in FIG. 13), and Br is an R channel in the case of a stereo sound signal ("R3" in FIG. 13). Then, the same processing as in the above embodiment is performed, and the modified acoustic frame correction unit 45 corrects the Br signal. The embedding process is not performed for Lf for the bass speaker.

  In this way, when a 5.1 channel surround sound signal in which an interfering sound is embedded is reproduced, if someone tries to copy illegally, the Fl and Fc signals are mixed, or the Fr and Fc signals are mixed. Is mixed, the signals of Bl and Br are mixed, or mixing of each signal occurs at the same time, thereby causing aliasing. As a result, a component corresponding to the disturbing sound is recorded in the duplicate sound signal, and when the duplicate sound signal is reproduced, the disturbing sound is reproduced.

<4. Modified example>
The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the above embodiments, and various modifications can be made. For example, in the above-described embodiment, in the first acoustic signal, the signal component having the frequency Ft or higher is removed, and the signal component having the frequency Ft or lower is folded around the frequency Ft in the high frequency direction. The value of is not limited to the above embodiment, and various values can be set.

  In step S6 in the first and third embodiments, and in step S14 in the second and fourth embodiments, the time-frequency conversion means 20 adds a low frequency cancellation component to the high frequency of the first spectrum. However, this process may be omitted.

  In the second and fourth embodiments, the low-frequency cancellation component and white noise addition processing in steps S14 and S15 are performed on the odd-numbered acoustic frame or the even-numbered acoustic frame. The additional processing may be executed with the above frame interval, or may be executed for all sound frames.

1 ... CPU
2 ... RAM
3 ... Storage device 4 ... Key input I / F
5. Data input / output I / F
6 ... Display output I / F
8: Upsampling means 10 ... Sound frame reading means 20 ... Time-frequency conversion means 30 ... Frequency component modification means 40 ... Frequency-time conversion means 45 ... Modified sound frame correction means 50 ... Modified acoustic frame output means 60 ... Storage means 61 ... Acoustic signal storage section 62 ... Modified acoustic signal storage section

Claims (9)

A device that embeds a second acoustic signal composed of a time-series sample sequence in an inaudible state as an interfering sound with respect to an original acoustic signal composed of a time-series sample sequence of at least two channels,
Two channels are selected from the original acoustic signal, an acoustic frame composed of a predetermined number of samples is read as a first acoustic frame for each channel, and an acoustic frame composed of a predetermined number of samples from the second acoustic signal Sound frame reading means for reading as a second sound frame;
Time-frequency conversion is performed for each channel on the first sound frame to obtain a first spectrum that is a complex frequency component, and time-frequency conversion is performed on the second sound frame to obtain a complex frequency component. Time-frequency conversion means for obtaining a second spectrum;
Among the signal components of the second spectrum, a signal component of a predetermined frequency Ft or higher is removed, and a signal component of the frequency Ft or lower is folded back in the high frequency direction with the frequency Ft as a center, and from the folded frequency Ft. The frequency component of the first spectrum is added to the signal component of the corresponding frequency of the first spectrum by multiplying the signal component of the predetermined range included in the frequency 2Ft by a predetermined coefficient value, for each channel. A frequency component modifying means for modifying;
Frequency-time conversion means for generating a modified sound frame by performing frequency-time conversion on the first spectrum in which the frequency component is modified;
When two channels are added for each corresponding sample to the generated modified acoustic frame for two channels, a total of four adjacent two samples for two channels are matched so that the two adjacent samples match. A modified acoustic frame correction means for performing correction to change the value of two samples of each set with a set of samples;
Modified acoustic frame output means for sequentially outputting the modified acoustic frames corrected by the modified acoustic frame correcting means;
A device for embedding a disturbing sound for an acoustic signal, comprising: An apparatus that embeds noise composed of a time-series sample sequence in an inaudible state as an interfering sound with respect to an original sound signal composed of a time-series sample sequence of at least two channels,
Sound frame reading means for selecting two channels from the original sound signal and reading a sound frame composed of a predetermined number of samples for each channel as a first sound frame;
Time-frequency conversion means for performing time-frequency conversion for each channel on the first acoustic frame to obtain a first spectrum as a complex frequency component;
Among the signal components of the first spectrum, the frequency components of the first spectrum such that the absolute value of the signal increases by a predetermined value with respect to the signal components in the predetermined range included in the frequency 2Ft from the predetermined frequency Ft. Frequency component modifying means for modifying
Frequency-time conversion means for generating a modified sound frame by performing frequency-time conversion on the first spectrum in which the frequency component is modified;
When two channels are added for each corresponding sample to the generated modified acoustic frame for two channels, a total of four adjacent two samples for two channels are matched so that the two adjacent samples match. A modified acoustic frame correction means for performing correction to change the value of two samples of each set with a set of samples;
Modified acoustic frame output means for sequentially outputting the modified acoustic frames corrected by the modified acoustic frame correcting means;
A device for embedding a disturbing sound for an acoustic signal, comprising: In claim 2,
The frequency component altering means performs processing on acoustic frames at predetermined intervals among the acoustic frames read by the acoustic frame reading means. . In any one of Claims 1-3,
The frequency component modifying means removes a signal component having a frequency Ft or higher from the signal component of the first spectrum obtained by the time-frequency converting means, and a signal having the frequency Ft or lower with the frequency Ft as a center. The components of the first spectrum are folded back in the high frequency direction, and the sign component of the first spectrum in the range of the folded frequency Ft to the frequency 2Ft is inverted with the sign of the first spectrum and multiplied by a predetermined coefficient value. An interference sound embedding apparatus for an acoustic signal, wherein a process of modifying the frequency component of the first spectrum for each channel by adding to a signal component of a corresponding frequency is performed in advance. In any one of Claims 1-4,
The original sound signal is sampled at a sampling frequency Fs,
Upsampling means for upsampling the original sound signal at a sampling frequency Fus (Fus> Fs) to create a wideband sound signal that is a wideband original sound signal;
Ft = Fs / 2,
The time-frequency conversion means, the frequency component modification means, the frequency-time conversion means, the modified sound frame correction means, and the modified sound frame output means perform processing on the wideband sound signal. An embedding device for disturbing sound to an acoustic signal. In any one of Claims 1-4,
The original sound signal is sampled at a sampling frequency Fs,
The interference sound embedding device for an acoustic signal, wherein Ft = Fs / 4. In claim 6,
The predetermined coefficient value to be multiplied or the predetermined value to be increased in the frequency component modifying means is within a predetermined range included in the frequency 2Ft from the frequency Ft in the signal component of the first spectrum obtained by the time-frequency converting means. An interfering sound embedding device for an acoustic signal, wherein the device is varied based on an average value of signal components. In any one of Claims 1-7,
The time-frequency conversion means has a weight W (i) (0 ≦ W (i) ≦ 1) at a sample position i (0 ≦ i ≦ N−1) as window width N samples, and W (i) = 0. A device for embedding interfering sound into an acoustic signal, characterized in that time-frequency conversion is performed using a Hanning window function defined by 5-0.5 cos (2πi / N).   A non-transitory computer-readable storage medium storing a program for causing a computer to function as an embedding device for interfering sound with respect to an acoustic signal according to any one of claims 1 to 8.

Images