JP2006227330A - Embedding device/method of information to audio signals, extraction device/method of information from audio signals - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide the embedding device/method of information to audio signals which can embed mass additional information to audio signals of a plurality of channels as un-audible electronic watermark information by hardly affecting original sound quality and also can extract embedded additional information from audio signals under being reproduced correctly. <P>SOLUTION: In the embedding device of information to audio signals, additional information which is to be embedded is read (S101), audio frames are extracted from respective channels of a stereo audio signal (S104), the extracted frames are subjected to frequency conversion (S105), when intensity of low frequency components is below the reference, low frequency components of the left and right channels are processed evenly as information showing a delimiter (S108), when the intensity of the low frequency components is equal to or larger than the reference, the distribution between the left and right channels is changed so that the left is larger or the right is larger according to the bit values of the additional information (S107). Then, the additional information is embedded in the audio signals by subjecting the signals whose low frequency components are changed to frequency inverse conversion (S109). <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to the protection of music copyright (monitoring illegal copying) in the field of package music for viewing for consumer and business use using CDs and DVDs, and the field of broadcasting and network music distribution distributed for commercial purposes by broadcasters and the like. ) And the provision of music attribute information (music title search service).

  As a service to provide music attribute information that allows you to know the titles of music that has been played recently, you can query the broadcast station for the date and time of the broadcast music, and record music fragments that are being played on mobile phones. Services that collate with melodies registered in the database have been put into practical use (see, for example, Patent Documents 1 and 2).

  In the inventions described in Patent Documents 1 and 2, since the recorded music fragments are checked against the melodies registered in the database, the processing load increases as the number of songs registered in the database increases, and similar melodies are mistaken. The possibility of judging increases. Therefore, a method of embedding music attribute information such as a song name and artist information as an inaudible digital watermark in an acoustic signal has also been proposed (see, for example, Patent Documents 3 to 6).

In the methods described in Patent Documents 1 to 6, the amount of information that can be embedded is small, the sound quality is deteriorated to some extent, watermark information is lost by various signal processing, and watermark detection is difficult for analog copies. There is a problem. Therefore, the present applicant has proposed a method of embedding attribute information by changing the ratio of low frequency components of an acoustic signal having a plurality of channels according to the bit value of the attribute information (see Patent Document 7).
JP 2002-259421 A Japanese Patent Laid-Open No. 2003-157087 JP-A-11-145840 JP-A-11-219172 Japanese Patent No. 3321767 JP 2003-99077 A Japanese Patent Application No. 2005-5157

  However, in the method described in Patent Document 7, when additional information such as attribute information is extracted via analog, it is not always possible to extract from the top position where the additional information is embedded. There is a problem that extraction may not be possible.

  Therefore, the present invention embeds a large amount of additional information as an inaudible digital watermark and reproduces it for an audio signal of a plurality of channels provided on a CD or broadcast with little influence on the original sound quality. It is an object of the present invention to provide an information embedding apparatus / method for an acoustic signal and an apparatus / method for extracting information from an acoustic signal, which can accurately extract additional information embedded from the existing acoustic signal.

  In order to solve the above-described problem, in the present invention, additional information cannot be heard for a stereo acoustic signal having two left and right channels, and each of the left and right channels is composed of a time-series sample string. An apparatus for embedding in a state, additional information reading means for sequentially reading a bit array of additional information to be embedded, and acoustic frame reading means for reading a predetermined number of samples as acoustic frames from the left and right channels of the acoustic signal, Frequency conversion means for performing frequency conversion on each of the read sound frames and generating a plurality of frame spectra corresponding to the respective channels, and corresponding to a component having a predetermined frequency or less from the generated plurality of frame spectra The low frequency intensity data is extracted, and each bit value of the read additional information bit array can be taken. Based on the information indicating the bit array delimiter of the additional information, the channel-to-channel ratio of the low-frequency intensity data corresponding between the channels is any one of three patterns that are the left channel is large, the right channel is large, and the left and right are equal. Low frequency component changing means for changing to, and frequency reverse converting means for performing frequency inverse conversion on the plurality of frame spectra including the changed low frequency component to generate a plurality of modified acoustic frames, and the generated Provided is an information embedding device for an acoustic signal having modified acoustic frame output means for sequentially outputting modified acoustic frames.

  According to the present invention, with respect to an acoustic signal composed of two channels, the ratio between the channels of the low frequency components of each acoustic signal is represented in the information array of additional information to be embedded and the information indicating the separation of the information array. As a result, the large amount of additional information is embedded as an inaudible digital watermark, and the embedded additional information is played back with little influence on the original sound quality. The effect that it becomes possible to extract correctly from the sound signal which has been carried out is produced.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(1. Information embedding device for acoustic signals)
FIG. 1 is a functional block diagram showing the configuration of an information embedding device for an acoustic signal according to the present invention. In FIG. 1, 10 is an acoustic frame reading means, 20 is a frequency converting means, 30 is a low frequency component changing means, 40 is a frequency inverse converting means, 50 is a modified acoustic frame output means, 60 is a storage means, and 61 is an acoustic signal storage. , 62 is an additional information storage unit, 63 is a modified acoustic signal storage unit, and 70 is an additional information reading means.

  The sound frame reading means 10 has a function of reading a predetermined number of samples as one frame from each channel of the original stereo sound signal to be embedded with additional information. The frequency conversion means 20 has a function of generating a frame spectrum by frequency-converting the frame of the acoustic signal read by the acoustic frame reading means 10 by Fourier transformation or the like. The low frequency component changing means 30 extracts each low frequency intensity data corresponding to a predetermined frequency or less from a plurality of generated frame spectra, and responds between channels based on the additional information extracted from the additional information storage unit 62. It has a function of changing the inter-channel ratio of low frequency intensity data. The frequency reverse conversion means 40 has a function of generating a plurality of modified sound frames by performing frequency reverse conversion on a plurality of frame spectra including the changed low frequency intensity data. The modified sound frame output means 50 has a function of sequentially outputting the generated modified sound frames. The storage means 60 includes an acoustic signal storage unit 61 that stores a stereo acoustic signal to be embedded with additional information, an additional information storage unit 62 that is configured as a bit array and stores additional information embedded in the stereo acoustic signal, and an additional information It has a modified acoustic signal storage unit 63 for storing the modified acoustic signal after information is embedded, and stores various information necessary for other processing. The additional information reading means 70 has a function of extracting additional information from the additional information storage unit 62. The additional information is information that should be added to the sound information and embedded, and includes attribute information such as a title and artist name, and other information other than the attribute information. Each component shown in FIG. 1 is actually realized by installing a dedicated program in hardware such as a computer and its peripheral devices. That is, the computer executes the contents of each means according to a dedicated program.

(2. Processing operation of embedded device)
Next, the processing operation of the information embedding device for the acoustic signal shown in FIG. 1 will be described with reference to the flowchart of FIG. FIG. 2 corresponds to processing of 1 byte of additional information. First, the additional information reading means 70 reads additional information from the additional information storage unit 62 in units of 1 byte (S101). Specifically, 1 byte is read into the register. Subsequently, the mode is set to the separation mode (S102). There are two types of modes: separation mode and bit mode. The delimiter mode indicates a mode for performing processing at a delimiter in units of 1 byte, and the bit mode indicates a mode for performing processing based on the value of each bit of 1 byte. When one byte is read from the additional information storage unit 62, the separator mode is always set immediately after that.

  Subsequently, the acoustic frame reading means 10 reads a predetermined number of samples as one acoustic frame from each of the left and right channels of the stereo acoustic signal stored in the acoustic signal storage unit 61 (S104). The number of samples of one sound frame read by the sound frame reading means 10 can be set as appropriate, but is desirably about 4096 samples when the sampling frequency is 44.1 kHz. Therefore, the acoustic frame reading means 10 sequentially reads 4096 samples for each of the left channel and the right channel as acoustic frames.

  Subsequently, the frequency conversion means 20 performs frequency conversion on each read sound frame to obtain a frame spectrum that is a spectrum of the sound frame (S105). As frequency conversion, Fourier transform, wavelet transform, and other various known methods can be used. In the present embodiment, a case where Fourier transform is used will be described as an example. When performing Fourier transform, the left channel signal xl (i) and the right channel signal xr (i) (i = 0,..., N−1) are processed according to the following [Equation 1], and left The real part Al (j) and imaginary part Bl (j) of the conversion data corresponding to the channel and the real part Ar (j) and imaginary part Br (j) of the conversion data corresponding to the right channel are obtained.

[Formula 1]
Al (j) = Σ _{i = 0,..., N−1} xl (i) · cos (2πij / N)
Bl (j) = Σ _{i = 0,..., N−1} xl (i) · sin (2πij / N)
Ar (j) = Σ _{i = 0,..., N−1} xr (i) · cos (2πij / N)
Br (j) = Σ _{i = 0,..., N−1} xr (i) · sin (2πij / N)

  In [Expression 1], i is a serial number assigned to N samples in each acoustic frame, and takes an integer value of i = 0, 1, 2,... N−1. Further, j is a serial number assigned in order from the smallest value of the frequency value, and takes an integer value of j = 0, 1, 2,... N−1 similarly to i. When the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by 10.8 Hz. At this time, the acoustic signals xl (i) and xr (i) are each weighted with a window function (Hanning window) expressed by W (i) = 0.5−0.5 · cos (2πi / N). Multiply. Such a window function is used to reduce high-frequency noise generated by dividing the waveform into frequency components when performing Fourier transform, and between analysis intervals (corresponding to acoustic frames) when performing inverse Fourier transform. This is a well-known technique used to connect signal levels so as not to be discontinuous.

  By executing the processing according to the above [Equation 1], a frame spectrum in which the signal component of each acoustic frame is expressed by a spectrum corresponding to the frequency is obtained. Subsequently, the low frequency component changing unit 30 extracts low frequency intensity data having a frequency equal to or lower than a predetermined frequency from the generated plurality of frame spectra. Human hearing is known to be less sensitive to directionality for low frequency components up to about 200-300 Hz (Corona Corp., issued October 30, 1990, "Sound Engineering Course 1. Basic Acoustics"). Engineering, Acoustical Society of Japan ”p.247 (see FIGS. 9 and 26). Therefore, in this embodiment, about 200 Hz is set as the predetermined frequency, and low frequency intensity data having a frequency of 200 Hz or less is extracted. In the vicinity of a frequency of 200 Hz, j corresponds to 20, so the real part Al (j), imaginary part Bl (j), real part Ar (j), and imaginary part Br (j) calculated by the above [Equation 1]. Among them, those with j ≦ 20 are extracted.

  Subsequently, the low-frequency component changing unit 30 uses the extracted real part Al (j), imaginary part Bl (j), real part Ar (j), and imaginary part Br (j) as follows: ], The total value of the low frequency intensity up to j = 1 to M (20 when extracting 200 Hz or less) is calculated.

[Formula 2]
E = Σ _{j = 1,..., M} {Al (j) ² + Bl (j) ² + Ar (j) ² + Br (j) ² }

  E calculated by the above [Equation 2] indicates the sum of low frequency components in the frame spectrum. Subsequently, it is determined whether or not the total value E is equal to or higher than the level lower limit value. The level lower limit value is set to about 4 when the maximum amplitude value of the acoustic signals xl (i) and xr (i) is normalized to 1 and M = 20. The reason why it is determined whether or not the total value E is equal to or higher than the lower limit of the level is that if the signal strength is small, the change cannot be detected on the extraction side even if the signal is changed. Therefore, when the total value E is less than the lower level limit value, recording is not performed according to the bit value of the additional information, but processing is started again from the first bit, so the reading position is returned to the first bit and the mode is set to the delimiter mode. (S106). On the other hand, when the total value E is equal to or higher than the level lower limit value, the mode is determined.

  When the mode is the separation mode, the low frequency component changing unit 30 performs a process of making the low frequency intensity the same between the left and right channels (S108). That is, the low frequency component sound source is moved to the center. Specifically, processing according to the following [Equation 3] is executed.

[Formula 3]
E (j) = [0.5 · {Al (j) ² + Bl (j) ² + Ar (j) ² + Br (j) ² }] ^1/2
Al (j) ← Al (j) · E (j) / {Al (j) ² + Bl (j) ² } ^1/2
Bl (j) ← Bl (j) · E (j) / {Al (j) ² + Bl (j) ² } ^1/2
Ar (j) ← Ar (j) · E (j) / {Ar (j) ² + Br (j) ² } ^1/2
Br (j) ← Br (j) · E (j) / {Ar (j) ² + Br (j) ² } ^1/2

  In the above [Formula 3], “←” in the second to fifth formulas indicates that the calculation result of the right side is substituted into the left side. By executing the processing according to the above [Equation 3], the low frequency component of the frame spectrum is changed to a pattern equivalent to the left and right channels. This equal pattern of left and right channels is information indicating the head position (separation) of the additional information.

  On the other hand, when the mode is the bit mode, the low frequency component changing means 30 changes the distribution of the low frequency component between the left and right channels according to the bit value of the bit array of the additional information extracted from the additional information storage unit 62. A process of changing to either large or right channel is performed (S107). That is, the sound source of the low frequency component is moved to the left or right. Specifically, by executing processing according to any of the following [Formula 4] and [Formula 5] according to the bit value that can take the first value and the second value, Change the left-right distribution. For example, when the first value is 1 and the second value is 0, when the bit value is 1, processing according to the following [Equation 4] is executed.

[Formula 4]
E (j) = {Al (j) ² + Bl (j) ² + Ar (j) ² + Br (j) ² } ^1/2
Ar (j) ← Ar (j) · E (j) / {Ar (j) ² + Br (j) ² } ^1/2
Br (j) ← Br (j) · E (j) / {Ar (j) ² + Br (j) ² } ^1/2
Al (j) = 0
Bl (j) = 0

  When the bit value is 0, processing according to the following [Formula 5] is executed.

[Formula 5]
E (j) = {Al (j) ² + Bl (j) ² + Ar (j) ² + Br (j) ² } ^1/2
Al (j) ← Al (j) · E (j) / {Al (j) ² + Bl (j) ² } ^1/2
Bl (j) ← Bl (j) · E (j) / {Al (j) ² + Bl (j) ² } ^1/2
Ar (j) = 0
Br (j) = 0

  By executing the processing according to any one of [Equation 4] and [Equation 5], the low frequency component is either the left channel or the right channel according to each bit value of the bit array of the additional information. It will be changed to a pattern. Eventually, the low frequency component changing means 30 performs the processing based on [Equation 3] in the case of the separation mode in S108, and performs the processing based on [Equation 4] or [Equation 5] in S107 in the case of the bit mode. Become.

  Next, the frequency inverse transform means 40 performs a process of obtaining a modified acoustic frame by performing frequency inverse transform on the frame spectrum in which the inter-channel ratio of the low frequency intensity data is changed (S109). As a matter of course, this frequency inverse transform needs to correspond to the method executed by the frequency transform unit 20 in S105. In the present embodiment, since the frequency transform unit 20 performs the inverse Fourier transform, the frequency inverse transform unit 40 performs the inverse Fourier transform. Specifically, the real part Al (j), the imaginary part Bl (j), and the real part Ar (j) of the right channel of the spectrum obtained by any one of [Formula 3] to [Formula 5]. Then, using imaginary part Br (j), processing according to the following [Equation 6] is performed to calculate xl ′ (i) and xr ′ (i).

[Formula 6]
xl ′ (i) = 1 / N · {Σj _{= 0,..., N-1} Al (j) · cos (2πij / N) −Σj _{= 0,..., N−1} B1 (j) · sin ( 2πij / N)} + {1-W (i)} · xl (i)
xr ′ (i) = 1 / N · {Σ _{j = 0,..., N−1} Ar (j) · cos (2πij / N) −Σ _{j = 0,..., N−1} Br (j) · sin ( 2πij / N)} + {1-W (i)} · xr (i)

  The terms “+ {1−W (i)} · xl (i)” in the first equation and “+ {1−W (i)} · xr (i)” in the second equation in the above [Equation 6] are When the frequency conversion is performed in the above [Equation 1], the signal component removed by multiplying by the window function W (i) is restored. By the above [Equation 6], each sample xl ′ (i) of the left channel and each sample xr ′ (i) of the right channel of the modified sound frame are obtained. The modified sound frame output unit 50 sequentially outputs the obtained modified sound frames to the output file (S110). When the processing for one acoustic frame is completed in this way, after the mode is set to the bit mode (S111), the additional information reading means 70 reads the next bit in the bit array of the additional information (S103). The above processing is executed over all samples of both channels of the acoustic signal. That is, a predetermined number of samples are read as sound frames, and when there are no more sound frames to be read from the sound signal (S104), the process ends. When the processing corresponding to each bit of the 1-byte data read in S101 is completed, the process returns from S103 to S101, and the next byte of the additional information is read. When the processing is completed for all the bytes of the additional information, the processing returns to the first byte of the additional information. As a result, all modified acoustic frames that have been processed for all acoustic frames are recorded in the output file and obtained as modified acoustic signals. The obtained modified acoustic signal is output to and stored in the modified acoustic signal storage unit 63 in the storage unit 60.

  A state of signal change by the above processing will be described with reference to FIG. In FIG. 3, the left-right direction is a time axis and is proportional to the number of samples. L and R indicate the left channel and the right channel of the modified acoustic signal, respectively. Further, a large number of rectangles in the figure indicate low-frequency components of the modified acoustic frame, the horizontal width indicates the number of samples (4096 in the present embodiment), and the vertical width indicates the intensity. FIG. 3A shows a case where there is no acoustic frame in which the total value E calculated by the above [Equation 2] is less than the level lower limit value, that is, a case where the signal is a good signal for embedding additional information. ing. FIG. 3B shows a case where there is an acoustic frame in which the total value E calculated by the above [Equation 2] is less than the level lower limit value, that is, a signal that is not good for embedding additional information. ing.

  For example, it is assumed that a 2-byte bit array in which the first byte is “11011100” and the second byte is “11000001” is embedded as additional information. First, at the head of each byte, left and right are set as information indicating delimiters. This is obtained as a result of executing the processing in accordance with the above [Equation 3] in S108 by setting the separation mode in S102. Further, in the example of FIG. 3A, since there is no acoustic frame in which the total value E is less than the level lower limit value, one byte is continuously processed by the above [Formula 4] or [Formula 5]. Become. This indicates that the loop from S103 to S111 was repeated 8 times in succession, and during that time, it did not pass through S106 and S108 because it was less than the level lower limit value. As shown in the figure, when the bit value of the additional information is 1, it exists on the R side, and when the bit value of the additional information is 0, the low frequency component is set on the L side. As can be seen from the above [Formula 4] and [Formula 5], in this case, the other low frequency component is zero.

  In the example of FIG. 3B, as a result of the processing according to the above [Equation 2], there is an acoustic frame that is less than the level lower limit value. In this case, the above [Equation 3] is obtained via S106 and S108. As a result of executing the processing according to this, the left and right state is set to be equal. In this case, since the reading position is returned to the first bit in S106, the same bit is read again. In the example of FIG. 3B, when “11011100” of the first byte is embedded, an acoustic frame less than the level lower limit value appears at the time when the 2-bit processing of “11” is first performed, and “11011” is performed the second time. When the 5-bit processing is performed, an acoustic frame less than the level lower limit value appears, indicating that the 8-bit processing was finally possible at the third time.

  However, when the processing according to FIG. 2 is performed, as shown in FIG. 3B, every time an acoustic frame having a low signal level of low frequency components appears, it is repeated from the beginning in units of 1 byte. Since it will be processed, efficiency is bad. Therefore, instead of FIG. 2, processing as shown in FIG. 4 may be performed.

  The processing shown in FIG. 4 differs greatly from the processing shown in FIG. 2 in that it has a continuous identification mode in addition to the separation mode and the bit mode, and has three modes, and an acoustic frame with a low signal level of low frequency components has appeared. Even in this case, the process is continued without returning to the top. Therefore, in the continuous identification mode, information for identifying whether the next bit is a new one starting from the beginning or continued because it is interrupted is recorded. In the process of FIG. 4, the additional information is processed in units of words instead of units of bytes. This is because, as shown in S106 of FIG. 2, there is no process for forcibly returning the reading position to the beginning, and it is easy to process in units other than bytes. The number of bits in one word can be set freely and can be set to 1 byte.

  Since the process of FIG. 4 has many parts in common with the process of FIG. 2, different parts will be described. First, the additional information reading means 70 reads additional information from the additional information storage unit 62 in units of one word (S201). Thereafter, the mode is set to the separation mode (S202), and the acoustic frame reading means 10 reads the acoustic frames from the left and right channels of the stereo acoustic signal stored in the acoustic signal storage unit 61 (S204), and the frequency converting means 20 However, frequency conversion is performed on each read acoustic frame according to [Equation 1] to obtain a frame spectrum (S205), and the low frequency component changing means 30 extracts the real part Al (j) and imaginary part Bl. (J) Using the real part Ar (j) and the imaginary part Br (j) to calculate the total value E by [Formula 2] and determine whether the total value E is equal to or higher than the level lower limit Are the same. However, in the process of FIG. 4, when the sum E is less than the level lower limit value, the mode is simply set to the separation mode (S206). On the other hand, when the total value E is equal to or higher than the level lower limit value, the mode is determined.

  When the mode is the delimiter mode, the low frequency component changing unit 30 performs the process of making the low frequency intensity the same in the left and right channels according to the above [Equation 3] (S208). On the other hand, when the mode is the bit mode, the low frequency component changing means 30 according to the above [Equation 4] and [Equation 5] according to the bit value of the bit array of the additional information extracted from the additional information storage unit 62, A process of changing the ratio between the left and right channels of low frequency intensity is performed (S207). Further, in S207, unlike S107 in FIG. 2, processing is performed even in the continuous identification mode. In the continuous identification mode, when it is new, the distribution between the left and right channels of the low frequency component is changed to the right channel according to [Equation 4]. Change the distribution between to the left channel size.

  Next, the frequency inverse transform means 40 performs a process of performing frequency inverse transform on the frame spectrum in which the inter-channel ratio of the low frequency intensity data is changed according to [Equation 6] to obtain a modified acoustic frame (S209), and the modified acoustic frame The output unit 50 sequentially outputs the obtained modified sound frames to the output file (S210). When the processing for one acoustic frame is completed in this manner, the mode is determined (S211). If the mode is the separation mode, the mode is set to the continuous identification mode (S212), and then the acoustic frame reading means 10 A frame is read (S204). On the other hand, when the mode is the bit mode or the continuous identification mode, after setting the mode to the bit mode (S213), the low frequency component changing means 30 reads the next bit in the bit array of the additional information (S203). Then, all the sound frames are processed, and when there are no more sound frames to be read from the sound signal (S204), the process ends.

  The state of signal change by the processing according to FIG. 4 will be described with reference to FIG. In FIG. 5, as in FIG. 3, the horizontal direction is the time axis, and a large number of rectangles in the figure indicate the low frequency components of the modified acoustic frame, the horizontal width indicates the number of samples, and the vertical width indicates the intensity. . FIG. 5A shows a case where there is no acoustic frame in which the total value E calculated by the above [Equation 2] is equal to or higher than the level lower limit value, as in FIG. 3A. ) Shows a case where there is an acoustic frame in which the total value E calculated by the above [Equation 2] is less than the level lower limit value, as in FIG.

  For comparison with the processing of FIG. 2, the additional information to be embedded is also a 2-byte bit array in which the first byte is “11011100” and the second byte is “11000001”, as in FIG. As in FIG. 3, the left and right states are set at the beginning of each byte as information indicating a delimiter. This is obtained as a result of executing the processing according to the above [Equation 3] in S208 by setting the separation mode in S202. Subsequently, before the processing corresponding to each bit of the additional information is performed, information indicating whether the information is new or continued is recorded. According to the processing shown in FIG. 2, when there is an acoustic frame that is less than the lower level limit value, the processing bit is returned to the head of each byte and processing is performed. Then, even if there is an acoustic frame that is less than the lower limit level, the bit processed at that time is valid and is continuously performed from there, so information on whether the bit is new or continued Must be recorded. Therefore, after recording the information indicating the break, information indicating whether it is new or continued is recorded. Specifically, the mode determination is performed in the separation mode state (S211), the continuous identification mode is set (S212), and the sound frame is extracted without reading the bits of the additional information (S204). Then, after frequency conversion (S205), if it is new, the distribution between the left and right channels of the low frequency component is changed to the right channel size by processing according to [Equation 4] (S207).

  After recording the information indicating whether it is new or continued in this way, in order to make a mode determination in the state of the continuous identification mode (S211), the bit mode is set (S213), and the first bit is read from the register ( (S203), an acoustic frame is extracted (S204). In the example of FIG. 5A, since there is no acoustic frame that is less than the level lower limit value, one byte is continuously processed by the above [Expression 4] or [Expression 5]. This indicates that the loop from S203 to S213 was repeated 8 times in succession, and during that time, it was not via S206, S208, and S212 that the level was less than the lower limit value.

  In the example of FIG. 5B, as a result of the processing according to the above [Equation 2], there is an acoustic frame that is less than the level lower limit value. In this case, the above [Equation 3] is obtained via S206 and S208. As a result of executing the processing according to this, the left and right state is set to be equal. In this case, since the separation mode is set in S206, information indicating whether it is new or continued is recorded via S212. In the example of FIG. 5B, when embedding “11011100” of the first byte, an acoustic frame less than the level lower limit value appears when 1 bit of “1” of the first bit is initially processed. Therefore, after the information indicating the break is recorded, the information indicating the continuation is recorded, and the processing is continued from “1” of the second bit. Then, since “1011” from the 2nd bit to the 5th bit has been processed, an acoustic frame having a level lower than the lower limit value has appeared. Therefore, after recording the information indicating the break, the information indicating the continuation is recorded. The processing is continued from “1” of the sixth bit.

  In the example of FIG. 5, the case where the additional information is recorded in units of bytes has been described for comparison with FIG. 3, but the process shown in FIG. 4 records information indicating whether it is new or continued. Additional information can be recorded in an arbitrary number of bits.

  In the above example, the delimiter information is inserted with the information indicating the delimiter in fixed-length byte units or variable-length word units. However, it is also possible to insert the delimiter information in bit units. In this case, when the acoustic frame reading means 10 extracts the acoustic frame, it extracts an overlapping acoustic frame that overlaps the preceding and following acoustic frames, performs frequency conversion on the overlapping acoustic frame according to [Equation 1], and , [Equation 3] is performed to equalize the low frequency components of the left and right channels. The overlapping sound frames are set so that half of the samples overlap with the preceding and following sound frames. For example, when the preceding sound frame is sample number 1 to 4096 and the subsequent sound frame is sample number 4097 to 8192, the overlapping sound frame set between this is sample number 2049 to 6144. Similarly, for all sections of the acoustic signal, a duplicate acoustic frame is read, and processing for equalizing the low frequency components of the left and right channels is performed.

  As described above, when the overlapping acoustic frame is set and the processing for equalizing the low frequency components of the left and right channels is performed, the low frequency components of the left and right channels are equalized in order to reflect this in the modified acoustic signal. It is necessary to perform a process of performing inverse frequency conversion on the overlapped frame spectrum after processing to obtain a modified overlapped acoustic frame and further connecting to the acoustic frame. In this case, the signal component removed from the acoustic frame by multiplying by the window function W (i) can be compensated by connecting it with the signal of the overlapping acoustic frame. “+ {1−W (i)} · xl (i)” and “+ {1−W (i)} · xr (i)” in the second equation are not necessary. Therefore, in this case, the frequency inverse transformation for the overlapping frame spectrum and the frequency inverse transformation (S109, S209) for the frame spectrum are processed according to the following [Equation 7] instead of the above [Equation 6].

[Formula 7]
xl ′ (i) = 1 / N · {Σj _{= 0,..., N-1} Al (j) · cos (2πij / N) −Σj _{= 0,..., N−1} B1 (j) · sin ( 2πij / N)}
xr ′ (i) = 1 / N · {Σ _{j = 0,..., N−1} Ar (j) · cos (2πij / N) −Σ _{j = 0,..., N−1} Br (j) · sin ( 2πij / N)}

  By the above [Expression 7], each sample xl ′ (i) of the left channel and each sample xr ′ (i) of the right channel of the modified acoustic frame and the modified overlapping acoustic frame are obtained. In S110 and S210, the modified sound frame output means 50 sequentially connects the obtained modified sound frame and the modified overlapping sound frame and outputs them to the output file. As described above, when the sound frame and the overlapping sound frame are read from the sound signal by the sound frame reading means 10, the sound frame and the overlapping sound frame are processed so that the same sample is included in duplicate. Therefore, the acoustic frame output means 50 records the sample read in duplicate in the output file as the sum of the values of the samples.

  Of the modified acoustic signal obtained in this way, for the part where the additional information is embedded, whether the low frequency component exists equally in both channels or is unevenly distributed in one of the channels. There are only three distributions. However, since the high-frequency component remains as the original acoustic signal, it has various distributions based on the settings of the creator from both channels. Human auditory senses directionality with respect to high-frequency components, but it is difficult to sense directionality with respect to low-frequency components. Therefore, even if the low frequency component is biased to one side, it will be heard as if it is a normal acoustic signal for the listener.

(3. Device for extracting information from acoustic signals)
Next, an apparatus for extracting information from an acoustic signal according to the present invention will be described. FIG. 6 is a block diagram showing an embodiment of an apparatus for extracting information from an acoustic signal according to the present invention. In FIG. 6, 100 is an acoustic signal input means, 110 is an acoustic frame acquisition means, 120 is a frequency conversion means, 130 is an interchannel ratio encoding means, and 140 is an additional information extraction means.

  The acoustic signal input unit 100 has a function of acquiring and inputting a flowing sound as a digital acoustic signal. In reality, it is realized by a microphone and an A / D converter. The microphone needs to be a directional microphone that can input sound from the left and right channels. The sound frame acquisition means 110 has a function of reading a predetermined number of samples as one frame from each channel of the input digital stereo sound signal. Therefore, although the same processing as the acoustic frame reading means 10 shown in FIG. 1 is performed, the number of samples to be overlapped with the adjacent acoustic frames is different from that of the acoustic frame reading means 10. The frequency conversion means 120 has the same function as the frequency conversion means 20 shown in FIG. The channel-to-channel ratio encoding unit 130 extracts each low frequency intensity data corresponding to a predetermined frequency or less from the generated plurality of frame spectra, calculates a sum value of each low frequency intensity data for each of the left and right channels, It has a function of outputting a predetermined code based on the inter-channel ratio of the sum value. The additional information extraction unit 140 has a function of converting the ternary array, which is a set of codes output from the inter-channel ratio encoding unit 130, according to a predetermined rule and extracting it as meaningful additional information. Each component shown in FIG. 6 is actually realized by mounting a dedicated program on hardware such as a small computer having an information processing function and its peripheral devices. In particular, in order to achieve the object of the present invention more easily, it is desirable to use a portable terminal device as hardware.

(4. Processing operation of extraction device)
Next, the processing operation of the apparatus for extracting information from the acoustic signal shown in FIG. 6 will be described with reference to the flowchart of FIG. When the user wants to know the attribute information such as the song name of the music that is playing, first, the extraction device is instructed to start as the extraction device. For example, this can be executed by operating a predetermined button when the extraction device is realized by a mobile terminal such as a mobile phone. When an instruction is input to the extraction device, the acoustic signal input unit 100 records the flowing music, digitizes it, and inputs it as a digital acoustic signal. Specifically, the audio input from the left and right sides of the directional microphone is digitized by an A / D converter.

  Subsequently, the acoustic frame acquisition unit 110 reads a predetermined number of samples as one acoustic frame from each channel of the stereo acoustic signal input from the acoustic signal input unit 100 (S310). The number of samples of one acoustic frame read by the acoustic frame acquisition unit 110 needs to be the same as that set by the acoustic frame reading unit 10 shown in FIG. Therefore, in the case of this embodiment, the acoustic frame acquisition unit 110 sequentially reads 4096 samples for each of the left channel and the right channel as acoustic frames.

  Subsequently, the frequency conversion unit 120 and the inter-channel ratio encoding unit 130 determine embedded information from each read sound frame and output a corresponding code (S320). The format of the information to be output is a binary format corresponding to the bit value on the embedding side and a ternary format of values input as delimiters. The process in S320 will be described later.

  If a code corresponding to the bit value is extracted as a result of the processing in S320, the bit value is stored in the buffer (S330). Subsequently, the bit counter is counted up (S340). Then, it is determined whether the bit counter is 8 bits or more (S350). As a result, when the bit counter is 8 bits or more, since the bit value for 1 byte is stored in the buffer, the additional information extraction unit 140 outputs the data for 1 byte in the buffer. (S360). On the other hand, if a value corresponding to the delimiter information is output as a result of the processing in S320, the bit counter is initialized to 0 (S370). By executing the processing shown in FIG. 7 for each acoustic frame, additional information is extracted. If it is determined in S310 that all sound frames have been extracted, the process ends.

  Next, the details of the code output process of S320 will be described with reference to the flowchart of FIG. First, the frequency conversion means 120 performs frequency conversion on each read sound frame to obtain a frame spectrum (S321). This process is the same as the process in the frequency conversion means 20 shown in FIG. Therefore, in the present embodiment, the processing according to the above [Equation 1] is performed, and the real part Al (j), the imaginary part Bl (j) of the conversion data corresponding to the left channel, and the conversion data corresponding to the right channel are processed. Part Ar (j) and imaginary part Br (j) are obtained. At this time, the acoustic signals xl (i) and xr (i) are each weighted with a window function (Hanning window) expressed by W (i) = 0.5−0.5 · cos (2πi / N). The multiplication process is also performed in the same manner as the frequency conversion means 20.

  By the processing in the frequency conversion means 120, a frame spectrum expressed by a spectrum that is a component corresponding to the frequency is obtained. Subsequently, the inter-channel ratio encoding unit 130 extracts each low frequency intensity data in a predetermined frequency range from the generated plurality of frame spectra. The frequency range to be extracted needs to correspond to the embedding device. Therefore, here, low frequency intensity data having a frequency of 200 Hz or less is extracted, and the real part Al (j) and imaginary part Bl (j) calculated by the above [Equation 1] as in the case of the embedding device. , J ≦ 20 are extracted from the real part Ar (j) and the imaginary part Br (j).

  Subsequently, the inter-channel ratio encoding unit 130 extracts the extracted real part Al (j), imaginary part Bl (j), real part Ar (j), and imaginary part Br, similarly to the low-frequency component changing unit 30 of the embedding device. Using (j), the total value E of the low frequency intensity from j = 1 to M (for example, 20) is calculated by the above [Equation 2]. Further, similarly to the low-frequency component changing means 30, it is determined whether or not the total value E is equal to or higher than the level lower limit value (S322).

  When the total value E is equal to or higher than the lower limit of the level, processing according to the following [Equation 8] is executed.

[Formula 8]
El = Cl · Σ _{j = 1,..., M} {Al (j) ² + Bl (j) ² }
Er = Cr · Σ _{j = 1,..., M} {Ar (j) ² + Br (j) ² }

  In the above [Equation 8], Cl and Cr are correction coefficients using calibration data, and are set according to the use environment of the extraction device. In this embodiment, both Cl = Cr = 1 are set. is there. Further, the inter-channel ratio encoding means 130 performs the comparison determination of the calculated El and Er according to the following [Equation 9] (S323), and outputs a code corresponding to the comparison result.

[Formula 9]
When Er / El> 2, the right channel is large. When El / Er> 2, the left channel is large. When Er / El ≦ 2 and El / Er ≦ 2, the left and right are equal.

  The inter-channel ratio encoding unit 130 outputs a ternary code according to the determination result for each acoustic frame. That is, when it is determined that the right channel is large, a first bit value (for example, “1”) is output (S324), and when it is determined that the left channel is large, the second bit value (for example, “0”). ) Is output (S325), and when it is determined that the left and right are equal, a code indicating delimiter information is output (S326).

  In the process of S360, the additional information extracting unit 140 first sets a code indicating equal left and right among the ternary codes output by the inter-channel ratio encoding unit 130 as a delimiter position, and sets the next code as the head. The bit array is created by associating the signs of the right channel size and the left channel size with the bit values. Subsequently, this bit arrangement is converted according to a predetermined rule and extracted as meaningful additional information. As the predetermined rule, various rules can be applied as long as the information intended by the person who embeds the information can be recognized by the person who has received it. As a rule. That is, the additional information extracting unit 140 recognizes the code output from the inter-channel ratio encoding unit 130 in units of 1 byte (8 bits), and recognizes the character information according to the set code system. The character information thus obtained is displayed and output on a screen of a display device (not shown).

  Therefore, if the embedding device embeds the attribute information such as the song title or artist in the sound signal as the character information, the user wants to know the song title or artist by listening to the music being played. If a predetermined operation is performed on the mobile terminal that functions as the extraction device, attribute information such as a song title and an artist is displayed as character information on the screen of the mobile terminal.

  The flowchart in FIG. 7 corresponds to the additional information recorded on the embedding side in byte units. When additional information is recorded in units of words on the embedding side, processing according to the flowchart of FIG. 9 is performed. First, as in S310 of FIG. 7, the acoustic frame acquisition unit 110 reads a predetermined number of samples as one acoustic frame from each channel of the stereo acoustic signal input from the acoustic signal input unit 100 (S401).

  Subsequently, as in S310 of FIG. 7, the frequency conversion unit 120 and the inter-channel ratio encoding unit 130 determine embedded information from each read acoustic frame and output a corresponding code (S402). The details of the processing in S402 are as shown in FIG.

  If a code corresponding to the bit value is extracted as a result of the process in S402, the mode is determined (S403). Two modes, a delimited mode and a bit output mode, are prepared. If it is in the bit output mode, the bit value is stored in the buffer (S408). Subsequently, the bit counter is counted up (S409). On the other hand, if the result of determination in S403 is that the mode is a separation mode, it is further determined whether the extracted code means new or continuation (S404). As a result, if it is new, it means that one word has been completed immediately before, and the additional information extracting means 140 outputs the data for one word recorded in the buffer (S405). Then, the bit counter is initialized to 0 (S406). Further, the mode is set to the bit output mode (S407). If it is determined to continue in S404, the value should be output to the bit in the buffer, so only the processing for setting the bit output mode is performed. If a code corresponding to the delimiter information is extracted in S402, the mode is set to the delimiter mode in order to extract new or continued information from the next sound frame (S410). By executing the processing shown in FIG. 9 for each acoustic frame, additional information is extracted. If it is determined in S401 that all sound frames have been extracted, the process ends.

It is a functional block diagram of an information embedding device for an acoustic signal according to the present invention. It is a flowchart which shows the process outline | summary of the apparatus shown in FIG. FIG. 3 shows how a low-frequency component is changed by processing according to FIG. 2. It is a flowchart which shows the process outline | summary by the other method of the apparatus shown in FIG. FIG. 5 shows how the low frequency component changes due to the processing according to FIG. 4. FIG. 1 is a functional block diagram of an apparatus for extracting information from an acoustic signal according to the present invention. It is a flowchart which shows the process outline | summary of the apparatus shown in FIG. It is a flowchart which shows the detail of the code | symbol output process of S320 of FIG. It is a flowchart which shows the process outline | summary by the other method of the apparatus shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Sound frame reading means 20 ... Frequency conversion means 30 ... Low frequency component change means 40 ... Frequency reverse conversion means 50 ... Modified sound frame output means 60 ... Storage means 61 ... -Acoustic signal storage unit 62 ... Additional information storage unit 63 ... Modified acoustic signal storage unit 70 ... Additional information reading means 100 ... Acoustic signal input means 110 ... Acoustic frame acquisition means 120 ... Frequency conversion means 130 ... Inter-channel ratio encoding means 140 ... Additional information extraction means

Claims (8)

A device that embeds additional information in an inaudible state with respect to a stereo acoustic signal having two left and right channels, each of the left and right channels being composed of a time-series sample sequence,
Additional information reading means for sequentially reading a bit array of additional information to be embedded;
From each of the left and right channels of the acoustic signal, acoustic frame reading means for reading a predetermined number of samples as acoustic frames,
Frequency conversion means for performing frequency conversion on each of the read sound frames and generating a plurality of frame spectra corresponding to the channels;
Extracting low-frequency intensity data corresponding to a component below a predetermined frequency from the generated plurality of frame spectra, a value that each bit value of the read bit sequence of the additional information can take, and a bit sequence of the additional information Low-frequency component changing means for changing the channel-to-channel ratio of the low-frequency intensity data corresponding to each channel to any one of three patterns equivalent to the left channel being large, the right channel being large, and the left and right based on the information indicating the separation. When,
Frequency inverse transform means for performing frequency inverse transform on the plurality of frame spectra including the changed low frequency component, and generating a plurality of modified acoustic frames;
Modified acoustic frame output means for sequentially outputting the generated modified acoustic frames;
An information embedding device for an acoustic signal, comprising: In claim 1,
The low frequency component changing means equalizes the ratio between the left and right channels of the low frequency intensity data of both channels when the sum of the low frequency intensity data corresponding to a predetermined frequency or less does not reach a predetermined level. An information embedding device for an acoustic signal characterized by In claim 2,
The low frequency component changing means, when the sum of the low frequency intensity data does not reach a predetermined level, again, the low frequency intensity for the subsequent frame spectrum from the word leading bit of the bit array of the additional information. An apparatus for embedding information in an acoustic signal, characterized by performing processing for changing a ratio between data channels. In claim 2,
The low frequency component changing means, when the interchannel ratio of the low frequency intensity data of the additional information is equal to the left and right with respect to the immediately preceding frame spectrum, the word of the bit array of the additional information for the subsequent frame spectrum In order to indicate whether it is the beginning or the middle of a word, the ratio between channels of the low frequency intensity data is changed to a pattern of either a large left channel or a large right channel. An information embedding device for acoustic signals. In claim 1,
The acoustic frame reading means is for reading an overlapping acoustic frame that is overlapped by ½ of the length of the acoustic frame when reading each acoustic frame,
The frequency converting means performs frequency conversion on the overlapping acoustic frame, and generates a plurality of overlapping frame spectra corresponding to the channels,
The low frequency component changing means extracts each low frequency intensity data corresponding to a predetermined frequency or less from the generated overlapping frame spectrum, and equalizes the ratio of the low frequency intensity data of both channels,
The frequency inverse transform means performs frequency inverse transform on the plurality of overlapping frame spectra including the changed low frequency intensity data, and generates a plurality of modified overlapping acoustic frames,
The apparatus for embedding information in an acoustic signal, wherein the modified acoustic frame output means connects and outputs the modified acoustic frame and the modified overlapping acoustic frame according to a sample order in the original acoustic signal. A method of embedding additional information in an inaudible state with respect to a stereo acoustic signal having two left and right channels, each of the left and right channels being composed of a time-series sample sequence,
An additional information reading stage for sequentially reading a bit array of additional information to be embedded;
A sound frame reading step for reading a predetermined number of samples as sound frames from the left and right channels of the sound signal,
Performing a frequency conversion on each of the read sound frames to generate a plurality of frame spectra corresponding to the channels;
Extracting low-frequency intensity data corresponding to a component below a predetermined frequency from the generated plurality of frame spectra, a value that each bit value of the read bit sequence of the additional information can take, and a bit sequence of the additional information A low-frequency component changing step of changing the channel-to-channel ratio of the low-frequency intensity data corresponding to each channel to any one of three patterns equivalent to the left channel is large, the right channel is large, and left and right based on the information indicating the delimiter. When,
A frequency inverse transform stage that performs frequency inverse transform on the plurality of frame spectra including the modified low frequency component to generate a plurality of modified acoustic frames;
A modified sound frame output step of sequentially outputting the generated modified sound frames;
A method for embedding information in an acoustic signal, comprising: An apparatus for extracting additional information embedded in an inaudible state in advance from a stereo sound signal having two left and right channels, each of the left and right channels being composed of a time-series sample sequence,
Sound frame acquisition means for digitizing a predetermined section of each channel of the sound signal and acquiring a plurality of sound frames composed of a predetermined number of samples corresponding to each channel;
Frequency conversion means for performing frequency conversion on each acquired acoustic frame and generating a plurality of frame spectra corresponding to each channel;
Extract low frequency intensity data corresponding to a component below a predetermined frequency from the generated plurality of frame spectra, calculate the total value of each low frequency intensity data for each channel, and the inter-channel ratio of the total value is An inter-channel ratio encoding means for outputting a ternary code corresponding to each state based on whether the left and right are equal, the left channel is large, or the right channel is large;
When the output code is information indicating that the ratio between channels is uniform, a bit array composed of other binary codes excluding the code is converted according to a predetermined rule. Additional information extracting means for extracting additional information;
An apparatus for extracting information from an acoustic signal, comprising: A method of extracting additional information embedded in an inaudible state in advance from a stereo acoustic signal having two left and right channels, each of which is composed of a time-series sample sequence,
A sound frame acquisition step of digitizing a predetermined section of each channel of the sound signal and acquiring a plurality of sound frames composed of a predetermined number of samples corresponding to each channel;
Performing a frequency conversion on each acquired acoustic frame to generate a plurality of frame spectra corresponding to each channel;
Extract low frequency intensity data corresponding to a component below a predetermined frequency from the generated plurality of frame spectra, calculate the total value of each low frequency intensity data for each channel, and the inter-channel ratio of the total value is An inter-channel ratio encoding step for outputting a ternary code corresponding to each state based on whether the left and right are equal, the left channel is large, or the right channel is large;
When the output code is information indicating that the ratio between channels is uniform, a bit array composed of other binary codes excluding the code is converted according to a predetermined rule. An additional information extraction stage for extracting additional information;
A method for extracting information from an acoustic signal.

Images