JP2007048340A - Device for extracting information from acoustic signal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device for extracting information from an acoustic signal by which additional information such as attribute information embedded in the state of inhibiting listening can more correctly be extracted while suppressing the load of a calculation amount regarding the acoustic signals of a plurality of channels provided in CD or broadcasting. <P>SOLUTION: The sample group of a predetermined number of samples is set as a reference frame from an input acoustic signal, an optimal phase frame is decided therefrom after this reference frame and a phase changed frame are set, and a bit value is detected from the optimal phase frame. When the optimal phase frame fluctuates to return again, an offset value is added to set a reference frame and a phase changed frame for a next sample group. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to the protection of music copyright (monitoring illegal copying) in the field of package music for viewing for consumer and business use using CDs and DVDs, and the field of broadcasting and network music distribution distributed for commercial purposes by broadcasters and the like. ) And provision of music attribute information (music title search service), text information provision service linked to exhibition explanation narration in the fields, museums, event venues, information such as URLs from broadcast programs and audio signals of CD / DVD packages The present invention relates to a non-contact Internet gateway service field in which a mobile phone is used to access a web site related to a predetermined content and extract detailed information or answer a questionnaire.

  As a service to provide music attribute information that allows you to know the titles of music that has been played recently, you can query the broadcast station for the date and time of the broadcast music, and record music fragments that are being played on mobile phones. Services that collate with melodies registered in the database have been put into practical use (see, for example, Patent Documents 1 and 2).

  In the inventions described in Patent Documents 1 and 2, since the recorded music fragments are checked against the melodies registered in the database, the processing load increases as the number of songs registered in the database increases, and similar melodies are mistaken. The possibility of judging increases. Therefore, a method of embedding music attribute information such as a song name and artist information as an inaudible digital watermark in an acoustic signal has also been proposed (see, for example, Patent Documents 3 to 6).

In the methods described in Patent Documents 1 to 6, the amount of information that can be embedded is small, the sound quality is deteriorated to some extent, watermark information is lost by various signal processing, and watermark detection is difficult for analog copies. There is a problem. Therefore, the present applicant proposes a method of embedding attribute information by changing the ratio of the low frequency component of the sound signal having a plurality of channels according to the bit value of the attribute information, and the sound embedded in this way. In order to extract attribute information from a signal with high accuracy, a method of analyzing by changing the phase has been proposed (see Patent Documents 7 and 8).
JP 2002-259421 A Japanese Patent Laid-Open No. 2003-157087 JP-A-11-145840 JP-A-11-219172 Japanese Patent No. 3321767 JP 2003-99077 A Japanese Patent Application No. 2005-5157 Japanese Patent Application No. 2005-51381

  However, in the method described in Patent Document 8, since the phase is changed step by step, the determination of the optimum phase is unstable if the beginning of the signal captured on the extraction side deviates near the middle of each step. Thus, there are problems such as extremely low detection accuracy and impossible detection. As a countermeasure, it is conceivable to make the phase correction step fine, but there is a problem that the calculation time is increased and the real-time extraction process becomes difficult, and the calculation error of the optimum phase determination is increased and the determination accuracy is not improved. .

  Further, in the method described in Patent Document 8, when the intensity of the low frequency component is smaller than the threshold value, the bit value is not detected. However, the signal level suddenly fluctuates or the signal level temporarily decreases. There is also a problem that when a silent state occurs, it cannot be properly determined.

  Therefore, the present invention more accurately extracts additional information such as attribute information embedded in an inaudible state from a plurality of channels of audio signals provided on a CD or broadcast while suppressing the burden of calculation amount. It is an object of the present invention to provide an apparatus for extracting information from an acoustic signal. In addition, the present invention provides an apparatus for extracting information from an acoustic signal that can appropriately extract load information even when a sudden change in signal level, temporary signal level drop, or silence occurs. The task is to do.

  In order to solve the above problems, the present invention is an apparatus for extracting information embedded in an inaudible state in advance from an acoustic signal composed of a time-series sample sequence, wherein a predetermined number of information is extracted from the acoustic signal. An acoustic frame acquisition means for acquiring an acoustic frame composed of samples; a reference frame setting means for setting a reference frame by changing the phase by moving the acoustic frame by an offset value for a predetermined number of samples; and the reference A phase change frame setting means for setting a plurality of acoustic frames set by changing a phase by moving samples from a frame corresponding to a step value that is a value larger than the offset value as a phase change frame; and the reference Frequency conversion for each acoustic frame set as frame and phase change frame Performing frequency conversion means for generating a frame spectrum corresponding to each of the acoustic frames, extracting low frequency intensity data corresponding to a component of a predetermined frequency or less from the generated frame spectrum, and based on the low frequency intensity data Based on the code determination parameter calculating means for calculating the code determination parameter and the code determination parameter calculated in the same in-phase acoustic frame having a different reference frame, one of the reference frame and the plurality of phase change frames is selected. The acoustic frame is determined to be the optimal phase frame having the optimum phase, and a predetermined code is output based on the code determination parameter determined for the optimal phase frame. The phase is different from that of the frame. Code output means for changing the offset value when the two are the same, and a bit array composed of codes output for each optimum phase frame is converted and added according to a predetermined rule An apparatus for extracting information from an acoustic signal having additional information extracting means for extracting information is provided.

  In the present invention, the code output means determines that the acoustic frame is an invalid frame when the sum of low frequency intensity data extracted from the generated frame spectrum is less than a predetermined lower threshold. The lower threshold used for the determination is calculated by adding low-frequency intensity data that has been determined as valid frames in the past, and the addition of the low-frequency intensity data is far away in time. In addition, the past effective frames are characterized by reducing the influence thereof.

  According to the present invention, when the acquired acoustic signal is analyzed in a predetermined acoustic frame unit, the position of the reference frame is determined based on the change in the optimum phase in the past acoustic frame, and the phase is determined with respect to the reference frame. The optimal phase is determined for each acoustic frame while shifting, and the optimal phase is determined by minimizing the computation load to determine the embedded information based on the state of the acoustic frame determined to be the optimal phase. The additional information embedded in an inaudible state can be accurately extracted from the reproduced sound signal.

  Further, in the present invention, when calculating the threshold value of the low frequency intensity necessary for extracting the bit value, the influence of the past frame farther away in time is reduced, so that the signal level is rapidly increased. It is possible to appropriately extract additional information even when fluctuations, a temporary signal level decrease, or a silent state occurs.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(1. Embedding device)
First, an embedding device for embedding additional information to be extracted in an acoustic signal by an apparatus for extracting information from an acoustic signal according to the present invention will be described. FIG. 1 is a functional block diagram showing the configuration of the embedding device. In FIG. 1, 10 is an acoustic frame reading means, 20 is a frequency converting means, 30 is a low frequency component changing means, 40 is a frequency inverse converting means, 50 is a modified acoustic frame output means, 60 is a storage means, and 61 is an acoustic signal storage. , 62 is an additional information storage unit, 63 is a modified acoustic signal storage unit, and 70 is an additional information reading means.

  The sound frame reading means 10 has a function of reading a predetermined number of samples as one frame from each channel of the original stereo sound signal to be embedded with additional information. The frequency conversion means 20 has a function of generating a frame spectrum by frequency-converting the frame of the acoustic signal read by the acoustic frame reading means 10 by Fourier transformation or the like. The low frequency component changing means 30 extracts each low frequency intensity data corresponding to a predetermined frequency or less from a plurality of generated frame spectra, and responds between channels based on the additional information extracted from the additional information storage unit 62. It has a function of changing the inter-channel ratio of low frequency intensity data. The frequency reverse conversion means 40 has a function of generating a plurality of modified sound frames by performing frequency reverse conversion on a plurality of frame spectra including the changed low frequency intensity data. The modified sound frame output means 50 has a function of sequentially outputting the generated modified sound frames. The storage means 60 includes an acoustic signal storage unit 61 that stores a stereo acoustic signal to be embedded with additional information, an additional information storage unit 62 that is configured as a bit array and stores additional information embedded in the stereo acoustic signal, and an additional information It has a modified acoustic signal storage unit 63 for storing the modified acoustic signal after information is embedded, and stores various information necessary for other processing. The additional information reading means 70 has a function of extracting additional information from the additional information storage unit 62. The additional information is information that should be added to the sound information and embedded, and includes attribute information such as a title and artist name, and other information other than the attribute information. Each component shown in FIG. 1 is actually realized by installing a dedicated program in hardware such as a computer and its peripheral devices. That is, the computer executes the contents of each means according to a dedicated program.

(2. Processing operation of embedded device)
Next, the processing operation of the embedding apparatus shown in FIG. 1 will be described according to the flowchart of FIG. First, the additional information reading means 70 reads additional information from the additional information storage unit 62 in units of one word (S101). Specifically, one word is read into the register. Subsequently, the mode is set to the separation mode (S102). There are three types of modes: separation mode, bit mode, and continuous identification mode. The delimiter mode indicates a mode for performing processing in a delimiter in units of one word, and the bit mode indicates a mode for performing processing based on the value of each bit of one word. When one byte is read from the additional information storage unit 62, the separator mode is always set immediately after that.

  Subsequently, the acoustic frame reading means 10 reads a predetermined number of samples as one acoustic frame from each of the left and right channels of the stereo acoustic signal stored in the acoustic signal storage unit 61 (S104). The number of samples of one sound frame read by the sound frame reading means 10 can be set as appropriate, but is desirably about 4096 samples when the sampling frequency is 44.1 kHz. Therefore, the acoustic frame reading means 10 sequentially reads 4096 samples for each of the left channel and the right channel as acoustic frames.

  Subsequently, the frequency conversion means 20 performs frequency conversion on each read sound frame to obtain a frame spectrum that is a spectrum of the sound frame (S105). Specifically, each acoustic frame is performed using three window functions. As frequency conversion, Fourier transform, wavelet transform, and other various known methods can be used. In the present embodiment, a case where Fourier transform is used will be described as an example.

  Here, a case where general Fourier transform is performed will be described. When Fourier transform is performed on a predetermined signal, it is necessary to divide the signal into predetermined lengths. In this case, if Fourier transform is performed on a signal of a predetermined length as it is, the delimiter part is discontinuous. become. Therefore, in general, when performing Fourier transform, a signal value is changed using a window function called a Hanning window, and then Fourier transform is performed on the changed value.

  Here, FIG. 3 shows how the signal waveform changes when general Fourier transform is performed. In FIG. 3, the horizontal axis is the time axis (t), and FIGS. 3 (a) to 3 (c) all correspond to each other. In FIGS. 3A and 3C, the vertical axis indicates the amplitude value (level) of the signal. In FIG. 3B, the vertical axis indicates the value of the window function W (t). Note that W (t) = 0.5−0.5 · cos (2πt / T), and the maximum value of W (t) is 1.

  In the case of a general Fourier transform, a signal having a predetermined length as shown in FIG. 3A is multiplied by a window function W (t) as shown in FIG. Into a signal as shown in Then, the Fourier transform is executed with a signal having a waveform as shown in FIG.

  In this embodiment, in order to create a plurality of states for embedding information from one acoustic frame, not to remove the discontinuity after Fourier transform, a plurality of window functions are prepared, and one acoustic frame is created. On the other hand, Fourier transform is performed using each window function to obtain a plurality of spectra. Here, FIG. 4 shows a signal waveform of one acoustic frame and a plurality of window functions W (1, i), W (2, i), and W (3, i). In FIG. 4, the horizontal axis is the time axis (i). As described later, i is a serial number assigned to N samples in each acoustic frame, and is proportional to time t. In FIGS. 4A, 4E, 4F, and 4G, the vertical axis indicates the amplitude value (level) of the signal. 4B to 4D, the vertical axis indicates the values of the window functions W (1, i), W (2, i), W (3, i), and W (1, i), W The maximum values of (2, i) and W (3, i) are both 1.

  The first window function W (1, i) is for extracting the front part of the acoustic frame. As shown in FIG. 4B, the first window function W (1, i) has a maximum value of 1 at the position of the predetermined sample number i. And the rear part is set to have a minimum value of 0. Which sample number has the maximum value depends on the design of the first window function W (1, i). By multiplying the first window function W (1, i), the signal waveform of the acoustic frame as shown in FIG. 4A has a signal component remaining in the front part as shown in FIG. The signal component is deleted and becomes a Fourier transform target. The second window function W (2, i) is for extracting the central portion of the acoustic frame, and as shown in FIG. 4C, at the position of the predetermined sample number i in the central portion. The maximum value is 1, and the minimum value is set to 0 at the front and rear portions. By multiplying by the second window function W (2, i), the signal waveform of the acoustic frame as shown in FIG. 4A has a signal component remaining in the center as shown in FIG. The signal components of the rear part and the rear part are deleted, and this becomes a Fourier transform target. The third window function W (3, i) is for extracting the rear part of the acoustic frame. As shown in FIG. 4 (d), the front part has a minimum value of 0, and the rear part has a predetermined value. The maximum value 1 is set at the position of the sample number i. By multiplying the third window function W (3, i), the signal waveform of the acoustic frame as shown in FIG. 4A is deleted as shown in FIG. The signal component remains in the rear part, and this becomes a Fourier transform target. Thus, after extracting the front part, the central part, and the rear part, the Fourier transform is executed, so that spectra corresponding to the front part, the central part, and the rear part are obtained. In order to embed a bit value in one acoustic frame, the bit value is originally divided into two parts, a front part and a rear part. However, on the extraction side, it is not always possible to read a signal synchronously. Therefore, in order to clearly distinguish the front portion and the rear portion, the present invention also extracts the signal at the center portion.

  Specifically, each window function has a window separation function Wc (i) for performing window division based on the Hanning window function W (i) = 0.5−0.5 · cos (2πi / N) as follows. [Formula 1] and defined using this window separation function Wc (i).

[Formula 1]
When 3N / 8 ≦ i ≦ 5N / 8, Wc (i) = 0.5−0.5 · cos (8π (i−3N / 8) / N)
When i <3N / 8 or i> 5N / 8, Wc (i) = 0.0

  Then, using this window separation function Wc (i), the first window function W (1, i), the second window function W (2, i), and the third window function W (3, i) are It is defined as shown in [Formula 2].

[Formula 2]
W (2, i) = W (i) · Wc (i)
When i ≦ N / 2, W (1, i) = {1−Wc (i)} W (i), W (3, i) = 0.0
When i> N / 2, W (1, i) = 0.0, W (3, i) = {1-Wc (i)} W (i)

  As can be seen from FIG. 4 and [Formula 1] and [Formula 2], the first window function, the second window function, and the third window function are obtained by dividing the Hanning window function on the time axis in one acoustic frame. . Of these, the first window function and the second window function, and the second window function and the third window function are such that there are places where both have non-zero values at the same time, that is, on the time axis. Set to overlap each other. Further, the first window function and the third window function are set so that when one has a non-zero value, the other always becomes zero, that is, does not overlap each other on the time axis. The first window function and the third window function are set so that both sides thereof have asymmetric cosine functions, as shown in FIGS. This is to make the difference between the result processed through the first window function and the result processed through the third window function clear, and the processed acoustic frame is shifted on the time axis. However, it is possible to prevent the analysis by the inverse window functions.

  When the Fourier transform is performed in S105, specifically, the left channel signal xl (i) and the right channel signal xr (i) (i = 0,..., N−1) are in accordance with the above [Equation 2]. Using the first window function W (1, i), the second window function W (2, i), and the third window function W (3, i), which are the three window functions, Processing is performed in accordance with the real part Al (1, j), Al (2, j), Al (3, j), imaginary part Bl (1, j), Bl (2,2) of the conversion data corresponding to the left channel. j), Bl (3, j), real part Ar (1, j), Ar (2, j), Ar (3, j), imaginary part Br (1, j), corresponding to the right channel, Br (2, j) and Br (3, j) are obtained. Note that the window functions W (1, i), W (2, i), and W (3, i) are functions whose values increase near the front (front), near the center, and near the rear of the acoustic frame, respectively. ing.

[Formula 3]
Al (1, j) = Σi _{= 0,..., N-1} W (1, i) · xl (i) · cos (2πij / N)
Bl (1, j) = Σi _{= 0,..., N-1} W (1, i) · xl (i) · sin (2πij / N)
Al (2, j) = Σi _{= 0,..., N-1} W (2, i) · xl (i) · cos (2πij / N)
Bl (2, j) = Σ i = 0, ..., N-1 W (2, i) · xl (i) · sin (2πij / N)
Al (3, j) = Σi _{= 0,..., N-1} W (3, i) · xl (i) · cos (2πij / N)
Bl (3, j) = Σi _{= 0,..., N-1} W (3, i) · xl (i) · sin (2πij / N)
Ar (1, j) = Σi _{= 0,..., N-1} W (1, i) .xr (i) .cos (2πij / N)
Br (1, j) = Σ _{i = 0,..., N−1} W (1, i) · xr (i) · sin (2πij / N)
Ar (2, j) = Σi _{= 0,..., N-1} W (2, i) .xr (i) .cos (2πij / N)
Br (2, j) = Σi _{= 0,..., N-1} W (2, i) .xr (i) .sin (2πij / N)
Ar (3, j) = Σi _{= 0,..., N-1} W (3, i) .xr (i) .cos (2πij / N)
Br (3, j) = Σi _{= 0,..., N-1} W (3, i) .xr (i) .sin (2πij / N)

  In the above [Expression 3], i is a serial number assigned to N samples in each acoustic frame, and takes an integer value of i = 0, 1, 2,... N−1. Further, j is a serial number assigned in order from the smallest value of the frequency value, and takes an integer value of j = 0, 1, 2,... N−1 similarly to i. When the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by 10.8 Hz.

  By executing the processing according to the above [Equation 3], a frame spectrum in which the signal component of each acoustic frame is expressed by a spectrum corresponding to the frequency is obtained. Subsequently, the low frequency component changing unit 30 extracts a spectrum set of three predetermined frequency ranges from the generated frame spectrum. Human hearing is known to be less sensitive to directionality for low frequency components up to about 200-300 Hz (Corona Corp., issued October 30, 1990, "Sound Engineering Course 1. Basic Acoustics"). Engineering, Acoustical Society of Japan ”p.247 (see FIGS. 9 and 26). Therefore, in this embodiment, the low frequency component is about 200 Hz or less. In the vicinity of a frequency of 200 Hz, j corresponds to 20, so the real part Al (1, j), Al (2, j), Al (3, j), and imaginary part Bl ( 1, j), Al (2, j), Al (3, j), real part Ar (1, j), Ar (2, j), Ar (3, j), imaginary part Br (1, j) , Br (2, j), Br (3, j), j ≦ 20 are extracted.

Subsequently, the low frequency component changing means 30 extracts the extracted real part Al (1, j) and the like (the same applies to other real parts and imaginary parts including (2, j) (3, j)), and the imaginary part. Among the real parts Ar (1, j), etc., the imaginary parts Br (1, j), etc., among the real parts Al (1, j), etc., the imaginary parts Bl (1, j), etc. ) And the like, the total value E ₁ and the total value E ₂ are calculated by the following [Equation 4].

[Formula 4]
E ₁ = Σj _{= 1,..., M-3} {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² }
E ₂ = Σ _{j = 1,..., M-3} {Al (3, j) ² + Bl (3, j) ² + Ar (3, j) ² + Br (3, j) ² }

E ₁ calculated by the above [Equation 4] indicates the sum of the component intensities of the spectrum set near the front of the acoustic frame, and E ₂ indicates the sum of the component intensities of the spectrum set near the rear of the acoustic frame. Subsequently, it is determined whether or not the combined values E ₁ and E ₂ are equal to or higher than the level lower limit value Lev. The level lower limit value Lev is set to 0.05 when the maximum amplitude value of the acoustic signals xl (i) and xr (i) is normalized to 1 and M = 20 is set. Under this condition, the level at which the resistance to analog conversion can be maintained empirically is “0.5”. However, in the present invention, as will be described later, the digital watermark extraction / collation means 200 performs collation, so It is necessary to have. Therefore, in this embodiment, it is set to 0.05, which is about 1/10 of the level at which the resistance to analog conversion can be maintained. In the above [Equation 4], the range for calculating the sum is from 1 to M-3, and M-2, M-1, and M are excluded, as will be described later. This is to make the separation clear.

The reason why it is determined whether or not the combined values E ₁ and E ₂ are equal to or greater than the level lower limit value Lev is that if the signal strength is small, even if the signal is changed, the change cannot be detected on the extraction side. is there. In this embodiment, when the bit value that can take the first value (for example, “1”) and the second value (for example, “0”) is “1”, the bit value is “0” for the three components of the window. In the case of "," the window 1 component is embedded. Therefore, when the bit value to be embedded is “1”, the sum value E ₁ is less than the lower limit value Lev, and when the bit value to be embedded is “0”, the sum value E ₂ is less than the level lower limit value Lev. The mode is set to the separation mode without recording according to the bit value of the additional information (S106). On the other hand, when the bit value to be embedded is “1” and the total value E ₁ is greater than or equal to the level lower limit value Lev, or when the bit value to be embedded is “0” and the total value E ₂ is greater than or equal to the level lower limit value Lev, the mode is determined. It will be.

  When the mode is the separation mode, the low-frequency component changing unit 30 performs a process for equalizing the window 1 component and the window 3 component (including the case where all are 0) in the left (L) channel signal (S108). . Specifically, according to the following [Equation 5], processing for setting both L side to 0 is executed. In this case, the window 1 component and the window 3 component of the right (R) channel signal are not necessarily equal.

[Formula 5]
For j = 1 to M,
Al ′ (1, j) = 0
Bl ′ (1, j) = 0
Al ′ (3, j) = 0
Bl ′ (3, j) = 0
In the case of stereo, the following corresponding to the right signal is also calculated: E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^{1 / 2}
Ar ′ (1, j) = Ar (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Br ′ (1, j) = Br (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
E (3, j) = {Al (3, j) ² + B1 (3, j) ² + Ar (3, j) ² + Br (3, j) ² } ^1/2
Ar ′ (3, j) = Ar (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2
Br ′ (3, j) = Br (3, j) · E (3, j) / {Ar (3, j) ² + Br (1, j) ² } ^1/2

  By executing the processing according to the above [Equation 5], the low frequency component of the left channel frame spectrum is the same when both the window 1 component and the window 3 component are “0”. The pattern in which the window 1 component and the window 3 component are equal is information indicating the head position (separation) of the additional information. In the above [Equation 5], Al ′ (j) = Bl ′ (j) = 0 is set for both the window 1 component and the window 3 component, but the purpose is to make it possible to recognize the separation on the extraction side. Therefore, if the value is sufficiently small, it is not always necessary to set it to zero. Further, the window 1 component and the window 3 component do not necessarily have to be the same, and it is sufficient that the difference is small. In this sense, the term “equal” is used here.

  On the other hand, when the mode is the bit mode or the continuous identification mode, the low-frequency component changing unit 30 determines the window 1 component of the left channel signal and the bit value of the bit array of the additional information extracted from the additional information storage unit 62. Processing is performed to change the ratio of the spectral intensity of the three window components to a state in which the first window component is dominant or the third window component is dominant (S107). Here, “dominant” indicates that the spectral intensity in the spectrum set of one window component is larger than the spectral intensity in the spectrum set of the other window component. Therefore, in S107, the window 1 component is obtained by executing the processing according to any of the following [Equation 6] and [Equation 7] according to the bit value that can take the first value and the second value. The magnitude relationship between the spectral intensity of the window 3 and the spectral intensity of the window 3 component is changed, and processing is performed to change the window 1 component to dominant or the window 3 component to dominant. For example, when the first value is 1 and the second value is 0, when the bit value is 1, the processing according to the following [Equation 6] is executed for the window 1 component. In the case of the continuous identification mode, if it is new, the magnitude relationship between the spectral intensity of the window 1 component and the spectral intensity of the window 3 component is changed according to [Equation 6], and the window 3 component is predominated. In some cases, the window 1 component is changed according to [Equation 7].

[Formula 6]
For j = 1 to M, Al ′ (1, j) = 0
Bl ′ (1, j) = 0
In the case of stereo, the following corresponding to the right signal is also calculated: E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^{1 / 2}
Ar ′ (1, j) = Ar (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Br ′ (1, j) = Br (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
E (3, j) = {Al (3, j) ² + B1 (3, j) ² + Ar (3, j) ² + Br (3, j) ² } ^1/2
Al ′ (3, j) = Al (3, j) · E (3, j) / {Al (3, j) ² + Bl (3, j) ² } ^1/2
Bl ′ (3, j) = B1 (3, j) · E (3, j) / {Al (3, j) ² + Bl (3, j) ² } ^1/2

  In the above [Expression 6], in the last three expressions, the corresponding frequency component of the right channel signal is added to the window 3 component of the left channel signal in which information is embedded to obtain Al ′ (3, j), Bl ′ (3, j) is obtained. Thereby, the magnitudes of the strengths of the window 3 component and the other window components are clarified, and information can be easily extracted on the extraction side.

  In this case, the processing according to the following [Equation 7] is executed for some of the frequency components of the window 3 component.

[Formula 7]
For three components j = M−2, M−1 and M, Al ′ (3, j) = 0
Bl ′ (3, j) = 0
In the case of stereo, the following corresponding to the right signal is also calculated. E (3, j) = {Al (3, j) ² + Bl (3, j) ² + Ar (3, j) ² + Br (3, j) ² } ^{1 / 2}
Ar ′ (3, j) = Ar (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2
Br ′ (3, j) = Br (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2

  When the bit value is 0, the processing according to the following [Equation 8] is executed for the three window components.

[Formula 8]
For each component of j = 1 to M, Al ′ (3, j) = 0
Bl ′ (3, j) = 0
In the case of stereo, the following corresponding to the right signal is also calculated. E (3, j) = {Al (3, j) ² + Bl (3, j) ² + Ar (3, j) ² + Br (3, j) ² } ^{1 / 2}
Ar ′ (3, j) = Ar (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2
Br ′ (3, j) = Br (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2
E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^1/2
Al ′ (1, j) = Al (1, j) · E (1, j) / {Al (1, j) ² + Bl (1, j) ² } ^1/2
Bl ′ (1, j) = Bl (1, j) · E (1, j) / {Al (1, j) ² + Bl (1, j) ² } ^1/2

  In the above [Equation 8], as in the above [Equation 6], E (1, j), Al ′ (1, j), and Bl ′ (1, j) are changed using the last three equations. Calculated. In this way, by adding the corresponding frequency component of the right channel signal to the window 1 component of the left channel signal in which the information is embedded, Al ′ (1, j) and Bl ′ (1, j) are obtained, The magnitudes of the strengths of the window 1 component and the other window components are clarified, and information can be easily extracted on the extraction side.

  In this case, the processing according to the following [Equation 9] is executed for some of the frequency components of the window 1 component.

[Formula 9]
For three components j = M−2, M−1 and M, Al ′ (1, j) = 0
Bl ′ (1, j) = 0
In the case of stereo, the following corresponding to the right signal is also calculated: E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^{1 / 2}
Ar ′ (1, j) = Ar (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Br ′ (1, j) = Br (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2

  As a result of performing the processing according to the above [Equation 8] and [Equation 9], a value of “0” is obtained at j = M−2, M−1, and M of one component of the window, but signals other than the predetermined value are obtained at other times. Ingredients will be present. Therefore, in this case, the ratio of the spectrum intensity is changed so that the window 1 component is dominant.

  By executing the processing according to any one of [Formula 6] and [Formula 7], or [Formula 8] and [Formula 9], the left channel signal is determined according to each bit value of the bit array of the additional information. Thus, the pattern is changed so that the window 1 component is dominant or the window 3 component is dominant.

  In this case, there is always a portion where the signal component is “0” between the high frequency band and the low frequency band, thereby preventing the signal components of the high frequency band and the low frequency band from being mixed. Eventually, the low frequency component changing means 30 performs processing based on [Equation 5] in the case of the separation mode in S108, and [Equation 6], [Equation 7] or [Equation 8] [Equation 8] in the bit mode or the continuous identification mode. The processing based on Equation 9] is performed in S107.

  In either case of S107 and S108, the low-frequency component changing unit 30 then deletes the two window components (S109). Specifically, the processing according to the following [Formula 10] is executed for the two components of the window.

[Formula 10]
For each component of j = 1 to M, Al ′ (2, j) = 0
Bl ′ (2, j) = 0
In the case of stereo, the following corresponding to the right signal is also calculated: E (2, j) = {Al (2, j) ² + Bl (2, j) ² + Ar (2, j) ² + Br (2, j) ² } ^{1 / 2}
Ar ′ (2, j) = Ar (2, j) · E (2, j) / {Ar (2, j) ² + Br (2, j) ² } ^1/2
Br ′ (2, j) = Br (2, j) · E (2, j) / {Ar (2, j) ² + Br (2, j) ² } ^1/2

  Next, the frequency inverse transform means 40 performs the process of obtaining the modified acoustic frame by performing the frequency inverse transform on the frame spectrum in which the ratio between the spectrum sets of the window components is changed by the processes of S107 to S109 (S110). As a matter of course, this frequency inverse transform needs to correspond to the method executed by the frequency transform unit 20 in S105. In the present embodiment, since the frequency transform unit 20 performs the inverse Fourier transform, the frequency inverse transform unit 40 performs the inverse Fourier transform. Specifically, the real part Al ′ (1, j), etc., the imaginary part Bl ′ (1, j), etc. of the left channel of the spectrum obtained by any of the above [Formula 5] to [Formula 10], right Using the real part Ar ′ (1, j) of the channel, the imaginary part Br ′ (1, j), etc., processing according to the following [Equation 11] is performed, and xl ′ (i), xr ′ (i ) Is calculated. For frequency components not processed in the above [Formula 5] to [Formula 10], Al ′ (1, j), etc., Bl ′ (1, j), etc., Ar ′ (1, j), etc., Br, etc. As ′ (1, j) etc., Al (1, j) etc., Bl (1, j) etc., Ar (1, j) etc., Br (1, j) etc. are used.

[Formula 11]
xl' (i) = 1 / N · {Σ j Al' (1, j) · cos (2πij / N) -Σ j Bl' (1, j) · sin (2πij / N)} + 1 / N · { Σ _j Al ′ (2, j) · cos (2πij / N) −Σ _j Bl ′ (2, j) · sin (2πij / N)} + 1 / N · {Σ _j Al ′ (3, j) · cos (2πij / N) −Σ _j Bl ′ (3, j) · sin (2πij / N)} + xlp (i + N / 2)
xr' (i) = 1 / N · {Σ j Ar' (1, j) · cos (2πij / N) -Σ j Br' (1, j) · sin (2πij / N)} + 1 / N · { _{Σ j Ar' (2, j)} · cos (2πij / N) -Σ j Br' (2, j) · sin (2πij / N)} + 1 / N · {Σ j Ar' (3, j) · cos _{(2πij / N) -Σ j Br'} (3, j) · sin (2πij / N)} + xrp (i + N / 2)

In the above [Expression 11], Σ _{j = 0,} ... _{, N−1} is shown as Σ _{j in} order to prevent the expression from becoming complicated.

  The terms “+ xlp (i + N / 2)” in the first equation and “+ xrp (i + N / 2)” in the second equation in the above [Equation 11] are the data xlp (i) of the modified acoustic frame modified immediately before, When xrp (i) exists, the addition is performed in consideration of overlapping of N / 2 samples on the time axis. By the above [Equation 11], each sample xl ′ (i) of the left channel and each sample xr ′ (i) of the right channel of the modified acoustic frame are obtained. The modified sound frame output means 50 sequentially outputs the obtained modified sound frames to the output file (S111). When the processing for one acoustic frame is completed in this manner, the mode is determined (S112). If the mode is the separation mode, the mode is set to the continuous identification mode (S113), and then the acoustic frame reading means 10 A frame is read (S104). On the other hand, when the mode is the bit mode or the continuous identification mode, after the mode is set to the bit mode (S114), the low frequency component changing means 30 reads the next bit in the bit array of the additional information (S103). The above processing is executed over all samples of both channels of the acoustic signal. That is, a predetermined number of samples are read as sound frames, and when there are no more sound frames to be read from the sound signal (S104), the process ends. When the processing corresponding to each bit of 1-word data read in S101 is completed, the process returns from S103 to S101, and the next word of the additional information is read. When the processing is completed for all the words of the additional information, the processing is returned to the first word of the additional information. As a result, all modified acoustic frames that have been processed for all acoustic frames are recorded in the output file and obtained as modified acoustic signals. The obtained modified acoustic signal is output to and stored in the modified acoustic signal storage unit 63 in the storage unit 60.

The change of the left channel signal by the above processing will be described with reference to FIG. In FIG. 5, the horizontal direction in the drawing is the time axis and is proportional to the number of samples. In addition, a large number of rectangles in the figure indicate the window 1 component and the window 3 component of the modified acoustic frame. The horizontal width of the rectangle indicating the window component indicates the number of samples, and the vertical width indicates the strength. However, in FIG. 3, neither the horizontal width nor the vertical width is accurately shown, and a strong signal is applied to the leading portion corresponding to the window 1 component. It indicates whether there is a component and whether there is a strong signal component in the rear portion corresponding to the window 3 component. FIG. 5A shows a good signal when there is no acoustic frame in which the total values E ₁ and E ₂ calculated by the above [Equation 4] are less than the level lower limit value Lev, that is, for embedding additional information. The case is shown. FIG. 5B shows a signal that is not good for embedding additional information when there is an acoustic frame in which the combined values E ₁ and E ₂ calculated by the above [Equation 4] are less than the level lower limit value Lev. The case is shown.

  For example, it is assumed that a 2-byte bit array in which the first byte is “11011100” and the second byte is “11000001” is embedded as additional information. First, at the beginning of each byte, the window 1 component and the window 3 component are set to be equal as information indicating a break. This is obtained as a result of executing the processing according to the above [Equation 5] in S108, which is set to the separation mode in S102. Subsequently, before the processing corresponding to each bit of the additional information is performed, information indicating whether the information is new or continued is recorded. This is because even if there is an acoustic frame that is less than the lower limit level, the bit processed at that time is valid and is continuously performed from there, so whether the bit is new or continuous. This is because it is necessary to record information. Therefore, after recording the information indicating the break, information indicating whether it is new or continued is recorded. Specifically, the mode determination is performed in the separation mode (S112), the continuous identification mode is set (S113), and the acoustic frame is extracted without reading the bits of the additional information (S104). Then, after the frequency conversion (S105), if it is new, the processing according to [Equation 6] and [Equation 7] is performed so that the distribution of the low frequency component between the window 1 component and the window 3 component is superior to the window 3 component. (S107).

  After recording the information indicating whether it is new or continued in this way, the mode determination is performed in the state of the continuous identification mode (S112), so the bit mode is set (S114), and the first bit is read from the register ( (S103), an acoustic frame is extracted (S104). In the example of FIG. 5A, since there is no acoustic frame that is less than the level lower limit value Lev, one byte is continuously processed by S107. This indicates that the loop from S103 to S114 was repeated eight times in succession, and during that time, it was assumed that the level was lower than the lower limit value Lev, and that it did not pass through S106, S108, and S113.

  In the example of FIG. 5B, as a result of the processing according to the above [Formula 6] and [Formula 7], there is an acoustic frame that is less than the level lower limit value Lev. As a result of executing the processing according to [Equation 5], the window 1 component and the window 3 component are set to be equal. In this case, since the separation mode is set in S106, information indicating whether it is new or continued is recorded via S112. In the example of FIG. 5B, when embedding “11011100” of the first byte, an acoustic frame less than the level lower limit value Lev appears when 1 bit of “1” of the first bit is initially processed. Therefore, after the information indicating the break is recorded, the information indicating the continuation is recorded, and the processing is continued from “1” of the second bit. Then, since “1011” from the second bit to the fifth bit has been processed, an acoustic frame less than the level lower limit value Lev has appeared, so after recording the information indicating the break, the information indicating the continuation is recorded. Then, processing is continued from “1” of the sixth bit.

  In the example of FIG. 5, the case where the additional information is 1 word = 1 byte has been described. However, since the process illustrated in FIG. 2 records information indicating whether the additional information is new or continued, the additional information may be any number of bits. It is possible to record in units.

  In the above example, information indicating a delimiter is inserted in word units of variable length, but information indicating a delimiter can be further inserted in bit units. In this case, when the acoustic frame reading means 10 extracts the acoustic frame, it extracts an overlapping acoustic frame that overlaps the preceding and following acoustic frames, performs frequency conversion on the overlapping acoustic frame according to [Equation 1], and In accordance with [Equation 5], the processing for equalizing the window 1 component and the window 3 component is performed. The overlapping sound frames are set so that half of the samples overlap with the preceding and following sound frames. For example, when the preceding sound frame is sample number 1 to 4096 and the subsequent sound frame is sample number 4097 to 8192, the overlapping sound frame set between this is sample number 2049 to 6144. Similarly, for all sections of the acoustic signal, a duplicate acoustic frame is read and processing for equalizing the window 1 component and the window 3 component is performed.

  As described above, when the overlapping acoustic frame is set and the processing for equalizing the window 1 component and the window 3 component is performed, the window 1 component and the window 3 component are equalized in order to reflect this in the modified acoustic signal. It is necessary to perform a process of performing inverse frequency conversion on the duplicated frame spectrum after the processing to obtain a modified duplicated audio frame and further connecting it to the modified audio frame.

  Of the left channel of the modified acoustic signal obtained as described above, for the portion where the additional information is embedded, the low frequency component is equal to the window 1 component and the window 3 component, or the window 1 There are only three distributions: the component predominates or the window three component predominates. However, since the high frequency component remains the original acoustic signal, it has various distributions based on the setting of the producer. Further, as shown in the above example, when a stereo sound signal is used, the low frequency component changed in the left channel is apparent from the processing of [Formula 5] to [Formula 10]. In addition, it is always added to the low frequency component of the right channel. Therefore, since the right channel supplements the deleted component in the left channel, there is no signal degradation when viewed as both channels as a whole. Human auditory senses directionality with respect to high-frequency components, but it is difficult to sense directionality with respect to low-frequency components. Therefore, even if the low frequency component is biased to one side, it will be heard as if it is a normal acoustic signal for the listener.

(3. Device for extracting information from acoustic signals)
Next, an apparatus for extracting information from an acoustic signal according to the present invention will be described. FIG. 6 is a block diagram showing an embodiment of an apparatus for extracting information from an acoustic signal according to the present invention. In FIG. 6, 100 is an acoustic signal input means, 110 is an acoustic frame acquisition means, 120 is a reference frame setting means, 130 is a phase change frame setting means, 140 is a frequency conversion means, 150 is a code determination parameter calculation means, and 160 is a code. An output means, 170 is an additional information extraction means, and 180 is an acoustic frame holding means.

The acoustic signal input unit 100 has a function of acquiring and inputting a flowing sound as a digital acoustic signal. In reality, it is realized by a microphone and an A / D converter. The microphone needs to be a directional microphone capable of inputting sound from the left and right two channels. The acoustic frame acquisition means 110 has a function of reading an acoustic frame composed of a predetermined number of samples from each channel of the input digital stereo acoustic signal. The reference frame setting unit 120 has a function of setting, as a reference frame, a read sound frame that has been moved by an offset value. The phase change frame setting means 130 has a function of setting, as a phase change frame, an acoustic frame whose phase has been changed by moving the reference frame and predetermined samples. The frequency conversion means 140 has the same function as the frequency conversion means 20 shown in FIG. The code determination parameter calculation unit 150 extracts each low frequency intensity data corresponding to a predetermined frequency or less from the generated plurality of frame spectra, and adds the low frequency intensity data for each of the window 1 component and the window 3 component. _c1 and _Ec2 are calculated, the combined values _Ec1 and _Ec2 are used as code determination parameters, and a function is determined based on the ratio of the code determination parameters _Ec1 and _Ec2. Yes.

  The code output means 160 determines which one of the acoustic frames (reference frame and phase change frame) corresponding to one reference frame is determined to have the optimum phase, and selects a code corresponding to the state of the acoustic frame. It has a function to output. It also has a function of determining an offset value for setting a subsequent reference frame. The additional information extracting unit 170 has a function of converting the ternary array, which is a set of codes output by the code output unit 160, according to a predetermined rule and extracting it as meaningful additional information. The acoustic frame holding means 180 is a buffer memory capable of holding two consecutive acoustic frames for each channel. Each component shown in FIG. 6 is actually realized by mounting a dedicated program on hardware such as a small computer having an information processing function and its peripheral devices. In particular, in order to achieve the object of the present invention more easily, it is desirable to use a portable terminal device as hardware.

(4. Processing operation of extraction device)
Next, the processing operation of the apparatus for extracting information from the acoustic signal shown in FIG. 6 will be described with reference to the flowchart of FIG. When the user wants to know the attribute information such as the song name of the music that is playing, first, the extraction device is instructed to start as the extraction device. For example, this can be executed by operating a predetermined button when the extraction device is realized by a mobile terminal such as a mobile phone. When an instruction is input to the extraction device, the acoustic signal input unit 100 records the flowing music, digitizes it, and inputs it as a digital acoustic signal. Specifically, the audio input from the left and right sides of the directional microphone is digitized by an A / D converter.

Subsequently, the average code levels HL1, HL2, the phase determination table S (p), the non-code counter Nn, and the minute offset value are initialized (S200). These will be described. The average code levels HL1 and HL2 are low frequencies calculated by the following [Equation 12] for an acoustic frame (hereinafter referred to as an effective frame) that is determined to have a binary value corresponding to a bit value embedded therein. the average value of the sum E _c1, E _c2 components, i.e., which is given by the average value of the sum E _c1, E _c2 in the past of the effective frame, the initial value is, the level limit is also used in the above embedding device It is set to twice the value Lev. Therefore, in this embodiment, the initial values of HL1 and HL2 are both set to “0.1”. The phase determination table S (p) is a table for determining the phase, and p takes an integer value of 0 to 5. The initial value is set to S (p) = 0. The non-sign counter Nn is a counter of the number of frames determined to have a low signal level and non-sign (same as a code indicating a delimiter), and is set to Nn = 0 in the initial state. The minute offset value indicates the offset value when the reference frame extracted from the acoustic signal is moved. The offset value takes eight values, and each value is a value obtained by further reducing the phase change amount of each frame size to 1/8, that is, in this embodiment, set in units of 4096 × 1/6 × 1/8. Is done.

  Subsequently, the acoustic frame acquisition unit 110 extracts an acoustic frame composed of a predetermined number of samples from each channel of the stereo acoustic signal input from the acoustic signal input unit 100 (S201). Specifically, an acoustic frame is extracted and read into the acoustic frame holding unit 180. The number of samples of one acoustic frame read as the reference frame by the acoustic frame acquisition unit 110 needs to be the same as that set by the acoustic frame reading unit 10 shown in FIG. Therefore, in the present embodiment, the acoustic frame acquisition unit 110 sequentially reads 4096 samples for the left channel and the right channel as reference frames. As described above, the acoustic frame holding unit 180 can store two reference frames for each channel. When a new reference frame is read, the old reference frame is discarded. Accordingly, the acoustic frame holding means 180 always stores two reference frames (continuous 8192 samples) for each channel. When the offset value is “0”, the reference frame setting unit 120 sets each sound frame held in the sound frame holding unit 180 as a reference frame as it is.

  When the offset value is “0”, the acoustic frame to be processed by the embedding device can be divided into a reference frame that is set adjacently without interruption from the beginning, and a phase change frame in which the phase is changed with the reference frame. it can. For the reference frame, after setting sample number 1 to sample number 4096 as the first reference frame, the next reference frame is sample number 4097 to sample number 8192, and the next reference frame is sample number 8193 to sample number 12288. As long as the offset value is “0”, it is set without interruption. Then, for each reference frame, five phase change frames moved by 1/6 frame (about 683 samples) are set. For example, for the first reference frame, five phase change frames configured by 4096 samples starting from sample numbers 683, 1366, 2049, 2732, and 3413 are set.

  Subsequently, the frequency conversion unit 140 and the code determination parameter calculation unit 150 determine embedded information from each read sound frame and output a corresponding code (S202). The format of the information to be output is a binary format corresponding to the bit value on the embedding side and a ternary format of values input as delimiters.

  Details of the code output process of S202 will be described with reference to the flowchart of FIG. First, the code determination parameter calculation unit 150 initializes the candidate code table (S304). The candidate code table records a phase number of 0 to 5 that specifies one reference frame and five phase change frames, and a ternary code obtained from the states of the six acoustic frames.

  Next, an address fine offset process is performed (S300). Specifically, according to the offset number (offset value), the reading position of the reference frame and the phase change frame is set to a position moved by a predetermined number of frames.

  Subsequently, the frequency conversion unit 140 performs frequency conversion on each acoustic frame read from the offset-processed position to obtain a frame spectrum (S301). This process is the same as the process in the frequency conversion means 20 shown in FIG. However, since only the left channel is used for extraction, the processing according to the above [Equation 1] is performed, and the imaginary part Bl (1) such as the real part Al (1, j) of the conversion data corresponding to the left channel. , J) etc.

By the processing in the frequency conversion means 140, a frame spectrum expressed by a spectrum that is a component corresponding to the frequency is obtained. Subsequently, the code determination parameter calculation unit 150 calculates the average code levels HL1 and HL2 (S302). Specifically, v1 which is an integrated value of the total value E _c1 of the acoustic frames determined to have the dominant past window 1 component is represented by the number of acoustic frames determined to have the dominant past window 1 component. HL1 is calculated by dividing by a certain n1, and v2 that is an integrated value of the total value E _c2 for the acoustic frame determined to be in the dominant state of the past window three components is determined to be in the dominant state of the past window three components. HL2 is calculated by dividing by n2, which is the number of sound frames that have been performed. Here, E _c1 and E _c2 are the sum of the window 1 component and the window 3 component of the left channel signal, which will be described later, and are given by the following [Equation 12].

[Formula 12]
E _c1 = Σ _{j = 1,} ... _{, M-3} {Al (1, j) ² + Bl (1, j) ² }
E _c2 = Σ _{j = 1,} ... _{, M-3} {Al (3, j) ² + Bl (3, j) ² }

Therefore, the average code levels HL1 and HL2 are close to the average value of the sum value of the low-frequency intensity data of the sound frame for which it has been determined that the window component corresponding to the past is dominant. As described above, the initial value of HL1 and HL2 is both set to “0.1”, and the summation value E _c1 and the summation value E _c2 are added to the numerator. It is to do. In addition, n1 and n2 used as denominators for calculating the average code levels HL1 and HL2 are added one by one each time the summed value E _c1 and summed value E _c2 are added to v1 and v2, respectively. However, if n1 = nmax, v1 is reset to v1 / 2, n1 is reset to n1 / 2, and if n2 = nmax, v2 is Reset v2 / 2 to n2 / 2. As a result, the average code levels HL1 and HL2 are almost always determined by the latest nmax / 2 summation value E _c1 and summation value E _c2 , and it becomes possible to quickly follow fluctuations in the signal level.

Further, the code determination parameter calculation means 150 extracts each low frequency intensity data in a predetermined frequency range from the generated frame spectrum. The frequency range to be extracted needs to correspond to the embedding device. Therefore, here, low frequency intensity data having a frequency of 200 Hz or less is extracted, and the real part Al (j) and imaginary part Bl of the left channel calculated by the above [Equation 1] as in the case of the embedding device. Among (j), those with j ≦ 20 are extracted. The code determination parameter calculating means 150, the sum of the window 1 component of the left channel signal by performing a process in accordance with the [Equation 12] E _c1, calculates the sum E _c2 of the window 3 components. These E _c1 and E _c2 are used as sign determination parameters.

Subsequently, the code determination parameter calculating means 150 makes a determination window 1 sum of components E _c1, sum E _c2 of the window 3 components of whether each is below a predetermined value (S305). Here, the code determination levels FL1 and FL2 are used as predetermined values. As the code determination levels FL1 and FL2, values obtained by multiplying the average code levels HL1 and HL2 by 0.001 are set. When the total value E _c1 is less than FL1 and the total value E _c2 is less than FL2, the code determination parameter calculation unit 150 determines that the information is delimiter information (S309).

On the other hand, when either one of the total values E _c1 and E _c2 is equal to or greater than a predetermined value, that is, when the condition of E _c1 > FL1 or E _c2 > FL2 is satisfied, the code determination parameter calculation unit 150 calculates the above. The code determination parameters E _c1 and E _c2 are compared and determined according to the following [Equation 13] (S306), and a code corresponding to the comparison result is output.

[Formula 13]
When E _c2 > FL2 and E _c2 / E _c1 > 2, the window 3 component is dominant. When E _c1 > FL1 and E _c1 / E _c2 > 2, the window 1 component is dominant. E _c2 ≦ FL2 or E _c2 / E _c1 ≦ 2 and E _c1 ≦ FL1 or E _c1 / E _c2 ≦ 2), both window components are equal

  The code determination parameter calculation means 150 outputs a ternary code according to the determination result for each acoustic frame. That is, when the window 3 component is determined to be dominant, the first bit value (eg, “1”) is output (S307), and when the window 1 component is determined to be dominant, the second bit value ( For example, “0”) is output (S308), and when it is determined that the two window components are equal, a code indicating delimiter information is output (S309). When it is determined that the window 3 component is dominant and the first bit value is output (S307), or when the window 1 component is determined to be dominant and the second bit value is output (S308), The phase determination table S (p) is updated according to the following [Equation 14] (S310).

[Formula 14]
When the three window components are dominant, S (p) ← S (p) + E _c2 / E _c1
When the window 1 component is dominant, S (p) ← S (p) + E _c1 / E _c2

Subsequently, the code determination parameter calculation unit 150 stores the candidate for the optimum phase in the candidate code table (S311). Specifically, the value of the phase number p that maximizes the value of S (p) recorded in the phase determination table, one of the three values determined in S307 to S309, and the sound frame The values of E _c2 and E _c1 are stored in the candidate code table as optimum phase candidates.

  Subsequently, it is determined whether or not the processing corresponding to all the phase numbers p has been completed (S312). This determines whether all phase change frames have been processed for a certain reference frame. In this embodiment, since p takes a value from 0 to 5, if the processing is not performed six times, an acoustic frame having a different phase is set by shifting a predetermined number of samples from the acoustic frame being processed, and the process proceeds to S305. Return and repeat the process. The case where p = 0 is a reference frame, and the case where p = 1 to 5 is a phase change frame. When the processing corresponding to all the phase numbers p is completed, it is determined that the phase corresponding to the phase number p recorded in the candidate storage table is the optimum phase, and the code recorded in the candidate storage table is output. (S313). At this time, the phase number p is stored in the optimum phase determination table in order to obtain the optimum phase for the subsequent acoustic frame.

  Next, an offset change process is performed (S314). Details of the offset change processing are shown in the flowchart of FIG. In the offset changing process, first, it is determined whether or not the determination phase has changed (S401). Specifically, referring to the optimum phase determination table, whether or not the optimum phase of the acoustic frame just processed is the same as that of the previous acoustic frame and that it is different from the immediately preceding acoustic frame. Judging.

  If the variation condition is met, the minute offset value is updated (S402). Specifically, the offset value is changed to the closest value greater than the current offset value among the eight types of offset values prepared. When the minute offset value is updated or when it is determined in S401 that the variation condition is not met, mode determination is performed (S403). Two modes, a delimited mode and a bit output mode, are prepared.

In the bit output mode, the parameters of the average code levels HL1 and HL2 and the phase determination table S (p) are updated (S404). To update the parameters of the average code levels HL1 and HL2, the sum values E _c1 and E _c2 are added to the integrated values v1 and v2 which are numerators when calculating the average code levels HL1 and HL2, and the integrated values v1 and v2 are updated. , By adding 1 to the number of frames n1 and n2 as denominators and updating the number of frames n1 and n2. The phase determination table S (p) is updated in the same manner as the processing according to S310 and [Equation 14]. In S404, the update counter is also updated. The value of the update counter is incremented by 1 each time the process in S404 is executed.

  Then, when the number of updates exceeds a predetermined value, that is, when the value of the update counter exceeds a predetermined value, the parameters of the average code levels HL1 and HL2 and the phase determination table S (p) are refreshed ( S405). Specifically, the number of past data to be referred to is limited to 50 times, and the accumulated values v and S (p) are halved every time 100 times are accumulated. Then, the value of the update counter is reset to “0”.

  If it is determined in S403 that the separation mode is selected, if the number of updates is less than a predetermined value in S404, or if refresh is performed in S405, the offset change process (S314) is terminated.

Since the code determination process shown in FIG. 8 is completed by the end of the offset changing process, the description will be returned to the flowchart of FIG. If the code corresponding to the bit value is output as a result of the process in S202, the parameters of the average code levels HL1 and HL2 are updated and the non-code counter is initialized (S203). Specifically, the update of the parameter of the average code level HL is performed by adding the total values E _c1 and E _c2 to the integrated values v1 and v2 which are numerators when calculating the average code level HL, as in S404. This is performed by updating v1 and v2, and adding 1 to the number of frames n1 and n2 as denominators to update the number of frames n1 and n2. In the initialization process of the non-sign counter, the non-sign counter Nn = 0 is set as in the process in the initialization process of S200.

  Subsequently, the mode is determined (S204). Specifically, as in S403, it is determined whether the mode is a separation mode or a bit output mode. If it is in the bit output mode, the bit value is stored in the buffer (S209). Subsequently, the bit counter is counted up (S210). On the other hand, if the result of determination in S204 is that the mode is separation mode, it is further determined whether the extracted code means new or continuation (S205). As a result, if it is new, it means that one word has been completed immediately before, so that the additional information extracting means 170 outputs the data for one word recorded in the buffer (S206). Then, the bit counter is initialized to 0 (S207). Further, the mode is set to the bit output mode (S208). In S205, if it is determined to continue, the value should be output to the bit in the buffer, so only the processing for setting the bit output mode is performed.

  Subsequently, the mode is set to the delimiter mode in order to extract information on whether it is new or continued from the next sound frame (S212). Then, the non-sign counter is incremented (S213). Specifically, 1 is added to the value of the unsigned counter Nn. By executing the process shown in FIG. 7 for each reference frame, additional information is extracted. If it is determined in S201 that all reference frames have been extracted, the process ends.

  In the process of S206, the additional information extraction unit 170 first sets a code indicating equality of both window components among the ternary codes output by the code determination parameter calculation unit 150 as a delimiter position, and sets the next code as the head. The bit array is created by associating the codes of the window 3 component dominant and the window 1 component dominant with the bit values. Subsequently, this bit arrangement is converted according to a predetermined rule and extracted as meaningful additional information. As the predetermined rule, various rules can be applied as long as the information intended by the person who embeds the information can be recognized by the person who has received it. As a rule. That is, the additional information extraction unit 170 recognizes the code output from the code output unit 160 in units of 1 byte (8 bits) as determined by the code determination parameter calculation unit 150, and character information according to the set code system. Recognize The character information thus obtained is displayed and output on a screen of a display device (not shown).

  Therefore, if the embedding device embeds the attribute information such as the song title or artist in the sound signal as the character information, the user wants to know the song title or artist by listening to the music being played. If a predetermined operation is performed on the mobile terminal that functions as the extraction device, attribute information such as a song title and an artist is displayed as character information on the screen of the mobile terminal.

  In the above processing, in order to extract additional information accurately in the extraction device, processing for correcting the phase, processing for correcting the difference in intensity between the left and right low frequency components, and a lower limit for determining that the frame is an invalid frame Processing to correct the threshold is performed. Next, supplementary explanation will be given for these three correction processes.

(5. About phase correction processing and offset change processing)
As described above, at the time of extraction, it is not always possible to read an acoustic signal corresponding to the acoustic frame embedded at the time of embedding. Therefore, the phase of the acoustic frame is shifted and read in a plurality of ways (six in this embodiment), the optimum phase is determined, and a code corresponding to the acoustic frame specified by the phase is output. Yes. At this time, in order to further improve the detection accuracy, in the present invention, the position of the reference frame is moved by a minute amount to determine the optimum acoustic frame as a code extraction target.

  Regarding the phase correction processing, for example, when reading in six ways, the first acoustic frame is originally a sample with sample numbers 1 to 4096, but with 4096 samples starting with sample numbers 1, 683, 1366, 2049, 2732, and 3413. Processing is performed on each of the six acoustic frames that are configured, and a code corresponding to the optimal acoustic frame is output. This phase correction process is performed centering on the processes in S304, S310, S311, S312 and S313.

  As for the offset changing process, the offset value is changed only when the optimum phase frame has the same phase as the two previous optimum phase frames and a phase different from the previous optimum phase frame. . This offset changing process is performed centering on the processes in S304 and S314.

  Here, FIG. 10 shows a state of change of the acoustic frame that is a target of information extraction by the offset changing process. In FIG. 10, the left-right direction indicates the time-series direction, and indicates the future as it goes to the right. FIG. 10A schematically shows a digital acoustic signal that is an object to be acquired by the acoustic signal input unit 100 as a sample string, and is added to four locations indicated as acoustic frames A to D. It is assumed that information is embedded. Each of FIG. 10B schematically shows a digital acoustic signal acquired by the acoustic signal input unit 100 while shifting the phase as a sample sequence in order to extract the additional information. As described above, the sound frame acquisition unit 110 reads a predetermined number of samples (sample group) from each channel of the input digital stereo sound signal as the reference frame, and the phase change frame setting unit 130 determines the reference frame and the predetermined frame. An acoustic frame whose phase is changed by moving sample by sample is set as a phase change frame. FIG. 10B shows an acoustic frame as a reference frame and a phase change frame. In the example of FIG. 10, the sample group A is read as a reference frame (acoustic frame A0), and the acoustic frames A1 to A5 are read as phase change frames. The same applies to the relationship between the sample groups B and C and the acoustic frames B0 to B5 and the acoustic frames C0 to C5. That is, the alphabet of the acoustic frame corresponds to the alphabet of the sample group, and the numeral of the acoustic frame corresponds to the phase number p.

  Under such circumstances, the acoustic frame A2 is in the acoustic frame A0 to the acoustic frame A5, the acoustic frame B1 is in the acoustic frame B0 to the acoustic frame B5, and the acoustic frame C2 is in the optimal phase in the acoustic frame C0 to the acoustic frame C5. Assume that the frame is determined. In this case, the phase number p is recorded in the optimum phase determination table in the order of “2” “1” “2” corresponding to each reference frame by the process in S313. Such an unstable phenomenon is caused by the fact that the phase of the acoustic frame A is located between the acoustic frame A1 and the acoustic frame A2, and since it cannot be determined which is optimal, a different phase number is determined for each sample group. Therefore, this corresponds to the determination phase fluctuation condition in S401. That is, as shown in FIG. 10B, the optimum phase (C2) of the acoustic frame just processed is the same as the previous acoustic frame (A2) and is different from the previous acoustic frame (B1). . Then, in S402, the minute offset value is updated, and in S300, the reading position of the reference frame and the phase change frame is set to a position shifted by a predetermined number of frames in accordance with the address minute offset process, that is, the updated offset value. To do. Therefore, as shown in FIG. 10B, the sample group D is moved by the offset value. Then, any candidate acoustic frame of the acoustic frames D0 to D5 moves by the offset value, and in the example of FIG. 10B, there is no room for selection other than the acoustic frame D1, and after that, the phase continues for a while. The number p = 1 is determined stably. However, in practice, as in this example, it is rare to be stabilized by a single minute offset process, and on the contrary, the instability of phase determination may worsen by performing a minute offset process. Even in such a case, in this embodiment, the phase determination becomes stable if the minute offset process is performed at most seven times.

(7. Lower threshold (average code level) correction process)
When the signal level is low, the size of both window components cannot be determined, and the extraction side often makes a wrong determination. Therefore, frames whose low frequency intensities E _c1 and E _c2 are lower than the code determination levels FL1 and FL2 are determined to be invalid frames in S305 (not subject to bit value extraction). The code determination levels FL1 and FL2 are corrected using the low frequency intensity integrated values E _c1 and E _c2 for the past effective frames. Thus, by varying the code determination levels FL1 and FL2, which are threshold values, it is possible to accurately determine whether the frame is invalid or valid even if the signal level varies. In particular, in the present invention, when the input signal level remains so low that it is determined that the signal has been interrupted, the process of initializing the average code levels HL1 and HL2 to the level lower limit value Lev that is the initial value is performed. Do. Further, in the present invention, even when the signal level changes throughout the acoustic signal, the low frequency of a frame that is distant in time is calculated when the average code levels HL1 and HL2 are calculated. The strengths E _c1 and E _c2 are used while reducing the specific gravity. This lower limit threshold correction process is performed centering on the processes in S302, S203, and S404.

Here, as in the case of the present invention, when the low frequency intensities E _c1 and E _c2 that are the basis for calculating the average code levels HL1 and HL2 are treated equally for all the past frames, FIG. 11 shows the state of code determination when the importance of a distant frame is reduced and the signal is extremely small, and initialization is performed. In FIG. 11, the horizontal axis indicates the time axis in frame units, and the vertical axis indicates the signal level. In addition, solid lines with thick frame widths indicate low frequency intensities E _c1 and E _c2 (shown as E in the figure), and broken lines with thick frame widths show code determination levels FL1 and FL2 (shown as FL in the figure). Note that the low frequency intensity E shown in FIGS. 11A and 11B is exactly the same in the corresponding frame.

When such a low frequency intensity E is obtained, in the conventional method, as shown in FIG. 11A, in the sixth frame from the left, the low frequency intensity E _c1 and E _c2 are less than the code determination levels FL1 and FL2. Therefore, in S309, it is determined as delimiter information. As described above, in the conventional method, the sixth frame from the left has appropriate low-frequency intensities E _c1 and E _c2 , and although bit values can be extracted originally, It will be judged. On the other hand, in the present invention, the specific gravity of the low frequency intensities E _c1 and E _c2 for a frame that is distant in time is reduced. Therefore, the value of the low frequency intensity E for a frame close in time is greatly reflected in the code determination levels FL1 and FL2. For this reason, when the low frequency intensity E gradually decreases in the fourth to sixth frames from the left, this is reflected and the values of the code determination levels FL1 and FL2 also decrease. Then, as shown in FIG. 11 (b), in the sixth frame from the left, the low frequency intensities E _c1 and E _c2 are equal to or higher than the code determination levels FL1 and FL2, so the bit values are set in S306 to S308. Will be.

Further, the low frequency intensities E _c1 and E _c2 are “0” in the 10th and 11th frames from the left, but in the conventional method, special processing is applied to the code determination levels FL1 and FL2 in such a case. Therefore, the values of the code determination levels FL1 and FL2 do not decrease rapidly. For this reason, as shown in FIG. 11A, in the 12th and 13th frames from the left, the low frequency intensities E _c1 and E _c2 are less than the code determination levels FL1 and FL2, so that it is determined as delimiter information in S309. Is done. On the other hand, in the present invention, when the low frequency intensity E is “0” for a predetermined number of frames or more, the code determination levels FL1 and FL2 are reset to the initial values. For this reason, in the eleventh frame from the left, the values of the code determination levels FL1 and FL2 are also reduced. Then, as shown in FIG. 11B, in the twelfth and thirteenth frames from the left, the low frequency intensity E becomes equal to or higher than the code determination levels FL1 and FL2, so that a bit value is set in S306 to S308. become. In the example of FIG. 11, the code determination levels FL1 and FL2 are set to the initial values only when the low-frequency intensities E _c1 and E _c2 are “0” for two consecutive frames. In reality, the sign determination levels FL1 and FL2 are set to initial values when the low frequency intensity E is continuously “0” for a number of frames.

It is a functional block diagram of an information embedding device for an acoustic signal. It is a flowchart which shows the process outline | summary of the apparatus shown in FIG. It is a figure which shows the mode of the change of the signal waveform in the case of performing general Fourier transform. It is a figure which shows the window function used in this invention. FIG. 3 shows how a low-frequency component is changed by processing according to FIG. 2. 1 is a functional block diagram of an apparatus for extracting information from an acoustic signal according to the present invention. It is a flowchart which shows the process outline | summary of the apparatus shown in FIG. 5 It is a flowchart which shows the detail of the code | symbol determination process of S202 of FIG. It is a flowchart which shows the detail of the code | symbol determination process of S314 of FIG. It is a block diagram which shows the mode of a change of the acoustic frame used as the object of information extraction by an offset change process. It is a figure which shows the mode of the code determination in a conventional method and this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Sound frame reading means 20 ... Frequency conversion means 30 ... Low frequency component change means 40 ... Frequency reverse conversion means 50 ... Modified sound frame output means 60 ... Storage means 61 ... -Acoustic signal storage unit 62 ... Additional information storage unit 63 ... Modified acoustic signal storage unit 70 ... Additional information reading means 100 ... Acoustic signal input means 110 ... Acoustic frame acquisition means 120 ... Reference frame setting means 130: Phase change frame setting means 140 ... Frequency conversion means 150 ... Code determination parameter calculation means 160 ... Code output means 170 ... Additional information extraction means 180 ... Acoustic frame Holding means

Claims (5)

A device that extracts information embedded in advance in an inaudible state from an acoustic signal composed of a time-series sample sequence,
Acoustic frame acquisition means for acquiring an acoustic frame composed of a predetermined number of samples from the acoustic signal;
A reference frame setting means for setting a reference frame by changing the phase by moving the acoustic frame by an offset value for a predetermined number of samples;
Phase change frame setting means for setting a plurality of acoustic frames set by changing the phase by moving from the reference frame by a sample corresponding to a step value that is a value larger than the offset value, as a phase change frame;
Frequency conversion means for performing frequency conversion on each acoustic frame set as the reference frame and the phase change frame, and generating a frame spectrum corresponding to each acoustic frame;
Code determination parameter calculation means for extracting low frequency intensity data corresponding to a component of a predetermined frequency or less from the generated frame spectrum, and calculating a code determination parameter based on the low frequency intensity data;
Based on the code determination parameter calculated in the past in-phase acoustic frames having different reference frames, it is determined that one of the reference frames and the plurality of phase change frames is the optimum phase frame having the optimum phase. And outputting a predetermined code based on the code determination parameter determined for the optimum phase frame, and the optimum phase frame is different in phase from the immediately preceding optimum phase frame, and the phase and phase of the two previous optimum phase frames. Code output means for changing the offset value when they are the same,
Additional information extracting means for extracting the additional information by converting the bit arrangement constituted by the code output for each optimum phase frame according to a predetermined rule;
An apparatus for extracting information from an acoustic signal, comprising: In claim 1,
The frequency conversion means includes
Frequency conversion is performed on the acoustic frame using a first window function and a third window function, respectively, and a first window spectrum and a spectrum corresponding to the third window function are spectra corresponding to the first window function. To generate a third window spectrum,
The code output means extracts a spectrum set corresponding to a predetermined low frequency band from each generated window spectrum, calculates a sum value of spectrum intensities for each spectrum set, and a spectrum set of the sum value An apparatus for extracting information from an acoustic signal, which outputs a predetermined code based on a ratio between the two. In claim 1,
The code output means calculates the code determination parameter for each phase with respect to the acoustic frame having the same phase in the past, and the sound corresponding to the phase having the maximum value in the phase determination table calculated using the code determination parameter. The code corresponding to the state of the frame is output, and in the calculation of the phase determination table, the past acoustic frame that is farther away in time is less affected by the acoustic signal. Information extraction device. In claim 1 or claim 2,
The code output means includes
When the sum of the low frequency intensity data extracted from the generated frame spectrum is less than a predetermined lower threshold, the acoustic frame is determined to be an invalid frame,
The lower threshold used for the determination is calculated by adding low frequency intensity data that has been determined as an effective frame in the past, and in the addition of the low frequency intensity data, past effective frames that are far apart in time An apparatus for extracting information from an acoustic signal, characterized in that the influence is reduced. In claim 4,
The code output means includes
When the acoustic frame is determined to be an invalid frame and a predetermined number of consecutive acoustic frames are also determined to be invalid frames, the lower threshold is reset to an initial value. An apparatus for extracting information from an acoustic signal.

Images