JP4877007B2 - Information embedding device for sound signal and device for extracting information from sound signal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device for embedding information in a sound signal and a device for extracting information from the sound signal, such that even when data is embedded in a stereo sound signal and is monaurally reproduced from either of channels, information is extracted from the sound, while suppressing degradation in reproduced quality. <P>SOLUTION: When additional information is embedded in the sound signal, a sound frame is obtained from a prescribed period of the sound signal, and a prescribed frequency range from F1 to F2 of the sound frame is divided into multiple numbers (4 or larger). Intensity of a spectrum set extracted from them are changed so that it may become about the same for right and left channels, both in even numbered sound frames (c) and (d), and in odd numbered sound frames (e). <P>COPYRIGHT: (C)2009,JPO&amp;INPIT

Description

  The present invention relates to the protection of music copyright (monitoring illegal copying) in the field of package music for viewing for consumer and business use using CDs and DVDs, and the field of broadcasting and network music distribution distributed for commercial purposes by broadcasters and the like. ) And music attribute information (music title search service) field, museum, event information service service field linked to exhibition explanation narration, URL and other information extracted from audio signals of broadcast programs and CD / DVD packages In addition, the present invention relates to a non-contact Internet gateway service field in which a mobile phone is used to access a web site related to a predetermined content and extract detailed information or answer a questionnaire.

  Conventionally, as a service for providing music attribute information that allows you to know the title of music that is playing, you can query the broadcast station for the date and time of the broadcasted music, and record music fragments that are played on mobile phones. In addition, there is a service that matches the melody registered in the database. The present applicant has further developed this and proposed a method of embedding attribute information (additional information) by changing the ratio of frequency components of an acoustic signal according to the bit value of the attribute information (Patent Literature). 1 and 2).

  In the inventions described in Patent Documents 1 and 2, data is embedded in a 2-channel stereo sound signal while moving corresponding signal components between the left and right channels, and at the time of extraction, the L side of two speakers being reproduced in stereo Since the microphones are placed close to each other and monaural input is used for extraction, when two speakers are separated by a wide hall etc., data cannot be physically extracted if they are close to the R-side speakers. In some cases, the L side and the R side are reversed, or there are places where it is difficult to determine which side is the L side, and the extraction operation may be confused.

  Since the L-side signal and the R-side signal are in a complementary relationship, the R-side signal has been modified based on the embedded data. However, in order to facilitate misalignment correction, etc. On the L side, a signal component in a predetermined region is cut compared to the R side, and the amount is added to the R side, so that the design is basically difficult to extract from the R side signal. In addition, when the operation mode in which only the L-side signal is reproduced in monaural is used, there is a problem that noise is conspicuous because the signal degradation is biased to the L side.

In order to solve such a problem, the present applicant has proposed a technique capable of extracting information from either sound of a two-channel stereo sound signal (patent). Reference 3).
JP 2006-235359 A JP 2006-323246 A Japanese Patent Application No. 2007-53324

  However, in the invention described in Patent Document 3, it cannot be extracted near the center where sounds from both channels are mixed. For this reason, in order to enable extraction from anywhere, it is possible to cope with monaural reproduction using either one of the channels. However, in the case of monaural reproduction, there is a problem in reproduction quality.

  Therefore, according to the present invention, even when data is embedded in a stereo sound signal and monaural reproduction is performed from one of the channels, information can be extracted from the sound while suppressing deterioration in reproduction quality. It is an object of the present invention to provide an information embedding device for an acoustic signal and an information extracting device for the acoustic signal.

  In order to solve the above problems, the present invention is an apparatus that embeds additional information in an inaudible state in an acoustic signal composed of a stereo time-series sample sequence consisting of two channels, channel L and channel R. Then, from the acoustic signal, acoustic frame reading means for reading a predetermined number of samples for the channel L and channel R as an L acoustic frame and an R acoustic frame, and one of the odd numbered and even numbered acoustic frames among the read acoustic frames. A first window function and a second window function that mainly extract signal components of the front, middle, and rear in the time direction with respect to the L acoustic frame and the R acoustic frame of the A type. The frequency conversion is performed using the third window function, and the B type L sound frame and R sound frame Frequency conversion using a window function, a first window L spectrum that is a spectrum corresponding to the first window function, a first window R spectrum, a second window L spectrum that is a spectrum corresponding to the second window function, Second window R spectrum, third window L spectrum, which is a spectrum corresponding to the third window function, third window R spectrum, fourth window L spectrum, which is a spectrum corresponding to the fourth window function, fourth window R Frequency conversion means for obtaining a spectrum, and 2Q (Q is an integer of 2 or more) spectrum sets that do not overlap each other in a predetermined frequency range are extracted from each of the generated window spectra, and a first window L spectrum, a first window The spectrum set extracted from the R spectrum, the third window L spectrum, and the third window R spectrum is an adjacent spectrum in the same window spectrum of the same channel. The second window L spectrum, the second window R spectrum, the intensity of the plurality of groups are changed to be different from each other based on the bit arrangement to be embedded, so as to belong to different groups. For the fourth window L spectrum and the fourth window R spectrum, frequency component changing means for changing the intensity of adjacent spectrum sets in the same window spectrum of the same channel to be different from each other regardless of the bit arrangement of the additional information to be embedded. Frequency inverse transform means for performing frequency inverse transform on each window spectrum including the modified spectrum set to generate modified acoustic frames, and modified acoustic frame output for sequentially outputting the generated modified acoustic frames An information embedding device for an acoustic signal having means is provided.

In the present invention, the additional information embedded in a non listening state by embedding device information for said acoustic signal, there in the desired mono audio signal or we Extraction to apparatus of a single channel of a two-channel stereo audio signals An acoustic frame acquisition means for digitizing a predetermined section of the acoustic signal to acquire an acoustic frame composed of a predetermined number of samples; and a signal component at a front portion and a rear portion in a time direction with respect to the acoustic frame. mainly the first window function to extract, respectively, using the third window function, performs frequency conversion, the first window spectrum is a spectrum corresponding to the first window function, in the spectrum corresponding to the third window function A frequency converting means for obtaining a third window spectrum; and 2Q spectra not overlapping each other in a predetermined frequency range from the first window spectrum. Extracts the torque set, extracts 2Q number of spectral set that do not overlap with each other in the predetermined frequency range from said third window spectrum, first window spectrum, the spectral set extracted from the third window spectrum, the same window spectrum Is divided into two groups so that adjacent spectrum sets belong to different groups, and an embedded bit value is extracted based on the magnitude of each spectral element at the same frequency between the groups. And an apparatus for extracting information from an acoustic signal having additional information extraction means for converting the extracted bit value in units of words according to a predetermined rule to extract additional information.

  According to the present invention, when the additional information is embedded in the stereo sound signal, the predetermined frequency range information of each channel is divided into four or more fine ranges, and the bits to be embedded for each divided fine range. Since it is changed so that it becomes the same intensity on the left and right according to the value, even if it is monaural playback from one of the channels, information is extracted from that sound while suppressing deterioration in playback quality It becomes possible.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(1.1. Configuration of embedded device)
First, an information embedding device for an acoustic signal will be described. FIG. 1 is a functional block diagram showing the configuration of an information embedding device for an acoustic signal according to the present invention. In FIG. 1, 10 is an acoustic frame reading means, 20 is a frequency converting means, 30 is a frequency component changing means, 40 is a frequency inverse converting means, 50 is a modified acoustic frame output means, 60 is a storage means, and 61 is an acoustic signal storage section. 62 is an additional information storage unit, 63 is a modified acoustic signal storage unit, and 70 is a bit array creation means. The apparatus shown in FIG. 1 can deal with both a stereo sound signal and a monaural sound signal. Here, a case where processing is performed on a stereo sound signal will be described.

  The sound frame reading means 10 has a function of reading a predetermined number of samples as one frame from each channel of the original stereo sound signal to be embedded with additional information. The frequency conversion means 20 has a function of generating a spectrum by frequency-converting the frame of the acoustic signal read by the acoustic frame reading means 10 by Fourier transformation or the like. The frequency component changing unit 30 extracts a plurality of spectrum sets corresponding to a predetermined frequency range from the generated spectrum, and based on the bit arrangement created by the bit arrangement creating unit 70 from the additional information extracted from the additional information storage unit 62. The function of changing the state of the spectrum set is provided. The frequency inverse transform means 40 has a function of generating a modified acoustic frame by performing frequency inverse transform on a plurality of spectra including the changed spectrum set. The modified sound frame output means 50 has a function of sequentially outputting the generated modified sound frames.

  The storage means 60 includes an acoustic signal storage unit 61 that stores a stereo acoustic signal to be embedded with additional information, an additional information storage unit 62 that is configured as a bit array and stores additional information embedded in the stereo acoustic signal, and an additional information It has a modified acoustic signal storage unit 63 for storing the modified acoustic signal after information is embedded, and stores various information necessary for other processing. The bit array creation means 70 extracts additional information from the additional information storage unit 62, adds a 1-bit parity bit to each word of the additional information, and then adds a bit array according to a predetermined rule. Has the function to create. The additional information is information that should be added to the sound information and embedded, and includes attribute information such as a title and artist name, and other information other than the attribute information. In the present embodiment, since the ASCII code is adopted as the code format of the additional information, 7 bits are set as one word in the additional information, and the bit array generated by the bit array generating means 70 is a check code after adding a parity bit. The 16 bits subjected to the additional processing are defined as one word. Each component shown in FIG. 1 is actually realized by installing a dedicated program in hardware such as a computer and its peripheral devices. That is, the computer executes the contents of each means according to a dedicated program.

(1.2. Processing operation of embedded device)
Next, the processing operation of the information embedding device for the acoustic signal shown in FIG. 1 will be described. The sound frame reading means 10 reads a predetermined number of samples as one sound frame from each of the left and right channels of the stereo sound signal stored in the sound signal storage unit 61. The number of samples of one sound frame read by the sound frame reading means 10 can be set as appropriate, but is desirably about 4096 samples when the sampling frequency is 44.1 kHz. Therefore, the acoustic frame reading means 10 sequentially reads 4096 samples for each of the left channel and the right channel as acoustic frames.

  There are A type and B type as acoustic frames. In the A-type acoustic frame and the B-type acoustic frame, the next sample after the last sample of the preceding acoustic frame of the same type is set as the first sample. The A-type and B-type sound frames are set by overlapping a predetermined number (2048 in this embodiment) of samples. For example, if the A type acoustic frame is A1, A2, A3... From the top and the B type acoustic frame is B1, B2, B3... From the top, A1 is samples 1 to 4096, A2 is samples 4097 to 8192, A3. Is samples 8193-12288, B1 is samples 2049-6144, B2 is samples 6145-10240, and B3 is samples 10241-14336. Since the A type and the B type are relative, either one may be first. That is, contrary to the above, A1 is samples 2049 to 6144, A2 is samples 6145 to 10240, A3 is samples 10241 to 14336, B1 is samples 1 to 4096, B2 is samples 4097 to 8192, and B3 is samples 8193 to 12288. May be.

  The frequency conversion unit 20 performs frequency conversion on the sound frame read by the sound frame reading unit 10 to obtain a spectrum of the sound frame. Specifically, frequency conversion is performed using a window function. As frequency conversion, Fourier transform, wavelet transform, and other various known methods can be used. In the present embodiment, a case where Fourier transform is used will be described as an example.

  In general, when Fourier transform is performed on a predetermined signal, it is necessary to divide the signal into predetermined lengths. In this case, if Fourier transform is performed on a signal of a predetermined length as it is, a pseudo-harmonic wave is generated. Ingredients are generated. Therefore, in general, when performing Fourier transform, a signal value is changed using a window function called a Hanning window, and then Fourier transform is performed on the changed value.

  In the present invention, not only to prevent the generation of pseudo-harmonic components but also to create a plurality of states for embedding information from one acoustic frame, a plurality of window functions are prepared, Then, Fourier transform is performed using each window function to obtain a plurality of spectra. As a plurality of window functions, in the present embodiment, the first window function W (1, i), the second window function W (2, i), and the third window as shown in FIGS. A function W (3, i) is prepared so that it can be easily recognized on the extraction side. The first window function W (1, i) is for extracting the front part of the acoustic frame, and has a maximum value of 1 at the position of a predetermined sample number i in the front part as shown in FIG. And the rear part is set to have a minimum value of 0. Which sample number takes the maximum value depends on the design of the window function W (1, i), but in this embodiment, it is defined by [Equation 1] described later. By multiplying the window function W (1, i), the signal waveform of the acoustic frame as shown in FIG. 2 (a) has a signal component remaining in the front part as shown in FIG. The component is deleted, and this becomes a Fourier transform target.

  The second window function W (2, i) is for extracting the central portion of the acoustic frame, and as shown in FIG. 2 (c), at the position of the predetermined sample number i in the central portion. The maximum value is 1, and the minimum value is set to 0 at the front and rear portions. Which sample number takes the maximum value depends on the design of the window function W (2, i), but in this embodiment, it is defined by [Expression 2] described later. By multiplying the window function W (2, i), the signal waveform of the acoustic frame as shown in FIG. 2A has a signal component remaining at the center as shown in FIG. The rear signal component is removed, and this is subjected to Fourier transform.

  The third window function W (3, i) is for extracting the rear part of the acoustic frame. As shown in FIG. 2 (d), the front part has a minimum value of 0 and a predetermined value in the rear part. The maximum value 1 is set at the position of the sample number i. Which sample number takes the maximum value depends on the design of the window function W (3, i), but in this embodiment, it is defined by [Equation 3] described later. By multiplying the window function W (3, i), the signal waveform of the acoustic frame as shown in FIG. 2 (a) is removed from the front signal component as shown in FIG. 2 (h). The signal component remains and becomes a Fourier transform target. Thus, after extracting the front part, the central part, and the rear part, the Fourier transform is executed, so that spectra corresponding to the front part, the central part, and the rear part are obtained. In order to embed a bit value in one acoustic frame, the bit value is originally divided into two parts, a front part and a rear part. However, on the extraction side, it is not always possible to read a signal synchronously. Therefore, in order to clearly distinguish the front part from the rear part, in this embodiment, the signal component at the center part is always deleted at the time of embedding, and the front part and the rear part are separated in time (however, at the time of extraction) Only need to analyze the front and rear, and ignore the middle). In the window function used in the present embodiment, the window function W (1, i) and the window function W (3, i) are asymmetrical, so that erroneous recognition of embedded information is less likely to occur during extraction.

  In the present embodiment, the acoustic frames are read in duplicate, and the window functions W (1, i), W (2, i), W (3, i) are used for the odd frames (or even frames), For even frames (or odd frames), the fourth window function W (4, i) as shown in FIG. 2 (e) is used.

  In the present invention, sound frames are read in duplicate. That is, a predetermined number of samples are redundantly read in the odd-numbered sound frames and the even-numbered sound frames. As described above, the window function used is different between the odd frame and the even frame, but since either the odd frame or the even frame is simply the difference between the odd frame and the even frame, either process may be performed on either. . Therefore, in this specification, one of the odd-numbered frame and the even-numbered frame is referred to as an A-type frame, and the other is referred to as a B-type frame. In the present embodiment, an odd frame is described as an A type frame and an even frame is described as a B type frame. Conversely, an even frame may be an A type frame and an odd frame may be a B type frame.

  In the present embodiment, the window functions W (1, i) to W (4, i) are defined by the following [Equation 1] to [Equation 4]. In FIG. 2, the horizontal axis is the time axis (i). As described later, i is a serial number assigned to N samples in each acoustic frame, and is proportional to time t. 2A, 2F, 2G, 2H, and 2I, the vertical axis indicates the amplitude value (level) of the signal. 2B to 2E, the vertical axis indicates the values of the window functions W (1, i), W (2, i), W (3, i), and W (4, i). The maximum values of (1, i), W (2, i), W (3, i), and W (4, i) are all 1.

[Formula 1]
When i ≦ 3N / 8, W (1, i) = 0.5−0.5 cos (8πi / (3N))
When 3N / 8 <i ≦ N / 2, W (1, i) = 0.5−0.5 cos (8π (i−N / 4) / N)
When i> N / 2, W (1, i) = 0.0

[Formula 2]
When i ≦ 3N / 8, W (2, i) = 0.0
When 3N / 8 <i ≦ N / 2, W (2, i) = 0.5−0.5 cos (8π (i−3N / 8) / N)
When N / 2 <i ≦ 3N / 4, W (2, i) = 0.5−0.5 cos (4π (i−N / 4) / N)
When i> 3N / 4, W (2, i) = 0.0

[Formula 3]
When i ≦ N / 2, W (3, i) = 0.0
When i> N / 2, W (3, i) = 0.5−0.5 cos (4π (i−N / 2) / N)

[Formula 4]
When i ≦ N / 4, W (4, i) = 0.0
When N / 4 <i ≦ N / 2, W (4, i) = 0.5−0.5 cos (4π (i−N / 4) / N)
When N / 2 <i ≦ 7N / 8, W (4, i) = 0.5−0.5 cos (8π (i−N / 8) / (3N))
When i> 7N / 8, W (4, i) = 0.0

  As is clear from FIG. 2 and [Formula 1] to [Formula 4], the window functions W (1, i) and W (3, i) have asymmetric shapes. This is for facilitating identification between the two on the extraction side described later. The window functions W (1, i), W (2, i), and W (3, i) have a maximum value of 1 when i is a predetermined value, and i takes other values. , I is a window function that monotonically increases or decreases according to the value of i, and therefore, when the window functions W (1, i) and W (3, i) are determined, the window function W (2, i ) Is inevitably determined. For this reason, the window function W (2, i) has a left-right asymmetric shape.

  In the present invention, since odd frames and even frames are redundantly read by a predetermined number of samples, after embedding information and then restoring to an acoustic signal, the odd frame multiplied by the window function and the window function are multiplied. When overlapping samples of even frames are added, it is necessary to return almost to the original value. Therefore, the shape of the window function W (4, i) is inevitably determined according to the values of the window functions W (1, i), W (2, i), and W (3, i). That is, when the window functions W (1, i), W (2, i), W (3, i), and W (4, i) are added in the overlapping portion of the odd and even frames, the fixed value 1 for the entire section is obtained. Is defined to be

  When the frequency conversion means 20 performs Fourier transform on the A type sound frame, the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1). Using the three window functions W (1, i), W (2, i), W (3, i), processing according to the following [Equation 5] is performed, and the conversion data corresponding to the left channel is obtained. Real part Al (1, j), Al (2, j), Al (3, j), imaginary part Bl (1, j), Bl (2, j), Bl (3, j), corresponding to the right channel Real part Ar (1, j), Ar (2, j), Ar (3, j), imaginary part Br (1, j), Br (2, j), Br (3, j) obtain. Note that the window functions W (1, i), W (2, i), and W (3, i) are functions whose values increase near the front (front), near the center, and near the rear of the acoustic frame, respectively. ing.

[Formula 5]
Al (1, j) = Σi _{= 0,..., N-1} W (1, i) .Xl (i) .cos (2πij / N)
Bl (1, j) = Σi _{= 0,..., N-1} W (1, i) · Xl (i) · sin (2πij / N)
Al (2, j) = Σi _{= 0,..., N-1} W (2, i) · Xl (i) · cos (2πij / N)
Bl (2, j) = Σi _{= 0,..., N-1} W (2, i) · Xl (i) · sin (2πij / N)
Al (3, j) = Σi _{= 0,..., N-1} W (3, i) · Xl (i) · cos (2πij / N)
Bl (3, j) = Σi _{= 0,..., N-1} W (3, i) .Xl (i) .sin (2πij / N)
Ar (1, j) = Σi _{= 0,..., N-1} W (1, i) .Xr (i) .cos (2πij / N)
Br (1, j) = Σ _{i = 0,..., N-1} W (1, i) · Xr (i) · sin (2πij / N)
Ar (2, j) = Σi _{= 0,..., N-1} W (2, i) .Xr (i) .cos (2πij / N)
Br (2, j) = Σi _{= 0,..., N-1} W (2, i) .Xr (i) .sin (2πij / N)
Ar (3, j) = Σi _{= 0,..., N-1} W (3, i) · Xr (i) · cos (2πij / N)
Br (3, j) = Σi _{= 0,..., N-1} W (3, i) · Xr (i) · sin (2πij / N)

  When the frequency conversion means 20 performs Fourier transform on the B type sound frame, the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1). The window function W (4, i) is used to perform processing according to the following [Equation 6], and the real part Al (4, j) and imaginary part Bl (4, j) of the conversion data corresponding to the left channel ), Real part Ar (4, j) and imaginary part Br (4, j) of the conversion data corresponding to the right channel are obtained.

[Formula 6]
Al (4, j) = Σi _{= 0,..., N-1} W (4, i) · Xl (i) · cos (2πij / N)
Bl (4, j) = Σi _{= 0,..., N-1} W (4, i) .Xl (i) .sin (2πij / N)
Ar (4, j) = Σi _{= 0,..., N-1} W (4, i) .Xr (i) .cos (2πij / N)
Br (4, j) = Σi _{= 0,..., N-1} W (4, i) · Xr (i) · sin (2πij / N)

  In the above [Formula 5] and [Formula 6], i is a serial number assigned to N samples in each acoustic frame, and takes an integer value of i = 0, 1, 2,. Further, j is a serial number assigned in order from the smallest value of the frequency value, and takes an integer value of j = 0, 1, 2,... N / 2-1 like i. When the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by 10.8 Hz.

  By executing the processing according to the above [Equation 5] and [Equation 6], a spectrum corresponding to each window function of each acoustic frame is obtained. Subsequently, the frequency component changing unit 30 extracts a spectrum set in a predetermined frequency range from the generated spectrum. In the present embodiment, a range between F1 and F2 is extracted.

  The frequency component changing unit 30 performs a process of changing the ratio of the predetermined frequency component for the A type sound frame according to the bit arrangement created by the bit arrangement creating unit 70. In the present invention, the bit arrangement is read bit by bit, and 1-bit information is embedded in one acoustic frame. There are two 1-bit values to be embedded: “0” and “1”. In the present embodiment, these are defined as value 1 and value 2. Since two types of codes can be embedded, these can also be expressed as code 1 and code 2. At this time, any one of “0” and “1” may be defined as a value 1 and a value 2 (reference numerals 1 and 2). This is because it is sufficient that one bit embedded on the extraction side can be specified on the extraction side. Therefore, this definition must match between the embedding side and the extraction side.

  In this embodiment, the component of the change target frequency band of the acoustic frame is changed to two states, and 1-bit information is embedded. Here, the state of change of the predetermined frequency component of the acoustic frame before and after the embedding process will be described. FIG. 3 shows a state of a predetermined frequency component of each channel 1 sound frame of A type and B type according to the present embodiment. In each acoustic frame shown in FIG. 3, the horizontal axis indicates the time direction, and the vertical axis indicates the frequency direction. A shaded portion indicates a portion where a frequency component exists, and the darker the shade, the stronger the component strength.

  In FIG. 3, the frequency region is divided into six in the frequency direction of the vertical axis, but the second to fifth regions from the bottom, that is, the frequency range from F1 to F2 is the change target frequency band. The uppermost part, that is, the frequency exceeding F2, and the lowermost part, that is, less than F1, are frequency bands not to be changed. That is, in the present embodiment, the intensity of the spectrum set is changed by setting the frequency F1 to F2 as a predetermined frequency range. For the change target frequency band of the A type acoustic frame of the L channel, the spectrum set in the front part is sequentially repeated from the low frequency side, L1Dq, L1Uq (q = 1,..., Q), and the spectrum set in the middle part is set from the high frequency side. In turn, L2Dq and L2Uq are repeated, and the spectrum set in the rear part is expressed in order from the high-frequency side by repetition of L3Dq and L3Uq. For the change target frequency band of the A type acoustic frame of the R channel, the spectrum set in the front part is In this order, R1Dq and R1Uq are repeated, the spectrum set in the middle is expressed in order from the high frequency side, R2Dq and R2Uq are repeated, and the spectrum set in the rear is expressed in order from the high frequency side by repetition of R3Dq and R3Uq. In this embodiment, since Q = 2, as shown in FIG. 3A, the repetition of L3Dq and L3Uq is L3D1, L3U1, L3D2, and L3U2. Also, for the change target frequency band of the L channel B type acoustic frame, the spectrum set is expressed by repeating L4Dq and L4Uq in order from the high frequency side, and for the change target frequency band of the R channel B type acoustic frame, the spectrum Assume that the set is expressed by repetition of R4Dq and R4Uq in order from the high frequency side. Therefore, as shown in FIG. 3B, the repetition of L4Dq and L4Uq is L4D1, L4U1, L4D2, and L4U2.

  In the present embodiment, when the code 1 is embedded, as shown in FIG. 3C, the components L1Uq and L3Dq are removed, and components equivalent to the removed components are added to the components R1Uq and R3Dq. Also, the components R1Dq and R3Uq are removed, and components equivalent to the removed components are added to the components L1Dq and L3Uq. This state is referred to as “state 1”. When embedding the code 2, as shown in FIG. 3D, the components L1Dq and L3Uq are removed, and components equivalent to the removed components are added to the components R1Dq and R3Uq. Further, the components R1Uq and R3Dq are removed, and components equivalent to the removed components are added to the components L1Uq and L3Dq. This state is referred to as “state 2”. Regardless of the bit value to be embedded, as shown in FIGS. 3C and 3D, the component of L2Dq is removed, and a component equivalent to the removed component is added to the component of R2Dq. Further, the R2Uq component is removed, and a component equivalent to the removed component is added to the L2Uq component.

  Regardless of the bit value to be embedded, as shown in FIG. 3E, the L4Dq component is removed, and a component equivalent to the removed component is added to the R4Dq component. Further, the R4Uq component is removed, and a component equivalent to the removed component is added to the L4Uq component.

  In this embodiment, information is embedded by changing the frequency component of the A-type sound frame in two states as shown in FIGS. Since there are two states, this corresponds to an information amount of 1 bit.

  As shown in FIGS. 3C, 3D, and 3E, in the present invention, the difference between the component intensities in the frequency band to be changed between channels is not so large in either state 1 or state 2. Will be changed as follows. Therefore, the signal of one of the channels is not extremely deteriorated, and the noise is not conspicuous.

  In the present embodiment, the change target frequency bands F1 to F2 are set to either “1.7 kHz to 3.4 kHz” or “3.4 kHz to 6.8 kHz”. This is due to the following reasons.

  The human sound source localization sensation tends to increase in the treble part, but the energy of the source sound source has a characteristic that it becomes smaller as the treble becomes higher, especially when it exceeds the telephone line bandwidth (300 Hz to 3.4 kHz), it becomes only the harmonic component. As a result, the audible sound source localization variation is small. As a result of the experiment, the audible sound source localization mutation increases in the region of 400 Hz to 1.5 kHz, but tends to decrease when the frequency exceeds 1.5 kHz, and almost disappears when the frequency exceeds 4 kHz. This is because there is almost no sound component above 4 kHz, and the fundamental tone of the instrument sound exceeds the highest range, so that only the overtone component is obtained.

  Therefore, the frequency range to be embedded is specifically examined. When a mobile phone having a high degree of spread as voice communication is used as a receiving terminal, the upper limit needs to be 3.4 kHz which is the upper limit of the telephone line band and the mobile phone. Therefore, the lower limit is set to 1.7 kHz, which is one octave lower than the upper limit of 3.4 kHz. Further, when a device other than a mobile phone is used as a receiving terminal, a frequency region higher than 3.4 kHz can be used as long as it is below the upper limit of audible frequency (22 kHz), but in a high sound region exceeding 10 kHz, compression / modulation is possible. The lower limit is set to 3.4 kHz, which is the upper limit of the telephone line bandwidth, and the upper limit is set to 6.8 kHz, which is one octave higher than 3.4 kHz. It was. Note that the values “1.7 kHz”, “3.4 kHz”, and “6.8 kHz” are representative values, and are not necessarily accurate values, and may be slightly deviated from them. In the present specification, “1.7 kHz to 3.4 kHz” is referred to as “telephone high frequency band”, and “3.4 kHz to 6.8 kHz” is referred to as “super telephone frequency band”. Further, since the upper limit of the telephone line band is around 3.4 kHz as described above, the above “telephone high frequency band” and “super telephone frequency band” are slightly lower than the upper limit of the telephone line band in the audible frequency range. This corresponds to a predetermined frequency band on the high sound side.

  The change of the frequency component will be described in more detail. When the information to be embedded is “value 1”, the processing according to the following [Equation 7] is executed for Q regions of q = 1 to Q. Thus, the state of the frequency component is changed to “state 1”, that is, the state shown in FIG.

[Formula 7]
For each component of j = m + 2P (q−1) to m + 2P (q−1) + P−1, Al ′ (2, j) = 0
Bl ′ (2, j) = 0
E (2, j) = {Al (2, j) ² + Bl (2, j) ² + Ar (2, j) ² + Br (2, j) ² } ^1/2
Ar ′ (2, j) = Ar (2, j) · E (2, j) / {Ar (2, j) ² + Br (2, j) ² } ^1/2
Br ′ (2, j) = Br (2, j) · E (2, j) / {Ar (2, j) ² + Br (2, j) ² } ^1/2
Al ′ (3, j) = 0
Bl ′ (3, j) = 0
E (3, j) = {Al (3, j) ² + B1 (3, j) ² + Ar (3, j) ² + Br (3, j) ² } ^1/2
Ar ′ (3, j) = Ar (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2
Br ′ (3, j) = Br (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2
E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^1/2
Al ′ (1, j) = Al (1, j) · E (1, j) / {Al (1, j) ² + Bl (1, j) ² } ^1/2
Bl ′ (1, j) = Bl (1, j) · E (1, j) / {Al (1, j) ² + Bl (1, j) ² } ^1/2
Ar ′ (1, j) = 0
Br ′ (1, j) = 0
For each component of j = m + 2P (q−1) + P to m + 2P (q−1) + 2P−1, Al ′ (1, j) = 0
Bl ′ (1, j) = 0
E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^1/2
Ar ′ (1, j) = Ar (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Br ′ (1, j) = Br (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
E (2, j) = {Al (2, j) ² + Bl (2, j) ² + Ar (2, j) ² + Br (2, j) ² } ^1/2
Al ′ (2, j) = Al (2, j) · E (2, j) / {Al (2, j) ² + Bl (2, j) ² } ^1/2
Bl ′ (2, j) = Bl (2, j) · E (2, j) / {Al (2, j) ² + Bl (2, j) ² } ^1/2
Ar ′ (2, j) = 0
Br ′ (2, j) = 0
E (3, j) = {Al (3, j) ² + B1 (3, j) ² + Ar (3, j) ² + Br (3, j) ² } ^1/2
Al ′ (3, j) = Al (3, j) · E (3, j) / {Al (3, j) ² + Bl (3, j) ² } ^1/2
Bl ′ (3, j) = B1 (3, j) · E (3, j) / {Al (3, j) ² + Bl (3, j) ² } ^1/2
Ar ′ (3, j) = 0
Br ′ (3, j) = 0

  In the above [Equation 7], m is the number of the lower limit component of the change target frequency band, and M is the number of the upper limit component of the change target frequency band. For example, when 1.7 kHz to 3.4 kHz is set as the change target frequency band, m = 160 and M = 320. Further, P = (M−m) / (2Q) (Q = 2 in this embodiment).

  In the above [Expression 7], when j = m + 2P (q−1) to m + 2P (q−1) + P−1, Al ′ (2, j), Bl ′ (2, j), Al ′ (3, j ), Bl ′ (3, j), Ar ′ (1, j), Br ′ (1, j) are set to 0, and j = m + 2P (q−1) + P to m + 2P (q−1) + 2P−1 Al ′ (1, j), Bl ′ (1, j), Ar ′ (2, j), Br ′ (2, j), Ar ′ (3, j), Br ′ (3, j) are set to 0 Yes. This indicates that each component in L1Uq, L3Dq, R1Dq, and R3Uq is set to 0 as shown in FIG. 3 (c). Since it is sufficient if the difference can be clarified, it is not necessarily required to be 0, and a small value is sufficient.

  When the information to be embedded is “value 2”, the processing according to the following [Equation 8] is executed for Q regions of q = 1 to Q, thereby changing the frequency component state to “state”. 2 ″, that is, the state shown in FIG.

[Formula 8]
For each component of j = m + 2P (q−1) to m + 2P (q−1) + P−1, Al ′ (2, j) = 0
Bl ′ (2, j) = 0
E (2, j) = {Al (2, j) ² + Bl (2, j) ² + Ar (2, j) ² + Br (2, j) ² } ^1/2
Ar ′ (2, j) = Ar (2, j) · E (2, j) / {Ar (2, j) ² + Br (2, j) ² } ^1/2
Br ′ (2, j) = Br (2, j) · E (2, j) / {Ar (2, j) ² + Br (2, j) ² } ^1/2
Al ′ (1, j) = 0
Bl ′ (1, j) = 0
E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^1/2
Ar ′ (1, j) = Ar (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Br ′ (1, j) = Br (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
E (3, j) = {Al (3, j) ² + B1 (3, j) ² + Ar (3, j) ² + Br (3, j) ² } ^1/2
Al ′ (3, j) = Al (3, j) · E (3, j) / {Al (3, j) ² + Bl (3, j) ² } ^1/2
Bl ′ (3, j) = B1 (3, j) · E (3, j) / {Al (3, j) ² + Bl (3, j) ² } ^1/2
Ar ′ (3, j) = 0
Br ′ (3, j) = 0
For each component of j = m + 2P (q−1) + P to m + 2P (q−1) + 2P−1, Al ′ (3, j) = 0
Bl ′ (3, j) = 0
E (3, j) = {Al (3, j) ² + B1 (3, j) ² + Ar (3, j) ² + Br (3, j) ² } ^1/2
Ar ′ (3, j) = Ar (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2
Br ′ (3, j) = Br (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2
E (2, j) = {Al (2, j) ² + Bl (2, j) ² + Ar (2, j) ² + Br (2, j) ² } ^1/2
Al ′ (2, j) = Al (2, j) · E (2, j) / {Al (2, j) ² + Bl (2, j) ² } ^1/2
Bl ′ (2, j) = Bl (2, j) · E (2, j) / {Al (2, j) ² + Bl (2, j) ² } ^1/2
Ar ′ (2, j) = 0
Br ′ (2, j) = 0
E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^1/2
Al ′ (1, j) = Al (1, j) · E (1, j) / {Al (1, j) ² + Bl (1, j) ² } ^1/2
Bl ′ (1, j) = Bl (1, j) · E (1, j) / {Al (1, j) ² + Bl (1, j) ² } ^1/2
Ar ′ (1, j) = 0
Br ′ (1, j) = 0

  In the above [Expression 8], when j = m + 2P (q−1) to m + 2P (q−1) + P−1, Al ′ (2, j), Bl ′ (2, j), Al ′ (1, j ), Bl ′ (1, j), Ar ′ (3, j), Br ′ (3, j) are set to 0, and Al in j = m + 2P (q−1) + P to m + 2P (q−1) + 2P−1 '(3, j), Bl' (3, j), Ar '(2, j), Br' (2, j), Ar '(1, j), Br' (1, j) are set to 0 . This indicates that each component in L1Dq, L3Uq, R1Uq, and R3Dq is set to 0 as shown in FIG. 3 (d). In “state 2”, the intensity of the spectrum set other than these components Since it is sufficient if the difference can be clarified, it is not necessarily required to be 0, and a small value is sufficient.

  As described above, since it is necessary to change the frequency component for the A type sound frame according to the bit value to be embedded, the frequency component changing means 30 performs the above [Formula 7] and [Formula 8]. Execute the process according to. On the other hand, the B-type acoustic frame is used to compensate for the difference in signal degradation between both channels and prevent the listener from feeling noise, so it is necessary to change the frequency component according to the bit value. There is no. Therefore, the frequency component changing unit 30 always executes the process according to the following [Equation 9] for B type acoustic frames for Q regions of q = 1 to Q.

[Formula 9]
For each component of j = m + 2P (q−1) to m + 2P (q−1) + P−1, Al ′ (4, j) = 0
Bl ′ (4, j) = 0
E (4, j) = {Al (4, j) ² + Bl (4, j) ² + Ar (4, j) ² + Br (4, j) ² } ^1/2
Ar ′ (4, j) = Ar (4, j) · E (4, j) / {Ar (4, j) ² + Br (4, j) ² } ^1/2
Br ′ (4, j) = Br (4, j) · E (4, j) / {Ar (4, j) ² + Br (4, j) ² } ^1/2
For each component of j = m + 2P (q−1) + P to m + 2P (q−1) + 2P−1, E (4, j) = {Al (4, j) ² + Bl (4, j) ² + Ar (4 , J) ² + Br (4, j) ² } ^1/2
Al ′ (4, j) = Al (4, j) · E (4, j) / {Al (4, j) ² + Bl (4, j) ² } ^1/2
Bl ′ (4, j) = B1 (4, j) · E (4, j) / {Al (4, j) ² + Bl (4, j) ² } ^1/2
Ar ′ (4, j) = 0
Br ′ (4, j) = 0

  As described above, the frequency inverse transform means 40 performs the process of obtaining the modified sound frame by performing the frequency inverse transform on the frame spectrum in which the state of the frequency component is changed. Naturally, the inverse frequency conversion needs to correspond to the method executed by the frequency conversion means 20. In the present embodiment, since the frequency transform unit 20 performs the Fourier transform, the frequency inverse transform unit 40 executes the Fourier inverse transform.

  Specifically, for the A type sound frame, the frequency inverse transform means 40 performs the real part Al ′ (1,1) of the left channel of the spectrum obtained by any of the above [Formula 7] and [Formula 8]. j), imaginary part Bl ′ (1, j), etc., real part Ar ′ (1, j), etc. of the right channel, imaginary part Br ′ (1, j), etc. The process according to this is performed and Xl '(i) and Xr' (i) are calculated. For frequency components that are not modified in the above [Equation 7] and [Equation 8], the original frequency component Al (1, j) or the like is used as Al ′ (1, j) or the like.

[Formula 10]
Xl' (i) = 1 / N · {Σ j Al' (1, j) · cos (2πij / N) -Σ j Bl' (1, j) · sin (2πij / N)} + 1 / N · { Σ _j Al ′ (2, j) · cos (2πij / N) −Σ _j Bl ′ (2, j) · sin (2πij / N)} + 1 / N · {Σ _j Al ′ (3, j) · cos (2πij / N) −Σ _j Bl ′ (3, j) · sin (2πij / N)} + Xlp (i + N / 2)
Xr' (i) = 1 / N · {Σ j Ar' (1, j) · cos (2πij / N) -Σ j Br' (1, j) · sin (2πij / N)} + 1 / N · { _{Σ j Ar' (2, j)} · cos (2πij / N) -Σ j Br' (2, j) · sin (2πij / N)} + 1 / N · {Σ j Ar' (3, j) · cos _{(2πij / N) -Σ j Br'} (3, j) · sin (2πij / N)} + Xrp (i + N / 2)

In the above [Expression 10], Σ _{j = 0,} ... _{, N−1} is shown as Σ _{j in} order to prevent the expression from becoming complicated. The terms “+ Xlp (i + N / 2)” in the first equation and “+ Xrp (i + N / 2)” in the second equation in the above [Equation 10] indicate the data Xlp (i) of the modified acoustic frame modified immediately before, When Xrp (i) exists, the addition is performed in consideration of the overlap of N / 2 samples on the time axis. By the above [Equation 10], each sample Xl ′ (i) of the left channel and each sample Xr ′ (i) of the right channel of the A type modified acoustic frame are obtained.

  For the B type acoustic frame, the frequency inverse transform means 40 performs real part Al ′ (4, j) and imaginary part Bl ′ (4, j) of the left channel of the spectrum obtained by the above [Equation 9]. , Using the real part Ar ′ (4, j) and the imaginary part Br ′ (4, j) of the right channel, the processing according to the following [Equation 11] is performed, and Xl ′ (i), Xr ′ (i ) Is calculated. For frequency components that are not modified in the above [Equation 9], in the following [Equation 11], Al ′ (4, j), Bl ′ (4, j), Ar ′ (4, j), Br The original values Al (4, j), Bl (4, j), Ar (4, j), and Br (4, j) are used as ′ (4, j).

[Formula 11]
Xl' (i) = 1 / N · {Σ j Al' (4, j) · cos (2πij / N) -Σ j Bl' (4, j) · sin (2πij / N)} + Xlp (i + N / 2 )
Xr' (i) = 1 / N · {Σ j Ar' (4, j) · cos (2πij / N) -Σ j Br' (4, j) · sin (2πij / N)} + Xrp (i + N / 2 )

  By the above [Equation 11], each sample Xl ′ (i) of the left channel and each sample Xr ′ (i) of the right channel of the B type modified acoustic frame are obtained.

  The modified sound frame output unit 50 sequentially outputs the A type modified sound frame and the B type modified sound frame obtained by the processing of the frequency inverse transform unit 40 to an output file.

  Next, the overall flow of processing of the information embedding device for the acoustic signal shown in FIG. 1 will be described with reference to the flowchart of FIG. Each component which comprises the apparatus shown in FIG. 1 cooperates, and performs the process according to FIG. FIG. 4 corresponds to processing of one word of additional information. One word can be set to an arbitrary number of bits, but as described above, in this embodiment, it is set to substantially 7 bits. Further, since information embedding is performed on an A type acoustic frame, FIG. 4 illustrates the A type acoustic frame. The B type acoustic frame is read by the acoustic frame reading means 10 in parallel with the A type acoustic frame, and is frequency converted by the frequency converting means 20 using the window function W (4, i). The frequency component is changed by the processing according to [Equation 9] by the frequency component changing means 30, and the frequency is inversely converted by the frequency inverse converting means 40, and then output by the modified acoustic frame output means 50.

  In FIG. 4, first, the bit array creation means 70 performs a predetermined process on the additional information extracted from the additional information storage unit 62 to create a new bit array (S101). Specifically, first, it is extracted from the additional information storage unit 62 in units of 1 word (7 bits), and 1 bit of parity bit is added thereto to make 8 bits. Then, the bits having the value of the exclusive OR of two consecutive 8 bits are alternately arranged in 8 bits to create a 16-bit bit array. FIG. 5 shows how the bit arrangement is changed by this processing. FIG. 5A shows a bit arrangement after adding parity bits. In FIG. 5A, the eighth bit is a parity bit. In such a case, first, a check code string that is an exclusive OR of the first to eighth bits is obtained, and a bit string obtained by alternately replacing each bit obtained by the exclusive OR from the beginning is the ninth. A new bit array having bits to 16th bits is created. As a result, the arrangement as shown in FIG.

  The inspection codes P1 to P8 are defined as shown in FIG. That is, the check code P1 is calculated as an exclusive OR of the bits D1 and D2. The check code P2 is calculated as an exclusive OR of the bits D3 and D4. The check code P3 is calculated as an exclusive OR of the bits D5 and D6. The check code P4 is calculated as an exclusive OR of the bits D7 and D8. The check code P5 is calculated as an exclusive OR of the bit value “1” and the bit D1. The check code P6 is calculated as an exclusive OR of the bit value “1” and the bit D3. The check code P7 is calculated as an exclusive OR of the bit value “1” and the bit D5. The check code P8 is calculated as an exclusive OR of the bit value “1” and the bit D7. Therefore, the bit array creation means 70 creates a 16-bit bit array as shown in FIG. 5B in S101 according to the definition shown in FIG. 5C. These 16 bits are read into a register in a computer used as an information embedding device for an acoustic signal. Thus, in the additional information storage unit 62, one word is 7 bits, but at the time of embedding processing, processing for one word in the additional information is performed with this 16-bit array. In the bit array thus created, the first half bit string is referred to as the first half bit string, and the second half bit string is referred to as the second half bit string.

  Next, the frequency component changing unit 30 performs a process of reading 1 bit from 16 bits held in the register (S102). Subsequently, the acoustic frame reading means 10 reads a predetermined number of samples as one acoustic frame from each of the left and right channels of the stereo acoustic signal stored in the acoustic signal storage unit 61 (S103). Subsequently, the frequency converting unit 20 and the frequency component changing unit 30 perform processing for changing the state of the frequency component of the acoustic frame to either “state 1” or “state 2” (S104). Specifically, first, frequency conversion is performed on the read sound frame to obtain a frame spectrum that is a spectrum of the sound frame. That is, for each acoustic frame, the processing according to [Formula 5] is performed using the three window functions W (1, i), W (2, i), and W (3, i). Then, the frequency component changing unit 30 executes processing according to the above [Formula 7] and [Formula 8] according to the value 1 and the value 2 received from the bit array creating unit 70, and the state of the component of the frequency band to be changed Is changed to either “state 1” or “state 2”.

  Next, the frequency inverse transform means 40 performs the process of obtaining the modified acoustic frame by performing the frequency inverse transform on the frame spectrum in which the ratio between the spectrum sets of each window component is changed by the process of S104 (S105). As a matter of course, this frequency inverse transform needs to correspond to the method executed by the frequency converter 20 in S104. In the present embodiment, since the frequency transform unit 20 performs the inverse Fourier transform, the frequency inverse transform unit 40 performs the inverse Fourier transform. Specifically, the real channel Al ′ (1, j), etc., the imaginary part Bl ′ (1, j), etc. of the left channel of the spectrum obtained by any of the above [Equation 7] and [Equation 8], the right channel. Using the real part Ar ′ (1, j), etc., the imaginary part Br ′ (1, j), etc., the processing according to the above [Equation 10] is performed, and Xl ′ (i), Xr ′ (i) calculate.

  The modified sound frame output means 50 sequentially outputs the obtained modified sound frames to an output file. When the processing for one acoustic frame is completed in this way, the frequency component changing means 30 reads the next 1 bit in the bit array (S102). The above processing is executed over all samples of both channels of the acoustic signal. That is, a predetermined number of samples are read as sound frames, and when there are no more sound frames to be read from the sound signal (S103), the process ends. When the processing corresponding to each bit of the bit arrangement (16 bits) for one word read in S101 is completed, the process returns from S102 to S101, the next 7 bits of the additional information are read, and the 16-bit bit arrangement is changed. The process to create will be performed. When the processing is completed for all the words of the additional information, the processing returns to the first word of the additional information. As a result, all modified acoustic frames that have been processed for all acoustic frames are recorded in the output file and obtained as modified acoustic signals. The obtained modified acoustic signal is output to and stored in the modified acoustic signal storage unit 63 in the storage unit 60.

  In this embodiment, additional information is set to 7 bits per word, 1 bit of parity is added, a check code is added to 16 bits, and processing is performed for 1 word of additional information. As long as there is an agreement with the side, it is possible to record one word of additional information in an arbitrary number of bits.

  Of the left channel of the modified acoustic signal obtained as described above, with respect to the portion where the additional information is embedded, the component of the change target frequency band has only two distributions of the state 1 and the state 2 described above. become. However, since the components other than the component of the frequency band to be changed remain as the original acoustic signals, various distributions are made based on the setting of the producer. Further, as is apparent from the processing of [Formula 7] and [Formula 8], the component of the change target frequency band changed in the left channel is always added to the component of the change target frequency band of the right channel, and vice versa. In addition, the component of the change target frequency band changed in the right channel is always added to the component of the change target frequency band of the left channel. Accordingly, since the left and right channels supplement each other with the components deleted from each other, there is no signal degradation when viewed as both channels as a whole.

(2.1. Configuration of extraction device)
Next, an apparatus for extracting information from an acoustic signal according to the present invention will be described. FIG. 6 is a block diagram showing an embodiment of an apparatus for extracting information from an acoustic signal according to the present invention. In FIG. 6, 100 is an acoustic signal input unit, 110 is a reference frame acquisition unit, 120 is a phase change frame setting unit, 130 is a frequency conversion unit, 140 is a code determination parameter calculation unit, 150 is a code output unit, and 160 is additional information. Extraction means 170 is an acoustic frame holding means.

  The acoustic signal input unit 100 has a function of acquiring and inputting a flowing sound as a digital acoustic signal. In reality, it is realized by a microphone and an A / D converter. Any microphone can be used, regardless of whether it is monaural omnidirectional or stereo directional, as long as the component in the frequency band to be changed can be detected. Even if it is stereo-directional, only one channel needs to be used. Further, when information is embedded by the apparatus shown in FIG. 1, the information can be extracted even if a microphone with a general accuracy is used instead of a high accuracy. The reference frame acquisition unit 110 has a function of reading an audio frame composed of a predetermined number of samples as a reference frame from the input digital monaural audio signal (or one channel of a stereo audio signal). The phase change frame setting means 120 has a function of setting, as a phase change frame, an acoustic frame whose phase has been changed by moving the reference frame and a predetermined sample at a time. The frequency conversion means 130 has the same function as the frequency conversion means 20 shown in FIG.

  The code determination parameter calculation unit 140 extracts a spectrum set corresponding to a predetermined frequency range from the generated spectrum, and calculates the intensity values E1q to E4q of each spectrum set based on the following [Equation 12] from q = 1 to 1. Q set is calculated corresponding to Q. Similarly, the code determination parameters C1q to C4q are calculated in Q sets using the intensity values of the components corresponding to the respective elements j, and the magnitude relationship of the sum values of q = 1 to Q of the code determination parameters C1q to C4q is calculated. Based on this, it has a function of determining that it is in a predetermined state.

[Formula 12]
E1q = Σj = _{m + 2P (q-1), ..., m + 2P (q-1) + P-1} {Al (1, j) ² + Bl (1, j) ² } .F (j-m -2P (q-1)). (Αj) ²
E2q = Σ _{j = m + 2P (q-1), ..., m + 2P (q-1) + P-1} {Al (3, j) ² + Bl (3, j) ² } · F (j−m -2P (q-1)). (Αj) ²
E3q = Σj = _{m + 2P (q-1) + P, ..., m + 2P (q-1) + 2P-1} {Al (1, j) ² + Bl (1, j) ² } · F (j -M-P-2P (q-1)). (Αj) ²
E4q = Σ _{j = m + 2P (q-1) + P, ..., m + 2P (q-1) + 2P-1} {Al (3, j) ² + Bl (3, j) ² } · F (j -M-P-2P (q-1)). (Αj) ²

In the above [Equation 12], (αj) ² and F (j−m−2P (q−1)) are both functions used to correct the spectral elements corresponding to each j. Among these, (αj) ² is a correction function generally expressed by (αf) ^β , and shows a case where β = 2.0. Since j is a value proportional to the frequency f, in [Equation 12], j is used instead of the frequency f. Here, the relationship between the frequency of the acoustic signal and the energy distribution is shown in FIG. As shown in FIG. 7, the energy distribution of the acoustic signal tends to decrease in proportion to the square of the frequency f. For this reason, when it is determined whether or not information is embedded in the high frequency component, it is more difficult to make an erroneous determination if the signal intensity is corrected to be larger than that of the low frequency component. Therefore, in the present invention, the intensity values E1q to E4q corrected by the correction function (αf) ^β are obtained. In addition, α is a real number constant of 1 or less, and in this embodiment, α = 0.178 · (512 / N). F (j−m−2P (q−1)) is corrected by the lower limit m + 2P (q−1) of each frequency region in the frequency direction window function F (j) defined by the following [Equation 13]. Is. Although these functions act on each spectral element, in calculating the intensity of the spectrum set, the above [Equation 12] calculates the total sum of the intensities of each spectral element, so that it acts on the sum. Yes.

[Formula 13]
When 0 ≦ j <P / 8, F (j) = 0.5j ² / (P / 8) ²
When P / 8 ≦ j <P / 4, F (j) = 1.0−0.5 · (j−P / 4) ² / (P / 8) ²
When P / 4 ≦ j <3P / 4, F (j) = 1.0
When 3P / 4 ≦ j <7P / 8, F (j) = 1.0−0.5 · (j−3P / 4) ² / (P / 8) ²
When 7P / 8 ≦ j <P, F (j) = 0.5 (P−j) ² / (P / 8) ²

In the above [Formula 13], P indicates the number of components of the spectrum set in the predetermined frequency band, and P = (M−m) / (2Q). Here, a graph of the frequency direction window function F (j) and the correction function (αj) ² is shown in FIG. In the above [Equation 12], the frequency direction window function F (j) and the correction function (αj) ² are multiplied for each of the P spectral elements, where j is m to m + P−1 and m + P to M−1. Will be. A more detailed graph of the frequency direction window function F (j) is shown in FIG. As can be seen from a comparison between FIG. 9 and the above [Equation 13], the frequency direction window function F (j) is divided into five according to the value of j, and particularly when j <P / 8 and j ≧ 7P /. When the value is 8, the value decreases. For this reason, when the frequency direction window function F (j) extracts a spectrum set by dividing a predetermined frequency band into a plurality of frequency directions, the spectrum set on the low frequency component side and the spectrum set on the high frequency component side are clearly defined. Help to distinguish.

In addition, the sign determination parameter calculation unit 140 includes Al (1, j) ² + Bl (1, j = m + 2P (q−1),..., M + 2P (q−1) + P−1. j) The number of elements satisfying ² > Al (3, j) ² + Bl (3, j) ² is calculated as the code determination parameter C1q, and P−C1q is calculated as the code determination parameter C2q. In addition, among P elements of j = m + 2P (q−1) + P,..., M + 2P (q−1) + 2P−1, Al (1, j) ² + Bl (1, j) ² > Al (3 , J) ² + Bl (3, j) ² is calculated as the code determination parameter C3q, and P−C3q is calculated as the code determination parameter C4q.

  The code output means 150 determines what is determined to be the optimum phase from the acoustic frames (reference frame and phase change frame) corresponding to one reference frame, and selects a code corresponding to the state of the acoustic frame. It has a function to output. The code determination parameter calculation unit 140 and the code output unit 150 constitute an encoding unit. The additional information extraction unit 160 has a function of converting the binary array output by the code output unit 150 according to a predetermined rule and extracting it as meaningful additional information. The acoustic frame holding means 170 is a buffer memory that can hold two consecutive reference frames. Each component shown in FIG. 6 is actually realized by mounting a dedicated program on hardware such as a small computer having an information processing function and its peripheral devices. In particular, in order to more easily achieve the object of the present invention, it is desirable to use a portable terminal device such as a cellular phone as hardware. Note that even a portable terminal device such as a cellular phone is a type of computer having an arithmetic processing function.

(2.2. Processing operation of extraction device)
Next, the processing operation of the apparatus for extracting information from the acoustic signal shown in FIG. 6 will be described. The extraction apparatus according to the present invention can be set not to perform error correction when an error is detected by error checking, or can be set to perform 1-bit error correction. Hereafter, the processing operation of the extraction apparatus in a setting in which error correction is not performed will be described according to the flowchart of FIG. First, in this apparatus, the channel mode, the phase determination table S (p), the phase determination log, the phase determination flag, and the bit counter are initialized (S200). The channel mode specifies a channel from which information is to be extracted, and is set to “L” in the initial state. The phase determination table S (p) is a table for determining the phase, and p takes an integer value of 0 to 5. The initial value is set to S (p) = 0. The phase determination log records the determined phase, that is, the phase number p for each set of one reference frame and five phase change frames, and 0 is set in the initial state. The phase determination flag is a flag indicating whether or not the phase is fixed, and is set to Off in the initial state. For the bit counter, 0 is set as an initial value.

  In this way, when the initial value is set and the user wants to know the attribute information such as the song name of the music that is flowing, first, the extraction device Instruct startup. For example, this can be executed by operating a predetermined button when the extraction device is realized by a mobile terminal such as a mobile phone. When an instruction is input to the extraction device, the acoustic signal input unit 100 records the flowing music, digitizes it, and inputs it as a digital acoustic signal. More specifically, the audio input from the omnidirectional microphone (or one channel of the directional microphone) is digitized by the A / D converter.

  Subsequently, the reference frame acquisition unit 110 extracts an acoustic frame including a predetermined number of samples from the acoustic signal input from the acoustic signal input unit 100 as a reference frame (S201). Specifically, the reference frame is extracted and read into the acoustic frame holding unit 170. The number of samples of one acoustic frame read as the reference frame by the reference frame acquisition unit 110 needs to be the same as that set by the acoustic frame reading unit 10 shown in FIG. Therefore, in the present embodiment, the reference frame acquisition unit 110 sequentially reads 4096 samples as reference frames. The acoustic frame holding means 170 can store two reference frames as described above, and when a new reference frame is read, the old reference frame is discarded. Therefore, the sound frame holding means 170 always stores two reference frames (continuous 8192 samples).

  The acoustic frame processed by the extraction device can be divided into a reference frame that is set adjacently without interruption from the beginning, and a phase change frame in which the phase is changed. For the reference frame, after setting sample number 1 to sample number 4096 as the first reference frame, the next reference frame is sample number 4097 to sample number 8192, and the next reference frame is sample number 8193 to sample number 12288. It is set without interruption. Then, for each reference frame, five phase change frames moved by 1/6 frame (about 683 samples) are set. For example, for the first reference frame, five phase change frames configured by 4096 samples starting from sample numbers 683, 1366, 2049, 2732, and 3413 are set. Subsequently, the frequency conversion unit 130 and the code determination parameter calculation unit 140 specify the phase of each read sound frame, determine embedded information, and output a corresponding code (S202). The format of the information to be output has two formats of value 1 and value 2 corresponding to the case where the additional information is embedded.

  Here, details of phase determination and code determination in step S202 will be described with reference to the flowchart of FIG. First, it is confirmed whether the phase determination flag is On or Off (S301). When the phase determination flag is On, the phase determination process (S303 to S309) is not performed, and only the code determination process is performed (S302). However, since the phase is not fixed in the initial state and the phase determination flag is Off, the candidate code table is initialized (S303). The candidate code table records a phase number of 0 to 5 that specifies one reference frame and five phase change frames, and a binary code obtained from the states of the six acoustic frames.

  Subsequently, the code determination parameter calculation unit 140 performs a code determination process (S302). Details of the code determination process are shown in FIG. First, the frequency conversion means 130 performs frequency conversion on each read sound frame to obtain each window spectrum (S401). This process is the same as the process in the frequency conversion means 20 shown in FIG. However, since only the left channel is used for extraction, the processing according to [Formula 5] is performed, and the imaginary part Bl (1) such as the real part Al (1, j) of the conversion data corresponding to the left channel is performed. , J) etc.

  By the processing in the frequency conversion means 130, a frame spectrum expressed by a spectrum that is a component corresponding to the frequency is obtained. Subsequently, the code determination parameter calculation unit 140 calculates the code determination parameter C1q to the code determination parameter C4q as described above, and then uses the code determination parameter C1q to the code determination parameter C4q to calculate the component of the frequency band to be changed. Processing is performed to determine what state is, that is, what value is embedded as a 1-bit value (S402). Specifically, first, the following determination processing is executed, and as a result, it is determined that the corresponding state is obtained, and the corresponding value is output.

If Σq _{= 1,..., Q} (C1q + C4q)> Σq _{= 1,..., Q} (C2q + C3q), it is determined that the state is “1”, and a value 1 is output.
In cases other than the above, it is determined that the state is “state 2”, and the value 2 is output.

  The code determination parameter calculation means 140 sets either value 1 or value 2 in the candidate code table B (p) according to the determination result for each acoustic frame (S403). Specifically, when the value is 1, B (p) = 0, and when the value is 2, B (p) = 1.

  Further, as a result of the above determination, when either value 1 or value 2 is set in the candidate code table B (p) in the phase p, the phase determination table S (p) Update is performed (S404).

[Formula 14]
If it is determined that the status is "1" and the value 1 is output,
S (p) ← S (p) + Σ _{q = 1,..., Q} (E1q + E4q−E2q−E3q)
If it is determined that the status is "2" and the value 2 is output,
S (p) ← S (p) + Σ _{q = 1,..., Q} (E2q + E3q−E1q−E4q)

  Next, it is determined whether or not the channel mode is R (S405). If the channel mode is R as a result of the determination, a sign inversion process is executed (S406). That is, when value 1 is set in S403, the value is inverted to value 2, and when value 2 is set in S403, it is inverted to value 1. That is, the candidate code table B (p) is changed to 1-B (p). As a result of the determination in S405, if the channel mode is not R, that is, if the channel mode is L, the sign inversion process is not performed and the value (sign) set in S403 is output as it is.

  Returning to the flowchart of FIG. 11, the code determination parameter calculation unit 140 stores the code temporarily determined in the code determination process (S302) in the phase p in the candidate code table (S304).

  Subsequently, it is determined whether or not the processing corresponding to all the phase numbers p has been completed (S305). This determines whether all phase change frames have been processed for a certain reference frame. In this embodiment, since p takes a value from 0 to 5, if the processing is not performed six times, an acoustic frame having a different phase is set by shifting a predetermined number of samples from the acoustic frame being processed. Return and repeat the process. The case where p = 0 is a reference frame, and the case where p = 1 to 5 is a phase change frame. When the processing corresponding to all the phase numbers p is completed, it is determined that the phase corresponding to the phase number pmax having the maximum value in the phase determination table S (p) is the optimum phase, and the candidate code table B (p The code B (pmax) recorded in () is output (S306).

  Subsequently, the phase determination log is updated (S307). The phase determination log records the determined phase, that is, the phase number p for each set of one reference frame and five phase change frames. Then, referring to the phase determination log, it is determined whether or not the phase has been the same a predetermined number of times in the past (S308). In the present embodiment, this number is 10 times. If the phase has been the same a predetermined number of times in the past, the phase determination flag is set to On (S309). As a result, when the same phase continues for a predetermined number of times, the optimum phase is likely to be pmax. Therefore, the phase determination process (S303 to S309) is not performed, and the code determination process is performed only for the phase number p = pmax. (S302) is performed.

  Returning to the flowchart of FIG. When the phase is determined and the code is output, 1 bit corresponding to the output code value is stored in the buffer (S203). Next, the bit counter is incremented by “1” (S204).

  Next, it is determined whether the bit counter is 15 or less or 16 or more (S205). If the bit counter is equal to or smaller than 15, the process returns to S201 to perform processing for extracting the next reference frame.

  If the bit counter is 16 or more, a check bit value is calculated from the first half bit string in the bit array stored in the buffer (S206). Specifically, an exclusive OR of predetermined two bits in the first half bit string and the bit value “1” and the predetermined bit is calculated. This follows the rule of the check code calculation performed on the embedding side. As a result, eight check codes are obtained. Subsequently, the calculated check code is compared with the latter half check code (S207). Specifically, the eight check codes calculated as the exclusive OR in S206 and the latter half check code are collated in order. As a result, when all the eight pieces match, it is determined that they are perfectly matched. Further, if the first check code to the fourth check code are all inconsistent and the fifth check code to the eighth check code are all in agreement, it is determined that they are completely incompatible. In other cases, it is judged as nonconforming. If it is determined in S207 that it is nonconforming, the process returns to S201 to perform processing for extracting the next reference frame.

  If it is determined in S207 that it is a perfect match, the additional information extraction unit 160 adds 1 bit to the previous 7 bits in the first 8 bits and outputs the result (S208). In the case of perfect match in S207, there is a high possibility that the first bit in the buffer is the first bit of the word in the additional information. Therefore, by adding bit “0” to the 7 bits from the head, it is output as one word in the ASCII code. In the case of non-conformity other than complete non-conformity, there is a high possibility that the first half bit string used for collation is shifted from the word in the additional information. In this case, the first 1 bit is discarded, and processing for obtaining a new 1 bit is performed by the processing from S201 to S204.

  If it is determined in S207 that it is completely non-conforming, if all the bit values are inverted, it is completely conforming. In this case, the word is correctly extracted from the beginning of the word, but the extraction target channel may be reversed. Therefore, in this case, the bit in the buffer is inverted (S210). Further, since the channel is reversed, the channel mode is changed (S211). Then, a bit “0” is added to the previous 7 bits among the bits in the buffer inverted in S210 and output (S208).

  If it is judged as perfect conformity or complete nonconformity, there is a high possibility that the part is a word delimiter, so if it is really a delimiter, then if it is extracted 16 bits at a time, all will be accurately extracted in word units It can be performed. On the other hand, even if it is determined that it is a perfect match or a complete non-conformity, it may be a coincidence and not actually a word break. In such a case, there is a high possibility of nonconformity at the next inspection, and the correct division can be accurately determined after repeating several times. In S208, when 1 bit is added to the previous 7 bits and output, the bit counter is initialized to 0 (S209). Then, the process returns to S201 to perform processing for extracting the next reference frame.

  By executing the processing shown in FIG. 10 for each reference frame, additional information is extracted. If it is determined in S201 that all reference frames have been extracted, the process ends.

  In the process of S208, the additional information extraction unit 160 first converts the value output by the code output unit 150 according to a predetermined rule and extracts it as meaningful additional information. As the predetermined rule, various rules can be adopted as long as the information intended by the person who embeds the information can be recognized by the recipient. In this embodiment, the ASCII code is adopted. . That is, the additional information extraction unit 160 recognizes the bit value array obtained from the code determined by the code determination parameter calculation unit 140 and output from the code output unit 150 in units of 1 byte (8 bits), and recognizes this as ASCII. Recognizes character information according to the code. The character information thus obtained is displayed and output on a screen of a display device (not shown).

  Therefore, if the embedding device embeds the attribute information such as the song title or artist in the sound signal as the character information, the user wants to know the song title or artist by listening to the music being played. If a predetermined operation is performed on the mobile terminal that functions as the extraction device, attribute information such as a song title and an artist is displayed as character information on the screen of the mobile terminal.

  In the above example, the processing operation of the extraction apparatus in a setting in which error correction is not performed has been described according to the flowchart of FIG. When error correction is not performed, it is possible to output one word of additional information only when no error occurs.

  Next, the processing operation of the extraction apparatus in a setting for performing 1-bit error correction will be described with reference to the flowchart of FIG. 13 includes a portion that performs the same processing as in FIG. Therefore, portions that perform the same processing as in FIG. Also in the example of FIG. 13, first, initialization processing is performed (S200). In this initialization process, the channel mode, phase determination table S (p), phase determination log, phase determination flag, and bit counter are initialized as in the example of FIG. 10, but the automatic correction mode is set to OFF. I do.

  Subsequently, as in FIG. 10, after extracting an acoustic frame composed of a predetermined number of samples as a reference frame (S201), after identifying the phase of each read acoustic frame, the embedded information is determined. The corresponding code is output (S202). Note that the processing of S202 is as shown in FIGS.

  When the phase is determined and the code is output, 1 bit corresponding to the output code value is stored in the buffer (S203), and the bit counter is incremented by "1" (S204). Then, it is determined whether the bit counter is 15 or less or 16 or more (S205). If the bit counter is 15 or less, the process returns to S201 to perform processing for extracting the next reference frame.

  If the bit counter is 16 or more, a check code is calculated from the first half bit string in the bit array stored in the buffer (S206). Subsequently, the calculated check code is compared with the latter half check code (S207). If it is determined in S207 that it is completely conforming or completely incompatibility, processing for setting the automatic correction mode to ON is performed immediately before the bit string output processing in S208 (S212).

  If it is determined in S207 that it is nonconforming, it is determined whether one bit in the first half bit string which is an error can be specified (S213). Specifically, when any one of the first check code to the fourth check code does not match, it can be determined that the second bit, the fourth bit, the sixth bit, and the eighth bit are errors, respectively, If any one of the fifth check code, the second check code and the sixth check code, the third check code and the seventh check code, the fourth check code and the eighth check code does not match, the first bit, It can be judged as an error of 3 bits, 5th bit and 7th bit. If one error bit cannot be specified, the process of setting the automatic setting mode to OFF is performed (S214), and the process returns to S201 to perform the process of extracting the next reference frame.

  As a result of the determination in S213, if one error bit can be identified, it is determined whether the automatic correction mode is ON or OFF (S215). If the automatic correction mode is OFF, the process returns to S201 to perform processing for extracting the next reference frame.

  If the automatic correction mode is ON, the specified error 1 bit in the first half bit string is inverted and corrected (S216). Subsequently, a bit “0” is added to the previous 7 bits of the in-buffer bits that have been inverted and corrected (S208).

  As shown in the example of FIG. 13, in the case of setting for 1-bit error correction, even if a 1-bit error occurs, it is possible to output 1 word of additional information.

(2.3. About phase correction processing)
As described above, at the time of extraction, it is not always possible to read an acoustic signal corresponding to the acoustic frame embedded at the time of embedding. Therefore, the phase of the acoustic frame is shifted and read in a plurality of ways (six in this embodiment), the optimum phase is determined, and a code corresponding to the acoustic frame specified by the phase is output. Yes. For example, in the case of reading in six ways, the top acoustic frame is originally a sample of sample numbers 1 to 4096, but six pieces composed of 4096 samples starting from sample numbers 1, 683, 1366, 2049, 2732, and 3413 Are processed, and a code corresponding to the optimum acoustic frame is output. As described with reference to the flowchart of FIG. 11, in the present embodiment, when the same phase continues a predetermined number of times, the processing is performed after that phase is determined.

(3. Method of reliably embedding information even if the signal component is small)
As described so far, in the present invention, the signal component of the frequency band to be changed is always changed regardless of the state of the original signal component, but the magnitude of the component is the signal of both channels. There is no greater than strength. For this reason, if the original signal component does not exist or is too small, the signal component necessary for extraction cannot be obtained, and information cannot be appropriately extracted, resulting in an extraction error. Therefore, hereinafter, a method for enabling signal embedding so that information can be appropriately extracted even when the original signal component is small will be described.

  Conventionally, in order to appropriately extract information when the original signal component is small, as shown in Patent Document 3, by giving a predetermined fixed value as the intensity of the component in the spectrum set, It was possible to embed information. However, this method has a problem that the reproduction quality is deteriorated. Therefore, in this embodiment, weak white noise that is difficult to hear is superimposed on the acoustic signal.

  In this case, the acoustic frame reading means 10 shown in FIG. 1 has a function of adding or subtracting the same value in units of a predetermined number of samples within a predetermined amplitude range after reading a predetermined number of samples as one acoustic frame. Yes. In this embodiment, with respect to the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1), an amplitude range of −32 to +32 (Xl (i), Xr ( i) is a value between −32768 to +32767), and a uniform random number H is generated, and the same value H is used between K (K = 5 in the present embodiment) samples as follows: The processing as shown in FIG. 5 is executed to update the values of Xl (i) and Xr (i).

[Formula 15]
Xl (i) ← Xl (i) + H
Xr (i) ← Xr (i) -H

  That is, white noise is generated in one acoustic frame by executing the process according to the above [Formula 15] over N samples. The white noise generation process is performed immediately after the acoustic frame extraction process of S103 in the flowchart of FIG.

  As described above, even when white noise is generated, the configuration of the extraction device for extracting information from the acoustic signal on the extraction side is the same as that in FIG. 6, and the processing operation is shown in the flowcharts of FIGS. It is the same as followed.

(4.1 When embedding multiple bits in a sound frame)
In the above embodiment, the case where 1 bit is embedded in one acoustic frame has been described as an example, but it is also possible to embed 2 bits or more in one acoustic frame. Next, such a method will be described.

  First, a case will be described in which the components of the frequency band to be changed of the acoustic frame are changed to four states and 2-bit information is embedded. FIG. 14 shows a state of a predetermined frequency component of each type A channel 1 acoustic frame in this case. The B type channel 1 sound frame is the same as that shown in FIG. Also in FIG. 14, as in FIG. 3, the horizontal axis indicates the time direction, and the vertical axis indicates the frequency direction. A shaded portion indicates a portion where a frequency component exists, and the darker the shade, the stronger the component strength.

  Also in FIG. 14, as in FIG. 3, the frequency region is divided into six in the frequency direction of the vertical axis, and the second to fifth regions from the bottom, that is, between F1 and F2 is the frequency to be changed. The uppermost band, that is, the frequency above F2, and the lowermost area, that is, less than F1, are frequency bands that are not to be changed.

  In this embodiment, when the code 1 is embedded, the intensity of the spectrum set is changed to “state 1” as shown in FIG. When the code 2 is embedded, the intensity of the spectrum set is changed to “state 2” as shown in FIG. 14A to 14C are in exactly the same state as FIGS. 3A, 3C, and 3D, respectively.

  In this embodiment, when the code 3 is embedded, the intensity of the spectrum set is changed to “state 3” as shown in FIG. When embedding the code 4, the intensity of the spectrum set is changed to “state 4” as shown in FIG. When 1 bit is embedded in one acoustic frame, as shown in FIGS. 3C and 3D, among the four frequency regions in the change target frequency band, the first region from the bottom and the third region from the bottom are the same. Yes, the second area from the bottom and the fourth area from the bottom are changed to be the same. That is, the first and second regions from the bottom and the third and fourth regions from the bottom are each set as one set, and the two sets are changed to be the same. On the other hand, when 2 bits are embedded in one acoustic frame, the first and second areas from the bottom, and the third and fourth areas from the bottom are each set as one set. Is done. This creates a large number of patterns and embeds many bits.

  Even when 2 bits are embedded in one acoustic frame, the embedding process follows the flowchart of FIG. However, in S102, 2-bit reading is performed, and in S104, the state is set to any one of states 1 to 4 as shown in FIGS.

  Even when 2 bits are embedded in one acoustic frame, the extraction process follows the flowcharts of FIGS. However, in S202, any one of the values 1 to 4 is output. In S204, the bit counter is incremented by 2. In S402, the state 1 to the state 4 are determined. In the determination of the state in S402, the following determination process is executed, and as a result, it is determined that the state is a corresponding state, and a corresponding value is output.

For each q = 1, ..., Q,
When C1q + C4q> C2q + C3q, it is determined that the bit value is “1” (or “0”).
Otherwise, it is determined that the bit value is “0” (or “1”).

  In the example of FIG. 14, Q = 2, the bit value is extracted from the lower two regions (q = 1) in the change target frequency band, and the upper two regions ( A bit value is extracted from q = 2), and is output as a value 1 to a value 4 determined in advance according to the combination of these two bits.

  In the example shown in FIG. 3 and FIG. 14, the change target frequency band F1 to F2 is divided into four regions, but can be divided into more regions. Here, F1 to F2 are divided into eight, the components of the frequency band to be changed in the acoustic frame are changed to four states, and 2-bit information is embedded, and each of the A type channel 1 acoustic frames is predetermined. The state of the frequency component is shown in FIG. In this case, Q = 4, and it is possible to input up to 4 bits by changing the state of each set to be different. However, in the example of FIG. When the number of bits embedded in one acoustic frame is G, in the present invention, G can be set to be a divisor of Q.

  As shown in FIG. 15, even when the area is divided into a larger number of areas, the embedding process follows the flowchart of FIG. 4, and the extraction process follows the flowcharts of FIGS. 10 to 13. However, in the state determination in S402, the following determination process is executed, and as a result, it is determined that the state is a corresponding state, and a corresponding value is output.

For each g = 1, ..., G,
When Σq ₌ gG _{+ 1,..., GG + Q / G} (C1q + C4q)> Σq ₌ gG _{+ 1,...,} GG + _{Q / G} (C2q + C3q), the bit value is “1” (or “0”). Judge that there is.
Otherwise, it is determined that the bit value is “0” (or “1”).

(5. Modified example of bit array)
In the above embodiment, the bit array creation means 70 extracts 7 bits from the additional information, adds a 1-bit parity code, and further adds an 8-bit check code. In this case, however, the additional information It takes 16 bits to embed 1 byte. Therefore, in the following, two methods for reducing check codes while maintaining a 1-bit error correction function and an all-bit inversion detection function will be described.

  First, as shown in FIG. 16A, the first technique creates a bit array in which five check codes P1 to P5 are added to 7 bits D1 to D7 extracted from the additional information. . Each check code P1 to P5 is defined as shown in FIG. Among these, the check code P1 is a parity code, and the check codes P2 to P5 are 4-bit Hamming codes. Specifically, the check code P1 is calculated as an exclusive OR of the bits D1 to D7. The check code P2 is calculated as an exclusive OR of the bits D1, D2, D3, and D7. The check code P3 is calculated as an exclusive OR of the bits D1, D4, D5, D7, and P1. The check code P4 is calculated as an exclusive OR of the bits D2, D4, D6, D7, and P1. The check code P5 is calculated as an exclusive OR of the bits D3, D5, D6, and P1. Therefore, the bit array creation means 70 creates a 12-bit bit array as shown in FIG. 16A in S101 according to the definition shown in FIG. 16B.

  In this case, the extraction apparatus determines whether the bit counter is 11 or less or 12 or more in S205 of FIG. If the bit counter is 12 or more, in S206, a check bit value is calculated from the previous 7 bits in the bit array stored in the buffer. Specifically, the calculation is performed according to the definition shown in FIG. As a result, five check codes are obtained. Subsequently, in S207, the calculated check code is compared with the last 5 bits in the bit array.

  As a result, when all of P1 to P5 are acceptable, it is determined that they are completely compatible. Further, when P1 is acceptable and 2 bits of P2 and P5 are unacceptable, it can be determined that the entire 12 bits are inverted, and therefore, it is determined that it is completely nonconforming. In other cases, it is determined whether or not one error bit can be specified in S213 as incompatibility.

  In S213, if P1 is rejected and P2 and P3, P2 and P4, P2 and P5, P3 and P4, P3 and P5, and any two bits of P4 and P5 are rejected, D1, D2, D3, It is specified as a one-bit error of D4, D5, and D6. If P1 is unsuccessful, and any 3 bits of P2, P3, and P4, and P3, P4, and P5 are unsuccessful, they are identified as 1-bit errors of D7 and P1, respectively. If P1 fails and any one of P2 to P5 fails, each is identified as its own 1-bit error.

  As shown in FIG. 17A, the second method is to create a bit array in which four check codes P1 to P4 are added to 7 bits D1 to D7 extracted from the additional information. The check codes P1 to P4 are 4-bit hamming codes defined as shown in FIG. That is, the check code P1 is calculated as an exclusive OR of the bits D1, D2, D3, D5, D6, and D7. The check code P2 is calculated as an exclusive OR of the bits D1, D4, D6, and D7. The check code P3 is calculated as an exclusive OR of the bits D2, D4, D5, and D7. The check code P4 is calculated as an exclusive OR of the bits D3, D4, D5, and D6. Therefore, the bit array creation means 70 creates an 11-bit bit array as shown in FIG. 17A according to the definition shown in FIG. 17B in S101.

  In this case, the extraction apparatus determines whether the bit counter is 10 or less or 11 or more in S205 of FIG. If the bit counter is 11 or more, in S206, a check bit value is calculated from the previous 7 bits in the bit array stored in the buffer. Specifically, the calculation is performed according to the definition shown in FIG. As a result, four check codes are obtained. Subsequently, in S207, the calculated check code is compared with the last 4 bits in the bit array.

  As a result, when all of P1 to P4 are acceptable, it is determined that they are completely compatible. Further, when all of P1 to P4 are unacceptable, it can be determined that the entire 11 bits are inverted, and therefore, it is determined that it is completely incompatible. In other cases, it is determined whether or not one error bit can be specified in S213 as incompatibility.

  In S213, when any two bits of P1 and P2, P1 and P3, and P1 and P4 are unacceptable, they are identified as 1-bit errors of D1, D2, and D3, respectively. When any 3 bits of P2, P3 and P4, P1 and P3 and P4, P1 and P2 and P4, and P1 and P2 and P3 are unacceptable, they are identified as 1-bit errors of D4, D5, D6 and D7, respectively. If any one of P1 to P4 fails, each is identified as its own 1-bit error.

  As for the bit arrangement, three forms have been described as shown in FIGS. 5, 16, and 17. As a result of the experiment, the extraction accuracy is the highest in the case of 12 bits shown in FIG. it was high.

(6. Others)
As mentioned above, although it limited about the suitable embodiment of the present invention, the present invention is not limited to the above-mentioned embodiment, and various modifications are possible. For example, in the above embodiment, the number of samples of one acoustic frame is N = 4096, but N = 2048, 1024, 512, etc. may be set. As a result, the number of sound frames per same time is doubled, quadrupled, and quadrupled, and 2 to 8 times of information can be embedded as a whole.

  In the above-described embodiment, the case where the device for extracting information from the acoustic signal is realized by a single mobile terminal device such as a mobile phone has been described as an example. However, the device may be realized in cooperation with other computers. good. Specifically, the portable terminal device and the dedicated computer are connected so as to be capable of wireless communication, and among the components of the acoustic signal input unit 100 to the acoustic frame holding unit 170, those having a large calculation load are processed by the dedicated computer. For example, the portable terminal device includes the acoustic signal input unit 100, the reference frame acquisition unit 110, the phase change frame setting unit 120, and the additional information extraction unit 160, and the frequency conversion unit 130, the code determination parameter calculation unit 140, and the code output unit 150. The sound frame holding means 170 is provided in a dedicated computer so that necessary information is communicated between the two. This makes it possible to perform high-speed processing even when the processing performance of the portable terminal device is low.

It is a functional block diagram of an information embedding device for an acoustic signal. It is a figure which shows the time direction window function used by this invention. It is a figure which shows the state of a change of the component of the change object frequency band in one Embodiment of this invention. It is a flowchart which shows the process outline | summary of the apparatus shown in FIG. It is a figure which shows the mode of preparation of a bit arrangement | sequence. 1 is a functional block diagram of an apparatus for extracting information from an acoustic signal according to the present invention. It is a figure which shows the relationship between the frequency of an acoustic signal, and energy distribution. It is a figure which shows the frequency direction window function F (j) used in one Embodiment of this invention, and a correction function. It is a figure which shows the detail of the frequency direction window function F (j) used by one Embodiment of this invention. 7 is a flowchart illustrating an outline of processing in a setting in which error correction is not performed in the apparatus illustrated in FIG. 6. 11 is a flowchart showing details of phase determination and code output in S202 of FIG. It is a flowchart which shows the detail of the code | symbol determination process of S302 of FIG. 7 is a flowchart showing an outline of processing in a setting for performing 1-bit error correction in the apparatus shown in FIG. 6. It is a figure which shows the state of a change of the component of the frequency band for change in the case of embedding 2-bit information. It is a figure which shows the state of a change of a component at the time of dividing a change object frequency band into eight. It is a figure which shows the modification of a bit arrangement | sequence. It is a figure which shows the modification of a bit arrangement | sequence.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Acoustic frame reading means 20 ... Frequency conversion means 30 ... Frequency component change means 40 ... Frequency reverse conversion means 50 ... Modified acoustic frame output means 60 ... Storage means 61 ... Acoustic signal storage unit 62 ... additional information storage unit 63 ... modified acoustic signal storage unit 70 ... bit array creation means 100 ... acoustic signal input means 110 ... reference frame acquisition means 120 ... phase Change frame setting means 130 ... frequency conversion means 140 ... code determination parameter calculation means 150 ... code output means 160 ... additional information extraction means 170 ... sound frame holding means

Claims (20)

An apparatus for embedding additional information in an inaudible state with respect to an acoustic signal composed of a stereo time-series sample sequence consisting of two channels of channel L and channel R,
A sound frame reading means for reading a predetermined number of samples as an L sound frame and an R sound frame for the channel L and the channel R from the sound signal;
Of the read sound frames, odd-numbered and even-numbered ones are A type and the other is B type, and the front, middle, and rear in the time direction with respect to the L type sound frame and the R type sound frame of the A type Frequency conversion using the first window function, the second window function, and the third window function that mainly extract the signal components of the B type, and the fourth window function for the B type L acoustic frame and the R acoustic frame. The first window L spectrum, which is a spectrum corresponding to the first window function, the first window R spectrum, the second window L spectrum which is a spectrum corresponding to the second window function, Window R spectrum, third window L spectrum, which is a spectrum corresponding to the third window function, third window R spectrum, fourth window L spectrum, which is a spectrum corresponding to the fourth window function, A frequency converting means for obtaining the window R spectrum,
2Q (Q is an integer greater than or equal to 2) spectrum sets in a predetermined frequency range are extracted from the generated window spectra, and the first window L spectrum, the first window R spectrum, and the third window L spectrum are extracted. The spectrum set extracted from the third window R spectrum is divided into a plurality of groups such that adjacent spectrum sets in the same window spectrum of the same channel belong to different groups, and based on the bit arrangement to be embedded, The second window L spectrum, the second window R spectrum, the fourth window L spectrum, and the fourth window R spectrum are the same regardless of the bit arrangement of the additional information to be embedded. The intensity of adjacent spectral sets in the same window spectrum of the channel is different from each other. A frequency component changing means for changing,
Frequency inverse transform means for performing frequency inverse transform on each window spectrum including the modified spectrum set to generate a modified acoustic frame;
Modified acoustic frame output means for sequentially outputting the generated modified acoustic frames;
An information embedding device for an acoustic signal, comprising: In claim 1,
The frequency component changing means sets the spectrum set extracted from each window spectrum as one set in units of two adjacent regions, and performs the same change to the Q set spectrum set. An information embedding device. In claim 1,
The frequency component changing means makes one set of spectrum sets extracted from each window spectrum in units of two adjacent regions, and makes different changes to the Q set of spectrum sets. An information embedding device. In claim 1,
The frequency component changing means sets one set of spectrum sets extracted from each window spectrum in units of two adjacent areas, and makes different changes to G (G is a divisor of Q) sets of spectrum sets. An information embedding device for an acoustic signal. In claim 1,
The frequency component changing means changes the intensity by removing the component of the spectrum set of one channel and adding a component equivalent to the removed component to the component of the corresponding spectrum set of the other group. An apparatus for embedding information in an acoustic signal. In any one of Claims 1-5,
A bit array creating means for creating the bit array by adding a check code for detecting an error to the bit string in the additional information;
The apparatus for embedding information in an acoustic signal, wherein the frequency component changing means changes the intensity of the two groups to be different from each other based on the bit arrangement. In claim 6,
The apparatus for embedding information in an acoustic signal, wherein the bit array creation means uses an exclusive OR of a bit string in the additional information as the check code. In any one of Claims 1-7,
The acoustic frame reading means reads the A type acoustic frame and the B type acoustic frame by overlapping a predetermined number of samples,
The apparatus for embedding information in an acoustic signal, wherein the modified acoustic frame output means outputs the generated modified acoustic frame by connecting it with a preceding modified acoustic frame. In any one of Claims 1-8,
The frequency component changing means sets the predetermined frequency range as 1.7 kHz or more and 3.4 kHz or less, or sets the predetermined frequency range as 3.4 kHz or more and 6.8 kHz or less. An information embedding device. In any one of Claims 1-9,
The sound frame reading means generates white noise with extremely low amplitude that is difficult to hear during reproduction with the same number of samples as the L sound frame and the R sound frame, and adds the white noise to the L sound frame and the R sound frame. An information embedding device for an acoustic signal, characterized in that: In claim 10,
The sound frame reading means creates 1 / K uniform random numbers of the number of samples of the L sound frame and the R sound frame, and sets the same random value in K sample units for the L sound frame and the R sound frame. An apparatus for embedding information with respect to an acoustic signal, wherein white noise is added by adding the L acoustic frame and the R acoustic frame with the signs reversed. The computer, as an embedded device information for the acoustic signal as claimed in any one of claims 11, a program to function. The additional information embedded in a non listening state by embedding device information for the acoustic signal as claimed in any one of claims 11, desired single-channel monophonic audio signal or these two-channel stereo audio signals there is provided an apparatus to extract,
An acoustic frame acquisition means for digitizing a predetermined section of the acoustic signal and acquiring an acoustic frame composed of a predetermined number of samples;
With respect to the acoustic frame, the front portion of the time direction, the first window function to extract mainly the rear signal components, respectively, using the third window function, performs frequency conversion, corresponding to the first window function Frequency conversion means for obtaining a first window spectrum that is a spectrum, a third window spectrum that is a spectrum corresponding to the third window function;
It extracts the 2Q number of spectral set not overlapping with each other in a predetermined frequency range of the first window spectrum, extracting 2Q number of spectral set that do not overlap with each other in the predetermined frequency range from said third window spectrum, first The spectrum set extracted from the window spectrum and the third window spectrum is divided into two groups such that adjacent spectrum sets in the same window spectrum belong to different groups, and the intensity of each spectral element at the same frequency between the groups is determined. An encoding means for extracting the embedded bit value based on the magnitude;
Additional information extracting means for converting the extracted bit value in units of words according to a predetermined rule and extracting additional information;
An apparatus for extracting information from an acoustic signal, comprising: In claim 13,
The encoding means sets one set of spectrum sets extracted from each window spectrum in units of two adjacent regions, and sets a 1-bit bit value based on the magnitude of the intensity of the spectrum elements in all spectrum sets of the Q set. An apparatus for extracting information from an acoustic signal, characterized in that In claim 13,
The encoding means sets one set of spectrum sets extracted from each window spectrum in units of two adjacent regions, and sets one bit based on the magnitude of the intensity of the spectrum elements in all spectrum sets for each set of Q sets. An apparatus for extracting information from an acoustic signal, which extracts Q bits as a whole by extracting the bit value of. In claim 13,
The encoding means sets one set of spectrum sets extracted from each window spectrum in units of two adjacent regions, and sets the spectral elements in all spectrum sets for each set of G (G is a divisor of Q) set. An apparatus for extracting information from an acoustic signal, wherein G bits are extracted as a whole by extracting a 1-bit bit value based on the magnitude of intensity. In any one of Claims 13-16,
The additional information extraction means uses a predetermined number of bits from the back of the bit array as a check code when the set of extracted bit values becomes a bit array of a predetermined number of bits, and the check code is in a predetermined state In this case, a predetermined number of bits are inverted from the front of the bit string and output as a bit string constituting the additional information to extract the additional information. Extraction device. In claim 17,
The acoustic signal is characterized in that the additional information extraction means uses an exclusive OR of a predetermined number of bits from the front of the bit string when determining whether or not the check code is in a predetermined state. Information extraction device. In any one of Claims 13-18,
The acoustic frame acquisition means changes a phase by moving a reference frame and a predetermined sample from a reference frame acquisition means for acquiring an acoustic frame composed of a predetermined number of samples as a reference frame from the acoustic signal. It is constituted by phase change frame setting means for setting a plurality of set sound frames as phase change frames,
The encoding means is based on the code determination parameter calculation means for calculating a code determination parameter based on the extracted spectrum set, and on the code determination parameter calculated in a past in-phase acoustic frame having a different reference frame, It is determined that one of the reference frame and the plurality of phase change frames has an optimal phase, and a predetermined code is determined based on the code determination parameter determined for the acoustic frame having the optimal phase. An apparatus for extracting information from an acoustic signal, characterized by comprising code output means for outputting.   A program for causing a computer to function as an apparatus for extracting information from an acoustic signal according to any one of claims 13 to 19.

Images