JP5310498B2 - Information embedding device for acoustic signals - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an embedding device for information into a sound signal, capable of extracting embedded information with high accuracy, even when a signal component in the previous section overlaps in each section, in embedding additional information by a predetermined section unit of the sound signal. <P>SOLUTION: When the additional information is embedded into the sound signal, an odd numbered sound frame and an even numbered sound frame are obtained from the predetermined section of the sound signal. According to embedded information, two sound frames are changed so that either component strength of a high frequency side or a low frequency side may become a predetermined ratio or more than the other. After changing (a), a phase of the frequency component which is changed to be smaller in an object frame is corrected toward an opposite direction to a corresponding frequency component of the previous frame (b). <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a technique for embedding additional information such as attribute information in an acoustic signal, and in particular, it is possible to extract embedded information with high accuracy by preventing the influence of reverberation due to sound emitted from a speaker installed in a room. It relates to technology.

  Conventionally, as a service for providing music attribute information that allows you to know the title of music that is playing, you can query the broadcast station for the date and time of the broadcasted music, and record music fragments that are played on mobile phones. In addition, there is a service that matches the melody registered in the database. The present applicant has further developed this and proposed a method of embedding attribute information (additional information) by changing the ratio of frequency components of an acoustic signal according to the bit value of the attribute information (Patent Document 1). reference).

In addition, in the invention described in Patent Document 1, the present applicant has a problem that when two speakers are separated by a wide hall or the like, it cannot be extracted near the center where the sound from both channels is mixed. In order to solve this problem, a technique for embedding and extracting information using the principle of sound pulse fractionation has been proposed (see Patent Document 2). The applicant has also proposed a technology for embedding and extracting information by setting double frequency bands in which information is embedded and using double sound wave condensation that causes sound wave condensation in each frequency band. (See Patent Document 3).
JP 2006-323246 A JP 2008-256948 A JP 2009-75332 A

  In any of Patent Document 1 to Patent Document 3, the additional information is embedded in units of an acoustic frame that is a predetermined section of the acoustic signal. In this acoustic frame, the frequency component is changed in order to embed information. However, the larger the frequency band that can be changed in the acoustic frame, the higher the embedding efficiency and the higher the extraction accuracy. Become. Furthermore, in the inventions described in Patent Documents 2 and 3, even if information is embedded in a frequency band where human auditory sensitivity is high, it is possible to obtain a sound that does not cause humans to feel noise by the principle of segregation. It is.

  However, even if the inventions described in Patent Documents 2 and 3 are used, in the case where a sound emitted from a speaker is acquired by a microphone and digital watermark extraction is performed, the problem of room reverberation cannot be avoided. As a result, the propagation distance is limited. If the distance between the speaker and the microphone is more than a certain distance, the proportion of the reflected sound that passes through the floor, ceiling, and wall becomes larger than the direct sound from the speaker, which has the positive effect that the volume can be heard by human beings, In acoustic signal analysis, a phase-delayed sound wave is recorded at the same time, which causes a negative effect that high-precision analysis cannot be performed. In particular, in the case of digital watermark embedding described in Patent Documents 2 and 3, there is a problem in that the previous frame components are overlapped and recorded, and the volume pattern of the signal components is broken, leading to an extraction bit error. Therefore, there is a problem that even if it can be heard by humans, it cannot be stably extracted unless the terminal is brought close to the speaker. On the other hand, in Patent Document 3, bit determination is performed by subtracting the immediately preceding frame component at a constant rate in order to reduce the reverberation component during watermark extraction. However, it has been found that the method of correcting only the influence of the immediately preceding frame in Patent Document 3 cannot sufficiently reduce the influence of multiple reflected waves.

  Therefore, according to the present invention, when embedding additional information in units of a predetermined section of an acoustic signal, even if the signal component in the previous section overlaps each section, the embedded information can be extracted with high accuracy. It is an object to provide an information embedding device for a signal.

  In order to solve the above-described problem, according to the first aspect of the present invention, an apparatus for embedding additional information in an inaudible state with respect to an acoustic signal composed of a time-series sample sequence, Acoustic frame reading means for reading a number of samples, and among the read acoustic frames, odd-numbered and even-numbered one is A type and the other is B type, and the first window function is applied to the A type acoustic frame. A first window spectrum, which is a complex frequency component corresponding to the first window function, and the second window. Frequency conversion means for obtaining a second window spectrum, which is a complex frequency component corresponding to the function, and two scans that do not overlap each other in a predetermined frequency range from the generated window spectra. Based on the bit sequence to be embedded, the spectrum set intensities (the magnitudes of complex vectors that are complex frequency components) extracted from the first window spectrum and the second window spectrum are extracted. Frequency component changing means for changing the intensity of each spectrum set so that the intensity of the spectrum set becomes larger on the basis of the intensity of the other two spectrum sets, The direction of the complex frequency component of each frequency of the changed spectrum set (the direction in the complex two-dimensional space, that is, the phase) is corrected to the direction opposite to the direction of the complex frequency component of each frequency of the spectrum set in the immediately preceding acoustic frame. A phase correcting means, and performing frequency inverse transform on each window spectrum including the phase-corrected spectrum set. Providing a frequency inverse conversion means for generating a modified acoustic frames, the embedded device information for the acoustic signal having the altered acoustic frame output means for sequentially outputting the generated modified acoustic frame.

  The second aspect of the present invention is a device for embedding additional information in an inaudible state with respect to an acoustic signal composed of a time-series sample sequence, and reads a predetermined number of samples from the acoustic signal. An acoustic frame reading means, and among the read acoustic frames, odd-numbered and even-numbered one is A type and the other is B type, and frequency conversion is performed on the A type acoustic frame using a first window function. And performing frequency conversion on the B-type acoustic frame using a second window function, a first window spectrum, which is a complex frequency component corresponding to the first window function, and a complex corresponding to the second window function. Frequency conversion means for obtaining a second window spectrum, which is a frequency component, and four spectrum sets that do not overlap each other in a predetermined frequency range from each of the generated window spectra. Extraction is performed so that the frequency width of the two spectrum sets having the higher number is larger than the frequency width of the two spectrum sets having the lower frequency, and four spectrum sets for each window spectrum are extracted from the first window spectrum. 1D1, 1D2, 1U1, and 1U2 are set in the order from the lowest frequency, and the spectrum set extracted from the second window spectrum is set to 2D1, 2D2, 2U1, and 2U in the order from the lowest frequency. Based on the bit arrangement, one of the product of the intensity value of 1D1 and the intensity value of 2D2 and the product of the intensity value of 1D2 and the intensity value of 2D1 is larger than the other by a predetermined ratio, and at the same time, 1U1 Either the product of the intensity value and the intensity value of 2U2 or the product of the intensity value of 1U2 and the intensity value of 2U1 is more than a predetermined ratio from the other. The frequency component changing means for changing the intensity of each spectrum set so as to be determined, and the direction of the complex frequency component of each frequency of the spectrum set that has been changed so as to become smaller in the spectrum set, in the immediately preceding acoustic frame Phase correction means for correcting the spectrum frequency in the direction opposite to the direction of the complex frequency component of each frequency of the spectrum set, and performing a frequency inverse transform on each window spectrum including the phase corrected spectrum set to generate a modified acoustic frame There is provided a device for embedding information with respect to an acoustic signal, comprising frequency inverse transforming means for performing, and modified acoustic frame output means for sequentially outputting the generated modified acoustic frames.

  According to the present invention, when embedding additional information, after changing the intensity state of the complex frequency component according to the information to be embedded, the phase of the complex frequency component whose intensity is weakened is opposite to the direction of the complex frequency component in the immediately preceding section. Since the correction is made in the direction of the side, it is possible to prevent the complex frequency component intensity from greatly changing due to the reverberation of the complex frequency component in the immediately preceding section, and it is possible to improve the information extraction accuracy on the extraction side It becomes.

It is a functional block diagram of an information embedding device for an acoustic signal. It is a figure which shows the time direction window function used by this invention. It is a figure which shows the state of a change of the component of the change object frequency band in one Embodiment of this invention. It is a flowchart which shows the process outline | summary of code conversion table preparation. It is a figure which shows an example of the code conversion table of an 8-bit code and a 16-bit code. It is a flowchart which shows the process outline | summary of the apparatus shown in FIG. It is a figure for demonstrating the embedding process which does not use amplitude conversion. It is a figure for demonstrating the extraction process which does not use amplitude conversion. It is a figure for demonstrating the embedding process using amplitude conversion. It is a figure for demonstrating the extraction process using amplitude conversion. It is a functional block diagram of the extraction apparatus of the information from an acoustic signal. It is a figure which shows an example of the code conversion table of an 8-bit code and a 16-bit code. 12 is a flowchart illustrating an outline of processing in a setting in which error correction is not performed in the apparatus illustrated in FIG. 11. It is a flowchart which shows the detail of the time-axis shift confirmation and code | symbol output of S202 of FIG. It is a flowchart which shows the detail of the code | symbol determination process of S302 of FIG. It is explanatory drawing of the principle of the sound pulse fractionation which is a human auditory psychological characteristic. It is a figure which shows the mode of the intensity | strength change of each acoustic frame at the time of embedding of patent document 2. FIG. It is a figure which shows the mode of the intensity | strength change of each sound frame at the time of embedding in patent document 3. FIG. It is a conceptual diagram which shows the basic principle (simple model) of reverberation tolerance realization by phase correction. It is explanatory drawing of the basis of the dereverberation in the simple model shown in FIG. It is a conceptual diagram which shows the basic principle (practical model) of realization of reverberation tolerance by phase correction. It is explanatory drawing of the basis of the reverberation removal in the practical model shown in FIG. It is a figure which shows the energy distribution with respect to a frequency, and the embedding area | region of patent document 2 and patent document 3. FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(1. Basic concept of the present invention)
First, the basic concept of the present invention will be described. In the present invention, as in Patent Document 2 and Patent Document 3, the principle of sound pulse segregation, which is a human psychoacoustic characteristic, is used. The syllable segregation is an illusion that humans interpolate and hear sounds as if two tracks of high and low are flowing continuously against a pattern in which high and low sounds alternate in time series. It is.

  For example, as shown in FIG. 16 (a), three low sounds, low tone 1, low sound 3, low sound 5, and three high sounds, high sound 2, high sound 4, and high sound 6, are low sound 1, high sound 2, and low sound. Consider a case in which 3, 3 high sounds, 5 low sounds, and 6 high sounds are played in this order. Low pitch 1, low pitch 3, low pitch 5 and high pitch 2, high pitch 4, high pitch 6 are about 1 octave apart, and the low and high sounds are not played at the same time, but the time intervals are almost continuous. Shall. In this case, as shown in FIG. 16 (b), a human sounds as if the portion where the low and high sounds are not played is interpolated and played. That is, although the actual performance is a single melody, as shown in FIG. 16 (b), humans can obtain a high tone 1 ′, a high tone 3 ′, a high tone 5 ′, a low tone 2 ′, a low tone 4 ′, and a low tone 6 ′. Sounds like it is interpolated. For example, the high sound 3 ′ is heard as an average component of the high sound 2 and the high sound 4 so that the high sound 2 and the high sound 4 are continuously connected. Further, the high pitch 1 'and the low pitch 6' at the end also sound like components corresponding to the low tone 1 and the high tone 6 with components close to the adjacent high tone 2 and low tone 5, respectively. However, an electroacoustic apparatus such as a microphone acquires the sound shown in FIG. 16A as it is. The present invention and the invention of Patent Document 2 utilize such properties. Note that the interpolated sound is not necessarily interpolated at the same level as the sound played back and forth, but roughly speaking, about 50% of the sound played back and forth is interpolated. Sounds like

  In both the present invention and the inventions of Patent Document 2 and Patent Document 3, processing is performed using a predetermined number of samples obtained by sampling an acoustic signal as one acoustic frame. The odd-numbered and even-numbered sound frames from the top are set in such a manner that half samples are overlapped with each other. The difference between the invention of Patent Document 2 and the invention of Patent Document 3 is the number of divisions of the frequency band to be changed used for embedding. In the invention of Patent Document 2, it is divided into two frequency regions, and in Patent Document 3, it is divided into four frequency regions. It is divided. In either case, information is embedded by changing the relationship between the intensities of complex frequency components in the two frequency regions. In the invention of Patent Document 2, 1 bit can be embedded in the change target frequency band. In this invention, the change target frequency band is divided into an upper frequency band and a lower frequency band, and two bits can be embedded by embedding different information in the upper frequency band and the lower frequency band.

  Here, FIG. 17 shows a state of intensity change of each acoustic frame at the time of embedding when the change target frequency band is composed of two frequency regions. In the case of a stereo sound signal, processing is performed in the same way for both L-ch (left channel) and R-ch (right channel), but only L-ch (left channel) is shown in FIG. U and d represent relatively high and low frequency components, respectively, and 1 to 6 represent the order of the acoustic frames from the beginning. Therefore, in the example of FIG. 17, six acoustic frames are shown, and for example, “Uu1” indicates the high frequency side component of the first acoustic frame. In addition, although the odd-numbered and even-numbered sound frames are actually set by overlapping half of the samples, they are shown in an independent form for convenience of explanation in FIG. In FIG. 17, the size of each frequency component such as “Uu1” indicates the relative strength.

  In FIG. 17, FIG. 17A shows the frequency band to be changed of the original sound signal, FIG. 17B shows the frequency band to be changed after the conventional embedding process, and FIG. 17C shows the embedding process of the present invention. The frequency band to be changed later is shown. In the invention of Patent Document 2, the magnitude relationship between the complex frequency components on the high-frequency side and the low-frequency side of the odd-numbered acoustic frame and the magnitude of the complex frequency components on the high-frequency side and the low-frequency side of the even-numbered acoustic frame. Processing is performed so that the relationship is reversed. The bit value to be embedded can be changed depending on whether the high-frequency side of the odd-numbered or even-numbered acoustic frame is increased. In the example of FIG. 17, “0” is embedded by increasing the high frequency side of the even-numbered acoustic frame, and “1” is embedded by increasing the high-frequency side of the odd-numbered acoustic frame. Therefore, when the 3-bit information “010” is embedded in the original acoustic signal, the state of the acoustic frame changes as shown in FIG.

  In the invention of Patent Document 2, since the intensity on the high frequency side and the intensity on the low frequency side are reversed in the odd-numbered and even-numbered acoustic frames, as shown in FIG. There is always a strong signal strength on both the high and low frequency sides. For this reason, the sound does not sound as if the sound has been interrupted by the principle of sound-band division, but the extraction device can recognize the clear change.

  In the present invention, in order to increase the reverberation tolerance, the phase correction is further performed on the complex frequency component in which the intensity of the complex frequency component is relatively reduced. The reason why the complex frequency component is applied to the complex frequency component whose intensity is relatively reduced is that if the complex frequency component is applied to the complex frequency component whose intensity is relatively increased, quality deterioration will occur. . Therefore, as illustrated in FIG. 17C, the correction target is “Ud2”, “Uu4”, and “Ud6”. As will be described in detail later, the phase correction is performed so as to approach the opposite phase direction of the immediately preceding acoustic frame.

  With respect to the invention of Patent Document 2 as shown in FIGS. 17 (a) and 17 (b), in response to a desire to ensure a large frequency region for embedding additional information, by ensuring a large frequency region, the extraction sensitivity and accuracy are increased. The invention of Patent Document 3 has improved the amount of information that can be embedded in one acoustic frame. There are two methods for widening the frequency region: a method of widening to the high sound side and a method of widening to the low sound side, but the former method is easy to realize. This is because the signal intensity is low on the high sound side and the human auditory sensitivity is lowered, so that the reproduction quality is not greatly affected even if the frequency region is expanded. However, in order to make it possible to implement digital watermark extraction by a mobile phone, in the invention of Patent Document 3, it is decided to extend to the low sound side up to 0.3 kHz which is a telephone line band. In this case, the signal intensity in the bass band is strong and the auditory sensitivity is relatively high, so if you simply expand the frequency range to be embedded to the bass side and change that information, the principle of sound pulse condensation works well. Without sounding uncomfortable to the human ear.

  Therefore, in the invention of Patent Document 3, the frequency region is not simply expanded, but a region independent of the conventional embedding region is provided in the low frequency region of 1.7 kHz or less, and the sound pulse is doubled in the two high and low bands. Made the principle of separation work. In addition, since the two high and low bands function the sound pulse separation function independently, both frequency widths do not necessarily have to be the same (however, each band is further divided into two in the frequency direction, but divided into two). Since the lower limit of the telephone line band is 0.3 kHz according to the standard, the bandwidth on the low sound side must be set slightly narrower than that on the high sound side. Therefore, in the invention of Patent Document 3, as a preferable example, the high sound side is in the range of 1.7 kHz to 3.4 kHz, and the low sound side is in the range of 0.34 kHz to 1.7 kHz, as in the invention of Patent Document 2. did.

  Here, FIG. 18 shows a state of intensity change of each acoustic frame at the time of embedding when the change target frequency band is composed of four frequency regions. In this case as well, in the case of a stereo sound signal, the same processing is performed for both L-ch (left channel) and R-ch (right channel), but only L-ch (left channel) is shown in FIG. ing. Also, u and d are relatively high frequency and low frequency components, respectively, and 1 to 6 are the same as in FIG. In the present invention, the complex frequency component changing process is performed even in a frequency range lower than the conventional frequency range as shown in FIG. In FIG. 18, “Uu1” to “Uu6” and “Ud1” to “Ud6” are the same complex frequency components as shown in FIG. 17, but “Du1” to “Du6”, “Dd1” to “Dd6”. "Is a complex frequency component newly used for embedding information according to the present invention. In FIG. 18 as well, the size of characters of each complex frequency component such as “Uu1” indicates the relative strength.

  18, FIG. 18 (a) shows the frequency band to be changed of the original sound signal, FIG. 18 (b) shows the frequency band to be changed after the conventional embedding process, and FIG. 18 (c) shows the embedding process of the present invention. The frequency band to be changed later is shown. In the invention of Patent Document 3, in each of the upper frequency band and the lower frequency band, the magnitude relationship between the high-frequency side and low-frequency side component intensities of the odd-numbered acoustic frames, and the high-frequency side and low-frequency side of the even-numbered acoustic frames. Processing is performed so that the magnitude relationship between the component intensities is reversed. The bit value to be embedded can be changed depending on whether the high-frequency side of the odd-numbered or even-numbered acoustic frame is increased. In the example of FIG. 18, in both the upper frequency band and the lower frequency band, “0” is set by increasing the high frequency side of the even-numbered sound frame, and “1” is set by increasing the high-frequency side of the odd-numbered sound frame. Indicates embedding. Therefore, when the 3-bit information “010” is embedded in the original acoustic signal, the state of the acoustic frame changes as shown in FIG.

  In the present invention, in order to increase the reverberation tolerance, the phase correction is further performed on the complex frequency component in which the intensity of the complex frequency component is relatively reduced. Therefore, as shown in FIG. 18C, the correction target is “Ud2” “Uu4” “Ud6” “Dd2” “Du4” “Dd6”. Similarly to the case of FIG. 17, the phase correction is performed so as to approach the opposite phase direction of the immediately preceding acoustic frame.

  The reason why the phase correction is performed to increase the reverberation tolerance will be described. FIG. 19 is a conceptual diagram showing the basic principle of realizing reverberation tolerance by phase correction. In FIG. 19, complex frequency components in the change target frequency band of the target frame (acoustic frame to be changed) and the immediately preceding frame (acoustic frame immediately before the change target acoustic frame) are complex vectors with the real part and the imaginary part as axes. expressing.

  FIG. 19A shows a state after changing the complex frequency component (the states of FIG. 17B and FIG. 18B). FIG. 19B shows a complex vector representation of the target frame component after phase correction in addition to the immediately preceding frame component and the target frame component shown in FIG. In the example of FIG. 19, the amplitude of the previous frame component (the norm of the complex vector) is larger than the amplitude of the target frame component, so the phase of the target frame component is corrected in the direction opposite to the previous frame component. That is, the phase of the target frame component is corrected in a direction 180 degrees opposite to the previous frame component.

  The principle that reverberation can be removed by phase correction as shown in FIG. 19B will be described with reference to FIG. 20 (a) and 20 (b) show the states after the frequency components are changed and after the phase correction, as in FIGS. 19 (a) and 19 (b). Further, the immediately preceding frame component, the target frame component, and the target frame component after phase correction shown in FIG. 20 are the same as those in FIG. In FIG. 20, in addition to the frame component shown in FIG. 19, the target frame component when reverberation occurs is shown.

  As shown in FIG. 20A, when the immediately preceding frame component is superimposed on the target frame component due to reverberation, the amplitude of the target frame component increases, and the embedded bit value is likely to be inverted and extracted. Become. When the phase of the target frame component is corrected in the direction opposite to the previous frame component, as shown in FIG. 20B, even if the previous frame component is superimposed on the target frame component due to reverberation, the amplitude of the target frame component is There is almost no change, and the possibility that the embedded bit value is inverted and extracted is reduced. That is, by performing phase correction, it is possible to prevent erroneous detection of embedded information due to reverberation.

  19 and 20 are simple models that take into account the reverberation from the immediately preceding frame, but in reality, they may be affected by acoustic frames before the immediately preceding frame. Therefore, in the following, a practical model that takes into account reverberation from the two immediately preceding frames, which are the two immediately preceding the acoustic frame to be changed, and the two immediately preceding frames is described.

  FIG. 21 is a conceptual diagram showing a basic principle practical model for realizing reverberation tolerance by phase correction. In FIG. 21, in addition to the target frame component, the previous frame component, and the target frame component after phase correction shown in FIG. 19, two previous frame components are also shown. FIG. 21A shows the state after the frequency component is changed. FIG. 21B is a vector representation of the target frame component after phase correction in addition to the two previous frame component, the previous frame component, and the target frame component. Also in the example of FIG. 21, since the amplitude of the immediately preceding frame component (the norm of the complex vector) is larger than the amplitude of the subject frame component, the phase of the subject frame component is corrected in the direction opposite to the previous frame component. At this time, unlike the case of the simple model, the phase of the target frame component is corrected in the direction of the opposite phase of the immediately preceding frame component obtained by combining the immediately preceding frame component and the immediately preceding frame component instead of the immediately preceding frame component. That is, the phase of the target frame component is corrected in a direction 180 degrees opposite to the frame component immediately before the synthesis. Therefore, the phase of the target frame component is corrected slightly before the opposite phase of the immediately preceding frame component.

  The principle that reverberation can be removed by phase correction as shown in FIG. 21B will be described with reference to FIG. FIGS. 22 (a) and 22 (b) show the states after the frequency components are changed and after the phase correction, as in FIGS. 21 (a) and 21 (b). Also, the immediately preceding frame component, the immediately preceding frame component, the target frame component, and the target frame component after phase correction shown in FIG. 22 are the same as those in FIG. FIG. 22 shows a target frame component when reverberation occurs in addition to the frame components shown in FIG.

  As shown in FIG. 22A, when the frame component immediately before synthesis is superimposed on the target frame component due to reverberation, the amplitude of the target frame component increases, and the embedded bit value may be inverted and extracted. Get higher. When the phase of the target frame component is corrected in the opposite phase direction to the frame component immediately before synthesis, as shown in FIG. 22B, even if the frame component immediately before synthesis is superimposed on the target frame component due to reverberation, The amplitude hardly changes, and the possibility that the embedded bit value is inverted and extracted becomes low. That is, by performing phase correction, it is possible to prevent erroneous detection of embedded information due to reverberation.

  FIG. 23 shows the energy distribution with respect to the frequency and the embedding area when the change target frequency band is one stage (example in FIG. 17: two frequency areas) and two stages (example in FIG. 18: four frequency areas). Indicates. As shown in FIG. 23A, the energy distribution of the acoustic signal increases as the frequency decreases. As shown in FIG. 23B, when the change target frequency band is two stages, the frequency range lower than the embedding area in the case where the change target frequency band is one stage is also used as the embedding area. ing. When the change target frequency band has two stages, the upper frequency band (1.7 kHz to 3.4 kHz) and the lower frequency band (0.34 kHz to 1.7 kHz) have a slightly wider range in the upper frequency band (FIG. 23). As shown in (a), since the energy distribution in the lower order is larger, the energy required for embedding is rather larger in the lower order.

(2.1. Configuration of embedded device)
Next, an information embedding device for an acoustic signal according to the present invention will be described. FIG. 1 is a functional block diagram showing the configuration of an information embedding device for an acoustic signal according to the present invention. In FIG. 1, 10 is an acoustic frame reading means, 20 is a frequency converting means, 30 is a frequency component changing means, 40 is a frequency inverse converting means, 50 is a modified acoustic frame output means, 60 is a storage means, and 61 is an acoustic signal storage section. , 62 is an additional information storage unit, 63 is a modified acoustic signal storage unit, 70 is a bit array creation unit, and 80 is a conversion table creation unit. The apparatus shown in FIG. 1 can deal with both a stereo sound signal and a monaural sound signal. Here, a case where processing is performed on a stereo sound signal will be described.

  The sound frame reading means 10 has a function of reading a predetermined number of samples as one frame from each channel of the original stereo sound signal to be embedded with additional information. The frequency conversion means 20 has a function of generating a complex frequency component by frequency-converting a frame represented by a real component of the acoustic signal read by the acoustic frame reading means 10 by Fourier transformation or the like. The frequency component changing unit 30 extracts a plurality of spectrum sets corresponding to a predetermined frequency range from the generated complex frequency component, and converts the spectrum set into the bit array created by the bit array creating unit 70 from the additional information extracted from the additional information storage unit 62. Based on this, it has the function of changing the state of the spectrum set. The phase correction unit 31 converts the phase of the complex frequency component of the spectrum set that has been changed so as to reduce the intensity by the frequency component change unit 30 to the complex frequency of the spectrum set in the corresponding frequency range in the acoustic frame that was read immediately before. It has a function of correcting in the direction opposite to the component direction. At this time, the intensity of the phase component of the complex fractional component (the norm of the complex vector) is not changed. The frequency inverse transform means 40 has a function of generating a modified acoustic frame by performing frequency inverse transform on a plurality of complex frequency components including the changed spectrum set. The modified sound frame output means 50 has a function of sequentially outputting the generated modified sound frames.

  The storage means 60 includes an acoustic signal storage unit 61 that stores a stereo acoustic signal to be embedded as additional information, an additional information storage unit 62 that is configured as a bit string and stores additional information embedded in the stereo acoustic signal, and additional information. It has a modified acoustic signal storage unit 63 for storing the modified acoustic signal after embedding, and stores various information necessary for other processing. The bit array creating unit 70 has a function of extracting additional information from the additional information storage unit 62 and referring to the conversion table creating unit 80 for each word of the additional information to create a corresponding bit array. The conversion table creation means 80 includes Nh (> Nw) that has a Hamming distance of at least 3 or more, including a bit pattern that circulates bit data, for all 2 Nw powers of registration rank that Nw bits can take. By assigning a cyclic hamming code of bits, it has a function of creating a code conversion table in which the registration order of Nw bits and the cyclic hamming code of Nh bits are associated with each other. Here, the registration order indicates a value when Nw bits are expressed in decimal. When Nw = 7, the registration order is 0 to 127 (actually, registration is possible from 0 to 131). Expressed.

  The additional information is information that should be added to the sound information and embedded, and includes attribute information such as a title and artist name, and other information other than the attribute information. In the present embodiment, Nw and Nh are set to Nw = 7 and Nh = 16, respectively. In the present embodiment, since the ASCII code is adopted as the code format of the additional information, Nw = 7 in the additional information, and 7 bits are one word. However, in practice, code expression slightly exceeding the 7-bit range is possible, so 8 bits are defined as 1 word. The bit array created by the bit array creating means 70 is 16 bits, and after the bit array is created, this is processed as one word. Each component shown in FIG. 1 is actually realized by installing a dedicated program in hardware such as a computer and its peripheral devices. That is, the computer executes the contents of each means according to a dedicated program.

(2.2. Processing operation of embedded device)
Next, the processing operation of the information embedding device for the acoustic signal shown in FIG. 1 will be described. As described above, in the present invention, as shown in FIG. 17, the change target frequency band is set to one stage and a single sound pulse coagulation (hereinafter referred to as “single sound wave coagulation”) is generated. As shown in FIG. 18, the change target frequency band is divided into an upper frequency band and a lower frequency band, and a double sound pulse coagulation (hereinafter referred to as “double sound wave coagulation”) is generated. It can also be applied to cases. In the following, explanation will be given by taking “double sound volume fractionation” as an example. The sound frame reading means 10 reads a predetermined number N of samples as one sound frame from each of the left and right channels of the stereo sound signal stored in the sound signal storage unit 61. The number N of samples of one acoustic frame read by the acoustic frame reading means 10 can be set as appropriate, but is desirably about 4096 samples when the sampling frequency is 44.1 kHz. Therefore, the acoustic frame reading means 10 sequentially reads 4096 samples for each of the left channel and the right channel as acoustic frames.

  There are A type and B type as acoustic frames. In the A-type acoustic frame and the B-type acoustic frame, the next sample after the last sample of the preceding acoustic frame of the same type is set as the first sample. The A-type and B-type sound frames are set by overlapping a predetermined number (2048 in this embodiment) of samples. For example, if the A type acoustic frame is A1, A2, A3... From the top and the B type acoustic frame is B1, B2, B3... From the top, A1 is samples 1 to 4096, A2 is samples 4097 to 8192, A3. Is samples 8193-12288, B1 is samples 2049-6144, B2 is samples 6145-10240, and B3 is samples 10241-14336. Since the A type and the B type are relative, either one may be first. In other words, A1 is samples 2049 to 6144, A2 is samples 6145 to 10240, A3 is samples 10241 to 14336, B1 is samples 1 to 4096, B2 is samples 4097 to 8192, and B3 is samples 8193 to 12288. May be.

  The frequency conversion means 20 performs frequency conversion on the real-dimensional acoustic frame after amplitude conversion, and obtains a complex frequency component of the acoustic frame. Specifically, frequency conversion is performed using a window function. As the frequency transformation, various known methods such as Fourier transformation, wavelet transformation, and the like that can cope with transformation to a complex number dimension can be used. In the present embodiment, a case where Fourier transform is used will be described as an example.

  In general, when Fourier transform is performed on a predetermined signal, it is necessary to divide the signal into predetermined lengths. In this case, if Fourier transform is performed on a signal of a predetermined length as it is, a pseudo-harmonic wave is generated. Ingredients are generated. Therefore, in general, when performing Fourier transform, a signal value is changed using a window function called a Hanning window, and then Fourier transform is performed on the changed value.

  In this embodiment, the window function is used, but the window function to be used is divided into the A type acoustic frame and the B type acoustic frame. In the present embodiment, the first window function W (1, i) and the second window function W (2, i) as shown in FIGS. 2A and 2B are prepared so that the extraction side can easily recognize them. I made it. The first window function W (1, i) is for use with an A type acoustic frame, and has a maximum value of 1 at the position of a predetermined sample number i as shown in FIG. In the rear part, the minimum value is set to 0. Which sample number takes the maximum value depends on the design of the window function W (1, i), but in this embodiment, it is defined by [Equation 1] described later. The Fourier transform for the A type acoustic frame is performed on the product of the window function W (1, i).

  The second window function W (2, i) is for use with a B-type acoustic frame, and has a maximum value at the position of a predetermined sample number i as shown in FIG. 1 is set, and the front portion is set to have a minimum value of 0. Which sample number takes the maximum value depends on the design of the window function W (2, i), but in this embodiment, it is defined by [Expression 2] described later. The Fourier transform for the B type acoustic frame is performed on the product of the window function W (2, i).

  As described above, in the present embodiment, the sound frame is read in duplicate. That is, a predetermined number of samples are redundantly read in the odd-numbered sound frames and the even-numbered sound frames. As described above, the window function used is different between the odd frame and the even frame, but since either the odd frame or the even frame is simply the difference between the odd frame and the even frame, either process may be performed on either. . Therefore, in this specification, one of the odd-numbered frame and the even-numbered frame is referred to as an A-type frame, and the other is referred to as a B-type frame. In the present embodiment, an odd frame is described as an A type frame and an even frame is described as a B type frame. Conversely, an even frame may be an A type frame and an odd frame may be a B type frame.

  In the present embodiment, the window functions W (1, i) and W (2, i) are defined by the following [Equation 1] and [Equation 2]. In FIG. 2, the horizontal axis is the time axis (i). As described later, i is a serial number assigned to N samples in each acoustic frame, and is proportional to time t. 2A and 2B, the vertical axis indicates the amplitude value (level) of the signal. 2A and 2B, the vertical axis indicates the values of the window functions W (1, i) and W (2, i), and the maximum values of W (1, i) and W (2, i). Are all 1.

[Formula 1]
When i ≦ N / 8, W (1, i) = 0.0
When N / 8 <i ≦ 3N / 8, W (1, i) = 0.5−0.5 cos (4π (i−N / 8) / N)
When 3N / 8 <i ≦ 11N / 16, W (1, i) = 1.0
When 11N / 16 <i ≦ 13N / 16, W (1, i) = 0.5 + 0.5 cos (8π (i-11N / 16) / N)
When i> 13N / 16, W (1, i) = 0.0

[Formula 2]
When i ≦ 3N / 16, W (2, i) = 0.0
When 3N / 16 <i ≦ 5N / 16, W (2, i) = 0.5−0.5 cos (8π (i−3N / 16) / N)
When 5N / 16 <i ≦ 5N / 8, W (2, i) = 1.0
When 5N / 8 <i ≦ 7N / 8, W (2, i) = 0.5 + 0.5 cos (4π (i−5N / 8) / N)
When i> 7N / 8, W (2, i) = 0.0

  As is clear from FIG. 2 and [Formula 1] and [Formula 2], the window functions W (1, i) and W (2, i) are asymmetrical to each other. This is for facilitating identification between the two on the extraction side described later.

  In the present invention, since odd frames and even frames are redundantly read by a predetermined number of samples, after embedding information and then restoring to an acoustic signal, the odd frame multiplied by the window function and the window function are multiplied. When overlapping samples of even frames are added, it is necessary to return almost to the original value. For this reason, when the window functions W (1, i) and W (2, i) are added in the overlapping portion of the odd frame and the even frame, it is defined to be a fixed value 1 for all sections.

  When the frequency conversion means 20 performs Fourier transform on the A type sound frame, the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1). The window function W (1, i) is used to perform processing according to the following [Equation 3], and real part Al (1, j) and imaginary part Bl (1, j) of the conversion data corresponding to the left channel are performed. ), Real part Ar (1, j) and imaginary part Br (1, j) of the conversion data corresponding to the right channel are obtained.

[Formula 3]
Al (1, j) = Σi _{= 0,..., N-1} W (1, i) .Xl (i) .cos (2πij / N)
Bl (1, j) = Σi _{= 0,..., N-1} W (1, i) · Xl (i) · sin (2πij / N)
Ar (1, j) = Σi _{= 0,..., N-1} W (1, i) .Xr (i) .cos (2πij / N)
Br (1, j) = Σ _{i = 0,..., N-1} W (1, i) · Xr (i) · sin (2πij / N)

  When the frequency converting means 20 performs Fourier transform on the B type sound frame, the left channel signal Xl (i + N / 2), the right channel signal Xr (i + N / 2) (i = 0,..., N−1) ) Is processed using the window function W (2, i) according to the following [Equation 4], and the real part Al (2, j) and imaginary part Bl of the conversion data corresponding to the left channel are performed. (2, j), real part Ar (2, j) and imaginary part Br (2, j) of the conversion data corresponding to the right channel are obtained.

[Formula 4]
Al (2, j) = Σi _{= 0,..., N-1} W (2, i) · Xl (i + N / 2) · cos (2πij / N)
Bl (2, j) = Σi _{= 0,..., N-1} W (2, i) .Xl (i + N / 2) .sin (2πij / N)
Ar (2, j) = Σi _{= 0,..., N-1} W (2, i) .Xr (i + N / 2) .cos (2πij / N)
Br (2, j) = Σi _{= 0,..., N-1} W (2, i) .Xr (i + N / 2) .sin (2πij / N)

  In the above [Expression 3] and [Expression 4], i is a serial number assigned to N samples in each acoustic frame, and takes an integer value of i = 0, 1, 2,... N−1. Further, j is a serial number assigned in order from the smallest value of the frequency value, and takes an integer value of j = 0, 1, 2,... N / 2-1 like i. When the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by 10.8 Hz.

  By executing the processing according to the above [Equation 3] and [Equation 4], a complex frequency component corresponding to each window function of each acoustic frame is obtained. Subsequently, the frequency component changing unit 30 extracts a spectrum set in a predetermined frequency range from the generated complex frequency component. In the present embodiment, a range between F1 and F2 is extracted.

  After performing the frequency conversion, the frequency conversion means 20 performs conversion so that the average value of each spectrum set becomes a set value. A so-called amplitude conversion process is performed. The purpose of this amplitude conversion is to reduce the level difference for each frame so that appropriate information can be embedded. Therefore, the set value Zo can be set as appropriate.

  The frequency conversion means 20 first calculates the conversion magnification when performing the amplitude conversion on the obtained spectrum set. The conversion magnification is calculated by dividing the set value by an average effective value that is an average of effective intensity values of a spectrum set in a predetermined frequency range. Specifically, conversion magnifications Zl (1) and Zl (2) for spectrum sets of the L-channel A-type acoustic frame, the B-type acoustic frame, the R-channel A-type acoustic frame, and the B-type acoustic frame. , Zr (1), Zr (2) are calculated by processing according to the following [Equation 5]. When the target sound frame is close to silence and the square sum of the denominator is less than a predetermined value, the conversion magnification is set to 1.0 and the amplitude conversion is not performed. By performing the amplitude conversion, all the sound frames are in the state of amplitude conversion so that the average effective value of the spectrum set of each sound frame becomes the set value regardless of the signal intensity in the original state. Will be embedded.

[Formula 5]
Zl (1) = Zo / [Σ _{j = m,..., M−1} {Al (1, j) ² + Bl (1, j) ² }] ^1/2
Zl (2) = Zo / [Σ _{j = m,..., M−1} {Al (2, j) ² + Bl (2, j) ² }] ^1/2
Zr (1) = Zo / [Σ _{j = m,..., M−1} {Ar (1, j) ² + Br (1, j) ² }] ^1/2
Zr (2) = Zo / [Σ _{j = m,..., M−1} {Ar (2, j) ² + Br (2, j) ² }] ^1/2

  In the above [Formula 5], m and M are the lower limit and upper limit of the frequency band to be changed, and Zo = M−m. In this embodiment, M = 320, m = 32, and Zo = 288.

  Further, Zl (1) for each element of Al (1, j) and Bl (1, j) in the range of j = m,..., M−1 (corresponding to frequencies F1,..., F2). ), Multiply each element of Al (2, j) and Bl (2, j) by Zl (2), and multiply each element of Ar (1, j) and Br (1, j) The amplitude is converted by multiplying Zr (1) and multiplying each element of Ar (2, j) and Br (2, j) by Zr (2). In the following description, Al (1, j), Bl (1, j), Al (2, j), Bl (2, j), Ar (1, j), Br (1, j), Ar (2, j) and Br (2, j) are values obtained by performing these amplitude conversions.

  The frequency component changing unit 30 performs a process of changing the ratio of the predetermined complex frequency component for the A type sound frame according to the bit arrangement created by the bit arrangement creating unit 70. In the present invention, the bit array is read bit by bit, and 1-bit information is embedded in a pair of acoustic frames of A type and B type. There are two 1-bit values to be embedded: “0” and “1”. In the present embodiment, these are defined as value 1 and value 2. These can be expressed as code 1 and code 2 in that two types of codes can be embedded. At this time, any one of “0” and “1” may be defined as a value 1 and a value 2 (reference numerals 1 and 2). This is because it is sufficient that one bit embedded on the extraction side can be specified on the extraction side. Therefore, this definition must match between the embedding side and the extraction side.

  Specifically, when changing the complex frequency component of a predetermined complex frequency range in an acoustic frame composed of a predetermined number of samples extracted from an acoustic signal, the strength of the embedding device causes sound pulse segregation. Change to the correct state. As a result, although it does not sound as if the sound is interrupted by a human, the extraction device can recognize the clear change.

  In the present embodiment, using the principle of sound pulse segregation, the components of the change target frequency band of the acoustic frame are changed to two states, and 1-bit information is embedded. Here, the state of change of the predetermined complex frequency component of the acoustic frame before and after the embedding process will be described. FIG. 3 shows the states of the predetermined complex frequency components of the A type and B type L channel 1 sound frames according to this embodiment. The R channel is omitted because it is the same as the L channel. In each acoustic frame shown in FIG. 3, the horizontal axis indicates the time direction, and the vertical axis indicates the frequency direction.

  In FIG. 3, the frequency region is divided into six in the frequency direction of the vertical axis, but the change target frequency band is the second to fifth regions from the top, that is, between frequency F1 and F2 or less. The uppermost part, that is, the frequency exceeding F2, and the lowermost part, that is, less than F1, are frequency bands not to be changed. That is, in the present embodiment, the intensity of the spectrum set is changed by setting the frequency F1 to F2 as a predetermined frequency range. As shown in FIG. 3A, regarding the change target frequency band of the A type acoustic frame, the spectrum set is expressed by 1D1, 1D2, 1U1, and 1U2 in order of increasing frequency. Also, as shown in FIG. 3B, for the change target frequency band of the B-type acoustic frame, the spectrum set is represented by 2D1, 2D2, 2U1, and 2U2 in order of increasing frequency.

  When embedding code 1 by the 1-bit embedding method, as shown in FIGS. 3C and 3E, the product of the strengths of 1D1 and 2D2 and the product of the strengths of 1U1 and 2U2 are changed to a relatively strong state, The product of the strengths of 1D2 and 2D1 is changed to a relatively weak state. This state is referred to as “state 1”. When embedding the code 2, as shown in FIGS. 3D and 3F, the product of the strength of 1D2 and 2D1 is changed to a relatively strong state, and the product of the strength of 1D2 and 2U1 is changed to a strength of 1D1 and 2D2. The product of the intensity of 1U1 and 2U2 is changed to a relatively weak state. This state is referred to as “state 2”. The shaded portions indicate that the ones having the same concentration are a spectrum set that is a set for obtaining a product. The darker shaded color indicates a group that is changed to a relatively strong state.

  Information is embedded by changing the frequency components of the A-type and B-type sound frames in two states as shown in FIGS. 3C, 3E, 3D, and 3F. Since there are two states, this corresponds to an information amount of 1 bit. As shown in FIGS. 3C to 3F, the upper frequency band and the lower frequency band of the change target frequency band have the same pattern.

  In the present embodiment, the change target frequency bands F1 to F2 are set to “0.34 kHz to 3.4 kHz”. This is due to the following reasons. That is, when a mobile phone having a high degree of spread as voice communication is used as a receiving terminal, the upper limit needs to be 3.4 kHz which is the upper limit of the telephone line band and the mobile phone. This is because the low-pass filter mounted inside the mobile phone corresponds to 3.4 kHz or less in accordance with the telephone exchange. Also, the lower limit of the upper frequency band is set to 1.7 kHz, which is one octave lower than the upper limit of 3.4 kHz. The upper limit of the lower frequency band is set to 1.7 kHz, similar to the lower limit of the upper frequency band, and the lower limit of the lower frequency band needs to be 0.3 kHz or more, which is the lower limit of the telephone line band and the mobile phone. This is because the high-pass filter mounted inside the mobile phone corresponds to 0.3 kHz or more according to the telephone exchange. Therefore, the upper limit of 1.7 kHz is set to 0.34 kHz, which is slightly lower by 2 octaves. The frequency range of the lower frequency band is slightly narrower than the frequency range of the upper frequency band, but the intensity of the distributed signal components is about 4 times on average, so the sensitivity of the lower frequency band is greater than that of the upper frequency band. Can be estimated. The values “0.34 kHz”, “1.7 kHz”, and “3.4 kHz” are representative values, and are not necessarily accurate values, and may be slightly deviated from them.

In the example shown in FIG. 3, the change to the relatively strong state and the weak state has been described. However, the degree of the strength can be set according to the situation. As will be described below, the larger the ratio between the two, the higher the accuracy at the time of extraction. However, the ratio of interpolation becomes incomplete, and noise due to discontinuous components is heard during reproduction. On the other hand, as the ratio between the two is equal, the reproduction quality approaches the original sound, but the embedded bits cannot be extracted, and the reproduction quality and extraction accuracy are in a trade-off relationship. For example, when the strong side is set to 100% and the weak side is set to 0%, the sound of the portion to be interpolated is 50% of the sound that was played in the original acoustic signal before the change as shown in FIG. % Has been confirmed. Therefore, if the strong side is set to 70% and the weak side is set to 30%, the sound of the portion to be interpolated is almost the same as the sound played by the original acoustic signal before the change as shown in FIG. It has been confirmed that this ratio is the limit that can maintain the extraction accuracy. For this reason, it is preferable to set the intensity ratio of the relatively strong spectrum set and the relatively weak spectrum set as 70% and 30%. In order to realize this, in this embodiment, in a specific process described later, a coefficient α = 0.7 for setting a strong state and a coefficient β = 0.3 for setting a weak state are set. However, when the intensity of the spectrum set to be changed to a strong state is originally small, it is necessary to correct the coefficients α and β. For this reason, the frequency component changing means 30 first executes processing according to the following [Equation 6] to thereby obtain the intensities E _1D1 , E _2D1 , E _1D2 , E _2D2 , E _1U1 , E _{2U1 of} each spectrum set. , to calculate the E _1U2, E _2U2.

[Formula 6]
E _1D1 = Σ _{j = m,..., M + G−1} {Al (1, j) ² + Bl (1, j) ² }
E _2D1 = Σ _{j = m,..., M + G−1} {Al (2, j) ² + Bl (2, j) ² }
E _1D2 = Σ _{j = m + G,..., M + 2G−1} {Al (1, j) ² + Bl (1, j) ² }
E _2D2 = Σ _{j = m + G,..., M + 2G−1} {Al (2, j) ² + Bl (2, j) ² }
E _1U1 = Σ _{j = 2m,..., 2m + 2G-1} {Al (1, j) ² + Bl (1, j) ² }
E _2U1 = Σ _{j = 2m,..., 2m + 2G−1} {Al (2, j) ² + Bl (2, j) ² }
E _1U2 = Σ _{j = 2m + 2G,..., 2m + 4G-1} {Al (1, j) ² + Bl (1, j) ² }
E _2U2 = Σ _{j = 2m + 2G,..., 2m + 4G-1} {Al (2, j) ² + Bl (2, j) ² }

  In the above [Expression 6], m is the number of the complex frequency component at the lower limit of the frequency band to be changed, G is the width of the divided region of the lower frequency band in the frequency band to be changed, and 2m + 4G is the upper limit of the frequency band to be changed. This is the number of the complex frequency component. For example, when 0.34 kHz to 3.4 kHz is set as the change target frequency band, m = 32 and 2m + 4G = 320 (= M). Therefore, the width G (= (M−2m) / 4) = 64 of the two divided regions in the lower frequency band and the width 2G = 128 of the two divided regions in the upper frequency band.

  Further, the frequency component changing unit 30 uses the calculated intensity of each spectrum set to calculate the intensity ratio γ of the spectrum set to be changed to a strong state with respect to the spectrum set to be changed to the weak state. The intensity ratio γ is calculated according to the following [Equation 7].

[Formula 7]
_{γ = (E 1D1 · E 1U1} · E 2D2 · E 2U2) / (E 1D2 · E 1U2 · E 2D1 · E 2U1)

  Further, according to the value of the intensity ratio γ, the frequency component changing unit 30 performs processing according to the following [Equation 8] to correct the coefficients α and β to obtain the coefficients α ′ and β ′.

[Formula 8]
In the case of 0.01 ≦ γ <1.0, α'= α · γ -1/4, β'= β · γ 1/4
When γ <0.01, α ′ = 10.0 · α, β ′ = 0.1 · β
When γ ≧ 1.0, no correction is performed.

  Further, the frequency component changing means 30 is provided with real parts Al (1, j), Ar (1, j), Al (2, j), Ar (2, j) in the continuous A type acoustic frame and B type acoustic frame. , Bl (1, j), Br (1, j), Bl (2, j), Br (2, j) as frequency domain parameters, the lower limit m (= 32) to the upper limit M (= 320) And m to M / 2 should be divided into two regions with width G (= (M-2m) / 4), and 2m to M should be divided into two regions with width 2G and embedded Each modification is made according to the bit value. As an example, when the bit value to be embedded in one sound frame is “value 1”, the state of the frequency component is changed to “state 1”, that is, FIG. (C) Change to the state shown in (e).

[Formula 9]
For each component of j = m to m + G−1, E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^1/2
Al ′ (1, j) = Al (1, j) · E (1, j) · α / {Al (1, j) ² + Bl (1, j) ² } ^1/2
Bl ′ (1, j) = Bl (1, j) · E (1, j) · α / {Al (1, j) ² + Bl (1, j) ² } ^1/2
Ar ′ (1, j) = Ar (1, j) · E (1, j) · α / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Br ′ (1, j) = Br (1, j) · E (1, j) · α / {Ar (1, j) ² + Br (1, j) ² } ^1/2
E (2, j) = {Al (2, j) ² + Bl (2, j) ² + Ar (2, j) ² + Br (2, j) ² } ^1/2
Al ′ (2, j) = Al (2, j) · E (2, j) · β / {Al (2, j) ² + Bl (2, j) ² } ^1/2
Bl ′ (2, j) = Bl (2, j) · E (2, j) · β / {Al (2, j) ² + Bl (2, j) ² } ^1/2
Ar ′ (2, j) = Ar (2, j) · E (2, j) · β / {Ar (2, j) ² + Br (2, j) ² } ^1/2
Br ′ (2, j) = Br (2, j) · E (2, j) · β / {Ar (2, j) ² + Br (2, j) ² } ^1/2
For each component of j = m + G to m + 2G-1, E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^1/2
Al ′ (1, j) = Al (1, j) · E (1, j) · β / {Al (1, j) ² + Bl (1, j) ² } ^1/2
Bl ′ (1, j) = B1 (1, j) · E (1, j) · β / {Al (1, j) ² + B1 (1, j) ² } ^1/2
Ar ′ (1, j) = Ar (1, j) · E (1, j) · β / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Br ′ (1, j) = Br (1, j) · E (1, j) · β / {Ar (1, j) ² + Br (1, j) ² } ^1/2
E (2, j) = {Al (2, j) ² + Bl (2, j) ² + Ar (2, j) ² + Br (2, j) ² } ^1/2
Al ′ (2, j) = Al (2, j) · E (2, j) · α / {Al (2, j) ² + Bl (2, j) ² } ^1/2
Bl ′ (2, j) = Bl (2, j) · E (2, j) · α / {Al (2, j) ² + Bl (2, j) ² } ^1/2
Ar ′ (2, j) = Ar (2, j) · E (2, j) · α / {Ar (2, j) ² + Br (2, j) ² } ^1/2
Br ′ (2, j) = Br (2, j) · E (2, j) · α / {Ar (2, j) ² + Br (2, j) ² } ^1/2
For each component of j = 2m to 2m + 2G-1, E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^1/2
Al ′ (1, j) = Al (1, j) · E (1, j) · α / {Al (1, j) ² + Bl (1, j) ² } ^1/2
Bl ′ (1, j) = Bl (1, j) · E (1, j) · α / {Al (1, j) ² + Bl (1, j) ² } ^1/2
Ar ′ (1, j) = Ar (1, j) · E (1, j) · α / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Br ′ (1, j) = Br (1, j) · E (1, j) · α / {Ar (1, j) ² + Br (1, j) ² } ^1/2
E (2, j) = {Al (2, j) ² + Bl (2, j) ² + Ar (2, j) ² + Br (2, j) ² } ^1/2
Al ′ (2, j) = Al (2, j) · E (2, j) · β / {Al (2, j) ² + Bl (2, j) ² } ^1/2
Bl ′ (2, j) = Bl (2, j) · E (2, j) · β / {Al (2, j) ² + Bl (2, j) ² } ^1/2
Ar ′ (2, j) = Ar (2, j) · E (2, j) · β / {Ar (2, j) ² + Br (2, j) ² } ^1/2
Br ′ (2, j) = Br (2, j) · E (2, j) · β / {Ar (2, j) ² + Br (2, j) ² } ^1/2
For each component of j = 2m + 2G to M−1, E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^1/2
Al ′ (1, j) = Al (1, j) · E (1, j) · β / {Al (1, j) ² + Bl (1, j) ² } ^1/2
Bl ′ (1, j) = B1 (1, j) · E (1, j) · β / {Al (1, j) ² + B1 (1, j) ² } ^1/2
Ar ′ (1, j) = Ar (1, j) · E (1, j) · β / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Br ′ (1, j) = Br (1, j) · E (1, j) · β / {Ar (1, j) ² + Br (1, j) ² } ^1/2
E (2, j) = {Al (2, j) ² + Bl (2, j) ² + Ar (2, j) ² + Br (2, j) ² } ^1/2
Al ′ (2, j) = Al (2, j) · E (2, j) · α / {Al (2, j) ² + Bl (2, j) ² } ^1/2
Bl ′ (2, j) = Bl (2, j) · E (2, j) · α / {Al (2, j) ² + Bl (2, j) ² } ^1/2
Ar ′ (2, j) = Ar (2, j) · E (2, j) · α / {Ar (2, j) ² + Br (2, j) ² } ^1/2
Br ′ (2, j) = Br (2, j) · E (2, j) · α / {Ar (2, j) ² + Br (2, j) ² } ^1/2

  Since one bit is embedded in one acoustic frame, the intensity pattern of the spectrum set unit is the same on the high frequency side and the low frequency side.

  When the coefficients α ′ and β ′ are obtained by executing the processing according to the above [Equation 8], the coefficients α ′ and β ′ are replaced with the coefficients α and β in the [Equation 9]. Use.

  Next, the phase correction unit 31 corrects the phase of the low frequency component in which the above component is changed. The correction target is a low frequency component whose intensity has been changed to be small by the frequency component changing means 30. In order to perform phase correction for a low-frequency component of a certain acoustic frame, the low-frequency component after change processing in the immediately preceding acoustic frame is used. Therefore, when the phase correction unit 31 performs the phase change process, the low frequency component after the change process by the frequency component change unit 30 is temporarily held in the memory.

  As a specific process, the phase correction unit 31 uses the frequency component changing unit 30 to change the low frequency components (j = m,..., M−) of the B type sound frame that has been changed using β ′. 1 for each frequency component), the phase correction means 31 executes the processing according to the following [Equation 10] to correct the phase of the low frequency component.

[Formula 10]
El ′ (1, j) = {Al ′ (1, j) ² + Bl ′ (1, j) ² } ^1/2
El ′ (2, j) = {Al ′ (2, j) ² + Bl ′ (2, j) ² } ^1/2
Er ′ (1, j) = {Ar ′ (1, j) ² + Br ′ (1, j) ² } ^1/2
Er ′ (2, j) = {Ar ′ (2, j) ² + Br ′ (2, j) ² } ^1/2
Al '(2, j) ← (1.0-w) .Al' (2, j) -w.Al '(1, j) .El' (2, j) / El '(1, j)
Bl ′ (2, j) ← (1.0−w) · B1 ′ (2, j) −w · B1 ′ (1, j) · El ′ (2, j) / E1 ′ (1, j)
Ar ′ (2, j) ← (1.0−w) · Ar ′ (2, j) −w · Ar ′ (1, j) · Er ′ (2, j) / Er ′ (1, j)
Br ′ (2, j) ← (1.0−w) · Br ′ (2, j) −w · Br ′ (1, j) · Er ′ (2, j) / Er ′ (1, j)
Al ′ (2, j) ← Al ′ (2, j) .El ′ (2, j) / {Al ′ (2, j) ² + B1 ′ (2, j) ² } ^1/2
(El ′ (2, j) of the numerator multiplied by (Al ′ (2, j)) is the size of the complex vector calculated at the beginning of [Formula 10], and the denominator {Al ′ (2, j) ² + B1 ′ (2, j) ² } ^1/2 changes in value according to the magnitude of the complex vector after the above update processing is performed on Al ′ (2, j) and B1 ′ (2, j). Therefore, the correction is made so that the initial value is obtained.
Bl ′ (2, j) ← Bl ′ (2, j) · El ′ (2, j) / {Al ′ (2, j) ² + Bl ′ (2, j) ² } ^1/2
Ar' (2, j) ← Ar' (2, j) · Er' (2, j) / {Ar' (2, j) 2 + Br' (2, j) 2} 1/2
Br' (2, j) ← Br' (2, j) · Er' (2, j) / {Ar' (2, j) 2 + Br' (2, j) 2} 1/2

  In the above [Equation 10], w is a coefficient that determines the degree of phase correction. The coefficient w is a real number between 0.0 and 1.0, and is set according to the reverberation characteristic to be used. It is. In this embodiment, w = 0.8. When the coefficient w is the minimum value 0.0, it means that no phase correction is performed. When w is the maximum value 1.0, the phase is opposite (180 degrees opposite) to the corresponding component of the immediately preceding acoustic frame. It means to correct. In the present invention, reverberation is corrected by correcting the direction of the complex frequency component in the direction opposite to the complex frequency component of the immediately preceding acoustic frame. This “opposite side” is the complex frequency component in the immediately preceding acoustic frame. It means different angles in the range of 90 degrees or more and 180 degrees or less. In this case, if w> 0.8 (144 degrees in angle), it is not preferable because reverberation due to the complex frequency component of two or more previous acoustic frames is not removed. Eventually, the phase correction means 31 performs the processing according to [Equation 10], and uses the immediately preceding A type acoustic frame to perform the low frequency component of the B type acoustic frame as shown in FIG. Thus, phase correction as shown in FIG. In addition, Al ′ (2, j), Ar ′ (2, j), Bl ′ (2, j), and Br ′ (2, j) obtained in [Equation 10] They are also used as Al ′ (0, j), Ar ′ (0, j), Bl ′ (0, j) and Br ′ (0, j), respectively.

  For the low frequency component of the A type sound frame that has been changed using β ′ by the frequency component changing means 30, the phase correcting means 31 performs processing according to the following [Equation 11]. By executing, the phase of the low frequency component is changed.

[Formula 11]
El ′ (0, j) = {Al ′ (0, j) ² + Bl ′ (0, j) ² } ^1/2
El ′ (1, j) = {Al ′ (1, j) ² + Bl ′ (1, j) ² } ^1/2
Er ′ (0, j) = {Ar ′ (0, j) ² + Br ′ (0, j) ² } ^1/2
Er ′ (1, j) = {Ar ′ (1, j) ² + Br ′ (1, j) ² } ^1/2
Al '(1, j) ← (1.0-w) .Al' (1, j) -w.Al '(0, j) .El' (1, j) / El '(0, j)
Bl ′ (1, j) ← (1.0−w) · B1 ′ (1, j) −w · B1 ′ (0, j) · El ′ (1, j) / E1 ′ (0, j)
Ar ′ (1, j) ← (1.0−w) · Ar ′ (1, j) −w · Ar ′ (0, j) · Er ′ (1, j) / Er ′ (0, j)
Br ′ (1, j) ← (1.0−w) · Br ′ (1, j) −w · Br ′ (0, j) · Er ′ (1, j) / Er ′ (0, j)
Al ′ (1, j) ← Al ′ (1, j) .El ′ (1, j) / {Al ′ (1, j) ² + B1 ′ (1, j) ² } ^1/2
Bl ′ (1, j) ← Bl ′ (1, j) · El ′ (1, j) / {Al ′ (1, j) ² + Bl ′ (1, j) ² } ^1/2
Ar' (1, j) ← Ar' (1, j) · Er' (1, j) / {Ar' (1, j) 2 + Br' (1, j) 2} 1/2
Br' (1, j) ← Br' (1, j) · Er' (1, j) / {Ar' (1, j) 2 + Br' (1, j) 2} 1/2

  In the above [Equation 11], Al ′ (0, j) and Ar ′ (0, j) indicate the real part of the frequency component after phase correction in the immediately preceding B-type acoustic frame, and Bl ′ (0, j), Br ′ (0, j) represents the imaginary part of the frequency component after phase correction in the immediately preceding B-type sound frame. Eventually, the phase correction means 31 performs the process according to [Formula 11], and uses the immediately preceding B type acoustic frame to perform the low frequency component of the A type acoustic frame as shown in FIG. Thus, phase correction as shown in FIG.

  As described above, the frequency inverse transform means 40 performs frequency inverse transform on the complex frequency component in which the phase of the frequency component is corrected to obtain a real-dimensional modified acoustic frame. Naturally, the inverse frequency conversion needs to correspond to the method executed by the frequency conversion means 20. In the present embodiment, since the frequency transform unit 20 performs the Fourier transform, the frequency inverse transform unit 40 executes the Fourier inverse transform.

  Specifically, for the A type sound frame, the frequency inverse transform means 40 performs the real part Al ′ (1, j) and the imaginary part of the left channel of the complex frequency component obtained by the above [Equation 9]. Using Bl ′ (1, j), the real part Ar ′ (1, j) of the right channel, and the imaginary part Br ′ (1, j), processing according to the following [Equation 12] is performed, and Xl ′ ( i) and Xr ′ (i) are calculated. For frequency components that are not modified in the above [Equation 9], Al (1, j) or the like that is the original frequency component is used as Al ′ (1, j) or the like. In calculating the inverse frequency transform, for Al ′ (1, j) and Bl ′ (1, j), Zl (1) in [Formula 5] is replaced with Ar ′ (1, j) and Br ′ (1 , J), it is necessary to simultaneously perform inverse amplitude transformation by dividing Zr (1) in [Formula 5].

[Formula 12]
Xl' (i) = 1 / N · {Σ j Al' (1, j) · cos (2πij / N) / Zl (1) -Σ j Bl' (1, j) · sin (2πij / N) / Zl (1)} + Xlp (i + N / 2)
Xr' (i) = 1 / N · {Σ j Ar' (1, j) · cos (2πij / N) / Zr (1) -Σ j Br' (1, j) · sin (2πij / N) / Zr (1)} + Xrp (i + N / 2)

In the above [Expression 12], Σ _{j = 0,..., N−1} is shown as Σ _{j in} order to prevent the expression from becoming complicated. The terms “+ Xlp (i + N / 2)” in the first equation and “+ Xrp (i + N / 2)” in the second equation in the above [Equation 12] indicate the data Xlp (i) of the modified acoustic frame modified immediately before, When Xrp (i) exists, the addition is performed in consideration of the overlap of N / 2 samples on the time axis. By the above [Equation 12], each sample Xl ′ (i) of the left channel and each sample Xr ′ (i) of the right channel of the A type modified acoustic frame are obtained.

  For the B type acoustic frame, the frequency inverse transforming means 40 performs the real part Al ′ (2, j) and imaginary part Bl ′ (2, 2) of the left channel of the complex frequency component obtained by the above [Equation 9]. j) Using the real part Ar ′ (2, j) and imaginary part Br ′ (2, j) of the right channel, the processing according to the following [Equation 13] is performed, and Xl ′ (i), Xr ′ (I) is calculated. For frequency components that are not modified in the above [Equation 9], in the following [Equation 13], Al ′ (2, j), Bl ′ (2, j), Ar ′ (2, j), Br The original values Al (2, j), Bl (2, j), Ar (2, j), and Br (2, j) are used as ′ (2, j). In calculating the frequency inverse transform, for Al ′ (2, j) and Bl ′ (2, j), Zl (2) in [Equation 5] is replaced with Ar ′ (2, j) and Br ′ (2 , J), it is necessary to simultaneously perform inverse amplitude transformation by dividing Zr (2) in [Equation 5].

[Formula 13]
Xl' (i + N / 2) = 1 / N · {Σ j Al' (2, j) · cos (2πij / N) / Zl (2) -Σ j Bl' (2, j) · sin (2πij / N ) / Zl (2)} + Xlp (i + N)
Xr' (i + N / 2) = 1 / N · {Σ j Ar' (2, j) · cos (2πij / N) / Zr (2) -Σ j Br' (2, j) · sin (2πij / N ) / Zr (2)} + Xrp (i + N)

  By the above [Equation 13], each sample Xl ′ (i) of the left channel and each sample Xr ′ (i) of the right channel of the B type modified acoustic frame are obtained.

  The modified sound frame output unit 50 sequentially outputs the A type modified sound frame and the B type modified sound frame obtained by the processing of the frequency inverse transform unit 40 to an output file.

  Next, the overall flow of processing of the information embedding device for the acoustic signal shown in FIG. 1 will be described. First, a code conversion table in which an Nw-bit reference code and an Nh-bit Hamming code are associated with each other as shown in FIG. 5 is prepared in advance. This code conversion table may be created in any way as long as the Hamming distance between each Hamming code is 3 or more. In this embodiment, it is created by causing a computer to function as a conversion table creation device by executing a dedicated program. FIG. 4 shows a flowchart of code conversion table creation by the conversion table creation device.

  The conversion table creation device first performs initialization processing (S601). Specifically, the 8-bit reference code “0” is associated with the 16-bit code “1” and registered in the i (= 0) -th code conversion table, and the initial value of the 8-bit reference code KF is set to 1, 16 The initial value of the bit code HF is 2. Subsequently, i = 0 is initially set (S602). Next, the Hamming distance between the 16-bit code HF and its 15 cyclic codes and the other 16-bit code already registered in the code conversion table and its 15 cyclic codes is calculated (S603). ). At this stage, 16 ways × 16 ways and 256 ways of Hamming distances are calculated.

  If at least one of the calculated Hamming distances is less than 3, the value of HF is updated by 1 (S604), and then the process returns to S602 and the process for the next 16-bit code HF is performed. On the other hand, if all the calculated Hamming distances are 3 or more, the value of i is incremented by 1 and updated (S605). If i is less than KF-1, the process returns to S603 to calculate the Hamming distance between the i-th 16-bit code registered in the code conversion table and its 15 cyclic codes. When i becomes KF-1 or more after S605, the 16-bit code HF is registered at the position of the 8-bit code KF in the code conversion table, and the values of KF and HF are incremented by 1 and updated. (S606). If HF is less than 65536, the process returns to S612 to perform the process for the next 8-bit code KF. If the HF is 65536 or more, all 16-bit codes HF that can be expressed in a 16-bit range have been registered, and the code conversion table creation process ends.

  In this way, the code conversion table shown in FIG. 5 is created. The code conversion table in FIG. 5 is obtained by extracting only the corresponding portion with the Hamming code that does not consider the cyclic code. In this code conversion table, an 8-bit reference code is defined so that 16 cyclic codes each have a Hamming distance of 3 or more within a range of 0 to 65535 in which a 16-bit Hamming code can be defined. Thus, as the reference code, a value of 0 to 131 slightly more than 7 bits can be expressed.

  Next, the processing of the embedding device after the code conversion table creation processing will be described with reference to the flowchart of FIG. Each component which comprises the apparatus shown in FIG. 1 cooperates, and performs the process according to FIG. FIG. 6 corresponds to processing for one word of additional information. The number of bits Nw of one word can be set to an arbitrary number of bits, but as described above, in this embodiment, it is set to 8 bits of the effective storage capacity of the ASCII code.

  In FIG. 6, first, the bit array creation means 70 creates a corresponding bit array by referring to the conversion table creation means 80 for each word of the additional information extracted from the additional information storage unit 62 (S101). Specifically, first, a 16-bit bit array that is extracted from the additional information storage unit 62 in units of 1 word (8 bits) and refers to the code conversion table shown in FIG. To extract.

  These 16 bits are read into a register in a computer used as an information embedding device for an acoustic signal. Thus, in the additional information storage unit 62, one word is 8 bits, but at the time of embedding processing, processing for one word in the additional information is performed with this 16-bit array.

  Next, the frequency component changing unit 30 performs a process of reading one bit from Nh (= 16) bits held in the register (S102). Subsequently, the acoustic frame reading means 10 reads a predetermined number of samples from the left and right channels of the stereo acoustic signal stored in the acoustic signal storage unit 61 as one A-type acoustic frame, and the frequency converting means 20 Conversion is performed, and amplitude conversion is performed on the obtained complex frequency component (S103). Specifically, first, frequency conversion is performed on the read sound frame to obtain a complex frequency component that is a complex frequency component of the sound frame. That is, for each acoustic frame, processing according to the above [Equation 2] is performed using the window function W (1, i). Then, the process according to the above [Equation 5] is executed to calculate Zl (1) and Zr (1) and perform amplitude conversion. Similarly, the acoustic frame reading means 10 reads a predetermined number of samples as one B-type acoustic frame from the left and right channels of the stereo acoustic signal stored in the acoustic signal storage unit 61, and the frequency conversion means 20 Conversion is performed, and amplitude conversion is performed on the obtained complex frequency component (S104). Specifically, first, frequency conversion is performed on the read sound frame to obtain a complex frequency component of the sound frame. That is, for each acoustic frame, processing according to the above [Equation 3] is performed using the window function W (2, i). Then, the process according to the above [Equation 5] is executed to calculate Zl (2) and Zr (2) and perform amplitude conversion. Subsequently, when the frequency component changing unit 30 changes the state of the frequency component of the A type sound frame and the B type sound frame according to the read bit value, the processing according to [Formula 6] and [Formula 8] is executed. Then, the conversion ratios α and β are determined (S105). Using this determined conversion ratio, the frequency component changing unit 30 executes the processing according to the above [Equation 9] according to the value 1 and the value 2 received from the bit array creating unit 70, and the frequency band to be changed The state of the component is changed to either “state 1” or “state 2” (S106).

  Next, the phase correction means 31 performs phase correction of the component of the change target frequency band (S107). Specifically, the processing according to [Formula 10] and [Formula 11] is executed on the A type sound frame and the B type sound frame to correct the phase of each frequency component.

  Next, the frequency inverse transform means 40 performs amplitude inverse transform and frequency inverse transform on the complex frequency component in which the phase of each spectrum set corresponding to the A-type acoustic frame is corrected by the process of S107 to obtain a modified acoustic frame. Processing is performed (S108). This inverse amplitude transformation is performed by multiplying the complex frequency component by the inverse of Zl (1) and Zr (1) calculated by [Equation 5]. This frequency inverse transformation is naturally performed by the frequency transformation means 20 at S103. It is necessary to correspond to the method executed in the above. In the present embodiment, since the frequency transform unit 20 performs the inverse Fourier transform, the frequency inverse transform unit 40 performs the inverse Fourier transform. Specifically, the real part Al ′ (1, j) of the left channel and the imaginary part Bl ′ (1, j) of the complex frequency component obtained by [Equation 11], the real part Ar ′ (1) of the right channel. , J) and imaginary part Br ′ (1, j), the process according to the above [Equation 12] is performed to calculate Xl ′ (i) and Xr ′ (i). The modified sound frame output means 50 sequentially outputs the obtained modified sound frames to an output file.

  Similarly, the frequency inverse transform means 40 performs amplitude inverse transform and frequency inverse transform on the complex frequency component in which the phase of each spectrum set corresponding to the B-type acoustic frame is corrected by the processing of S107 to obtain a modified acoustic frame. Processing is performed (S109). Specifically, the inverse amplitude transformation is performed by multiplying the complex frequency component by the inverse of Zl (2) and Zr (2) calculated in [Formula 5], and the complex frequency component obtained by the above [Formula 10] is calculated. Using the real part Al ′ (2, j), the imaginary part Bl ′ (2, j) of the left channel, the real part Ar ′ (2, j), and the imaginary part Br ′ (2, j) of the right channel, Processing according to [Formula 13] is performed to calculate Xl ′ (i) and Xr ′ (i).

  The modified sound frame output means 50 sequentially outputs the obtained modified sound frames to an output file. When the processing for the two A-type and B-type sound frames is finished for each channel in this way, the frequency component changing means 30 reads the next 1 bit in the bit array (S102). The above processing is executed over all samples of both channels of the acoustic signal. That is, a predetermined number of samples are read as sound frames, and when there are no more sound frames to be read from the sound signal (S103, S104), the process ends. When the processing corresponding to each bit of the bit arrangement (Nh = 16 bits) for one word read in S101 is completed, the process returns from S102 to S101 to read the next word of the additional information and create a bit arrangement. Processing will be performed. When the processing is completed for all the words of the additional information, the processing returns to the first word of the additional information. As a result, all modified acoustic frames that have been processed for all acoustic frames are recorded in the output file and obtained as modified acoustic signals. The obtained modified acoustic signal is output to and stored in the modified acoustic signal storage unit 63 in the storage unit 60.

  In the present embodiment, the case has been described in which the additional information is converted into a 16-bit bit array using the code conversion table by converting the additional information into 8 bits per word, and processing for one word of additional information has been described in the present invention. As long as there is an agreement with the side, it is possible to record one word of additional information in units of other number of bits.

  Of the modified acoustic signal obtained as described above, for the portion in which the additional information is embedded, the components of the change target frequency band have only two distributions of state 1 and state 2. However, since the components other than the component of the frequency band to be changed remain as the original acoustic signals, various distributions are made based on the setting of the producer. Further, as is clear from the processing of [Formula 9], the components of the frequency band to be changed are changed at the same rate in the left channel and the right channel. Therefore, even at a position equidistant from both speakers, the components in the frequency band to be changed are in a relationship to be amplified without being canceled out, and information can be easily extracted.

  The embedding apparatus has been described above. Here, embedding and extraction using amplitude conversion performed in the present embodiment will be described with reference to FIGS. For comparison, FIGS. 7 and 8 show an embedding process and an extraction process when amplitude conversion is not used, and FIGS. 9 and 10 show an embedding process and an extraction process using amplitude conversion in the present embodiment. In these explanatory diagrams, the signal waveform of the acoustic signal assuming that 3-bit information is embedded is shown for the odd and even three pairs of acoustic frames, and the acoustic frames are shown in a state where they do not overlap each other for convenience of explanation. ing. In each figure, the waveform at the left end shows a normal acoustic signal waveform in the time dimension, the horizontal direction is the time axis, the time advances toward the right, and the vertical direction is intensity (amplitude). The waveform at the center or right end expresses the sum of complex frequency components in a predetermined frequency range after frequency conversion as a signal waveform, and is a signal waveform after virtually passing through a bandpass filter. The vertical direction is similarly strong, and is originally significantly lower than the left end, but here, for convenience of explanation, it is enlarged and expressed to the same extent as the left end. Although the levels of the two types of waveforms are originally different, they are represented at the same level here. Actually, there are high-frequency components and low-frequency components outside the predetermined frequency range as the data after frequency conversion, but these are not shown on the assumption that they are not modified, and are embedded according to FIG. The two signal waveforms are limited to the two frequency band components for performing the above.

  FIG. 7 (a) shows the original sound signal. The first pair of sound frames has a substantially constant intensity throughout, and the center pair of sound frames has a smaller odd frame intensity and an even frame intensity. The last pair of sound frames has a large odd-numbered frame and a small even-numbered frame. FIG. 7B shows the result of frequency conversion performed on such an acoustic signal. As shown in FIG. 3, the embedding target component is divided into upper and lower parts and the components are changed for convenience of FIG. ) Are expressed separately as two waveforms (b-1) and (b-2). The waveform shape after frequency conversion is different for each frequency and is not necessarily similar to that shown in FIG. 7A, but here, it is assumed that it is the same as before frequency conversion. By embedding 3-bit data [0, 0, 0] based on the method described with reference to FIG. 3 with respect to FIGS. 1 (corresponding to 1), FIGS. 7 (c-1) and (c-2) are obtained. The first pair can express a pattern as shown in FIG. 3, but the other two pairs of upper and lower ones are required to construct a magnitude relationship in the opposite direction to the original stage. It turns out that an appropriate pattern has not been constructed. FIG. 7D shows the result of performing frequency inverse transformation on the result of such modification. In general, the external shape (referred to as an envelope) of an acoustic signal waveform is determined by low-frequency components having a large energy distribution, which are included in signal components outside a predetermined frequency range in this application, and are modified in the stage of FIG. 7 (c). Therefore, FIG. 7D after frequency inverse transformation has a shape similar to FIG. 7A before frequency transformation.

  FIG. 8 shows how extraction processing is performed on the result of such embedding. FIGS. 8A and 8B correspond to FIGS. 7D and 7C, respectively. In FIG. 8B, when performing bit determination, since the pattern assumed in FIG. 3 is not formed except for the first pair, it is difficult to determine a correct bit.

  Next, a method in which amplitude conversion is introduced in the present embodiment will be described with reference to FIGS. 9 and 10. FIG. 9A shows the same original sound signal as FIG. 7A, and FIG. 9B in which the frequency conversion is performed in the same manner is also the same as FIG. 7B. Here, FIG. 9C shows the result of amplitude conversion performed in units of six frames. In the case of FIG. 9, since the amplitude in each frame is flat in FIG. 9 (a), the whole is flat in FIG. 9 (c). Since the intra-frame variation is followed even in the stage of FIG. 9C, it does not usually become completely flat as in the figure. (Actually, since FIG. 9C is frequency dimension data, the frequency dimension data is not flat and fluctuates.) Also, the conversion magnification is set for each frame, and the upper and lower two frequency components are set. Since the conversion is performed at the same magnification, there is usually a significant difference between the upper and lower sides (however, the time axis direction is relatively uniform). On the other hand, when 3-bit data [0, 0, 0] is embedded, FIGS. 9D-1 and 9D-2 are obtained. Since FIG. 9C, which is the original stage, has a flat waveform, it can be seen that an ideal pattern as shown in FIG. 3 can be easily constructed in all frames. Subsequently, FIG. 9E shows the result of inverse amplitude transformation performed by multiplying the inverse of the magnification set for each frame. At this stage, a pattern similar to that shown in FIG. 7C is often generated, but any shape may be used. Finally, when frequency inverse transform is performed, FIG. 9F is obtained, and similarly, the shape is similar to that of FIG. 9A of the original signal waveform.

  FIG. 10 shows how extraction processing is performed on the result of such embedding. FIGS. 10 (a), (b) and (c) correspond to FIGS. 9 (f), (e) and (d), respectively. Although the waveform shape after frequency conversion in FIG. 10B is basically different from that in FIG. 9B, the calculated amplitude conversion magnification is a similar value, and the amplitude conversion is performed at substantially the same magnification. 10 (c) is obtained. If bit determination is performed in the stage of FIG. 10C, an ideal pattern as shown in FIG. 3 is formed in all frames, and therefore correct bits can be determined.

  As described above, in the present embodiment in which the amplitude conversion is introduced, since the signal component to be embedded is converted and embedded so as to be flat in the time axis direction as shown in FIG. The probability of making an unnatural change that completely reverses the magnitude relationship between the component intensity on the low frequency side and the low frequency side is reduced, and the extraction accuracy on the extraction side can be increased while maintaining the quality.

(3.1. Configuration of extraction device)
Next, an apparatus for extracting information from an acoustic signal according to the present invention will be described. FIG. 11 is a block diagram showing an embodiment of an apparatus for extracting information from an acoustic signal according to the present invention. In FIG. 11, 100 is an acoustic signal input means, 110 is a reference frame acquisition means, 120 is a time-axis shift frame setting means, 130 is a frequency conversion means, 140 is a code determination parameter calculation means, 150 is a code output means, and 160 is an addition Information extraction means, 170 is an acoustic frame holding means, and 180 is a conversion table creation means.

  The acoustic signal input unit 100 has a function of acquiring and inputting a flowing sound as a digital acoustic signal. In reality, it is realized by a microphone and an A / D converter. Any microphone can be used, regardless of whether it is monaural omnidirectional or stereo directional, as long as the component in the frequency band to be changed can be detected. Even if it is stereo-directional, only one channel needs to be used. In addition, when information is embedded by the apparatus shown in FIG. 1, the information is reproduced in stereo. However, the sound from either the left or right speaker may be input, or the sound from both speakers may be mixed and input. There is no restriction on the position of the microphone. Of course, when the acoustic signal in which the additional information is embedded is monaurally reproduced, or when the acoustic signal itself in which the additional information is embedded is monaural, the microphone may be directed to the single speaker to be reproduced. This microphone is not highly accurate, and information can be extracted using a microphone with general accuracy. The reference frame acquisition unit 110 has a function of reading an audio frame composed of a predetermined number of samples as a reference frame from the input digital monaural audio signal (or one channel of a stereo audio signal). As the reference frame, A type and B type are set as in the case of embedding. The time axis shift frame setting means 120 has a function of setting an acoustic frame whose time axis is shifted by moving each of the A type and B type reference frames and predetermined samples as a time axis shift frame.

  The frequency conversion unit 130 basically has substantially the same function as the frequency conversion unit 20 shown in FIG. However, since the timing to capture the sound does not start from the beginning of the acoustic signal, in order to specify the correct time axis shift, amplitude conversion and frequency conversion are performed with multiple time axis shifts, and the original acoustic signal is stereo. Even in this case, the difference is that it is performed on one channel.

When performing the Fourier transform on the A type acoustic frame, the frequency converting unit 130 applies the window function W to the signal X (i−N / 2 + pN / 6) (i = 0,..., N−1). (1, i) is used to perform processing according to the following [Equation 14] to obtain the real part A (1, j, p) and imaginary part B (1, j, p) of the converted data. Have.
p is a time axis shift value and takes an integer value of 0 to 5.

[Formula 14]
A (1, j, p) = Σi _{= 0,..., N-1} W (1, i) .X (i-N / 2 + p.N / 6) .cos (2πij / N)
B (1, j, p) = Σi _{= 0,..., N-1} W (1, i) .X (i-N / 2 + p.N / 6) .sin (2πij / N)

  When the Fourier transform is performed on the B type acoustic frame, the frequency converting unit 130 applies the window function W (2) to the signal X (i + p · N / 6) (i = 0,..., N−1). , I) is used to perform processing according to the following [Formula 15] to obtain a real part A (2, j, p) and an imaginary part B (2, j, p) of the converted data. ing.

[Formula 15]
A (2, j, p) = Σ _{i = 0,..., N-1} W (2, i) · X (i + p · N / 6) · cos (2πij / N)
B (2, j, p) = Σi _{= 0,..., N-1} W (2, i) .X (i + p.N / 6) .sin (2πij / N)

  The frequency conversion unit 130 performs amplitude conversion in the same manner as the frequency conversion unit 20. In performing amplitude conversion, first, conversion magnification is calculated. The conversion magnification is calculated by dividing the set value by the average effective value that is the average of the effective intensity values of the spectrum set in the predetermined frequency range. Although the set value can be determined as appropriate, it is necessary to set the same value as in the case of amplitude conversion at the time of embedding. Therefore, in the present embodiment, the set value Zo needs to be 288 (= M−m). Specifically, the conversion magnifications Z (1, p) and Z (2, p) for the A type acoustic frame and the B type acoustic frame are calculated by processing according to the following [Equation 16]. . By performing the amplitude conversion, all the sound frames are extracted in a state where the amplitude is converted so that the average effective value of each sound frame becomes a set value regardless of the signal strength in the original state. Will be done.

[Formula 16]
Z (1, p) = Zo / [Σ _{j = m,..., M−1} {A (1, j, p) ² + B (1, j, p) ² }] ^1/2
Z (2, p) = Zo / [Σ _{j = m,..., M−1} {A (2, j, p) ² + B (2, j, p) ² }] ^1/2

  j = m,..., M-1 (corresponding to frequencies F1,..., F2) and p = 0,. . . , 5, each element of A (1, j, p) and B (1, j, p) is multiplied by Z (1, p) and A (2, j, p) and B ( The amplitude conversion is performed by multiplying each element of (2, j, p) by Z (2, p). In the following description, A (1, j, p), B (1, j, p), A (2, j, p), and B (2, j, p) are values obtained by performing these amplitude conversions. .

  The code determination parameter calculation unit 140 extracts a spectrum set corresponding to a predetermined frequency range from the generated complex frequency component, calculates an intensity value of each spectrum set, and uses this intensity value to calculate a code determination parameter. It has a function of calculating and determining that it is in a predetermined state based on the magnitude relationship of the code determination parameters.

  First, the basic intensity values E (1, j, p) and E (2, j, p) of each spectrum set are calculated by processing according to the following [Equation 17]. At the time of extraction, only the intensity value of the complex frequency component corresponding to the left side of [Expression 17] obtained is used for determination, and the complex number information corresponding to the right side of the same expression is not used. It is also possible to directly calculate E (1, j, p) and E (2, j, p).

[Formula 17]
E (1, j, p) = A (1, j, p) ² + B (1, j, p) ²
E (2, j, p) = A (2, j, p) ² + B (2, j, p) ²

  The intensity values E1, E2, E3, E4, E5, E6, E7, and E8 of each spectrum set are calculated as follows using the calculated basic intensity values E (1, j, p) and E (2, j, p). It calculates based on [Formula 18].

[Formula 18]
E1 (p) = Σ _{j = m,..., M + G-1} E (1, j, p)
E2 (p) = Σ _{j = m, ..., m + G-1} E (2, j, p)
E3 (p) = Σ _{j = m + G,..., M + 2G-1} E (1, j, p)
E4 (p) = Σ _{j = m + G,..., M + 2G-1} E (2, j, p)
E5 (p) = Σj _{= 2m, ..., 2m + 2G-1} E (1, j, p)
E6 (p) = Σ _{j = 2m, ..., 2m + 2G-1} E (2, j, p)
E7 (p) = Σ _{j = 2m + 2G, ..., 2m + 4G-1} E (1, j, p)
E8 (p) = Σ _{j = 2m + 2G, ..., 2m + 4G-1} E (2, j, p)

  Eventually, the intensity values E1, E2, E3, E4, E5, E6, E7, and E8 of each spectrum set are calculated by [Equation 17] and [Equation 18].

  Further, the code determination parameter calculation unit 140 calculates the code determination parameter C using the intensity values E1, E2, E3, E4, E5, E6, E7, and E8. By executing the processing according to the following [Equation 19], the candidate code B is provisionally determined and the code determination parameter C is calculated.

[Formula 19]
1) In the case of E1 (p) · E5 (p)> E2 (p) · E6 (p) and E4 (p) · E8 (p)> E3 (p) · E7 (p)
C = E1 (p) .E5 (p). {E4 (p) .E8 (p) -E2 (p) .E6 (p) .E3 (p) .E7 (p)} / {E1 (p). E5 (p), E4 (p), E8 (p) + E2 (p), E6 (p), E3 (p), E7 (p)}
2) When E2 (p) · E6 (p)> E1 (p) · E5 (p) and E3 (p) · E7 (p)> E4 (p) · E8 (p)
C = {E2 (p) .E6 (p) .E3 (p) .E7 (p) -E1 (p) .E5 (p) .E4 (p) .E8 (p)} / {E1 (p). E5 (p), E4 (p), E8 (p) + E2 (p), E6 (p), E3 (p), E7 (p)}
3) In the case of E1 (p), E5 (p), E4 (p), E8 (p)> E2 (p), E6 (p), E3 (p), E7 (p)
C = {E1 (p) .E5 (p) .E4 (p) .E8 (p) -E2 (p) .E6 (p) .E3 (p) .E7 (p)} / {E1 (p). E5 (p), E4 (p), E8 (p) + E2 (p), E6 (p), E3 (p), E7 (p)}
4) In cases other than the above 1) to 3) B = 1 and provisional determination,
C = {E2 (p) .E6 (p) .E3 (p) .E7 (p) -E1 (p) .E5 (p) .E4 (p) .E8 (p)} / {E1 (p). E5 (p), E4 (p), E8 (p) + E2 (p), E6 (p), E3 (p), E7 (p)}

  The code output means 150 determines which one of the sound frames (reference frame and time-axis shift frame) corresponding to one reference frame is determined to be the optimum time-axis shift, and corresponds to the state of the sound frame. The function to output the code to The code determination parameter calculation unit 140 and the code output unit 150 constitute an encoding unit. The additional information extraction unit 160 extracts the binary array output by the code output unit 150 in units of Nh bits, refers to the code inverse conversion table, converts it to the registration rank of Nw bits, and further, a predetermined rule It has a function to extract as additional information having meaning after conversion by the above. The acoustic frame holding means 170 is a buffer memory capable of holding two consecutive reference frames (a total of four reference frames for each channel) for each of the A type and B type for each channel. Similarly to the conversion table creation means 80 shown in FIG. 1, the conversion table creation means 180 hums each other with Nh cyclic bit patterns for all 2 Nw power registration orders that can be taken by Nw bits. By assigning a Nh (> Nw) -bit Hamming code having a distance of at least 3 or more, it has a function of creating a code conversion table in which the registration rank of Nw bits and the Nh-bit Hamming code are associated with each other. Each component shown in FIG. 11 is actually realized by mounting a dedicated program on hardware such as a small computer having an information processing function and its peripheral devices. In particular, in order to more easily achieve the object of the present invention, it is desirable to use a portable terminal device such as a cellular phone as hardware. Note that in this specification, even a portable terminal device such as a cellular phone has an arithmetic processing function as a kind of computer.

(3.2. Processing operation of extraction device)
Next, the processing operation of the apparatus for extracting information from the acoustic signal shown in FIG. 11 will be described. When the extraction device is activated, first, the conversion table creation means 180 creates a code conversion table. The code conversion table is created by the conversion table creating unit 180 in the same manner as the conversion table creating unit 80, and the code conversion table as shown in FIG. 12 is obtained. In this code conversion table, a maximum of 16 types of cyclic codes are defined including a 16-bit code Hamming code and a cyclic bit pattern corresponding to each 8-bit reference code.

  Next, the processing operation of the extraction device after the code conversion table creation process will be described. The extraction apparatus according to the present invention performs error detection based on the code conversion table of FIG. 12, and all bit patterns not listed in this table are determined to be errors. The processing operation of the extraction device will be described with reference to the flowchart of FIG. First, in this apparatus, a time axis shift determination table S (p), a time axis shift determination log, a time axis shift determination flag, and a bit counter are initialized (S200). The time axis shift determination table S (p) is a table for determining a time axis shift, and p takes an integer value of 0 to 5. The initial value is set to S (p) = 0. The time axis shift determination log records the determined time axis shift, that is, the time axis shift value p for each set of one reference frame and five time axis shift frames, and is set to 0 in the initial state. Has been. The time axis shift confirmation flag is a flag indicating whether or not the time axis shift is confirmed, and is set to Off in the initial state. For the bit counter, 0 is set as an initial value.

  In this way, when the code conversion table is created and the initial value is set, if the user wants to know the attribute information such as the song name of the music that is playing, first, the extraction device On the other hand, an activation instruction as an extraction device is given. For example, this can be executed by operating a predetermined button when the extraction device is realized by a mobile terminal such as a mobile phone. When an instruction is input to the extraction device, the acoustic signal input unit 100 records the flowing music, digitizes it, and inputs it as a digital acoustic signal. More specifically, the audio input from the omnidirectional microphone (or one channel of the directional microphone) is digitized by the A / D converter.

  Subsequently, the reference frame acquisition unit 110 extracts an acoustic frame including a predetermined number of samples from the acoustic signal input from the acoustic signal input unit 100 as a reference frame (S201). Specifically, reference frames for A type and B type are extracted and read into the acoustic frame holding means 170. The number of samples of one acoustic frame read as the reference frame by the reference frame acquisition unit 110 needs to be the same as that set by the acoustic frame reading unit 10 shown in FIG. Therefore, in the present embodiment, the reference frame acquisition unit 110 sequentially reads 4096 samples for each of the A type and B type as reference frames. The acoustic frame holding unit 170 can store two reference frames of A type and B type for each channel, that is, 2.5N samples. When a new reference frame is read, the old reference frame is discarded. It is supposed to be. Therefore, the sound frame holding means 170 always stores four reference frames (continuous 10240 samples).

  The sound frame processed by the extraction device can be divided into a reference frame that is set adjacently without interruption from the beginning, and a time axis shift frame in which the reference frame and the time axis are shifted. As for the reference frame, when the A type sound frame and the B type sound frame overlap each other by 2048 samples, after setting sample number 1 to sample number 4096 as the first reference frame, the next reference frame is Sample number 2049 to sample number 6144, the next reference frame is set without interruption, such as sample number 4097 to sample number 8192, and the next reference frame is set to sample number 6145 to sample number 10240. Then, for each reference frame, five time axis shift frames moved by 1/6 frame (about 683 samples) are set. For example, for the first reference frame, five time-axis shift frames configured with 4096 samples starting from sample numbers 683, 1366, 2049, 2732, and 3413 are set. Subsequently, the frequency conversion unit 130 and the code determination parameter calculation unit 140 specify the time axis shift for each read sound frame, determine embedded information, and output a corresponding code (S202). The format of the information to be output has two formats of value 1 and value 2 corresponding to the case where the additional information is embedded.

  Here, details of time axis shift confirmation and sign determination in step S202 will be described with reference to the flowchart of FIG. First, it is confirmed whether the time axis shift confirmation flag is On or Off (S301). When the time axis shift confirmation flag is On, the time axis shift confirmation process (S303 to S309) is not performed, and only the sign determination process is performed (S302). However, since the time axis shift is not fixed in the initial state and the time axis shift determination flag is Off, the candidate code table B (p) is initialized (S303). Candidate code table B (p) includes a time base shift value of p = 0 to 5 that specifies one reference frame and five time axis shift frames, and binary values obtained from the states of these six sound frames. A code is recorded.

  Subsequently, the code determination parameter calculation unit 140 performs a code determination process (S302). Details of the code determination process are shown in FIG. First, the frequency conversion means 130 performs frequency conversion on each read sound frame to obtain each window spectrum (S401). Specifically, the processing according to the above [Equation 14] and [Equation 15] is executed, and the real part A (1, j, p), imaginary part B (1, j, p), real part A of the converted data is executed. (2, j, p) and imaginary part B (2, j, p) are obtained.

  Regarding the processing of S401, in practice, after frequency conversion is performed on the A type sound frame, the frequency conversion is performed on the B type sound frame after being shifted by N / 2 samples. For these conversion data A (2, j, p), B (2, j, p), A (2, j, p), B (2, j, p), the above [Expression 16] is followed. The process is executed to perform amplitude conversion (S402).

  By the processing in the frequency conversion means 130, a frame spectrum expressed by a spectrum that is a component corresponding to the frequency is obtained. Subsequently, after the code determination parameter calculation unit 140 calculates the code determination parameter C as described above, the code determination parameter C is used to determine what state the component of the frequency band to be changed is, That is, a process of determining what value is embedded as a 1-bit value is performed (S403). Specifically, the code determination parameter C is calculated by executing processing according to the above [Equation 17] to [Equation 19]. Candidate code B is set in candidate code table B (p).

  As a specific processing procedure of S403, first, the basic intensity values E (1, j, p) and E (2, j, p) are calculated using the above [Expression 17], and then the above [Expression 18]. ], E1 (p), E2 (p), E3 (p), E4 (p) are calculated, and the candidate code B and the code determination parameter C are calculated according to the above [Equation 19].

  As a result of the above determination, when either value 1 or value 2 is output to the candidate code table B (p) for the time-axis shift value p, the time-axis shift determination table is further calculated according to the following [Equation 20]. S (p) is updated (S404).

[Formula 20]
S (p) ← S (p) + C

  Returning to the flowchart of FIG. 14, the code determination parameter calculation unit 140 stores the code temporarily determined in the code determination process (S302) in the time axis shift value p in the candidate code table B (p) ( S304).

  Subsequently, it is determined whether or not the processing corresponding to all the time axis shift values p has been completed (S305). This determines whether all time-axis shift frames have been processed for a certain reference frame. In the present embodiment, since p takes a value from 0 to 5, if processing is not performed 6 times, a predetermined number of samples are shifted from the processed acoustic frame, and an acoustic frame having a different time axis shift is set. The process returns to S302 and is repeated. The case where p = 0 is a reference frame, and the case where p = 1 to 5 is a time axis shift frame. When the processing corresponding to all the time axis shift values p is completed, the time axis shift corresponding to the time axis shift value pmax that maximizes the value of the time axis shift determination table S (p) is the optimum time axis shift. And the code B (pmax) recorded in the candidate code table B (p) is output (S306).

  Subsequently, the time axis shift determination log is updated (S307). The time axis shift determination log is for recording the determined time axis shift, that is, the time axis shift value p for each set of one reference frame and five time axis shift frames. Then, with reference to the time axis shift determination log, it is determined whether or not the time axis shift has been the same for a predetermined number of times in the past (S308). In the present embodiment, this number is 10 times. If the time axis shift has been the same a predetermined number of times in the past, the time axis shift confirmation flag is set to On (S309). As a result, when the same time-axis shift continues for a predetermined number of times, the optimal time-axis shift is likely to be pmax. Therefore, the time-axis shift determination process (S303 to S309) is not performed, and the time-axis shift value p = The code determination process (S302) is performed only for pmax.

  Returning to the flowchart of FIG. When the time axis shift is confirmed and the code is output, 1 bit corresponding to the output code value is stored in the buffer (S203). Next, the bit counter is incremented by “1” (S204).

  Next, it is determined whether the bit counter is 15 or less or 16 or more (S205). If the bit counter is 15 or less, the process returns to S201 to perform processing for extracting the next A type and B type reference frames.

  If the bit counter is 16 or more, the 16-bit bit array stored in the buffer is decoded (S206). This is a process in which a 16-bit bit pattern is checked against all cyclic bit patterns corresponding to all the reference codes listed in the code conversion table of FIG. 12, and an 8-bit reference code corresponding to a completely matching bit pattern is obtained. Explore.

  A bit pattern that is the minimum hamming distance with the cyclic hamming code is searched, and it is determined whether the obtained minimum hamming distance is 0, 1 or more (S207). If it is determined in S207 that the minimum hamming distance is 1 or more (no bit pattern that is completely matched), the process returns to S201 to perform processing for extracting the next reference frame.

  If it is determined in S207 that the minimum Hamming distance is 0, the additional information extraction unit 160 outputs an 8-bit reference code (S208).

  By executing the processing shown in FIG. 13 for each reference frame, additional information is extracted. If it is determined in S201 that all reference frames have been extracted, the process ends.

  In the process of S208, the additional information extraction unit 160 first converts the value output by the code output unit 150 according to a predetermined rule and extracts it as meaningful additional information. As the predetermined rule, various rules can be adopted as long as the information intended by the person who embeds the information can be recognized by the recipient, but in this embodiment, the ASCII code is adopted. doing. That is, the additional information extraction unit 160 recognizes the bit value array obtained from the code determined by the code determination parameter calculation unit 140 and output from the code output unit 150 in units of 1 byte (8 bits), and recognizes this as ASCII. Recognizes character information according to the code. The character information thus obtained is displayed and output on a screen of a display device (not shown).

  Therefore, if the embedding device embeds the attribute information such as the song title or artist in the sound signal as the character information, the user wants to know the song title or artist by listening to the music being played. If a predetermined operation is performed on the mobile terminal that functions as the extraction device, attribute information such as a song title and an artist is displayed as character information on the screen of the mobile terminal.

(3.3. Time axis shift correction processing)
As described above, at the time of extraction, it is not always possible to read an acoustic signal corresponding to the acoustic frame embedded at the time of embedding. Therefore, the time axis of the acoustic frame is shifted and read in a plurality of ways (six in this embodiment), an optimum time axis shift is determined, and a code corresponding to the acoustic frame specified by the time axis shift is determined. Is to be output. For example, in the case of reading in six ways, the top acoustic frame is originally a sample of sample numbers 1 to 4096, but six pieces composed of 4096 samples starting from sample numbers 1, 683, 1366, 2049, 2732, and 3413 Are processed, and a code corresponding to the optimum acoustic frame is output. As described with reference to the flowchart of FIG. 14, in the present embodiment, when the same time axis shift continues for a predetermined number of times, the process is performed after that time axis shift is determined.

(4. Method of embedding information more reliably when the signal component is small)
As described so far, according to the present invention, information can be embedded even if the original signal component has a portion close to silence. Of course, this is sufficient, but in the present invention, it is possible to add a process for embedding information more reliably. Specifically, before performing the frequency conversion, weak white noise that is difficult to hear is superimposed on the acoustic signal.

  In this case, the acoustic frame reading means 10 shown in FIG. 1 has a function of adding or subtracting the same value in units of a predetermined number of samples within a predetermined amplitude range after reading a predetermined number of samples as one acoustic frame. Yes. In this embodiment, with respect to the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1), an amplitude range of −32 to +32 (Xl (i), Xr ( i) is a value between −32768 to +32767), and a uniform random number H is generated, and the same value H is used between K (K = 5 in the present embodiment) samples as follows: The processing as shown in FIG. 5 is executed to update the values of Xl (i) and Xr (i).

[Formula 21]
Xl (i) ← Xl (i) + H
Xr (i) ← Xr (i) -H

  That is, white noise is generated in one acoustic frame by executing the processing according to the above [Equation 21] over N samples. This white noise generation process is performed immediately after the acoustic frame extraction process of S103 and S104 in the flowchart of FIG.

  As described above, even when white noise is generated, the configuration of the extraction device for extracting information from the acoustic signal on the extraction side is the same as that in FIG. 11, and the processing operation is shown in the flowcharts of FIGS. It is the same as followed.

(5. Single sound content)
In the above-described embodiment, the case of double sound wave aggregation has been described as an example, but the present invention can naturally be applied to single sound wave condensation. In the case of a single tone distribution, information embedding and extraction processing can be realized by performing processing only on the upper frequency band of the change target frequency band in the above embodiment.

(6. Modified examples of frequency range)
In this embodiment, in order to improve the extraction accuracy, a method of expanding the embedding area in the entire telephone band is adopted. However, in order to improve reproduction quality, the width of the lower change target frequency band may be narrower than that in the above embodiment. Is possible. Specifically, the lower limit F1 of the change target frequency band is set to 0.85 kHz. In this case, the upper frequency band is 1.7 kHz to 3.4 kHz, and the lower frequency band is 0.85 kHz to 1.7 kHz. However, as a result of the experiment, there is no significant difference in quality compared to the method in which the embedded area is set over the entire telephone band, but rather the disadvantage of a decrease in extraction accuracy is greater, and as a result, superiority is seen. There wasn't. In addition, when setting the embedding area in the entire telephone band, the applicant narrows the lower frequency band to 0.85 kHz to 1.7 kHz instead of the method shown in the above embodiment, and further separates the frequency band below it. Experiments were also conducted on a method of installing 0.34 kHz to 0.85 kHz in triplicate, but as a result of comparison, it is more effective to suppress noise by combining the lower frequency bands into one of 0.34 kHz to 1.7 kHz, and the sensitivity is also high. It was found that it was magnified about twice. The bass part has a large signal energy component, so it seems that the sensitivity was significantly improved by a slight enlargement. In the end, the method that divides the frequency range into four frequency regions and makes the upper and lower frequency bands close to each other to form a double sound pulse concentration can suppress the noise most for human hearing. all right.

(7. Others)
As mentioned above, although it limited about the suitable embodiment of the present invention, the present invention is not limited to the above-mentioned embodiment, and various modifications are possible. For example, in the above embodiment, the number of samples of one acoustic frame is N = 4096, but N = 2048, 1024, 512, etc. may be set. As a result, the number of sound frames per same time is doubled, quadrupled, and quadrupled, and 2 to 8 times of information can be embedded as a whole.

  In the above-described embodiment, the case where the device for extracting information from the acoustic signal is realized by a single mobile terminal device such as a mobile phone has been described as an example. However, the device may be realized in cooperation with other computers. good. Specifically, the portable terminal device and the dedicated computer are connected so as to be capable of wireless communication, and among the components of the acoustic signal input unit 100 to the acoustic frame holding unit 170, those having a large calculation load are processed by the dedicated computer. For example, the portable terminal device includes the acoustic signal input unit 100, the reference frame acquisition unit 110, the time axis shift frame setting unit 120, the additional information extraction unit 160, and the conversion table creation unit 180, and the frequency conversion unit 130 and the code determination parameter calculation. Means 140, code output means 150, and sound frame holding means 170 are provided in a dedicated computer so that necessary information can be communicated between them. This makes it possible to perform high-speed processing even when the processing performance of the portable terminal device is low.

  In the above embodiment, a case where a two-channel stereo sound signal is used has been described as an example. However, a one-channel monaural sound signal may be used. In this case, the process performed on either the L channel or the R channel may be executed.

  Further, the change of the predetermined frequency component by the frequency component changing means 30 is executed according to FIG. 3 and [Equation 9], but various changes can be used for changing the frequency component according to the bit value to be embedded. For example, it is possible to use a technique as shown in Patent Document 1. In this case, on the extraction side, extraction is naturally performed by a method corresponding to the embedding method.

  The present invention relates to the protection of music copyright (monitoring illegal copying) in the field of package music for viewing for consumer and business use using CDs and DVDs, and the field of broadcasting and network music distribution distributed for commercial purposes by broadcasters and the like. ) And music attribute information (music title search service) field, museum, event information service service field linked to exhibition explanation narration, URL and other information extracted from audio signals of broadcast programs and CD / DVD packages In addition, it can be used for industries such as a non-contact Internet gateway service field in which a mobile phone is used to access a web site related to a predetermined content to extract detailed information or answer a questionnaire.

DESCRIPTION OF SYMBOLS 10 ... Sound frame reading means 20 ... Frequency conversion means 30 ... Frequency component change means 31 ... Phase correction means 40 ... Frequency reverse conversion means 50 ... Modified sound frame output means 60 ... Storage means 61: Acoustic signal storage section 62: Additional information storage section 63: Modified acoustic signal storage section 70: Bit array creation means 80: Conversion table creation means 100: Acoustic signal Input means 110: Reference frame acquisition means 120 ... Time axis shift frame setting means 130 ... Frequency conversion means 140 ... Code determination parameter calculation means 150 ... Code output means 160 ... Additional information extraction Means 170: Acoustic frame holding means 180 ... Conversion table creation means

Claims (13)

A device for embedding additional information in an inaudible state with respect to an acoustic signal composed of a time-series sample sequence,
An acoustic frame reading means for reading a predetermined number of samples from the acoustic signal;
Of the read acoustic frames, odd-numbered and even-numbered ones are A type and the other is B type, and frequency conversion is performed on the A type acoustic frames using a first window function, and the B type A frequency conversion is performed on the acoustic frame using a second window function, and a first window spectrum that is a complex frequency component corresponding to the first window function and a second that is a complex frequency component corresponding to the second window function. A frequency conversion means for obtaining a window spectrum;
Two spectrum sets that do not overlap each other in a predetermined frequency range are extracted from each of the generated window spectra, and the intensity of a total of four spectrum sets extracted from the first window spectrum and the second window spectrum is stored in the bit array to be embedded. Based on frequency component changing means for changing the intensity of each spectrum set so that the intensity of two spectrum sets is larger than the intensity of the other two spectrum sets on a predetermined basis,
A phase that corrects the direction of the complex frequency component of each frequency of the spectrum set that is changed to be smaller in the spectrum set in a direction opposite to the direction of the complex frequency component of each frequency of the spectrum set in the immediately preceding acoustic frame. Correction means;
Frequency inverse transform means for performing frequency inverse transform on each window spectrum including the phase-corrected spectrum set to generate a modified acoustic frame;
Modified acoustic frame output means for sequentially outputting the generated modified acoustic frames;
An information embedding device for an acoustic signal, comprising: A device for embedding additional information in an inaudible state with respect to an acoustic signal composed of a time-series sample sequence,
An acoustic frame reading means for reading a predetermined number of samples from the acoustic signal;
Of the read acoustic frames, odd-numbered and even-numbered ones are A type and the other is B type, and frequency conversion is performed on the A type acoustic frames using a first window function, and the B type A frequency conversion is performed on the acoustic frame using a second window function, and a first window spectrum that is a complex frequency component corresponding to the first window function and a second that is a complex frequency component corresponding to the second window function. A frequency conversion means for obtaining a window spectrum;
Four spectrum sets that do not overlap each other in a predetermined frequency range from each of the generated window spectra are set so that the frequency width of the two spectrum sets having the higher frequency is larger than the frequency width of the two spectrum sets having the lower frequency. The four spectral sets for each window spectrum were extracted from the second window spectrum, with the spectral sets extracted from the first window spectrum as 1D1, 1D2, 1U1, 1U2 in order of increasing frequency. When the spectrum set is 2D1, 2D2, 2U1, and 2U2 in ascending order of frequency, based on the bit arrangement to be embedded, the product of the intensity value of 1D1 and the intensity value of 2D2, the intensity value of 1D2, and the intensity of 2D1 One of the products with the value is larger than the other by a predetermined ratio, and at the same time, the intensity value of 1U1 and 2 Frequency component changing means for changing the intensity of each spectrum set such that one of the product of the intensity value of 2 and the product of the intensity value of 1U2 and the intensity value of 2U1 is greater than the other by a predetermined ratio. When,
A phase that corrects the direction of the complex frequency component of each frequency of the spectrum set that is changed to be smaller in the spectrum set in a direction opposite to the direction of the complex frequency component of each frequency of the spectrum set in the immediately preceding acoustic frame. Correction means;
Frequency inverse transform means for performing frequency inverse transform on each window spectrum including the phase-corrected spectrum set to generate a modified acoustic frame;
Modified acoustic frame output means for sequentially outputting the generated modified acoustic frames;
An information embedding device for an acoustic signal, comprising: In claim 2,
The frequency component changing means, based on the state of 1 bit in the bit array, a product of an intensity value of 1D1, an intensity value of 1U1, an intensity value of 2D2, and an intensity value of 2U2, and an intensity value of 1D2 and 1U2 The intensity of each spectrum set is changed so that any one of the product of the intensity value of 2D1, the intensity value of 2D1 and the intensity value of 2U1 is larger than the other by a predetermined ratio or more. An information embedding device for signals. In any one of Claims 2 to 3,
The frequency component changing means includes a product of an intensity value of 1D1 and an intensity value of 2D2, a product of an intensity value of 1D2 and an intensity value of 2D1, a product of an intensity value of 1U1 and an intensity value of 2U2, and 1U2. Of the product of the intensity value of 2 and the intensity value of 2U1, the spectrum set on the side to be changed to be larger is multiplied by the coefficient α, and the spectrum set on the side to be changed to be smaller. Is a device for embedding information in an acoustic signal, characterized in that the intensity is multiplied by a coefficient β smaller than the coefficient α. In claim 4,
The frequency component changing means has a value γ obtained by dividing the product of the intensity of the group to be changed so as to increase the product of the intensity by the product of the intensity of the group to be changed so as to reduce the product of the intensity. In the case where the coefficient is small, a coefficient α ′ obtained by dividing the coefficient α by a square root of γ and a coefficient β ′ obtained by multiplying the coefficient β by the square root of γ are used instead of the coefficients α and β. An information embedding device for an acoustic signal. In any one of Claims 1-5,
The phase correcting means has a direction in the range of 0.5 to 0.8 when the original position of the component vector to be corrected is 0 and the direction of the component vector in the immediately preceding acoustic frame is 1 as the opposite direction. A device for embedding information into an acoustic signal, wherein In any one of Claims 1-6,
The acoustic signal is a stereo signal of two or more channels;
The acoustic signal is characterized in that the acoustic frame reading means, the frequency converting means, the frequency component changing means, the frequency inverse converting means, and the modified acoustic frame output means execute processing for each channel. An information embedding device. In any one of Claims 1-7,
The additional information is composed of Nw bits as one word;
Nh (> Nw) bits for which the minimum hamming distance is at least 3 or more in Nh × Nh combinations of Nh cyclic Nh bit patterns with respect to all 2 Nw power registration orders that Nw bits can take Conversion table creation means for creating a code conversion table to which the Hamming code is assigned,
For each word in the additional information, referring to the code conversion table, a bit array creating means for creating a bit array of Nh bits composed of a corresponding Hamming code;
An apparatus for embedding information in an acoustic signal, further comprising: In claim 8,
The conversion table creating means is a 16-bit 16-bit combination of 16 cyclic 16-bit patterns with a minimum Hamming distance of 3 or more for all 128 registration orders that can take at least 7 bits. An apparatus for embedding information in an acoustic signal, which creates a code conversion table to which a Hamming code is assigned. In any one of Claims 1-9,
The acoustic frame reading means reads the A type acoustic frame and the B type acoustic frame by overlapping a predetermined number of samples,
The apparatus for embedding information in an acoustic signal, wherein the modified acoustic frame output means outputs the generated modified acoustic frame by connecting it with a preceding modified acoustic frame. In any one of Claims 1-10,
The apparatus for embedding information in an acoustic signal, wherein the frequency component changing means sets the predetermined frequency range as 0.34 kHz or more and 3.4 kHz or less. In any one of Claims 1-11,
The acoustic frame reading means generates white noise with extremely low amplitude that is difficult to hear during reproduction with the same number of samples as the acoustic frame, and adds the white noise to the acoustic frame. An information embedding device.   The program for functioning a computer as an information embedding apparatus with respect to the acoustic signal in any one of Claims 1-12.

Images