JP2007292827A - Acoustic signal retrieving apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an acoustic signal retrieving apparatus capable of accurately performing collating processing of a melody obtained by recording and a melody registered in a data base. <P>SOLUTION: For an acoustic signal obtained by recording, in a predetermined interval unit, a plurality of feature words in which a feature of the acoustic signal in the interval unit is expressed with a feature pattern and a weight value, are generated. The plurality of feature words are combined to make one set, and collating processing of the feature word within one set and a registered feature word registered in a related information data base (a) are performed from a heading feature word in one set, and a plurality of candidate records in which a non-match degree is lower than a predetermined degree are extracted (b), and related information corresponding to one candidate record which is narrowed down, is extracted. The non-matching degree is calculated by considering weighting on the number of non-matching bits of the feature patterns. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to the protection of music copyright (monitoring illegal copying) in the field of package music for viewing for consumer and business use using CDs and DVDs, and the field of broadcasting and network music distribution distributed for commercial purposes by broadcasters and the like. ) And the provision of music attribute information (music title search service).

As a service to provide music attribute information that allows you to know the titles of music that has been played recently, you can query the broadcast station for the date and time of the broadcast music, and record music fragments that are being played on mobile phones. Services that collate with melodies registered in the database have been put into practical use (see, for example, Patent Documents 1 and 2).
JP-T-2004-536348 JP-T-2004-537760 Japanese Patent Application No. 2006-49503

  However, in the above conventional method, in order to improve the accuracy of matching the melodies registered in the database and the recorded melody, the flowing melody must be recorded for a long time, and the melody and database obtained by recording are recorded. There is also a problem that the verification process with the melody registered in is also burdened.

  In order to solve this, the applicant of the present invention disclosed in Patent Document 3 described above that generates a plurality of 32-bit feature words obtained by performing spectral decomposition on an acoustic signal and accumulating spectral displacements between adjacent frames over a plurality of frames. We proposed a technique for searching for acoustic signals by collating the database with this feature word unit and narrowing down candidate records with a high degree of bit match.

  However, since the method described in Patent Document 3 does not reflect the absolute volume, the spectrum becomes unstable at a low volume, and the bit matching degree between the conforming record and the non-conforming record is increased. There is a problem that a remarkable difference is difficult to occur and erroneous determination is likely to occur.

  Therefore, an object of the present invention is to provide an acoustic signal search device that can more accurately perform a collation process between a melody obtained by recording and a melody registered in a database.

  In order to solve the above-described problem, in the first aspect of the present invention, information related to a recorded sound signal is obtained by collating the characteristics of the sound signal obtained by recording with the characteristics of the sound signal registered in the database. For each acoustic signal, a plurality of feature words expressing the feature pattern and weight value, a related information database in which the relevant information of the acoustic signal is registered, and a sound transmitted from the sound source An acoustic signal acquisition means for acquiring an acoustic signal, and a feature word for generating a feature word having a predetermined number of bytes expressing a feature pattern and a weight value of the acoustic signal in the section unit for the acoustic signal in a predetermined section unit A plurality of feature words and a plurality of feature words are made into one set, the feature words in the set and the registered feature words registered in the related information database. And a process of extracting a plurality of candidate records in which the degree of non-matching between the generated feature word and the registered feature word is lower than a predetermined value. There is provided an acoustic signal search device having database collation means for narrowing down to one candidate record and related information extraction means for extracting related information corresponding to the narrowed candidate record from the related information database.

  In the second aspect of the present invention, the feature word conversion means in the acoustic signal search device of the first aspect of the present invention generates a feature pattern having a predetermined number of bits based on an average spectrum of the predetermined section, and A weight value of a predetermined number of bits is generated based on the average volume of the section, and a feature word composed of the feature pattern and the weight value is generated.

  In the third aspect of the present invention, the feature word conversion means in the acoustic signal search device of the second aspect of the present invention calculates an average spectrum of the predetermined section, and 8K bands in the frequency direction within the predetermined frequency region. And binarization is performed by giving 0 or 1 to the spectral components of each band so that the number of 0's and 1's is uniform in the entire 8K bands, and the spectrum of the 8K bands Is converted to a K-byte characteristic word.

  In the fourth aspect of the present invention, the feature word conversion means in the acoustic signal search device of the third aspect of the present invention classifies 8K bands into a plurality of groups, and the number of 0s and 1s is equal in each group. Thus, binarization is performed by giving 0 or 1 to the spectral component of each band.

  According to a fifth aspect of the present invention, in the acoustic signal search device according to the second to fourth aspects of the present invention, the predetermined section is constituted by T acoustic frames, and the feature word conversion means Fourier transform is performed on each acoustic frame, and a difference value from adjacent acoustic frames is added over the T acoustic frames to calculate an average spectrum of the predetermined section. To do.

  According to a sixth aspect of the present invention, in the acoustic signal search device according to the fifth aspect of the present invention, each of the feature word conversion means performs a Fourier transform on each of the acoustic frames, and then shifts each unit by a predetermined number of frames. A plurality of feature words are generated by calculating an average spectrum at the position, and the database collating means extracts candidate records using the plurality of feature words.

  According to a seventh aspect of the present invention, in the acoustic signal search device according to the first aspect of the present invention, the database collating means is a characteristic word generated by the characteristic word generating means and a feature registered in the related information database. When comparing the registered feature word that is a pattern, the generated feature word and the feature pattern included in the registered feature word are compared to calculate the number of mismatch bits, and the generated feature word and the registered A weight value included in the feature word is compared to calculate a correction value obtained by dividing a large weight value by a small weight value, and a non-matching degree is calculated using a value obtained by multiplying the number of mismatch bits by the correction value, A candidate record is extracted based on the non-matching degree.

  According to an eighth aspect of the present invention, in the acoustic signal search device according to the first aspect of the present invention, the database collating means is a feature word generated by the feature word generating means and a feature registered in the related information database. When matching the registered feature word that is a pattern, the generated feature word and the feature pattern included in the registered feature word are compared in units of bits, and if the two bits do not match, the component is It is characterized in that a non-matching degree is calculated by performing a process of adding a weight value included in a feature word indicating largeness, and a candidate record is extracted based on the non-matching degree.

  According to a ninth aspect of the present invention, in the acoustic signal search device according to the first to eighth aspects of the present invention, the related information is address information for accessing a predetermined site on the network, and the address information is used. Network access means for accessing a predetermined site on the network.

  According to a tenth aspect of the present invention, in the acoustic signal search device according to the first to eighth aspects of the present invention, the feature word registered in the related information database is created for an acoustic signal in which additional information is embedded. The recorded acoustic signal is embedded with additional information, and further includes additional information extracting means for extracting additional information from the acoustic signal.

  According to the first aspect of the present invention, an acoustic signal is converted into a feature word having a predetermined number of bytes expressing a feature pattern and a weight value of the acoustic signal in a predetermined section unit. A set of feature words in the one set and the registered feature words registered in the related information database are collated from the first feature word in the one set, and the generated feature words are registered. Processed to extract a plurality of candidate records whose feature word non-matching degree is lower than a predetermined value, narrowed down to one candidate record, and extracted related information corresponding to the narrowed candidate records. There is an effect that it is possible to more accurately perform the matching process between the melody and the melody registered in the database.

  According to the second aspect of the present invention, the feature word expressing the characteristics of the acoustic signal is obtained by converting the feature pattern having a predetermined number of bits based on the average spectrum of the predetermined section and the predetermined bits based on the volume of the predetermined section. Since it is configured with a number of weight values, there is no need to remove the silent part from the object of collation, and it is possible to perform stable collation.

  According to the third aspect of the present invention, the feature word representing the feature of the acoustic signal is calculated as an average spectrum of a predetermined section, and the predetermined frequency region is divided into 8K bands in the frequency direction, thereby obtaining 8K pieces. The binarization is performed by giving 0 or 1 to the spectral components of each band so that the number of 0s and 1s in the entire band is equal, and the 8K band spectrum is used as a K-byte feature word. Since the feature word is a unit of multiples of 8 bits that can be easily processed by a computer, that is, a byte unit, there is an effect that collation can be performed at high speed.

  According to the fourth aspect of the present invention, in the generation of the feature word, the average spectrum is calculated and divided into 8K bands, and then the 8K bands are classified into a plurality of groups. Since the binarization is performed by giving 0 or 1 to the spectral components of the respective bands so that the number of bands is equal, the influence of the frequency characteristics due to passing through the microphone can be corrected. There is an effect that it becomes possible.

  According to the fifth aspect of the present invention, the feature word is generated on the basis of the sum of the spectral differences for a plurality of acoustic frames. Therefore, even for a portion where the signal level of the acoustic signal changes greatly. Thus, it is possible to generate a feature word that is not affected by the signal level.

  According to the sixth aspect of the present invention, after performing Fourier transform on each acoustic frame, a plurality of feature words are generated by calculating an average spectrum at each position shifted by a predetermined number of frames. Since the candidate records are extracted using the plurality of feature words, it is possible to perform a highly accurate search even when the recorded acoustic signal is misaligned.

  According to the seventh aspect of the present invention, when the feature word is collated, the generated feature word and the feature pattern included in the registered feature word are compared to calculate the number of mismatch bits, and the generated feature word Since the correction value based on the comparison of the weight values included in the registered feature word is calculated, and the non-matching degree is calculated using a value obtained by multiplying the correction value by the number of mismatch bits, especially the time for music, etc. When collating with acoustic data with a small general expansion and contraction, there is an effect that a highly accurate search becomes possible.

  According to the eighth aspect of the present invention, when the feature word is collated, the feature pattern included in the registered feature word and the generated feature word are compared bit by bit, and when the two bits do not match, Since the disagreement level is calculated by adding the weight value included in the feature word indicating that the component is large, acoustic data that has a large temporal expansion and contraction, such as natural sounds (animal calls, etc.) When collating with, there is an effect that a highly accurate search becomes possible.

  According to the ninth aspect of the present invention, as related information, address information for accessing a predetermined site on the network is registered, and the predetermined site on the network is accessed using the address information. As a result, recording and acquiring the sound that is playing has the effect of making it possible to obtain information on the corresponding site.

  According to the tenth aspect of the present invention, the feature word registered in the related information database is generated from the acoustic signal in which the additional information is embedded by the digital watermark technique, and the digital watermark is also applied to the reproduced acoustic signal. Since the additional information is embedded by the technique, the reproduced acoustic signal is recorded, and the embedded additional information is extracted, so that it is possible to perform a more accurate search together with the feature word. There is an effect.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(1. Preparation of related information database)
First, a feature word to be registered in the related information database is generated by a registered feature word generation device. FIG. 1 is a functional block diagram showing a configuration of a registered feature word generation device. In FIG. 1, 10 is an acoustic frame reading means, 20 is a feature word generation means, 30 is a feature word registration means, and 40 is a related information database.

  The sound frame reading means 10 has a function of sequentially reading a predetermined number of samples as sound frames from a digital sound signal recorded with a certain sound as a material. The feature word generation means 20 has a function of performing frequency analysis using the read sound frame and generating a feature word expressing the feature of the sound signal for the material. This feature word expresses a feature of a certain acoustic signal with a small amount of data, and is composed of a feature pattern that represents a spectrum feature and volume data. The feature word registering unit 30 associates the generated feature word with the related information related to the material represented by the original acoustic signal and the ID for specifying the material of the acoustic signal in the related information database 40 as a registered feature word. Has the function to register. Here, the material indicates music or sound recorded in the acoustic signal. Each component shown in FIG. 1 is actually realized by installing a dedicated program in hardware such as a computer and its peripheral devices. That is, the computer executes the contents of each means according to a dedicated program.

  Next, the processing operation of the registered feature word generation device shown in FIG. 1 will be described with reference to the flowchart of FIG. First, the registered feature word generation device reads a digital acoustic signal from an acoustic data file. This digital sound signal is obtained by sampling an analog sound signal by a technique such as PCM. In the registered feature word generation device, the acoustic frame reading means 10 reads a predetermined number of samples as one acoustic frame from the acoustic signal file. The number of samples of one sound frame read by the sound frame reading means 10 can be set as appropriate, but is desirably about 4096 samples when the sampling frequency is 44.1 kHz. This corresponds to about 0.092 seconds. However, in consideration of samples whose values decrease by using a Hanning window function in frequency conversion described later, the acoustic frame is read by overlapping a predetermined number of samples. In this embodiment, 2048 samples that are exactly half the section length of the acoustic frame are overlapped. Therefore, the first acoustic frame is sequentially read as samples 1 to 4096, the second acoustic frame is samples 2049 to 6144, and the third acoustic frame is samples 4097 to 8192.

  Subsequently, the frequency conversion means 20 performs frequency conversion on each read sound frame to obtain a frame spectrum that is a spectrum of the sound frame (S101). Specifically, frequency conversion is performed on the acoustic frame read by the acoustic frame reading means 10 using a window function. As frequency conversion, Fourier transform, wavelet transform, and other various known methods can be used. In the present embodiment, a case where Fourier transform is used will be described as an example.

  Here, a window function used for Fourier transform in the present embodiment will be described. In general, when performing Fourier transform on a predetermined signal, it is necessary to divide the signal into predetermined lengths. In this case, if Fourier transform is performed on a signal having a predetermined length, A pseudo component is generated. Therefore, in general, when performing Fourier transform, a signal value is changed using a window function called a Hanning window, and then Fourier transform is performed on the changed value.

  When performing the Fourier transform in S101, specifically, the Hanning window function W (i) is used for the value X (i) (i = 0,..., N−1) in the sample i, and the following [ The processing according to the first and second formulas of Formula 1] is performed to obtain the real part A (j) and the imaginary part B (j) at each frequency.

  Subsequently, a spectral component is calculated (S102). Specifically, processing according to the following [Formula 1] third formula is performed to obtain an intensity value E (j) at each frequency.

[Formula 1]
A (j) = Σ _{i = 0,..., N-1} W (i) · X (i) · cos (2πij / N)
B (j) = Σ _{i = 0,..., N-1} W (i) · X (i) · sin (2πij / N)
E (j) = A (j) ² + B (j) ²

  In [Expression 1], i is a serial number assigned to N samples in each acoustic frame, and takes an integer value of i = 0, 1, 2,... N−1. Further, j is a serial number assigned in order from the smallest value of the frequency value, and takes an integer value of j = 0, 1, 2,... N / 2-1. When the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by 10.8 Hz.

  Subsequently, thinning processing of spectral components is performed (S103). Although the spectral components up to around 22 kHz are obtained by the above frequency conversion, a characteristic word lower than that near 4 kHz is used for generation of the feature word in this embodiment. This is in order to correspond to a level that can be handled by voice recording of a mobile phone. Therefore, j = 385 to 2047 higher than the vicinity of 4 kHz among the frequency components of j = 0 to 2047 is not used. In addition, the low frequency components of j = 0 to 32 where it is difficult to extract sound features are not used. That is, in the present embodiment, j = 33 to 384 frequency components are used. Specifically, processing according to the following [Equation 2] is executed, and P (n) in units of 11 frequency components is thinned out.

[Formula 2]
P (0) = (E ₃₃ + E ₃₄ +... + E ₄₃ ) ^1/4
_{P (1) = (E 44} + E 45 + ... + E 54) 1/4
:
:
P (31) = (E ₃₇₄ + E ₃₇₅ +... + E ₃₈₄ ) ^1/4

  According to the above [Expression 2], 352 frequency components of j = 33 to 384 are thinned out to 32 frequency components of n = 0 to 31. The above process is performed for each acoustic frame, and 32 frequency components are obtained for each acoustic frame.

  Next, for each acoustic frame, a difference from the spectral component of the immediately preceding acoustic frame is calculated (S104). The processes of S101 to S103 are sequentially performed on each acoustic frame. This inter-frame processing in S104 uses P (0) to P (31) obtained as a result of performing the processing up to S103 for each acoustic frame. Specifically, the process according to the following [Equation 3] is performed to obtain the difference Dn (t).

[Formula 3]
Dn (t) = | Pn (t) −Pn (t−1) |, n = 0,..., 31

  Thus, the difference between adjacent acoustic frames is calculated by a feature word that reflects the characteristics of the acoustic signal without being affected by the magnitude of the signal level, even in places where the signal level of the acoustic signal changes greatly. It is for producing | generating.

  When the processing of S101 to S104 is sequentially performed on each acoustic frame and T differences (n in this embodiment) Dn (t) between the acoustic frames are obtained, the sum total of T is obtained (S106). ). That is, processing according to the following [Equation 4] is performed to obtain the sum Sn of the differences.

[Formula 4]
Sn = Σ _{t = 0,..., T-1} Dn (t)

  Subsequently, the binarization process of Sn obtained by the above [Equation 4] is performed (S107). Specifically, first, the Sn array is divided into two in the upper and lower bands of n ≧ 14 and n ≦ 13, and 1 is given to 7 of 14 large values of n ≦ 13, and 0 is given to 7 of the smaller values. In addition, 1 is given to 9 of 18 large values of n ≧ 14, and 0 is given to 9 of the small values. Here, instead of simply giving 1 and 0 to the 32 large Sn medium values of 16 and 16 small ones, the group of 18 bands having a high frequency and the band of 14 bands having a high frequency The reason why 1 and 0 are equally given to each group is to correct the influence of the frequency characteristics caused by passing through the microphone in the acoustic signal search apparatus described later. The reason why the upper and lower bands are divided at the positions of the 18 band and the 14 band is that, as a result of the experiment, the search accuracy is the highest when divided at this position. By the processing in S107, Sn for each n can be expressed by 1 bit. Then, a 32-bit feature pattern is obtained with n = 0 as LSB and n = 31 as MSB.

  Next, the volume data is calculated (S108). Specifically, the volume data Vol is calculated using the following [Formula 5].

[Formula 5]
_{Vol = [Σ t = 0,} ..., T-1 {Σ n = 0, ..., 31 Pn (t) 4} 1/4] · 64 / T

  As shown in [Formula 5] above, the values of all the thinned components Pn (t) are added for T acoustic frames. As a result, volume data Vol that is the average volume for the T acoustic frames is obtained. When the value of the volume data Vol does not fall within the range of 0 to 255, normalization is performed so as to fall within this range. Therefore, the volume data Vol is represented by 8 bits. Since the volume data Vol is calculated by the equation shown in [Formula 5], the volume data Vol is an accumulation of volumes over the T acoustic frames. Therefore, the volume data Vol is not the volume of each frame unit but the average volume of the T acoustic frames. It is expressing.

  The processes of S107 and S108 can be performed by changing the order. As a result of the processing in S107 and S108, a 40-bit feature word composed of a 32-bit feature pattern and 8-bit volume data is obtained.

  By executing the above processing for each acoustic frame, a large number of feature words for the acoustic signal are generated. For example, if the sampling frequency is 44.1 kHz, the sound frame is 4096 samples, and the sound frame is overlapped by 2048 samples as in the above example, one feature word is approximately 0.506 seconds, Will generate approximately 120 feature words.

  When a feature word is generated for each acoustic signal as described above, related information about the material of the acoustic signal, ID and feature word are associated with each acoustic signal and registered in the related information database 40. The related information may be any information as long as it is related to the material. For example, if the material is a song, the name of the song or the performer, and if the material is CM audio, the sponsor is used. A company name, URL, or the like can be used.

  Here, the feature word generation processing will be described with reference to the conceptual diagram of FIG. FIG. 3A is a diagram illustrating a waveform of an acoustic signal that is a generation target of a feature word. In the registered feature word generation apparatus, the acoustic signal is read in units of acoustic frames, but as shown in FIG. Then, each acoustic frame is divided into 32 bands as shown in FIG. This corresponds to S101 to S103. Next, as shown in FIG. 3D, the difference processing of the separated components and the total processing of the separated components are performed. The separation component difference process corresponds to S104, and the separation component summation process corresponds to the summation process of each frequency component (n = 0 to 31) in S108. Next, as shown in FIG. 3E, a summation process of 32-band difference components and a summation process of volume are performed. The summation process of the 32-band difference component corresponds to S106, and the volume summation process corresponds to the summation process of each acoustic frame (t = 0 to T-1) in S108. Next, as shown in FIG. 3F, binarization processing of the 32-band sum component and compression processing of the sound volume (processing for compressing the sound volume level in 256 steps based on the above [Equation 5]) are performed. The binarization process of the 32-band sum component corresponds to S107, and the volume compression process corresponds to S108. As shown in FIG. 3, a process of sequentially reading sound frames from the sound signal and generating one feature word in units of T sound frames is performed.

(2. Search for acoustic signals)
Next, the acoustic signal search device according to the present invention will be described. FIG. 4 is a block diagram of an acoustic signal search device according to the present invention. In FIG. 4, 50 is a microphone, 11 is an acoustic frame reading means, 21 is a feature word generating means, 60 is a feature word collating means, and 70 is a related information output means.

  The microphone 50 is a general-purpose microphone used in a mobile phone or the like. The microphone 50 does not need to have the performance of taking in low-frequency components and high-frequency components, as long as it has a frequency sensitivity corresponding to the above 400 Hz to slightly above 4 kHz (corresponding to j = 33 to 384). good. Similar to the sound frame reading means 10, the sound frame reading means 11 has a function of sequentially reading a predetermined number of samples as sound frames from a digital sound signal. However, the acoustic frame reading means 11 also has a function of digitizing the analog acoustic signal captured by the microphone 50. In the present embodiment, the microphone 50 and the acoustic frame reading means 11 serve as acoustic signal acquisition means for acquiring the sound transmitted from the sound source as an acoustic signal.

  Similar to the feature word generation unit 20, the feature word generation unit 21 has a function of performing frequency analysis using the read acoustic frame and generating a feature word expressing the characteristics of the acoustic signal for the material. Yes. However, as will be described later, more feature words are generated than the feature word generation means 20. The feature word collating unit 60 has a function of collating the generated feature word with the registered feature word registered in the related information database 40. The related information output means 70 has a function of extracting the related information about the material most similar to the feature of the acoustic signal obtained by recording as a result of the collation by the feature word collating means 60 from the related information database 40 and outputting it. is doing. Each component shown in FIG. 4 is actually realized by installing a dedicated program in a device including an arithmetic processing unit. As a preferred example, there is a case where a mobile phone having an arithmetic processing function executes the contents of each means according to a dedicated program.

  Next, the processing operation of the apparatus shown in FIG. 4 will be described. First, when the user wants to know related information about the sound that is flowing, the activation instruction is given to the acoustic signal search device. This can be executed by operating a predetermined button, for example, when the acoustic signal search device is realized by a mobile terminal such as a mobile phone. When an instruction is input, the sound signal search device takes in music flowing from the microphone 50, records it, and inputs it as a digital sound signal. More specifically, the voice input from the microphone 50 is digitized by an A / D converter.

  Next, the acoustic frame reading means 11 reads a predetermined number of samples as one acoustic frame from the acoustic signal captured from the microphone 50 and digitized. This processing is performed in the same manner as that performed by the registered feature word generation device. That is, the number of samples in one acoustic frame is 4096 samples when the sampling frequency is 44.1 kHz. In addition, the sound frame is read by overlapping 2048 samples.

  The processing from here to the generation of the feature word follows the flowchart of FIG. The flowchart of FIG. 5 is substantially the same as the flowchart of FIG. 2 for the registered feature word generation device. The feature word generation unit 21 performs frequency conversion on each read sound frame to obtain a frame spectrum that is a spectrum of the sound frame (S201). As with the registered feature word generation device, Fourier transform, wavelet transform, and other various known methods can be used as the frequency conversion. However, since it is necessary to match the processing of the feature word generation device, in this embodiment, the Fourier transform is performed. Use transformation.

  Further, in S202 to S208, after performing Sn according to the above [Formula 2] to [Formula 4] to obtain Sn, a 32-bit feature pattern is generated and the process according to the above [Formula 5] is performed. To generate 8-bit volume data Vol and obtain a 40-bit feature word. However, the acoustic signal search device generates five times as many feature words as the registered feature word generation device. In the acoustic signal search device, since a sound that is flowing is recorded to acquire an acoustic signal, a positional shift may occur depending on the recording start timing. Even in the method generated by the registered feature word generation device, since 11 acoustic frames are generated by averaging, they are relatively resistant to displacement. However, in the case of a music material with a sharp rhythm change, if it deviates by about 5 sound frames, which is almost half of 11 sound frames that are the generation unit of feature words, a significantly different feature word is generated and erroneous information is searched. End up. Therefore, a plurality of feature words are generated by delaying by two frames within the range of 11 acoustic frames, which is one analysis unit, and collated with registered feature words in the related information database 40.

  Specifically, in the registered feature word generation device, one feature word is generated from frame 1 to frame 11, but in the acoustic signal search device, a feature word is generated from frame 1 to frame 11, and for two frames. A feature word is also generated from a frame group shifted by 4 frames, 6 frames, and 8 frames. That is, feature words are also generated in frames 3 to 13, frames 5 to 15, frames 7 to 17, and frames 9 to 19.

  In this embodiment, the user is allowed to record for about 3 seconds. This is because if the length is too long, the user feels troublesome work. If the length is too short, there is a possibility that a feature word necessary for collation may not be generated. As described above, in the case of the present embodiment, one feature word is about 0.506 seconds. Therefore, six feature words are generated by recording for about 3 seconds. These six feature words are set as one set. However, as described above, since feature words shifted by 2, 4, 6, and 8 frames are generated, a total of 30 feature words are generated as one set.

  Subsequently, the feature word collating unit 60 collates the feature word generated by the feature word generating unit 21 with the feature word registered in the related information database 40. In the present invention, two methods can be used as a matching method. In any of the methods, bit matching between feature patterns constituting a feature word is performed, and the degree of non-match is calculated by using volume data as a weight for the result. Then, the search target is narrowed down using this non-matching degree. First, the first matching method will be described.

  An example of information registered in the related information database 40 is shown in FIG. As shown in FIG. 6A, in the related information database 40, a material ID that is an ID for specifying an acoustic material and a registered feature word are registered in association with each other. Actually, related information corresponding to the acoustic material is also registered, but is omitted in FIG. Each registered feature word is composed of a feature pattern and volume data as described above. Here, the feature pattern is represented by Sd, and the volume data is represented by Vd. Therefore, in the material A, the registered feature word A1 is represented by [Sd1, Vd1], the registered feature word A2 is represented by [Sd2, Vd2], and the registered feature word A3 is represented by [Sd3, Vd3]. Correspondingly, the feature pattern constituting the feature word generated by the feature word generation means 21 is expressed by Si and the volume data is expressed by Vi. Accordingly, the three feature words from the top are represented by [Si1, Vi1], [Si2, Vi2], and [Si3, Vi3].

  In the case of the first matching method, the feature word matching means 60 is first generated using the volume data Vi1 and Vd1 at the beginning of both to match the volume balance between the generated feature word and the registered feature word. The second and third volume data Vi2 and Vi3 of the feature word are corrected by the following [Equation 6].

[Formula 6]
Vi2´ ＝ Vi2 ・ Vd1 / Vi1
Vi3´ ＝ Vi3 ・ Vd1 / Vi1

  Subsequently, the generated feature word and the feature pattern of the registered feature word are collated in order from the top. Since the feature pattern of the generated feature word and the feature pattern of the registered feature word in the related information database 40 are both created according to the same rule and have a 32-bit configuration in which the low frequency component is LSB and the high frequency component is MSB, The collation can be performed depending on whether or not these bit values match. In particular, since it is an 8-bit multiple unit that is easy for a computer to process, high-speed collation is possible. Here, the number of mismatched bits between the feature patterns Si1 and Sd1 is D1, and hereinafter, D2 and D3 are similarly used. The obtained D1 to D3 are corrected by the following [Equation 7] using the relationship between the corrected volume data and the volume data of the registered feature word.

[Formula 7]
When Vi2´> Vd2, D2´ ＝ D2 ・ Vi2´ / Vd2
When Vi2´ <Vd2, D2´ ＝ D2 ・ Vd2 / Vi2´
When Vi2´ = Vd2, D2´ = D2
When Vi3´> Vd3, D3´ ＝ D3 ・ Vi3´ / Vd3
When Vi3´ <Vd3, D3´ ＝ D3 ・ Vd3 / Vi3´
When Vi3´ = Vd3, D3´ ＝ D3

  Then, D1 + D2 ′ + D3 ′ is calculated as the non-matching degree MA1 using the value obtained by the correction.

  The above processing is first performed for the first three feature words 01 to 03 of the shift 0 group and the registered feature words A1 to A3 for the material A registered in the related information database 40 to obtain the non-matching degree MA1. Next, the registered feature words are moved one by one, and the feature words 01 to 03 and the registered feature words A2 to A4 are collated to calculate the non-matching degree MA1. Similarly, the registered feature words are moved one by one and collated with the feature words 01 to 03, and the minimum non-matching degree MA1 is set to the non-matching in the case of the shift 0 for the material A. Degree MA1. In the example of FIG. 6B, this value is “15”.

  Subsequently, the three feature words 21 to 23 of the two shift groups are collated with the material A to obtain the non-matching degree MA1. In this way, the non-matching degree MA1 with the material A is obtained for the shift 4 group, the shift 6 group, and the shift 8 group. As a result, as shown in the first line of FIG. 6B, a non-matching degree MA1 with the material A corresponding to each shift amount is obtained.

  Similarly, the feature word is collated with registered feature words of other materials. As a result, as shown in the second and third lines in FIG. 6B, a non-matching degree MA1 with the materials B and C corresponding to each shift amount is obtained.

  If the non-matching degree MA1 is smaller than a predetermined threshold, only the record of the material in which the non-matching degree MA1 is recorded is extracted as a candidate record, and the fourth to sixth feature word collation targets are extracted. To do.

  Then, the narrowed material is collated with the latter feature words 04 to 06, and the material with the smallest non-matching degree MA1 is determined as the matching acoustic material. When the suitable acoustic material is determined, the related information output means 70 extracts the related information and outputs it to a display device or the like provided in the acoustic signal search device.

  Next, the second matching method will be described. In the second collation method, the feature pattern of the registered feature word is represented by Sd, the volume data is represented by Vd, the feature pattern of the feature word generated by the feature word generation means 21 is represented by Si, and the volume data is represented by Vi. To do. In the second matching method, the feature pattern Sd and the feature pattern Si are compared, and weighting is performed using the volume data Vd and Vi according to the number of mismatch bits. Specifically, when the non-matching degree is set to 0 as an initial state, when the bit on the Sd side is 1 and the bit on the Si side is 0, Vd is added to the non-matching degree MA2, and the bit on the Sd side Is 0 and the Si side bit is 1, Vi is added to the non-matching degree MA2. In this way, the non-matching degree MA2 is obtained by collating the 32 bits of the feature pattern.

  As the collation order, first, among the six feature words 01 to 06 of the shift 0 group, the first feature word 01 is extracted, and this feature word 01 and all the feature words registered in the related information database 40 are extracted. Is checked.

  Then, the value of the non-matching degree MA2 when the non-matching degree MA2 is minimized is recorded for each material. Similarly, the first feature word 21 to the feature word 81 of the shifted 2 group to shifted 8 group are also stored in the database. The number of bits when the non-matching degree MA2 is minimized is recorded for each material. As a result, a non-matching degree MA2 for each shifted group similar to that shown in FIG. 6B is obtained.

  If the non-matching degree MA2 is smaller than a predetermined threshold, only the record of the material in which the non-matching degree MA2 is recorded is extracted as a candidate record, and is used as a target for collation of the second feature word.

  Next, the second feature word 02 is extracted from the two shifted groups, and this feature word 02 is collated with all registered feature words for candidate records narrowed down by the first feature word. The non-matching degree MA2 when the non-matching degree MA2 is minimized is recorded for each material (candidate record). Similarly, the first feature word 22 to the feature word 82 of the shifted 2 group to shifted 8 group are recorded. Also, collation with the registered feature word of the narrowed material is performed, and the non-matching degree MA2 when the non-matching degree MA2 is minimized is recorded for each material unit. If the non-matching degree MA2 is smaller than a predetermined threshold value, only the record of the material in which the non-matching degree MA2 is recorded is extracted as a new candidate record, and is used as the third feature word collation target. .

  As described above, the sixth feature word 06 to feature word 86 are collated with the registered feature words of the narrowed material. However, when the material is narrowed down to one on the way, the material is determined as the compatible acoustic material. When the suitable acoustic material is determined, the related information output means 70 extracts the related information and outputs it to a display device or the like provided in the acoustic signal search device.

  Comparing the first verification method and the second verification method, in the first verification method, the non-matching degree MA1 increases when there is a positional deviation or a difference in amplitude change. This will be specifically described. When a displacement occurs in the acoustic frame, the feature pattern and volume data in the feature word are also affected by the displacement. Then, the number of mismatched bits D1 to D3 of the feature patterns Si1 and Sd1, Si2 and Sd2, and Si3 and Sd3 tends to increase, and at the same time, the ratio of Vd2 and Vi2 ′ and the ratio of Vd3 and Vi3 ′ tend to increase. As a result, D2 ′ and D3 ′ are also increased. In addition, if there is a difference in amplitude change, the values of the mismatched bit numbers D1 to D3 do not change, but the ratio of Vd2 to Vi2 ′ and the ratio of Vd3 to Vi3 ′ tend to increase, and D2 ′, D3 ′ Also grows.

  On the other hand, in the second collation method, the non-matching degree MA2 does not increase even if there is a positional deviation or a difference in amplitude change. As described above, the non-matching degree MA2 is obtained by adding one of the volume data Vd and Vi when there is a mismatching bit between the feature patterns Si and Sd. do not become. Even if the misalignment occurs, the mismatch bits between the feature patterns Si and Sd increase, and the difference between Vd and Vi increases, MA2 does not increase if the added Vd or Vi value itself is small. Even if the difference in amplitude changes and the difference between Vd and Vi increases, similarly, if the added Vd or Vi value itself is small, MA2 does not increase. On the contrary, if the added Vd or Vi value itself is large, MA2 becomes large, but MA2 becomes large except when the mismatch bits of the feature patterns Si and Sd are large, that is, when they do not match the registered feature word. In the case of matching with the registered feature word, the value of the non-matching degree MA2 is relatively less affected even if there is a positional shift or a difference in amplitude change.

  For this reason, the first matching method is suitable for matching with acoustic data such as music that has a small temporal expansion and contraction, and the second matching method is suitable for temporal analysis of natural sounds (such as animal calls). It is suitable for collation with acoustic data with large expansion and contraction.

  By recording the flowing sound as described above, information related to the flowing sound can be extracted. When the search device is realized by a mobile phone, related information can be displayed on the liquid crystal screen.

  In addition, in a mobile phone having a network connection function, it is possible to access another computer using a URL. However, as related information acquired by the apparatus of the present invention, a URL that is address information is recorded in a database. In addition, when the URL is acquired as the related information, if the network connection function is used, it is possible to access a site corresponding to the sound being played. Here, a functional block diagram of an acoustic signal search apparatus having a network connection function is shown in FIG. In FIG. 7, 80 is a network access unit, 100 is a network, and 200 is a site.

  The network access unit 80 has a function of accessing the site 200 corresponding to the address information via the network 100 using the address information extracted by the related information extraction unit 70 as the related information. The network access unit 80 is realized by a Web browser that acquires address information and accesses a site on the network. According to the apparatus shown in FIG. 7, for example, the characteristic word created from the lion's call and the URL of the site having the lion's information are recorded in the related information database 40 in association with each other. By simply recording the cry of the lion, the site 200 having the lion information can be accessed from the mobile phone.

(3. Combination with information embedding by digital watermark)
The acoustic signal search apparatus according to the present invention can improve the accuracy of related information extraction by combining with a method of embedding additional information in an acoustic signal as a digital watermark. Here, FIG. 8 shows a functional block diagram of the acoustic signal search apparatus in the case of combining digital watermark embedding and extraction. In FIG. 8, an acoustic signal reproduction device 500 and an embedded information extraction unit 90 are further added to those shown in FIG. 7. The acoustic signal reproducing device is for reproducing an acoustic signal in which additional information is embedded. For this, a commercially available music player capable of reproducing a digital sound signal can be employed. The embedded information extraction unit 90 has a function of extracting embedded additional information from the digital sound signal. The processing of the embedded information extraction unit 90 will be described later. According to the apparatus shown in FIG. 8, since the embedded information extraction unit extracts the additional information, the information obtained by collating the feature words by registering the additional information in the related information database 40 in advance. It becomes possible to perform a search further narrowed down. Hereinafter, a method of embedding additional information in an acoustic signal reproduced by the acoustic signal reproduction device 500 and extraction of additional information in the embedded information extraction unit 90 will be described.

(3.1. Embedding additional information in acoustic signals)
Next, embedding of additional information in an acoustic signal will be described. The acoustic signal in which the additional information is embedded is reproduced by the acoustic signal reproduction device 500. FIG. 7 is a functional block diagram showing a configuration of an information embedding device for an acoustic signal. In FIG. 7, 310 is an acoustic frame reading means, 320 is a frequency converting means, 330 is a low frequency component changing means, 340 is a frequency inverse converting means, 350 is a modified acoustic frame output means, 360 is a storing means, and 361 is an acoustic signal storage. 362 is an additional information storage unit, 363 is a modified acoustic signal storage unit, and 370 is an additional information reading means. Note that the apparatus shown in FIG. 7 can handle both a stereo sound signal and a monaural sound signal. Here, a case where processing is performed on a stereo sound signal will be described.

  The sound frame reading means 310 has a function of reading a predetermined number of samples as one frame from each channel of the original stereo sound signal to be embedded with additional information. The frequency conversion means 320 has a function of generating a frame spectrum by frequency-converting the frame of the acoustic signal read by the acoustic frame reading means 310 by Fourier transformation or the like. The low frequency component changing unit 330 extracts three sets of spectrum sets corresponding to three predetermined frequency ranges from the generated frame spectrum for the A type sound frame, and based on the additional information extracted from the additional information storage unit 62. The ratio (ratio) between the spectrum sets of the low frequency intensity data is changed, and for the B type sound frame, the low frequency intensity data in the predetermined frequency range of the generated frame spectrum is set to “0”. is doing. The frequency inverse transform unit 340 has a function of generating a modified acoustic frame by performing frequency inverse transform on a plurality of frame spectra including the changed low frequency intensity data. The modified sound frame output unit 350 has a function of sequentially outputting the generated modified sound frames. The storage means 360 includes an acoustic signal storage unit 361 that stores a stereo acoustic signal to be embedded with additional information, an additional information storage unit 62 that is configured as a bit array and stores additional information embedded in the stereo acoustic signal, and an additional information It has a modified acoustic signal storage unit 363 that stores the modified acoustic signal after information is embedded, and stores various information necessary for other processing. The additional information reading unit 370 has a function of extracting additional information from the additional information storage unit 62. Note that the additional information is information other than the acoustic signal itself to be embedded as an additional to the acoustic signal, and whether the content is address information for accessing a computer on the network, such as an IP address or a URL, Or it is ID for converting into address information. In this embodiment, a case where an ID is used will be described. Each component shown in FIG. 7 is actually realized by mounting a dedicated program on hardware such as a computer and its peripheral devices. That is, the computer executes the contents of each means according to a dedicated program.

  Next, the processing operation of the information embedding device for the acoustic signal shown in FIG. 7 will be described with reference to the flowchart of FIG. Here, a case where processing is performed on a stereo sound signal having two channels of L (left) and R (right) as sound signals will be described. FIG. 10 corresponds to processing for one word of additional information. One word can be set to an arbitrary number of bits, but is usually set to 1 byte. First, the additional information reading unit 370 reads additional information from the additional information storage unit 62 in units of one word (S301). Specifically, one word is read into a register in a computer used as an information embedding device for an acoustic signal. Subsequently, the mode is set to the separation mode (S302). There are three types of modes: separation mode, bit mode, and continuous identification mode. The delimiter mode indicates a mode for performing processing in a delimiter in units of one word, and the bit mode indicates a mode for performing processing based on the value of each bit of one word. When one word is read from the additional information storage unit 62, the delimiter mode is always set immediately after that. In the continuous identification mode, when an acoustic frame with a low low-frequency component signal level appears, if the delimiter information is embedded, it is a new one starting from the beginning or continued because it was interrupted. A mode for recording information for identifying whether or not there is present is shown.

  Subsequently, the acoustic frame reading means 310 reads a predetermined number of samples as one acoustic frame from each of the left and right channels of the stereo acoustic signal stored in the acoustic signal storage unit 361 (S304). The number of samples of one sound frame read by the sound frame reading means 310 can be set as appropriate, but is desirably about 4096 samples when the sampling frequency is 44.1 kHz. Therefore, the acoustic frame reading means 310 sequentially reads 4096 samples for each of the left channel and the right channel as acoustic frames. In the present invention, there are A type and B type as acoustic frames. In the A-type acoustic frame and the B-type acoustic frame, the next sample after the last sample of the preceding acoustic frame of the same type is set as the first sample. The A-type and B-type sound frames are set by overlapping a predetermined number (2048 in this embodiment) of samples. For example, if the A type acoustic frame is A1, A2, A3... From the top and the B type acoustic frame is B1, B2, B3... From the top, A1 is samples 1 to 4096, A2 is samples 4097 to 8192, A3. Is samples 8193-12288, B1 is samples 2049-6144, B2 is samples 6145-10240, and B3 is samples 10241-14336. Since the A type and the B type are relative, either one may be first. That is, contrary to the above, A1 is samples 2049 to 6144, A2 is samples 6145 to 10240, A3 is samples 10241 to 14336, B1 is samples 1 to 4096, B2 is samples 4097 to 8192, and B3 is samples 8193 to 12288. May be. In the example of FIG. 10, the case where the B type sound frame is set first is shown.

  Subsequently, the frequency conversion unit 320 performs frequency conversion on the read B-type sound frame to obtain a frame spectrum that is a spectrum of the sound frame (S305). Specifically, frequency conversion is performed on the B type acoustic frame read in S304 using a window function. As frequency conversion, Fourier transform, wavelet transform, and other various known methods can be used. In the present embodiment, a case where Fourier transform is used will be described as an example. In S309, which will be described later, similarly, frequency conversion is performed using the window function for the A type acoustic frame read in S308.

  Here, a window function used for Fourier transform in the present embodiment will be described. In general, when performing Fourier transform on a predetermined signal, it is necessary to divide the signal into predetermined lengths. In this case, if Fourier transform is performed on a signal having a predetermined length, A pseudo component is generated. Therefore, in general, when performing Fourier transform, a signal value is changed using a window function called a Hanning window, and then Fourier transform is performed on the changed value. In the present embodiment, for A type acoustic frames, window functions W (1, i), W (2, i) as shown in FIGS. 11B to 11D are used instead of the Hanning window function. ), W (3, i) is used for Fourier transform. This makes it easier for the extraction side to recognize the embedded information. The window function W (1, i) is for extracting the front part of the acoustic frame, and takes a maximum value 1 at the position of a predetermined sample number i in the front part as shown in FIG. In the rear part, the minimum value is set to 0. Which sample number takes the maximum value depends on the design of the window function W (1, i), but in this embodiment, it is defined by [Equation 8] described later. By multiplying the window function W (1, i), the signal waveform of the acoustic frame as shown in FIG. 11A has a signal component remaining at the front part as shown in FIG. The component is deleted, and this becomes a Fourier transform target.

  Further, the window function W (2, i) is for extracting the central portion of the acoustic frame. As shown in FIG. 11C, the maximum value is obtained at the position of the predetermined sample number i in the central portion. 1 is set, and the front and rear portions are set to have a minimum value of 0. Which sample number takes the maximum value varies depending on the design of the window function W (2, i), but in this embodiment, it is defined by [Equation 9] described later. By multiplying the window function W (2, i), the signal waveform of the acoustic frame as shown in FIG. 11A has a signal component remaining in the center as shown in FIG. The rear signal component is removed, and this is subjected to Fourier transform.

  The window function W (3, i) is for extracting the rear part of the acoustic frame. As shown in FIG. 11D, the window function W (3, i) takes a minimum value of 0 at the front part, and a predetermined sample at the rear part. The maximum value 1 is set at the position of the number i. Which sample number takes the maximum value depends on the design of the window function W (3, i), but in this embodiment, it is defined by [Equation 10] described later. By multiplying the window function W (3, i), the signal waveform of the acoustic frame as shown in FIG. 11A is removed from the front signal component as shown in FIG. The signal component remains and becomes a Fourier transform target. Thus, after extracting the front part, the central part, and the rear part, the Fourier transform is executed, so that spectra corresponding to the front part, the central part, and the rear part are obtained. In order to embed a bit value in one acoustic frame, the bit value is originally divided into two parts, a front part and a rear part. However, on the extraction side, it is not always possible to read a signal synchronously. Therefore, in order to clearly distinguish the front part from the rear part, in this embodiment, the signal component at the center part is always deleted at the time of embedding, and the front part and the rear part are separated in time (however, at the time of extraction) Only need to analyze the front and rear, and ignore the middle). The greatest feature of the window function used in the present embodiment is that the window function W (1, i) and the window function W (3, i) are asymmetrical. For this reason, in particular, bit inversion hardly occurs.

  In the present embodiment, the acoustic frames are read in duplicate, and the window functions W (1, i), W (2, i), W (3, i) are used for the odd frames (or even frames), For even frames (or odd frames), the window function W (4, i) as shown in FIG. 11 (e) is used.

  In the present embodiment, the sound frame is read in duplicate. That is, a predetermined number of samples are redundantly read in the odd-numbered sound frames and the even-numbered sound frames. As mentioned above, the window function used is different between odd frames and even frames, but because odd frames and even frames are simply the difference between odd and even, which one is processed for which? Also good. Therefore, in this specification, one of the odd-numbered frame and the even-numbered frame is referred to as an A-type frame, and the other is referred to as a B-type frame. According to the example of FIG. 10, odd frames are B type frames and even frames are A type frames. Conversely, even frames may be B type frames and odd frames may be A type frames.

  In the present embodiment, the window functions W (1, i) to W (4, i) are defined by the following [Equation 8] to [Equation 11]. In FIG. 11, the horizontal axis is the time axis (i). As described later, i is a serial number assigned to N samples in each acoustic frame, and is proportional to time t. 11A, 11F, 11G, 11H, and 11I, the vertical axis indicates the amplitude value (level) of the signal. 11B to 11E, the vertical axis indicates the values of the window functions W (1, i), W (2, i), W (3, i), and W (4, i). The maximum values of (1, i), W (2, i), W (3, i), and W (4, i) are all 1.

[Formula 8]
When i ≦ 3N / 8, W (1, i) = 0.5−0.5 cos (8πi / (3N))
When 3N / 8 <i ≦ N / 2, W (1, i) = 0.5−0.5 cos (8π (i−N / 4) / N)
When i> N / 2, W (1, i) = 0.0

[Formula 9]
When i ≦ 3N / 8, W (2, i) = 0.0
When 3N / 8 <i ≦ N / 2, W (2, i) = 0.5−0.5 cos (4π (i−N / 4) / N)
When i> 3N / 4, W (2, i) = 0.0

[Formula 10]
When i ≦ N / 2, W (3, i) = 0.0
When i> N / 2, W (3, i) = 0.5−0.5 cos (4π (i−N / 2) / N)

[Formula 11]
When i ≦ N / 4, W (4, i) = 0.0
When N / 4 <i ≦ N / 2, W (4, i) = 0.5−0.5 cos (4π (i−N / 4) / N)
When N / 2 <i ≦ 7N / 8, W (4, i) = 0.5−0.5 cos (8π (i−N / 8) / (3N))
When i> 7N / 8, W (4, i) = 0.0

  As is clear from FIG. 16 and [Formula 8] to [Formula 11], the window functions W (1, i) and W (3, i) are asymmetrical to each other. This is for facilitating identification between the two on the extraction side described later. The window functions W (1, i), W (2, i), and W (3, i) have a maximum value of 1 when i is a predetermined value, and i takes other values. , I is a window function that monotonically increases or decreases according to the value of i, and therefore, when the window functions W (1, i) and W (3, i) are determined, the window function W (2, i ) Is inevitably determined. For this reason, the window function W (2, i) has a left-right asymmetric shape.

  In this embodiment, since odd frames and even frames are read in duplicate by predetermined samples, after embedding information, when restoring to an acoustic signal, an odd frame multiplied by a window function and a window function are When overlapping samples of even frames multiplied are added, it must be returned to the original value. Therefore, the shape of the window function W (4, i) is inevitably determined according to the values of the window functions W (1, i), W (2, i), and W (3, i). That is, when the window functions W (1, i), W (2, i), W (3, i), and W (4, i) are added in the overlapping portion of the odd and even frames, the fixed value 1 for the entire section is obtained. Is defined to be The above is the outline of the window function used in this embodiment.

  Returning to S305, description will be made. When performing Fourier transform in S305, specifically, the window function W (4, 4) is applied to the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1). i) is used to perform processing according to the following [Equation 12] and correspond to the real part Al (4, j), the imaginary part Bl (4, j), and the right channel of the conversion data corresponding to the left channel. Real part Ar (4, j) and imaginary part Br (4, j) of the conversion data are obtained.

[Formula 12]
Al (4, j) = Σi _{= 0,..., N-1} W (4, i) · Xl (i) · cos (2πij / N)
Bl (4, j) = Σi _{= 0,..., N-1} W (4, i) .Xl (i) .sin (2πij / N)
Ar (4, j) = Σi _{= 0,..., N-1} W (4, i) .Xr (i) .cos (2πij / N)
Br (4, j) = Σi _{= 0,..., N-1} W (4, i) · Xr (i) · sin (2πij / N)

  In [Equation 12], i is a serial number assigned to N samples in each acoustic frame, and takes an integer value of i = 0, 1, 2,... N−1. Further, j is a serial number assigned in order from the smallest value of the frequency value, and takes an integer value of j = 0, 1, 2,... N−1 similarly to i. When the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by 10.8 Hz. Next, the low frequency component changing means 330 removes the window 4 components (low frequency spectrum components by the fourth window function) (S306). Specifically, the processing according to the following [Equation 13] is executed for the window 4 components.

  By executing the processing according to the above [Equation 12], a frame spectrum in which the signal component of each acoustic frame is expressed by a spectrum corresponding to the frequency is obtained. Subsequently, the low frequency component changing unit 330 extracts a spectrum set of three predetermined frequency ranges from the generated frame spectrum. Human hearing is known to be less sensitive to directionality for low frequency components up to about 200-300 Hz (Corona Corp., issued October 30, 1990, "Sound Engineering Course 1. Basic Acoustics"). Engineering, Acoustical Society of Japan ”p.247 (see FIGS. 9 and 26). Therefore, in this embodiment, the low frequency component is about 200 Hz or less. In the vicinity of a frequency of 200 Hz, j corresponds to 20, so the real part Al (4, j), the imaginary part Bl (4, j), the real part Ar (4, j) calculated by the above [Equation 12], Among the imaginary parts Br (4, j), those with j ≦ 20 (= M) are extracted.

[Formula 13]
For each component of j = 1 to M, Al ′ (4, j) = 0
Bl ′ (4, j) = 0
In the case of stereo, the following corresponding to the right signal is also calculated. E (4, j) = {Al (4, j) ² + B1 (4, j) ² + Ar (4, j) ² + Br (4, j) ² } ^{1 / 2}
Ar ′ (4, j) = Ar (4, j) · E (4, j) / {Ar (4, j) ² + Br (4, j) ² } ^1/2
Br ′ (4, j) = Br (4, j) · E (4, j) / {Ar (4, j) ² + Br (4, j) ² } ^1/2

  Next, the frequency inverse transform means 340 performs a process of obtaining a modified acoustic frame by performing frequency inverse transform on the frame spectrum from which the window 4 component has been removed by the process of S306 (S307). As a matter of course, the inverse frequency conversion needs to correspond to the method executed by the frequency conversion means 320 in S305. In the present embodiment, since the frequency transform unit 320 performs the Fourier transform, the frequency inverse transform unit 340 performs the Fourier inverse transform. Specifically, the real part Al ′ (4, j), the imaginary part Bl ′ (4, j) of the left channel of the spectrum obtained by any of the above [Equation 13], the real part Ar ′ ( 4, j) and imaginary part Br ′ (4, j) are used to perform processing according to the following [Equation 14] to calculate Xl ′ (i) and Xr ′ (i). For frequency components that are not modified in the above [Equation 13], in the following [Equation 14], Al ′ (4, j), Bl ′ (4, j), Ar ′ (4, j), Br The original values Al (4, j), Bl (4, j), Ar (4, j), and Br (4, j) are used as ′ (4, j).

[Formula 14]
Xl' (i) = 1 / N · {Σ j Al' (4, j) · cos (2πij / N) -Σ j Bl' (4, j) · sin (2πij / N)} + Xlp (i + N / 2 )
Xr' (i) = 1 / N · {Σ j Ar' (4, j) · cos (2πij / N) -Σ j Br' (4, j) · sin (2πij / N)} + Xrp (i + N / 2 )

  According to the above [Equation 14], each sample Xl ′ (i) of the left channel and each sample Xr ′ (i) of the right channel of the type A modified acoustic frame are obtained. The modified sound frame output unit 350 sequentially outputs the obtained modified sound frames to an output file. Subsequently, the acoustic frame reading means 310 reads a predetermined number of samples as A-type acoustic frames from the left and right channels of the stereo acoustic signal stored in the acoustic signal storage unit 361 (S308). As described above, the B type sound frame and the A type sound frame are overlapped by 2048 samples. Therefore, in S308, the acoustic frame reading means 310 moves the 2048 samples from the position where the B-type acoustic frame is read in S304, and reads the acoustic frame.

  Subsequently, the frequency conversion unit 320 performs frequency conversion on the read A-type sound frame to obtain a frame spectrum that is a spectrum of the sound frame (S309). Specifically, each acoustic frame is performed using three window functions of window functions W (1, i), W (2, i), and W (3, i). As the frequency conversion, Fourier transform, wavelet transform, and other various known methods can be used. In the present embodiment, Fourier transform is used as in the case of S305.

  When performing the Fourier transform in S309, specifically, three window functions W (1) for the left channel signal Xl (i) and the right channel signal Xr (i) (i = 0,..., N−1). , I), W (2, i), and W (3, i), the processing according to the following [Formula 15] is performed, and the real part Al (1, j) of the conversion data corresponding to the left channel , Al (2, j), Al (3, j), imaginary part Bl (1, j), Bl (2, j), Bl (3, j), real part Ar () of the conversion data corresponding to the right channel 1, j), Ar (2, j), Ar (3, j), imaginary part Br (1, j), Br (2, j), Br (3, j) are obtained. Note that the window functions W (1, i), W (2, i), and W (3, i) are functions whose values increase near the front (front), near the center, and near the rear of the acoustic frame, respectively. ing.

[Formula 15]
Al (1, j) = Σi _{= 0,..., N-1} W (1, i) .Xl (i) .cos (2πij / N)
Bl (1, j) = Σi _{= 0,..., N-1} W (1, i) · Xl (i) · sin (2πij / N)
Al (2, j) = Σi _{= 0,..., N-1} W (2, i) · Xl (i) · cos (2πij / N)
Bl (2, j) = Σi _{= 0,..., N-1} W (2, i) · Xl (i) · sin (2πij / N)
Al (3, j) = Σi _{= 0,..., N-1} W (3, i) · Xl (i) · cos (2πij / N)
Bl (3, j) = Σi _{= 0,..., N-1} W (3, i) .Xl (i) .sin (2πij / N)
Ar (1, j) = Σi _{= 0,..., N-1} W (1, i) .Xr (i) .cos (2πij / N)
Br (1, j) = Σ _{i = 0,..., N-1} W (1, i) · Xr (i) · sin (2πij / N)
Ar (2, j) = Σi _{= 0,..., N-1} W (2, i) .Xr (i) .cos (2πij / N)
Br (2, j) = Σi _{= 0,..., N-1} W (2, i) .Xr (i) .sin (2πij / N)
Ar (3, j) = Σi _{= 0,..., N-1} W (3, i) · Xr (i) · cos (2πij / N)
Br (3, j) = Σi _{= 0,..., N-1} W (3, i) · Xr (i) · sin (2πij / N)

  As in [Formula 12] above, in [Formula 15], i is a serial number assigned to N samples in each acoustic frame, and an integer value of i = 0, 1, 2,. Take. Further, j is a serial number assigned in order from the smallest value of the frequency value, and takes an integer value of j = 0, 1, 2,... N−1 similarly to i. When the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by 10.8 Hz.

  By executing the processing according to the above [Equation 15], a frame spectrum in which the signal component of each acoustic frame is represented by a spectrum corresponding to the frequency is obtained. Subsequently, the low frequency component changing unit 330 extracts a spectrum set of three predetermined frequency ranges from the generated frame spectrum. As described above, the human auditory sense makes it difficult to detect the directionality of the low frequency component up to about 200 to 300 Hz, so the low frequency component is set to about 200 Hz or less here as well. In the vicinity of a frequency of 200 Hz, j corresponds to 20, so the real part Al (1, j), Al (2, j), Al (3, j), and imaginary part Bl ( 1, j), Bl (2, j), Bl (3, j), real part Ar (1, j), Ar (2, j), Ar (3, j), imaginary part Br (1, j) , Br (2, j), Br (3, j), j ≦ 20 (= M) are extracted.

Subsequently, the low frequency component changing means 330 extracts the real part Al (1, j), Al (3, j), imaginary part Bl (1, j), Bl (3, j) of the extracted left channel, and the right channel. Using the real part Ar (1, j), Ar (3, j), imaginary part Br (1, j), Br (3, j), the following equation (16) gives the combined value E ₁ , to calculate the total value E _2.

[Formula 16]
E ₁ = Σj _{= 1,..., M-3} {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² }
E ₂ = Σ _{j = 1,..., M-3} {Al (3, j) ² + Bl (3, j) ² + Ar (3, j) ² + Br (3, j) ² }

E ₁ calculated by the above [Equation 16] represents the sum of the component intensities of the spectrum set near the front of the acoustic frame, and E ₂ represents the sum of the component intensities of the spectrum set near the rear of the acoustic frame. Subsequently, it is determined whether or not the combined values E ₁ and E ₂ are equal to or higher than the level lower limit value Lev. The level lower limit value Lev is set to 0.5 when the maximum amplitude value of the acoustic signals Xl (i) and Xr (i) is normalized to 1 and M = 20. This value of Lev = 0.5 is a level at which the resistance to analog conversion can be maintained empirically, and when there are few low frequency components, it will be reduced as appropriate. In this case, the detection accuracy also decreases due to analog conversion. Will do.

The reason why it is determined whether or not the combined values E ₁ and E ₂ are equal to or greater than the level lower limit value Lev is that if the signal strength is small, even if the signal is changed, the change cannot be detected on the extraction side. is there. In this embodiment, when the bit value that can take the first value (for example, “1”) and the second value (for example, “0”) is “1”, the window three components (low by the third window function). When the bit value is “0” in each component of the frequency spectrum, it is embedded in the window 1 component (each component of the low frequency spectrum by the first window function). Therefore, when the bit value to be embedded is “1”, the sum value E ₁ is less than the lower limit value Lev, and when the bit value to be embedded is “0”, the sum value E ₂ is less than the level lower limit value Lev. In order to process again from the first bit without recording according to the bit value of the access information, the reading position is returned to the first bit, and the mode is set to the delimiter mode (S310). On the other hand, when the bit value to be embedded is “1” and the total value E ₁ is greater than or equal to the level lower limit value Lev, or when the bit value to be embedded is “0” and the total value E ₂ is greater than or equal to the level lower limit value Lev, the mode is determined. It will be.

  The low frequency component changing unit 330 performs processing for equalizing the low frequency components of the window 1 component and the window 3 component (including the case where all are 0) in the left (L) channel signal when the mode is the separation mode. It performs (S312). Specifically, according to the following [Equation 17], processing for setting both L side to 0 is executed. In this case, the window 1 component and the window 3 component of the right (R) channel signal are not necessarily equal.

[Formula 17]
For j = 1 to M,
Al ′ (1, j) = 0
Bl ′ (1, j) = 0
Al ′ (3, j) = 0
Bl ′ (3, j) = 0
In the case of stereo, the following corresponding to the right signal is also calculated: E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^{1 / 2}
Ar ′ (1, j) = Ar (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Br ′ (1, j) = Br (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
E (3, j) = {Al (3, j) ² + B1 (3, j) ² + Ar (3, j) ² + Br (3, j) ² } ^1/2
Ar ′ (3, j) = Ar (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2
Br ′ (3, j) = Br (3, j) · E (3, j) / {Ar (3, j) ² + Br (1, j) ² } ^1/2

  By executing the processing according to the above [Equation 17], the low frequency component of the left channel frame spectrum is the same when both the window 1 component and the window 3 component are “0”. The pattern in which the window 1 component and the window 3 component are equal is information indicating the head position (separation) of the additional information. In the above [Equation 17], Al ′ (j) = Bl ′ (j) = 0 is set for both the window 1 component and the window 3 component. Therefore, if the value is sufficiently small, it is not always necessary to set it to zero. Further, the window 1 component and the window 3 component do not necessarily have to be the same, and it is sufficient that the difference is small. In this sense, the term “equal” is used here.

  On the other hand, when the mode is the bit mode or the continuous identification mode, the low frequency component changing unit 330 determines the window 1 component of the left channel signal according to the bit value of the bit array of the additional information extracted from the additional information storage unit 62. Processing for changing the ratio of the spectral intensity of the window 3 component to either the window 1 component being dominant or the window 3 component being dominant (S311). Here, “dominant” indicates that the spectral intensity in the spectrum set of one window component is larger than the spectral intensity in the spectrum set of the other window component. Therefore, in S311, the window 1 component is obtained by executing the processing according to any of the following [Equation 18] and [Equation 19] according to the bit value that can take the first value and the second value. The magnitude relationship between the spectral intensity of the window 3 and the spectral intensity of the window 3 component is changed, and processing is performed to change the window 1 component to dominant or the window 3 component to dominant. For example, when the first value is 1 and the second value is 0, when the bit value is 1, the process according to the following [Equation 18] is executed for the window 1 component.

[Formula 18]
For j = 1 to M, Al ′ (1, j) = 0
Bl ′ (1, j) = 0
In the case of stereo, the following corresponding to the right signal is also calculated: E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^{1 / 2}
Ar ′ (1, j) = Ar (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Br ′ (1, j) = Br (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2

  In this case, the processing according to the following [Equation 19] is executed for the three components of the window.

[Formula 19]
For three components j = M−2, M−1 and M, Al ′ (3, j) = 0
Bl ′ (3, j) = 0
In the case of stereo, the following corresponding to the right signal is also calculated. E (3, j) = {Al (3, j) ² + Bl (3, j) ² + Ar (3, j) ² + Br (3, j) ² } ^{1 / 2}
Ar ′ (3, j) = Ar (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2
Br ′ (3, j) = Br (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2
Further, in the case of stereo, processing for moving the right channel component to the left channel component is performed as follows in order to emphasize the remaining component. For j = 1 to M-3, Ar ′ (3, j) = 0
Br ′ (3, j) = 0
Al ′ (3, j) = Al (3, j) · E (3, j) / {Al (3, j) ² + Bl (3, j) ² } ^1/2
Bl ′ (3, j) = B1 (3, j) · E (3, j) / {Al (3, j) ² + Bl (3, j) ² } ^1/2

  As a result of performing the processing according to the above [Equation 18] and [Equation 19], the value becomes “0” when the three components of the window j = M−2, M−1, and M, but signals other than the predetermined value are obtained. Ingredients will be present. Therefore, in this case, the ratio of the spectral intensity is changed so that the window 3 component is dominant. Subsequently, when the bit value is 0, the processing according to the following [Equation 20] is executed for the window three components.

[Formula 20]
For each component of j = 1 to M, Al ′ (3, j) = 0
Bl ′ (3, j) = 0
In the case of stereo, the following corresponding to the right signal is also calculated. E (3, j) = {Al (3, j) ² + Bl (3, j) ² + Ar (3, j) ² + Br (3, j) ² } ^{1 / 2}
Ar ′ (3, j) = Ar (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2
Br ′ (3, j) = Br (3, j) · E (3, j) / {Ar (3, j) ² + Br (3, j) ² } ^1/2

  In this case, the processing according to the following [Equation 21] is executed for one component of the window.

[Formula 21]
For three components j = M−2, M−1 and M, Al ′ (1, j) = 0
Bl ′ (1, j) = 0
In the case of stereo, the following corresponding to the right signal is also calculated: E (1, j) = {Al (1, j) ² + Bl (1, j) ² + Ar (1, j) ² + Br (1, j) ² } ^{1 / 2}
Ar ′ (1, j) = Ar (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Br ′ (1, j) = Br (1, j) · E (1, j) / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Further, in the case of stereo, processing for moving the right channel component to the left channel component is performed as follows in order to emphasize the remaining component. For j = 1 to M-3, Ar ′ (1, j) = 0
Br ′ (1, j) = 0
Al ′ (1, j) = Al (1, j) · E (1, j) / {Al (1, j) ² + Bl (1, j) ² } ^1/2
Bl ′ (1, j) = Bl (1, j) · E (1, j) / {Al (1, j) ² + Bl (1, j) ² } ^1/2

  As a result of performing the processing according to the above [Equation 20] and [Equation 21], a value of “0” is obtained at j = M−2, M−1, and M of one component of the window, but signals other than the predetermined value are obtained at other times. Ingredients will be present. Therefore, in this case, the ratio of the spectrum intensity is changed so that the window 1 component is dominant.

  By executing the processing according to any one of [Equation 18] and [Equation 19], or [Equation 20] and [Equation 21], the left channel signal is generated in accordance with each bit value of the bit array of the additional information. Thus, the pattern is changed so that the window 1 component is dominant or the window 3 component is dominant. In S311, in the case of the continuation identification mode, if it is new, the distribution between the low frequency component window 1 component and the window 3 component is changed to a state in which the window 3 component is dominant according to [Equation 18], and is continued. In this case, the distribution of the low frequency component between the window 1 component and the window 3 component is changed to a state in which the window 1 component is dominant according to [Equation 19].

  In this case, there is always a portion where the signal component is “0” between the high frequency band and the low frequency band, thereby preventing the signal components of the high frequency band and the low frequency band from being mixed. Eventually, the low frequency component changing means 330 performs processing based on [Equation 17] in the separation mode in S312 and in the bit mode or the continuous identification mode, [Equation 18] [Equation 19] or [Equation 20] The processing based on Equation 21] is performed in S311.

  In either case of S311, S312, next, the low frequency component changing means 330 deletes the window two components (each component of the low frequency spectrum by the second window function) (S313). Specifically, the processing according to the following [Formula 22] is executed for the two components of the window.

[Formula 22]
For each component of j = 1 to M, Al ′ (2, j) = 0
Bl ′ (2, j) = 0
In the case of stereo, the following corresponding to the right signal is also calculated: E (2, j) = {Al (2, j) ² + Bl (2, j) ² + Ar (2, j) ² + Br (2, j) ² } ^{1 / 2}
Ar ′ (2, j) = Ar (2, j) · E (2, j) / {Ar (2, j) ² + Br (2, j) ² } ^1/2
Br ′ (2, j) = Br (2, j) · E (2, j) / {Ar (2, j) ² + Br (2, j) ² } ^1/2

  Next, the frequency inverse transform means 340 performs the process of obtaining the modified acoustic frame by performing the frequency inverse transform on the frame spectrum in which the ratio between the spectrum sets of each window component is changed by the processes of S311 to S313 (S314). As a matter of course, the inverse frequency conversion needs to correspond to the method executed by the frequency conversion means 320 in S305. In the present embodiment, since the inverse frequency Fourier transform is performed in the frequency transforming unit 320, the inverse frequency transforming unit 340 executes the inverse Fourier transform. Specifically, the real part Al ′ (1, j), etc., the imaginary part Bl ′ (1, j), etc. of the left channel of the spectrum obtained by any of the above [Equation 17] to [Equation 22], the right Using the real part Ar ′ (1, j) of the channel, the imaginary part Br ′ (1, j), etc., processing according to the following [Equation 23] is performed, and Xl ′ (i), Xr ′ (i ) Is calculated. For the frequency components that are not modified in the above [Equation 17] to [Equation 22], Al (1, j), which is the original frequency component, is used as Al ′ (1, j).

[Formula 23]
Xl' (i) = 1 / N · {Σ j Al' (1, j) · cos (2πij / N) -Σ j Bl' (1, j) · sin (2πij / N)} + 1 / N · { Σ _j Al ′ (2, j) · cos (2πij / N) −Σ _j Bl ′ (2, j) · sin (2πij / N)} + 1 / N · {Σ _j Al ′ (3, j) · cos (2πij / N) −Σ _j Bl ′ (3, j) · sin (2πij / N)} + Xlp (i + N / 2)
Xr' (i) = 1 / N · {Σ j Ar' (1, j) · cos (2πij / N) -Σ j Br' (1, j) · sin (2πij / N)} + 1 / N · { _{Σ j Ar' (2, j)} · cos (2πij / N) -Σ j Br' (2, j) · sin (2πij / N)} + 1 / N · {Σ j Ar' (3, j) · cos _{(2πij / N) -Σ j Br'} (3, j) · sin (2πij / N)} + Xrp (i + N / 2)

In the above [Equation 23], Σ _{j = 0,} ... _{, N−1} is shown as Σ _{j in} order to prevent the equation from becoming complicated. The terms “+ Xlp (i + N / 2)” in the first equation and “+ Xrp (i + N / 2)” in the second equation in the [Equation 23] are the data Xlp (i) of the modified acoustic frame modified immediately before, When Xrp (i) exists, the addition is performed in consideration of the overlap of N / 2 samples on the time axis. With the above [Equation 23], each sample Xl ′ (i) of the left channel and each sample Xr ′ (i) of the right channel of the A type modified acoustic frame are obtained. The modified sound frame output unit 350 sequentially outputs the obtained modified sound frames to an output file. When the processing for one acoustic frame is completed in this manner, the mode is determined (S316). If the mode is the separation mode, the mode is set to the continuous identification mode (S317), and then the acoustic frame reading means 310 is set to B A type acoustic frame is read (S304). On the other hand, when the mode is the bit mode or the continuous identification mode, after setting the mode to the bit mode (S318), the low frequency component changing unit 330 reads the next bit in the bit array of the access information (S303). The above processing is executed over all samples of both channels of the acoustic signal. That is, a predetermined number of samples are read as sound frames, and when there are no more sound frames to be read from the sound signal (S304), the process ends. When the processing corresponding to each bit of 1-word data read in S301 is completed, the process returns from S303 to S301, and the next word of the access information is read. When the processing is completed for all the words of the access information, the process returns to the first word of the access information. As a result, all modified acoustic frames that have been processed for all acoustic frames are recorded in the output file and obtained as modified acoustic signals. The obtained modified acoustic signal is output to and stored in the modified acoustic signal storage unit 363 in the storage unit 360.

The change of the left channel signal by the above processing will be described with reference to FIG. In FIG. 12, the horizontal direction in the drawing is a time axis and is proportional to the number of samples. In addition, a large number of rectangles in the figure indicate the window 1 component and the window 3 component of the modified acoustic frame. The horizontal width of the rectangle indicating the window component indicates the number of samples, and the vertical width indicates the intensity. However, in FIG. 12, neither the horizontal width nor the vertical width is accurately shown, and a strong signal is applied to the head portion corresponding to the window 1 component. It indicates whether there is a component and whether there is a strong signal component in the rear portion corresponding to the window 3 component. FIG. 12A shows a good signal when there is no acoustic frame in which the total values E ₁ and E ₂ calculated by the above [Equation 16] are less than the level lower limit value Lev, that is, for embedding additional information. The case is shown. FIG. 12 (b), the sum calculated by [Equation 16] E _1, if the E ₂ audio frames is less than the level lower limit value Lev present, i.e., to embed the additional information, poor signal The case is shown.

  For example, it is assumed that as additional information, a 2-word bit array in which the first word is “11011100” and the second word is “11000001” is embedded. First, at the beginning of each word, the window 1 component and the window 3 component are set to be equal as information indicating a break. This is obtained as a result of executing the processing according to the above [Equation 17] in S312 when the separation mode is set in S302. Subsequently, before the processing corresponding to each bit of the additional information is performed, information indicating whether the information is new or continued is recorded.

  In the present embodiment, even if there is an acoustic frame that is less than the level lower limit value Lev, since the bit processed at that time is valid and is continuously performed from there, whether the bit is new or continued. It is necessary to record some information. Therefore, after recording the information indicating the break, information indicating whether it is new or continued is recorded. Specifically, the mode determination is performed in the separation mode state (S316), the continuous identification mode is set (S317), and the process returns to the extraction of the B type sound frame without reading the bits of the additional information (S304). ). Then, after frequency conversion (S309), if it is new, the processing according to [Equation 18] makes the distribution between the window 1 component and the window 3 component, which are low frequency components, the window 3 component dominant. Change (S311).

  After recording the information indicating whether it is new or continuation in this way, the mode determination is performed in the continuation identification mode (S316), so that the bit mode is set (S318), and the first bit is read from the register ( S303), the B type sound frame is extracted (S304). In the example of FIG. 12A, since there is no acoustic frame that is less than the level lower limit value Lev, one word is continuously processed in S311. This is because the loop from S303 to S318 was repeated continuously 8 times (in the case of 1 word = 1 byte), and during that time, it was not passed through S310, S312, and S317 because it was less than the level lower limit value Lev. Is shown. As shown in FIG. 12, when the bit value of the additional information is 1, a low frequency component exists in the window 3 component, and when the bit value of the additional information is 0, the low frequency component exists in the window 1 component. . As can be seen from the above [Formula 18] to [Formula 21], in this case, the low frequency component of the other window component is zero.

  In the example of FIG. 12B, as a result of the processing according to the above [Equation 16], there is an acoustic frame that is less than the level lower limit value Lev. In this case, the above [Equation 17] passes through S310 and S312. As a result of executing the processing according to the above, the window 1 component and the window 3 component are set to be equal. In this case, since the separation mode is set in S310, information indicating whether it is new or continued is recorded via S317. In the example of FIG. 12B, when embedding “11011100” of the first word, an acoustic frame less than the level lower limit value Lev appears at the time when 1 bit of “1” of the first bit is first processed. Therefore, after the information indicating the break is recorded, the information indicating the continuation is recorded, and the processing is continued from “1” of the second bit. Then, since “1011” from the second bit to the fifth bit has been processed, an acoustic frame less than the level lower limit value Lev has appeared, so after recording the information indicating the break, the information indicating the continuation is recorded. Then, processing is continued from “1” of the sixth bit.

  In the example of FIG. 12, the case where one word of the additional information is 1 byte has been described. However, in order to record information indicating whether it is new or continued, one word of the additional information is recorded in an arbitrary number of bits. It is possible.

  Of the left channel of the modified acoustic signal obtained as described above, for the portion where the additional information is embedded, the low frequency component is equal to the window 1 component and the window 3 component, or the window 1 There are only three distributions: the component predominates or the window three component predominates. However, since the high frequency component remains the original acoustic signal, it has various distributions based on the setting of the producer. Further, as shown in the above example, when a stereo sound signal is used, the low frequency component changed in the left channel is apparent from the processing of [Expression 17] to [Expression 22]. In addition, it is always added to the low frequency component of the right channel. Therefore, since the right channel supplements the deleted component in the left channel, there is no signal degradation when viewed as both channels as a whole. Human auditory senses directionality with respect to high-frequency components, but it is difficult to sense directionality with respect to low-frequency components. Therefore, even if the low frequency component is biased to one side, it will be heard as if it is a normal acoustic signal for the listener.

  Therefore, the sound signal in which the additional information is embedded as described above is reproduced in the sound signal reproduction device 500, and the sound emitted as the sound from the speaker can be heard as the normal sound signal, but the sound is acquired. In the acoustic signal search device, additional information can be extracted.

(3.2 Details of Embedded Information Extraction Unit 90)
Next, details of the embedded information extraction unit 90 will be described. FIG. 13 is a diagram showing details of the embedded information extraction unit 90. In FIG. 13, reference numeral 410 is a reference frame acquisition unit, 420 is a phase change frame setting unit, 430 is a frequency conversion unit, 440 is a code determination parameter calculation unit, 450 is a code output unit, 460 is an additional information extraction unit, and 470 is an acoustic frame. Holding means.

The reference frame acquisition unit 410 has a function of reading an audio frame composed of a predetermined number of samples from the input digital monaural audio signal (or one channel of a stereo audio signal) as a reference frame. The phase change frame setting means 420 has a function of setting, as a phase change frame, an acoustic frame whose phase has been changed by moving the reference frame and predetermined samples. The frequency conversion means 430 has the same function as the frequency conversion means 320 shown in FIG. The code determination parameter calculation means 440 extracts each low frequency intensity data corresponding to a predetermined frequency or less from the generated frame spectrum, and adds a total value E _C1 of each low frequency intensity data for each of the window 1 component and the window 3 component. E _C2 is calculated based on the following [Equation 24], and the total value E _C1 and E _C2 are used as code determination parameters. Based on the ratio of the code determination parameters E _C1 and E _C2 , the predetermined state is obtained. Has the function to judge. The following [Equation 24] is obtained by deleting the right channel component in [Equation 16], and does not refer to the right channel component during extraction.

[Formula 24]
E _C1 = Σ _{j = 1,..., M-3} {Al (1, j) ² + Bl (1, j) ² }
E _C2 = Σ _{j = 1,..., M-3} {Al (3, j) ² + Bl (3, j) ² }

  The code output means 450 determines which one of the sound frames (reference frame and phase change frame) corresponding to one reference frame is determined to have the optimum phase, and sets a code corresponding to the state of the sound frame. It has a function to output. The additional information extraction unit 460 has a function of converting the ternary array, which is a set of codes output by the code output unit 450, according to a predetermined rule and extracting it as meaningful additional information. The acoustic frame holding means 470 is a buffer memory that can hold two consecutive reference frames.

Next, the processing operation of the embedded information extraction unit 90 shown in FIG. 13 will be described with reference to the flowchart of FIG. First, in this apparatus, the average code levels HL1 and HL2 and the phase determination table are initialized. These will be described. The average code levels HL1 and HL2 are low frequencies calculated by the above [Equation 24] for an acoustic frame (hereinafter referred to as an effective frame) that is determined to have embedded a binary value corresponding to a bit value. sum the average value of E _C1, E _C2 components, i.e., which is given by the average value of the sum E _C1, E _C2 in previous valid frame, the initial value is, the level limit is also used in the above embedding device The value Lev is set. The phase determination table S (p) is a table for determining the phase, and p takes an integer value of 0 to 5. The initial value is set to S (p) = 0.

  As described above, in a state where the initial value is set, the reference frame acquisition unit 410 extracts an acoustic frame including a predetermined number of samples as a reference frame from the digitized acoustic signal (S401). Specifically, the reference frame is extracted and read into the acoustic frame holding unit 470. The number of samples of one acoustic frame read as the reference frame by the reference frame acquisition unit 410 needs to be the same as that set by the acoustic frame reading unit 310 shown in FIG. Therefore, in this embodiment, the reference frame acquisition unit 410 sequentially reads 4096 samples as reference frames. The acoustic frame holding means 470 can store two reference frames as described above, and when a new reference frame is read, the old reference frame is discarded. Therefore, the sound frame holding means 470 always stores two reference frames (continuous 8192 samples).

  The acoustic frame processed by the embedded information extraction unit 90 can be divided into a reference frame that is set adjacently without interruption from the head, and a phase change frame in which the phase is changed. For the reference frame, after setting the first reference frame from sample number 1 to sample number 4096, the next reference frame is sample number 4097 to sample number 8192, and the next reference frame is sample number 8193 to sample number 12288. It is set without interruption. Then, for each reference frame, five phase change frames moved by 1/6 frame (about 683 samples) are set. For example, for the first reference frame, five phase change frames configured by 4096 samples starting from sample numbers 683, 1366, 2049, 2732, and 3413 are set.

  Subsequently, the frequency conversion unit 430 and the code determination parameter calculation unit 440 determine embedded information from each read sound frame and output a corresponding code (S402). The format of the information to be output is a binary format corresponding to the bit value on the embedding side and a ternary format of values input as delimiters.

  Details of the code determination process in step S402 will be described with reference to the flowchart of FIG. First, the frequency conversion means 430 performs frequency conversion on each read sound frame to obtain a frame spectrum (S501). This process is the same as the process in the frequency conversion unit 320 shown in FIG. However, since only the left channel is used for extraction, the processing according to [Equation 15] is performed, and the imaginary part Bl (1) such as the real part Al (1, j) of the converted data corresponding to the left channel is performed. , J) etc.

By the processing in the frequency conversion means 430, a frame spectrum expressed by a spectrum that is a component corresponding to the frequency is obtained. Subsequently, the code determination parameter calculation unit 440 calculates the average code levels HL1 and HL2 (S502). Specifically, v1 which is the integrated value E _C1 of the sound frames for which the past window 1 component is determined to be dominant is represented by the number of sound frames for which the past window 1 component is determined to be dominant. HL1 is calculated by dividing by a certain n1, and v2 that is an integrated value of the total value E _C2 for the sound frame for which the past window 3 component is determined to be dominant is determined to be the state where the past window 3 component is dominant. HL2 is calculated by dividing by n2, which is the number of sound frames that have been performed. Therefore, the average code levels HL1 and HL2 are average values of the sum values of the low-frequency intensity data of the acoustic frames that have been determined to have a dominant window component corresponding to the past.

Further, the code determination parameter calculation means 440 extracts each low frequency intensity data in a predetermined frequency range from the generated frame spectrum. The frequency range to be extracted needs to correspond to the embedding device. Therefore, here, low frequency intensity data having a frequency of about 200 Hz or less is extracted, and the real part Al (j) and imaginary part of the left channel calculated by the above [Equation 15] as in the case of the embedding device. Of Bl (j), those with j ≦ 20 are extracted. The code determination parameter calculating means 440, by executing the processing according to the above [Equation 24], the window 1 component of the sum E _C1, calculating a sum value E _C2 of the window 3 components. The embedded information extraction unit 90 uses this as a code determination parameter.

  Subsequently, the code determination parameter calculation unit 440 initializes the candidate code table (S503). The candidate code table records a phase number of 0 to 5 that specifies one reference frame and five phase change frames, and a ternary code obtained from the states of the six acoustic frames.

Subsequently, the code determination parameter calculating means 440 makes a determination window 1 component of the sum E _C1, window 3 components sum E _C2 of whether each is below a predetermined value (S504). Specifically, 1/20 of the average code levels HL1 and HL2 are set as predetermined values, respectively. When the sum value E _C1 is equal to or less than 1/20 of the average code level HL1 and the sum value E _C2 is equal to or less than 1/20 of the average code level HL2, the code determination parameter calculation unit 440 uses delimiter information. It is determined that there is (S508).

On the other hand, the code determination parameter calculation means 440 performs the comparison determination with the predetermined values of the calculated code determination parameters E _C1 and E _C2 and the mutual comparison determination according to the following [Equation 25] (S505), and the comparison result is obtained. Output the corresponding code.

[Formula 25]
When E _C2 > (predetermined value) and E _C2 / E _C1 > 2, the window 3 component is dominant. When E _C1 > (predetermined value) and E _C1 / E _C2 > 2, the window 1 component is dominant. Otherwise, both window components are equal

The code determination parameter calculation means 440 outputs a ternary code according to the determination result for each acoustic frame. That is, when it is determined that the window 3 component is dominant, the first bit value (for example, “1”) is output (S506), and when it is determined that the window 1 component is dominant, the second bit value is output. A bit value (for example, “0”) is output (S507), and when it is determined that both window components are equal, a code indicating delimiter information is output (S508). In step S505, if it is determined that the window 3 component is in a dominant state, E _C1 is HL1 or higher. If it is determined that the window 1 component is in a dominant state, is E _C2 higher than HL2. If these conditions are not satisfied, a code indicating delimiter information is output (S508).

  When it is determined that the window 3 component is dominant and the first bit value is output (S506), or when it is determined that the window 1 component is dominant and the second bit value is output (S507). Further, the phase determination table S (p) is updated according to the following [Equation 26] (S509).

[Formula 26]
When the three components of the window are dominant, S (p) ← S (p) + E _C1 / E _C2
When the window 1 component is dominant, S (p) ← S (p) + E _C2 / E _C1

Subsequently, the code determination parameter calculation unit 440 saves the candidate for the optimum phase in the candidate code table (S410). Specifically, the value of the phase number p that maximizes the value of S (p) recorded in the phase determination table, one of the three values determined in S506 to S508, and the sound frame The values of E _C1 and E _C2 corresponding to the low frequency component calculated by executing the process according to the above [Equation 16] are stored in the candidate code table as candidates for the optimum phase.

  Subsequently, it is determined whether or not the processing corresponding to all the phase numbers p has been completed (S411). This determines whether all phase change frames have been processed for a certain reference frame. In the present embodiment, since p takes a value from 0 to 5, if the processing is not performed six times, an acoustic frame having a different phase is set by shifting a predetermined number of samples from the acoustic frame that has been processed. Return and repeat the process. The case where p = 0 is a reference frame, and the case where p = 1 to 5 is a phase change frame. When the processing corresponding to all the phase numbers p is completed, it is determined that the phase corresponding to the phase number p recorded in the candidate storage table is the optimum phase, and the code recorded in the candidate storage table is output. (S412).

Returning to the flowchart of FIG. If the code corresponding to the bit value is output as a result of the process in S402, the average code level parameter is updated (S403). Specifically, the sum values E _C1 and E _C2 are added to the accumulated values v1 and v2 which are numerators when calculating the average code levels HL1 and HL2, respectively, and the accumulated values v1 and v2 are updated, and the number of frames serving as the denominator. The number of frames n1 and n2 is updated by adding 1 to n1 and n2, respectively. Subsequently, the mode is determined (S404). Two modes, a delimited mode and a bit output mode, are prepared. If it is in the bit output mode, the bit value is stored in the buffer (S409). Subsequently, the bit counter is counted up (S410). On the other hand, if the result of determination in S404 is that the mode is the separation mode, it is further determined whether the extracted code means new or continuation (S405). As a result, if it is new, it means that one word is completed immediately before, and the additional information extracting means 460 outputs the data for one word recorded in the buffer (S406). Then, the bit counter is initialized to 0 (S407). Further, the mode is set to the bit output mode (S408). If it is determined in S405 to continue, the value should be output to the bit in the buffer, so only the processing for setting the bit output mode is performed. If a code corresponding to the delimiter information is extracted in S402, the mode is set to the delimiter mode in order to extract new or continued information from the next sound frame (S411). By executing the processing shown in FIG. 14 for each reference frame, additional information is extracted. If it is determined in S401 that all the reference frames have been extracted, the process ends.

  In the process of S406, the additional information extraction unit 460 first delimits a code indicating that the window 3 component and the window 1 component are equal among the ternary codes output by the code determination parameter calculation unit 440. Then, the next code is the head, and a bit array is created by associating a code indicating that the window 3 component is dominant and the window 1 component is dominant with the bit value. Subsequently, this bit arrangement is converted according to a predetermined rule and extracted as meaningful additional information. As the predetermined rule, various rules can be applied as long as the information intended by the person who embeds the information can be recognized by the person who has received it. As a rule. That is, the additional information extraction unit 460 recognizes the code output from the code output unit 450 in units of 1 byte (8 bits) as determined by the code determination parameter calculation unit 440, and character information according to the set code system. To recognize the access information. In this embodiment, the URL is recognized.

  Therefore, if the embedded device embeds the URL of the web site related to the song name or artist of the song as character information in the acoustic signal, the user can hear the music flowing and listen to the song name or When you want to know detailed information related to the artist, if you perform a specified operation on your mobile terminal that functions as an extraction device, the URL of the web site related to the song or artist on your mobile terminal Can be acquired.

Here, advantages of using window function window functions W (1, i), W (2, i), W (3, i), and W (4, i) as shown in the present embodiment will be described. FIG. 16 is a diagram showing the state of low-frequency components immediately before and after embedding when a window function having no features of this embodiment is used, and FIG. It is a figure which shows the state of a frequency component. Comparing FIG. 16B and FIG. 17B showing the state of the low frequency component after embedding, the present embodiment has a value in a wider range in the A type acoustic frame than in the case where it is not so. I understand that. Therefore, the sum values E _C1 and E _C2 calculated using [Equation 24] in S502 are increased, and the probability that a bit value is extracted in S506 and S507 is increased.

  In the above processing, in order to accurately extract additional information in the embedded information extraction unit 90, processing for correcting the phase, processing for correcting the intensity balance between the window 3 component and the window 1 component, and an invalid frame The process which correct | amends the lower limit threshold value for judging this is performed. Next, supplementary explanation will be given for these three correction processes.

  As described above, at the time of extraction, it is not always possible to read an acoustic signal corresponding to the acoustic frame embedded at the time of embedding. Therefore, the phase of the acoustic frame is shifted and read in a plurality of ways (six in this embodiment), the optimum phase is determined, and a code corresponding to the acoustic frame specified by the phase is output. Yes. For example, in the case of reading in six ways, the top acoustic frame is originally a sample of sample numbers 1 to 4096, but six pieces composed of 4096 samples starting from sample numbers 1, 683, 1366, 2049, 2732, and 3413 Are processed, and a code corresponding to the optimum acoustic frame is output. This phase correction process is performed centering on the processes in S503, S509, S410, S411, and S412.

When the signal level is low, the magnitude of the intensity of the window component cannot be determined, and erroneous determination is often made on the extraction side. Therefore, it is determined that the frames whose sum values E _C1 and E _C2 are equal to or less than the predetermined threshold are invalid frames. The threshold at this time is set as the integrated value of the low frequency intensity for the past effective frames. Correction processing is performed using. By varying the threshold value in this way, it is possible to accurately determine whether the frame is invalid or valid even if the signal level varies. This lower limit threshold correction process is performed centering on the processes in S502 and S403.

  In the above embodiment, the case where both the embedding device and the embedding information extraction unit 90 embed additional information in the left channel signal of a stereo sound signal having left and right channels has been described as an example. Additional information may be embedded in. This is because the present invention is not related to the left and right characteristics. Further, when processing is performed on a monaural sound signal having only one channel, the processing performed on the left channel signal is performed in the above embodiment. Since the present invention embeds and extracts additional information from one channel signal, it can be similarly performed for a monaural sound signal or a stereo sound signal.

  FIG. 18 is a conceptual diagram in the case where additional information is embedded in the stereo sound signal and the monaural sound signal according to the present invention. FIG. 18A shows a stereo sound signal, and FIG. 18B shows a monaural sound signal. In the example of FIG. 18, the low frequency component for one acoustic frame is represented by a waveform, and the bit value “0” is embedded as an example.

  In the case of a stereo sound signal, embedding is performed on the left channel (L-ch) signal. As shown in FIG. 18A, after frequency conversion, signal separation is performed, and bit embedding processing is performed. Specifically, bit embedding is performed as a result of the processing of [Equation 17] to [Equation 22]. Here, as described above, when “0” is embedded, [Equation 20] and [Equation 21] are used. Therefore, after bit embedding processing, the signal component in the low frequency band is 0 (represented by no waveform in the figure) near the center of the acoustic frame (corresponding to window 2) and near the rear (corresponding to window 3). Become. At this time, as is clear from the contents of [Equation 20] and [Equation 21], the signal component from which the left channel signal is deleted is added to the right channel (R-ch) signal. Therefore, as shown in the lower part of FIG. 18A, the low frequency component of the right channel signal is increased. After bit embedding processing, signal synthesis including high frequency components is performed, and then frequency inverse conversion is performed to obtain a modified acoustic signal. On the other hand, as is apparent from the last three equations of [Equation 18], the right channel (R-ch) signal component corresponding to the remaining signal component of the left channel signal is added to the left channel signal. . Therefore, as shown in the upper part of FIG. 18A, the low frequency component corresponding to the window 1 of the left channel signal becomes large.

  In the case of a monaural sound signal, processing is performed as shown in FIG. 18B, but as can be seen from the upper part of FIG. 18A, processing similar to that for the left channel of a stereo sound signal is performed. Become.

  In the processing described so far, it is necessary that a signal component having a predetermined size or more exists in the window 1 component and the window 3 component, and both the window 1 component and the window 3 component are not more than a predetermined size. In this case, information cannot be embedded. Therefore, hereinafter, a method for enabling signal embedding even when the window 1 component and the window 3 component are both equal to or smaller than a predetermined size will be described.

  In this case, the information embedding process in the embedding apparatus shown in FIG. 9 is performed according to the flowchart of FIG. The flowchart of FIG. 19 differs from the flowchart of FIG. 10 in that the low frequency component changing unit 330 does not determine the level in the frequency conversion process in S709, and there is no setting process for the separation mode corresponding to S310. Is a point. This is because, in the processing according to FIG. 19, information is forcibly embedded even if the signal level is low. Therefore, it is determined whether there is a portion with a low signal level where information cannot be embedded. This is because it is not necessary to set the mode.

Therefore, as a process for setting one of the window 1 component and the window 3 component in S710 to be in an advantageous state, first, the fixed value V calculated according to the following [Equation 27] is used as the intensity of the low frequency component. It is set in place of the total values E ₁ and E ₂ .

[Formula 27]
V = {0.5 · Lev / (M−3)} ^1/2

  When the first value is 1 and the second value is 0, when the bit value is 1, the processing according to the above [Formula 18] and [Formula 19] is executed, and then the following [Formula 28 ] Is executed.

[Formula 28]
For the three components of the window, Al ′ (3, j) = Al (3, j) · V / {Ar (3, j) ² + Br (3, j) ² } ^1/2
Bl ′ (3, j) = Bl (3, j) · V / {Ar (3, j) ² + Br (3, j) ² } ^1/2

  When the bit value is 0, the process according to the above [Expression 20] and [Expression 21] is executed, and then the process according to the following [Expression 29] is executed.

[Formula 29]
For one component of window, Al ′ (1, j) = Al (1, j) · V / {Ar (1, j) ² + Br (1, j) ² } ^1/2
Bl ′ (1, j) = Bl (1, j) · V / {Ar (1, j) ² + Br (1, j) ² } ^1/2

  After performing the above processing in S710, the processing after the window two component deletion processing (S712) is performed in the same manner as the processing after S313 shown in FIG.

  As described above, even when information is embedded when the frequency component is small, the configuration of the extraction device for extracting information from the acoustic signal on the extraction side is the same as that of FIG. 13, and the processing operation is the flowchart of FIG. Is the same as

  FIG. 20 shows a conceptual diagram when embedding additional information when the signal component is small with respect to the stereo sound signal and the monaural sound signal. FIG. 20A shows a stereo sound signal, and FIG. 20B shows a monaural sound signal. In the example of FIG. 20, as in the case of FIG. 18, a low frequency component for one acoustic frame is represented by a waveform and a bit value “0” is embedded as an example.

  In FIG. 20, the difference from the case of FIG. 18 is that the original signal component is small. As in the example shown in FIG. 20 (a), at the stage after signal separation, even if the values of the window 1 component and the window 3 component are small, the bits are processed according to the processing of [Equation 27] to [Equation 29]. By embedding, the same signal component as in FIG. 18 is obtained.

  In the case of a monaural sound signal, processing is performed as shown in FIG. 20B, but as can be seen from the upper part of FIG. 20A, processing similar to that for the left channel of a stereo sound signal is performed. Become.

It is a block diagram of a registration characteristic word production | generation apparatus. It is a flowchart which shows the processing operation of the apparatus shown in FIG. It is a conceptual diagram of the production | generation process of a characteristic word. It is a block diagram of the acoustic signal search device according to the present invention. It is a flowchart which shows the processing operation of the apparatus shown in FIG. It is a figure which shows the information registered in the related information database 40, and the information which arises in a process. It is a figure which shows the case where the address information of a site is registered as related information. It is a figure which shows the example of the combination with embedding and extraction of the additional information by a digital watermark. It is a functional block diagram of an information embedding device for an acoustic signal. FIG. 10 is a flowchart showing an outline of processing of the apparatus shown in FIG. 9. FIG. It is a figure which shows the window function used by this invention. FIG. 11 shows how the low frequency component changes due to the processing according to FIG. 10. 4 is a functional block diagram showing details of an embedded information extraction unit 90. FIG. It is a flowchart which shows the process outline | summary of the apparatus shown in FIG. It is a flowchart which shows the detail of the code | symbol determination process of S402 of FIG. It is a conceptual diagram which shows the low frequency part of the acoustic signal before and behind embedding when not using the window function of this embodiment. It is a conceptual diagram which shows the low frequency part of the acoustic signal before and behind embedding by this embodiment. It is a conceptual diagram of the embedding process of the additional information by this invention. FIG. 10 is a flowchart showing an outline of processing in the apparatus shown in FIG. 9 when information can be embedded even if the original signal component is small. It is a conceptual diagram of an additional information embedding process when the original signal component is small.

Explanation of symbols

10, 11, 310, 410 ... sound frame reading means 20, 21 ... feature word generating means 30 ... feature word registering means 40 ... related information database 50 ... microphone 60 ... feature word Matching means 70: Related information output means 80 ... Network access section 90 ... Embedded information extraction section 100 ... Network 200 ... Site 320 ... Frequency conversion means 330 ... Low frequency component Change means 340... Frequency inverse transform means 350... Modified acoustic frame output means 360... Storage means 361... Acoustic signal storage section 362 .. additional information storage section 363. 370 ... additional information reading means 410 ... reference frame acquisition means 420 ... phase change frame setting means 430 ... round Wave number conversion means 440 ... code determination parameter calculation means 450 ... code output means 460 ... additional information extraction means 470 ... acoustic frame holding means 500 ... acoustic signal reproduction device

Claims (11)

A device that obtains information related to a recorded sound signal by collating the characteristics of the sound signal obtained by recording with the features of the sound signal registered in the database,
For each acoustic signal, a plurality of feature words expressing the feature pattern and weight value, a related information database in which the relevant information of the acoustic signal is registered,
Acoustic signal acquisition means for acquiring the sound transmitted from the sound source as an acoustic signal;
A feature word generating unit that generates a feature word of a predetermined number of bytes expressing a feature pattern and a weight value of the acoustic signal in the unit of the section in a unit of the section for the acoustic signal;
A plurality of the feature words are set as one set, and the process of collating the feature words in the set with the registered feature words registered in the related information database is sequentially performed from the first feature word in the one set. A database collation unit that sequentially performs a process of extracting a plurality of candidate records in which the degree of mismatch between the generated feature word and the registered feature word is lower than a predetermined value, and narrows down to one candidate record;
Related information extracting means for extracting related information corresponding to the narrowed candidate records from the related information database;
An acoustic signal search device characterized by comprising: In claim 1,
The feature word generation means generates a feature pattern of a predetermined number of bits based on an average spectrum of the predetermined section, generates a volume of the predetermined section as a weight value of a predetermined number of bits, and An acoustic signal search device for generating a feature word composed of weight values. In claim 2,
The feature word generation means calculates an average spectrum of the predetermined section as the feature pattern, divides the inside of the predetermined frequency region into 8K bands in the frequency direction, and 0 and 1 in the entire 8K bands. In order to equalize the number, binarization is performed by giving 0 or 1 to the spectral components of each band, and a feature word of K bytes is generated based on the spectrum of the 8K band. An acoustic signal search device characterized by the above. In claim 3,
The feature word generation means classifies the 8K bands into a plurality of groups, and sets 0 or 1 to the spectral components of each band so that the number of 0s and 1s is equal in each group. An acoustic signal search device characterized by binarization by giving. In claims 2 to 4,
The predetermined section is composed of T acoustic frames,
The feature word generation means performs a Fourier transform on each of the sound frames, and calculates a difference value from adjacent sound frames as an average spectrum of the predetermined section obtained by adding the difference values over the T sound frames. An acoustic signal retrieval device characterized by comprising: In claim 5,
The feature word conversion means generates a plurality of feature words by calculating an average spectrum at each position shifted by a predetermined number of frames after performing a Fourier transform on each acoustic frame,
The database collating means extracts a candidate record using the plurality of feature words, and an acoustic signal search device characterized in that In claim 1,
The database collating unit, when collating the feature word generated by the feature word generating unit with a registered feature word that is a feature pattern registered in the related information database, The feature pattern included in the registered feature word is compared to calculate the number of mismatch bits, and the generated feature word is compared with the weight value included in the registered feature word, and the large weight value is divided by the small weight value. A sound that calculates a correction value, calculates a non-matching degree using a value obtained by multiplying the number of non-matching bits by the correction value, and extracts candidate records based on the non-matching degree Signal search device. In claim 1,
The database collating unit, when collating the feature word generated by the feature word generating unit with a registered feature word that is a feature pattern registered in the related information database, The feature pattern included in the registered feature word is compared bit by bit, and if the two bits do not match, the weight value included in the feature word indicating that the component is large is added and the degree of non-match And a candidate record is extracted based on the non-matching degree. In claims 1 to 8,
The related information is address information for accessing a predetermined site on the network,
An acoustic signal search apparatus further comprising network access means for accessing a predetermined site on the network using the address information. In claims 1 to 8,
The feature word registered in the related information database is created for an acoustic signal in which additional information is embedded,
The recorded sound signal is embedded with additional information,
An acoustic signal search device further comprising additional information extraction means for extracting additional information from the acoustic signal. In claims 1 to 8,
An acoustic signal search apparatus characterized in that the acoustic signal acquisition means, the feature word generation means, the database collation means, and the related information extraction means are realized by a single mobile phone.

Images