JP5454161B2 - Acoustic data related information registration device and acoustic data related information search device - Google Patents

Description

  The present invention relates to an apparatus that registers related information related to acoustic data in which music data such as music is recorded in an acoustic database, and an apparatus that searches related information related to acoustic data from the acoustic database.

  As a service to provide music attribute information that allows you to know the titles of music that has been played recently, you can query the broadcast station for the date and time of the broadcast music, and record music fragments that are being played on mobile phones. Services that collate with melodies registered in the database have been put into practical use (see, for example, Patent Documents 1 and 2). On the other hand, the applicant also has characteristics of a predetermined section of an acoustic signal (including an analog acoustic signal recorded with a microphone. A digitized signal that can be processed by a computer or the like is hereinafter referred to as “acoustic data”). Is converted into a feature word of a predetermined number of bytes, a plurality of feature words are made into one set, a process of sequentially comparing the feature words in one set and the registered feature words registered in the database, A technique has been proposed in which a record having a high degree of matching is extracted to perform a collation process between a melody obtained by recording and a melody registered in a database at high speed (see Patent Document 3). At this time, in the matching process, it is determined whether or not the matching degree is high based on a uniform threshold value, and a single record having the smallest matching degree is extracted.

JP-T-2004-536348 JP-T-2004-537760 JP 2007-226071 A

  However, in the above conventional method, it is impossible in principle to perfectly align the acoustic signal (including the analog acoustic signal) used as a search key with the registered acoustic data, so there is a slight possibility. Occurrence of misalignment is inevitable, and due to the slight misregistration, search failure occurs when judgment is made with a uniform threshold value independent of individual songs, so let's search for all multiple songs that match the threshold value Then, excessive search occurs, and there is a problem that the target music cannot be extracted accurately.

  Therefore, the present invention provides a related information search device for acoustic data, which can search for related information related to the acoustic data registered in the acoustic database using acoustic data such as music without excess or deficiency. This is the issue.

  In order to solve the above-described problem, in the first aspect of the present invention, a feature word expressing the feature is created from the given original sound data, and related information related to the original sound data is stored in the sound database together with the feature word. A set of feature words expressing a characteristic pattern of original sound data based on information obtained by setting a unit section of a predetermined length for the original sound data and analyzing the unit section. A registered feature word generation unit that generates a feature word group as a registered feature word group, and a partial cutout of the original sound data to obtain a partial sound data. A feature word group, which is a set of feature words representing feature patterns of partial acoustic data, is set based on information obtained by setting a unit section of length and analyzing the unit section. A partial feature word generating means for generating a group of words, collating between the registered feature word group and the partial feature word group while shifting the temporal positional relationship between the registered feature word group and the partial feature word group, Is based on a minimum difference that is the degree of the minimum difference in the position condition that best fits, a determination threshold value calculation means for calculating an optimum determination threshold value, the registered feature word group and the optimum determination threshold value, Provided is a related information registration device for acoustic data having registration means for registering in the acoustic database in association with information for specifying the original acoustic data, and features of the given acoustic data and the original data registered in the acoustic database. A device that performs matching with the characteristics of the acoustic data and retrieves related information related to the given acoustic data from the acoustic database. For the data, a registered feature word group that is a set of registered feature words representing the feature pattern, an acoustic database in which the optimum determination threshold of the original sound data and related information related to the original sound data are registered, and A feature word group, which is a set of feature words representing a feature pattern of the search sound data in the section unit, is generated as a search feature word group in a predetermined section unit for the search acoustic data that is given acoustic data. The search feature word generation means, the search feature word group, and the registered feature word group registered in the acoustic database are collated with each other while shifting the temporal positional relationship, and the registered feature word group and the search The minimum difference degree, which is the degree of the minimum difference in the position condition that best matches the feature word group, is the registered feature word. And a feature word collating unit that selects related information of the original sound data corresponding to the registered feature word group as a candidate of related information for the search sound data when the threshold value is smaller than the optimum determination threshold value registered for each group. Provided is a related information retrieval apparatus for acoustic data.

  According to the first aspect of the present invention, on the basis of information obtained by analyzing original sound data for each unit section, a registered feature word group expressing the feature pattern is generated, and a part partially cut out from the original sound data A partial feature word group that expresses the feature pattern is generated from the acoustic data, and the matching is performed between the registered feature word group and the partial feature word group while shifting the temporal positional relationship between the two. Based on the minimum dissimilarity that gives the minimum value in the conditions, the optimum judgment threshold value is calculated and associated with the original sound data and registered in the database. A search feature word group expressing the feature pattern is generated from search acoustic data that can be stored, and the search feature word group and a registered feature word group registered in the database When the minimum dissimilarity that gives the minimum value in the position condition that best suits both is smaller than the judgment judgment threshold registered for each registered feature word Since the related information of the original sound data corresponding to the registered feature word group is selected as a candidate of the related information of the search sound data, the original information registered in the sound database using the search sound data such as music is used. The related information related to the sound data can be searched as the related information of the search sound data without excess or deficiency.

  Further, in the second aspect of the present invention, in the related information registration device for acoustic data according to the first aspect of the present invention, the partial feature word group is generated based on a plurality of partial acoustic data whose cut positions are different from each other, The determination threshold value calculation unit collates the registered feature word group and each partial feature word group in bit units while shifting the temporal positional relationship between the registered feature word group and each partial feature word group. As the minimum dissimilarity that gives the minimum value in the position condition that best matches, the minimum number of mismatch bits is obtained for each partial feature word group, and based on the average value of the minimum mismatch bits calculated for each partial feature word group And calculating an optimum determination threshold value. In the related information search apparatus for acoustic data according to the first aspect of the present invention, the feature word collating means includes the registration The minimum which gives the minimum value in the position condition where the registered feature word group and the search feature word group are best matched by shifting the temporal positional relationship between the collected word group and the search feature word group in bit units between them. As the degree of difference, the minimum number of mismatch bits is obtained.

  According to the second aspect of the present invention, in the related information registration apparatus for acoustic data, both of the registered feature word group and the plurality of partial feature word groups are collated in bit units while shifting the temporal positional relationship between the two. Is determined for each partial feature word group, and the optimum judgment threshold is determined based on the average value of the minimum mismatch bit numbers obtained for each partial feature word group. As a result, it is possible to obtain an optimum determination threshold value with high accuracy, and in the related information search device for acoustic data, the temporal positional relationship between the registered feature word group and the search feature word group can be determined. The two are collated with each other while shifting, and the minimum number of unmatched bits that is the minimum value in the position condition where the two are most suitable is obtained. Since as compared with registered optimal determination threshold to database, it is possible to perform the matching process with high accuracy.

In the third aspect of the present invention, in the related information registration apparatus for acoustic data according to the first aspect of the present invention, the feature pattern indicates intensity for each predetermined frequency range, and the feature word is added before Kitoku symptom patterns have the volume data, the partial characteristic word group the way cut each other is generated plurality of groups based on a plurality of different partial sound data, the determination threshold calculation means, said registered feature After shifting the temporal positional relationship between the word and each partial feature word, the feature pattern between the two is collated in bit units to obtain the number of mismatch bits, and then the weight based on the volume data of each feature word is set to the number of mismatch bits Is added or multiplied to calculate the number of weighted mismatch bits, and the minimum weighted mismatch bit number that is the minimum value in the position condition where the two match best is determined for each feature word group. And the optimum determination threshold value is calculated based on an average value for each partial feature word group of the minimum difference degree, and the related information of the acoustic data according to the first aspect of the present invention in search device, the characteristic pattern, which shows the intensity for each predetermined frequency range, the characteristic word is added before Kitoku symptom patterns have the volume data, the feature word collating means, said After shifting the temporal positional relationship between the registered feature word group and the search feature word group, the feature pattern is collated in bit units between the two to obtain the number of mismatch bits, and the weight based on the volume data of each feature word is The number of weighted mismatch bits is calculated by adding or multiplying the number of mismatch bits, and the minimum weight mismatch bit number that is the minimum value in the position condition in which both are most suitable is calculated as the maximum number. And obtaining a degree of difference.

  According to the third aspect of the present invention, the feature word has volume data, and the related information registration apparatus obtains the mismatch bit number between the feature patterns, and then adds or multiplies the weight based on the volume data to the mismatch bit number. Thus, the number of weighted mismatch bits is calculated, the minimum value is obtained as the minimum dissimilarity, and the average value for each partial feature word of the minimum dissimilarity is used as the optimum determination threshold value. It is possible to obtain a determination threshold value, and in a related information retrieval apparatus for acoustic data, after obtaining a minimum mismatch bit number between feature patterns, a weight based on volume data is added to or multiplied by a mismatch bit number. The number of weighted mismatch bits is calculated, and the minimum value is obtained as the minimum dissimilarity. Using this minimum dissimilarity, the optimum registered in the acoustic database is obtained. Since as compared with constant threshold, it is possible to perform collation processing in consideration of variations in the volume data.

  In the fourth aspect of the present invention, in the acoustic data related information registration apparatus according to any one of the first to third aspects of the present invention, the acoustic database has an optimum determination threshold range that does not overlap each other. The registration unit is configured by a plurality of divided acoustic databases, and the registration unit stores the registered feature word group, the optimum in the corresponding divided acoustic database according to the optimum determination threshold calculated by the determination threshold calculation unit. The determination threshold value and related information related to the original sound data are registered, and in the related information search device for sound data according to any one of the first to third aspects of the present invention, the sound database includes a plurality of sound databases. The feature word matching means is composed of a plurality of divided sound databases corresponding to the plurality of divided sound databases. When the comparison in parallel with respect to each of the divided acoustic database, characterized by selecting the relevant information of the original sound data corresponding to the registered feature word groups as a candidate of the relevant information of the search acoustic data.

  According to the fourth aspect of the present invention, a plurality of divided acoustic databases are prepared according to the range of the optimum determination threshold, and after calculating the optimum determination threshold in the related information registration device for acoustic data, The original sound corresponding to the registered feature word group is registered in different divided acoustic databases according to the calculated optimum determination threshold, and collated against each divided acoustic database in parallel in the acoustic data related information search device. Since the related information of the data is selected as a candidate for the related information of the search sound data, it can be distributed on different CPUs and computers by being arranged in different HDDs.

  Further, according to a fifth aspect of the present invention, in the feature word search device for acoustic data according to the fourth aspect of the present invention, the divided acoustic database is stored in an independent storage medium, and the feature word collating means It is characterized by comprising a plurality of programs having the same processing contents so as to be processed by independent processors corresponding to the divided acoustic database. According to the fifth aspect of the present invention, since the feature word matching means is provided independently corresponding to the divided acoustic database, when the computer has a multi-core configuration, the feature word matching means corresponding to the divided acoustic database is independent. If the OS is assigned to the thread and the search is executed, the OS distributes the feature word collating means to a plurality of storage devices and CPUs, so that the speed can be easily increased.

  Also, in the sixth aspect of the present invention, in the acoustic data related information registration apparatus according to any one of the first to fourth aspects of the present invention, when a plurality of the original sound data are stored in one folder, One by one is extracted from the folder, and each original sound data is sequentially processed by a registered feature word generating means, a partial feature word generating means, and a determination threshold value calculating means. A file name of data is registered as one of the related information. According to the sixth aspect of the present invention, when a plurality of original sound data are stored in one folder, the registration process is sequentially executed for all the original sound data in the folder. A batch registration process of a plurality of original sound data becomes possible.

  Further, in the seventh aspect of the present invention, in the related information search device for acoustic data according to any one of the first to fifth aspects of the present invention, when a plurality of the search acoustic data are stored in one folder, Each of the registered sound data selected by the feature word collating means is extracted one by one from the folder and sequentially processed by the search feature word generating means and the feature word collating means for each searched sound data. And the information output means for storing the extracted related information in a predetermined log file. According to the seventh aspect of the present invention, when a plurality of search sound data are stored in one folder, the search process is sequentially executed for all the search sound data in the folder. Batch search processing of a plurality of search sound data becomes possible.

  According to an eighth aspect of the present invention, in the related information registration device according to any one of the first to fourth and sixth aspects of the present invention, the partial feature word group and a registered feature word group registered in the acoustic database. And a feature word collating unit that extracts a record having the registered feature word group when the condition based on the optimum determination threshold registered for each registered feature word group is satisfied. The characteristic word generation unit, the determination threshold value calculation unit, and the registration unit perform processing only when there is no corresponding record as a result of the collation by the characteristic word collation unit. According to the eighth aspect of the present invention, the partial feature word group and the registered feature word group registered in the acoustic database are first collated, and as a result of the collation, only when there is no corresponding record, Since the registered feature word generation unit, the determination threshold value calculation unit, and the registration process are performed, it is regarded as the same as the original sound data that is the basis of the registered feature word that has already been registered (already registered) (Even if there is a difference in the data due to the difference in encoding format and compression method from that of the existing sound data, the original sound signal is considered to be the same). Can be prevented.

  Moreover, in the ninth aspect of the present invention, in the acoustic data related information search apparatus according to any one of the first to fifth and seventh aspects of the present invention, a unit section having a predetermined length with respect to the searched acoustic data. A registered feature word generating means for generating a set of feature words of a predetermined number of bytes expressing a feature pattern of the search acoustic data as a registered feature word group based on the information obtained by analyzing the unit section, and the search sound Partial sound data is obtained by partially cutting out the data, a unit section of a predetermined length is set for the partial sound data, and the partial sound data is based on information obtained by analyzing the unit section. A partial feature word generating means for generating a set of feature words of a predetermined number of bytes representing a feature pattern of the above as a partial feature word group, and temporal positions of the registered feature word group and the partial feature word group Judgment threshold value calculation means for performing bit-by-bit collation between the two while shifting the relationship, and calculating an optimum determination threshold value based on the minimum degree of difference that is the degree of the minimum difference in the position condition where the two are most suitable And registration means for registering the registered feature word group and the optimum determination threshold in an acoustic database in association with information for specifying the search acoustic data, the registered feature word generating means, and the partial feature The word generation unit, the determination threshold value calculation unit, and the registration unit perform processing only when a corresponding record does not exist as a result of collation by the feature word collation unit.

  According to the ninth aspect of the present invention, as a result of performing a search using the search sound data, when there is no relevant information of the corresponding search sound data, the search sound data is registered in the sound database. It is possible to automatically add a registered feature word group, optimum determination threshold value, and related information of unregistered search sound data.

  Advantageous Effects of Invention According to the present invention, there is an effect that it is possible to search related information related to acoustic data registered in an acoustic database without excess or deficiency using acoustic data such as music.

It is a block diagram of the related information registration apparatus of the acoustic data which concerns on this invention. It is a flowchart which shows the production | generation process of the characteristic word. It is a conceptual diagram of the generation process of the characteristic word. It is a flowchart which shows the optimal determination threshold value calculation process. It is a flowchart which shows the detail of the collation process of the partial area z in S120 of FIG. It is a flowchart which shows the detail of the calculation process of sum total mismatch bit number S (y, z, h, x) in S122 of FIG. It is a block diagram of the related information search apparatus of the acoustic data which concerns on this invention. It is a flowchart which shows the production | generation process of a search characteristic word. 10 is a flowchart of a search using a search feature word group by a feature word collating unit 70. It is a flowchart which shows the detail of the collation process of the record r in S220 of FIG. FIG. 11 is a flowchart showing details of a process of calculating a total mismatch bit number S (r, y, h, x) in S222 of FIG. 10. It is a block diagram which shows the modification of the related information registration apparatus of the acoustic data which concerns on this invention. It is a block diagram which shows the modification of the related information search apparatus of the acoustic data which concerns on this invention. It is a block diagram which shows the other modification of the related information registration apparatus of the acoustic data which concerns on this invention. It is a block diagram which shows the other modification of the related information search apparatus of the acoustic data which concerns on this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(1. Related information registration device for acoustic data)
First, a related information registration apparatus (hereinafter referred to as “related information registration apparatus”) for acoustic data according to the present invention will be described. The related information registration apparatus according to the present invention creates a registered feature word from acoustic data, and uses the registered feature word and the optimum determination threshold value created together with related information related to the acoustic data (generally called metadata) The data is registered in the acoustic database in association with information specifying the data (for example, acoustic data ID). This acoustic database is used for searching related information of acoustic data, and the acoustic data itself is not registered. The acoustic data is recorded in a digital format, such as music and voice, and is obtained by sampling an analog acoustic signal by a technique such as PCM, such as MP3 (MPEG-1 / Layer3) in advance. In general, an acoustic data file subjected to various lossy compression processes is provided. (According to copyright protection measures, license data is often not distributed for PCM format audio data with CD master disc quality in general.) FIG. 1 is a functional block diagram showing a configuration of a related information registration apparatus. In FIG. 1, 10 is a registered feature word generating means, 20 is a partial feature word generating means, 30 is a determination threshold value calculating means, 40 is a registration means, and 50 is an acoustic database.

  The registered feature word generation unit 10 sequentially reads a predetermined number of samples as acoustic frames from the acoustic data, performs frequency analysis using the read acoustic frames, and generates a feature word expressing the features of the acoustic data have. This feature word expresses features of certain acoustic data with a small amount of data, and is composed of a feature pattern representing the features of the spectrum and volume data. It is required that the acoustic data cannot be reproduced, that is, cannot be copied, and registration in the acoustic database is permitted. A feature word registered in the acoustic database 50 is particularly referred to as a “registered feature word”. In addition, the sound data on which the feature word is created is particularly referred to as “original sound data”. As this "original sound data", it is normal to use the "original" that has been modified to protect the copyright, not the original data itself owned by the copyright holders. It is. Of course, the data itself that is the “original” can also be used as the “original sound data”. As will be described later, in the present invention, other feature words such as a partial feature word and a search feature word appear. These are the same as the basic structure of the feature word, but the partial feature word. Unlike the registered feature word, the search feature word has a set of a plurality of feature words groups (five types in the embodiment of the present application) whose phases are shifted in order to correspond to collation with shifted phases in the search processing described later. There is a difference. In this specification, the minimum unit of 40-bit configuration is called a “feature word”, and a set of feature words used for collation is called a “feature word group”.

  Similar to the registered feature word generation unit 10, the partial feature word generation unit 20 sequentially reads a predetermined number of samples from the partial sound data as sound frames, performs frequency analysis using the read sound frames, and generates the partial sound data. It has a function of generating a feature word expressing the features of Both the registered feature word generation means 10 and the partial feature word generation means 20 are the same in that they read an acoustic frame from the same original sound data and create a feature word. The partial feature word generation means 20 performs processing on the whole (that is, from the beginning to the end), while the partial feature word generation means 20 has a plurality of short partial sections (same as the search feature words generated at the time of search). This is different in that a plurality of (five types in the present embodiment) partial sound data are generated by blindly cutting out and extracting a section of about 15 seconds and shifting the phase for each partial section. The number of partial sections to be extracted from one original sound data will be set in advance. For example, when extracting five pieces, the entire original sound data is 5 minutes long, If the length of the partial section is 15 seconds, the head (0 minutes) to 15 seconds, 1 minute 0 seconds to 1 minute 15 seconds, 2 minutes 0 seconds to 2 minutes 15 seconds, 3 minutes 0 seconds to 3 minutes 15 Five partial sections of seconds, 4 minutes to 4 minutes 15 seconds are extracted, and processing is performed on each partial acoustic data generated from the five partial sections.

  For the acoustic frame, the registered feature word generation unit 10 and the partial feature word generation unit 20 also acquire the same number of samples and perform a predetermined process to generate a feature word. The number of samples constituting the sound frame and the feature word generation method need to be the same in the registered feature word generation means 10 and the partial feature word generation means 20, but the partial feature words are the partial sounds in the same partial section. There is a difference in that a plurality of sets of partial feature word groups whose phases are shifted based on data are generated.

  Further, as described above, the registered feature word generation unit 10 and the partial feature word generation unit 20 generate a feature word by performing slightly different processing on the same number of sampled sound frames, but are generated as a minimum unit. The format of the 40-bit feature word is the same. However, as described above, the feature word created by the registered feature word generation unit 10 for the entire original sound data is “registered feature word”, and the feature word created by the partial feature word generation unit 20 for partial acoustic data is the target word. This is called “partial feature word”.

  The determination threshold value calculation means 30 performs collation between a single registered feature word group and a plurality of partial feature word groups. Compared to the former registered feature word group, each of the latter partial feature word groups Since the number of feature words is remarkably small, collation is performed while shifting the temporal position. Furthermore, since each partial feature word group is configured as a set of a plurality of types having different phases, it is necessary to collate each group individually. For example, a registered feature word group has a length of 5 minutes, there are five types of partial feature word groups corresponding to five types of 15-second partial sections, and each partial feature word group changes its phase to 5 If it is created as follows, using 5 × 5 15-second partial feature word groups, the registered feature word group with a length of 5 minutes and the time position overlap each other for 15/2 seconds, and the matching is performed by shifting. When performing, collation is performed with 5 × 5 × 300 × 2/15 = 1000 combinations. As a result of collation using these combinations, a value indicating the minimum difference is calculated for each of the five types of partial sections, and an optimum determination threshold value is calculated based on the average value thereof.

  The registration unit 40 uses the registered feature word group generated by the registered feature word generation unit 10 and the optimum determination threshold value calculated by the determination threshold value calculation unit 30 as the original original sound data production or copyright. Related related information (generally called metadata) and registered in the acoustic database 50 in association with an ID individually defined by a company that manages copyright information of the original acoustic data in order to identify the original acoustic data It has a function to do. Here, related information refers to text information that identifies the song, such as the song name and genre name, and the copyright owner and copyright owner who are involved in the production of the original acoustic data such as the lyrics, composer, arranger name, artist name, producer name, etc. Indicates text information about the name. However, the original sound data itself is not normally registered in the sound database 50 due to restrictions on the copyright law. Also, a series of binary-format material data (individual recording data before mixing down, MIDI input data) used for production and mastering of the acoustic data is normally registered due to copyright law restrictions. Absent. Each component shown in FIG. 1 is actually realized by incorporating a dedicated program into hardware such as a computer and its peripheral devices. That is, the computer executes the contents of each means according to a dedicated program.

(1.2. Processing operation of related information registration device)
Next, the processing operation of the related information registration apparatus shown in FIG. 1 will be described. First, in the related information registration device, the registered feature word generation means 10 generates a registered feature word from the designated original sound data. FIG. 2 is a flowchart showing a registration feature word generation process. First, the registered feature word generation means 10 reads the original sound data. In the related information registration device, the registered feature word generation means 10 reads a predetermined number of samples as one acoustic frame from the designated original acoustic data. The number of samples of one acoustic frame read by the registered feature word generation unit 10 can be set as appropriate, but is desirably about 4096 samples when the sampling frequency is 44.1 kHz. This corresponds to about 0.092 seconds. However, in consideration of the continuity between adjacent windows by using a Hanning window function in frequency conversion, which will be described later, the acoustic frame is read by overlapping a predetermined number of samples. In this embodiment, 2048 samples that are exactly half the section length of the acoustic frame are overlapped. Therefore, the first acoustic frame is sequentially read as samples 1 to 4096, the second acoustic frame is samples 2049 to 6144, and the third acoustic frame is samples 4097 to 8192.

  Subsequently, the registered feature word generation unit 10 performs frequency conversion on each read sound frame to obtain a frame spectrum that is a spectrum of the sound frame (S11). Specifically, frequency conversion is performed on the acoustic frame read by the registered feature word generation means 10 using a window function. As frequency conversion, Fourier transform, wavelet transform, and other various known methods can be used. In the present embodiment, a case where Fourier transform is used will be described as an example.

  Here, a window function used for Fourier transform in the present embodiment will be described. In general, when Fourier transform is performed on a predetermined signal, it is necessary to divide the signal into predetermined lengths. In this case, the Fourier transform (precisely, for a short time) is performed on a signal having a predetermined length. When called (Fourier transform), a pseudo component is generated in the high frequency region. Therefore, in general, when performing Fourier transform, the signal value is changed so that the weight is dropped by the cosine wave shape at the boundary of the window using a window function called a Hanning window, and then the value after the change is changed. To perform the Fourier transform.

  When the Fourier transform is performed in S11, specifically, the value X (i) (i = 0,..., N−1) in the sample i has a real value of 0 to 1 and is defined in the N sample section. Using the Hanning window function W (i) (= 0.5−0.5 cos (2πi / N)), the processing according to the first and second formulas of [Formula 1] Real part A (j) and imaginary part B (j) are obtained.

  Subsequently, a spectral component is calculated (S12). Specifically, processing according to the following [Formula 1] third formula is performed to obtain an intensity value E (j) at each frequency.

[Formula 1]
A (j) = Σ _{i = 0,..., N-1} W (i) · X (i) · cos (2πij / N)
B (j) = Σ _{i = 0,..., N-1} W (i) · X (i) · sin (2πij / N)
E (j) = A (j) ² + B (j) ²

  In [Expression 1], i is a serial number assigned to N samples in each acoustic frame, and takes an integer value of i = 0, 1, 2,... N−1. Further, j is a serial number assigned in order from the smallest value of the frequency value, and takes an integer value of j = 0, 1, 2,... N / 2-1. When the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by 10.8 Hz.

  Subsequently, thinning processing of spectral components is performed (S13). Although the spectral component up to about 22 kHz is obtained by the above frequency conversion, the spectral component in the range of 340 Hz or higher and lower than about 4 kHz is used for generation of the feature word in this embodiment. This is in order to correspond to a voice compression format such as 3GPP standard used for voice reproduction of a cellular phone (however, since digital acoustic data is always given in this embodiment, a voice recording signal of a cellular phone is used. There is no need to deal with matching.) For this reason, in order to accurately match the voice reproduction range of the mobile phone, the upper limit of the feature word generation may be set to around 3.4 kHz. In the present embodiment, j = 385 to 2047 higher than the vicinity of 4 kHz among the frequency components of j = 0 to 2047 is not used. Also, the low frequency component of j = 0 to 32 that is 340 Hz or less is not used. That is, in the present embodiment, j = 33 to 384 frequency components are used. Specifically, the processing according to the following [Equation 2] is executed and thinned out to Pn in units of 11 frequency components.

[Formula 2]
_{P0 = (E 33 + E 34} + ... + E 43) 1/4
_{P1 = (E 44 + E 45} + ... + E 54) 1/4
:
:
P31 = ( _E374 + _E375 + ... + _E384 ) ^1/4

  According to the above [Expression 2], 352 frequency components of j = 33 to 384 are thinned out to 32 frequency components of n = 0 to 31. The above process is performed for each acoustic frame, and 32 frequency components are obtained for each acoustic frame.

  Next, for each acoustic frame, a difference from the spectral component of the immediately preceding acoustic frame is calculated (S14). The processes of S11 to S13 are sequentially performed on each acoustic frame. The process between frames in S14 uses P0 to P31 obtained as a result of performing the processes up to S13 for each acoustic frame. Specifically, the process according to the following [Equation 3] is performed to obtain the difference Dn (t).

[Formula 3]
Dn (t) = | Pn (t) −Pn (t−1) |, n = 0,..., 31

  In this way, the difference between adjacent acoustic frames is calculated by emphasizing changes in the amplitude level and reflecting the characteristics of the acoustic data even in places where the amplitude level of the acoustic data slightly changes. It is for producing | generating.

  When the processes of S11 to S14 are sequentially performed on each acoustic frame and T differences (n in this embodiment) Dn (t) between the acoustic frames are obtained, a total sum of T is obtained (S16). ). That is, processing according to the following [Equation 4] is performed to obtain the sum Sn of the differences.

[Formula 4]
Sn = Σ _{t = 0,..., T-1} Dn (t)

  Subsequently, the binarization process of Sn obtained by the above [Equation 4] is performed (S17). Specifically, first, the Sn array is divided into two in the upper and lower bands of n ≧ 14 and n ≦ 13, and 1 is given to 7 of 14 large values of n ≦ 13, and 0 is given to 7 of the smaller values. In addition, 1 is given to 9 of 18 large values of n ≧ 14, and 0 is given to 9 of the small values. Here, instead of simply giving 1 and 0 to the 32 large Sn medium values of 16 and 16 small ones, the group of 18 bands having a high frequency and the band of 14 bands having a high frequency The reason why 1 and 0 are equally given to each group is to correct the influence of the frequency characteristics associated with various data compression processes. The reason why the upper and lower bands are divided at the positions of the 18 band and the 14 band is that, as a result of the experiment, when divided at this position, the search acoustic data used for the search is subjected to various data compression processes such as MP3, This is because the search accuracy was the highest. By the processing in S17, Sn for each n can be expressed by 1 bit. Then, a 32-bit feature pattern is obtained with n = 0 as LSB and n = 31 as MSB.

  Next, the volume data is calculated (S18). Specifically, the volume data Vol is calculated using the following [Formula 5].

[Formula 5]
Vol = Σ _{t = 0, ..., T-1} {Σ _{n = 0, ..., 31} Pn (t)}

  As shown in [Formula 5] above, the values of all the thinned components Pn (t) are added for T acoustic frames. As a result, volume data Vol that is the total volume of the T acoustic frames is obtained. The value of the volume data Vol is multiplied by a fixed scaling value set as appropriate, and normalized so that it falls within the range of 0 to 255. Therefore, the volume data Vol is represented by 8 bits. Since the volume data Vol is the sum of the volumes over the T acoustic frames as shown in [Formula 5], it represents the total volume of the T acoustic frames, not the volume of each frame. Will be.

  The processes of S17 and S18 can be performed by changing the order. As a result of the processing in S17 and S18, a 40-bit feature word composed of a 32-bit feature pattern and 8-bit volume data is obtained.

  By executing the above processing for each sound frame, a large number of feature words for the sound data are generated. For example, when the sampling frequency is 44.1 kHz, the sound frame is 4096 samples, and the sound frame is overlapped by 2048 samples as in the above example, one feature word is about 0.506 seconds, and the sound data from 5 minutes Will generate approximately 600 feature words.

  Here, the feature word generation processing will be described with reference to the conceptual diagram of FIG. FIG. 3A is a diagram illustrating a waveform of acoustic data that is a generation target of a feature word. In the related information registration apparatus, the acoustic data is read in units of acoustic frames. However, as shown in FIG. Then, each acoustic frame is divided into 32 bands as shown in FIG. This corresponds to S11 to S13. Next, as shown in FIG. 3D, difference processing between adjacent acoustic frames for each band of separated components and total processing of 32 band separated components are performed. The separation component difference process corresponds to S14, and the separation component summation process corresponds to the summation process of each frequency component (n = 0 to 31) in S18. Next, as shown in FIG. 3E, a summation process of 32-band difference components and a summation process of volume are performed. The summation process of the 32-band difference component corresponds to S16, and the volume summation process corresponds to the summation process of each acoustic frame (t = 0 to T-1) in S18. Next, as shown in FIG. 3 (f), the binarization process of the 32-band sum component and the compression process of the sound volume (256 levels obtained by multiplying the value calculated based on the above [Formula 5] by a predetermined scaling value). To the volume level). The binarization process for the 32-band sum component corresponds to S17, and the volume compression process corresponds to S18. As shown in FIG. 3, a process of sequentially reading sound frames from the sound data and generating one feature word in units of T sound frames is performed.

  When a registered feature word group (a set of feature words) is obtained for the entire specified original sound data, next, sections of a predetermined length from the beginning of the original sound data are sequentially extracted as partial sections. A feature word (partial feature word) is generated for each extracted partial sound data. The length of the partial section can be set as appropriate. The partial section is extracted blindly (equally spaced in the actual embodiment) from the given original sound data, assuming a typical section to be used at the time of search, and some leakage has occurred. However, it is set at a constant interval from the beginning so that there is no overlap, and Z partial sections are obtained for one original sound data. The generation of the partial feature words is basically created by the procedure and concept as shown in FIGS. 2 and 3 as in the case of the registration feature words generated for the entire original sound data. . However, unlike the registered feature words, it is necessary to additionally generate a plurality of partial feature words with a phase shift.

  Since the partial sound data is a piece of music cut out, the timing does not necessarily coincide with the original sound data, and a positional shift may occur. The feature word is generated by averaging T sound frames (T = 11 in the embodiment of the present invention), and is relatively resistant to displacement. However, in the case of original sound data with a sharp change in rhythm, a feature word that is remarkably different is generated when it is shifted by about 5 sound frames, which is almost half of 11 sound frames, which is a feature word generation unit, and is compared with a registered feature word. A significant difference will occur. Therefore, a plurality of partial feature words are generated within the acoustic database 50 by delaying several acoustic frames within a range of 11 acoustic frames, which is one analysis unit (two acoustic frames in this embodiment by changing the phase). Match with the registered feature word to be registered.

In S16, instead of calculating Sn based on [Equation 4], h = 0,. . . , H−1, H types of Sn (h) are calculated based on [Formula 4 ′] below. hs is a suitable positive integer, and hs = 2 in this embodiment. Subsequently, in S17, the binarization process is similarly performed on each of the H types of Sn (h), and each of them is converted into a 32-bit feature pattern.

[Formula 4 ']
Sn (h) = Σ _{t = 0,..., T-1} Dn (t + h · hs)

  Furthermore, in S18, instead of calculating Vol based on [Formula 5], h = 0,. . . , H−1, H types of Vol (h) are calculated based on [Formula 5 ′] below. Similarly, hs is a suitable positive integer, and hs = 2 in this embodiment.

[Formula 5 ']
Vol (h) = Σ _{t = 0,..., T-1} {Σ _{n = 0,..., 31} Pn (t + h · hs)}

  From the above processing, assuming that each of the Z partial sections has a length of 15 seconds, 30 × H partial feature words (H = 5 in this embodiment) are generated.

  Once a registered feature word group and a partial feature word group for each partial section are obtained for a certain original acoustic data, the original acoustic data is then obtained using the registered feature word group and the partial feature word group for each partial section. An optimal determination threshold is calculated for. The optimum determination threshold value calculation process will be described with reference to the flowcharts of FIGS.

4 to 6, each variable is defined as follows.
[Registration feature word]
Y: Number of registered feature words registered for original sound data Fd (y): Feature pattern array (y = 0,..., Y-1), 32 bits Vd (y): Volume data array (y = 0) , ..., Y-1), 8 bits
[Partial feature word]
Z: number of partial sections set in the original sound data X (z, h): partial section z (z = 0,..., Z-1) and shift (phase) h (h = 0,. Number of partial feature words in H-1) F (z, h, x): Feature pattern array (x = 0,..., X (z, h) -1), 32 bits V (z, h, x) : Volume data array (x = 0,..., X (z, h) -1), 8 bits
[Collation variable]
W: Number of collation words (w = 0,..., W−1), number of registered feature words and partial feature words to be collated (eg, W = 6)
D (y + w, z, h, x + w): Number of mismatch bits in word unit of feature pattern (0 to 32)
S (y, z, h, x): the sum of the number of mismatched bits, the number of matching word counts D (y + w, z, h, x + w) (Σw _{= 0,} ... _W-1 D (y + w, z, h, x + w))

  FIG. 4 is a flowchart showing the optimum determination threshold value calculation process. First, initial setting is performed (S110). Specifically, the sum S of the total number of mismatch bits is set to S = 0, and the variable z = 0 for specifying the partial section is set.

  Subsequently, the partial feature word group generated from the partial section z is collated with the registered feature word group (S120). As a result of the collation in S120, the minimum mismatch bit number Smin is obtained for the partial section z. The process of S120 will be described later. When the minimum mismatch bit number Smin is obtained, it is determined whether Smin is larger than the initial value Big1 (S130). The initial value Big1 is a value set in S121 described later. Only when Smin is smaller than the initial value Big1, a process of adding to the sum S of the total mismatch bits is performed (S140). Next, the variable z specifying the partial section is incremented, that is, increased by 1 (S150). Then, it is determined whether or not the variable z that specifies the partial section has reached the number of partial sections Z (S160). If not, the process returns to S120, and the next partial section z is collated. The processing is executed for each partial section z, and when the variable z reaches the number of partial sections Z, that is, when the processing for all Z partial sections is completed, the sum S of the total number of mismatch bits is divided by the number of partial sections Z. The value S / Z is output as the optimum determination threshold value M (S170).

  Next, details of the collation processing of the section z in S120 of FIG. 4 will be described using the flowchart of FIG. First, initial setting is performed (S121). Specifically, the minimum mismatch bit number Smin = initial value Big1, the variable x = 0 specifying the partial feature word, the variable h = 0 specifying the phase, and the variable y = 0 specifying the registered feature word are set. The initial value Big1 may be a value that is sufficiently larger than the value that the minimum mismatch bit number Smin can take, and is set in advance. Subsequently, the sum mismatch bit number S (y, z, h, x) is calculated (S122). Since an error value may be output for the sum mismatch bit number S (y, z, h, x) as described later, the sum mismatch bit number S (y, z, h, x) is obtained as a normal value. If it is determined, it is determined whether or not it is smaller than the minimum mismatch bit number Smin (S123). Only when S (y, z, h, x) is smaller than Smin, a process of setting the value of S (y, z, h, x) to Smin is performed (S124). Then, the variable x is incremented, the processes of S122 to S124 are repeated, and when the variable x for specifying the partial feature word reaches the partial feature word number X (z, h), the variable for specifying the phase with x = 0. h is incremented, and the processing of S122 to S124 is repeated. When h reaches H, y is incremented with h = x = 0, and the processing of S122 to S124 is repeated. That is, first, the phase h is fixed and the partial feature word and the registered feature word are compared. When the processing is completed for all the partial feature words, the phase is changed and the next phase is processed. In this way, processing is executed for all phases.

  Then, it is determined whether or not y has reached the total number Y of registered feature words (S125). If not, the process returns to S122 to calculate the minimum mismatch bit number Smin for the next registered feature word y. Do. When processing is performed for each registered feature word y and the variable y reaches the total number Y of registered feature words, that is, when processing for all Y registered feature words is completed, the total number of mismatch bits S (y, z at that time) , H, x) is output as the minimum mismatch bit number Smin in the partial section z. This minimum mismatch bit number Smin is obtained in S120 of FIG.

  Next, details of the process of calculating the total mismatch bit number S (y, z, h, x) in S122 of FIG. 5 will be described using the flowchart of FIG. In this apparatus, at the time of collation, collation of W consecutive feature words, which is the number of collation words, is performed. That is, the consecutive W registered feature words and the consecutive W partial feature words are collated in order from the top. In FIG. 6, first, initial setting is performed (S181). Specifically, the total mismatch bit number S (y, z, h, x) = 0 and the variable w = 0 indicating the number of feature word collations are set. After the initial setting, it is determined whether or not the condition that the volume data Vd (y + w) of the registered feature word and the volume data V (z, h, x + w) of the partial feature word are both greater than 0 is satisfied (S182). The volume data Vd (y + w) of the registered feature word and the volume data V (z, h, x + w) of the partial feature word are expressed by [Equation 5] and [Equation 5 ′] with respect to the whole original sound data and the partial sound data, respectively. The processing according to the above is executed, and “Vol” and “Vol (h)” are calculated. As a result of the determination in S182, only when one of the registered feature words and the partial feature word volume data Vd (y + w) and the partial feature word volume data V (z, h, x + w) satisfy the condition that both are greater than 0, The word unit mismatch bit number D (y + w, z, h, x + w), which is the number of mismatch bits when one feature word is compared, is calculated (S183).

  Next, it is determined whether or not the word unit mismatch bit number D (y + w, z, h, x + w) is equal to or less than a predetermined value Mw (S184). As a result of the determination in S184, the word unit mismatch bit number D (y + w, z, h, x + w) is not added only if the word unit mismatch bit number D (y + w, z, h, x + w) is equal to or less than the predetermined value Mw. A process of adding to the bit number S (y, z, h, x) is performed (S185). In S185, the variable w indicating the number of feature words to be collated is incremented and the feature word w to be collated is changed to the next feature word w. Then, it is determined whether or not the variable w indicating the number of collated feature words has reached a predetermined number W (S186). If not, the process returns to S182 and the next feature word is processed. When processing is performed for each feature word and the variable w reaches a predetermined number W, the total mismatch bit number S (y, z, h, x) at that time is calculated as the total mismatch bit number for a certain x, h, y. Output as S (y, z, h, x). This sum mismatch bit number S (y, z, h, x) is obtained in S122 of FIG. If it is determined that the condition is not satisfied in S182 and S184, a calculation error value (negative value) for the number of unmatched bits is output.

(1.2. Calculation of word unit mismatch bit number)
The calculation of the word unit mismatch bit number D (y + w, z, h, x + w) in S183 of FIG. 6 will be described. Regarding the calculation of the word unit mismatch bit number D (y + w, z, h, x + w), the specific processing contents differ depending on the sound volume determination mode set by the user. There are four volume determination modes: Off, Weight, Match, and Both.

(1.2.1. Volume judgment mode “Off”)
The sound volume determination mode “Off” is a mode in which no weight is added. When the sound volume determination mode “Off” is set, the word unit mismatch bit number D (y + w, z, h, x + w) is the difference in word units as it is. Is a word unit dissimilarity D (y + w, z, h, x + w) indicating the degree of. In the sound volume judgment mode “Off”, the initial value is set with the word unit mismatch bit number D (y + w, z, h, x + w) = 0, and then 32 bits of Fd (y + w) and F (z, h, x + w). Are sequentially compared in corresponding bit units, and 1 is added to D (y + w, z, h, x + w) every time the bits differ. That is, an exclusive OR is calculated in units of words of Fd (y + w) and F (z, h, x + w), and the number of “1” bits in the obtained 32-bit word is counted. In both the registered feature word group and the partial feature word group, the feature pattern of the feature word is created according to the same rule, and has a 32-bit configuration with the low frequency component as LSB and the high frequency component as MSB. This can be done depending on whether or not the bit values match.

(1.2.2. Volume judgment mode “Weight”)
The volume determination mode “Weight” is a mode for adding a weight, and when the volume determination mode “Weight” is set, the initial value is set with the word unit mismatch bit number D (y + w, z, h, x + w) = 0. After that, 32 bits of Fd (y + w) and F (z, h, x + w) are sequentially compared in corresponding bit units. Based on the comparison result, the process according to the following [Formula 6] is executed to determine the value of D (y + w, z, h, x + w). As a result, D (y + w, z, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account volume data. This mode has no meaning in matching registered feature words and partial feature words generated from the same acoustic data, but becomes the latter source when matching registered feature words and search feature words, which will be described later. When the search sound data is accompanied by signal processing such as analog conversion, and the original sound data and the relative change of the sound volume and the absolute value of the sound volume are different, an appropriate matching result is given.

[Formula 6]
When the Fd (y + w) side is bit 1 and the F (z, h, x + w) side is bit 0,
D (y + w, z, h, x + w) ← D (y + w, z, h, x + w) + Vd (y + w) · 2 / {Vd (y + w) + V (z, h, x + w)}
When the Fd (y + w) side is bit 0 and the F (z, h, x + w) side is bit 1,
D (y + w, z, h, x + w) ← D (y + w, z, h, x + w) + V (z, h, x + w) · 2 / {Vd (y + w) + V (z, h, x + w)}

(1.2.3. Volume judgment mode “Match”)
When the sound volume determination mode “Match” is set, first, processing in the sound volume determination mode “Off” is performed to obtain D (y + w, z, h, x + w). Then, by executing processing according to the following [Equation 7], the weight is multiplied to determine the value of D (y + w, z, h, x + w). As a result, D (y + w, z, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account the difference in the fluctuation pattern of the volume data. This mode has no meaning in matching registered feature words and partial feature words generated from the same acoustic data, but becomes the latter source when matching registered feature words and search feature words, which will be described later. The search acoustic data is accompanied by signal processing such as various types of data compression, and there is not much difference between the original acoustic data and the relative change in volume, but an appropriate collation result is given when the absolute values are different. In this embodiment, this mode is most recommended.

[Formula 7]
When Vd (y + w) · Vd (y + w−1)> V (z, h, x + w) · V (z, h, x + w−1),
D (y + w, z, h, x + w) ← D (y + w, z, h, x + w) .Vd (y + w) .Vd (y + w-1) / {V (z, h, x + w) .V (z, h, x + w-1)}
When Vd (y + w) · Vd (y + w−1) <V (z, h, x + w) · V (z, h, x + w−1),
D (y + w, z, h, x + w) ← D (y + w, z, h, x + w) · V (z, h, x + w) · V (z, h, x + w−1) / {Vd (y + w) · Vd ( y + w-1)}

  When w = 0, in the above [Expression 7], Vd (y + w-1) = Vd (y + w) and V (z, h, x + w-1) = V (z, h, x + w).

(1.2.4. Volume judgment mode “Both”)
When the sound volume determination mode “Both” is set, first, processing in the sound volume determination mode “Weight” is performed to obtain D (y + w, z, h, x + w). Then, by executing the processing according to the following [Equation 8], the value of D (y + w, z, h, x + w) is determined by multiplying the weight. As a result, D (y + w, z, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account volume data. This mode has no meaning in matching registered feature words and partial feature words generated from the same acoustic data, but becomes the latter source when matching registered feature words and search feature words, which will be described later. Appropriate verification when the search sound data is accompanied by signal processing such as high compression ratio data compression with waveform distortion and analog conversion, and both the relative change in volume and the absolute value of volume are significantly different from the original sound data Give the result.

[Formula 8]
When Vd (y + w) · V (z, h, x + w−1)> V (z, h, x + w) · Vd (y + w−1),
D (y + w, z, h, x + w) ← D (y + w, z, h, x + w) · Vd (y + w) · V (z, h, x + w−1) / {V (z, h, x + w) · Vd ( y + w-1)}
If Vd (y + w) · V (z, h, x + w−1) <V (z, h, x + w) · Vd (y + w−1),
D (y + w, z, h, x + w) ← D (y + w, z, h, x + w) · V (z, h, x + w) · Vd (y + w−1) / {Vd (y + w) · V (z, h, x + w-1)}

  When w = 0, in the above [Expression 8], Vd (y + w−1) = Vd (y + w) and V (z, h, x + w−1) = V (z, h, x + w).

  In the case other than the sound volume determination mode “Off”, D (y + w, z, h, x + w) is not the word unit mismatch bit number but the word unit dissimilarity indicating the degree of difference of the word unit considering the sound volume data. In FIG. 5 and FIG. 6, S (y, z, h, x) represents the total dissimilarity, not the total mismatch bit number. Further, Smin in FIGS. 4 and 5 represents not the minimum mismatch bit number but the minimum difference.

  Eventually, the optimum determination threshold value M (= S / Z) is calculated in S170 of FIG. 4, so that for each original sound data, the related information about the original sound data and the original sound data can be uniquely identified. Are registered in the acoustic database 50 in association with the ID, the registered feature word, and the optimum determination threshold value. The related information may be any information as long as it is related to the original sound data. For example, if the original sound data is a song, the name of the song, the name of the player, and the original sound data are CM audio. If so, the name or URL of the sponsoring company can be used. However, a series of binary-format material data (individual recording data before mixing down, MIDI input data) used for production / mastering of the original sound data is not normally subject to restrictions due to copyright laws.

(2. Related information retrieval device)
Next, a related information search device for acoustic data according to the present invention (hereinafter referred to as “related information search device”) will be described. FIG. 7 is a configuration diagram of a related information retrieval apparatus according to the present invention. In FIG. 7, 50 is an acoustic database, 60 is a feature word generating means, 70 is a feature word collating means, and 80 is an information output means. The related information search device searches for related information related to the original sound data registered in the sound database as related information related to the search sound data using the search sound data held by the user. The search sound data is sound data used for search. At the time of search, it is necessary to collate a search feature word, which is a feature word generated from the search acoustic data, with a registered feature word registered in the acoustic database 50 in advance. For this reason, the search feature word and the registered feature word must basically have the same structure (note that the former search feature word group includes a plurality of feature words whose phases are changed in the same manner as the partial feature word group described above. A set of groups is generated), the search sound data and the original sound data that are based on it are compressed in various encoding formats, and are generally different from each other depending on the form of acquisition. It is necessary to convert to an encoding format. In this embodiment, the search sound data and the original sound data are converted into the PCM format having the same specifications (sampling frequency: 44.1 kHz, quantization bit number: 16 bits, channel number: 1 / monophonic PCM format parameter). I try to unify.

  Similar to the registered feature word generation unit 10 shown in FIG. 1, the search feature word generation unit 60 performs frequency analysis using the read sound frame and generates a feature word expressing the features of the search sound data. It has a function. However, similarly to the generation of the partial feature words described above, a set of a plurality of (H) feature word groups whose phases are shifted is generated. The feature word collating unit 70 has a function of collating the generated search feature word with the registered feature word registered in the acoustic database 50. The information output means 80 has a function of extracting the relevant information about the original sound data similar to the feature of the searched sound data as a result of the matching by the feature word matching means 70 and outputting it. Each component shown in FIG. 7 is actually realized by mounting a dedicated program on a device including an arithmetic processing unit. As a preferred example, there is a case where a personal computer having a high-performance arithmetic processing function connected to a server computer on which an acoustic database is operated through a network executes the contents of each means according to a dedicated program. .

  Subsequently, the processing operation of the apparatus shown in FIG. 7 will be described. First, when the user wants to search for the search sound data held by the user, the related information search device is instructed to start, and after the start, the search sound data to be searched is designated. This can be executed by operating a button on a predetermined computer screen and designating search acoustic data stored in the storage area of the related information search device. Actually, since the search sound data is often in a compression format such as MP3, it is processed after being converted into the PCM format. When an instruction is input, the feature word generation means 60 reads a predetermined number of samples as one acoustic frame from the designated search acoustic data. This process is performed in the same manner as that performed by the related information registration apparatus. That is, the number of samples in one acoustic frame is 4096 samples when the sampling frequency is 44.1 kHz. In addition, the sound frame is read by overlapping 2048 samples.

  The processing from here to the generation of the search feature word follows the flowchart of FIG. The flowchart in FIG. 8 is almost the same as the flowchart in FIG. 2 for the registration feature word generation. The search feature word generation unit 60 performs frequency conversion on each read sound frame to obtain a frame spectrum that is a spectrum of the sound frame (S21). As with the related information registration apparatus, as the frequency conversion, Fourier transform, wavelet transform, and other various known methods can be used. However, since it is necessary to match the processing of the related information registration apparatus, in this embodiment, the Fourier transform Is used.

  Further, in S22 to S28, after performing processing according to the above [Formula 2] to [Formula 4 ′] to obtain Sn (h), a 32-bit feature pattern is generated, and the above [Formula 5 ′ ] To generate 8-bit volume data Vol (h) to obtain a 40-bit feature word. That is, the related information search device generates H times (H = 5 in this embodiment) feature words as compared to the related information registration device. Since the search acoustic data to be searched is a piece of music cut out, the timing does not necessarily coincide with the registered acoustic data registered in the database, and a positional shift may occur. Even in the method generated in the related information registration apparatus, since 11 acoustic frames are generated by averaging, they are relatively resistant to displacement. However, in the case of searched sound data with a sharp rhythm change, if the sound sound is shifted by about 5 sound frames, which is almost half of 11 sound frames, which is a feature word generation unit, significantly different feature words are generated, and erroneous information is searched. End up. Therefore, a plurality of search feature words are generated by delaying (changing the phase) two sound frames within a range of 11 sound frames as one analysis unit, and collated with registered feature words in the sound database 50. To do.

  Specifically, the related information registration device generates one feature word from frame 1 to frame 11, whereas the related information search device generates a feature word from frame 1 to frame 11 and for two acoustic frames. A feature word is also generated from a group of acoustic frames shifted (by changing the phase) by four acoustic frames, six acoustic frames, and eight acoustic frames. That is, feature words are also generated in frames 3 to 13, frames 5 to 15, frames 7 to 17, and frames 9 to 19.

  When the search feature word group is obtained for the search acoustic data to be searched, the feature word collating unit 70 then uses the search feature word group to relate the related information of the original acoustic data registered in the acoustic database 50. Search for. Specifically, the search feature word group and the registered feature word group registered in the acoustic database 50 are compared, and a record satisfying a predetermined condition is extracted. This search process will be described with reference to the flowcharts of FIGS.

9 to 11, each variable is defined as follows.
[Registration feature word]
R: Number of records (number of original acoustic data managed by the acoustic database 50)
Y (r): Number of registered feature words of record r (r = 0,..., R-1) Fd (r, y): Feature pattern of record r (r = 0,..., R-1) Array (y = 0,..., Y-1), 32-bit Vd (r, y): Volume data array (y = 0,...) Of record r (r = 0,..., R-1) ., Y-1), 8 bits
[Search feature word]
X (h): Number of search feature words in shift (phase) h (h = 0,..., H−1) F (h, x): Feature pattern array (x = 0,..., X (h) ) -1), 32 bits V (h, x): Volume data array (x = 0,..., X (h) -1), 8 bits
[Collation variable]
W: Number of collation words (w = 0,..., W−1), number of registered feature words and partial feature words to be collated (eg, W = 6)
D (r, y + w, h, x + w): Number of mismatch bits in word unit of feature pattern (0 to 32)
S (r, y, h, x): the sum of the mismatched bit number, the word count mismatched bit number D (r, y + w, h, x + w) of the collation word number (Σw _{= 0,..., W -1} D (r, y + w, h, x + w))

  FIG. 9 is a flowchart of the acoustic data search using the search feature word group by the feature word collating unit 70. First, initial setting is performed (S210). Specifically, the matching table Smin (c) = initial value Big2, the matching table Rmin (c) = − 1, the matching number c = 0, and the record number r = 0 are set. The initial value Big2 may be a value sufficiently larger than a value that can be taken as the minimum number of mismatch bits, and is set in advance. Subsequently, the registered feature word group registered in association with the record r is collated with the search feature word group (S220). As a result of the collation in S220, the record number r is given to Rmin (c) and outputted. The process of S220 will be described later.

  If Rmin (c) is obtained, it is determined whether Rmin (c) is 0 or more (S230). When Rmin (c) is 0 or more, it is determined that the record is matched, and a process of adding 1 to the number of matching cases c is performed (S240). If Rmin (c) is less than 0, it is determined that the record has not been matched, and the matching number c is not added. As will be described in detail later, in S220, a value less than 0 is set as an initial value in Rmin (c), and when it is determined that the record is suitable, a record number r that is 0 or more is set to Rmin (c). To give. For this reason, in S240, the suitability of the record is determined by determining whether Rmin (c) is 0 or more.

  Next, the variable r specifying the record is incremented, that is, increased by 1 (S250). Then, it is determined whether or not the variable r for specifying the record has reached the total number R of records in the acoustic database (S260). If not, the process returns to S220 and the next record r is collated. When processing is performed for each record r and the record r reaches the total number of records R, that is, when processing for all R total records is completed, the matching table Rmin (c) is changed to the value of the matching table Smin (c). Based on the ascending order, the list is output together with the matching number c (S270).

    Next, details of the record r matching processing in S220 of FIG. 9 will be described using the flowchart of FIG. First, initial setting is performed (S221). Specifically, the matching table Smin (c) = initial value Big2, the matching table Rmin (c) = − 1, the variable x = 0 for specifying the search feature word, the variable h = 0 for specifying the phase, and the registered feature word The variable y to be specified is set to 0. Subsequently, the sum mismatch bit number S (r, y, h, x) is calculated (S222). When the total mismatch bit number S (r, y, h, x) is obtained, the optimum determination threshold value M registered in association with the record r is extracted from the acoustic database 50, and the total mismatch bit number S (r , Y, h, x) is determined whether it is equal to or less than the optimum determination threshold value M (S223). When the total mismatch bit number S (r, y, h, x) is less than or equal to the optimum determination threshold value M, the total mismatch bit number S (r, y, h, x) is smaller than the total mismatch bit number minimum value Smin. It is determined whether it is small (S224).

  Only when the sum mismatch bit number S (r, y, h, x) is smaller than the sum mismatch bit number minimum value Smin, the value of r is set in the adaptation table Rmin (c), and S (r, y, h, A process of setting the value of x) in the matching table Smin (c) is performed (S225). Next, the variable x is incremented, and the processing of S222 to S225 is repeated. When x becomes X (h), x is incremented by x = 0, the processing of S222 to S225 is repeated, and X (h ) Becomes H, y is incremented with h = x = 0, and the processing of S222 to S225 is repeated (S226).

  Then, it is determined whether or not y has reached the number of all registered feature words Y (r) of the record r (S227). If not, the process returns to S222 to add the sum mismatch bit for the next registered feature word y. The number minimum value Smin is calculated. Processing is performed for each registered feature word y, and when the variable y reaches the total number Y (r) of registered feature words in the record r, that is, when processing for all the registered feature words Y (r) is completed, The matching table Rmin (c) and the matching table Smin (c) are output. The matching table Rmin (c) and the matching table Smin (c) are obtained in S220 of FIG.

  Next, details of the calculation processing of the total mismatch bit number S (r, y, h, x) in S222 of FIG. 10 will be described using the flowchart of FIG. First, initial setting is performed (S281). Specifically, the total mismatch bit number S (r, y, h, x) = 0 and the variable w = 0 indicating the number of feature word collations are set. After the initial setting, it is determined whether or not the condition that the volume data Vd (r, y + w) of the registered feature word and the volume data V (h, x + w) of the search feature word are both greater than 0 is satisfied (S282). The volume data Vd (r, y + w) of the registered feature word and the volume data V (h, x + w) of the search feature word are expressed in [Formula 5] and [Formula 5 ′] for the entire original sound data and the retrieved sound data, respectively. The processing according to the above is executed, and “Vol” and “Vol (h)” are calculated. As a result of the determination in S282, the registered feature word 1 only when the volume data Vd (r, y + w) of the registered feature word and the volume data V (z, h, x + w) of the search feature word are both greater than 0. The word unit mismatch bit number D (r, y + w, h, x + w), which is the number of mismatch bits when the search feature word is compared with one search feature word, is calculated (S283).

  Next, it is determined whether or not the word unit mismatch bit number D (r, y + w, h, x + w) is equal to or smaller than a predetermined value Mw (S284). As a result of the determination in S284, the word unit mismatch bit number D (r, y + w, h, x + w) is not integrated only when the word unit mismatch bit number D (r, y + w, h, x + w) is equal to or less than the predetermined value Mw. A process of adding to the bit number S (r, y, h, x) is performed (S285). In S285, a process of incrementing the variable w indicating the number of feature word collations is also performed. Then, it is determined whether or not the variable w indicating the number of collated feature words has reached a predetermined number W (S286). If not, the process returns to S282 and the next feature word is processed. When processing is performed for each feature word and the variable w reaches a predetermined number W, the total mismatch bit number S (r, y, h, x) at that time is calculated as the total mismatch bit number for a certain x, h, y. Output as S (r, y, h, x). This sum mismatch bit number S (r, y, h, x) is obtained in S222 of FIG. If it is determined that the condition is not satisfied in S282 and S284, a calculation error of the number of total mismatch bits is output.

(2.2. Calculation of word unit mismatch bit number)
The calculation of the word unit mismatch bit number D (r, y + w, h, x + w) in S283 in FIG. 11 will be described. Regarding the calculation of the word unit mismatch bit number D (r, y + w, h, x + w), the specific processing contents differ depending on the sound volume determination mode set by the user. There are four volume determination modes: Off, Weight, Match, and Both.

(2.2.1. Volume judgment mode “Off”)
The volume determination mode “Off” is a mode in which no weight is added, and when the volume determination mode “Off” is set, the word unit mismatch bit number D (r, y + w, h, x + w) is directly different in word units. This is a word unit dissimilarity D (r, y + w, h, x + w) indicating the degree of. In the sound volume determination mode “Off”, the initial value is set with the word unit mismatch bit number D (r, y + w, h, x + w) = 0, and then 32 bits of Fd (r, y + w) and F (h, x + w). Are sequentially compared in corresponding bit units, and 1 is added to D (r, y + w, h, x + w) every time the bits differ. In both the registered feature word group and the search feature word group, the feature pattern of the feature word is created according to the same rule, and has a 32-bit configuration in which the low frequency component is LSB and the high frequency component is MSB. This can be done depending on whether or not the bit values match.

(2.2.2. Volume judgment mode “Weight”)
The sound volume determination mode “Weight” is a mode for adding a weight, and when the sound volume determination mode “Weight” is set, the initial value is set with the word unit mismatch bit number D (r, y + w, h, x + w) = 0. After that, the 32 bits of Fd (r, y + w) and F (h, x + w) are sequentially compared in corresponding bit units. Based on the result of the comparison, processing according to the following [Equation 9] is executed to determine the value of D (r, y + w, h, x + w). As a result, D (y + w, z, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account volume data. This mode has no meaning in matching registered feature words and partial feature words generated from the same original sound data, but the latter search is performed when matching registered feature words and search feature words. When the acoustic data is accompanied by signal processing such as analog conversion, and the original acoustic data, the relative change in volume and the absolute value of the volume are different, an appropriate matching result is given.

[Formula 9]
When the Fd (r, y + w) side is bit 1 and the F (h, x + w) side is bit 0,
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) + Vd (r, y + w) · 2 / {Vd (r, y + w) + V (h, x + w)}
When the Fd (r, y + w) side is bit 0 and the F (h, x + w) side is bit 1,
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) + V (h, x + w) · 2 / {Vd (r, y + w) + V (h, x + w)}

(2.2.3. Volume judgment mode “Match”)
When the sound volume determination mode “Match” is set, first, processing in the sound volume determination mode “Off” is performed to obtain D (r, y + w, h, x + w). Then, by executing the processing according to the following [Equation 10], the value of D (r, y + w, h, x + w) is determined by multiplying the weight. As a result, D (y + w, z, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account the difference in the fluctuation pattern of the volume data. This mode has no meaning in matching registered feature words and partial feature words generated from the same original sound data, but the latter search is performed when matching registered feature words and search feature words. The acoustic data is accompanied by signal processing such as various data compression, and there is not much difference between the original acoustic data and the relative change in volume, but an appropriate collation result is given when the absolute values are different. In this embodiment, this mode is most recommended.

[Formula 10]
When Vd (r, y + w) · Vd (r, y + w−1)> V (h, x + w) · V (h, x + w−1),
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) · Vd (r, y + w) · Vd (r, y + w−1) / {V (h, x + w) · V (h, x + w-1)}
When Vd (r, y + w) · Vd (r, y + w−1) <V (h, x + w) · V (h, x + w−1),
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) · V (h, x + w) · V (h, x + w−1) / {Vd (r, y + w) · Vd (r, y + w-1)}

  When w = 0, in the above [Equation 10], Vd (r, y + w−1) = Vd (r, y + w) and V (h, x + w−1) = V (h, x + w).

(2.2.4. Volume judgment mode “Both”)
When the sound volume determination mode “Both” is set, first, processing in the sound volume determination mode “Weight” is performed to obtain D (r, y + w, h, x + w). Then, by executing processing according to the following [Equation 11], the value of D (r, y + w, h, x + w) is determined by multiplying the weight. As a result, D (y + w, z, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account volume data. This mode has no meaning in matching registered feature words and partial feature words generated from the same original sound data, but the latter search is performed when matching registered feature words and search feature words. Appropriate matching results when the acoustic data is accompanied by signal processing such as high compression ratio data compression and analog conversion with waveform distortion, and both the relative change in volume and the absolute value of volume are significantly different from the original sound data give.

[Formula 11]
When Vd (r, y + w) · V (h, x + w−1)> V (h, x + w) · Vd (r, y + w−1),
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) · Vd (r, y + w) · V (h, x + w−1) / {V (h, x + w) · Vd (r, y + w-1)}
When Vd (r, y + w) · V (h, x + w−1) <V (h, x + w) · Vd (r, y + w−1),
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) · V (h, x + w) · Vd (r, y + w−1) / {Vd (r, y + w) · V (h, x + w-1)}

  When w = 0, in the above [Equation 11], Vd (r, y + w-1) = Vd (r, y + w) and V (h, x + w-1) = V (h, x + w).

  In cases other than the sound volume determination mode “Off”, D (r, y + w, h, x + w) is not the number of word unit mismatch bits, but the word unit difference degree indicating the degree of difference in word units taking into account volume data. 10 and 11, S (r, y, h, x) represents the total dissimilarity, not the total mismatch bit number. Further, Smin in FIGS. 9 and 10 represents not the minimum mismatch bit number but the minimum dissimilarity.

  Eventually, in S370 of FIG. 9, the matching table Rmin (c) is sorted in ascending order based on the value of the matching table Smin (c) and is listed together with the number of matching cases c. This is searched using the search acoustic data. It becomes a search result. When one is selected from the output list, the information output unit 80 extracts the record from the acoustic database 50. At that time, when an instruction to acquire the original sound data is given, the information output unit 80 uses the file name recorded in the record to obtain related information related to the original sound data stored in the storage device or the original sound data. An ID for specifying is output.

(3. Modification 1: Multiple databases)
Next, modifications of the related information registration device and the related information search device according to the present invention will be described. In the above embodiment, information related to all the original sound data is registered in one sound database, and an optimum determination threshold value is calculated for each registered original sound data. To make the judgment of applicable / non-applicable. In the following, a plurality of divided sound databases are prepared, and information related to each original sound data is distributed and registered in any of the plurality of divided sound databases according to the optimum determination threshold. A modification in which a search is performed by accessing a divided acoustic database corresponding to the optimum determination threshold among a plurality of divided acoustic databases will be described.

  FIG. 12 is a diagram illustrating a configuration of a related information registration apparatus in a modification in which related information related to original sound data is registered in a plurality of divided sound databases. In FIG. 12, 11 is a registered feature word generating means, 21 is a partial feature word generating means, 31 is a determination threshold value calculating means, 41 is a registering means, and 51a, 51b and 51c are divided acoustic databases.

  The registered feature word generation unit 11, the partial feature word generation unit 21, and the determination threshold value calculation unit 31 are the same as the registered feature word generation unit 10, the partial feature word generation unit 20, and the determination threshold value calculation unit 30 illustrated in FIG. Has the same function. The registration unit 41 has a function of registering the registered feature word and the optimum determination threshold in the acoustic database in association with the related information related to the original original sound data and the ID for specifying the original sound data. 1 is the same as the registration means 40 shown in FIG. 1 except that the divided acoustic database to be registered is distinguished according to the optimum determination threshold value. The divided acoustic databases 51a, 51b, and 51c are shown in FIG. 1 in that the registered feature word, the optimum determination threshold value are stored in association with the related information related to the original sound data, and the ID for specifying the original sound data. The acoustic database 50 is the same as the acoustic database 50 shown in FIG. 6 except that the difference is distinguished by the registered optimum determination threshold range (upper limit and lower limit). The range of the optimum determination threshold value can be set as appropriate, but can be changed as appropriate so that the number of registered cases is as uniform as possible by referring to the number of registered cases.

  As the specific processing operation of the related information registration apparatus shown in FIG. 12, first, the registered feature word generation means 11, the partial feature word generation means 21, and the determination are performed in the same manner as the related information registration apparatus shown in FIG. An optimum determination threshold value is calculated by the threshold value calculation means 31. When the optimum determination threshold value is calculated, the registration unit 41 compares the calculated optimum determination threshold value with the upper and lower limits of the optimum determination threshold value set for each of the divided acoustic databases. The optimum determination threshold value, registered feature word, and related information are registered in the divided acoustic database.

  Next, a modified example of the related information search device according to the present invention will be described. FIG. 13 is a diagram illustrating a configuration of a related information search device in a modified example in which related information of search sound data is searched from a plurality of sound databases. In FIG. 13, 51a, 51b and 51c are divided acoustic databases, 61 is a search feature word generation means, 71a, 71b and 71c are feature word matching means corresponding to the divided acoustic databases 51a, 51b and 51c, and 81 is an information output means. is there.

  The search feature word generation unit 61 and the information output unit 81 have the same functions as the search feature word generation unit 60 and the information output unit 80 shown in FIG. Each of the feature word collating means 71a, 71b, 71c is the same as the feature word collating means 70 in that the generated search feature word and the registered feature word registered in the acoustic database are collated. It is different in that the collation processing can be performed in parallel for each of the acoustic databases.

  As a specific processing operation of the related information search device shown in FIG. 13, first, the search feature word generation means 61 generates a search feature word from the search acoustic data in the same manner as the related information search device shown in FIG. To do. When the search feature word is generated, each of the feature word collating units 71a to 71c independently accesses the corresponding divided acoustic databases 51a to 51c, and simultaneously performs the search process. Specifically, the search feature word group is compared with the registered feature word group registered in the acoustic databases 51a to 51c, and a record satisfying a predetermined condition is extracted. Search processing for each acoustic database is performed in the procedure as shown in FIGS.

  The processing shown in FIG. 13 has a configuration suitable for parallel processing by a computer, and when it is realized by a computer program, it is recommended that it be realized by the following parallel processing. A multi-core CPU or PC cluster is constructed so that the constituent acoustic databases 51a to 51c are mounted on a storage device such as an independent HDD or semiconductor memory, and the feature word collating means 71a to 71c can be executed in parallel by a plurality of independent CPUs. Implement on a type computer. Then, the search process can be speeded up in proportion to the number of distributed databases. On the other hand, at the time of registration, it is only necessary to update one of the distributed divided acoustic databases, so there is an advantage that the load of the registration process is also reduced.

  When the search results are obtained from the respective acoustic databases 51a to 51c, the feature word collating means 71a to 71c are individually transferred to the information output means 81, and the information output means 81 receives the Smin (passed from each feature word collating means 71a to 71c. c) and the array of Rmin (c) are integrated into one, sorting is performed in ascending order based on the integrated value of Smin (c), and the matching table Rmin (c) integrated with the number of matching cases c is integrated. Output a list.

(4. Modification 2: Batch registration and batch search)
In the related information registration device according to the present invention, not only one original sound data is registered, but when a plurality of original sound data are stored in one folder, all the original sound data stored in the designated folder are stored. The acoustic data may be registered in a lump. In such a configuration, when one folder in the storage device is designated, the original sound data in the folder is sequentially read into the registered feature word generation unit 10 and the partial feature word generation unit 20 shown in FIG. Then, it is processed by the determination threshold value calculation means 30 and the registration means 40 and registered in the acoustic database 50. At that time, the file name can be used as the related information registered for each original sound data, and if the file name is composed of an ID for specifying the music, the related information defined separately based on the file name is used. It is also possible to automatically search the information database and register related information. The related information database stores only metadata such as the song title / artist name of a song by defining a unique identification number for each song, and does not store binary data such as acoustic data and feature words. For example, as a representative related information database in Japan, JASRAC-JWID published by the Japan Music Copyright Association can be used. Even in the case of registration in a plurality of divided sound databases shown in FIG. 12, the processing is sequentially performed on all the original sound data stored in the designated folder, and registration is performed.

  In the related information search device according to the present invention, not only search for one search sound data but also a plurality of search sound data stored in one folder, all searches stored in the designated folder It is good also as a structure which searches acoustic data collectively. In such a configuration, when one folder in the storage device is designated, the search acoustic data in the folder is sequentially read into the search feature word generation means 60 shown in FIG. The information output means 80 processes the related information as a search result. In the case of performing a search for a plurality of divided sound databases shown in FIG. 13, similarly, all the search sound data stored in the designated folder is sequentially processed. The search result of is output. In both FIG. 7 and FIG. 13, the information output means records the search result in the log file.

  In this way, by performing batch registration and batch search, there is no need to individually execute execution instructions for a large amount of sound data, and all registrations and searches can be performed only by giving execution instructions to one folder. Can be done.

(5. Modification 3: Prevention of double registration)
In the related information registration device according to the present invention, the specified original sound data is not registered unconditionally, but when it is determined that the same is the same as the already registered original sound data, It is good also as a structure which prevents double registration. FIG. 14 is a diagram illustrating a configuration of a related information registration apparatus in a modified example for preventing double registration. The related information registration apparatus of FIG. 14 has a configuration in which a feature word collating unit 72 is added to the related information registration apparatus shown in FIG. 1, and the same reference numerals as those of the related information registration apparatus shown in FIG. The requirements have the same functions as those shown in FIG.

  In the related information registration apparatus of FIG. 14, after the partial feature word generation unit 20 generates a partial feature word, the feature word collating unit 72 searches the acoustic database 50 using the partial feature word. The specific procedure of the search by the feature word collating unit 72 is the same as that of the feature word collating unit 70 in FIG. 7 and the feature word collating unit 71 in FIG. If the corresponding record exists as a result of the search, the related information registration device stops the registration process and processes the registration feature word generation unit 10, the determination threshold value calculation unit 30, and the registration unit 40 in the subsequent stage. Control stop.

  As described above, after generating the partial feature word from the original sound data, the acoustic database is searched using the generated partial feature word before calculating the optimum determination threshold and performing the registration process. Therefore, it becomes possible to prevent double registration.

(6. Modification 4: Additional registration when there is no match at the time of search)
The related information search device according to the present invention may be configured to register the acoustic data used for the search when there is no match as a result of the search. FIG. 15 is a diagram illustrating a configuration of a related information search apparatus in a modification in which such additional registration is performed. The related information search apparatus of FIG. 15 has a configuration in which the registered feature word generation means 10, the partial feature word generation means 20, the determination threshold value calculation means 30, and the registration means 40 are added to the related information search apparatus shown in FIG. Basically, the related information search device of FIG. 7 and the related information registration device of FIG. 1 are integrated.

  In the related information search device of FIG. 15, similar to the related information search device of FIG. 7, after the search feature word generating unit 60 creates the search feature word, the feature word collating unit 70 includes the generated search feature word, The registered feature word registered in the acoustic database 50 is collated. As a result, when there is no corresponding record, the information output means 80 outputs that the corresponding record does not exist and additional registration is executed. The registered feature word generation unit 10, the partial feature word generation unit 20, the determination threshold value calculation unit 30, and the registration unit 40 are activated, and the registration feature word generation unit 10 generates a registration feature word from the original acoustic data to be searched. At the same time, the partial feature word generation means 20 generates a partial feature word. Then, similarly to the related information registration device shown in FIG. 1, the determination threshold value calculation unit 30 calculates the optimum determination threshold value, and the registration unit 40 acquires related information related to the acoustic data to be searched, The ID, the optimum determination threshold value, and the registered feature word are associated with each other and registered in the acoustic database 50. As a result of the collation by the feature word collating unit 70, if there is even one record, the information output unit 80 outputs the information of the hit record as in the related information search device of FIG. No action is performed on the registered feature word generation unit 10, the partial feature word generation unit 20, the determination threshold value calculation unit 30, and the registration unit 40.

  The preferred embodiments of the present invention have been described above, but the present invention is not limited to the above embodiments, and various modifications can be made. For example, in the above-described embodiment, in the related information registration device, a plurality of partial sound data is extracted from a plurality of partial sections and a partial feature word group is generated from each partial sound data. However, only one partial section is generated. It is also possible to extract only one partial sound data and generate one partial feature word group.

  In the above embodiment, four modes can be selected as the sound volume determination mode. However, two or three of the four modes may be set to be selectable, and one mode may be selected. It may be fixedly set.

  The present invention relates to the protection of music copyright (monitoring illegal copying) in the field of package music for viewing for consumer and business use using CDs and DVDs, and the field of broadcasting and network music distribution distributed for commercial purposes by broadcasters and the like. ) And provision of music attribute information (music title search service).

10, 11 ... registered feature word generation means 20, 21 ... partial feature word generation means 30, 31 ... determination threshold value calculation means 40, 41 ... registration means 50 ... acoustic database 51a- 51c ... Divided acoustic database 60, 61 ... Search feature word generation means 70, 71a-71c, 72 ... Feature word collation means 80, 81 ... Information output means

Claims (13)

A device that generates a feature word expressing the feature from given original sound data, and registers related information related to the original sound data together with the feature word in an acoustic database,
A unit section of a predetermined length is set for the original sound data, and a feature word group that is a set of feature words expressing a feature pattern of the original sound data is registered based on information obtained by analyzing the unit section. Registered feature word generation means for generating as a feature word group;
Based on information obtained by partially cutting out the original sound data, obtaining partial sound data, setting a unit section of a predetermined length for the partial sound data, and analyzing the unit section, Partial feature word generation means for generating a feature word group that is a set of feature words expressing a feature pattern of partial acoustic data as a partial feature word group;
The degree of the minimum difference in the position condition where the registered feature word group and the partial feature word group are best matched by collating between the registered feature word group and the partial feature word group while shifting the temporal positional relationship between them. A determination threshold value calculating means for calculating an optimum determination threshold value based on the minimum dissimilarity,
Registration means for registering the related feature information related to the original acoustic data in the acoustic database in association with the information specifying the original acoustic data, the registered feature word group, the optimum determination threshold value, and the original acoustic data;
An apparatus for registering related information of acoustic data, comprising: In claim 1,
The partial feature word group is generated in a plurality of groups based on a plurality of partial sound data having different cut-out positions, and the determination threshold value calculation unit is configured to determine a temporal position between the registered feature word group and each partial feature word group. The minimum number of inconsistent bits is used as the minimum dissimilarity value for the minimum difference in the position condition where the registered feature word group and each partial feature word group are best matched by collating each other while shifting the relationship. An acoustic data-related information registration apparatus characterized in that an optimum determination threshold value is calculated based on an average value of minimum mismatch bit numbers calculated for each partial feature word group. In claim 1,
The feature pattern indicates the intensity for each predetermined frequency range,
The feature word, added before Kitoku symptom patterns have the volume data, the partial characteristic word group the way cut each other is generated plurality of groups based on a plurality of different partial sound data,
The determination threshold value calculating means obtains the number of mismatch bits by collating the feature pattern bit by bit while shifting the temporal positional relationship between the registered feature word and each partial feature word, A weighting mismatch bit number is calculated by adding or multiplying the weight based on the volume data of the word to the mismatch bit number, and a minimum weight mismatch bit number that is the minimum value in the position condition in which both are the best match for each feature word group An acoustic data related information registration apparatus, characterized in that an optimum determination threshold value is calculated based on an average value for each partial feature word group of the minimum difference degree given as the minimum difference degree. In any one of Claims 1-3,
The acoustic database is composed of a plurality of divided acoustic databases in which ranges of optimum determination thresholds that do not overlap each other are set,
In accordance with the optimum determination threshold value calculated by the determination threshold value calculation means, the registration means is associated with the registered feature word group, the optimum determination threshold value, and the original sound data in a corresponding divided acoustic database. A related information registration device for acoustic data, wherein the related information is registered. In any one of Claims 1-4,
When a plurality of the original sound data are stored in one folder, the original sound data is extracted one by one from the folder and sequentially registered feature word generation means, partial feature word generation means, and determination for each original sound data Process by threshold calculation means,
The said registration means registers the file name of each original acoustic data as one of the said relevant information, The related information registration apparatus of the acoustic data characterized by the above-mentioned. In any one of Claims 1-5,
When the partial feature word group and the registered feature word group registered in the acoustic database are collated and the condition based on the optimum determination threshold registered for each registered feature word group is satisfied, the registered feature word A feature word matching means for extracting records having groups;
The registered feature word generation means, the determination threshold value calculation means, and the registration means perform processing only when there is no corresponding record as a result of matching by the feature word matching means. Related information registration device. A device that searches the related information related to the searched sound data from the sound database by collating the characteristics of the retrieved sound data, which is given sound data, with the features of the original sound data registered in the sound database. And
For each original sound data, an acoustic database in which a registered feature word group that is a set of registered feature words expressing the feature pattern, an optimum determination threshold for the original sound data, and related information related to the original sound data are registered When,
Search feature word generation means for generating, as a search feature word group, a feature word group that is a set of feature words expressing a feature pattern of the search acoustic data in the section unit for the search acoustic data in a predetermined section unit; ,
The search feature word group and the registered feature word group registered in the acoustic database are collated with each other while shifting the temporal positional relationship, and the registered feature word group and the search feature word group are most suitable. Related information of the original acoustic data corresponding to the registered feature word group when the minimum difference degree, which is the degree of the minimum difference in the position condition, is smaller than the optimum determination threshold value registered for each registered feature word group Characteristic word collating means for selecting as a candidate of related information of the search acoustic data,
An apparatus for retrieving related information of acoustic data, comprising: In claim 7,
The feature word collating means collates the registered feature word group and the search feature word group in bit units while shifting the temporal positional relationship between the registered feature word group and the search feature word group, and the registered feature word group and the search feature word group are most suitable. An apparatus for retrieving related information of acoustic data, wherein a minimum mismatch bit number is obtained as a minimum dissimilarity that gives a minimum value in a position condition. In claim 7,
The feature pattern indicates the intensity for each predetermined frequency range,
The feature word, added before Kitoku symptom patterns have the volume data,
The feature word collating means obtains the number of inconsistent bits after collating the feature pattern in bit units between the registered feature word group and the search feature word group while shifting the temporal positional relationship between the two. The weight based on the volume data is added to or multiplied by the mismatch bit number to calculate the weight mismatch bit number, and the minimum weight mismatch bit number that is the minimum value in the position condition where the two match best is obtained as the minimum dissimilarity An apparatus for retrieving related information of acoustic data characterized by In any one of Claims 7 to 9,
The acoustic database is composed of a plurality of divided acoustic databases,
The feature word collating means includes a plurality of divided acoustic databases corresponding to the plurality of divided acoustic databases, performs collation in parallel with each of the divided acoustic databases, and relates the original acoustic data corresponding to the registered feature word group. A related information search device for acoustic data, wherein information is selected as a candidate for related information related to the searched acoustic data. In claim 10,
The divided acoustic database is stored in an independent storage medium;
The characteristic word collating means is composed of a plurality of programs having the same processing content so as to be processed by an independent processor corresponding to each of the divided acoustic databases. In any one of Claims 7-11,
When a plurality of the search sound data are stored in one folder, the search sound data is extracted one by one from the folder, and each search sound data is sequentially processed by the search feature word generation means and the feature word collation means. What to do,
The system further includes information output means for extracting the related information of each original sound data selected by the feature word collating means as the related information of each search sound data, and storing the extracted related information in a predetermined log file. A related information retrieval device for characteristic acoustic data. In any one of Claims 7-12,
A unit section of a predetermined length is set for the search acoustic data, and a set of feature words having a predetermined number of bytes representing the feature pattern of the search acoustic data is registered based on information obtained by analyzing the unit section. Registered feature word generation means for generating as a feature word group;
Based on information obtained by partially cutting out the search sound data to obtain partial sound data, setting a unit section of a predetermined length for the partial sound data, and analyzing the unit section, Partial feature word generation means for generating a set of feature words of a predetermined number of bytes representing a feature pattern of partial acoustic data as a partial feature word group;
While comparing the temporal positional relationship between the registered feature word group and the bit unit of the partial feature word group, they are collated with each other, and based on the minimum difference that is the degree of the minimum difference in the position condition in which both match best Determination threshold value calculating means for calculating an optimum determination threshold value;
Registration means for registering the registered feature word group and the optimum determination threshold in an acoustic database in association with information for specifying the search acoustic data;
Further comprising
The registered feature word generation unit, the partial feature word generation unit, the determination threshold value calculation unit, and the registration unit perform processing only when a corresponding record does not exist as a result of collation by the feature word collation unit. A related information retrieval apparatus for acoustic data.

Images