JP5516274B2 - Relevant information retrieval device for acoustic data - Google Patents

Description

  The present invention relates to an apparatus for searching related information related to acoustic data in which music data such as music is recorded from an acoustic database.

  As a service to provide music attribute information that allows you to know the titles of music that has been played recently, you can query the broadcast station for the date and time of the broadcast music, and record music fragments that are being played on mobile phones. Services that collate with melodies registered in the database have been put into practical use (see, for example, Patent Documents 1 and 2). On the other hand, the applicant also converts the feature of a predetermined section of the sound data digitized by PCM or the like into a search feature word having a predetermined number of bytes, and sets a plurality of search feature words as one set of search features within one set. By sequentially processing the word group and the registered feature word group registered in the database and extracting records with a high degree of match, the melody obtained by recording and the melody registered in the database A technique for performing collation processing at high speed has been proposed (see Patent Document 3).

  In the technique of the above-mentioned patent document 3, when comparing the search feature word group created from the acoustic data (search acoustic data) given as the purpose of the search and the registered feature word group registered in the database, the search acoustic data is It may be obtained by recording the music being played. In this case, the position cannot be obtained from the head position, the position of the registered feature word group is shifted from the original sound data, and the comparison between the feature word groups is judged with a uniform threshold value, so that an accurate search cannot be performed. There was a problem. In order to solve this problem, the applicant has proposed a technique for setting a determination threshold value for each registered feature word group (for each record), not for the entire database (see Patent Document 4).

JP-T-2004-536348 JP-T-2004-537760 JP 2007-226071 A Japanese Patent Application No. 2010-009310

  However, in the technique disclosed in Patent Document 4, when a plurality of records correspond to the search result, there may be a problem in the priority order determination. For example, if the determination threshold values of the two records A and B are in the relationship of A <B, the degree of difference from the search feature word group is equal to or less than each determination threshold value as a result of matching, If priority is given to those having a smaller value, the record A having a smaller determination threshold value is more likely to be prioritized, making it difficult to perform appropriate evaluation.

  Therefore, in the present invention, when searching for related information related to acoustic data registered in the acoustic database using acoustic data such as music, when a plurality of records are matched, the priority order is accurately determined. It is an object of the present invention to provide a related information search apparatus for acoustic data that can be used.

  In order to solve the above-mentioned problem, in the first aspect of the present invention, the search sound data is collated with the characteristics of the search sound data which is given sound data and the characteristics of the original sound data registered in the sound database, and the search sound is recorded. A device that performs a search by identifying original sound data having features close to the data, and for each original sound data, a registered feature word group that is a set of registered feature words that express the features of the original sound data, and a registered feature Individual determination reference values for comparison of word groups, an acoustic database in which relevant information of the original sound data is registered, and the characteristics of searched acoustic data in each section in a predetermined section unit with respect to the searched acoustic data Search feature word generation means for generating a expressed search feature word and obtaining a feature word group that is a set of the search feature words, the search feature word group, and the sound The minimum is the degree of the smallest difference at the position where the search feature word group and the registered feature word group are best matched while collating between the two while moving the positional relationship with the registered feature word group registered in the database The difference is calculated, and the corrected minimum difference is obtained by correcting the minimum difference based on the ratio between the set standard minimum difference and the individual determination reference value registered corresponding to the registered feature word group. And an acoustic data related information retrieval device having a feature word collating unit that identifies the original acoustic data corresponding to the registered feature word group as a selection target when the corrected minimum difference is smaller than a predetermined determination threshold value. provide.

  According to the first aspect of the present invention, for each original sound data, a registered feature word group, which is a set of registered feature words representing the feature pattern, is registered in the sound database, and the search sound data is predetermined. A search feature word that expresses the feature of the search acoustic data is generated for each section, a feature word group that is a set of the search feature word is obtained, and the search feature word group and the registered feature word group are collated. , Calculate the minimum difference, which is the degree of the minimum difference at the best matching position, and based on the ratio of the set standard minimum difference and the individual judgment reference value registered corresponding to the registered feature word group If the corrected minimum dissimilarity is smaller than a predetermined judgment threshold by correcting the minimum dissimilarity, the original acoustic data corresponding to the registered feature word group is selected. Since so as to identify, when searching for related information relating to the sound data registered in the sound database, when a plurality of records are compatible, it is possible to determine the priority accurately.

  Further, according to the second aspect of the present invention, in the related information search apparatus for acoustic data according to the first aspect of the present invention, the related information regarding the original acoustic data specified as the selection target is based on the value of the corrected minimum difference. And an information output means for outputting in order.

  According to the second aspect of the present invention, the related information related to the original sound data specified as the selection target is output in order based on the value of the corrected minimum dissimilarity. In addition, the output is performed in the priority order according to the characteristics of each original sound data.

  According to the third aspect of the present invention, in the acoustic data related information search apparatus according to the first or second aspect of the present invention, the feature word includes each component of the frequency spectrum obtained by analyzing the acoustic data at a predetermined frequency. A feature pattern of a predetermined number of bits expressed by a bit value in units, and the feature word collating unit collates the registered feature word group and the search feature word group by comparing the feature pattern in bit units. The position where the registered feature word group and the search feature word group are most suitable is set as the position where the number of mismatch bits is the smallest, and the number of mismatch bits is used as the minimum dissimilarity. It is characterized in that the minimum number of inconsistent bits that minimizes is obtained.

  According to the third aspect of the present invention, the feature word has a feature pattern having a predetermined number of bits in which each component of the frequency spectrum obtained by analyzing the acoustic data is expressed by a bit value in a predetermined frequency unit. By comparing the registered feature word group and the search feature word group by comparing in bit units, the number of mismatch bits is obtained, and the minimum mismatch bit number that minimizes the number of mismatch bits is set as the minimum difference. It is possible to perform a collation process at high speed by determining the coincidence / non-coincidence between bits.

  According to the fourth aspect of the present invention, in the acoustic data related information search apparatus according to the first or second aspect of the present invention, the feature word includes each component of the frequency spectrum obtained by analyzing the acoustic data at a predetermined frequency. A feature pattern of a predetermined number of bits expressed by a bit value in units and volume data, and the feature word collating means compares the feature patterns bit by bit to thereby register the registered feature word group and the search feature word After the group is collated to determine the number of mismatch bits, the weight based on the volume data of each feature word is added to or multiplied by the mismatch bit number to calculate the weight mismatch bit number, and the registered feature word group and the search feature The position that best matches the word group is the position where the weighted mismatch bit number is the smallest, and the weight mismatch bit number is the smallest. And obtaining the minimum weighted number of inconsistent bits as said minimum dissimilarity.

  According to the fourth aspect of the present invention, the feature word has a feature pattern having a predetermined number of bits in which each component of the frequency spectrum obtained by analyzing the acoustic data is expressed by a bit value in a predetermined frequency unit, and volume data. By comparing the feature pattern bit by bit, the registered feature word group and the search feature word group are collated to obtain the number of mismatch bits, and the weight based on the volume data of each feature word is added to or multiplied by the mismatch bit number. Since the number of weighted mismatch bits is calculated and the minimum mismatch bit number that minimizes the weight mismatch bit number is set as the minimum dissimilarity, it is possible to perform high-precision collation processing taking into account fluctuations in volume data Become.

  In the fifth aspect of the present invention, in the acoustic data related information search apparatus according to any one of the first to fourth aspects of the present invention, the individual determination reference value corresponds to the registered feature word group in advance. Partial sound data is obtained by partially cutting out the original sound data, a unit section of a predetermined length is set for the partial sound data, and the partial sound is analyzed based on information obtained by analyzing the unit section. A set of feature words expressing the characteristics of the acoustic data is generated as a partial feature word group, and the registration feature word group and the partial feature word group are collated with each other while shifting the temporal positional relationship between the registered feature word group and the registration The feature word group and the partial feature word group are calculated on the basis of the minimum degree of difference which is the minimum degree of difference at the best matching position.

  According to the fifth aspect of the present invention, the individual determination reference value is information obtained by analyzing a unit section of a predetermined length with respect to partial sound data partially cut out from the original sound data corresponding to the registered feature word group. Based on the above, a set of feature words expressing the characteristics of the partial acoustic data is generated as a partial feature word group, the registered feature word group and the partial feature word group are collated, and the registered feature word group and the partial feature word group are Since the calculation is based on the minimum difference, which is the degree of the minimum difference at the most suitable position, an individual determination reference value suitable for each acoustic data can be obtained.

  According to a sixth aspect of the present invention, in the related information retrieval apparatus for acoustic data according to the fifth aspect of the present invention, the feature word is a bit of each component of the frequency spectrum obtained by analyzing the acoustic data in a predetermined frequency unit. The partial feature word group is generated for each of a plurality of partial sound data having different cut-out positions, and the registered features are compared in bit units. A word group and the partial feature word group are collated to determine the number of mismatch bits, and the position where the registered feature word group and the search feature word group are most suitable is set as the position where the number of mismatch bits is the minimum, and the minimum As the degree of difference, a minimum mismatch bit number that minimizes the mismatch bit number is obtained for each partial sound data, and each partial sound data Minimum mismatch based on the average value of the number of bits of, and obtains the individual judgment reference value.

  According to the sixth aspect of the present invention, the feature word has a feature pattern having a predetermined number of bits in which each component of the frequency spectrum obtained by analyzing the acoustic data is expressed by a bit value in a predetermined frequency unit, and the partial feature word A group is generated for each of a plurality of partial sound data whose cut-out positions are different from each other, and by comparing the feature pattern bit by bit, the registered feature word group and the partial feature word group are collated to obtain the number of mismatch bits. The minimum discrepancy bit number that minimizes the number is set as the minimum dissimilarity, and the individual judgment reference value is obtained based on the average value of this minimum discrepancy bit number, so the individual judgment reference value suitable for high-speed matching processing is obtained. Can be obtained.

  In the seventh aspect of the present invention, in the related information search apparatus for acoustic data according to the fifth aspect of the present invention, the feature word is a bit of each component of the frequency spectrum obtained by analysis of the acoustic data in a predetermined frequency unit. A feature pattern having a predetermined number of bits expressed by a value and volume data, and the partial feature word group is generated for each of a plurality of partial sound data having different cut-out positions, and the feature patterns are compared in bit units. The registered feature word group and the partial feature word group are collated to determine the number of mismatch bits, and the weight based on the volume data of each feature word is added to or multiplied by the mismatch bit number to calculate the weight mismatch bit number The position where the registered feature word group and the partial feature word group are most suitable is the position where the weighted mismatch bit number is the smallest. Position, the minimum weighted mismatch bit number that minimizes the number of weighted mismatch bits is determined for each partial acoustic data as the minimum dissimilarity, and based on the average value of the minimum mismatch bit numbers for each partial acoustic data, A determination reference value is obtained.

According to the seventh aspect of the present invention, the feature word has a feature pattern having a predetermined number of bits in which each component of the frequency spectrum obtained by analyzing the acoustic data is expressed by a bit value in a predetermined frequency unit, and volume data. A partial feature word group is generated for each of a plurality of partial sound data having different cut-out positions, and the feature pattern is compared bit by bit to check the registered feature word group and the partial feature word group to obtain the number of mismatch bits. After that, the weight based on the volume data of each feature word is added or multiplied to the number of mismatch bits to calculate the weight mismatch bit number, and the minimum weight mismatch bit number that minimizes the weight mismatch bit number is set as the minimum difference. ,
Since the individual determination reference value is obtained based on the average value of the minimum weighted mismatch bits, it is possible to obtain the individual determination reference value suitable for high-precision collation processing.

In the eighth aspect of the present invention, in the acoustic data related information search apparatus according to any one of the first to seventh aspects of the present invention, the search feature word generation means The feature word group is generated as H search feature word groups while changing the phase h, and the feature word collating means specifies the search feature word specifying one of the H phase h. The group is registered with the registered feature word group registered in the acoustic database as a first collation. As a result of the first collation, the corrected minimum dissimilarity is obtained from the first determination threshold (MM1). When the search feature word group is small, a second collation is performed while matching the search feature word group and the registered feature word group registered in the acoustic database while changing the phase h. As a result of the second collation, Above The original sound corresponding to the registered feature word group when the corrected minimum difference in phase h that best matches the recorded feature word group and the search feature word group is smaller than the second determination threshold (MM2). An apparatus for retrieving related information of acoustic data, characterized by identifying data.
It is characterized by that.

  According to the eighth aspect of the present invention, the search feature word group is generated in H ways while changing the phase h, and the search feature word group and the registered feature word group that specify one of the H phase phases h, Is collated as the first collation, and when the corrected minimum difference is smaller than the first determination threshold (MM1) as a result of the first collation, the search feature word group and the registered feature word group are collated. Is performed while changing the phase h. As a result of the second collation, the corrected minimum difference degree in the phase h that best matches the registered feature word group and the search feature word group is the second match. Since the original sound data corresponding to the registered feature word group is specified when the value is smaller than the determination threshold (MM2), the processing load is high when the characteristics of the search sound data and the original sound data are greatly different. Perform misalignment processing Ku, can be excluded from the search, it is possible to reduce the processing load on the search as a whole.

  In the ninth aspect of the present invention, in the acoustic data related information search apparatus according to any one of the first to eighth aspects of the present invention, the acoustic database further includes the registered feature word group for each original acoustic data. Representative registration feature data generated using the above-mentioned search feature data generation means for generating representative search feature data using the generated search feature word group, the representative search feature data, Representative feature data collating means for collating with representative registered feature data registered in the acoustic database, and the feature word collating means, as a result of collation by the representative feature data collating means, the representative search feature data Only when it is determined that the value obtained by correcting the difference between the representative registered feature data and the representative registered feature data using the individual determination reference value is within a predetermined range. The search characteristic word groups, and characterized in that for matching the registered feature word groups registered in the sound database.

  According to the ninth aspect of the present invention, the acoustic database registers representative registered feature data generated using a registered feature word group for each original acoustic data, and further uses a search feature word group. Generate representative search feature data, collate the representative search feature data with the representative registered feature data, and check the difference between the representative search feature data and the representative registered feature data as a result of the collation by the representative feature data matching means. The search feature word group and the registered feature word group are collated only when it is determined that the value corrected using the value is within the predetermined range, so the acoustic data registered in the acoustic database When searching for related information related to, the processing load is reduced by narrowing down the records to be matched using the rough characteristics of the acoustic data. It becomes possible.

  According to the present invention, when searching for related information related to acoustic data registered in the acoustic database using acoustic data such as music, when a plurality of records are matched, the priority is accurately determined. It has the effect that it becomes possible.

It is a hardware block diagram of a related information registration apparatus. It is a functional block diagram of a related information registration device. It is a flowchart which shows the production | generation process of a registration characteristic word. It is a conceptual diagram of the generation process of the characteristic word. It is a flowchart which shows the calculation process of the representative registration feature data based on a registration feature word. It is a conceptual diagram of the calculation process of the average value and standard deviation of the registration characteristic data array by S19 and S20. It is a flowchart which shows the calculation process of an individual determination reference value. It is a flowchart which shows the detail of the collation process of the partial area z which fixed (identified) the phase h of S130 of FIG. It is a flowchart which shows the detail of the calculation process of the sum mismatch bit number S (y, z, h, x) of S132 of FIG. It is a flowchart which shows the detail of the collation process of the partial area z which changed the phase h of S150 of FIG. It is a hardware block diagram of the related information search apparatus of the acoustic data which concerns on this invention. It is a functional block diagram of the related information search device for acoustic data according to the present invention. It is a figure which shows the outline | summary of a process of the search feature word production | generation means 50, the representative search feature data production | generation means 60, the representative feature data collation means 80, and the feature word collation means 90. FIG. It is a flowchart which shows the production | generation process of a search characteristic word. It is a flowchart which shows the calculation process of the representative search feature data based on a search feature word. It is a conceptual diagram of the calculation process of the average value of standard search feature data by S29 and S30, and a standard deviation. It is a figure which shows the concept of collation range determination. It is a figure which shows the concept of representative feature data. 5 is a flowchart of a search using a search feature word group by a feature word collating unit 90. It is a flowchart which shows the detail of the collation process of the record r in S240 of FIG. FIG. 21 is a flowchart showing details of a process of calculating a total mismatch bit number S (r, y, ho, x) in S242 of FIG. 20. It is a flowchart which shows the detail of the collation process in a record in S245 of FIG. It is a flowchart which shows the detail of the calculation process of sum total mismatch bit number S (r, y, h, x) in S322 of FIG.

<1. Sound data related information registration device>
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. First, registration of related information of acoustic data will be described. Registration of the related information of the acoustic data is performed by a registration device for the related information of the acoustic data (hereinafter referred to as “related information registration device”). The related information registration device creates a registered feature word and representative registered feature data from the original sound data, together with related information related to the sound data (generally called metadata), the created registered feature word, representative registered feature data, The individual determination reference value is associated with information (for example, an acoustic data ID) specifying acoustic data and registered in the acoustic database. The acoustic data ID, registered feature word, representative registered feature data, individual determination reference value, and related information related to one original acoustic data are registered in the acoustic database as one record. This acoustic database is used for searching related information of original sound data, and the original sound data itself is usually not registered. This is a copyright problem, and functionally, it is naturally possible to adopt a configuration in which the original sound data is registered.

  The acoustic data is recorded in a digital format such as music and voice, and is obtained by sampling an analog acoustic signal by a technique such as PCM. The original sound data is sound data of a sound material such as music to be searched. Because of copyright protection measures, PCM-format audio data with CD master disc quality is generally not distributed by license. Therefore, various irreversible data such as MP3 (MPEG-1 / Layer3) are pre-registered in the audio database. In general, an acoustic data file subjected to compression processing is given. However, even if the obtained data is in the MP3 format, in order to create the feature word, it is necessary to decompress the MP3 format data and generate the acoustic data of the sample sequence.

  FIG. 1 is a hardware configuration diagram of a related information registration apparatus. The related information registration device can be realized by a general-purpose computer. As shown in FIG. 1, a CPU 2a (CPU: Central Processing Unit), a main memory RAM 2b (RAM: Random Access Memory), and data A large-capacity data storage device 2c (for example, a hard disk), a program storage device 2d (for example, a hard disk) for storing programs executed by the CPU, and a key input I / F 2e such as a keyboard and a mouse And a data input / output interface 2f for data communication with an external device (data storage medium) and a display output interface 2g for sending information to a display device (display), which are connected to each other via a bus. ing.

  The program storage device 2d of the related information registration device is mounted with a dedicated program for operating the CPU 2a and causing the computer to function as the related information registration device. In addition, the data storage device 2c stores a registered feature word, representative registered feature data, individual determination reference value, and the like obtained as a processing result in association with related information, functions as an acoustic database, and performs various processes necessary for processing. Store the data.

  FIG. 2 is a functional block diagram of the related information registration apparatus. In FIG. 2, 10 is a registered feature word generating means, 15 is a partial feature word generating means, 20 is a representative registered feature data generating means, 25 is a representative partial feature data generating means, 30 is a judgment reference value calculating means, and 35 is a registering means. , 40 is an acoustic database. As described above, each unit is realized by the CPU 2a executing a dedicated program read from the program storage device 2d.

  The registered feature word generation unit 10 sequentially reads a predetermined number of samples as acoustic frames from the acoustic data, performs frequency analysis using the read acoustic frames, and generates a feature word expressing the features of the acoustic data have. This feature word expresses features of certain acoustic data with a small amount of data, and is composed of a feature pattern representing the features of the spectrum and volume data. It is required that the acoustic data cannot be reproduced, i.e., cannot be duplicated, and the feature word satisfies the condition, so registration in the acoustic database is permitted). A feature word registered in the acoustic database 40 is particularly called a “registered feature word”. In addition, the sound data on which the feature word is created is particularly referred to as “original sound data”. As this "original sound data", it is normal to use the "original" that has been modified to protect the copyright, not the original data itself owned by the copyright holders. It is. Of course, the data itself that is the “original” can also be used as the “original sound data”. As will be described later, in the present invention, other feature words such as a partial feature word and a search feature word appear. These are the same as the basic structure of the feature word, but the partial feature word. Unlike the registered feature word, the search feature word is provided with a set of a plurality of feature words (five types in the present embodiment) whose phases are shifted in order to correspond to collation in which phases are shifted in search processing described later. There is a difference. In this specification, the minimum unit of 40-bit configuration is called a “feature word”, and a set of feature words used for collation is called a “feature word group”.

  The partial feature word generation unit 15 sequentially reads a predetermined number of samples as acoustic frames for partial acoustic data, which is a part of the original acoustic data, in the same manner as the registered feature word generation unit 10, and uses the read acoustic frames. , It has a function of performing frequency analysis and generating a feature word representing the characteristics of the partial sound data. Both the registered feature word generation means 10 and the partial feature word generation means 15 are the same in that they read an acoustic frame from the same original sound data and create a feature word. The partial feature word generation means 15 performs processing on the whole (that is, from the beginning to the end), while the partial feature word generation means 15 has a plurality of short partial sections (same as the search feature words generated at the time of search). This is different in that a plurality of (five types in the present embodiment) partial sound data are generated by blindly cutting out and extracting a section of about 15 seconds and shifting the phase for each partial section. The number of partial sections to be extracted from one original sound data will be set in advance. For example, when extracting five pieces, the entire original sound data is 5 minutes long, If the length of the partial section is 15 seconds, the head (0 minutes) to 15 seconds, 1 minute 0 seconds to 1 minute 15 seconds, 2 minutes 0 seconds to 2 minutes 15 seconds, 3 minutes 0 seconds to 3 minutes 15 Five partial sections of seconds, 4 minutes to 4 minutes 15 seconds are extracted, and processing is performed on each partial acoustic data generated from the five partial sections.

  For the acoustic frame, the registered feature word generation unit 10 and the partial feature word generation unit 15 also acquire the same number of samples and perform a predetermined process to generate a feature word. The number of samples constituting the sound frame and the feature word generation method need to be the same in the registered feature word generation means 10 and the partial feature word generation means 15, but the partial feature words have partial sounds in the same partial section. There is a difference in that a plurality of sets of partial feature word groups whose phases are shifted based on data are generated.

  In addition, as described above, the registered feature word generation unit 10 and the partial feature word generation unit 15 generate a feature word by performing slightly different processing on the same number of sampled sound frames, but are generated as a minimum unit. The format of the 40-bit feature word is the same. However, as described above, the feature word created by the registered feature word generation unit 10 for the entire original sound data is “registered feature word”, and the feature word created by the partial feature word generation unit 20 for partial acoustic data is the target word. This is called “partial feature word”.

  The representative registered feature data generating unit 20 has a function of generating one representative registered feature data for one original sound data using the registered feature word group generated by the registered feature word generating unit 10. The registered feature word represents a partial feature of the original sound data, whereas the representative registered feature data represents the overall feature of one original sound data.

  The representative partial feature data generation unit 25 has a function of generating one representative partial feature data for one partial acoustic data using the partial feature word group generated by the partial feature word generation unit 15. The partial feature word represents a partial feature of partial acoustic data, while the representative partial feature data represents an overall feature of one partial acoustic data.

  The determination reference value calculation means 30 performs collation between one registered feature word group and a plurality of partial feature word groups, and one representative registered feature data and a plurality of representative partial feature data. At this time, since the number of feature words of each partial feature word group is significantly smaller than that of the registered feature word group, the temporal position is shifted between one registered feature word group and a plurality of partial feature word groups. While collating. Furthermore, since each partial feature word group is configured as a set of a plurality of types having different phases, it is necessary to collate each group individually. For example, a registered feature word group corresponds to a length of 5 minutes, there are five types of partial feature word groups corresponding to five types of 15-second partial sections, and each partial feature word group changes phase. 5 types of 15-second partial feature words are used, and the registered feature words with a length of 5 minutes and the time position overlap each other by 15/2 seconds. Thus, collation is performed with 5 × 5 × 300 × 2/15 = 1000 combinations. As a result of collation with these combinations, a value indicating the minimum difference is calculated for each of the five types of partial sections, and an individual determination reference value is calculated based on the average value thereof. Although details will be described later, for one original sound data, four individual determination reference values, one representative registered feature data, and a plurality of pieces of information are obtained by collation between one registered feature word group and a plurality of partial feature word groups. One individual determination reference value is calculated by collation with the representative partial feature data.

  The registration unit 35 includes a registered feature word group generated by the registered feature word generation unit 10, representative registration feature data generated by the representative registration feature data generation unit 20, and five types calculated by the determination reference value calculation unit 30. Manages the individual judgment reference values for the original original sound data, related information related to the copyright (generally called metadata), and the original sound data copyright information to identify the original sound data. It has a function of registering in the acoustic database 40 in association with an ID individually defined by a business operator. Here, related information refers to text information that identifies the song, such as the song name and genre name, and the copyright owner and copyright owner who are involved in the production of the original acoustic data such as the lyrics, composer, arranger name, artist name, producer name, etc. Indicates text information about the name. However, the original sound data itself is not normally registered in the sound database 40 due to restrictions on the copyright law. In addition, a series of binary-format material data (individual recording data before mixing down, MIDI input data), etc., used for production and mastering of the original sound data should be registered normally due to restrictions on copyright law. There is no.

<1.2. Processing operation of related information registration device>
Next, the processing operation of the related information registration apparatus shown in FIG. 2 will be described. First, in the related information registration device, the registered feature word generation means 10 generates a registered feature word from the designated original sound data. FIG. 3 is a flowchart showing a registration feature word generation process. First, the registered feature word generation means 10 reads the original sound data. In the related information registration device, the registered feature word generation means 10 reads a predetermined number of samples as one acoustic frame from the designated original acoustic data. The number of samples of one acoustic frame read by the registered feature word generation unit 10 can be set as appropriate, but is desirably about 4096 samples when the sampling frequency is 44.1 kHz. This corresponds to about 0.092 seconds. However, in consideration of the continuity between adjacent windows by using a Hanning window function in frequency conversion, which will be described later, the acoustic frame is read by overlapping a predetermined number of samples. In this embodiment, 2048 samples that are exactly half the section length of the acoustic frame are overlapped. Therefore, the first acoustic frame is sequentially read as samples 1 to 4096, the second acoustic frame is samples 2049 to 6144, and the third acoustic frame is samples 4097 to 8192.

  Subsequently, the registered feature word generation unit 10 performs frequency conversion on each read sound frame to obtain a frame spectrum that is a spectrum of the sound frame (S11). Specifically, frequency conversion is performed on the acoustic frame read by the registered feature word generation means 10 using a window function. As frequency conversion, Fourier transform, wavelet transform, and other various known methods can be used. In the present embodiment, a case where Fourier transform is used will be described as an example.

  Here, a window function used for Fourier transform in the present embodiment will be described. In general, when Fourier transform is performed on a predetermined signal, it is necessary to divide the signal into predetermined lengths. In this case, the Fourier transform (precisely, for a short time) is performed on a signal having a predetermined length. When called (Fourier transform), a pseudo component is generated in the high frequency region. Therefore, in general, when performing Fourier transform, the signal value is changed so that the weight is dropped by the cosine wave shape at the boundary of the window using a window function called a Hanning window, and then the value after the change is changed. To perform the Fourier transform.

  When the Fourier transform is performed in S11, specifically, the value X (i) (i = 0,..., N−1) in the sample i has a real value of 0 to 1 and is defined in the N sample section. Using the Hanning window function W (i) (= 0.5−0.5 cos (2πi / N)), the processing according to the first and second formulas of [Formula 1] Real part A (j) and imaginary part B (j) are obtained.

  Subsequently, a spectral component is calculated (S12). Specifically, the processing according to the third equation of the following [Equation 1] is performed to obtain the intensity value E (j) at each frequency.

[Formula 1]
A (j) = Σ _{i = 0,..., N-1} W (i) · X (i) · cos (2πij / N)
B (j) = Σ _{i = 0,..., N-1} W (i) · X (i) · sin (2πij / N)
E (j) = A (j) ² + B (j) ²

  In [Formula 1], i is a serial number assigned to N samples in each acoustic frame, and takes an integer value of i = 0, 1, 2,..., N−1. Further, j is a serial number assigned in order from the smallest value of the frequency value, and takes an integer value of j = 0, 1, 2,..., N / 2-1. When the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by 10.8 Hz.

  Subsequently, an integration process of spectrum components is performed (S13). Although the spectral component up to about 22 kHz is obtained by the above frequency conversion, the spectral component in the range of 340 Hz or higher and lower than about 4 kHz is used for generation of the feature word in this embodiment. This is in order to correspond to a voice compression format such as 3GPP standard used for voice reproduction of a cellular phone (however, since digital acoustic data is always given in this embodiment, a voice recording signal of a cellular phone is used. There is no need to deal with matching.) For this reason, in order to accurately match the voice reproduction range of the mobile phone, the upper limit of the feature word generation may be set to around 3.4 kHz. In the present embodiment, j = 385 to 2047 higher than the vicinity of 4 kHz among the frequency components of j = 0 to 2047 is not used. Also, the low frequency component of j = 0 to 32 that is 340 Hz or less is not used. That is, in the present embodiment, j = 33 to 384 frequency components are used. Specifically, processing according to the following [Equation 2] is executed and integrated into Pn in units of 11 frequency components.

[Formula 2]
_{P0 = (E 33 + E 34} + ... + E 43) 1/4
_{P1 = (E 44 + E 45} + ... + E 54) 1/4
:
:
P31 = ( _E374 + _E375 + ... + _E384 ) ^1/4

  According to the above [Equation 2], 352 frequency components of j = 33 to 384 are integrated into 32 frequency components of n = 0 to 31. The above process is performed for each acoustic frame, and 32 frequency components are obtained for each acoustic frame.

  Next, for each acoustic frame, a difference from the spectral component of the immediately preceding acoustic frame is calculated (S14). The processes of S11 to S13 are sequentially performed on each acoustic frame. The inter-frame difference calculation processing in S14 uses P0 to P31 obtained as a result of performing the processing up to S13 for each acoustic frame. Specifically, processing according to the following [Equation 3] is performed to obtain an inter-frame difference Dn (t).

[Formula 3]
Dn (t) = | Pn (t) −Pn (t−1) |, n = 0,..., 31

  In the above [Expression 3], Pn (t) is an integrated frequency component in the t-th acoustic frame. In this way, the difference between adjacent acoustic frames is calculated by emphasizing changes in the amplitude level and reflecting the characteristics of the acoustic data even in places where the amplitude level of the acoustic data slightly changes. It is for producing | generating.

  When the inter-frame difference calculation process is completed, it is determined whether the predetermined number of frames have been processed (S15). Specifically, it is determined whether t ≧ T. As a result, if t <T, t is incremented and the process returns to S11. If t ≧ T as a result of the determination in S15, the total sum of the obtained T differences Dn (t) is obtained (S16). That is, when the processes of S11 to S14 are sequentially performed on each acoustic frame and T differences Dn (t) between the acoustic frames are obtained (11 in this embodiment), T differences Dn (t) are obtained. Will be calculated. Specifically, processing according to the following [Equation 4] is performed to obtain a sum Sn of interframe differences.

[Formula 4]
Sn = Σ _{t = 0,..., T-1} Dn (t)

In the above [Equation 4], “Σt _{= 0,..., T−1} ” means the total sum when t is increased by 1 from t = 0 to T−1. Subsequently, the binarization process of Sn obtained by the above [Equation 4] is performed (S17). Specifically, first, the 32 Sn arrays are divided into two in the upper and lower bands of n ≧ 14 and n ≦ 13, and 1 is given to 7 of the 14 values of n ≦ 13, and 7 of the smaller value. In addition to giving 0 to each, 1 is given to 9 having a large value among 18 pieces of n ≧ 14, and 0 is given to 9 having a small value. Here, instead of simply giving 1 and 0 to 16 large values and 16 small values out of all 32 Sn, 32 bands are grouped into 18 bands with high frequency and 14 bands with low frequency. The reason why 1 and 0 are equally given in each group is to correct the influence of the frequency characteristics associated with various data compression processes. The reason why the upper and lower bands are divided at the positions of the 18 band and the 14 band is that, as a result of the experiment, when divided at this position, the search acoustic data used for the search is subjected to various data compression processes such as MP3, This is because the search accuracy was the highest. By the processing in S17, the sum Sn of inter-frame differences for each frequency band n can be expressed by 1 bit. Then, a 32-bit feature pattern Fd (y) is obtained with n = 0 as LSB and n = 31 as MSB. Here, y (= 0,..., Y−1) is a variable indicating the order of Y feature words generated from one original sound data. Therefore, y is a variable proportional to the time from the start of performance.

  Next, the volume data is calculated (S18). Specifically, first, the total volume Vol is calculated using the following [Formula 5].

[Formula 5]
Vol = Σ _{t = 0, ..., T-1} {Σ _{n = 0, ..., 31} Pn (t)}

In the above [Equation 5], “Σ _{t = 0,..., T−1} ” means the total when t is increased by 1 from t = 0 to T−1. As shown in the above [Formula 5], the values of all the components Pn (t) obtained by the integration process are added for T acoustic frames. As a result, a total volume Vol that is the total volume of the T acoustic frames is obtained. Multiplication volume is multiplied by a fixed scaling value set appropriately, and normalization is performed so as to be within the range of 0 to 255 to obtain volume data Vd (y). The sound volume data Vd (y) is represented by 8 bits by normalization. Since the volume data Vd (y) is based on the total volume Vol over T acoustic frames as shown in [Formula 5], the volume data Vd (y) is not the volume of each frame unit, but the total of T acoustic frames. It represents the volume.

  The processes of S17 and S18 can be performed by changing the order. As a result of the processing in S17 and S18, a 40-bit feature word composed of a 32-bit feature pattern and 8-bit volume data is obtained.

  By executing the above processing for each sound frame, a large number of feature words for the sound data are generated. For example, if the sampling frequency is 44.1 kHz, the sound frame is 4096 samples, and the sound frame is overlapped by 2048 samples as in the above example, one feature word is approximately 0.506 seconds, and the sound data from 5 minutes Will generate approximately 600 feature words.

  Here, the process of generating the feature word will be described with reference to the conceptual diagram of FIG. FIG. 4A is a diagram illustrating a waveform of acoustic data that is a generation target of a feature word. In the related information registration apparatus, the acoustic data is read in units of acoustic frames. However, as shown in FIG. Then, for each acoustic frame, as shown in FIG. 4C, frequency components in a predetermined frequency range are extracted and integrated into 32 bands. This corresponds to S11 to S13. Next, as shown in FIG. 4D, a difference process between adjacent acoustic frames for each band of the integrated component and a total process of the 32 band integrated components are performed. The integrated component difference process corresponds to S14, and the integrated component sum process corresponds to the sum process of each frequency component (n = 0 to 31) in S18. Next, as shown in FIG. 4E, a summation process of 32-band difference components and a summation process of volume are performed. The summation process of the 32-band difference component corresponds to S16, and the volume summation process corresponds to the summation process of each acoustic frame (t = 0 to T-1) in S18. Next, as shown in FIG. 4 (f), the binarization process of the 32-band sum component and the compression process of the sound volume (256 levels obtained by multiplying the value calculated based on the above [Formula 5] by a predetermined scaling value). To the volume level). The binarization process for the 32-band sum component corresponds to S17, and the volume compression process corresponds to S18. As shown in FIG. 4, a process of sequentially reading sound frames from sound data and generating one feature word in units of T sound frames is performed.

  When a registered feature word group (a set of registered feature words) is obtained for the entire specified original sound data, the representative registered feature data generation unit 20 then performs a representative registration using the registered feature word group. Generate feature data. The representative registered feature data is configured as a set of an average value and a standard deviation in the time direction of feature components based on the registered feature word. FIG. 5 is a flowchart showing a calculation process of an average value and a standard deviation in the time direction of feature components based on registered feature words. First, the representative registered feature data generation means 20 generates a registered feature data array by multi-value processing the registered feature word (S19). Specifically, by executing processing according to the following [Equation 6], a registered feature data array Q (n, y) (n = 0,..., 31) that is a feature component based on the registered feature word. Y = 0,..., Y-1) is generated.

[Formula 6]
When each bit n of Fd (y) is 1, Q (n, y) ← Vd (y)
When each bit n of Fd (y) is 0, Q (n, y) ← −Vd (y)

  Next, the representative registered feature data generation means 20 calculates an average value array Cd (n, r) and a standard deviation array Ld (n, r) in the time direction y of the registered feature data array Q (n, y) ( S20). Specifically, the average value array Cd (n, r) and the standard deviation array Ld (n, r) are calculated by executing processing according to the following [Equation 7].

[Formula 7]
Cd (n, r) = [Σ _{y = 0,..., Y-1} Q (n, y)] / Y
Ld (n, r) = [Σ _{y = 0,..., Y-1} (Q (n, y) −Cd (n, r)) ² / Y] ^1/2

  FIG. 6 is a conceptual diagram of the process of calculating the average value and standard deviation of the registered feature data array in S19 and S20. As shown in FIG. 6, the registered feature word has a 32-bit feature pattern F (y) and 8-bit volume data V (y) corresponding to each time y (y = 0,..., Y−1). doing. In FIG. 6, 32 bits of the feature pattern are indicated by Bit0 to Bit31.

  Then, depending on whether the value of bit n (Bit 0 to Bit 31) is 0 or 1 as shown in [Formula 6], the value of Q (n, y) is left as the value of the volume data, It is determined whether the value of the volume data is multiplied by −1, and the value of Q (n, y) corresponding to bit n is determined. At this time, since 8-bit volume data may be a negative value, Q (n, y) is expressed by 16 bits. In the 32-bit feature pattern, what is represented by 1 bit for each band is represented by 16 bits (2 bytes), so the registered feature data at each time y is 64 bytes.

  The average value array Cd (n, r) and the standard deviation array Ld (n, r) generated as representative registration feature data by the representative registration feature data generation means 20 are stored as acoustic data such as an acoustic data ID by the registration means 35. The information is registered in the acoustic database 40 in association with information to be identified (corresponding to r one-to-one).

  When the registered feature word group and the representative registered feature data are obtained for the entire designated original sound data, the partial feature word generation unit 15 then selects a section of a predetermined length from the beginning of the original sound data. Are sequentially extracted as partial sections, and a feature word (partial feature word) is generated for each extracted partial sound data. The length of the partial section can be set as appropriate. The partial section is extracted blindly (equally spaced in the actual embodiment) from the given original sound data, assuming a typical section to be used at the time of search, and some leakage has occurred. However, it is set at a constant interval from the beginning so that there is no overlap, and Z partial sections are obtained for one original sound data. The generation of the partial feature words is basically created by the procedure and concept as shown in FIGS. 3 and 4 as in the case of the registration feature words generated for the entire original sound data. . However, unlike the registered feature words, it is necessary to additionally generate a plurality of partial feature word groups whose phases are shifted.

  Since the partial sound data is a piece of music cut out, the timing does not necessarily coincide with the original sound data, and a positional shift may occur. The feature word is generated by averaging T sound frames (T = 11 in the present embodiment), and is relatively resistant to displacement. However, in the case of original sound data with a sharp change in rhythm, a feature word that is remarkably different is generated when it is shifted by about 5 sound frames, which is almost half of 11 sound frames, which is a feature word generation unit, and is compared with a registered feature word. A significant difference will occur. Therefore, a plurality of partial feature words are generated by delaying (changing the phase) several acoustic frames (in this embodiment, two acoustic frames) within a range of 11 acoustic frames, which is one generation unit of feature words, The database 40 collates with a registered feature word to be registered.

  In the case of generating the partial feature word group, instead of calculating Sn based on the above [Formula 4] in S16 of FIG. 3, H = 0,... 8]. hs is a suitable positive integer, and hs = 2 in this embodiment.

[Formula 8]
Sn (h) = Σ _{t = 0,..., T-1} Dn (t + h · hs)

  Subsequently, in S17, binarization processing is similarly performed on each of the H types of Sn (h), and a 32-bit feature pattern F (z, h) in which n = 0 is LSB and n = 31 is MSB. , X). Here, z (= 0,..., Z−1) is a variable indicating the order in Z pieces of partial sound data obtained from one original sound data. x (= 0,..., X−1) is a variable indicating the order of X feature words generated from one partial sound data. Therefore, x is a variable proportional to the time from the start of performance.

  Furthermore, in S18, instead of calculating Vol based on the above [Formula 5], H types of Vol (h) are similarly calculated based on [Formula 9] below with h = 0,..., H−1. Similarly, hs is a suitable positive integer, and hs = 2 in this embodiment.

[Formula 9]
Vol (h) = Σ _{t = 0,..., T-1} {Σ _{n = 0,..., 31} Pn (t + h · hs)}

  The value of the total volume Vol (h) is multiplied by a fixed scaling value appropriately set, and normalized so as to fall within the range of 0 to 255 to obtain volume data V (z, h, x). The sound volume data V (z, h, x) is represented by 8 bits by normalization. Since the volume data V (z, h, x) is based on the total volume Vol (h) over T acoustic frames as shown in [Formula 9], the volume data V (z, h, x) This represents the total volume of T acoustic frames.

  From the above processing, assuming that each of Z partial sections (partial acoustic data) has a length of 15 seconds, there are 30 × H partial feature words per partial acoustic data (in the present embodiment). H = 5) will be generated.

  Once a partial feature word group (a set of partial feature words) is obtained for the entire specified original sound data, the representative partial feature data generation means 25 then uses the partial feature word group to represent the representative part. Generate feature data. The representative partial feature data is configured as a set of an average value and a standard deviation in the time direction of the feature component based on the partial feature word. The generation of the representative partial feature data is performed in the same manner as the generation of the representative registered feature data shown in FIG. However, unlike the representative registered feature data, it is necessary to additionally generate a plurality of representative partial feature data whose phases are shifted.

  In the case of generating a partial feature word group, the representative partial feature data generation unit 25 generates h = 0 instead of generating the registered feature data array Q (n, y) based on the above [Equation 6] in S19 of FIG. ,..., H−1, a partial feature data array Q (z, n, h, x) is generated based on [Formula 10] below.

[Formula 10]
When each bit n of F (z, h, x) is 1, Q (z, n, h, x) ← V (z, h, x)
When each bit n of F (z, h, x) is 0, Q (z, n, h, x) ← −V (z, h, x)
(Z = 0, ..., Z-1; n = 0, ..., 31; h = 0, ..., H-1; x = 0, ..., X (z, h) -1)

  Next, the representative partial feature data generation means 25 executes a process according to the following [Formula 11] instead of the above [Formula 7] in S20 of FIG. , Z, h, x) in the time direction x and the phase h, the average value array C (z, n) and the standard deviation array L (z, n) are calculated.

[Formula 11]
C (z, n) = [Σh _{= 0,..., H-1, x = 0,..., X (z, h) -1} Q (z, n, h, x)] / (H.X)
L (z, n) = [Σh _{= 0,..., H-1, x = 0,..., X (z, h) -1} (Q (z, n, h, x) -C (z, n , H, x)) ² / (H · X)] ^1/2

  The average value array C (z, n) and the standard deviation array L (z, n) generated as the representative partial feature data by the representative partial feature data generation means 25 are used only for calculating the individual determination reference value, It is not registered in the acoustic database 40.

  When the registered feature word group, the representative registered feature data, the partial feature word group of each partial section, and the representative partial feature data are obtained for a certain original sound data, the determination reference value calculating unit 30 then performs the representative registration. Using the feature data, the representative partial feature data of each partial section, the registered feature word group, and the partial feature word group of each partial section, an individual determination reference value for the original sound data is calculated. This individual determination reference value calculation process will be described with reference to the flowcharts of FIGS.

7 to 10, each variable is defined as follows.
[Registration feature word]
Y: Number of registered feature words registered for original sound data Fd (y): Feature pattern array (y = 0,..., Y-1), 32 bits Vd (y): Volume data array (y = 0,... , Y-1), 8 bits Cd (n): Y average value of registered feature data array Q (n, y) Ld (n): Standard deviation of registered feature data array Q (n, y)

[Partial feature word]
Z: Number of partial sections set in original sound data (number of partial sound data)
X (z, h): Number of partial feature words in a phase h (h = 0,..., H−1) with a partial interval z (z = 0,..., Z−1) F (z, h, x): Feature pattern array (x = 0,..., X (z, h) -1), 32-bit V (z, h, x): Volume data array (x = 0,..., X (z, h) -1) , 8 bits C (z, n): H × X average value of partial feature data array Q (z, n, h, x) L (z, n): Partial feature data array Q (z, n, h) , X) standard deviation

[Collation variable]
W: Number of collation words (w = 0,..., W−1), number of registered feature words and partial feature words to be collated (eg, W = 6)
U (z): representative feature data distance D (y + w, z, h, x + w): number of mismatch bits in word unit of feature pattern (0 to 32)
S (y, z, h, x): the sum of the number of unmatched bits, the number of matching word numbers D (y + w, z, h, x + w) (Σw _{= 0,..., W) -1} D (y + w, z, h, x + w))

[Individual judgment reference value]
Mu: Individual determination reference value of representative feature data distance M1: Individual determination reference value of total mismatch bit number specifying phase M2: Individual determination of total mismatch bit number S (r, y, h, x) by changing phase Reference value Mw1: Individual determination reference value for the number of mismatched bits in word unit with phase specified Mw2: Reference value for individual determination of the number of mismatched bits in word unit with phase changed

  FIG. 7 is a flowchart showing the calculation process of the individual determination reference value. First, initial setting is performed (S110). Specifically, the sum Su = 0 of the representative feature data distance U (z), the sum Sw1 = 0 of the word unit mismatch bits specifying the phase, the sum Sw2 = 0 of the word unit mismatch bits changing the phase, The sum S1 = 0 of the total number of mismatch bits specifying the phase, the sum S2 = 0 of the number of word unit mismatch bits changing the phase, and the variable (section number) z = 0 for specifying the partial section are set.

  Subsequently, the determination reference value calculation unit 30 calculates the representative feature data distance U (z) in the partial section z (S120). Specifically, processing according to the following [Equation 12] is executed to calculate a representative feature data distance U (z) that is a distance between the representative registered feature data and the representative partial feature data for the partial section z.

[Formula 12]
U (z) = 256 × [Σ _{n = 0,..., 31} {(Cd (n) −C (z, n)) / (Ld (n) + L (z, n))} ² ] ^1/2 / [Σ _{n = 0,..., 31} {Cd (n) / (Ld (n) + L (z, n))} ² ] ^1/4 / [Σ _{n = 0,..., 31} {C (z, n) / (Ld (n) + L (z, n))} ² ] ^1/4

In the above [Equation 12], there are three terms enclosed by []. The square root of the first [] is divided by the fourth root of the second [] and the fourth root of the third []. Is multiplied by a coefficient “256” for normalization. The coefficient for normalization can be appropriately changed according to the range of normalization. In the above [Expression 12], “Σ _{n = 0,..., 31} ” means the total of 32 pieces when n is incremented by 1 from n = 0 to 31. As the power root of each term enclosed in [], the specific power can be changed as appropriate, but in this embodiment, as shown in [Formula 12], the square root, the fourth root, the fourth power Roots.

[Equation 12] indicates the distance between the representative partial feature data and the representative registered feature data, and is defined based on the normalized correlation coefficient of both. Specifically, each element n of the representative partial feature data is represented by the sum of the standard deviation value of the partial feature data array and the standard deviation value of the registered feature data array with respect to the average value of the partial feature data array. Each element is defined as a normalized value divided by (C (z, n) / (Ld (n) + L (z, n)). Similarly, each element of the representative registered feature data is registered as a feature. The sum of the standard deviation value of the partial feature data array and the standard deviation value of the registered feature data array with respect to the average value of the data array is defined as each value normalized by dividing each of the 32 elements, and Cd ( n) / (Ld (n) + L (z, n)) The normalized correlation coefficient between these 32 elements is [Σ _{n = 0,..., 31} {(Cd (n) −C (z, n)) / ( Ld (n) + L (z, n))} 2] / [Σ n = 0, ..., 31 {Cd (n) / (Ld (n) + L (z, n)) _{^{2] 1/2 / [Σ n =}} 0, ..., 31 {(C (z, n) / (Ld (n) + L (z, n))} 2] is given by ^1/2, -1 to +1 In the above [Equation 12], the normalized correlation coefficient value is multiplied by a predetermined integer value 256 to give an integer expression, and the fluctuation range of the integer value is expanded to show the corresponding record and the non-corresponding record. In order to make the difference, the normalized correlation coefficient value is used as it is, whether it is the square root or the fourth root. Just choose.

  Subsequently, the phase h = ho is fixed (specified) and the partial feature word group generated from the partial section z is collated with the registered feature word group (S130). As for ho, when H = 5, it can be set to any of ho = 0, 1, 2, 3, and 4, but it is set to ho = 0 with the least amount of normal arithmetic processing. As a result of the collation in S130, the minimum sum mismatch bit number S (ymin, z, ho, xmin) is obtained for the partial section z. The process of S130 will be described later. When the minimum sum mismatch bit number S (ymin, z, ho, xmin) is obtained, a process of adding to the sum S1 of the sum mismatch bits is performed (S140). In the process of S130, the minimum word unit mismatch bit number D (ymin + wmax, z, ho, xmin + wmax) is also calculated. In S140, the minimum word unit mismatch bit number D (ymin + wmax) , Z, ho, xmin + wmax) is also performed.

  Subsequently, the partial feature word group generated from the partial section z by changing the phase h is collated with the registered feature word group (S150). As a result of the collation in S150, the minimum sum mismatch bit number S (ymin, z, hmin, xmin) when the phase is the best for the partial section z is obtained. Details of the processing in S150 will also be described later. When the minimum sum mismatch bit number S (ymin, z, hmin, xmin) is obtained, a process of adding to the sum S2 of the sum mismatch bits is performed (S160). In the process of S150, the minimum word unit mismatch bit number D (ymin + wmax, z, hmin, xmin + wmax) is also calculated, but in S160, the minimum word unit mismatch bit number D (ymin + wmax) , Z, hmin, xmin + wmax) is also performed.

  Next, the variable z specifying the partial section is incremented, that is, increased by 1 (S170). Then, it is determined whether or not the variable z specifying the partial section has reached the number of partial sections Z (S180). If not, the process returns to S120, and the next partial section z is collated. When processing is performed for each partial section z and the variable z reaches the number of partial sections Z, that is, processing for all Z partial sections is completed, so five types of individual corresponding to one original sound data A determination reference value is determined (S170).

  Of the five types of individual determination reference values, Mu, M1, and M2 are calculated by executing processes according to the following [Equation 13] to [Equation 15], respectively.

[Formula 13]
Mu = Σ _{z = 0, ..., Z-1} U (z) / Z

[Formula 14]
M1 = Σ _{z = 0,..., Z-1} S (ymin, z, ho, xmin) / Z

[Formula 15]
M2 = Σz _{= 0,..., Z-1} S (ymin, z, hmin, xmin) / Z

  Then, the maximum value D (ymin + wmax, z, ho, xmin + wmax) of D (y + w, z, ho, x + w) constituting S (ymin, z, ho, xmin) is given as the individual determination reference value Mw1. Further, the maximum value D (ymin + wmax, z, hmin, xmin + wmax) of D (y + w, z, h, x + w) constituting S (ymin, z, hmin, xmin) is given as the individual determination reference value Mw2.

  Next, details of the collating process of the partial section z in which the phase h is fixed in S130 of FIG. 7 will be described using the flowchart of FIG. In S130, after h = ho is set, the process proceeds to the process shown in FIG. First, initial setting is performed (S131). Specifically, the minimum mismatch bit number Smin1 = initial value Big1, a variable x = 0 specifying a partial feature word, a variable y = 0 specifying a registered feature word, and x when the number of mismatch bits is minimized A variable xmin = −1 and a variable ymin = −1 that specifies y when the number of mismatch bits is minimized are set. The initial value Big1 may be a value that is sufficiently larger than the value that the total mismatch bit number S (y, z, h, x) can take, and is set in advance.

  Subsequently, the sum mismatch bit number S (y, z, h, x) is calculated (S132). Since an error value may be output for the sum mismatch bit number S (y, z, h, x) as described later, the sum mismatch bit number S (y, z, h, x) is obtained as a normal value. If it is determined, it is determined whether or not the minimum mismatch bit number Smin1 is smaller (S133). Only when S (y, z, h, x) is equal to or smaller than Smin1, a process of setting the value of S (y, z, h, x) to Smin1 is performed (S134). Then, the variable x is incremented (S135), the processes of S132 to S134 are repeated, and when the variable x specifying the partial feature word reaches the partial feature word number X (z, h) (S136), x = 0. The variable y specifying the registered feature word is incremented (S137), and the processing of S132 to S136 is repeated. That is, first, the registered feature word y is fixed, the partial feature word and the registered feature word are compared, and when the processing is completed for all the partial feature words, the registered feature word is changed, and the next registered feature word is changed. Process. In this way, processing is performed for all registered feature words.

  Then, it is determined whether or not y has reached the total number Y of registered feature words (S138). If not, the process returns to S132 to calculate the minimum mismatch bit number Smin1 for the next registered feature word y. Do. When processing is performed for each registered feature word y and the variable y reaches the total number Y of registered feature words, that is, when processing for all Y registered feature words is completed, the total number of mismatch bits S (y, z at that time) , H, x) is output as the minimum mismatch bit number Smin in the partial section z when the phase is fixed. This minimum mismatch bit number Smin1 is obtained in S130 of FIG.

  Next, details of the calculation processing of the total mismatch bit number S (y, z, h, x) in S132 of FIG. 8 will be described using the flowchart of FIG. In this apparatus, at the time of collation, collation of W consecutive feature words, which is the number of collation words, is performed. That is, the consecutive W registered feature words and the consecutive W partial feature words are collated in order from the top. In FIG. 9, first, initial setting is performed (S181). Specifically, the sum mismatch bit number S (y, z, h, x) = 0, variables w = 0, Dmax = 0, and wmax = 0 indicating the number of feature word collations are set. After the initial setting, it is determined whether or not the condition that the volume data Vd (y + w) of the registered feature word and the volume data V (z, h, x + w) of the partial feature word are both greater than 0 is satisfied (S182). The volume data Vd (y + w) of the registered feature word and the volume data V (z, h, x + w) of the partial feature word follow [Equation 5] and [Equation 9] for the whole original sound data and the partial sound data, respectively. Are calculated as “Vol” and “Vol (h)”. As a result of the determination in S182, only when one of the registered feature words and the partial feature word volume data Vd (y + w) and the partial feature word volume data V (z, h, x + w) satisfy the condition that both are greater than 0, The word unit mismatch bit number D (y + w, z, h, x + w), which is the number of mismatch bits when one feature word is compared, is calculated (S183).

  Next, the word unit mismatch bit number D (y + w, z, h, x + w) is compared with a predetermined value Dmax (S184). As a result of the determination in S184, the value of the word unit mismatch bit number D (y + w, z, h, x + w) is set to Dmax only when the word unit mismatch bit number D (y + w, z, h, x + w) is larger than the predetermined value Dmax. (S185). Then, the value of w is set to wmax. Subsequently, a process of adding the word unit mismatch bit number D (y + w, z, h, x + w) to the total mismatch bit number S (y, z, h, x) is performed (S185). In S185, the variable w indicating the number of feature words to be collated is incremented and the feature word w to be collated is changed to the next feature word w. Regardless of the result of the determination in S184, the process of S186 is executed.

  Then, it is determined whether or not the variable w indicating the number of collated feature words has reached the predetermined number W (S187). If not, the process returns to S182 and the next feature word is processed. When processing is performed for each feature word and the variable w reaches a predetermined number W, the total mismatch bit number S (y, z, h, x) at that time is calculated as the total mismatch bit number for a certain x, h, y. Output as S (y, z, h, x). This sum mismatch bit number S (y, z, h, x) is obtained in S132 of FIG. When it is determined in S182 that the condition is not satisfied, a calculation error value (negative value) for the number of unmatched bits is output.

  Next, details of the collating process of the partial section z in which the phase h is changed in S150 of FIG. 7 will be described using the flowchart of FIG. First, initial setting is performed (S151). Specifically, the minimum mismatch bit number Smin2 = initial value Big2, the variable h = 0 for specifying the phase, and the variable hmin = −1 for specifying h when the mismatch bit number is minimized are set. The initial value Big2 may be any value that is sufficiently larger than the value that the total mismatch bit number S (ymin, z, h, xmin) can take, and is set in advance.

  Subsequently, for the specified phase h, the partial feature word group generated from the partial section z is collated with the registered feature word group (S152). The specific process of S152 is executed according to FIG. 8 as in S130. As a result of the collation in S152, the minimum sum mismatch bit number S (ymin, z, h, xmin) is obtained for the partial interval z in the specified phase h.

  When the minimum total mismatch bit number S (ymin, z, h, xmin) is obtained, the comparison with the minimum mismatch bit number Smin2 is performed (S153). Only when S (ymin, z, h, xmin) is equal to or smaller than Smin2, a process of setting the value of S (ymin, z, h, xmin) to Smin2 is performed (S154). Then, the variable h is incremented (S155), and the processes of S152 to S154 are repeated. That is, for each phase h, the processes of S152 to S154 are repeated to obtain the minimum sum mismatch bit number S (ymin, z, h, xmin) of each phase h, and the smallest one is the minimum mismatch bit number Smin2. Perform the process.

  It is determined whether or not h has reached the total number of phases H (S156). If not, the process returns to S152, and the minimum number of unmatched bits S (ymin, z, h, xmin) for the next phase h. The calculation process is performed. When the processing is executed for each phase h and the variable h reaches the total number of phases H, that is, when the processing for all the H phases is completed, the minimum sum mismatch bit number S (ymin, z, h, xmin) at that time The minimum value Smin2 is output as the minimum mismatch bit number Smin2 in the partial interval z at the time of phase change. This minimum mismatch bit number Smin2 is obtained in S150 of FIG.

<1.3. Calculation of word unit mismatch bit count>
The calculation of the word unit mismatch bit number D (y + w, z, h, x + w) in S183 in FIG. 9 will be described. Regarding the calculation of the word unit mismatch bit number D (y + w, z, h, x + w), the specific processing contents differ depending on the sound volume determination mode set by the user. There are four volume determination modes: Off, Weight, Match, and Both.

<1.3.1. Volume judgment mode “Off”>
The sound volume determination mode “Off” is a mode in which no weight is added. When the sound volume determination mode “Off” is set, the word unit mismatch bit number D (y + w, z, h, x + w) is the difference in word units as it is. Is a word unit dissimilarity D (y + w, z, h, x + w) indicating the degree of. In the sound volume judgment mode “Off”, the initial value is set with the word unit mismatch bit number D (y + w, z, h, x + w) = 0, and then 32 bits of Fd (y + w) and F (z, h, x + w). Are sequentially compared in corresponding bit units, and 1 is added to D (y + w, z, h, x + w) every time the bits differ. In both the registered feature word group and the partial feature word group, the feature pattern of the feature word is created according to the same rule, and has a 32-bit configuration with the low frequency component as LSB and the high frequency component as MSB. This can be done depending on whether or not the bit values match.

<1.3.2. Volume judgment mode “Weight”>
The volume determination mode “Weight” is a mode for adding a weight, and when the volume determination mode “Weight” is set, the initial value is set with the word unit mismatch bit number D (y + w, z, h, x + w) = 0. After that, 32 bits of Fd (y + w) and F (z, h, x + w) are sequentially compared in corresponding bit units. Based on the result of the comparison, processing according to the following [Equation 16] is executed to determine the value of D (y + w, z, h, x + w). As a result, D (y + w, z, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account volume data. In this mode, when the registered feature word is matched with the search feature word, the search acoustic data that is the basis of the search feature word is accompanied by signal processing such as analog conversion. Appropriate matching results are given when both absolute values are different.

[Formula 16]
When the Fd (y + w) side is bit 1 and the F (z, h, x + w) side is bit 0,
D (y + w, z, h, x + w) ← D (y + w, z, h, x + w) + Vd (y + w) · 2 / {Vd (y + w) + V (z, h, x + w)}
When the Fd (y + w) side is bit 0 and the F (z, h, x + w) side is bit 1,
D (y + w, z, h, x + w) ← D (y + w, z, h, x + w) + V (z, h, x + w) · 2 / {Vd (y + w) + V (z, h, x + w)}

<1.3.3. Volume judgment mode “Match”>
When the sound volume determination mode “Match” is set, first, processing in the sound volume determination mode “Off” is performed to obtain D (y + w, z, h, x + w). Then, by executing processing according to the following [Equation 17], the weight is multiplied to determine the value of D (y + w, z, h, x + w). As a result, D (y + w, z, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account the difference in the fluctuation pattern of the volume data. This mode has no meaning in collation between registered feature words and partial feature words generated from the same original sound data, but when collating registered feature words and search feature words, which will be described later, search feature words The search acoustic data that is the basis of the above is accompanied by signal processing such as various data compression, and there is not much difference between the relative change of the original sound data and the volume, but an appropriate collation result is given when the absolute values are different. In this embodiment, this mode is most recommended.

[Formula 17]
When Vd (y + w) · Vd (y + w−1)> V (z, h, x + w) · V (z, h, x + w−1),
D (y + w, z, h, x + w) ← D (y + w, z, h, x + w) .Vd (y + w) .Vd (y + w-1) / {V (z, h, x + w) .V (z, h, x + w-1)}
When Vd (y + w) · Vd (y + w−1) <V (z, h, x + w) · V (z, h, x + w−1),
D (y + w, z, h, x + w) ← D (y + w, z, h, x + w) · V (z, h, x + w) · V (z, h, x + w−1) / {Vd (y + w) · Vd ( y + w-1)}

  When w = 0, in the above [Equation 17], Vd (y + w−1) = Vd (y + w) and V (z, h, x + w−1) = V (z, h, x + w).

<1.3.4. Volume judgment mode “Both”>
When the sound volume determination mode “Both” is set, first, processing in the sound volume determination mode “Weight” is performed to obtain D (y + w, z, h, x + w). Then, by executing processing according to the following [Equation 18], the value of D (y + w, z, h, x + w) is determined by multiplying the weight. As a result, D (y + w, z, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account volume data. This mode has no meaning in collation between registered feature words and partial feature words generated from the same original sound data, but when collating registered feature words and search feature words, which will be described later, search feature words If the search acoustic data that is the basis of the sound is accompanied by signal processing such as high compression ratio data compression or analog conversion with waveform distortion, both the relative change in volume and the absolute value of volume are significantly different from the original sound data Give an appropriate match result.

[Formula 18]
When Vd (y + w) · V (z, h, x + w−1)> V (z, h, x + w) · Vd (y + w−1),
D (y + w, z, h, x + w) ← D (y + w, z, h, x + w) · Vd (y + w) · V (z, h, x + w−1) / {V (z, h, x + w) · Vd ( y + w-1)}
If Vd (y + w) · V (z, h, x + w−1) <V (z, h, x + w) · Vd (y + w−1),
D (y + w, z, h, x + w) ← D (y + w, z, h, x + w) · V (z, h, x + w) · Vd (y + w−1) / {Vd (y + w) · V (z, h, x + w-1)}

  When w = 0, in the above [Equation 18], Vd (y + w−1) = Vd (y + w) and V (z, h, x + w−1) = V (z, h, x + w).

  In the case other than the sound volume determination mode “Off”, D (y + w, z, h, x + w) is not the word unit mismatch bit number but the word unit dissimilarity indicating the degree of difference of the word unit considering the sound volume data. In FIG. 8 and FIG. 9, S (y, z, h, x) represents the total dissimilarity, not the total mismatch bit number. Further, Smin1 and Smin2 in FIGS. 10 and 8 represent not the minimum mismatch bit number but the minimum difference.

  The registration means 35 registers the related information, the sound data ID, the registered feature word group, the representative registered feature data, and the individual determination reference value for each original sound data in the sound database 40 in association with each other. The related information may be any information as long as it is related to the original sound data. For example, if the original sound data is a song, the name of the song, the name of the player, and the original sound data are CM audio. If so, the name or URL of the sponsoring company can be used. However, a series of binary-format material data (individual recording data before mixing down, MIDI input data) used for production / mastering of the original sound data is not normally subject to restrictions due to copyright laws. The individual determination reference values Mu, M1, M2, Mw1, and Mw2 are registered as Mu (r), M1 (r), M2 (r), Mw1 (r), and Mw2 (r) for each record r.

<2. Related information search device>
Next, a related information search device for acoustic data according to the present invention (hereinafter referred to as “related information search device”) will be described. FIG. 11 is a hardware configuration diagram of the related information search apparatus. Similar to the related information registration device, the related information search device can be realized by a general-purpose computer. As shown in FIG. 11, a CPU 3a (CPU: Central Processing Unit) and a RAM 3b (RAM: main memory of the computer) Random Access Memory), a large-capacity data storage device 3c (for example, hard disk) for storing data, a program storage device 3d (for example, hard disk) for storing programs executed by the CPU, a keyboard, and a mouse A key input I / F 3e, a data input / output interface 3f for data communication with an external device (data storage medium), and a display output interface 3g for sending information to a display device (display). They are connected to each other via a bus.

  The program storage device 3d of the related information search device is mounted with a dedicated program for operating the CPU 3a and causing the computer to function as the related information search device. The data storage device 3c stores registered feature words, representative registered feature data, and the like in association with related information, functions as an acoustic database, and stores various data necessary for processing. FIG. 11 shows an example realized by a single computer. However, a personal computer having a high-performance arithmetic processing function connected to a server computer on which an acoustic database is operated via a network follows a dedicated program. The contents of each means may be executed.

  FIG. 12 is a functional block diagram of the related information search device according to the present invention. In FIG. 12, 40 is an acoustic database, 45 is a mode setting means, 50 is a search feature word generation means, 60 is a representative search feature data generation means, 70 is a collation range determination means, 80 is a representative feature data collation means, and 90 is a feature. A word collating unit 100 is an information output unit. The related information search device searches for related information related to the original sound data registered in the sound database as related information related to the search sound data using the search sound data held by the user. The search sound data is sound data used for search. At the time of search, it is necessary to collate a search feature word, which is a feature word generated from the search acoustic data, with a registered feature word registered in the acoustic database 40 in advance. Therefore, the search feature word and the registered feature word need to have basically the same structure (note that the former search feature word group generates a set of multiple (H) feature word groups with different phases. ) The search audio data and the original audio data that are the basis of the search feature word and the registered feature word are compressed in various encoding formats, and are generally in different encoding formats depending on the acquisition form. It needs to be converted to a format. In this embodiment, the search sound data and the original sound data are converted into the PCM format having the same specifications (sampling frequency: 44.1 kHz, quantization bit number: 16 bits, channel number: 1 / monophonic PCM format parameter). I try to unify.

  The mode setting unit 45 sets which mode is to be used for processing from among a plurality of modes provided in the related information search apparatus, and is realized by an input device such as a keyboard and a mouse and a key input I / F 3e. Is done. As modes that can be set, a search sound data mode and a sound volume determination mode are prepared. The search sound data mode indicates the state of the search sound data, and can be selected from an intro search and a full scale search. Intro search is a case where the search sound data includes the beginning of the sound material (in the case of music, it means the original music), and in the full length search, the search sound data is completely cut out from the sound material. This corresponds to the case where all the acoustic material is used, that is, the same time length as the acoustic material. The volume determination mode sets how to add a volume component to the word unit dissimilarity D (r, y + w, h, x + w), which will be described later, and can be selected from Off, Weight, Match, and Both. It has become.

  Similar to the registered feature word generation unit 10 shown in FIG. 2, the search feature word generation unit 50 performs frequency analysis using the read sound frame and generates a feature word expressing the features of the search sound data. It has a function. However, a set of a plurality of (H) feature word groups whose phases are shifted is generated. Similar to the representative registered feature data generation unit 20 shown in FIG. 2, the representative search feature data generation unit 60 uses the search feature word group generated by the search feature word generation unit 50, for each search acoustic data. It has a function of generating one representative search feature data. The search feature word expresses a partial feature of the search acoustic data, while the representative search feature data expresses the overall feature of one search acoustic data.

  The representative feature data matching unit 80 has a function of matching the generated representative search feature data with the representative registered feature data registered in the acoustic database 40. The feature word collating unit 90 has a function of collating the generated search feature word with the registered feature word registered in the acoustic database 40. The information output unit 100 has a function of extracting and outputting related information about the original sound data similar to the feature of the searched sound data as a result of the matching by the feature word matching unit 90 from the sound database 40.

  Here, an outline of the relationship among the search feature word generation unit 50, the representative search feature data generation unit 60, the representative feature data collation unit 80, and the feature word collation unit 90 will be described with reference to FIG. As shown in FIG. 13, for the original sound data, a feature word generation process is executed on the original sound data, and a plurality of obtained feature words are registered in the sound database. Further, representative feature data generation processing is executed for a plurality of feature words, and one representative feature data is registered for each original sound data. At the time of the search, the representative search feature data generation unit 60 executes the feature word generation process on the search acoustic data, and then the representative search feature data generation unit 60 executes the representative feature data generation process on the feature word, One representative feature data is generated. Then, the representative feature data matching unit 80 performs a matching process between the representative feature data obtained from the searched sound data and each representative feature data in the sound database. As a result of the collation, the feature word collating unit 90 collates the feature words only with respect to the original sound data that satisfies the condition.

  In this way, first, the representative feature data matching unit 80 generates only one for each acoustic data, and narrows down using the representative feature data representing the characteristics of the entire acoustic data, so that the acoustic features that are greatly different Exclude data from the target. And the characteristic word collation means 90 collates with respect to the comparatively similar original sound data using the characteristic word which expressed the partial feature, and can perform an exact search.

<2.2. Processing operation of related information retrieval device>
Next, the processing operation of the apparatus shown in FIG. 12 will be described. First, when the search operator wants to search the search sound data held by the search operator, the related information search device is instructed to start, and after the start, the search sound data to be searched is designated. This can be executed by operating a button on a predetermined computer screen via the key input I / F 3e and designating search acoustic data stored in the data storage device 3c of the related information search device. Actually, since the search sound data is often in a compression format such as MP3, it is processed after being converted into the PCM format. Further, the search operator sets the search mode and the sound volume determination mode by the mode setting means 45. When the setting is not performed, a normal search other than “Intro search” and “Full scale search” is executed for the search mode, and “Off” is set for the volume determination mode. The set information is written into a predetermined area of the RAM 3b by the CPU 3a, and each means can be referred to. When “Intro search” or “Full scale search” is selected in the search mode, the search acoustic data is set to start from the head of the acoustic material.

  When an instruction is input, the search feature word generation unit 50 reads a predetermined number of samples as one sound frame from the specified search sound data. This process is performed in the same manner as that performed by the related information registration apparatus. That is, the number of samples in one acoustic frame is 4096 samples when the sampling frequency is 44.1 kHz. In addition, the sound frame is read by overlapping 2048 samples.

  The processing from here to the generation of the search feature word follows the flowchart of FIG. The flowchart in FIG. 14 is substantially the same as the flowchart in FIG. 3 for the registration feature word generation. In the same manner as in S11, the search feature word generation unit 50 performs frequency conversion on each read sound frame to obtain a frame spectrum that is a spectrum of the sound frame (S21). As with the related information registration apparatus, as the frequency conversion, Fourier transform, wavelet transform, and other various known methods can be used. However, since it is necessary to match the processing of the related information registration apparatus, in this embodiment, the Fourier transform Is used.

  Subsequently, a spectral component is calculated (S22). Specifically, similarly to S12, the processing according to the above [Formula 1] third formula is performed to obtain the intensity value E (j) at each frequency.

  Subsequently, thinning processing of spectral components is performed (S23). Specifically, as in S13, the process according to the above [Equation 2] is executed and thinned out to Pn in units of 11 frequency components.

  According to the above [Expression 2], 352 frequency components of j = 33 to 384 are thinned out to 32 frequency components of n = 0 to 31. The above process is performed for each acoustic frame, and 32 frequency components are obtained for each acoustic frame.

  Next, for each acoustic frame, a difference from the spectral component of the immediately preceding acoustic frame is calculated (S24). The processes of S21 to S23 are sequentially performed on each acoustic frame. The inter-frame difference calculation processing in S24 uses P0 to P31 obtained as a result of performing the processing up to S23 for each acoustic frame. Specifically, similarly to S14, the process according to the above [Equation 3] is performed to obtain the inter-frame difference Dn (t).

  When the processes of S21 to S24 are sequentially performed on each acoustic frame and T differences (n in this embodiment) Dn (t) between the acoustic frames are obtained, a total sum of T is obtained (S26). ). That is, processing according to the following [Equation 19] is performed to obtain a sum Sn (h) of inter-frame differences. Hs in [Equation 19] is the number of acoustic frames in the minimum unit for shifting the phase, and is defined by (T / H. However, since hs has no meaning unless it is an integer value, in this embodiment, the decimal part is rounded down. , Hs = −2.

[Formula 19]
Sn (h) = Σ _{t = 0,..., T-1} Dn (t + h · hs)

  In the above [Equation 19], h is a phase number that identifies the phase, and is an integer that takes H values of 0 ≦ h ≦ H−1. Subsequently, the binarization process of Sn (h) obtained by the above [Equation 19] is performed (S27). Specifically, the same processing as S17 is executed for the Sn (h) array. That is, the Sn (h) array is divided into two in the upper and lower bands of n ≧ 14 and n ≦ 13, 1 is given to 7 of 14 large values of n ≦ 13, and 0 is given to 7 of the smaller values. , N ≧ 14 and 9 of the 18 large values are given 1 and 9 of the small values are given 0. By the processing in S27, Sn (h) for each n can be expressed by 1 bit. Then, a 32-bit feature pattern F (h, x) is obtained with n = 0 as LSB and n = 31 as MSB. Here, x (= 0,..., X) is a variable indicating the order of X feature words generated from the search sound data. Therefore, x is a variable proportional to the time from the start of performance.

  Next, the volume data is calculated (S28). Specifically, first, the total volume Vol (h) is calculated for each phase number h using the following [Equation 20].

[Formula 20]
Vol (h) = Σ _{t = 0,..., T-1} {Σ _{n = 0,..., 31} Pn (t + h · hs)}

  As shown in [Expression 5] above, the values of all the thinned components Pn (t + h · hs) are added for T acoustic frames. As a result, for each phase number h, a total volume Vol (h) that is the total volume for the T acoustic frames is obtained. The value of the total volume Vol is multiplied by a fixed scaling value set as appropriate, and is normalized so that it falls within the range of 0 to 255 to obtain volume data V (h, x). The sound volume data V (h, x) is represented by 8 bits by normalization. Since the volume data V (h, x) is based on the sum of the volumes over the T acoustic frames as shown in [Equation 20], the volume data V (h, x) is not the volume of each frame unit but the T acoustic frames. It represents the total volume of.

  The processes of S27 and S28 can be performed by changing the order. As a result of the processing in S27 and S28, a 40-bit feature word composed of a 32-bit feature pattern and 8-bit volume data is obtained.

  By executing the above processing for each acoustic frame, a large number (X) of characteristic words for the acoustic data are generated. For example, when the sampling frequency is 44.1 kHz, the sound frame is 4096 samples, and the sound frame is overlapped by 2048 samples as in the above example, one feature word is about 0.506 seconds, and the sound data from 30 seconds Will generate about 60 (= X) feature words.

  As described above, the related information search device generates H times (in this embodiment, H = 5) feature words per unit time of the acoustic data as compared with the related information registration device. (However, as a whole, the feature words generated by the related information search device are significantly fewer than the related information registration device.) The search acoustic data to be searched for compared to the related information registration device is music. Often part of this is cut off. This is because the user may obtain it by recording a part of the music being played. For this reason, the timing of the original sound data registered in the database does not necessarily match, and a positional shift may occur. Even in the method generated in the related information registration apparatus, 11 acoustic frames are averaged and generated, so that they are relatively resistant to positional deviation. However, in the case of searched sound data with a sharp rhythm change, if the sound sound is shifted by about 5 sound frames, which is almost half of 11 sound frames, which is a feature word generation unit, significantly different feature words are generated, and erroneous information is searched. End up. Therefore, a plurality of search feature words are generated by delaying (changing the phase) by hs (= 2 sound frames) within the range of 11 sound frames as one analysis unit, and the registered feature words in the sound database 40 Try to match.

  Specifically, the related information registration device generates one feature word from frame 1 to frame 11, whereas the related information search device generates a feature word from frame 1 to frame 11 and for two acoustic frames. A feature word is also generated from a group of acoustic frames shifted (by changing the phase) by four acoustic frames, six acoustic frames, and eight acoustic frames. That is, feature words are also generated in frames 3 to 13, frames 5 to 15, frames 7 to 17, and frames 9 to 19. Finally, by executing the processing according to the flowchart of FIG. 14 for each acoustic frame, [F (h, x), V (h, x)] ( h = 0, ..., H-1; x = 0, ..., X-1).

  After generating the H search feature word groups in consideration of the phase shift, the representative search feature data generating means 60 then generates representative search feature data using the search feature word groups. The representative search feature data is configured as a set of an average value and a standard deviation in the time direction of feature components based on the search feature word. FIG. 15 is a flowchart showing a calculation process of an average value and a standard deviation in the time direction of feature components based on a search feature word. First, the representative search feature data generation means 60 generates a search feature data array by multi-value processing the search feature words (S29). Specifically, by executing processing according to the following [Formula 21], a search feature data array Q (n, h, x) (n = 0,...) That is a feature component based on the search feature word. , 31; h = 0,..., H−1; x = 0,.

[Formula 21]
When each bit n of F (h, x) is 1, Q (n, h, x) ← V (h, x)
When each bit n of F (h, x) is 0, Q (n, h, x) ← −V (h, x)

  Next, the representative search feature data generation means 60 calculates the average value array C (n) and the standard deviation array L (n) using the search feature data array Z (n, h, x) (S30). Specifically, the average value array C (n) and the standard deviation array L (n) are calculated by executing processing according to the following [Equation 22].

[Formula 22]
C (n) = [Σx _{= 0, ..., X-1} Σh _{= 0, ..., H-1} Q (n, h, x)] / (HX)
L (n) = [Σx _{= 0, ..., X-1} Σh _{= 0, ..., H-1} (Q (n, h, x) -C (n)) ² / (HX)] ^1/2

FIG. 16 is a conceptual diagram of the processing for calculating the average value and standard deviation of representative search feature data in S29 and S30. FIG. 16 shows an average value and standard deviation calculation process of representative search feature data for a specific phase h. As shown in FIG. 16, the search feature word is stored at each time x (x = 0, .., X-1) has a 32-bit feature pattern and 8-bit volume data. In FIG. 16, 32 bits of the feature pattern are indicated by Bit0 to Bit31.

  Then, as shown in [Formula 21], the value of Q (n, h, x) is left as it is as the value of the volume data depending on whether the value of bit n (Bit 0 to Bit 31) is 0 or 1. Or the value of the volume data is multiplied by −1, and the value of Q (n, h, x) corresponding to bit n is determined. At this time, since 8-bit volume data may be a negative value, Q (n, h, x) is expressed by 16 bits. In the 32-bit feature pattern, what is represented by 1 bit for each band is represented by 16 bits (2 bytes), so the search feature data at each time x is 64 bytes.

  When the search feature word group and the representative search feature data are obtained for the search acoustic data to be searched, the collation process is actually executed with reference to the acoustic database 40. At this time, first, the collation range determining means 70 performs processing for determining the collation range of the registered feature word. The search sound data is the sound data used by the user for the search. Therefore, the entire music is not necessarily recorded, and only a part of the music may be recorded and acquired as the sound data. . For this reason, the collation range determination means 70 determines a collation range for efficient collation on the premise that it is not known from which part of the original music the retrieved acoustic data is actually acquired. . FIG. 17 shows a conceptual diagram for determining the collation range. The horizontal width of the rectangle in FIG. 17 indicates the number of feature words constituting the feature word group.

  Specifically, first, the collation range determination means 70 compares the number X of search feature words in the search feature word group with the number Y (r) of registered feature words in the registered feature word group for a certain record r. Here, the number of search feature words in the search feature word group is compared with the number of registered feature words in the registered feature word group as being proportional to the length at the time of reproduction. Therefore, the search feature word number X is assumed to be when the phase is not considered.

  Then, as a result of comparison between the search feature word number X of the search feature word group and the registered feature word number Y (r) of the registered feature word group, when X is larger than Y (r), that is, as shown in FIG. In such a case, the collation range determination unit 70 excludes the record from the search target and moves to the next record. When X is larger than Y (r), a part of the search sound data is longer than the original sound data. Since the original sound data records the entire sound data, it becomes clear that both are based on different original sound data. Therefore, in this case, the record is excluded from the search target.

  If X is smaller than Y (r) as a result of comparison between the search feature word number X of the search feature word group and the registered feature word number Y (r) of the registered feature word group, the collation range determination means 70 searches the search feature word group. Whether or not a feature word created from the head portion of the original sound data is included. Specifically, referring to a predetermined area of the RAM 3b, it is confirmed whether or not the intro search or full scale search is set by the mode setting means 45. When these search modes are set, it is clear that the search feature word group includes a feature word created from the beginning of the original sound data, so the search sound data and the original sound data are the same. If there is, the first search feature word and the registered feature word should match. Therefore, when the search feature word group includes a feature word created from the head portion of the original sound data, the collation range determination unit 70 sets the head search feature word and the registered feature word as a collation target. I do. However, just comparing the top one feature word may result in another song having the same feature word, so that a correct collation result cannot be obtained. Accordingly, the collation range determination unit 70 performs processing for targeting α feature words corresponding to a predetermined length of time. In the present embodiment, processing is performed on a feature word group corresponding to about 12 seconds. As described above, in this embodiment, one feature word is about 0.506 seconds, so 12 seconds corresponds to 24 feature words. That is, in this case, 24 feature words from the beginning of the search feature word group and 24 feature words from the beginning of the registered feature word group are targeted for collation. FIG. 17B shows the state at this time.

  If the search feature word group does not include the feature word created from the head part of the original sound data and if α (<X) from the head part of the search feature word group is to be collated, the collation range is determined. The means 70 sets the collation range with the first search feature word (x = 0) of the search feature word group in the registered feature word group from the start (y = 0) to the last registered feature word (y = Y) of the registered feature word group. From (r) -1), the number of search feature word groups (X) is the range up to the front. That is, the collation range of the search feature word group in the registered feature word group with the head search feature word (x = 0) is from (Y (r) −) from the head (y = 0) as shown in FIG. X-1) Up to the registered feature word.

  In the present embodiment, as a standard setting, α feature words from the beginning of the search feature word group are targeted for collation, but α feature words from the tail of the search feature word group and the feature at the center of the search feature word group are included. It is also possible to set α words as collation targets. When α feature words from the tail of the search feature word group are set as collation targets, the collation range of the search feature word group in the registered feature word group with the first search feature word (x = 0) is as shown in FIG. As shown in (d), from (X-α) th to (Y (r) -α) th registered feature word.

  Further, when α feature words in the center of the search feature word group are set as collation targets, the collation range with the first search feature word (x = 0) of the search feature word group in the registered feature word group is as shown in FIG. As shown in (d), from {(X−α) / 2} th to {Y (r) −α− (X−α) / 2} th registered feature words. The collation range determination unit 70 passes the collation range of the registered feature word determined as described above to the feature word collation unit 90 for each record.

  When the collation range of the registered feature word is determined, the representative feature data collating unit 80 next narrows down the records to be searched using the representative search feature data. Specifically, the representative search feature data is compared with the representative registered feature data registered in the acoustic database 40, and only records satisfying a predetermined condition are extracted. And the characteristic word collating means 90 searches the relevant information of the original acoustic data registered into the acoustic database 40 using the search characteristic word group from the extracted record. Specifically, the search feature word group and the registered feature word group registered in the acoustic database 40 are compared, and a record satisfying a predetermined condition is extracted. The processing by the representative feature data matching unit 80 is performed to narrow down the records to be searched because the calculation load of the comparison processing using the search feature word group is high.

  Here, the concept of representative feature data performed by the representative feature data matching unit 80 will be described. FIG. 18 is a diagram illustrating the concept of representative feature data. The representative feature data has values corresponding to the 32 integrated frequency bands as shown in FIGS. Therefore, the integrated frequency band has a one-dimensional structure. In the narrowing down using the representative feature data, only the original sound data having the representative registered feature data intersecting with the representative search feature data of the search sound data plotted in the 32-dimensional space is extracted as the narrowing result.

  In FIG. 18, for the purpose of conceptual illustration, a certain two-dimensional plot is plotted among the 32-dimensional representative feature data. In FIG. 18, the black circle is an average value for each acoustic data, and the circle centered on the black circle has a radius that is a standard deviation for each acoustic data from the black circle. Then, the original sound data whose circle does not intersect with the circle of the search sound data is excluded from the search target, and only the original sound data whose circle intersects with the circle of the search sound data is extracted as the narrowing result. Then, it can be set as appropriate whether to extract what intersects all 32 dimensions or extract what intersects a predetermined number of dimensions. In this embodiment, as shown in [Formula 7] and [Formula 22], each of the 32 frequency bands is two-dimensionalized with an average value and a standard deviation. The concept of the representative feature data distance U (z) is the same.

  Next, search processing by the representative feature data collating unit 80 and the feature word collating unit 90 will be described with reference to the flowcharts of FIGS. 19 to 23, each variable is defined as follows.

[Registration feature word]
R: Number of records (number of original acoustic data managed by the acoustic database 40)
Y (r): Number of registered feature words of record r (r = 0,..., R−1) Fd (r, y): Feature pattern array of record r (r = 0,..., R−1) (y = 0,..., Y-1), 32 bits Vd (r, y): Volume data array (y = 0,..., Y-1) of record r (r = 0,..., R-1), 8 bits Cd (N, r): average value of registered feature data array in record r (r = 0,..., R−1) Ld (n, r): registered feature in record r (r = 0,..., R−1) Standard deviation of data array

[Search feature word]
X (h): Number of search feature words in phase number h (h = 0,..., H−1) F (h, x): Feature pattern array (x = 0,..., X (h) −1), 32 Bit V (h, x): Volume data array (x = 0,..., X (h) -1), 8 bits C (n): Average value of search feature data array L (n): Search feature data array standard deviation

[Collation variable]
W: Number of collation words (w = 0,..., W-1), registered feature words to be collated, number of search feature words (eg, W = 6)
U (r): representative feature data distance D (r, y + w, h, x + w): word unit mismatch bit number of feature pattern (0 to 32)
S (r, y, h, x): the sum of the number of unmatched bits, the number of matching word numbers D (r, y + w, h, x + w) (Σw _{= 0,} ... _{, W) -1} D (r, y + w, h, x + w))

[Judgment threshold (preset)]
MMu: Determination threshold value of representative feature data distance U (r) MM1: Determination threshold value of total mismatch bit number S (r, y, ho, x) MM2: Total mismatch bit number S (r, y, h, x) Determination threshold value MMw1: Determination threshold value of word unit mismatch bit number D (r, y, ho, x) MMw2: Determination threshold value of word unit mismatch bit number D (r, y, h, x) value

  The determination threshold values MMu, MM1, MM2, MMw1, and MMw2 are set in advance. These determination threshold values are obtained for each original sound data as follows, and finally the maximum value is determined. It can be an official decision threshold. When MMu generates registered feature words and representative registered feature data for the original sound data to be registered and registers them in the record r, MMu creates a plurality of partial sound data that are randomly cut out from the original sound data at several locations. For each record r, a partial feature word and representative partial feature data are created for each partial acoustic data, and a representative feature data distance U (z) calculated based on [Equation 12] is calculated. The maximum value + β of the plurality of representative feature data distances U (z) calculated respectively is given as MMu. (In the sense that β is set slightly larger than the maximum value, for example, β = 1). As a result, the representative feature data distance U (z) for at least the corresponding record r does not exceed Mu, so that if the representative feature data distance U (z) exceeds MMu, it can be immediately determined that the record is not applicable.

  For MM1 and MMw1, the registered feature word and each of the plurality of partial feature words are fixed to h = ho, and the number D (y, z, ho, x) of word unit mismatches is determined by the method shown in FIG. The number of consecutive mismatched bits S (y, z, ho, x) is calculated based on the sum of these, and the minimum number of unmatched bits Smin1 when this value is the smallest is calculated. At the same time, the maximum word unit mismatch bit number Dmax of each element D (y, z, ho, x) of the sum at that time is obtained, and the maximum value + β of Smin1 obtained for each of the plurality of search feature words is set to MM1, The maximum value + β is assumed to be MMw1.

  For MM2 and MMw2, each of the registered feature words and the plurality of partial feature words and the number of word unit mismatch bits D (y, z, h, x) are changed while h is changed in the range of 0 to H-1. 9 is continuously calculated by a predetermined number of words by the method shown in FIG. 9, and the sum mismatch bit number S (y, z, h, x) is calculated based on the total sum of them, and the minimum when this value is the smallest The total mismatch bit number Smin2 is calculated, the maximum word unit mismatch bit number Dmax of each element D (y, z, h, x) of the sum at that time is obtained, and the maximum value of Smin2 obtained for each of the plurality of partial feature words + Β is MM2, and the maximum value of Dmax + β is MMw2.

  The processing as described above is performed by causing a computer to execute a dedicated program, and different determination threshold values MMu, MM1, MM2, MMw1, and MMw2 can be obtained for each record. Then, the maximum value of each determination threshold value calculated for all records is set as a determination threshold value common to all records. In this embodiment, MMu = 350, MM1 = 70, MM2 = 54, MMw1 = 20, and MMw2 = 13 are given. For example, record 1 is MMu = 350, MM1 = 60, MM2 = 54, MMw1 = 18, MMw2 = 13, and record 2 is MMu = 300, MM1 = 70, MM2 = 44, MMw1 = 20, MMw2 = 12. For example, MMu = 350, MM1 = 70, MM2 = 54, MMw1 = 20, and MMw2 = 13 are given as judgment threshold values common to both records.

  FIG. 19 is a flowchart of the acoustic data search using the search feature word group by the representative feature data matching unit 80 and the feature word matching unit 90. First, initial setting is performed (S210). Specifically, the matching table Smin (c) = initial value Big3, the matching table Rmin (c) = − 1, the number of matching cases c = 0, and the record number r = 0 are set. The initial value Big3 may be a value sufficiently larger than a value that can be taken as the minimum mismatch bit number, and is set in advance.

  Next, the representative feature data matching unit 80 calculates the representative feature data distance U (r) (S210). Specifically, processing according to the following [Equation 23] is executed to calculate a representative feature data distance U (r) that is a distance between the representative registered feature data and the representative search feature data for the record r.

[Formula 23]
U (r) = 256 × [Σ _{n = 0,..., 31} {(Cd (n, r) −C (n)) / (Ld (n, r) + L (n))} ² ] ^1/2 / [Σ _{n = 0, ..., 31} {Cd (n, r) / (Ld (n, r) + L (n))} ² ] ^1/4 / [Σ _{n = 0, ..., 31} {(C (n ) / (Ld (n, r) + L (n))} ² ] ^1/4

In [Equation 23], there are three terms enclosed by [], but the square root of the first [] is divided by the fourth root of the second [] and the fourth root of the third []. Is multiplied by a coefficient “256” for normalization. The coefficient for normalization can be appropriately changed according to the range of normalization. In the above [Equation 23], “Σ _{n = 0,..., 31} ” means a total of 32 pieces when n is incremented by 1 from n = 0 to 31. As the power root of each term enclosed in [], the specific power can be appropriately changed. However, in this embodiment, as shown in [Equation 23], the square root, the fourth root, the fourth power Roots.

[Equation 23] indicates the distance between the representative search feature data and the representative registered feature data, and is defined based on the normalized correlation coefficient of both. Specifically, each element n of the representative search feature data is represented by the sum of the standard deviation value of the search feature data array and the standard deviation value of the registered feature data array with respect to the average value of the search feature data array. Each value is defined by being normalized and divided by (C (n) / (Ld (n, r) + L (n)). Similarly, each element of the representative registered feature data is registered in the registered feature data array. Is defined by each value normalized by dividing each of the 32 elements by the sum of the standard deviation value of the search feature data array and the standard deviation value of the registered feature data array with respect to the average value of Cd (n, r) / (Ld (n, r) + L (n)) The normalized correlation coefficient between these 32 elements is [Σ _{n = 0,..., 31} {(Cd (n, r) -C (n)) / (Ld (n, r) + L (n))} 2] / [Σ n = 0, ..., 31 {Cd (n, r) / (Ld (n, r) + L (n ^{)} 2] 1/2 / [Σ} n = 0, ..., 31 {(C (n) / (Ld (n, r) + L (n))} 2] is given by ^1/2, -1 to +1 In the above [Equation 23], the normalized correlation coefficient value is multiplied by a predetermined integer value 256 to give an integer expression, and the fluctuation range of the integer value is expanded to show the corresponding record and the non-corresponding record. In order to make the difference, the normalized correlation coefficient value is used as it is, whether it is the square root or the fourth root. Just choose.

  The representative feature data distance U (r) corresponds to the distance between the circle of the search sound data and the circle of the original sound data in FIG. In reality, the determination of the target to be extracted as the narrowing-down result may be set as appropriate, for example, when the circle and the circle shown in FIG. 18 intersect or when the circle and the circle are within a predetermined distance range. it can. In this embodiment, the narrowing down target is determined by comparing the representative feature data distance U (r) calculated by [Equation 23] with the determination threshold value.

  Next, the representative feature data collating unit 80 corrects the calculated representative feature data distance U (r) (S220). Specifically, the process according to the following [Equation 24] is executed to calculate the correction value U ′ (r).

[Formula 24]
U ′ (r) = U (r) · {M ^S u / Mu (r)} ^1/2

In the above [Equation 24], M ^S u is a standard value and is a value that is ½ of the determination threshold value Mmu. Mu (r) is an individual determination reference value of the record r registered in the acoustic database 40. Accordingly, the correction value U '(r) is obtained by multiplying the square root representative feature data distance U (r) but by dividing the standard value M ^S u record r of the individual judgment reference value Mu (r).

  Next, the representative feature data collating unit 80 compares the calculated representative feature data distance correction value U ′ (r) with a preset determination threshold value MMu (S230). As a result of the comparison, when the correction value U ′ (r) of the representative feature data distance is equal to or larger than the determination threshold value Mmu, the process proceeds to S270, where r is incremented, and the process for the next record r + 1 is performed. When the correction value U ′ (r) of the representative feature data distance is equal to or greater than the determination threshold value MMu, detailed collation calculation for the record r is not performed. That is, in the present embodiment, the search target is narrowed down by the value of the representative feature data distance correction value U ′ (r).

  If the correction value U ′ (r) of the representative feature data distance is smaller than the determination threshold value MMu as a result of the comparison in S230, the feature word collating unit 90 is registered with the registered feature word group registered in association with the record r. Then, collation with the search feature word group is performed (S240). As a result of the collation in S240, a record satisfying a predetermined condition is output with the record number r given to Rmin (c). Details of the process of S240 will be described later.

  If Rmin (c) is obtained, it is determined whether Rmin (c) is 0 or more (S250). When Rmin (c) is 0 or more, it is determined that the record is matched, and a process of adding 1 to the number of matching cases c is performed (S260). If Rmin (c) is less than 0, it is determined that the record has not been matched, and the matching number c is not added. Although details will be described later, in S240, a value less than 0 is set as an initial value in Rmin (c), and when it is determined that the record is suitable, a record number r that is greater than or equal to 0 is set to Rmin (c). To give. For this reason, in S260, the suitability of the record is determined by determining whether Rmin (c) is 0 or more.

  Next, the variable r specifying the record is incremented, that is, increased by 1 (S270). Then, it is determined whether or not the variable r for specifying the record has reached the total number R of records in the acoustic database 40 (S280). If not, the process returns to S240 to perform the collation process for the next record r. . When processing is performed for each record r and the record r reaches the total number of records R, that is, when processing for all R total records is completed, the matching table Rmin (c) is changed to the value of the matching table Smin (c). Based on the ascending order, the list is output together with the matching number c (S290).

  Next, the details of the record r matching process in S240 of FIG. 19 will be described using the flowchart of FIG. First, initial setting is performed (S241). Specifically, the matching table Smin (c) = initial value Big3, the matching table Rmin (c) = − 1, the variable x = 0 for specifying the search feature word, the variable y = 0 for specifying the registered feature word, and the phase Set variable h = ho to be specified. As for ho, when H = 5, it can be set to any of ho = 0, 1, 2, 3, and 4, but it is set to ho = 0 with the least amount of normal arithmetic processing. Subsequently, the total mismatch bit number S (r, y, ho, x) is calculated (S242). Details of the processing of S242 will be described later.

  Next, the feature word collating unit 90 corrects the calculated sum mismatch bit number S (r, y, ho, x) (S243). Specifically, a process according to the following [Equation 25] is executed to calculate a correction value S ′ (r, y, ho, x).

[Formula 25]
S ′ (r, y, ho, x) = S (r, y, ho, x) · {M ^S 1 / M1 (r)} ^1/2

In the above [Equation 25], M ^S 1 is a standard value, which is a value half of the determination threshold value MM1. M1 (r) is an individual determination reference value of the record r registered in the acoustic database 40. Therefore, the correction value S ′ (r, y, ho, x) is obtained by dividing the square root of the standard value M ^S 1 by the individual determination reference value M1 (r) of the record r and adding the number S (r, y, obtained by multiplying ho, x).

  Next, the feature word collating unit 90 compares the calculated correction value S ′ (r, y, ho, x) of the total mismatch bit number with a preset determination threshold value MM1 (S244). As a result of the comparison, if the correction value S ′ (r, y, ho, x) is larger than the determination threshold MM1, the process proceeds to S246, x is incremented, and the process for the next search feature word x + 1 is performed. When the correction value S ′ (r, y, ho, x) is equal to or smaller than the determination threshold value MM1, the registered feature word y in the specified (x, y) is compared with the search feature word x (S245). ).

  Next, the variable x is incremented (S246). Then, x and X (ho) are compared (S247). As a result of the comparison, if x has not reached X (ho), the process returns to S242 and the process for the next search feature word x + 1 is performed. If x reaches X (ho) as a result of the comparison, x is set to 0 and y is incremented (S248). Then, y is compared with Y (r) (S249). As a result of the comparison, if y has not reached Y (r), the process returns to S242 to perform processing for the next registered feature word y + 1.

  If y reaches Y (r) as a result of the comparison, the processing for all Y (r) registered feature words has been completed. Therefore, the matching table Rmin (c) and the matching table Smin at that time point. (C) is output. The matching table Rmin (c) and the matching table Smin (c) are obtained in S240 of FIG.

    Next, details of the calculation processing of the total mismatch bit number S (r, y, ho, x) in S242 of FIG. 20 will be described using the flowchart of FIG. First, initial setting is performed (S281). Specifically, the variable h = ho for specifying the phase, the sum mismatch bit number S (r, y, ho, x) = 0, and the variable w = 0 indicating the number of feature word collations are set. After the initial setting, it is determined whether or not the condition that the volume data Vd (r, y + w) of the registered feature word and the volume data V (h, x + w) of the search feature word are both greater than 0 is satisfied (S282). The volume data Vd (r, y + w) of the registered feature word and the volume data V (h, x + w) of the search feature word are in accordance with [Formula 5] and [Formula 20] with respect to the original sound data and the retrieved sound data, respectively. The process is executed, and the calculated “Vol” and “Vol (h)” are normalized. As a result of the determination in S282, the registered feature word 1 only when the volume data Vd (r, y + w) of the registered feature word and the volume data V (z, h, x + w) of the search feature word are both greater than 0. The word unit mismatch bit number D (r, y + w, ho, x + w), which is the number of mismatch bits, is compared with one search feature word (S283).

  Next, the feature word collating unit 90 corrects the calculated word unit mismatch bit number D (r, y + w, ho, x + w) (S284). Specifically, a process according to the following [Equation 26] is executed to calculate a correction value D ′ (r, y + w, ho, x + w).

[Formula 26]
D'(r, y + w, ho, x + w) = D (r, y + w, ho, x + w) · {M S w1 / Mw1 (r)} 1/2

In the above [Equation 26], M ^s w1 is a standard value, which is a value half that of the determination threshold MMw1. Mw1 (r) is an individual determination reference value of the record r registered in the acoustic database 40. Therefore, the correction value D ′ (r, y + w, ho, x + w) is the word unit mismatch bit number D (r, y + w) obtained by dividing the square root of the standard value M ^S w1 by the individual determination reference value Mw1 (r) of the record r. , Ho, x + w).

  Next, the feature word collating unit 90 compares the calculated correction value D ′ (r, y + w, ho, x + w) of the word unit mismatch bit number with a predetermined determination threshold value MMw1 (S285). . Only when the correction value D ′ (r, y + w, ho, x + w) is equal to or smaller than the determination threshold MMw1 as a result of the determination in S285, the correction value D ′ (r, y + w, ho, x + w) A process of adding to S (r, y, ho, x) is performed (S286).

  In S286, a process of incrementing the variable w indicating the number of feature word collations is also performed. Then, it is determined whether or not the variable w indicating the number of collated feature words has reached the predetermined number W (S287). If not, the process returns to S282 and the next feature word is processed. When processing is performed for each feature word and the variable w reaches a predetermined number W, the total mismatch bit number S (r, y, ho, x) at that time is calculated for a certain x, h (= ho), y. Output as the total mismatch bit number S (r, y, ho, x). This sum mismatch bit number S (r, y, ho, x) is obtained in S242 of FIG. If it is determined in S282 and S285 that the condition is not satisfied, an error in calculating the total number of mismatch bits is output.

  Next, details of the intra-record matching process in S245 of FIG. 20 will be described using the flowchart of FIG. First, initial setting is performed (S321). Specifically, the variable h = 0 that specifies the phase is set. Subsequently, the sum mismatch bit number S (r, y, h, x) is calculated (S322). Details of the process of S322 will be described later.

  Next, the feature word collating unit 90 corrects the calculated sum mismatch bit number S (r, y, h, x) (S323). Specifically, the processing according to the following [Equation 27] is executed to calculate the correction value S ′ (r, y, h, x).

[Formula 27]
S ′ (r, y, h, x) = S (r, y, h, x) · {M ^S 2 / M2 (r)} ^1/2

In the above [Expression 27], M ^S 2 is a standard value, which is a value that is ½ of the determination threshold value MM2. M2 (r) is an individual determination reference value of the record r registered in the acoustic database 40. Accordingly, the correction value S ′ (r, y, h, x) is obtained by dividing the square root of the standard value M ^S 2 by the individual determination reference value M2 (r) of the record r and adding the number S (r, y, obtained by multiplying h, x).

  Next, the feature word collating unit 90 compares the calculated correction value S ′ (r, y, h, x) of the total mismatch bit number with the preset determination threshold value MM2 (S324). When the correction value S ′ (r, y, h, x) is less than or equal to the determination threshold value MM2, the correction value S ′ (r, y, h, x) is compared with the sum total mismatch bit number minimum value Smin (c). (S325).

  Only when the correction value S ′ (r, y, h, x) is smaller than the sum total mismatch bit number minimum value Smin (c), the value of r is set in the matching table Rmin (c), and the correction value S ′ (r , Y, h, x) is set in the matching table Smin (c) (S326). Next, the variable h is incremented (S327). Then, h and H are compared (S328). If h does not reach H as a result of the comparison, the process returns to S322 and the process for the next phase number h + 1 is performed.

  If h has reached H as a result of the comparison, the processing for all H phases has been completed, and the matching table Rmin (c) and matching table Smin (c) at that time are output. The matching table Rmin (c) and the matching table Smin (c) are obtained in S245 of FIG.

  Next, details of the calculation processing of the total mismatch bit number S (r, y, h, x) in S322 of FIG. 22 will be described using the flowchart of FIG. First, initial setting is performed (S381). Specifically, the total mismatch bit number S (r, y, h, x) = 0 and the variable w = 0 indicating the number of feature word collations are set. After the initial setting, it is determined whether or not the condition that the volume data Vd (r, y + w) of the registered feature word and the volume data V (h, x + w) of the search feature word are both greater than 0 is satisfied (S382). As a result of the determination in S382, the registered feature word 1 only when the volume data Vd (r, y + w) of the registered feature word and the volume data V (z, h, x + w) of the search feature word are both greater than 0. The word unit mismatch bit number D (r, y + w, h, x + w), which is the number of mismatch bits when one search feature word is compared, is calculated (S383).

  Next, the feature word collating unit 90 corrects the calculated word unit mismatch bit number D (r, y + w, h, x + w) (S384). Specifically, a process according to the following [Equation 28] is executed to calculate a correction value D ′ (r, y + w, h, x + w).

[Formula 28]
D'(r, y + w, h, x + w) = D (r, y + w, h, x + w) · {M S w2 / Mw2 (r)} 1/2

In the above [Equation 28], M ^S w2 is a standard value, which is a value that is ½ of the determination threshold value MMw2. Mw2 (r) is an individual determination reference value of the record r registered in the acoustic database 40. Therefore, the correction value D ′ (r, y + w, h, x + w) is obtained by dividing the square root of the standard value M ^S w2 by the individual determination reference value Mw2 (r) of the record r, and the number D (r, y + w) of word unit mismatch bits. , H, x + w).

  Next, the feature word collating unit 90 compares the calculated correction value D ′ (r, y + w, h, x + w) of the word unit mismatch bit number with a preset determination threshold value MMw2 (S385). . Only when the correction value D ′ (r, y + w, h, x + w) is equal to or smaller than the determination threshold value MMw2 as a result of the determination in S385, the correction value D ′ (r, y + w, ho, x + w) A process of adding to S (r, y, h, x) is performed (S386). In S386, a process of incrementing the variable w indicating the number of feature word collations is also performed. Then, it is determined whether or not the variable w indicating the number of collated feature words has reached the predetermined number W (S387). If not, the process returns to S382 and the next feature word is processed. When processing is performed for each feature word and the variable w reaches a predetermined number W, the total mismatch bit number S (r, y, h, x) at that time is calculated as the total mismatch bit number for a certain x, h, y. Output as S (r, y, h, x). This sum mismatch bit number S (r, y, h, x) is obtained in S322 of FIG. If it is determined in S382 and S385 that the condition is not satisfied, a calculation error of the total mismatch bit number is output.

<2.3. Calculation of word unit mismatch bit count>
The calculation of the word unit mismatch bit number D (r, y + w, h, x + w) in S283 in FIG. 21 and S383 in FIG. 23 will be described. Regarding the calculation of the word unit mismatch bit number D (r, y + w, h, x + w), the specific processing contents differ depending on the sound volume determination mode set by the user. There are four volume determination modes: Off, Weight, Match, and Both.

<2.3.1. Volume judgment mode “Off”>
The volume determination mode “Off” is a mode in which no weight is added, and when the volume determination mode “Off” is set, the word unit mismatch bit number D (r, y + w, h, x + w) is directly different in word units. This is a word unit dissimilarity D (r, y + w, h, x + w) indicating the degree of. In the sound volume determination mode “Off”, the initial value is set with the word unit mismatch bit number D (r, y + w, h, x + w) = 0, and then 32 bits of Fd (r, y + w) and F (h, x + w). Are sequentially compared in corresponding bit units, and 1 is added to D (r, y + w, h, x + w) every time the bits differ. In both the registered feature word group and the search feature word group, the feature pattern of the feature word is created according to the same rule, and has a 32-bit configuration in which the low frequency component is LSB and the high frequency component is MSB. This can be done depending on whether or not the bit values match.

<2.3.2. Volume judgment mode “Weight”>
The sound volume determination mode “Weight” is a mode for adding a weight, and when the sound volume determination mode “Weight” is set, the initial value is set with the word unit mismatch bit number D (r, y + w, h, x + w) = 0. After that, the 32 bits of Fd (r, y + w) and F (h, x + w) are sequentially compared in corresponding bit units. Based on the result of the comparison, processing according to the following [Equation 29] is executed to determine the value of D (r, y + w, h, x + w). As a result, D (r, y + w, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account volume data. In this mode, when the registered feature word is matched with the search feature word, the search acoustic data that is the basis of the search feature word is accompanied by signal processing such as analog conversion. Appropriate matching results are given when both absolute values are different.

[Formula 29]
When the Fd (r, y + w) side is bit 1 and the F (h, x + w) side is bit 0,
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) + Vd (r, y + w) · 2 / {Vd (r, y + w) + V (h, x + w)}
When the Fd (r, y + w) side is bit 0 and the F (h, x + w) side is bit 1,
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) + V (h, x + w) · 2 / {Vd (r, y + w) + V (h, x + w)}

<2.3.3. Volume judgment mode “Match”>
When the sound volume determination mode “Match” is set, first, processing in the sound volume determination mode “Off” is performed to obtain D (r, y + w, h, x + w). Then, by executing the processing according to the following [Equation 30], the value of D (r, y + w, h, x + w) is determined by multiplying the weight. As a result, D (r, y + w, h, x + w) is not the word unit mismatch bit number but the word unit dissimilarity indicating the degree of difference in word units taking into account the difference in the fluctuation pattern of the volume data. In this mode, when the registered feature word and the search feature word are collated, the search acoustic data that is the basis of the search feature word is accompanied by signal processing such as various data compression. Although there is not much difference, an appropriate matching result is given when the absolute values are different. In this embodiment, this mode is most recommended.

[Formula 30]
When Vd (r, y + w) · Vd (r, y + w−1)> V (h, x + w) · V (h, x + w−1),
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) · Vd (r, y + w) · Vd (r, y + w−1) / {V (h, x + w) · V (h, x + w-1)}
When Vd (r, y + w) · Vd (r, y + w−1) <V (h, x + w) · V (h, x + w−1),
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) · V (h, x + w) · V (h, x + w−1) / {Vd (r, y + w) · Vd (r, y + w-1)}

  When w = 0, in the above [Equation 30], Vd (r, y + w−1) = Vd (r, y + w) and V (h, x + w−1) = V (h, x + w).

<2.3.4. Volume judgment mode “Both”>
When the sound volume determination mode “Both” is set, first, processing in the sound volume determination mode “Weight” is performed to obtain D (r, y + w, h, x + w). Then, by executing processing according to the following [Equation 31], the value of D (r, y + w, h, x + w) is determined by multiplying the weight. As a result, D (r, y + w, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account volume data. In this mode, when the registered feature word and the search feature word are collated, the search acoustic data that is the basis of the search feature word is accompanied by signal processing such as high-compression data compression or analog conversion with waveform distortion. An appropriate collation result is given when both the original acoustic data, the relative change in volume, and the absolute value of volume are significantly different.

[Formula 31]
When Vd (r, y + w) · V (h, x + w−1)> V (h, x + w) · Vd (r, y + w−1),
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) · Vd (r, y + w) · V (h, x + w−1) / {V (h, x + w) · Vd (r, y + w-1)}
When Vd (r, y + w) · V (h, x + w−1) <V (h, x + w) · Vd (r, y + w−1),
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) · V (h, x + w) · Vd (r, y + w−1) / {Vd (r, y + w) · V (h, x + w-1)}

  When w = 0, in the above [Equation 31], Vd (r, y + w−1) = Vd (r, y + w) and V (h, x + w−1) = V (h, x + w).

  In cases other than the sound volume determination mode “Off”, D (r, y + w, h, x + w) is not the number of word unit mismatch bits, but the word unit difference degree indicating the degree of difference in word units taking into account volume data. In FIG. 20 and FIG. 21, S (r, y, h, x) represents the total dissimilarity, not the total mismatch bit number. Also, Smin (c) in FIGS. 19 and 20 represents not the minimum mismatch bit number but the minimum difference.

  The information output unit 100 sorts the record number table Rmin (c) in ascending order based on the value of the minimum difference degree table Smin (c) in S290 of FIG. 19, and outputs the list together with the number of matching cases c. Since the corrected minimum dissimilarity S ′ (r, y, h, x) is recorded in the minimum dissimilarity table, the minimum dissimilarity table is ordered according to the value of the corrected minimum dissimilarity S ′ (r, y, h, x). Is output. That is, the smaller the corrected minimum difference S ′ (r, y, h, x), the higher the output. This is a search result searched using the search acoustic data. When outputting the list, the content to be output can be set as appropriate. For example, all the related information may be output, or only the music title may be output. When one is selected from the output list, the information output unit 100 extracts the record from the acoustic database 40. At that time, when an instruction to acquire the original sound data is given, the information output means 100 uses the file name recorded in the record to obtain related information related to the original sound data stored in the storage device or the original sound data. An ID for specifying is output.

  The preferred embodiments of the present invention have been described above, but the present invention is not limited to the above embodiments, and various modifications can be made. In the above embodiment, various processes are combined to reduce the processing load as a whole and perform an accurate search. However, one or more processes may be omitted from a combination of a plurality of processes. Is possible. In this case, the processing load slightly increases or the search accuracy slightly decreases, but the effect of the present invention can be sufficiently exerted. For example, in the above-described embodiment, the collation range determining unit 70 determines the collation range corresponding to all cases shown in FIGS. 17A to 17E. The processing corresponding to any one or more of e) may be executed. If possible, it is desirable to include the two processes of FIGS. 17 (a) and 17 (b).

  In the above embodiment, as shown in FIG. 20 and FIG. 22, the feature word collating unit 90 performs the first collation in a state where the phase h is specified. When a certain degree of difference is smaller than the predetermined determination threshold value MM1, the second matching is performed while changing the phase h. However, the phase is changed without performing such a two-step matching process. Only the second collation to be changed may be performed. In the case of only the second collation, the processing load increases, but the collation accuracy does not change. Therefore, the collation range is limited by the collation range determining means 70, so that the effect of the present invention can be obtained.

  In the above embodiment, as shown in FIG. 20 and FIG. 21, the feature word collating means 90 is configured such that the W word unit mismatch bit numbers D (r, y + w, ho, x + w) are all determined threshold values MMw1. If it is smaller, it is determined whether the correction value S ′ (r, y, ho, x) of the total mismatch bit number is equal to or less than the determination threshold value MM1, and as shown in FIGS. The feature word collating means 90 corrects the total mismatch bit number correction value S ′ when all of the W word unit mismatch bit number correction values D ′ (r, y + w, h, x + w) are smaller than the determination threshold value MMw2. Whether or not (r, y, h, x) is equal to or less than the determination threshold MM2 is determined. In either case, the correction value D ′ of the word unit mismatch bit number is the determination threshold. Whether it is MMw1, MMw2 or less It may be omitted judgment. If the determination of the correction value D ′ for the word unit mismatch bit number is omitted, there may be a case where the difference between individual feature words is large, but if the overall difference is MM1 or MM2, the two are similar. This is because it can be within an allowable range. Further, it is determined whether or not the correction values D ′ (r, y + w, ho, x + w) for the W word unit mismatch bit numbers are all smaller than the determination threshold MMw1, and the correction value S ′ (r for the total mismatch bit number , Y, ho, x) omits the determination of whether or not the determination threshold value MM1 or less, and all the correction values D ′ (r, y + w, h, x + w) of the W word unit mismatch bit numbers are determined. It may be determined whether or not the correction value S ′ (r, y, h, x) of the number of unmatched bits that is smaller than the threshold MMw2 is equal to or less than the determination threshold MM2.

  In the above embodiment, the representative registration feature data is registered in the acoustic database in the related information registration device, the representative search feature data generation unit 60 generates the representative search feature data, and the representative feature data matching unit 80 The registered feature data and the representative search feature data are collated to narrow down the original acoustic data to be searched. However, if there is no need to particularly reduce the processing load, this narrowing processing is not necessarily performed.

  Moreover, in the said embodiment, although the number of the frequency bands after integration was 32 bands from n = 0 to 31, it can be appropriately increased or decreased according to the situation.

  In the above embodiment, four modes can be selected as the sound volume determination mode. However, two or three of the four modes may be set to be selectable, and one mode may be selected. It may be fixedly set.

  In the above embodiment, the case where the original sound data is not registered in the sound database due to copyright problems has been described. However, the original sound data itself is registered in the sound database as a part of the related information, and the search result is obtained. The original sound data corresponding to the extracted record may be acquired.

  The present invention relates to the protection of music copyright (monitoring illegal copying) in the field of package music for viewing for consumer and business use using CDs and DVDs, and the field of broadcasting and network music distribution distributed for commercial purposes by broadcasters and the like. ) And provision of music attribute information (music title search service).

2a, 3a ... CPU
2b, 3b ... RAM
2c, 3c ... data storage device 2d, 3d ... program storage device 2e, 3e ... key input I / F
2f, 3f ... Data I / O I / F
2g, 3g ... Display output I / F
DESCRIPTION OF SYMBOLS 10 ... Registered feature word generation means 15 ... Partial feature word generation means 20 ... Representative registration feature data generation means 25 ... Representative partial feature data generation means 30 ... Determination reference value calculation means 35 ... Registration means 40 ... Acoustic database 45 ... Mode setting means 50 ... Search feature word generation means 60 ... Representative search feature data generation means 70 ... Collation range determination means 80 ... Representative feature data Matching means 90 ... feature word matching means 100 ... information output means

Claims (10)

A search is performed by comparing the characteristics of the retrieved acoustic data, which is the given acoustic data, with the characteristics of the original acoustic data registered in the acoustic database, and specifying the original acoustic data having characteristics similar to the retrieved acoustic data. A device for performing
For each original sound data, a registered feature word group that is a set of registered feature words expressing the characteristics of the original sound data, an individual determination reference value for comparison of the registered feature word group, and related information of the original sound data The registered acoustic database,
A search feature word generation unit that generates a search feature word that expresses a feature of the search acoustic data in each section for each of the search acoustic data, and obtains a feature word group that is a set of the search feature words. When,
While matching the positional relationship between the search feature word group and the registered feature word group registered in the acoustic database, the matching is performed between the search feature word group and the registered feature word group. The minimum difference that is the degree of the minimum difference is calculated, and the minimum difference is calculated based on the ratio between the set standard minimum difference and the individual determination reference value registered corresponding to the registered feature word group. A feature word matching means for obtaining a corrected minimum difference and correcting the original acoustic data corresponding to the registered feature word group as a selection target when the corrected minimum difference is smaller than a predetermined determination threshold value; ,
An apparatus for retrieving related information of acoustic data, comprising: In claim 1,
The related information search apparatus for acoustic data, further comprising: information output means for outputting related information related to the original acoustic data specified as the selection target in an order based on the value of the corrected minimum difference. In claim 1 or claim 2,
The feature word has a feature pattern of a predetermined number of bits in which each component of a frequency spectrum obtained by analyzing acoustic data is expressed by a bit value in a predetermined frequency unit,
The feature word collating means compares the feature pattern bit by bit to collate the registered feature word group and the search feature word group to obtain a mismatch bit number, and the registered feature word group and the Acoustic data characterized in that a position that best matches a search feature word group is set to a position where the number of mismatch bits is minimum, and a minimum mismatch bit number that minimizes the number of mismatch bits is obtained as the minimum dissimilarity. Related information retrieval device. In claim 1 or claim 2,
The feature word has a feature pattern of a predetermined number of bits expressing each component of a frequency spectrum obtained by analysis of acoustic data by a bit value in a predetermined frequency unit, and volume data,
The feature word collating means compares the feature pattern bit by bit to collate the registered feature word group with the search feature word group to obtain the number of mismatch bits, and then based on the volume data of each feature word A weighted mismatch bit number is calculated by adding or multiplying the mismatch bit number with a weight, and a position where the registered feature word group and the search feature word group are most suitable is a position where the weight mismatch bit number is the smallest. An apparatus for retrieving related information of acoustic data, wherein the minimum weighted mismatch bit number that minimizes the weighted mismatch bit number is obtained as the minimum dissimilarity. In any one of Claims 1-4,
The individual determination reference value is in advance,
For the original sound data corresponding to the registered feature word group, partial sound data is obtained by partially cutting, a unit section of a predetermined length is set for the partial sound data, and the unit section is Based on the analyzed information, a set of feature words expressing the features of the partial acoustic data is generated as a partial feature word group,
While collating between the registered feature word group and the partial feature word group while shifting the temporal positional relationship, the difference between the registered feature word group and the partial feature word group is the smallest difference A related information search device for acoustic data, characterized in that the calculation is based on a minimum degree of difference which is a degree. In claim 5,
The feature word has a feature pattern of a predetermined number of bits in which each component of a frequency spectrum obtained by analyzing acoustic data is expressed by a bit value in a predetermined frequency unit,
The partial feature word group is generated for each of a plurality of partial sound data having different cut-out positions, and the registered feature word group and the partial feature word group are collated by comparing the feature patterns in bit units. The position where the registered feature word group and the search feature word group are most suitable is determined as the position where the number of mismatch bits is minimum, and the minimum mismatch is the minimum mismatch level where the number of mismatch bits is minimum. A related information search device for acoustic data, wherein the number of bits is obtained for each partial acoustic data, and the individual determination reference value is obtained based on an average value of the minimum number of mismatched bits for each partial acoustic data. In claim 5,
The feature word has a feature pattern of a predetermined number of bits expressing each component of a frequency spectrum obtained by analysis of acoustic data by a bit value in a predetermined frequency unit, and volume data,
The partial feature word group is generated for each of a plurality of partial sound data having different cut-out positions, and the registered feature word group and the partial feature word group are collated by comparing the feature patterns in bit units. After obtaining the number, a weight based on the volume data of each feature word is added to or multiplied by the number of mismatch bits to calculate a weight mismatch bit number, and the registered feature word group and the partial feature word group are most suitable. The position is the position where the number of weighted mismatch bits is minimum, the minimum weighted mismatch bit number at which the weighted mismatch bit number is minimum is determined for each partial acoustic data as the minimum difference, and the minimum for each partial acoustic data The sound for obtaining the individual determination reference value based on the average value of the number of weighted mismatch bits Over other relevant information retrieval system. In any one of Claims 1-7,
The search feature word generation means includes:
For the search acoustic data, the feature word group is generated as H search feature word groups while changing the phase h,
The feature word matching means includes
Collation of the search feature word group specifying one h out of the H phases h and the registered feature word group registered in the acoustic database is performed as a first collation, and the result of the first collation When the corrected minimum difference is smaller than the first determination threshold, the search feature word group and the registered feature word group registered in the acoustic database are collated while changing the phase h. The second matching is executed, and as a result of the second matching, the corrected minimum difference degree in the phase h that best matches the registered feature word group and the search feature word group is smaller than the second determination threshold value. In this case, the acoustic data related information search device is characterized in that the original acoustic data corresponding to the registered feature word group is specified. In any one of Claims 1-8,
The acoustic database further registers representative registered feature data generated using the registered feature word group for each original acoustic data,
Further, representative search feature data generating means for generating representative search feature data using the generated search feature word group,
Representative feature data collating means for collating the representative search feature data with the representative registered feature data registered in the acoustic database,
The feature word collating means has a value obtained by correcting the difference between the representative search feature data and the representative registered feature data using the individual determination reference value within a predetermined range as a result of collation by the representative feature data collating means. Only when it is determined that there is an acoustic data related information search device, the search feature word group is collated with a registered feature word group registered in the acoustic database.   A program for causing a computer to function as the related information retrieval apparatus for acoustic data according to any one of claims 1 to 9.

Images