JP5561041B2 - Relevant information retrieval device for acoustic data - Google Patents

Description

  The present invention relates to an apparatus for searching related information related to acoustic data in which music data such as music is recorded from an acoustic database.

  As a service to provide music attribute information that allows you to know the titles of music that has been played recently, you can query the broadcast station for the date and time of the broadcast music, and record music fragments that are being played on mobile phones. Services that collate with melodies registered in the database have been put into practical use (see, for example, Patent Documents 1 and 2). On the other hand, the applicant also has characteristics of a predetermined section of an acoustic signal (including an analog acoustic signal recorded with a microphone. A digitized signal that can be processed by a computer or the like is hereinafter referred to as “acoustic data”). Is converted into a search feature word having a predetermined number of bytes, a plurality of search feature words are set as one set, and a search feature word group in one set is compared with a registered feature word group registered in the database. A technique is proposed in which a melody obtained by recording and a melody registered in a database are collated at high speed by sequentially performing extraction of records having a high degree of match (see Patent Document 3).

JP-T-2004-536348 JP-T-2004-537760 JP 2007-226071 A

  However, in the above-described conventional method, whatever the given sound data (search sound data) is, for all search sound word groups in the search sound data and all registered sound data groups, Since collation processing is performed, there is a problem that processing load is high.

  Therefore, the present invention limits the range to be collated based on the state of the given acoustic data when searching related information related to the acoustic data registered in the acoustic database using acoustic data such as music. Accordingly, an object of the present invention is to provide a related information search device for acoustic data capable of suppressing processing load.

In order to solve the above-mentioned problem, in the first aspect of the present invention, the search sound data is collated with the characteristics of the search sound data which is given sound data and the characteristics of the original sound data registered in the sound database, and the search sound is recorded. A device that performs a search by identifying original sound data having features close to the data, and for each original sound data, a registered feature word group that is a set of registered feature words expressing the feature pattern, and the original sound data A search feature word expressing a feature pattern of the search acoustic data in each section is generated in a predetermined section unit with respect to the acoustic database in which the related information is registered and the search sound data, and a set of the search feature words The search feature word generation means for obtaining a feature word group, the search feature word group, and the registered feature word group, A collation range determination means for determining a movement range of a registered feature word to be collated in a registered feature word group as a collation range based on a relationship between a registered feature word group and a search feature word group, The collation target in the feature word group and the collation target in the registered feature word group registered in the acoustic database are collated while moving the collation target in the registered feature word group in the collation range. there dissimilarity, when the predetermined value (determination threshold M2) smaller than possess a characteristic word matching means for identifying the original sound data corresponding to the registered feature word groups as selected object, wherein the collation range determined The means is characterized in that the number of registered feature words constituting the registered feature word group is equal to the number of search feature words constituting the search feature word group. The registered feature word group is excluded from the search target when the number is smaller, and when the search acoustic data is set to start from the beginning of the acoustic material, the search feature word group is predetermined from the beginning. the number of search features words and comparison target, related from the beginning of the registered feature word groups of acoustic data, characterized in der Rukoto what determines only registered feature word of the predetermined number as a comparison target and the collation range An information retrieval apparatus is provided.

According to the first aspect of the present invention, for each original sound data, a registered feature word group, which is a set of registered feature words representing the feature pattern, is registered in the sound database, and the search sound data is predetermined. The search feature word expressing the feature pattern of the search acoustic data in each section is generated for each section, the feature word group that is a set of the search feature words is obtained, and the relationship between the registered feature word group and the search feature word group Based on the above, the movement range of the registered feature word to be collated in the registered feature word group is determined as the collation range, and the collation target in the search feature word group and the collation target in the registered feature word group are determined as the collation target in the registered feature word group. The matching feature word in the registered feature word group is determined as a result of matching. A part of the search feature word group to be collated, a search feature word group that is a collation target, and a registered feature word group that is included in the determined collation range among the registered feature word groups registered in the acoustic database; When the degree of difference, which is the degree of difference, is smaller than a predetermined value (determination threshold M2) , the original acoustic data corresponding to the registered feature word group is specified as the selection target. Therefore, when searching related information related to the acoustic data registered in the acoustic database, it is possible to reduce the processing load by limiting the range to be collated based on the state of the given acoustic data. Become. According to the first aspect of the present invention, when the number of feature words constituting the registered feature word group is smaller than the number of search feature words constituting the search feature word group, the collation range in the registered feature word group is Since the number of registered feature words is set to 0, if the performance time of the search sound data is longer than the original sound data, the original sound data can be excluded from the search target, and the processing load is reduced as a whole. It becomes possible. Further, according to the first aspect of the present invention, when the search acoustic data is set to start from the beginning of the acoustic material, a predetermined number of search feature words from the beginning of the search feature word group are to be collated. Only the same number of registered feature words from the beginning of the registered feature word group are determined as a collation target and a collation range. That is, the collation target is not moved within the collation range of the registered feature word. As a result, it is possible to perform collation processing by limiting the registered feature word group to an accurate range for the search acoustic data that is sure to start from the beginning of the acoustic material, and the processing load as a whole can be suppressed. .

Moreover, in the second aspect of the present invention, in the related information search device for acoustic data according to the first aspect of the present invention, the collation range determining means is not set so that the searched acoustic data starts from the beginning of the acoustic material. In this case, a predetermined number of search feature words from the beginning of the search feature word group are subject to collation, and others are not subject to collation, and the same number of registration as the search feature words that are not subject to collation from the top of the registered feature word group The remaining registered feature words excluding the feature words are determined as the collation range.

According to the second aspect of the present invention, when the search acoustic data is not set to start from the beginning of the acoustic material, a predetermined number of search feature words from the beginning of the search feature word group are collated and others are collated. The remaining registered feature words, excluding the same number of registered feature words as the non-matching search feature words from the beginning of the registered feature word group, are determined as the matching range from the beginning of the registered feature word group. With respect to search acoustic data that is unclear whether or not it starts, collation processing can be performed by restricting the registered feature word group to an appropriate range, and the processing load as a whole can be suppressed.

In the third aspect of the present invention, in the related information search apparatus for acoustic data according to the first aspect of the present invention, the collation range determination means is not set so that the searched acoustic data starts from the beginning of the acoustic material. In this case, a predetermined number of search feature words from the end of the search feature word group are to be collated, and others are not to be collated, and the same number of registration as the search feature words from the end of the registered feature word group that are not to be collated. The remaining registered feature words excluding the feature words are determined as the collation range.

According to the third aspect of the present invention, when the search acoustic data is not set to start from the beginning of the acoustic material, a predetermined number of search feature words from the end of the search feature word group are collated, and the others are collated. The remaining registered feature words, excluding the same number of registered feature words as the non-matching search feature words from the end of the registered feature word group, are determined as the matching range from the end of the registered feature word group. With respect to search acoustic data that is unclear whether or not it starts, collation processing can be performed by restricting the registered feature word group to an appropriate range, and the processing load as a whole can be suppressed.

In the fourth aspect of the present invention, in the related information search device for acoustic data according to the first aspect of the present invention, the collation range determination means is not set so that the searched acoustic data starts from the beginning of the acoustic material. In this case, a predetermined number of central search feature words in the search feature word group are to be collated, and others are not to be collated, and from the end of the registered feature word group, the same number of registration as the search feature words not to be collated A central registered feature word excluding the feature word is determined as the collation range.

According to the fourth aspect of the present invention, when the search sound data is not set to start from the head of the sound material, a predetermined number of search feature words in the center of the search feature word group are to be checked, and the others are checked. Since the central registered feature word that excludes the same number of registered feature words as the non-matching search feature words from the end of the registered feature word group is determined as the matching range from the end, it is determined from the beginning of the acoustic material. With respect to search acoustic data that is unclear whether or not it starts, collation processing can be performed by restricting the registered feature word group to an appropriate range, and the processing load as a whole can be suppressed.

In the fifth aspect of the present invention, in the acoustic data related information search apparatus according to any one of the first to fourth aspects of the present invention, the search feature word generation means changes the phase h of the search feature word. H types of search feature words are generated, and H search feature word groups, which are sets of the search feature words, are obtained. The feature word collating means includes W search feature words in the search feature word group, the registration The W registered feature words in the feature word group are set as the target of collation, and collation is performed with the W search feature words and the registered feature words while changing the phase h, and the word unit difference ( D (r , y + w, h, x + w)) is, when W times all at a collation predetermined determination threshold value (Mw2) smaller than, W pieces of said word units dissimilarity sum is a summation difference of the (S ( , Y, h, as the degree of difference of x)), and comparing the predetermined value (determination threshold M2).

According to the fifth aspect of the present invention, the W feature words are collated while changing the phase h, and the word unit difference ( D (r, y + w, h, x + w) ) of one feature word is When all of W collations are smaller than a predetermined determination threshold value ( Mw2 ) , the total dissimilarity ( S (r, y, h, x) ) that is the sum of W word unit dissimilarities and a predetermined value ( Since the comparison is made with the determination threshold value M2 ) , if there is a large difference between individual feature words, the collation processing as the feature word group is omitted, so that the processing load as a whole can be reduced.

  According to the present invention, when searching related information related to acoustic data registered in the acoustic database using acoustic data such as music, the range to be collated is limited based on the state of the given acoustic data. By doing so, it is possible to suppress the processing load.

It is a hardware block diagram of a related information registration apparatus. It is a functional block diagram of a related information registration device. It is a flowchart which shows the production | generation process of a registration characteristic word. It is a conceptual diagram of the generation process of the characteristic word. It is a flowchart which shows the calculation process of the representative registration feature data based on a registration feature word. It is a conceptual diagram of the calculation process of the average value and standard deviation of the registration characteristic data array by S19 and S20. It is a hardware block diagram of the related information search apparatus of the acoustic data which concerns on this invention. It is a functional block diagram of the related information search device for acoustic data according to the present invention. It is a figure which shows the outline | summary of a process of the search feature word production | generation means 50, the representative search feature data production | generation means 60, the representative feature data collation means 80, and the feature word collation means 90. FIG. It is a flowchart which shows the production | generation process of a search characteristic word. It is a flowchart which shows the calculation process of the representative search feature data based on a search feature word. It is a conceptual diagram of the calculation process of the average value of standard search feature data by S29 and S30, and a standard deviation. It is a figure which shows the concept of collation range determination. It is a figure which shows the concept of representative feature data. 5 is a flowchart of a search using a search feature word group by a feature word collating unit 90. It is a flowchart which shows the detail of the collation process of the record r in S220 of FIG. FIG. 17 is a flowchart showing details of a calculation process of a total mismatch bit number S (r, y, ho, x) in S222 of FIG. It is a flowchart which shows the detail of the collation process in a record in S224 of FIG. It is a flowchart which shows the detail of the calculation process of sum total mismatch bit number S (r, y, h, x) in S322 of FIG.

<1. Sound data related information registration device>
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. First, registration of related information of acoustic data will be described. Registration of the related information of the acoustic data is performed by a registration device for the related information of the acoustic data (hereinafter referred to as “related information registration device”). The related information registration device creates a registered feature word and representative registered feature data from the original sound data, and creates the registered feature word and representative registered feature data together with related information related to the sound data (generally called metadata). In association with information for specifying acoustic data (for example, acoustic data ID), it is registered in the acoustic database. The acoustic data ID, registered feature word, representative registered feature data, and related information related to one original acoustic data are registered in the acoustic database as one record. This acoustic database is used for searching related information of original sound data, and the original sound data itself is usually not registered. This is a copyright problem, and functionally, it is naturally possible to adopt a configuration in which the original sound data is registered.

  The acoustic data is recorded in a digital format such as music and voice, and is obtained by sampling an analog acoustic signal by a technique such as PCM. The original sound data is sound data of a sound material such as music to be searched. Because of copyright protection measures, PCM-format audio data with CD master disc quality is generally not distributed by license. Therefore, various irreversible data such as MP3 (MPEG-1 / Layer3) are pre-registered in the audio database. In general, an acoustic data file subjected to compression processing is given. However, even if the obtained data is in the MP3 format, in order to create the feature word, it is necessary to decompress the MP3 format data and generate the acoustic data of the sample sequence.

  FIG. 1 is a hardware configuration diagram of a related information registration apparatus. The related information registration device can be realized by a general-purpose computer. As shown in FIG. 1, a CPU 2a (CPU: Central Processing Unit), a main memory RAM 2b (RAM: Random Access Memory), and data A large-capacity data storage device 2c (for example, a hard disk), a program storage device 2d (for example, a hard disk) for storing programs executed by the CPU, and a key input I / F 2e such as a keyboard and a mouse And a data input / output interface 2f for data communication with an external device (data storage medium) and a display output interface 2g for sending information to a display device (display), which are connected to each other via a bus. ing.

  The program storage device 2d of the related information registration device is mounted with a dedicated program for operating the CPU 2a and causing the computer to function as the related information registration device. The data storage device 2c stores a registered feature word, representative registered feature data, and the like obtained as a processing result in association with related information, functions as an acoustic database, and stores various data necessary for processing.

  FIG. 2 is a functional block diagram of the related information registration apparatus. In FIG. 2, 10 is a registered feature word generating means, 20 is a representative registered feature data generating means, 30 is a registering means, and 40 is an acoustic database. As described above, each unit is realized by the CPU 2a executing a dedicated program read from the program storage device 2d.

  The registered feature word generation unit 10 sequentially reads a predetermined number of samples as acoustic frames from the acoustic data, performs frequency analysis using the read acoustic frames, and generates a feature word expressing the features of the acoustic data have. This feature word expresses features of certain acoustic data with a small amount of data, and is composed of a feature pattern representing the features of the spectrum and volume data. It is required that the acoustic data cannot be reproduced, i.e., cannot be duplicated, and the feature word satisfies the condition, so registration in the acoustic database is permitted). A feature word registered in the acoustic database 40 is particularly called a “registered feature word”. In addition, the sound data on which the feature word is created is particularly referred to as “original sound data”. As this "original sound data", it is normal to use the "original" that has been modified to protect the copyright, not the original data itself owned by the copyright holders. It is. Of course, the data itself that is the “original” can also be used as the “original sound data”. As will be described later, in the present invention, other feature words such as a partial feature word and a search feature word appear. These are the same as the basic structure of the feature word, but the partial feature word. Unlike the registered feature word, the search feature word is provided with a set of a plurality of feature words (five types in the present embodiment) whose phases are shifted in order to correspond to collation in which phases are shifted in search processing described later. There is a difference. In this specification, the minimum unit of 40-bit configuration is called a “feature word”, and a set of feature words used for collation is called a “feature word group”.

  The representative registered feature data generating unit 20 has a function of generating one representative registered feature data for each piece of acoustic data using the registered feature word group generated by the registered feature word generating unit 10. The registered feature word represents a partial feature of the original sound data, whereas the representative registered feature data represents the overall feature of one original sound data.

  The registration unit 30 relates the registered feature word group generated by the registered feature word generation unit 10 and the representative registered feature data generated by the representative registered feature data generation unit 20 to the production and copyright of the original original sound data. Related information (generally referred to as metadata) and registered in the acoustic database 40 in association with an ID individually defined by a company that manages copyright information of the original acoustic data in order to identify the original acoustic data. It has a function. Here, related information refers to text information that identifies the song, such as the song name and genre name, and the copyright owner and copyright owner who are involved in the production of the original acoustic data such as the lyrics, composer, arranger name, artist name, producer name, etc. Indicates text information about the name. However, the original sound data itself is not normally registered in the sound database 40 due to restrictions on the copyright law. In addition, a series of binary-format material data (individual recording data before mixing down, MIDI input data) used for production and mastering of the original sound data is normally registered due to copyright law restrictions. Absent.

<1.2. Processing operation of related information registration device>
Next, the processing operation of the related information registration apparatus shown in FIG. 2 will be described. First, in the related information registration device, the registered feature word generation means 10 generates a registered feature word from the designated original sound data. FIG. 3 is a flowchart showing a registration feature word generation process. First, the registered feature word generation means 10 reads the original sound data. In the related information registration device, the registered feature word generation means 10 reads a predetermined number of samples as one acoustic frame from the designated original acoustic data. The number of samples of one acoustic frame read by the registered feature word generation unit 10 can be set as appropriate, but is desirably about 4096 samples when the sampling frequency is 44.1 kHz. This corresponds to about 0.092 seconds. However, in consideration of the continuity between adjacent windows by using a Hanning window function in frequency conversion, which will be described later, the acoustic frame is read by overlapping a predetermined number of samples. In this embodiment, 2048 samples that are exactly half the section length of the acoustic frame are overlapped. Therefore, the first acoustic frame is sequentially read as samples 1 to 4096, the second acoustic frame is samples 2049 to 6144, and the third acoustic frame is samples 4097 to 8192.

  Subsequently, the registered feature word generation unit 10 performs frequency conversion on each read sound frame to obtain a frame spectrum that is a spectrum of the sound frame (S11). Specifically, frequency conversion is performed on the acoustic frame read by the registered feature word generation means 10 using a window function. As frequency conversion, Fourier transform, wavelet transform, and other various known methods can be used. In the present embodiment, a case where Fourier transform is used will be described as an example.

  Here, a window function used for Fourier transform in the present embodiment will be described. In general, when Fourier transform is performed on a predetermined signal, it is necessary to divide the signal into predetermined lengths. In this case, the Fourier transform (precisely, for a short time) is performed on a signal having a predetermined length. When called (Fourier transform), a pseudo component is generated in the high frequency region. Therefore, in general, when performing Fourier transform, the signal value is changed so that the weight is dropped by the cosine wave shape at the boundary of the window using a window function called a Hanning window, and then the value after the change is changed. To perform the Fourier transform.

  When the Fourier transform is performed in S11, specifically, the value X (i) (i = 0,..., N−1) in the sample i has a real value of 0 to 1 and is defined in the N sample section. Using the Hanning window function W (i) (= 0.5−0.5 cos (2πi / N)), the processing according to the first and second formulas of [Formula 1] Real part A (j) and imaginary part B (j) are obtained.

  Subsequently, a spectral component is calculated (S12). Specifically, processing according to the following [Formula 1] third formula is performed to obtain an intensity value E (j) at each frequency.

[Formula 1]
A (j) = Σ _{i = 0,..., N-1} W (i) · X (i) · cos (2πij / N)
B (j) = Σ _{i = 0,..., N-1} W (i) · X (i) · sin (2πij / N)
E (j) = A (j) ² + B (j) ²

  In [Expression 1], i is a serial number assigned to N samples in each acoustic frame, and takes an integer value of i = 0, 1, 2,... N−1. Further, j is a serial number assigned in order from the smallest value of the frequency value, and takes an integer value of j = 0, 1, 2,..., N / 2-1. When the sampling frequency is 44.1 kHz and N = 4096, if the value of j is different by one, the frequency will be different by 10.8 Hz.

  Subsequently, an integration process of spectrum components is performed (S13). Although the spectral component up to about 22 kHz is obtained by the above frequency conversion, the spectral component in the range of 340 Hz or higher and lower than about 4 kHz is used for generation of the feature word in this embodiment. This is in order to correspond to a voice compression format such as 3GPP standard used for voice reproduction of a cellular phone (however, since digital acoustic data is always given in this embodiment, a voice recording signal of a cellular phone is used. There is no need to deal with matching.) For this reason, in order to accurately match the voice reproduction range of the mobile phone, the upper limit of the feature word generation may be set to around 3.4 kHz. In the present embodiment, j = 385 to 2047 higher than the vicinity of 4 kHz among the frequency components of j = 0 to 2047 is not used. Also, the low frequency component of j = 0 to 32 that is 340 Hz or less is not used. That is, in the present embodiment, j = 33 to 384 frequency components are used. Specifically, processing according to the following [Equation 2] is executed and integrated into Pn in units of 11 frequency components.

[Formula 2]
_{P0 = (E 33 + E 34} + ... + E 43) 1/4
_{P1 = (E 44 + E 45} + ... + E 54) 1/4
:
:
P31 = ( _E374 + _E375 + ... + _E384 ) ^1/4

  According to the above [Equation 2], 352 frequency components of j = 33 to 384 are integrated into 32 frequency components of n = 0 to 31. The above process is performed for each acoustic frame, and 32 frequency components are obtained for each acoustic frame.

  Next, for each acoustic frame, a difference from the spectral component of the immediately preceding acoustic frame is calculated (S14). The processes of S11 to S13 are sequentially performed on each acoustic frame. The inter-frame difference calculation processing in S14 uses P0 to P31 obtained as a result of performing the processing up to S13 for each acoustic frame. Specifically, processing according to the following [Equation 3] is performed to obtain an inter-frame difference Dn (t).

[Formula 3]
Dn (t) = | Pn (t) −Pn (t−1) |, n = 0,..., 31

  In the above [Expression 3], Pn (t) is an integrated frequency component in the t-th acoustic frame. In this way, the difference between adjacent acoustic frames is calculated by emphasizing changes in the amplitude level and reflecting the characteristics of the acoustic data even in places where the amplitude level of the acoustic data slightly changes. It is for producing | generating.

  When the inter-frame difference calculation process is completed, it is determined whether the predetermined number of frames have been processed (S15). Specifically, it is determined whether t ≧ T. As a result, if t <T, t is incremented and the process returns to S11. If t ≧ T as a result of the determination in S15, the total sum of the obtained T differences Dn (t) is obtained (S16). That is, when the processes of S11 to S14 are sequentially performed on each acoustic frame and T differences Dn (t) between the acoustic frames are obtained (11 in this embodiment), T differences Dn (t) are obtained. Will be calculated. Specifically, processing according to the following [Equation 4] is performed to obtain a sum Sn of interframe differences.

[Formula 4]
Sn = Σ _{t = 0,..., T-1} Dn (t)

In the above [Equation 4], “Σt _{= 0,..., T−1} ” means the total sum when t is increased by 1 from t = 0 to T−1. Subsequently, the binarization process of Sn obtained by the above [Equation 4] is performed (S17). Specifically, first, the 32 Sn arrays are divided into two in the upper and lower bands of n ≧ 14 and n ≦ 13, and 1 is given to 7 of the 14 values of n ≦ 13, and 7 of the smaller value. In addition to giving 0 to each, 1 is given to 9 having a large value among 18 pieces of n ≧ 14, and 0 is given to 9 having a small value. Here, instead of simply giving 1 and 0 to 16 large values and 16 small values out of all 32 Sn, 32 bands are grouped into 18 bands with high frequency and 14 bands with low frequency. The reason why 1 and 0 are equally given in each group is to correct the influence of the frequency characteristics associated with various data compression processes. The reason why the upper and lower bands are divided at the positions of the 18 band and the 14 band is that, as a result of the experiment, when divided at this position, the search acoustic data used for the search is subjected to various data compression processes such as MP3, This is because the search accuracy was the highest. By the processing in S17, the sum Sn of inter-frame differences for each frequency band n can be expressed by 1 bit. Then, a 32-bit feature pattern Fd (y) is obtained with n = 0 as LSB and n = 31 as MSB. Here, y (= 0,..., Y−1) is a variable indicating the order of Y feature words generated from one original sound data. Therefore, y is a variable proportional to the time from the start of performance.

  Next, the volume data is calculated (S18). Specifically, first, the total volume Vol is calculated using the following [Formula 5].

[Formula 5]
Vol = Σ _{t = 0, ..., T-1} {Σ _{n = 0, ..., 31} Pn (t)}

In the above [Equation 5], “Σ _{t = 0,..., T−1} ” means the total when t is increased by 1 from t = 0 to T−1. As shown in the above [Formula 5], the values of all the components Pn (t) obtained by the integration process are added for T acoustic frames. As a result, a total volume Vol that is the total volume of the T acoustic frames is obtained. Multiplication volume is multiplied by a fixed scaling value set appropriately, and normalization is performed so as to be within the range of 0 to 255 to obtain volume data Vd (y). The sound volume data Vd (y) is represented by 8 bits by normalization. Since the volume data Vd (y) is based on the total volume Vol over T acoustic frames as shown in [Formula 5], the volume data Vd (y) is not the volume of each frame unit, but the total of T acoustic frames. It represents the volume.

  The processes of S17 and S18 can be performed by changing the order. As a result of the processing in S17 and S18, a 40-bit feature word composed of a 32-bit feature pattern and 8-bit volume data is obtained.

  By executing the above processing for each sound frame, a large number of feature words for the sound data are generated. For example, if the sampling frequency is 44.1 kHz, the sound frame is 4096 samples, and the sound frame is overlapped by 2048 samples as in the above example, one feature word is approximately 0.506 seconds, and the sound data from 5 minutes Will generate approximately 600 feature words.

  Here, the process of generating the feature word will be described with reference to the conceptual diagram of FIG. FIG. 4A is a diagram illustrating a waveform of acoustic data that is a generation target of a feature word. In the related information registration apparatus, the acoustic data is read in units of acoustic frames. However, as shown in FIG. Then, for each acoustic frame, as shown in FIG. 4C, frequency components in a predetermined frequency range are extracted and integrated into 32 bands. This corresponds to S11 to S13. Next, as shown in FIG. 4D, a difference process between adjacent acoustic frames for each band of the integrated component and a total process of the 32 band integrated components are performed. The integrated component difference process corresponds to S14, and the integrated component sum process corresponds to the sum process of each frequency component (n = 0 to 31) in S18. Next, as shown in FIG. 4E, a summation process of 32-band difference components and a summation process of volume are performed. The summation process of the 32-band difference component corresponds to S16, and the volume summation process corresponds to the summation process of each acoustic frame (t = 0 to T-1) in S18. Next, as shown in FIG. 4 (f), the binarization process of the 32-band sum component and the compression process of the sound volume (256 levels obtained by multiplying the value calculated based on the above [Formula 5] by a predetermined scaling value). To the volume level). The binarization process for the 32-band sum component corresponds to S17, and the volume compression process corresponds to S18. As shown in FIG. 4, a process of sequentially reading sound frames from sound data and generating one feature word in units of T sound frames is performed.

  When a registered feature word group (a set of registered feature words) is obtained for the entire specified original sound data, the representative registered feature data generation unit 20 then performs a representative registration using the registered feature word group. Generate feature data. The representative registered feature data is configured as a set of an average value and a standard deviation in the time direction of feature components based on the registered feature word. FIG. 5 is a flowchart showing a calculation process of an average value and a standard deviation in the time direction of feature components based on registered feature words. First, the representative registered feature data generation means 20 generates a registered feature data array by multi-value processing the registered feature word (S19). Specifically, by executing processing according to the following [Equation 6], a registered feature data array Z (n, y) (n = 0,..., 31) which is a feature component based on the registered feature word. Y = 0,..., Y-1) is generated.

[Formula 6]
When each bit n of Fd (y) is 1, Z (n, y) ← Vd (y)
When each bit n of Fd (y) is 0, Z (n, y) ← −Vd (y)

  Next, the representative registered feature data generation means 20 calculates an average value array Cd (n, r) and a standard deviation array Ld (n, r) in the time direction y of the registered feature data array Z (n, y) ( S20). Specifically, the average value array Cd (n, r) and the standard deviation array Ld (n, r) are calculated by executing processing according to the following [Equation 7].

[Formula 7]
Cd (n, r) = [Σ _{y = 0,..., Y-1} Z (n, y)] / Y
Ld (n, r) = [Σ _{y = 0,..., Y-1} (Z (n, y) −Cd (n, r)) ² / Y] ^1/2

  FIG. 6 is a conceptual diagram of the process of calculating the average value and standard deviation of the registered feature data array in S19 and S20. As shown in FIG. 6, the registered feature word has a 32-bit feature pattern F (y) and 8-bit volume data V (y) corresponding to each time y (y = 0,..., Y−1). doing. In FIG. 6, 32 bits of the feature pattern are indicated by Bit0 to Bit31.

  Depending on whether the value of bit n (Bit 0 to Bit 31) is 0 or 1 as shown in [Formula 6], the value of Z (n, y) is left as the value of the volume data, It is determined whether the value of the volume data is multiplied by −1, and the value of Z (n, y) corresponding to bit n is determined. At this time, since 8-bit volume data may be a negative value, Z (n, y) is expressed by 16 bits. In the 32-bit feature pattern, what is represented by 1 bit for each band is represented by 16 bits (2 bytes), so the registered feature data at each time y is 64 bytes.

  The average value array Cd (n, r) and standard deviation array Ld (n, r) generated as representative registered feature data by the representative registered feature data generating means 20 is information (r for identifying acoustic data such as an acoustic data ID). And one-to-one correspondence) and is registered in the acoustic database 40.

  For each original sound data, the related information about the original sound data, the sound data ID, the registered feature word group, and the representative registered feature data are registered in the sound database 40 in association with each other. The related information may be any information as long as it is related to the original sound data. For example, if the original sound data is a song, the name of the song, the name of the player, and the original sound data are CM audio. If so, the name or URL of the sponsoring company can be used. However, a series of binary-format material data (individual recording data before mixing down, MIDI input data) used for production / mastering of the original sound data is not normally subject to restrictions due to copyright laws.

<2. Related information search device>
Next, a related information search device for acoustic data according to the present invention (hereinafter referred to as “related information search device”) will be described. FIG. 7 is a hardware configuration diagram of the related information search apparatus. Similar to the related information registration device, the related information search device can be realized by a general-purpose computer. As shown in FIG. 7, a CPU 3a (CPU: Central Processing Unit) and a RAM 3b (RAM: RAM: the main memory of the computer). Random Access Memory), a large-capacity data storage device 3c (for example, hard disk) for storing data, a program storage device 3d (for example, hard disk) for storing programs executed by the CPU, a keyboard, and a mouse A key input I / F 3e, a data input / output interface 3f for data communication with an external device (data storage medium), and a display output interface 3g for sending information to a display device (display). They are connected to each other via a bus.

  The program storage device 3d of the related information search device is mounted with a dedicated program for operating the CPU 3a and causing the computer to function as the related information search device. The data storage device 3c stores registered feature words, representative registered feature data, and the like in association with related information, functions as an acoustic database, and stores various data necessary for processing. Although FIG. 7 shows an example realized by a single computer, a personal computer having a high-performance arithmetic processing function connected to a server computer on which an acoustic database is operated via a network follows a dedicated program. The contents of each means may be executed.

  FIG. 8 is a functional block diagram of the related information search apparatus according to the present invention. In FIG. 8, 40 is an acoustic database, 45 is a mode setting means, 50 is a search feature word generation means, 60 is a representative search feature data generation means, 70 is a collation range determination means, 80 is a representative feature data collation means, and 90 is a feature. A word collating unit 100 is an information output unit. The related information search device searches for related information related to the original sound data registered in the sound database as related information related to the search sound data using the search sound data held by the user. The search sound data is sound data used for search. At the time of search, it is necessary to collate a search feature word, which is a feature word generated from the search acoustic data, with a registered feature word registered in the acoustic database 40 in advance. Therefore, the search feature word and the registered feature word need to have basically the same structure (note that the former search feature word group generates a set of multiple (H) feature word groups with different phases. ) The search audio data and the original audio data that are the basis of the search feature word and the registered feature word are compressed in various encoding formats, and are generally in different encoding formats depending on the acquisition form. It needs to be converted to a format. In this embodiment, the search sound data and the original sound data are converted into the PCM format having the same specifications (sampling frequency: 44.1 kHz, quantization bit number: 16 bits, channel number: 1 / monophonic PCM format parameter). I try to unify.

  The mode setting unit 45 sets which mode is to be used for processing from among a plurality of modes provided in the related information search apparatus, and is realized by an input device such as a keyboard and a mouse and a key input I / F 3e. Is done. As modes that can be set, a search sound data mode and a sound volume determination mode are prepared. The search sound data mode indicates the state of the search sound data, and can be selected from an intro search and a full scale search. Intro search is a case where the search sound data includes the beginning of the sound material (in the case of music, it means the original music), and in the full length search, the search sound data is completely cut out from the sound material. This corresponds to the case where all the acoustic material is used, that is, the same time length as the acoustic material. The volume determination mode sets how to add a volume component to the word unit dissimilarity D (r, y + w, h, x + w), which will be described later, and can be selected from Off, Weight, Match, and Both. It has become.

  Similar to the registered feature word generation unit 10 shown in FIG. 2, the search feature word generation unit 50 performs frequency analysis using the read sound frame and generates a feature word expressing the features of the search sound data. It has a function. However, a set of a plurality of (H) feature word groups whose phases are shifted is generated. Similar to the representative registered feature data generation unit 20 shown in FIG. 2, the representative search feature data generation unit 60 uses the search feature word group generated by the search feature word generation unit 50, for each search acoustic data. It has a function of generating one representative search feature data. The search feature word expresses a partial feature of the search acoustic data, while the representative search feature data expresses the overall feature of one search acoustic data.

  The representative feature data matching unit 80 has a function of matching the generated representative search feature data with the representative registered feature data registered in the acoustic database 40. The feature word collating unit 90 has a function of collating the generated search feature word with the registered feature word registered in the acoustic database 40. The information output unit 100 has a function of extracting and outputting related information about the original sound data similar to the feature of the searched sound data as a result of the matching by the feature word matching unit 90 from the sound database 40.

  Here, an outline of the relationship among the search feature word generation unit 50, the representative search feature data generation unit 60, the representative feature data collation unit 80, and the feature word collation unit 90 will be described with reference to FIG. As shown in FIG. 9, for the original sound data, a feature word generation process is executed on the original sound data, and a plurality of obtained feature words are registered in the sound database. Further, representative feature data generation processing is executed for a plurality of feature words, and one representative feature data is registered for each original sound data. At the time of the search, the representative search feature data generation unit 60 executes the feature word generation process on the search acoustic data, and then the representative search feature data generation unit 60 executes the representative feature data generation process on the feature word, One representative feature data is generated. Then, the representative feature data matching unit 80 performs a matching process between the representative feature data obtained from the searched sound data and each representative feature data in the sound database. As a result of the collation, the feature word collating unit 90 collates the feature words only with respect to the original sound data that satisfies the condition.

  In this way, first, the representative feature data matching unit 80 generates only one for each acoustic data, and narrows down using the representative feature data representing the characteristics of the entire acoustic data, so that the acoustic features that are greatly different Exclude data from the target. And the characteristic word collation means 90 collates with respect to the comparatively similar original sound data using the characteristic word which expressed the partial feature, and can perform an exact search.

<2.2. Processing operation of related information retrieval device>
Next, the processing operation of the apparatus shown in FIG. 8 will be described. First, when the search operator wants to search the search sound data held by the search operator, the related information search device is instructed to start, and after the start, the search sound data to be searched is designated. This can be executed by operating a button on a predetermined computer screen via the key input I / F 3e and designating search acoustic data stored in the data storage device 3c of the related information search device. Actually, since the search sound data is often in a compression format such as MP3, it is processed after being converted into the PCM format. Further, the search operator sets the search mode and the sound volume determination mode by the mode setting means 45. When the setting is not performed, a normal search other than “Intro search” and “Full scale search” is executed for the search mode, and “Off” is set for the volume determination mode. The set information is written into a predetermined area of the RAM 3b by the CPU 3a, and each means can be referred to. When “Intro search” or “Full scale search” is selected in the search mode, the search acoustic data is set to start from the head of the acoustic material.

  When an instruction is input, the search feature word generation unit 50 reads a predetermined number of samples as one sound frame from the specified search sound data. This process is performed in the same manner as that performed by the related information registration apparatus. That is, the number of samples in one acoustic frame is 4096 samples when the sampling frequency is 44.1 kHz. In addition, the sound frame is read by overlapping 2048 samples.

  The processing from here to the generation of the search feature word follows the flowchart of FIG. The flowchart in FIG. 10 is substantially the same as the flowchart in FIG. 3 for the registration feature word generation. In the same manner as in S11, the search feature word generation unit 50 performs frequency conversion on each read sound frame to obtain a frame spectrum that is a spectrum of the sound frame (S21). As with the related information registration apparatus, as the frequency conversion, Fourier transform, wavelet transform, and other various known methods can be used. However, since it is necessary to match the processing of the related information registration apparatus, in this embodiment, the Fourier transform Is used.

  Subsequently, a spectral component is calculated (S22). Specifically, similarly to S12, the processing according to the above [Formula 1] third formula is performed to obtain the intensity value E (j) at each frequency.

  Subsequently, thinning processing of spectral components is performed (S23). Specifically, as in S13, the process according to the above [Equation 2] is executed and thinned out to Pn in units of 11 frequency components.

  According to the above [Expression 2], 352 frequency components of j = 33 to 384 are thinned out to 32 frequency components of n = 0 to 31. The above process is performed for each acoustic frame, and 32 frequency components are obtained for each acoustic frame.

  Next, for each acoustic frame, a difference from the spectral component of the immediately preceding acoustic frame is calculated (S24). The processes of S21 to S23 are sequentially performed on each acoustic frame. The inter-frame difference calculation processing in S24 uses P0 to P31 obtained as a result of performing the processing up to S23 for each acoustic frame. Specifically, similarly to S14, the process according to the above [Equation 3] is performed to obtain the inter-frame difference Dn (t).

  When the processes of S21 to S24 are sequentially performed on each acoustic frame and T differences (n in this embodiment) Dn (t) between the acoustic frames are obtained, a total sum of T is obtained (S26). ). That is, processing according to the following [Equation 8] is performed to obtain a sum Sn (h) of inter-frame differences. Hs in [Equation 8] is the number of sound frames in the minimum unit for shifting the phase, and is defined by (T / H. However, since hs has no meaning unless it is an integer value, in this embodiment, the decimal part is rounded down. , Hs = −2.

[Formula 8]
Sn (h) = Σ _{t = 0,..., T-1} Dn (t + h · hs)

  In the above [Equation 8], h is a phase number that identifies the phase, and is an integer that takes H values of 0 ≦ h ≦ H−1. Subsequently, the binarization process of Sn (h) obtained by the above [Equation 8] is performed (S27). Specifically, the same processing as S17 is executed for the Sn (h) array. That is, the Sn (h) array is divided into two in the upper and lower bands of n ≧ 14 and n ≦ 13, 1 is given to 7 of 14 large values of n ≦ 13, and 0 is given to 7 of the smaller values. , N ≧ 14 and 9 of the 18 large values are given 1 and 9 of the small values are given 0. By the processing in S27, Sn (h) for each n can be expressed by 1 bit. Then, a 32-bit feature pattern F (h, x) is obtained with n = 0 as LSB and n = 31 as MSB. Here, x (= 0,..., X) is a variable indicating the order of X feature words generated from the search sound data. Therefore, x is a variable proportional to the time from the start of performance.

  Next, the volume data is calculated (S28). Specifically, first, the total volume Vol (h) is calculated for each phase number h using [Formula 9] below.

[Formula 9]
Vol (h) = Σ _{t = 0,..., T-1} {Σ _{n = 0,..., 31} Pn (t + h · hs)}

  As shown in [Expression 5] above, the values of all the thinned components Pn (t + h · hs) are added for T acoustic frames. As a result, for each phase number h, a total volume Vol (h) that is the total volume for the T acoustic frames is obtained. The value of the total volume Vol is multiplied by a fixed scaling value set as appropriate, and is normalized so that it falls within the range of 0 to 255 to obtain volume data V (h, x). The sound volume data V (h, x) is represented by 8 bits by normalization. Since the volume data V (h, x) is based on the sum of volume over T acoustic frames as shown in [Equation 9], the volume data V (h, x) is not the volume of each frame unit, but T sound frames. It represents the total volume of.

  The processes of S27 and S28 can be performed by changing the order. As a result of the processing in S27 and S28, a 40-bit feature word composed of a 32-bit feature pattern and 8-bit volume data is obtained.

  By executing the above processing for each acoustic frame, a large number (X) of characteristic words for the acoustic data are generated. For example, when the sampling frequency is 44.1 kHz, the sound frame is 4096 samples, and the sound frame is overlapped by 2048 samples as in the above example, one feature word is about 0.506 seconds, and the sound data from 30 seconds Will generate about 60 (= X) feature words.

  As described above, the related information search device generates H times (in this embodiment, H = 5) feature words per unit time of the acoustic data as compared with the related information registration device. (However, as a whole, the feature words generated by the related information search device are significantly fewer than the related information registration device.) The search acoustic data to be searched for compared to the related information registration device is music. Often part of this is cut off. This is because the user may obtain it by recording a part of the music being played. For this reason, the timing of the original sound data registered in the database does not necessarily match, and a positional shift may occur. Even in the method generated in the related information registration apparatus, 11 acoustic frames are averaged and generated, so that they are relatively resistant to positional deviation. However, in the case of searched sound data with a sharp rhythm change, if the sound sound is shifted by about 5 sound frames, which is almost half of 11 sound frames, which is a feature word generation unit, significantly different feature words are generated, and erroneous information is searched. End up. Therefore, a plurality of search feature words are generated by delaying (changing the phase) by hs (= 2 sound frames) within the range of 11 sound frames as one analysis unit, and the registered feature words in the sound database 40 Try to match.

  Specifically, the related information registration device generates one feature word from frame 1 to frame 11, whereas the related information search device generates a feature word from frame 1 to frame 11 and for two acoustic frames. A feature word is also generated from a group of acoustic frames shifted (by changing the phase) by four acoustic frames, six acoustic frames, and eight acoustic frames. That is, feature words are also generated in frames 3 to 13, frames 5 to 15, frames 7 to 17, and frames 9 to 19. Eventually, by executing the processing according to the flowchart of FIG. 10 for each acoustic frame, [F (h, x), V (h, x)] ( h = 0, ..., H-1; x = 0, ..., X-1).

  After generating the H search feature word groups in consideration of the phase shift, the representative search feature data generating means 60 then generates representative search feature data using the search feature word groups. The representative search feature data is configured as a set of an average value and a standard deviation in the time direction of feature components based on the search feature word. FIG. 11 is a flowchart showing a calculation process of an average value and a standard deviation in the time direction of feature components based on a search feature word. First, the representative search feature data generation means 60 generates a search feature data array by multi-value processing the search feature words (S29). Specifically, by executing processing according to the following [Equation 10], a search feature data array Z (n, h, x) (n = 0,...) That is a feature component based on the search feature word. , 31; h = 0,..., H−1; x = 0,.

[Formula 10]
When each bit n of F (h, x) is 1, Z (n, h, x) ← V (h, x)
When each bit n of F (h, x) is 0, Z (n, h, x) ← −V (h, x)

  Next, the representative search feature data generation means 60 calculates the average value array C (n) and the standard deviation array L (n) using the search feature data array Z (n, h, x) (S30). Specifically, the average value array C (n) and the standard deviation array L (n) are calculated by executing processing according to the following [Equation 11].

[Formula 11]
C (n) = [Σx _{= 0, ..., X-1} Σh _{= 0, ..., H-} 1Z (n, h, x)] / (5X)
L (n) = [Σx _{= 0, ..., X-1} Σh _{= 0, ..., H-1} (Z (n, h, x) -C (n)) ² / (5X)] ^1/2

  FIG. 12 is a conceptual diagram of processing for calculating the average value and standard deviation of representative search feature data in S29 and S30. FIG. 12 shows a process for calculating the average value and standard deviation of the representative search feature data for a specific phase h. As shown in FIG. 12, the search feature word has a 32-bit feature pattern and 8-bit volume data corresponding to each time x (x = 0,..., X−1). In FIG. 12, 32 bits of the feature pattern are indicated by Bit0 to Bit31.

  Then, as shown in the above [Equation 10], depending on whether the value of the bit n (Bit 0 to Bit 31) is 0 or 1, the value of Z (n, h, x) remains as the value of the volume data. Or the value of the volume data is multiplied by −1, and the value of Z (n, h, x) corresponding to bit n is determined. At this time, since 8-bit volume data may be a negative value, Z (n, h, x) is expressed by 16 bits. In the 32-bit feature pattern, what is represented by 1 bit for each band is represented by 16 bits (2 bytes), so the search feature data at each time x is 64 bytes.

  When the search feature word group and the representative search feature data are obtained for the search acoustic data to be searched, the collation process is actually executed with reference to the acoustic database 40. At this time, first, the collation range determining means 70 performs processing for determining the collation range of the registered feature word. The search sound data is the sound data used by the user for the search. Therefore, the entire music is not necessarily recorded, and only a part of the music may be recorded and acquired as the sound data. . For this reason, the collation range determination means 70 determines a collation range for efficient collation on the premise that it is not known from which part of the original music the retrieved acoustic data is actually acquired. . FIG. 13 shows a conceptual diagram for determining the collation range. The horizontal width of the rectangle in FIG. 13 indicates the number of feature words constituting the feature word group.

  Specifically, first, the collation range determination means 70 compares the number X of search feature words in the search feature word group with the number Y (r) of registered feature words in the registered feature word group for a certain record r. Here, the number of search feature words in the search feature word group is compared with the number of registered feature words in the registered feature word group as being proportional to the length at the time of reproduction. Therefore, the search feature word number X is assumed to be when the phase is not considered.

  Then, as a result of comparison between the search feature word number X of the search feature word group and the registered feature word number Y (r) of the registered feature word group, when X is larger than Y (r), that is, as shown in FIG. In such a case, the collation range determination unit 70 excludes the record from the search target and moves to the next record. When X is larger than Y (r), a part of the search sound data is longer than the original sound data. Since the original sound data records the entire sound data, it becomes clear that both are based on different original sound data. Therefore, in this case, the record is excluded from the search target.

  If X is smaller than Y (r) as a result of comparison between the search feature word number X of the search feature word group and the registered feature word number Y (r) of the registered feature word group, the collation range determination means 70 searches the search feature word group. Whether or not a feature word created from the head portion of the original sound data is included. Specifically, referring to a predetermined area of the RAM 3b, it is confirmed whether or not the intro search or full scale search is set by the mode setting means 45. When these search modes are set, it is clear that the search feature word group includes a feature word created from the beginning of the original sound data, so the search sound data and the original sound data are the same. If there is, the first search feature word and the registered feature word should match. Therefore, when the search feature word group includes a feature word created from the head portion of the original sound data, the collation range determination unit 70 sets the head search feature word and the registered feature word as a collation target. I do. However, just comparing the top one feature word may result in another song having the same feature word, so that a correct collation result cannot be obtained. Accordingly, the collation range determination unit 70 performs processing for targeting α feature words corresponding to a predetermined length of time. In the present embodiment, processing is performed on a feature word group corresponding to about 12 seconds. As described above, in this embodiment, one feature word is about 0.506 seconds, so 12 seconds corresponds to 24 feature words. That is, in this case, 24 feature words from the beginning of the search feature word group and 24 feature words from the beginning of the registered feature word group are targeted for collation. FIG. 13B shows the state at this time.

  If the search feature word group does not include the feature word created from the head part of the original sound data and if α (<X) from the head part of the search feature word group is to be collated, the collation range is determined. The means 70 sets the collation range with the first search feature word (x = 0) of the search feature word group in the registered feature word group from the start (y = 0) to the last registered feature word (y = Y) of the registered feature word group. From (r) -1), the number of search feature word groups (X) is the range up to the front. That is, the collation range of the search feature word group in the registered feature word group with the head search feature word (x = 0) is as shown in FIG. 13C from the head (y = 0) to (Y (r) − X-1) Up to the registered feature word.

  In the present embodiment, as a standard setting, α feature words from the beginning of the search feature word group are targeted for collation, but α feature words from the tail of the search feature word group and the feature at the center of the search feature word group are included. It is also possible to set α words as collation targets. When α feature words from the tail of the search feature word group are set as collation targets, the collation range of the search feature word group in the registered feature word group with the first search feature word (x = 0) is as shown in FIG. As shown in (d), from (X-α) th to (Y (r) -α) th registered feature word.

When the central feature word α of the search feature word group is set as a collation target, the collation range of the search feature word group in the registered feature word group with the first search feature word (x = 0) is as shown in FIG. As shown in ( e ), from {(X−α) / 2} th to {Y (r) −α− (X−α) / 2} th registered feature word. The collation range determination unit 70 passes the collation range of the registered feature word determined as described above to the feature word collation unit 90 for each record.

  When the collation range of the registered feature word is determined, the representative feature data collating unit 80 next narrows down the records to be searched using the representative search feature data. Specifically, the representative search feature data and the representative registered feature data registered in the acoustic database 40 are compared, and only records satisfying a predetermined condition are extracted as the feature word matching means 90. And the characteristic word collation means 90 searches the relevant information of the original sound data registered into the sound database 40 using the search characteristic word group. Specifically, the search feature word group and the registered feature word group registered in the acoustic database 40 are compared, and a record satisfying a predetermined condition is extracted. The processing by the representative feature data matching unit 80 is performed to narrow down the records to be searched because the calculation load of the comparison processing using the search feature word group is high.

  Here, the concept of representative feature data performed by the representative feature data matching unit 80 will be described. FIG. 14 is a diagram illustrating the concept of representative feature data. The representative feature data has values corresponding to the 32 integrated frequency bands as shown in FIGS. Therefore, the integrated frequency band has a one-dimensional structure. In the narrowing down using the representative feature data, only the original sound data having the representative registered feature data intersecting with the representative search feature data of the search sound data plotted in the 32-dimensional space is extracted as the narrowing result.

  In FIG. 14, for the sake of conceptual illustration, a certain two-dimensional plot is plotted among the 32-dimensional representative feature data. In FIG. 14, the black circle is an average value for each acoustic data, and the circle around the black circle has a radius that is a standard deviation for each acoustic data from the black circle. Then, the original sound data whose circle does not intersect with the circle of the search sound data is excluded from the search target, and only the original sound data whose circle intersects with the circle of the search sound data is extracted as the narrowing result. Then, it can be set as appropriate whether to extract what intersects all 32 dimensions or extract what intersects a predetermined number of dimensions. In this embodiment, as shown in [Formula 7] and [Formula 11], each of the 32 frequency bands is two-dimensionalized with an average value and a standard deviation.

  Next, search processing by the representative feature data matching unit 80 and the feature word matching unit 90 will be described with reference to the flowcharts of FIGS. 15 to 19, each variable is defined as follows.

[Registration feature word]
R: Number of records (number of original acoustic data managed by the acoustic database 40)
Y (r): Number of registered feature words of record r (r = 0,..., R−1) Fd (r, y): Feature pattern array of record r (r = 0,..., R−1) (y = 0,..., Y-1), 32 bits Vd (r, y): Volume data array (y = 0,..., Y-1) of record r (r = 0,..., R-1), 8 bits Cd (N, r): average value of registered feature data array in record r (r = 0,..., R−1) Ld (n, r): registered feature in record r (r = 0,..., R−1) Standard deviation of data array

[Search feature word]
X (h): Number of search feature words in phase number h (h = 0,..., H−1) F (h, x): Feature pattern array (x = 0,..., X (h) −1), 32 Bit V (h, x): Volume data array (x = 0,..., X (h) -1), 8 bits C (n): Average value of search feature data array L (n): Search feature data array standard deviation

[Collation variable]
W: Number of collation words (w = 0,..., W-1), registered feature words to be collated, number of search feature words (eg, W = 6)
U (r): representative feature data distance D (r, y + w, h, x + w): word unit mismatch bit number of feature pattern (0 to 32)
S (r, y, h, x): the sum of the number of mismatched bits, the number of matching word counts D (r, y + w, h, x + w) (Σw _{= 0,...} , _{W -1} D (r, y + w, h, x + w))

[Judgment threshold (preset)]
Mu: Determination threshold value of representative feature data distance U (r) M1: Determination threshold value of total mismatch bit number S (r, y, ho, x) M2: Total mismatch bit number S (r, y, h, x) Determination threshold value Mw1: Determination threshold value of word unit mismatch bit number D (r, y, ho, x) Mw2: Determination threshold value of word unit mismatch bit number D (r, y, h, x) value

  The determination threshold values Mu, M1, M2, Mw1, and Mw2 can be obtained for each original sound data as follows. Mu generates a plurality of partial sound data that are randomly cut out from the original sound data when generating a registered feature word and representative registered feature data for the original sound data to be registered and registering it in the record r. , A search feature word and representative search feature data are created for each partial sound data, and a representative feature data distance U (r) calculated based on Equation 12 described later is calculated. The maximum value + β of the calculated representative feature data distances U (r) is given as Mu. (In the sense that β is set slightly larger than the maximum value, for example, β = 1). Thereby, since at least the representative feature data distance U (r) for the corresponding record r does not exceed Mu, if the representative feature data distance U (r) exceeds Mu, it can be immediately determined that the record is not applicable.

  For M1 and Mw1, the registered feature word and each of the plurality of search feature words are fixed to h = ho, and the word unit mismatch bit number D (r, y, ho, x) is determined by a method described later. Are calculated continuously, and the total number of unmatched bits S (r, y, ho, x) is calculated based on the sum of them, and the minimum total number of unmatched bits Smin (r, y, ho, x), and the maximum word unit mismatch bit number Dmax (r, y, ho, x) of each element D (r, y, ho, x) of the sum at that time is obtained, and a plurality of search feature words The maximum value + β of Smin (r, y, ho, x) obtained for each is defined as M1, and the maximum value + β of Dmax (r, y, ho, x) is defined as Mw1.

  As for M2 and Mw2, the word unit mismatch bit number D (r, y, h, x) is described later while changing each of the registered feature words and the plurality of search feature words and h in the range of 0 to H-1. In this way, a predetermined number of words are continuously calculated, and the total mismatch bit number S (r, y, h, x) is calculated based on the total sum of them, and the minimum total mismatch bit when this value is the smallest The number Smin (r, y, h, x) is calculated, and the maximum word unit mismatch bit number Dmax (r, y, h, x) of each element D (r, y, h, x) of the sum at that time is calculated. The maximum value + β of Smin (r, y, h, x) obtained for each of the plurality of search feature words is set as M2, and the maximum value + β of Dmax (r, y, h, x) is set as Mw1.

  The processing as described above is performed by causing a computer to execute a dedicated program, and different determination threshold values Mu, M1, M2, Mw1, and Mw2 can be obtained for each record. Then, the maximum value of each determination threshold value calculated for all records is set as a determination threshold value common to all records. In the present embodiment, Mu = 350, M1 = 70, M2 = 54, Mw1 = 20, and Mw2 = 13 are given. For example, record 1 is Mu = 350, M1 = 60, M2 = 54, Mw1 = 18, Mw2 = 13, and record 2 is Mu = 300, M1 = 70, M2 = 44, Mw1 = 20, Mw2 = 12. For example, Mu = 350, M1 = 70, M2 = 54, Mw1 = 20, and Mw2 = 13 are given as judgment threshold values common to both records.

  FIG. 15 is a flowchart of the acoustic data search using the search feature word group by the feature word collating unit 90. First, initial setting is performed (S210). Specifically, the matching table Smin (c) = initial value Big2, the matching table Rmin (c) = − 1, the matching number c = 0, and the record number r = 0 are set. The initial value Big2 may be a value sufficiently larger than a value that can be taken as the minimum number of mismatch bits, and is set in advance.

  Next, the feature word matching unit 90 calculates the representative feature data distance U (r) (S211). Specifically, processing according to the following [Equation 12] is executed to calculate the representative feature data distance U (r) that is the distance between the representative registered feature data and the representative search feature data for the record r.

[Formula 12]
U (r) = 256 × [Σ _{n = 0,..., 31} {(Cd (n, r) −C (n)) / (Ld (n, r) + L (n))} ² ] ^1/2 / [Σ _{n = 0, ..., 31} {Cd (n, r) / (Ld (n, r) + L (n))} ² ] ^1/4 / [Σ _{n = 0, ..., 31} {(C (n ) / (Ld (n, r) + L (n))} ² ] ^1/4

In the above [Equation 12], there are three terms enclosed by []. The square root of the first [] is divided by the fourth root of the second [] and the fourth root of the third []. Is multiplied by a coefficient “256” for normalization. The coefficient for normalization can be appropriately changed according to the range of normalization. In the above [Expression 12], “Σ _{n = 0,..., 31} ” means the total of 32 pieces when n is incremented by 1 from n = 0 to 31. As the power root of each term enclosed in [], the specific power can be changed as appropriate, but in this embodiment, as shown in [Formula 12], the square root, the fourth root, the fourth power Roots.

[Equation 12] indicates the distance between the representative search feature data and the representative registered feature data, and is defined based on the normalized correlation coefficient of both. Specifically, each element n of the representative search feature data is represented by the sum of the standard deviation value of the search feature data array and the standard deviation value of the registered feature data array with respect to the average value of the search feature data array. Each value is defined by being normalized and divided by (C (n) / (Ld (n, r) + L (n)). Similarly, each element of the representative registered feature data is registered in the registered feature data array. Is defined by each value normalized by dividing each of the 32 elements by the sum of the standard deviation value of the search feature data array and the standard deviation value of the registered feature data array with respect to the average value of Cd (n, r) / (Ld (n, r) + L (n)) The normalized correlation coefficient between these 32 elements is [Σ _{n = 0,..., 31} {(Cd (n, r) -C (n)) / (Ld (n, r) + L (n))} 2] / [Σ n = 0, ..., 31 {Cd (n, r) / (Ld (n, r) + L (n ^{)} 2] 1/2 / [Σ} n = 0, ..., 31 {(C (n) / (Ld (n, r) + L (n))} 2] is given by ^1/2, -1 to +1 In the above [Equation 12], the normalized correlation coefficient value is multiplied by a predetermined integer value 256 to give an integer expression, and the fluctuation range of the integer value is expanded to show the corresponding record and the non-corresponding record. In order to make the difference, the normalized correlation coefficient value is used as it is, whether it is the square root or the fourth root. Just choose.

  The representative feature data distance U (r) corresponds to the distance between the circle of the search sound data and the circle of the original sound data in FIG. In reality, the determination of the target to be extracted as the narrowing-down result may be set as appropriate, for example, when the circle and the circle shown in FIG. 14 intersect or when the circle and the circle are within a predetermined distance range. it can. In the present embodiment, a narrowing target is determined by comparing the representative feature data distance U (r) calculated by [Equation 12] with a determination threshold value.

  Next, the feature word collating unit 90 compares the calculated representative feature data distance U (r) with a preset determination threshold value Mu (S212). As a result of the comparison, if the representative feature data distance U (r) is greater than or equal to the threshold value Mu, the process proceeds to S250, where r is incremented, and the process for the next record r + 1 is performed. When the representative feature data distance U (r) is greater than or equal to the determination threshold value Mu, detailed collation calculation for the record r is not performed. That is, in this embodiment, the search target is narrowed down by the value of the representative feature data distance U (r).

  As a result of the comparison in S212, when the representative feature data distance U (r) is smaller than the determination threshold value Mu, the registered feature word group registered in association with the record r is collated with the search feature word group ( S220). As a result of the collation in S220, the record number r is given to Rmin (c) and outputted. The process of S220 will be described later.

  If Rmin (c) is obtained, it is determined whether Rmin (c) is 0 or more (S230). When Rmin (c) is 0 or more, it is determined that the record is matched, and a process of adding 1 to the number of matching cases c is performed (S240). If Rmin (c) is less than 0, it is determined that the record has not been matched, and the matching number c is not added. As will be described in detail later, in S220, a value less than 0 is set as an initial value in Rmin (c), and when it is determined that the record is suitable, a record number r that is 0 or more is set to Rmin (c). To give. For this reason, in S240, the suitability of the record is determined by determining whether Rmin (c) is 0 or more.

  Next, the variable r specifying the record is incremented, that is, increased by 1 (S250). Then, it is determined whether or not the variable r for specifying the record has reached the total number R of records in the acoustic database 40 (S260). If not, the process returns to S220 to perform the collation process for the next record r. . When processing is performed for each record r and the record r reaches the total number of records R, that is, when processing for all R total records is completed, the matching table Rmin (c) is changed to the value of the matching table Smin (c). Based on the ascending order, the list is output together with the matching number c (S270).

  Next, details of the record r matching process in S220 of FIG. 15 will be described using the flowchart of FIG. First, initial setting is performed (S221). Specifically, the matching table Smin (c) = initial value Big2, the matching table Rmin (c) = − 1, the variable x = 0 specifying the search feature word, the variable h = ho specifying the phase, and the registered feature word Set the variable y to be specified to 0 As for ho, when H = 5, it can be set to any of ho = 0, 1, 2, 3, and 4, but it is set to ho = 0 with the least amount of normal arithmetic processing. Subsequently, the total mismatch bit number S (r, y, ho, x) is calculated (S222). When the total mismatch bit number S (r, y, ho, x) is obtained, it is determined whether or not the total mismatch bit number S (r, y, ho, x) is equal to or less than the determination threshold M1 (S223). ). The value of the determination threshold value M1 can be set as appropriate, and is set in advance in the present embodiment. When the total mismatch bit number S (r, y, ho, x) is larger than the determination threshold value M1, the process proceeds to S225, x is incremented, and the process for the next search feature word x + 1 is performed. When the total mismatch bit number S (r, y, ho, x) is equal to or less than the determination threshold M1, the registered feature word y in the specified (x, y) is collated with the search feature word x ( S224).

  Next, the variable x is incremented (S225). Then, x and X (h) are compared (S226). As a result of the comparison, if x does not reach X (h), the process returns to S222 and the process for the next search feature word x + 1 is performed. If x reaches X (h) as a result of the comparison, x is incremented with x = 0, and the processing of S222 to S227 is repeated. When X (h) becomes H, x = 0 and y Is incremented (S227). Then, y is compared with Y (r) (S228). If y does not reach Y (r) as a result of the comparison, the process returns to S222 and the process for the next registered feature word y + 1 is performed.

  If y reaches Y (r) as a result of the comparison, the processing for all Y (r) registered feature words has been completed. Therefore, the matching table Rmin (c) and the matching table Smin at that time point. (C) is output. The matching table Rmin (c) and the matching table Smin (c) are obtained in S220 of FIG.

  Next, the details of the calculation processing of the total mismatch bit number S (r, y, ho, x) in S222 of FIG. 16 will be described with reference to the flowchart of FIG. First, initial setting is performed (S281). Specifically, the total mismatch bit number S (r, y, ho, x) = 0 and the variable w = 0 indicating the number of feature word collations are set. After the initial setting, it is determined whether or not the condition that the volume data Vd (r, y + w) of the registered feature word and the volume data V (h, x + w) of the search feature word are both greater than 0 is satisfied (S282). The volume data Vd (r, y + w) of the registered feature word and the volume data V (h, x + w) of the search feature word are in accordance with [Formula 5] and [Formula 9] with respect to the original sound data and the retrieved sound data, respectively. The process is executed, and the calculated “Vol” and “Vol (h)” are normalized. As a result of the determination in S282, the registered feature word 1 only when the volume data Vd (r, y + w) of the registered feature word and the volume data V (z, h, x + w) of the search feature word are both greater than 0. The word unit mismatch bit number D (r, y + w, ho, x + w), which is the number of mismatch bits, is compared with one search feature word (S283).

  Next, it is determined whether or not the word unit mismatch bit number D (r, y + w, ho, x + w) is equal to or less than the determination threshold value Mw1 (S284). The value of the determination threshold value Mw1 can be set as appropriate, and is set in advance in the present embodiment. As a result of the determination in S284, the word unit mismatch bit number D (r, y + w, ho, x + w) is obtained only when the word unit mismatch bit number D (r, y + w, ho, x + w) is equal to or less than the determination threshold value Mw1. A process of adding to the total mismatch bit number S (r, y, ho, x) is performed (S285).

  In S285, a process of incrementing the variable w indicating the number of feature word collations is also performed. Then, it is determined whether or not the variable w indicating the number of collated feature words has reached a predetermined number W (S286). If not, the process returns to S282 and the next feature word is processed. When processing is performed for each feature word and the variable w reaches a predetermined number W, the total mismatch bit number S (r, y, ho, x) at that time is calculated for a certain x, h (= 0), y. Output as the total mismatch bit number S (r, y, ho, x). This sum mismatch bit number S (r, y, ho, x) is obtained in S222 of FIG. If it is determined that the condition is not satisfied in S282 and S284, a calculation error of the number of total mismatch bits is output.

  Next, details of the intra-record matching process in S224 of FIG. 16 will be described using the flowchart of FIG. First, initial setting is performed (S321). A variable h = 0 specifying the phase is set. Subsequently, the sum mismatch bit number S (r, y, h, x) is calculated (S322). When the total mismatch bit number S (r, y, h, x) is obtained, it is determined whether or not the total mismatch bit number S (r, y, h, x) is equal to or smaller than the determination threshold M2 (S323). ). The value of the determination threshold value M2 can be set as appropriate, and is set in advance in the present embodiment. When the total mismatch bit number S (r, y, h, x) is equal to or less than the determination threshold value M2, the total mismatch bit number S (r, y, h, x) is the total mismatch bit number minimum value Smin (c It is determined whether it is smaller than (S324).

  Only when the total mismatch bit number S (r, y, h, x) is smaller than the total mismatch bit number minimum value Smin (c), the value of r is set in the matching table Rmin (c), and S (r, y , H, x) is set in the matching table Smin (c) (S325). Next, the variable h is incremented (S326). Then, h and H are compared (S327). If h does not reach H as a result of the comparison, the process returns to S322 and the process for the next phase number h + 1 is performed.

  If h has reached H as a result of the comparison, the processing for all H phases has been completed, and the matching table Rmin (c) and matching table Smin (c) at that time are output. The matching table Rmin (c) and the matching table Smin (c) are obtained in S224 of FIG.

  Next, details of the calculation processing of the total mismatch bit number S (r, y, h, x) in S322 of FIG. 18 will be described using the flowchart of FIG. First, initial setting is performed (S381). Specifically, the total mismatch bit number S (r, y, h, x) = 0 and the variable w = 0 indicating the number of feature word collations are set. After the initial setting, it is determined whether or not the condition that the volume data Vd (r, y + w) of the registered feature word and the volume data V (h, x + w) of the search feature word are both greater than 0 is satisfied (S382). As a result of the determination in S382, the registered feature word 1 only when the volume data Vd (r, y + w) of the registered feature word and the volume data V (z, h, x + w) of the search feature word are both greater than 0. The word unit mismatch bit number D (r, y + w, h, x + w), which is the number of mismatch bits when one search feature word is compared, is calculated (S383).

  Next, it is determined whether or not the word unit mismatch bit number D (r, y + w, h, x + w) is equal to or less than the determination threshold value Mw2 (S384). The value of the determination threshold value Mw2 can be set as appropriate, and is set in advance in the present embodiment. As a result of the determination in S384, the word unit mismatch bit number D (r, y + w, h, x + w) is obtained only when the word unit mismatch bit number D (r, y + w, h, x + w) is equal to or less than the determination threshold value Mw2. A process of adding to the total mismatch bit number S (r, y, h, x) is performed (S385). In S385, a process of incrementing the variable w indicating the number of feature word collations is also performed. Then, it is determined whether or not the variable w indicating the number of feature words to be collated has reached a predetermined number W (S386). If not, the process returns to S382 and the next feature word is processed. When processing is performed for each feature word and the variable w reaches a predetermined number W, the total mismatch bit number S (r, y, h, x) at that time is calculated as the total mismatch bit number for a certain x, h, y. Output as S (r, y, h, x). This sum mismatch bit number S (r, y, h, x) is obtained in S322 of FIG. If it is determined in S382 and S384 that the condition is not satisfied, an error in calculating the total number of mismatch bits is output.

<2.3. Calculation of word unit mismatch bit count>
The calculation of the word unit mismatch bit number D (r, y + w, h, x + w) in S283 in FIG. 17 and S383 in FIG. 19 will be described. Regarding the calculation of the word unit mismatch bit number D (r, y + w, h, x + w), the specific processing contents differ depending on the sound volume determination mode set by the user. There are four volume determination modes: Off, Weight, Match, and Both.

<2.3.1. Volume judgment mode “Off”>
The volume determination mode “Off” is a mode in which no weight is added, and when the volume determination mode “Off” is set, the word unit mismatch bit number D (r, y + w, h, x + w) is directly different in word units. This is a word unit dissimilarity D (r, y + w, h, x + w) indicating the degree of. In the sound volume determination mode “Off”, the initial value is set with the word unit mismatch bit number D (r, y + w, h, x + w) = 0, and then 32 bits of Fd (r, y + w) and F (h, x + w). Are sequentially compared in corresponding bit units, and 1 is added to D (r, y + w, h, x + w) every time the bits differ. In both the registered feature word group and the search feature word group, the feature pattern of the feature word is created according to the same rule, and has a 32-bit configuration in which the low frequency component is LSB and the high frequency component is MSB. This can be done depending on whether or not the bit values match.

<2.3.2. Volume judgment mode “Weight”>
The sound volume determination mode “Weight” is a mode for adding a weight, and when the sound volume determination mode “Weight” is set, the initial value is set with the word unit mismatch bit number D (r, y + w, h, x + w) = 0. After that, the 32 bits of Fd (r, y + w) and F (h, x + w) are sequentially compared in corresponding bit units. Based on the result of the comparison, processing according to the following [Equation 13] is executed to determine the value of D (r, y + w, h, x + w). As a result, D (y + w, z, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account volume data. In this mode, when the registered feature word is matched with the search feature word, the search acoustic data that is the basis of the search feature word is accompanied by signal processing such as analog conversion. Appropriate matching results are given when both absolute values are different.

[Formula 13]
When the Fd (r, y + w) side is bit 1 and the F (h, x + w) side is bit 0,
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) + Vd (r, y + w) · 2 / {Vd (r, y + w) + V (h, x + w)}
When the Fd (r, y + w) side is bit 0 and the F (h, x + w) side is bit 1,
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) + V (h, x + w) · 2 / {Vd (r, y + w) + V (h, x + w)}

<2.3.3. Volume judgment mode “Match”>
When the sound volume determination mode “Match” is set, first, processing in the sound volume determination mode “Off” is performed to obtain D (r, y + w, h, x + w). Then, by executing the processing according to the following [Equation 14], the value of D (r, y + w, h, x + w) is determined by multiplying the weight. As a result, D (y + w, z, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account the difference in the fluctuation pattern of the volume data. In this mode, when the registered feature word and the search feature word are collated, the search acoustic data that is the basis of the search feature word is accompanied by signal processing such as various data compression. Although there is not much difference, an appropriate matching result is given when the absolute values are different. In this embodiment, this mode is most recommended.

[Formula 14]
When Vd (r, y + w) · Vd (r, y + w−1)> V (h, x + w) · V (h, x + w−1),
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) · Vd (r, y + w) · Vd (r, y + w−1) / {V (h, x + w) · V (h, x + w-1)}
When Vd (r, y + w) · Vd (r, y + w−1) <V (h, x + w) · V (h, x + w−1),
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) · V (h, x + w) · V (h, x + w−1) / {Vd (r, y + w) · Vd (r, y + w-1)}

  When w = 0, in the above [Equation 14], Vd (r, y + w−1) = Vd (r, y + w) and V (h, x + w−1) = V (h, x + w).

<2.3.4. Volume judgment mode “Both”>
When the sound volume determination mode “Both” is set, first, processing in the sound volume determination mode “Weight” is performed to obtain D (r, y + w, h, x + w). Then, by executing processing according to the following [Equation 15], the value of D (r, y + w, h, x + w) is determined by multiplying the weight. As a result, D (y + w, z, h, x + w) is not a word unit mismatch bit number but a word unit dissimilarity indicating the degree of difference in word units taking into account volume data. In this mode, when the registered feature word and the search feature word are collated, the search acoustic data that is the basis of the search feature word is accompanied by signal processing such as high-compression data compression or analog conversion with waveform distortion. An appropriate collation result is given when both the original acoustic data, the relative change in volume, and the absolute value of volume are significantly different.

[Formula 15]
When Vd (r, y + w) · V (h, x + w−1)> V (h, x + w) · Vd (r, y + w−1),
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) · Vd (r, y + w) · V (h, x + w−1) / {V (h, x + w) · Vd (r, y + w-1)}
When Vd (r, y + w) · V (h, x + w−1) <V (h, x + w) · Vd (r, y + w−1),
D (r, y + w, h, x + w) ← D (r, y + w, h, x + w) · V (h, x + w) · Vd (r, y + w−1) / {Vd (r, y + w) · V (h, x + w-1)}

  When w = 0, in the above [Expression 15], Vd (r, y + w−1) = Vd (r, y + w) and V (h, x + w−1) = V (h, x + w).

  In cases other than the sound volume determination mode “Off”, D (r, y + w, h, x + w) is not the number of word unit mismatch bits, but the word unit dissimilarity indicating the degree of difference in word units taking into account the sound volume data. In FIG. 16 and FIG. 17, S (r, y, h, x) represents the total dissimilarity, not the total mismatch bit number. Further, Smin in FIGS. 15 and 16 represents not the minimum mismatch bit number but the minimum dissimilarity.

  Eventually, in S270 of FIG. 15, the matching table Rmin (c) is sorted in ascending order based on the value of the matching table Smin (c) and is listed together with the number of matching cases c. This is searched using the search acoustic data. It becomes a search result. When one is selected from the output list, the information output unit 100 extracts the record from the acoustic database 40. At that time, when an instruction to acquire the original sound data is given, the information output means 100 uses the file name recorded in the record to obtain related information related to the original sound data stored in the storage device or the original sound data. An ID for specifying is output.

  The preferred embodiments of the present invention have been described above, but the present invention is not limited to the above embodiments, and various modifications can be made. In the above embodiment, various processes are combined to reduce the processing load as a whole and perform an accurate search. However, one or more processes may be omitted from a combination of a plurality of processes. Is possible. In this case, the processing load slightly increases or the search accuracy slightly decreases, but the effect of the present invention can be sufficiently exerted. For example, in the above embodiment, the collation range determination means 70 determines the collation range corresponding to all cases shown in FIGS. 13A to 13E. The processing corresponding to any one or more of e) may be executed. If possible, it is desirable to include the two processes shown in FIGS.

  In the above embodiment, as shown in FIGS. 16 and 18, the feature word collating unit 90 performs the first collation in a state in which the phase h is specified. When a certain degree of difference is smaller than the predetermined determination threshold value M1, the second matching is performed while changing the phase h. However, the phase is changed without performing the two-step matching process. Only the second collation to be changed may be performed. In the case of only the second collation, the processing load increases, but the collation accuracy does not change. Therefore, the collation range is limited by the collation range determining means 70, so that the effect of the present invention can be obtained.

  In the above embodiment, as shown in FIG. 16 and FIG. 17, the feature word collating unit 90 determines that the W word unit mismatch bit numbers D (r, y + w, ho, x + w) are all determined threshold values Mw1. If it is smaller, it is determined whether or not the sum mismatch bit number S (r, y, ho, x) is equal to or less than the determination threshold value M1, and as shown in FIGS. 90, when all the W word unit mismatch bits D (r, y + w, h, x + w) are smaller than the determination threshold Mw2, the total mismatch bit number S (r, y, h, x) is determined. Whether or not the threshold value M2 is equal to or less than the threshold value M2 is determined. In any case, the determination as to whether or not the word unit mismatch bit number D is equal to or less than the determination threshold values Mw1 and Mw2 may be omitted. . If the determination of the word unit mismatch bit number D is omitted, there may be a case where the difference between the individual characteristic words is large, but if the overall difference is M1 or M2, the two are similar. This is because the allowable range can be made. In addition, the determination as to whether all the W word unit mismatch bits D (r, y + w, ho, x + w) are smaller than the determination threshold Mw1 and the total mismatch bit number S (r, y, ho, x) are The determination of whether or not the threshold value is less than or equal to the determination threshold value M1 is omitted, and the W word unit mismatch bit numbers D (r, y + w, h, x + w) are all smaller than the determination threshold value Mw2 and the total mismatch bit number S It may be determined only whether (r, y, h, x) is equal to or less than the determination threshold value M2.

  In the above embodiment, the representative registration feature data is registered in the acoustic database in the related information registration device, the representative search feature data generation unit 60 generates the representative search feature data, and the representative feature data matching unit 80 The registered feature data and the representative search feature data are collated to narrow down the original acoustic data to be searched. However, if there is no need to particularly reduce the processing load, this narrowing processing is not necessarily performed.

  Moreover, in the said embodiment, although the number of the frequency bands after integration was 32 bands from n = 0 to 31, it can be appropriately increased or decreased according to the situation.

  In the above embodiment, four modes can be selected as the sound volume determination mode. However, two or three of the four modes may be set to be selectable, and one mode may be selected. It may be fixedly set.

  In the above embodiment, the case where the original sound data is not registered in the sound database due to copyright problems has been described. However, the original sound data itself is registered in the sound database as a part of the related information, and the search result is obtained. The original sound data corresponding to the extracted record may be acquired.

  The present invention relates to the protection of music copyright (monitoring illegal copying) in the field of package music for viewing for consumer and business use using CDs and DVDs, and the field of broadcasting and network music distribution distributed for commercial purposes by broadcasters and the like. ) And provision of music attribute information (music title search service).

2a, 3a ... CPU
2b, 3b ... RAM
2c, 3c ... data storage device 2d, 3d ... program storage device 2e, 3e ... key input I / F
2f, 3f ... Data I / O I / F
2g, 3g ... Display output I / F
DESCRIPTION OF SYMBOLS 10 ... Registration feature word generation means 20 ... Representative registration feature data generation means 30 ... Registration means 40 ... Acoustic database 45 ... Mode setting means 50 ... Search feature word generation means 60 ... Representative search feature data generation means 70: collation range determination means 80 ... representative feature data collation means 90 ... feature word collation means 100 ... information output means

Claims (6)

A search is performed by comparing the characteristics of the retrieved acoustic data, which is the given acoustic data, with the characteristics of the original acoustic data registered in the acoustic database, and specifying the original acoustic data having characteristics similar to the retrieved acoustic data. A device for performing
For each original sound data, a registered feature word group that is a set of registered feature words expressing the feature pattern, an acoustic database in which related information of the original sound data is registered,
A search feature word that generates a search feature word that represents a feature pattern of the search acoustic data in each section for each of the search acoustic data, and obtains a search feature word group that is a set of the search feature words Generating means;
In the search feature word group and the registered feature word group, a part of the same number of feature words is set as a verification target, and the verification target registration in the registered feature word group is performed based on the relationship between the registered feature word group and the search feature word group. Collation range determining means for determining the movement range of the feature word as a collation range;
The collation target in the search feature word group and the collation target in the registered feature word group registered in the acoustic database are collated while moving the collation target in the registered feature word group in the collation range. When the degree of difference is smaller than a predetermined value , a feature word matching unit that identifies the original acoustic data corresponding to the registered feature word group as a selection target;
I have a,
The collation range determining means excludes the registered feature word group from the search target when the number of registered feature words constituting the registered feature word group is smaller than the number of search feature words constituting the searched feature word group. When the search acoustic data is set to start from the beginning of the acoustic material, a predetermined number of search feature words from the beginning of the search feature word group are to be collated, and the registered feature word group Related information retrieval apparatus of the sound data, characterized in der Rukoto what determines only registered feature word of the predetermined number as a comparison target and the collation range from the beginning of the. Oite to claim 1,
When the search acoustic data is not set to start from the beginning of the acoustic material, the collation range determination means is a collation target for a predetermined number of search feature words from the top of the search feature word group, and the other is a collation target The remaining registered feature words excluding the same number of registered feature words as the search feature words that are not to be collated from the beginning of the registered feature word group are determined as the collation range. Related information retrieval device. Oite to claim 1,
The collation range determining means is a collation target for a predetermined number of search feature words from the end of the search feature word group when the search acoustic data is not set to start from the beginning of the acoustic material, and the other collation target And the remaining registered feature words excluding the same number of registered feature words as the search feature words that are not to be collated from the end of the registered feature word group are determined as the collation range. Related information retrieval device. Oite to claim 1,
When the search acoustic data is not set to start from the beginning of the acoustic material, the collation range determination means is a collation target for a predetermined number of search feature words in the center of the search feature word group, and the other is a collation target The central registered feature word excluding the same number of registered feature words as the non-matching search feature words from the end of the registered feature word group is determined as the matching range. Related information retrieval device. In any one of Claims 1-4 ,
The search feature word generation means generates H search feature words while changing the phase h, and obtains H search feature word groups that are sets of the search feature words.
The characteristic word matching means, W number of search features word in the search feature word group, said W-pieces of registered feature words in the registered feature word groups as the checking target, W number of search features word while changing the phase h and collates the registration feature word, 1 wherein the word-word units dissimilarity each other, when all the W times of matching small Ri by the predetermined determination threshold value is the sum of the W number of the word unit dissimilarity The related information search device for acoustic data , wherein the sum difference is compared with the predetermined value as the difference . The program for functioning a computer as a related information search apparatus of the acoustic data as described in any one of Claims 1-5 .

Images