JP6588212B2 - Sound source search apparatus and sound source search method - Google Patents

Description

  The present invention relates to a sound source search device and a sound source search method.

  Conventionally, a sound source search device is known (see, for example, Non-Patent Document 1).

  Non-Patent Document 1 discloses a sound source search device using a mixed sound as a search key. This sound source search apparatus is configured to extract a feature amount from a mixed sound and to search a sound source from a sound source database using the extracted feature amount as a search key. Here, a chroma spectrum is used as the feature amount. A chroma spectrum is obtained by calculating a Fourier spectrum in each analysis frame of a mixed sound (signal) having a predetermined time length and then calculating an output energy of each band window. The element of the chroma spectrum is a scalar quantity (for example, a single precision floating point number, 32 bits). The sound source is searched for by pattern matching (the Euclidean distance between the feature amount vectors) of the feature amount vector of the mixed sound and the feature amount vector of the sound source in the sound source database.

“Evaluation of sound source retrieval system from mixed sound using cumulative distance between features”, IEICE Technical Report, vol. 114, no. 191, pp. 19-24.

  However, in the sound source search device described in Non-Patent Document 1, the chroma spectrum is used as a feature value, but there is a problem that the search accuracy is lowered due to a change in the sound pressure of the music signal. In addition, since a chroma spectrum having a scalar quantity element is used as a feature quantity, it takes time to perform pattern matching (search) between the feature quantity vector of the mixed sound and the feature quantity vector of the sound source in the sound source database. is there.

  The present invention has been made to solve the above-described problems, and one object of the present invention is to increase the search speed in a sound source search apparatus and sound source search method using a mixed sound as a search key. A sound source search apparatus and a sound source search method capable of improving the search accuracy.

In order to achieve the above object, a sound source search device according to a first aspect of the present invention includes a search device-side feature amount extraction unit that extracts a feature amount from a mixed sound including a music signal and a sound signal, and the extracted feature amount. And binarizing means for searching for a sound source from a sound source database using the characteristic value of the mixed sound binarized by the binarizing means for the searching apparatus as a search key. The feature amount of the mixed sound is a chroma spectrum that is output energy of each band window calculated based on the Fourier spectrum of the mixed sound, and the feature amount of the mixed sound is one analysis frame having a predetermined time length or are extracted for each of a plurality of analysis frame, the search apparatus binarizing means, the feature quantity of the mixed sound, where based on the chroma spectrum of feature quantity at 1 per analysis frame or multiple analyzes each frame Of the characteristic quantity 1 when above the reference value, and is configured to feature quantity in the case of less than the predetermined reference value to zero.

  In the sound source search device according to the first aspect of the present invention, as described above, by providing the search device side binarization means for binarizing the extracted feature value, a scalar quantity (for example, a single precision floating point number, Compared with the case where a sound source is searched from a sound source database using a feature amount having 32 bits) as a search key, the dimension is reduced (1 bit) as the feature amount is binarized, so that the search speed is increased. Can do.

  In addition, when an audio signal is mixed with a music signal, the envelope (shape) of the music signal changes as much as the audio signal is mixed. Therefore, in the present invention, by providing a binarization unit on the search device side that binarizes the extracted feature value, the voice signal is additively added to the portion “1” of the feature value after binarization. As long as it works, it remains “1”. On the other hand, even if the voice signal acts additively on the “0” portion of the binarized feature value, it remains “0” unless the reference value for binarization is exceeded. In the vicinity of the threshold value for binarization, the audio signal is mixed, so that the binarized feature quantity “0” or “1” may be reversed, while the audio signal output Since the frequency with high energy (frequency that can be inverted) is a relatively small range only in the vicinity of a frequency that is an integral multiple of the fundamental frequency, the influence of inversion is considered to be small. As a result, the binarized feature value is a robust feature value with respect to the mixed audio signal while representing the envelope (shape) of the music signal. In addition, since the predetermined reference value can be changed in accordance with the change in the volume of the mixed sound even when the sound pressure of the music signal changes, the change in the characteristic amount of the mixed sound (the characteristic amount is “0”). "Or change in the determination of" 1 ") is prevented. This point has been confirmed by the inventors' experiments. As a result, the search speed can be increased and the search accuracy can be improved.

  In the sound source search apparatus according to the first aspect, preferably, the sound source database includes database-side feature amount extraction means for extracting feature amounts from the music signal for database, and database-side binarization for binarizing the extracted feature amounts. And a construction means for constructing a sound source database from the feature value of the database music signal binarized by the database-side binarization means, and the feature value of the database music signal is the Fourier spectrum of the database music signal. Is a chroma spectrum that is the output energy of each band window calculated based on the database, and the feature amount of the music signal for the database is extracted for each analysis frame or a plurality of analysis frames having a predetermined time length. The database-side binarization means determines that the feature quantity of the music signal for the database is one analysis frame. The characteristic amount is 1 in the case of more than a predetermined reference value of the feature quantity at each of a plurality of analysis frame, and is configured to feature quantity in the case of less than the predetermined reference value to zero. With this configuration, since the feature amount of the sound source database is binarized, compared to a case where the sound source database is constructed from a feature amount having a scalar amount (for example, a single precision floating point number, 32 bits). As the feature value is binarized, the dimension is reduced (1 bit), so the database size of the sound source database can be reduced. As a result, the search speed can be increased.

A sound source search method according to a second aspect of the present invention includes a step of extracting a feature amount from a mixed sound including a music signal and a sound signal, a step of binarizing the extracted feature amount, and a binarized mixture And a step of searching for a sound source from a sound source database using the sound feature amount as a search key. The feature amount of the mixed sound is extracted for each analysis frame or a plurality of analysis frames having a predetermined time length. The step of extracting the feature amount from the mixed sound including the music signal and the sound signal includes the step of extracting the chroma spectrum, which is the output energy of each band window calculated based on the Fourier spectrum of the mixed sound, as the feature amount. a step of binarizing the extracted feature quantity, the feature quantity of the mixed sound, equal to or greater than the predetermined reference value based on the chroma spectrum of feature quantity at 1 per analysis frame or multiple analyzes each frame A feature value is 1 where, comprising the step of zero characteristic quantity if less than a predetermined reference value.

  In the sound source search method according to the second aspect of the present invention, as described above, by including the step of binarizing the extracted feature quantity, it has an element of scalar quantity (for example, single precision floating point number, 32 bits). Compared to the case where a sound source is searched from the sound source database using the feature amount as a search key, the dimension is reduced by 1 bit because the feature amount is binarized, and the binarized feature amount is an envelope of the music signal. (Shape) while being robust against mixed audio signals and invariant to changes in the sound pressure of music signals, it is possible to increase the search speed and improve the search accuracy. A sound source search method can be provided.

  According to the present invention, as described above, in the sound source search apparatus and sound source search method using the mixed sound as a search key, the search speed can be increased and the search accuracy can be improved.

It is a block diagram of a sound source search device according to an embodiment of the present invention. It is a schematic diagram of the waveform of a mixed sound. It is a schematic diagram which shows the Fourier spectrum of the mixed sound of FIG. It is a schematic diagram which shows the chroma spectrum calculated | required from the Fourier spectrum of the mixed sound of FIG. It is a block diagram of a sound source database according to an embodiment of the present invention. It is a flowchart of the construction method of the sound source database by one Embodiment of this invention. It is a flowchart of the sound source search method by one Embodiment of this invention. It is a figure which shows the frequency of the distance between the feature-value of mixed sound and the feature-value of the music memorize | stored in the sound source database. It is a figure which shows the chroma spectrum which is not binarized for 1 music. FIG. 10 is a diagram illustrating an unbinarized chroma spectrum for one piece of music that is 10 dB larger than FIG. 9. It is a figure which shows the chroma spectrum binarized for 1 music. It is a figure which shows the binarized chroma spectrum for 1 music larger 10 dB than FIG. It is a figure which shows the search result (F value) of the sound source search apparatus by a comparative example. It is a figure which shows the search result (F value) of the sound source search device by one Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Configuration of sound source search device]
With reference to FIGS. 1-4, the structure of the sound source search apparatus 100 by this embodiment is demonstrated. The sound source search device 100 is configured to search for a sound source constituting the mixed sound from a sound source database 50 described later.

  As shown in FIG. 1, the sound source search device 100 according to the present embodiment includes a feature amount extraction unit 10, a binarization unit 20, and a search unit 30. In the present embodiment, the mixed sound is generated by the mixing means 40 in order to evaluate the search performance of the sound source search device 100. Note that the feature amount extraction unit 10, the binarization unit 20, the search unit 30, and the mixing unit 40 are configured by a control unit such as a CPU (Central Processing Unit), for example. The feature quantity extraction means 10 and the binarization means 20 are examples of the “search apparatus side feature quantity extraction means” and the “search apparatus side binarization means” of the present invention. This will be specifically described below.

(Mixing means)
The mixing means 40 is configured to generate a mixed sound by mixing (editing) the music signal and the audio signal. Note that the music means a case where only a performance by an instrument is performed and a case where a performance by a musical instrument and a singing voice are included. The voice means a voice that is not noise. For example, the mixed sound is a sound composed of voice of narration in a TV program and BGM played in the background.

The mixed sound is a sound in which a plurality of sound sources are weighted and added at an arbitrary ratio. Assuming that the time waveform of the mixed sound is k (t) and the J sound sources s _j (t) are weighted by w _j , the mixed sound is expressed by the following equation (1).

Here, j = 1,. . . , J, t represents time. The general form of sound source search is that J sound sources s ₁ (t),. . . , S _J (t) is retrieved from the sound source database 50. In the present embodiment, if J = 2 and the weight (w) is arbitrary, the above equation (1) becomes the following equation (2).

Two sound sources s ₁ (t) and s ₂ (t) are a music signal s ₁ (t) and an audio signal s ₂ (t). As described above, the sound source search device 100 according to the present embodiment is configured to search for the feature value of s ₁ (t) from the sound source database 50 using the feature value of k (t) that is a mixed sound as a search key. Yes.

(Feature amount extraction means)
As shown in FIG. 1, the feature amount extraction unit 10 is configured to receive a mixed sound including a music signal and an audio signal generated by the mixing unit 40. The feature amount extraction unit 10 is configured to extract the feature amount from the mixed sound including the music signal and the audio signal. Specifically, the feature amount of the mixed sound is a chroma spectrum that is output energy of each band window calculated based on the Fourier spectrum of the mixed sound. Hereinafter, the extraction of the feature amount of the mixed sound will be specifically described.

It is assumed that the feature amount vector of the mixed sound is k (n), the feature amount vector of the music is s ₁ (n), and the feature amount vector of the sound is s ₂ (n). Here, n is an analysis frame number. The analysis frame will be described later. Each vector is D-dimensional, and the feature quantity vector of the mixed sound is expressed as k (n) = [k (n, 1), k (n, 2),. . . , K (n, D)] ^T. Here, ^T represents transposition of the vector. s ₁ (n) and s ₂ (n) are similarly represented.

(Analysis frame)
Next, the analysis frame will be described with reference to FIG. In FIG. 2, the horizontal axis represents time (t), and the vertical axis represents the amplitude of the mixed sound. Then, the waveform of the mixed sound in FIG. 2 is extracted every predetermined time length T (for example, 1 s). Specifically, the analysis frame having a predetermined time length T is extracted by applying a window (for example, a Hamming window) starting from a certain time t1. In addition, the analysis frame having a predetermined time length T is extracted by applying a Hamming window with time t2 after a certain time (frame shift length) has elapsed from time t2. Hereinafter, similarly, in all regions of the mixed sound, the waveform of the mixed sound is extracted for each analysis frame.

(Chroma spectrum)
Next, a chroma spectrum is demonstrated with reference to FIG. 3 and FIG. In FIG. 3, the horizontal axis represents the frequency, and the vertical axis represents the amplitude of the Fourier spectrum. A Fourier spectrum (solid line in FIG. 3) is calculated for the mixed sound (see FIG. 2) extracted for each analysis frame having a predetermined time length T. Then, the output energy (chroma spectrum) of each band window is calculated from the Fourier spectrum. Specifically, each band window (area surrounded by a dotted line in FIG. 3, filter bank) corresponding to the piano keyboard is set. The band window becomes a triangle having a wider width as the frequency becomes higher. Then, by integrating the Fourier spectrum included in each band window, the output energy (chroma spectrum) is calculated as shown in FIG. In FIG. 4, the horizontal axis represents the frequency, and the vertical axis represents the size of the chroma spectrum.

  Here, there are twelve band windows (A, A #, B, C, C #, D, D #, E, F, F #, G, so as to correspond to a piano keyboard for a pitch of one octave. , G #). In this embodiment, a chroma spectrum (feature value) is calculated for a band window (72 = 12 × 6) for 6 octaves. Thus, the mixed sound feature vector k (n) has a 72-dimensional dimension D.

(Binarization means)
Here, in this embodiment, the binarizing means 20 is configured to binarize the extracted feature quantity. Specifically, as shown in FIG. 4, the binarizing unit 20 determines that the feature amount (chroma spectrum) of the mixed sound is a predetermined reference value (specifically, an average value) of the feature amount for each analysis frame. (See the dotted line in FIG. 4) The feature amount is set to 1 in the above case, and the feature amount is set to 0 when the feature amount is less than a predetermined reference value for each analysis frame. That is, the binarizing means 20 is configured to binarize a chroma spectrum having a scalar quantity element.

Specifically, the chroma spectrum at time t is expressed as c (t) = [c ₁ (t), c ₂ (t),. . . , C _D (t)] ^T. Here, D represents the number of dimensions of the vector, and ^T represents the transposition of the vector. The binarized chroma spectrum b (t) = [b ₁ (t), b ₂ (t),. . . , B _D (t)] ^T is represented by the following equation (3).

(Search means)
The search means 30 is configured to search for a sound source from the sound source database 50 using the feature value of the mixed sound binarized by the binarization means as a search key. Specifically, the search means 30 searches for the sound source from the sound source database 50 using the binarized mixed sound feature values corresponding to a plurality (P) of analysis frames (cumulative analysis frames) as search keys. It is configured. That is, the sound source search results in a similar pattern search problem using a sequence of P feature quantity vectors as a search key. Specifically, the feature quantity vectors of the nth to (n + P-1) th analysis frames are expressed by the following equations (4) to (6).

Here, assuming that there are V music signals and W audio signals, let S _{1, v} (n) and S _{2, w} (m) be the v th and w th feature quantities. If the characteristic amount of the mixed sound obtained by mixing the v-th music signal and the w-th audio signal is K _{v, w} (n), the search is performed under conditions where v, w, n, and m are unknown. , K _{v, w} (n), the music number v ^* and the analysis frame number n ^* are estimated. When the search process is expressed as “search”, the search is expressed by the following formula (7).

  That is, the search process search determines one set of items corresponding to the search result (items with the smallest distance between feature quantity vectors). That is, the search result is a case where the distance between the feature quantity vector of the search key and the feature quantity vector to be searched becomes the minimum.

(Average false rejection rate and average false detection rate)
The performance of the search is evaluated by two indexes of false rejection (Miss) and false detection (False Alarm). The false rejection corresponds to a case where [v, n] corresponding to the music signal constituting the mixed sound is not included in the search result. Misdetection corresponds to a case where the search result includes [v, n] (music not related to the search key) other than the music signal constituting the mixed sound.

In the example of searching Q times, the average error rejection rate and the average error detection rate will be described. The search key for the q-th time is K _{v (q), w (q)} (n ^(q) ), the obtained set is φ ^(q), and K _{v (q), w (q)} (n ^(q) ) If S _{1, v ′ (q)} (n ′ ^(q) ) is a constituent sound source, the average error rejection rate is expressed by the following equation (8).

ここで、Ｉは、音源データベース５０中の［ｖ，ｎ］が取り得る組の総数を表す。また、φ^（ｑ）＼［ｖ´^（ｑ），ｎ´^（ｑ）］は、集合φ^（ｑ）から、要素［ｖ´^（ｑ），ｎ´^（ｑ）］を取り除く処理を意味する。また、｜｜は、集合の要素数を求める処理を意味する。そして、平均誤棄却率および平均誤検出率は、共に、値が小さいほど、検索性能が高いことを表す。 The average false detection rate is expressed by the following formula (9).

Here, I represents the total number of pairs that [v, n] in the sound source database 50 can take. Also, φ ^(q) \ [v ′ ^(q) , n ′ ^(q) ] means a process of removing elements [v ′ ^(q) , n ′ ^(q) ] from the set φ ^(q) . Also, || means a process for obtaining the number of elements in the set. The average false rejection rate and the average false detection rate both indicate that the smaller the value, the higher the search performance.

(Sound source database)
The sound source database 50 stores a plurality of music pieces. Specifically, similar to the above mixed sound, the feature amounts of a plurality of music pieces are stored in the sound source database 50 in a binarized state.

  Specifically, as shown in FIG. 5, in the present embodiment, the sound source database 50 includes a feature amount extraction unit 51 that extracts a feature amount from a music signal for database, and binarization that binarizes the extracted feature amount. Means 52 and construction means 53 for constructing the sound source database 50 from the feature quantity of the database music signal binarized by the binarization means 52. The feature quantity extraction unit 51, the binarization unit 52, and the construction unit 53 are configured by a control unit such as a CPU (Central Processing Unit), for example. The feature quantity extraction means 51 and the binarization means 52 are examples of the “database side feature quantity extraction means” and the “database side binarization means” of the present invention, respectively.

  Here, the feature amount of the database music signal is a chroma spectrum that is output energy of each band window calculated based on the Fourier spectrum of the database music signal. Further, the feature amount of the music signal for database is extracted for each analysis frame having a predetermined time length. Then, the binarizing means 52 sets the feature amount to 1 when the feature amount of the music signal for database is equal to or greater than a predetermined reference value (specifically, an average value) of the feature amount for each analysis frame. The feature amount is set to 0 when the value is less than the reference value. Note that the details of extraction of feature values from the music signal for database and binarization of the extracted feature values are the same as those of the sound source search device 100.

[How to build a sound source database]
Next, the construction method of the sound source database 50 according to the present embodiment will be described with reference to FIG.

  First, in step S <b> 11, a feature amount is extracted from the database music signal input to the feature amount extraction means 51. Specifically, after the Fourier spectrum of the mixed sound is calculated for each analysis frame, the chroma spectrum that is the output energy of each band window is calculated.

  Next, in step S12, the binarizing means 52 binarizes the extracted feature amount based on the above equation (3). In step S13, the construction unit 53 constructs the sound source database 50 from the binarized database music signal feature amount.

[Sound source search method]
Next, the sound source search method according to the present embodiment will be described with reference to FIG.

  First, in step S1, a feature value is extracted from a mixed sound including a music signal and a sound signal input to the feature value extraction unit 10. Specifically, after the Fourier spectrum of the mixed sound is calculated for each analysis frame, the chroma spectrum that is the output energy of each band window is calculated.

  Next, in step S2, the extracted feature quantity is binarized based on the above equation (3).

  Next, in step S3, a sound source is searched from the sound source database 50 using the binarized mixed sound feature amount as a search key. Specifically, when the distance between the binarized feature vector of the search key and the binarized feature vector of the music stored in the sound source database 50 is the minimum, the mixed sound is included. It is determined that the matching music has been searched (detected).

  Next, in step S4, the search result is evaluated. Specifically, the average error rejection rate and the average error detection rate are calculated by the above equations (8) and (9). Further, an F value (harmonic average) is calculated from the calculated average error rejection rate and average error detection rate.

(Experiment to confirm whether chroma spectrum is suitable for expressing music)
With reference to FIG. 8, an experiment conducted to confirm whether or not the chroma spectrum is suitable for expressing music will be described.

  First, the chroma spectra of 71 songs were compiled into a database. Then, among 71 music pieces, 500 chroma spectra of music pieces having a length of 10 seconds were randomly selected and used as search keys. And the distance distribution between the feature-value vector of 500 search keys and the feature-value vector of 71 music pieces was created. In FIG. 8, the horizontal axis represents distance, and the vertical axis represents frequency. In this experiment, it was confirmed that there are 500 places where the distance becomes zero. That is, it was confirmed that there is one place where the distance becomes 0 for one search key. Thus, it was confirmed that there was no numerical repetition (repetition of the same feature amount) in the structure of the music (music having a length of 10 seconds). That is, it was confirmed that the chroma spectrum is suitable for expressing music (suitable as a feature amount of music).

  Further, as shown in FIG. 8, it has been found that the frequency gradually increases from the distance 0, and then the frequency rapidly increases and then decreases rapidly. That is, it has been found that the frequency is generally formed in one convex shape.

(Experiment on binarization of chroma spectrum)
Next, with reference to FIG. 9 to FIG. 12, an experiment for binarization of the chroma spectrum will be described. In FIGS. 9 to 12, the larger the chroma spectrum value, the darker the color.

  FIG. 9 is a chroma spectrum for one piece of music that is not binarized. FIG. 10 is a chroma spectrum of one piece of music that is not binarized when the energy of the signal waveform of the music shown in FIG. 9 is relatively increased by 10 dB. As shown in FIG. 10, when the energy of the signal waveform of the music is relatively increased by 10 dB, it has been found that the chroma spectrum value increases as a whole (the color becomes darker). That is, it has been confirmed that the chroma spectrum (feature amount) that has not been binarized changes as the energy of the signal waveform of the music changes.

  FIG. 11 shows a binarized chroma spectrum for one piece of music. FIG. 12 is a binarized chroma spectrum for one music when the energy of the signal waveform of the music shown in FIG. 11 is relatively increased by 10 dB. As shown in FIG. 12, it was found that even when the energy of the signal waveform of the music was relatively increased by 10 dB, the binarized chroma spectra for one music completely matched the pattern. That is, it was confirmed that the binarized chroma spectrum (feature amount) is invariant to the change in energy of the signal waveform of the music.

(Sound source search experiment)
Next, referring to FIG. 13 and FIG. 14, a sound source search experiment by the sound source search device 100 according to the present embodiment will be described in comparison with a sound source search device according to a comparative example.

  The sound source search device according to the comparative example includes the feature amount extraction unit and the search unit, but does not include the binarization unit 20 unlike the sound source search device 100 according to the present embodiment. That is, in the sound source search device according to the comparative example, the feature amount is the chroma spectrum value itself (single precision floating point number, 32 bits). That is, the dimension of the feature quantity is 72 dimensions × 32 bits for 6 octaves. On the other hand, in the sound source search device 100 according to the present embodiment, since the feature amount (chroma spectrum) is binarized, the dimension of the feature amount is 72 dimensions × 6 bits for 6 octaves.

  As shown in FIG. 13 and FIG. 14, in the sound source search experiment, the mixed sound (mixing ratio−5 dB) in which the relative magnitude (sound pressure ratio) of the music signal relative to the audio signal is reduced by 5 dB is mutually compared. Equal mixed sound (mixing ratio 0 dB), 5 dB larger mixed sound (mixing ratio 5 dB), 10 dB larger mixed sound (mixing ratio 10 dB), 15 dB larger mixed sound (mixing ratio 15 dB), and 20 dB larger mixed sound Sounds (mixing ratio 20 dB) were prepared, sound source search was performed for each mixed sound, and F value of the search result was calculated. For example, the sound pressure ratio when BGM flows in the background of narration corresponds to −5 dB to 0 dB.

  In the sound source search experiment, the band window (filter bank) is composed of 6 octaves (72 banks) with 55 (Hz) to 3520 (Hz). The signal sampling rate was 16000 Hz. The analysis frame length was 16.384 (s) and the frame shift length was 1/16 (s). In the sound source database 50, music pieces for 72 music pieces (about 200,000 analysis frames) of a commercially available CD were stored.

  Then, as the cumulative number of frames, a 10 (s) section (16 × 10 analysis frames) and a 2 (s) section (16 × 2 analysis frames) were adopted. Then, using these as feature quantities (search keys), the time when successive 16 × 10 analysis frames (16 × 2 analysis frames) match was searched from about 200,000 candidates stored in the sound source database 50.

  The audio signal to be mixed with the music signal is JNAS (“JNAS: Japan special speech for large-scale continuous speech recognition research”, J. Acust. Soc. Therefore, the male and female utterances were connected and prepared without opening the pause between utterances. And it mixed so that utterance might become narration over one music.

  Furthermore, the sound source search was executed when the amplitude (relative value) of the signal was changed to 1 times, 0.5 times, 2 times, 0.1 times, and 10 times.

  The database size (32, see FIG. 13) of the sound source database according to the comparative example in which the feature quantity is not binarized is compared with the database size (1, see FIG. 14) of the sound source database 50 of this embodiment in which the feature quantity is binarized. It was confirmed that the size was 32 times.

  In the sound source search device according to the comparative example, the relative processing time (search time) is 250 or 1130, whereas in the sound source search device 100 according to the present embodiment, the relative processing time (search time) is 34. Or 170. Thus, it was confirmed that the search speed is increased by binarizing the feature amount.

  Further, in the sound source search device according to the comparative example, the value of the F value changes remarkably with respect to the change in the signal amplitude (1 ×, 0.5 ×, 2 ×, 0.1 ×, 10 ×). There was found. On the other hand, in the sound source search device 100 according to the present embodiment, it has been found that the F value is invariant with respect to a change in the amplitude of the signal. This is because, as shown in FIGS. 11 and 12, the binarized chroma spectrum (feature amount) is invariant to the change in energy of the signal waveform of the music. It is thought.

  When the amplitude of the signal is 1 time, the F value by the sound source search device according to the comparative example may be higher, while the amplitude of the signal is 0.5 times other than 1 time, 2 times, 0. At 1 × and 10 ×, in all cases, it was found that the F value by the sound source search device 100 according to the present embodiment was higher. As a result, it has been confirmed that the search accuracy is improved by binarizing the feature amount as a whole.

[Effect of this embodiment]
In the present embodiment, the following effects can be obtained.

  In the present embodiment, as described above, by providing the binarizing means 20 for binarizing the extracted feature quantity, a feature quantity having an element of a scalar quantity (for example, a single precision floating point number, 32 bits) is searched. Compared with the case where a sound source is searched from the sound source database 50 as a key, the dimension becomes smaller (1 bit) as the feature value is binarized, so that the search speed can be increased.

  In addition, when an audio signal is mixed with a music signal, the envelope (shape) of the music signal changes as much as the audio signal is mixed. Therefore, in the present embodiment, as described above, the binarizing unit 20 that binarizes the extracted feature quantity includes the binarized feature quantity so that the portion “1” of the binarized feature quantity is an audio signal. As long as is acting additively, it remains “1”. On the other hand, even if the voice signal acts additively on the “0” portion of the binarized feature value, it remains “0” unless the reference value for binarization is exceeded. In the vicinity of the threshold value for binarization, the audio signal is mixed, so that the binarized feature quantity “0” or “1” may be inverted, while the audio signal output Since the frequency with high energy (frequency that can be inverted) is a relatively small range only in the vicinity of a frequency that is an integral multiple of the fundamental frequency, the influence of inversion is considered to be small. As a result, the binarized feature value is a robust feature value with respect to the mixed audio signal while representing the envelope (shape) of the music signal. In addition, since the predetermined reference value can be changed in accordance with the change in the volume of the mixed sound even when the sound pressure of the music signal changes, the change in the characteristic amount of the mixed sound (the characteristic amount is “0”). "Or change in the determination of" 1 ") is prevented. As a result, the search speed can be increased and the search accuracy can be improved.

  Further, in the present embodiment, as described above, the feature amount of the mixed sound is extracted for each analysis frame having a predetermined time length T, and the binarizing unit 20 determines that the feature amount of the mixed sound is The feature amount is set to 1 when it is equal to or greater than the average value of the feature values in each analysis frame, and the feature value is set to 0 when it is less than the average value of the feature values in each analysis frame. Thereby, since the feature value is binarized based on the average value of the feature value for each analysis frame, the feature value can be binarized appropriately corresponding to the change in the volume of the mixed sound.

  In the present embodiment, as described above, the feature amount of the mixed sound is a chroma spectrum that is output energy of each band window calculated based on the Fourier spectrum of the mixed sound, and the binarizing means 20 The chroma spectrum is configured to be binarized. Thereby, a sound source can be appropriately searched using the chroma spectrum as a feature amount.

  In the present embodiment, as described above, the feature amount of the mixed sound is extracted for each analysis frame having a predetermined time length T, and the search means 30 is set to 2 corresponding to a plurality of analysis frames. The sound source is searched from the sound source database 50 using the characteristic value of the mixed sound that has been digitized as a search key. As a result, the feature amount (information amount) of the search key is increased compared to the case where the search is performed using the binarized mixed sound feature amount corresponding to one analysis frame as a search key, so that the search accuracy is improved. be able to.

  Further, in the present embodiment, as described above, the search unit 30 uses the feature value of the binarized mixed sound corresponding to the 10 × 16 analysis frame or the 2 × 16 analysis frame as a search key from the sound source database 50. Configure to search for sound sources. As a result, even when a search is performed using a binarized mixed sound feature amount corresponding to a plurality of analysis frames that are relatively short of a 10 × 16 analysis frame or a 2 × 16 analysis frame as a search key, By binarization, high-speed and high-precision search can be performed.

  In the present embodiment, as described above, the sound source database 50 includes the feature amount extraction unit 51 that extracts the feature amount from the database music signal, the binarization unit 52 that binarizes the extracted feature amount, And a construction means 53 for constructing the sound source database 50 from the feature quantity of the database music signal binarized by the binarization means 52. As a result, the feature amount of the sound source database 50 is binarized, so that the feature amount is compared with the case where the sound source database 50 is constructed from feature amounts having elements of scalar amounts (for example, single precision floating point numbers, 32 bits). As the quantity is binarized, the dimension is reduced (1 bit), so the database size of the sound source database 50 can be reduced. As a result, the search speed can be increased.

[Modification]
The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above description of the embodiment but by the scope of claims for patent, and further includes all modifications (modifications) within the meaning and scope equivalent to the scope of claims for patent.

  For example, in the above embodiment, the feature amount is set to 1 when the feature amount of the mixed sound and the music signal for the database is equal to or larger than the average value of the feature amount, and is set to 0 when the feature amount is less than the average value of the feature amount. However, the present invention is not limited to this. In the present invention, the feature value may be set to 1 or 0 with reference to a value other than the average value of the feature values. For example, the feature value may be set to 1 or 0 with reference to the median value of the feature values.

  In the above embodiment, the feature amounts of the mixed sound and the database music signal are extracted for each analysis frame having a predetermined time length. However, the present invention is not limited to this. For example, the characteristics of the mixed sound and database music signal may be extracted for each of a plurality of analysis frames.

  In the above embodiment (experiment), the sound source is searched from the sound source database using the binarized mixed sound feature amount corresponding to the 10 × 16 analysis frame or the 2 × 16 analysis frame as a search key. However, the present invention is not limited to this. For example, a binarized mixed sound feature amount corresponding to a number of analysis frames other than a 10 × 16 analysis frame or a 2 × 16 analysis frame may be used as a search key.

  In the above-described embodiment, an example in which the feature vector has a dimension corresponding to 6 octaves (72 dimensions) is shown, but the present invention is not limited to this. For example, the feature vector may be configured to have a number of octave dimensions other than six octaves.

  In the above-described embodiment, an example in which the search result is a case where the distance between the feature quantity vector of the search key and the feature quantity vector to be searched becomes the minimum is shown, but the present invention is not limited to this. For example, when the difference between the feature value of the mixed sound binarized by the binarization means and the feature value of the sound source in the sound source database is less than a predetermined threshold, the search result is configured. Also good. As a result, when a mixed sound is used as a search key, the sound source to be searched can be searched even when the distance between the feature vector of the search key and the feature vector of the search target (correct sound source to be searched) is not minimized. It can be prevented that it becomes impossible.

10. Feature amount extraction means (retrieval device side feature amount extraction means)
20 Binarization means (search device side feature quantity extraction means)
30 Search means 50 Sound source database 51 Feature quantity extraction means (database side feature quantity extraction means)
52 Binarization means (database side binarization means)
100 sound source search device T (predetermined) time length

Claims (3)

A search device side feature quantity extraction means for extracting a feature quantity from a mixed sound including a music signal and a voice signal;
A search device-side binarization means for binarizing the extracted feature quantity;
Search means for searching for a sound source from a sound source database, using the feature value of the mixed sound binarized by the search device side binarization means as a search key;
The feature amount of the mixed sound is a chroma spectrum that is output energy of each band window calculated based on a Fourier spectrum of the mixed sound,
The feature amount of the mixed sound is extracted for each analysis frame or a plurality of analysis frames having a predetermined time length,
The search device-side binarization means, when the feature amount of the mixed sound is equal to or greater than a predetermined reference value based on a chroma spectrum of the feature amount in each analysis frame or each of the plurality of analysis frames. Is a sound source search device configured to set the feature amount to 0 when the value is less than the predetermined reference value. The sound source database is
Database-side feature quantity extraction means for extracting feature quantities from the music signal for the database;
Database-side binarization means for binarizing the extracted feature quantity;
Construction means for constructing the sound source database from the feature quantities of the database music signal binarized by the database-side binarization means,
The feature amount of the database music signal is a chroma spectrum that is output energy of each band window calculated based on a Fourier spectrum of the database music signal,
The feature amount of the music signal for the database is extracted for each analysis frame or each of the plurality of analysis frames having the predetermined time length,
The database-side binarization means sets the feature amount to 1 when the feature amount of the database music signal is equal to or greater than the predetermined reference value of the feature amount for each analysis frame or for each of the plurality of analysis frames. The sound source search device according to claim 1, wherein the feature amount is set to 0 when the value is less than the predetermined reference value. Extracting a feature value from the mixed sound including the music signal and the audio signal;
Binarizing the extracted feature value;
Using the feature value of the binarized mixed sound as a search key, and searching for a sound source from a sound source database,
The feature amount of the mixed sound is extracted for each analysis frame or a plurality of analysis frames having a predetermined time length,
The step of extracting the feature amount from the mixed sound including the music signal and the sound signal includes a step of extracting a chroma spectrum that is output energy of each band window calculated based on the Fourier spectrum of the mixed sound as the feature amount. Including
The step of binarizing the extracted feature value is performed when the feature value of the mixed sound is equal to or greater than a predetermined reference value based on a chroma spectrum of the feature value for each analysis frame or for each of the plurality of analysis frames. The sound source search method includes a step of setting the feature amount to 1 and setting the feature amount to 0 when the feature amount is less than the predetermined reference value.

Images