JP2007193222A - Melody input device and musical piece retrieval device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a melody input device with which an improvement in retrieval accuracy is achieved by performing musical piece retrieval in consideration of chain probability of notes or pitches. <P>SOLUTION: The melody input device (1) equipped with an input means (4) inputting a melody, and an extraction means (6) extracting pitch information from the input melody is equipped with a database (10) including note chain information relating to the chains of the notes and a correction means (11) correcting the note information extracted by the extraction means based on the note chain information in the database. Even when the chains of the notes of the low probability are observed in the input melody, the chains are corrected according to the chain probability of the correct notes and therefore, when the device is applied to, for example, the musical piece retrieval device, wrong musical piece retrieval does not occur. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a melody input device and a music search device, for example, a melody input device that inputs a hummed melody and a music search device that searches a database for music corresponding to the hummed melody.

  For example, there is known a conventional technique (hereinafter referred to as Conventional Technique 1) in which information such as the name of the music can be checked by listening to music that flows on a television or in a city using a mobile phone (see Non-Patent Document 1). . However, since this prior art 1 needs to listen to the music itself, the sound source is indispensable, and there is a drawback that it cannot be used for searching for music that only remembers part of the melody.

As a prior art overcoming this drawback (hereinafter referred to as Prior Art 2), a humming input device is known (see Non-Patent Document 2).
In this conventional technique 2, when a music piece is registered in the database, one music piece is divided into a number of “music pieces”, and the user's Hamming input and the music pieces in the database are matched in units of music pieces. It is. You can search for any part of the song you're humming.

As a method of generating the above “music piece”, for example, a technique (hereinafter, “Prior Art 3”) in which a predetermined number n of note groups are cut out starting from every note in a song is known (patent 3). Reference 1).
In this prior art 3, one musical piece is a note a ₁ , a _2.
, A ₃ ,..., A _p and n = 4, a ₁
First musical piece ~a _4, the second musical piece _{a 2 ~a 5, a 3 ~a} 6
3rd music piece of ... _{ap-3 to ap}
Into the p th music pieces. That is, a group of n notes is cut out from the beginning of the notes constituting the music piece, and is used as the first piece of music. Thereafter, the second piece of music, the third piece of music, and the third piece of music are moved while shifting the cutout position of the notes one by one. ,..., To generate a p-th music piece.

Further, as an application example of the Hamming input device other than the score database, for example, a technique related to an automatic musical score device (hereinafter referred to as Conventional Technology 4) is also known (see Patent Document 2).
This prior art 4 extracts pitch information and power information of an acoustic signal input by humming or the like for each analysis period, and thereafter, a section (also referred to as a segment) in which the acoustic signal can be regarded as one sound from the extracted pitch information and power information. It is an automatic music transcription device that performs automatic music transcription by identifying the pitch of each section (denoted as the pitch in the literature), and its characteristic point is that when identifying the pitch of each section, First, for each pitch information of the section, a distance to the pitch candidate to be identified and a weighting coefficient determined by a position in the section are obtained, and the pitch with the smallest product sum value is identified. .
If the weighting factor is set small near the beginning and end of the section, the influence of the pitch at the beginning and end of the section on the product sum value will be small. It is possible to reduce the identification of the pitch to be high, and it is possible to identify the pitch more accurately.

“Au's“ Let me listen ””, [online], [Search January 12, 2006], Internet <URL: http://plusd.itmedia.co.jp/mobile/articles/0506/22/ news009.html> "Humming music search system", [online], Nippon Telegraph and Telephone Corporation, [Search January 12, 2006], Internet <URL: http://www.ntt.co.jp/saiyo/rd/review /2001/pf/10.html> JP 2000-172893 A JP 7-44163 A

  As described above, the prior art 1 needs to be able to listen to the music itself, so that the sound source is indispensable and cannot be used for searching for music that only remembers a part of the melody. Since hamming input is possible for -4, there is no such a fault.

  However, these prior arts 2 to 4 are only for performing a search in a partial unit such as a musical piece of music or a segment (section), and there is a problem that sufficient search accuracy cannot be obtained.

  That is, when one piece of music is composed of parts A, B, C, and D (music pieces or segments) and two music pieces are composed of parts a, B, C, and d (the same), the Hamming input is temporarily If A and B are used, for example, if A is misidentified as a due to pitch deviation or noise, one song should be hit, but the second song will be hit by mistake. There is an inconvenience. The cause of such inconvenience is that only collation is performed in units of parts such as music pieces and segments.

As described above, since the hamming input devices of the conventional techniques 2 to 4 perform the search using only the acoustic characteristics of the input partial units, a subtle shift in pitch at the time of input, ambient noise, or However, there is a problem that sufficient search accuracy cannot be obtained because it is easily affected by the low frequency resolution of the fast part of the song.
Further, although the prior art 4 eliminates the instability at both ends of the segment by weighting by the segment position, that alone can cause the pitch to become unstable in the middle of a long note (for example, after an extended sound). A sufficient effect cannot be expected when there is a decorative sound.

  Therefore, the present invention provides a melody input device that improves recognition accuracy and a music search device that improves music search accuracy by performing music search in consideration of the chain probability of notes or pitches. It is in.

The invention described in claim 1 is a melody input device comprising an input means for inputting a melody and an extracting means for extracting pitch information from the input melody, a database including note chain information relating to a chain of notes; A melody input device comprising correction means for correcting the pitch information extracted by the extraction means based on the note chain information in the database.
The invention according to claim 2 is the melody input device according to claim 1, wherein the note chain information is created for each musical instrument.
The invention according to claim 3 is the melody input device according to claim 1, wherein the note chain information is created for each genre of music.
The invention according to claim 4 is the melody input device according to claim 1, wherein the note chain information is created for each composer.
The invention according to claim 5 is the melody input device according to claim 1, wherein the note chain information does not take into account information relating to the length of a sound.
The invention described in claim 6 is extracted by the input means for inputting a melody, the extraction means for extracting pitch information from the input melody, the database including the note chain information relating to the chain of notes, and the extraction means. Correction means for correcting the pitch information based on the note chain information in the database, and music having similar pitch information from a plurality of music pieces using the pitch information corrected by the correction means. A music search apparatus comprising a search means for searching.
The invention described in claim 7 is the music search apparatus according to claim 6, wherein the note chain information is created for each musical instrument.
The invention described in claim 8 is the music search apparatus according to claim 6, wherein the note chain information is created for each genre of music.
The invention according to claim 9 is the music search apparatus according to claim 6, wherein the note chain information is created for each composer.
The invention according to claim 10 is the music search apparatus according to claim 6, wherein the note chain information does not take into account information relating to the length of a sound.

In the present invention, pitch information is extracted from the input melody, and the extracted pitch information is corrected based on the note chain information in the database.
Here, note chain information refers to data obtained by statistically modeling the probability that notes are chained in many music pieces. Such chain probabilities tend to be tentative depending on the genre of music, the type of musical instrument, and the composer, but at least there may be a chain of notes that are not linked at all empirically or very rarely linked. If such a chain of notes, that is, a chain of notes with low probability is found in the input melody, for example, if it is humming, it is caused by the influence of fluctuations in pronunciation, pitch deviation, noise, and the like.
Therefore, for example, when a music search is performed based on a melody composed of a chain of notes with low probability, an erroneous search result may be caused.
On the other hand, if the pitch information is extracted from the input melody and the extracted pitch information is corrected based on the note chain information in the database as in the present invention, the input melody is assumed to be probable. Even when a low note sequence is found, it is corrected according to the correct note sequence probability, so that the above-mentioned erroneous search result is not caused.

  Embodiments of the present invention will be described below with reference to the drawings. It should be noted that the specific details or examples in the following description and the illustrations of numerical values, character strings, and other symbols are only for reference in order to clarify the idea of the present invention, and the present invention may be used in whole or in part. Obviously, the idea of the invention is not limited. In addition, a well-known technique, a well-known procedure, a well-known architecture, a well-known circuit configuration, and the like (hereinafter, “well-known matter”) are not described in detail, but this is also to simplify the description. Not all or part of the matter is intentionally excluded. Such well-known matters are known to those skilled in the art at the time of filing of the present invention, and are naturally included in the following description.

[First Embodiment]
FIG. 1 is a functional block diagram of a hamming input device according to the first embodiment. In this figure, a hamming input device 1 includes an interface unit 2, a control unit 3, an acoustic signal input unit 4, an acoustic signal storage unit 5, a feature extraction unit 6, a feature storage unit 7, a hypothetical note string output unit 8, a note chain model. A generation unit 9, a note chain model storage unit 10, an input correction unit 11, and a note string output unit 12 are provided.

  Before describing each part, the note chain model will be outlined. The note chain model has the same meaning as the acoustic model referred to in the speech recognition technology, and is a statistical database that expresses how notes are chained in Ngram.

  FIG. 2 is a conceptual diagram showing a relationship between an acoustic model for speech recognition and a language model. In this figure, it is assumed that there is an utterance "Nihon Keizai" (Japanese economy). And if the pronunciation of “za” is somewhat ambiguous, for example, it may be heard as “ta”, the acoustic probability value of Nihon Keizai (Japanese economy) is 0.158, Nihon Keitai (Japanese mobile) If the same probability value is 0.165, “Nippon Mobile”, which has a higher acoustic probability value, is output, resulting in an incorrect result.

  On the other hand, the language model also stores the probability that all words are linked. For example, 0.065 is stored as the probability of chaining from “Japan” to “economy”, and 0.005 is stored as the probability of chaining from “Japan” to “mobile”. The linkage probability 0.005 from “Japan” to “mobile” is considerably lower (smaller) than the linkage probability 0.065 from “Japan” to “economy”. This means that the probability of chaining to “mobile” is extremely rare.

  Even in the hamming input, a correct recognition result can be obtained by considering the chain probability. That is, the correct recognition result can be obtained even in Hamming input by calculating the following equation (1) using the acoustic probability as the acoustic likelihood and the language probability (linkage probability) as the language likelihood. .

    Corrected likelihood = acoustic likelihood × language likelihood (1)

  The correction likelihood for the Japanese economy is 0.01027, and the correction likelihood for the Japanese mobile phone is 0.000825. In this case, since 0.01027> 0.000825, a correct result (Japanese economy) is obtained. In other words, “Japanese economy”, which was inferior in acoustic likelihood, is corrected by the probability that Japanese is chained, and is output as a correct answer.

  Such a language model is a morphological analysis of a huge amount of Japanese sentences such as newspapers (and its electronic data), and the frequency with which each word is linked is divided by the parameter. On the other hand, it can be said that the note chain model used in this embodiment is obtained by replacing “words” in the above language model with “notes”.

  FIG. 3 is a diagram showing a note chain model generation (construction) algorithm. In order to generate a note chain model, a large amount of musical score data (musical score data such as MIDI) is required in the same way as a language model. For example, existing musical score data such as a J-POP music book is used. Can be used.

  If there are 100,000 pieces of musical score data at hand, first, the musical score data is read (step S1), and the melody portion of the musical score data is cut out for each unit (for example, one bar). Step S3). Next, while looping by the number of units (the number of bars constituting the music according to the above example) (step S4), the first sound of each unit cut out is a reference sound (for example, C4). It normalizes so that it may become (step S5). Hereinafter, the cutout unit after normalization is referred to as a pattern.

Next, it is searched whether the pattern has appeared in the past (step S6). If it has not appeared in the past (it does not always appear first), it is registered as a new pattern, and the number of appearances is set to 1. (Step S7). On the other hand, when it appears, the number of appearances is increased by 1 (step S8). This process is repeated until there is no musical score data (100,000 musical score data according to the above example) (step S2).
When there is no musical score data, the number of pattern appearances is divided by the parameter (step S9), and the result is output (step S10). Note that a pattern whose appearance count is less than or equal to the frequency N (for example, N = 1) is regarded as a rare pattern and the result is not output.

  In this way, a note chain model is generated in advance.

  The note chain model generation unit 9 is a part that executes the algorithm of FIG. 3 to generate a note chain model in advance, and the note chain model storage unit 10 is a part that stores and holds the note chain model.

The interface unit 2 is an operation input unit for a user to instruct the start and end of a hamming input, and the control unit 3 is a unit that performs overall control of the overall operation of the hamming input device 1.
The acoustic signal input unit 4 includes a humming input acoustic microphone, an amplifier, an A / D converter, and the like, and the acoustic signal storage unit 5 stores and holds the acoustic signal captured from the acoustic signal input unit 4. Part.

The feature extraction unit 6 is a part that extracts the feature of the acoustic signal stored and held in the acoustic signal storage unit 5, and the feature storage unit 7 is a part that stores and holds the extraction result. The characteristics of the acoustic signal will be described later.
The hypothetical note string output unit 8 is a part that estimates the pitch based on the feature value stored and held in the feature storage unit 7 and outputs a hypothetical note string including the estimated pitch.
The input correction unit 11 corrects the hypothetical note string using the note chain model, and the note string output unit 12 outputs the note string having the largest corrected output value as a correct answer.

  FIG. 4 is a diagram showing an operation flowchart of the first embodiment. In this flowchart, first, a note chain model is generated by the note chain model generation unit 9, and the note chain model is stored in the note chain model storage unit 10 (step S21). As described above, the note chain model generation unit 9 may generate a note chain model in advance, and the note chain model may be stored in the note chain model storage unit 10 in advance.

  Next, humming is input (step S22), converted into an acoustic signal, and stored in the acoustic signal storage unit 5. This hamming input is performed by the user humming a part of a melody of a music piece desired to be searched toward the acoustic microphone of the acoustic signal input unit 4. For example, it may be sung like “LaLa La”.

  If humming is input, next, a feature is extracted from the acoustic signal (step S23). There are various methods for extracting features. Although not described in detail here, for example, a chroma vector is extracted as a feature.

  FIG. 5 is a schematic diagram showing chroma vectors. In this figure, the chroma vector is a feature indicating how much sound is present at the frequency corresponding to each pitch (G3 to B8 in this figure). Specifically, a short-time window is set for the input sound, a spectrogram is obtained by FFT (Fast Fourier Transform), and a filter in which the center of each band (for example, 262 Hz in the case of the center (C4)) has a peak is obtained. Extract features by multiplying.

  FIG. 6 is a schematic diagram of a filter. The vertical axis is the signal level, and the horizontal axis is the frequency. In the illustrated example, a first filter indicated by a square symbol, a second filter indicated by a circle symbol, a third filter indicated by a triangle symbol, and a fourth filter indicated by a hatched symbol are shown. By using a filter having an appropriate characteristic, a target chroma vector (feature) can be extracted. For example, if the chroma vector of the center do (C4) is extracted, a third filter indicated by a triangle symbol corresponding to the frequency of the sound of do (C4) (262 Hz) may be used.

  FIG. 7 is a diagram showing an example of a Hamming input, and FIG. 8 is a frequency distribution diagram of the Hamming input. However, although the hamming input in FIG. 7 is C4D4E4C4D4, it is assumed that the pitch is slightly removed.

  In FIG. 8, the vertical axis represents the pitch, and this pitch is lowered as it goes up and becomes higher as it goes down. The numerical value written to the right of the pitch indicates the peak value of the frequency for each pitch. Specifically, in order from the top, frequency peak of pitch G3 = 196 Hz, frequency peak of pitch Ab3 = 208 Hz, frequency peak of pitch A3 = 220 Hz,..., Frequency peak of pitch C # 5 = It indicates that the frequency is 554 Hz.

  In addition, the horizontal axis represents time, and the time scale values (0, 1, 2, 3,..., 53) indicating the unit of the time axis are shown at the top in the drawing. The maximum value of the time scale value is 53, but this is a convenient value. The maximum time scale value corresponds to the actual Hamming input length.

  Now, many numerical values are described within the range of the broken line 13 in the figure. These numerical values represent the level (spectrogram) for each frequency component of the humming input acoustic signal. For example, when paying attention to the time scale value 0, values “0000001000000000000000” are arranged from the top to the bottom, and this is the frequency of the first sound (the sound when the time scale value is 0) input from Hamming to 196 Hz. It shows that the level of 262 Hz is 0, the level of frequency 277 Hz is 1, and the level of frequencies 294 Hz to 554 Hz is 0. In other words, it means that only the level of the frequency 277 Hz is 1 and the levels of the other frequencies are all 0. Since the pitch of the frequency 277 Hz is C # 4, the sound at the time scale value 0 is eventually obtained. Means that the pitch of C # is C # 4.

The feature extraction unit 6 extracts a feature having the maximum level for each time scale value from the Hamming input frequency distribution diagram of FIG. 8 as a feature of the acoustic signal. For example, level 1 with a frequency of 277 Hz (pitch C # 4) is extracted at time scale value 0, and level 1 with frequencies 262 Hz and 277 Hz (pitch C4 and C # 4) is extracted at time scale value 1. Level 2 with frequency 262 Hz (pitch C4) is extracted for value 2, level 5 with frequency 277 Hz (pitch C # 4) is extracted for time scale value 3, and frequency 262 Hz (pitch C4) for time scale value 4 Level 6 is extracted at time scale value 5, level 7 at frequency 262 Hz (pitch C4) is extracted, level 7 at frequency 262 Hz (pitch C4) is extracted at time scale value 6, and time scale value 7 is extracted. Level 6 at frequency 262 Hz (pitch C4) is extracted, level 7 at frequency 262 Hz (pitch C4) is extracted at time scale value 8, and time scale value 9 Extract the level 6 of the wave number 262Hz and 277Hz (pitch C4 and C # 4), ····, to extract the level 1 of the time scale value 53 Frequency 294Hz (pitch D4).
As a result, feature extraction (feature vector extraction) is performed for each time scale value, as indicated by the black background in the figure.

  Once the characteristics of the acoustic signal are extracted, a hypothetical note string is generated and output based on the characteristics (step S24). In the generation of a hypothetical note string, a point where the feature value changes greatly is extracted from the maximum value of the change amount of the feature vector with respect to the feature vector string obtained in FIG. 8, and that point is set as a note change candidate.

  FIG. 9 is a schematic diagram of searching for note change candidates. This figure is similar to FIG. 8 described above, but differs in that vertical dividing lines L1 to L6 are inserted in some places along the time axis. The positions of these dividing lines L1 to L6 are points where feature values change greatly (points that are candidates for note change).

  Next, the time interval (separation line interval; hereinafter referred to as frame length) is checked against the average frame length for each predefined note type (quarter note or octal note, etc.) Estimate the type of

  If the obtained interval is estimated as a quarter note or an eighth note, the pitch is estimated after narrowing down the candidate positions to the sixteenth note level (see dotted lines L7 to L13 in FIG. 9).

  FIG. 10 is a conceptual diagram of hypothetical note string output focusing on one frame. As shown in this figure, the value of the chroma vector is added within the range, and the one showing the largest value (accuracy by sound: in this case, the accuracy is 47 with the C4 eighth note), the pruning threshold ( For example, all candidates within 10) are output. At the right end of FIG. 10, several hypothetical note strings output in this way, for example, “C4”, “C # 4 and C4”, and “B3 and C4” are shown.

  More specifically, one frame shown in FIG. 10 (a) is divided into a front frame F and a rear frame B with a sixteenth note level dividing line L7 as a boundary, and as shown in FIG. 10 (b), For each of the front frame F and the rear frame B, the level for each pitch (frequency) is added. As a result, in the previous frame F, the pitch Bb3 (frequency 156 Hz) is 0 + 0 + 0 + 1 + 0 = 1, the pitch B3 (frequency 165 Hz) is 0 + 0 + 1 + 4 + 1 = 6, the pitch C4 (frequency 262 Hz) is 0 + 1 + 3 + 4 + 6 = 14, the pitch C # 4 1 + 2 + 2 + 5 + 1 = 10 at (frequency 277 Hz), 0 + 0 + 1 + 2 + 0 = 3 at pitch D4 (frequency 294 Hz), 0 + 0 + 0 + 1 + 0 = 1 at pitch Eb4 (frequency 311 Hz), and in pitch Bb3 (frequency 156 Hz) in the subsequent frame B 0 + 1 + 1 + 0 + 0 = 2, 2 + 1 + 2 + 1 + 4 = 10 at pitch B3 (frequency 165 Hz), 7 + 7 + 6 + 7 + 6 = 33 at pitch C4 (frequency 262 Hz), 1 + 1 + 2 + 1 + 6 = 11 at pitch C # 4 (frequency 277 Hz), pitch D4 (frequency 294 Hz) 1 + 1 + 0 0 + 2 = 4, pitch Eb4 (Frequency 311Hz) 0 + 0 + 0 + 0 + 1 = 1 is obtained.

  Then, the maximum addition value 33 of the rear frame B and the maximum addition value 14 of the previous frame F are added to obtain 47, and the maximum addition value 33 of the rear frame B and the next highest addition value of the previous frame F are obtained. 49 is obtained by adding 10, and 43 is obtained by adding the maximum addition value 33 of the rear frame B and the second highest addition value 6 of the previous frame F.

  Similarly, by performing this process for all sections, a hypothetical note string of the entire Hamming input is output.

  FIG. 11 is a diagram showing a hypothetical note string output from the entire Hamming input. In this figure, “C4C4”, “C # 4C4”, and “B3C4” are output as hypothetical note sequences of frames between L1 and L7, and “D4D4” is output as hypothetical note sequences of frames between L2 and L8. “Eb4Eb4”, “Eb4E4”, “Eb4D4”, “E4Eb4”, “E4E4”, “E4D4” are output as hypothetical note strings of the frame between L3 and L9, and the frame between L4 and L10 “C4C4”, “B3C4”, and “B3B3” are output as hypothetical note strings, and “D4D4D4Eb4”, “D4D4D4D4”, “D4D4D4C # 4”, “D4D4Eb4Eb4”, “D4D4D4Eb4”, "D4D4Eb4D4", "D4D4E # 4Eb4", "D4D4C # 4D4", "D4D4D4E4", "D4D Eb4C # 4 "," D4D4C # 4C # 4 "is output.

  If the same pitch continues, it is difficult to recognize whether it is an eighth note or two sixteenth notes, so the output is expressed in units of sixteenths.

  After the hypothetical note string is output in this way, the correction process using the note chain model (step S25) is executed, and then the note string having the largest corrected output value is output as the correct answer (step S26).

  FIG. 12 is a diagram showing a correction process using a note chain model. In the correction process using the note chain model, pattern search is first performed (step S25a). If there is a pattern (“YES” in step S25b), “probability based on note chain model = chain probability of matched pattern”. (Step S25c). On the other hand, when there is no pattern (“NO” in step S25b), “probability by note chain model = α (α is a predetermined value)” (step S25d), and in either case, “output value = accuracy by sound × After calculating the probability based on the note chain model (step S25e), the flow returns to the flow of FIG.

  FIG. 13 is a diagram illustrating an example of the calculation result of step S25e. In this figure, on the left side is an output obtained acoustically by hypothetical note string output (step S24 in FIG. 4). Here, the symbols A to A are attached, the staff notation is changed to be easy to see, and further sorted in descending order of accuracy.

  Those candidates (A to F) are multiplied by a probability value by a note chain model. For example, the probability value of A's note chain model is 0.0045, the same probability value of A to D is 0.001, the same probability value of E is 0.00637,. It is assumed that 00701.

  The probability value of the note model represents how the notes are chained, and in this case, indicates the probability that the notes of the length of three quarter notes are chained. Of course, in the case of a melody progression that is generally possible, the probability is high, and in the case of a melody progression whose frequency is less than or equal to a threshold value, it is not registered in the model. The probability value in that case is fixed to α (in this case, α = 0.001) because it is floored in step S25d of FIG.

  Therefore, according to the example shown in the figure, the corrected output value of “a” is “272 × 0.0045 = 1.224”, the same output value of “272 × 0.001 = 0.272”, The same output value of “271 × 0.001 = 0.271”, the same output value of “271 × 0.00637 = 1.72627”,..., The same output value of “266 × 0” .00701 = 1.86466 ”, the musical note string having the largest corrected output value, that is,“ C4C4D4D4E4E4C4C4D4D4D4 ”is output as a correct answer.

  As described above, according to the first embodiment, an unstable melody input by humming is corrected using a note chain model, so that a melody that does not generally exist, that is, a probability of a note chain It is possible to avoid erroneous searches due to low melody. In this regard, when compared with the prior arts 2 to 4, the prior arts 2 to 4 each perform a search by a partial unit such as a musical piece or segment (section) of the music. For this reason, if there are inconveniences such as pitch deviation or noise in the music piece or segment of the humming input acoustic signal, or a lack of frequency resolution in the fast part of the song, these problems remain. Because music search is performed, the wrong music may be searched. This is because collation is performed only in units of parts such as music pieces and segments.

  On the other hand, in the present embodiment, since the humming input melody is corrected using the note chain model, even if an unstable melody is input by humming, a melody having a high chain probability, that is, a general Since the melody is corrected to a possible melody, the accuracy of the music search performed thereafter can be improved.

In the first embodiment, for the sake of simplification, a case has been described in which the start sound of the hypothetical note string matches the start sound of the note chain model. When the two do not match, the correction processing is performed by shifting the pitch of either the start sound of the hypothetical note string or the start sound of the note chain model to match the start sounds. Further, in calculating the correction likelihood, the product of the acoustic likelihood and the language likelihood is used. However, by introducing a weighting factor, the previous equation (1) may be transformed into the following equation (2). .
Corrected likelihood = acoustic likelihood + weighting coefficient × language likelihood (2)

  When building a note chain model, for example, notes for each instrument (piano, guitar, bass, trumpet, etc.), music genre (rock, waltz, Japanese folk song, etc.), or for each composer or singer. A chain model may be constructed. This is preferable because the volume of the note chain model can be reduced, the model construction time can be reduced and the storage capacity can be reduced, and the note chain model for each instrument, genre, and composer. Can be used to model the characteristics of note chains specific to each, thus improving recognition accuracy.

In the first embodiment described above, the model includes the type of note (quarter note, eighth note, etc.) in consideration of the length of the sound. However, the length of the sound is simply ignored. You can build a model.
In the first embodiment, the hamming input is applied. However, the input is not limited to a human voice and may be input by a musical instrument or other sound source.

In the first embodiment, a single tone input is taken as an example, but a chord input by a musical instrument or the like may be used, and a note chain model may be constructed in consideration of a chord.
In the first embodiment, the input sound and the note chain model are set to three quarter notes, but this is a representative example, and this is not necessarily the case. For example, when the input sound is longer, the correction process (see FIG. 12) according to the note chain model of the first embodiment may be repeated while shifting the input sound by n sounds, and the correct sum may be used.

[Second Embodiment]
FIG. 14 is a functional block diagram of the music search device according to the second embodiment. In this figure, the music search device 1a includes an interface unit 2, a control unit 3, an acoustic signal input unit 4, an acoustic signal storage unit 5, a feature extraction unit 6, a feature storage unit 7, a hypothetical note string output unit 8, a note chain model. The music database (abbreviated as DB in the figure) 15 is common to the hamming input device 1 of the first embodiment in that it includes a generation unit 9, a note chain model storage unit 10, an input correction unit 11, and a note string output unit 12. And it differs from the Hamming input device 1 of 1st Embodiment by the point provided with the music search part 14 which consists of a comparison part 16.

  In addition, a common component with the hamming input device 1 of 1st Embodiment, ie, the interface part 2, the control part 3, the acoustic signal input part 4, the acoustic signal storage part 5, the feature extraction part 6, the feature storage part 7, and a hypothesis For the description of the note string output unit 8, the note chain model generation unit 9, the note chain model storage unit 10, the input correction unit 11, and the note string output unit 12, the first embodiment will be referred to.

  The music database 15 stores music information to be searched for in advance. This music information includes note string information for each music.

  FIG. 15 is a diagram illustrating an operation flowchart of the second embodiment. In this flowchart, Step S21 to Step S25 and Step S26 are the same as the operation of the hamming input device 1 of the first embodiment (see FIG. 4), but the sort process (Step S25 and Step S26) Step S31), a loop for the number of songs in the music database 15 (Step S32), reading of reference data for one song from the music database 15 (Step S33), a loop for the number of candidates after correction (Step S34), and similarity calculation ( The difference is that step S35) is performed.

  That is, the comparison unit 16 first takes the corrected output from the input correction unit 11 (corrected using the note chain probability), sorts them, and sorts them out from the highest value within the threshold β. (M note strings and corrected output values) are held as candidates.

  Next, an operation of reading a note string for one song from the music database 15 and comparing the M note strings stored and the note strings read from the music database 15 to calculate the similarity is performed after correction. When the output is repeated and the similarity calculation with the musical note string for one song read from the song database 15 is completed, the next song is read from the song database 15 and the same operation is repeated to make all songs similar. When the degree is calculated, the music with the highest similarity is output as the final result.

  FIG. 16 is a conceptual diagram of similarity calculation. For example, with regard to the similarity evaluation value, 3 points are given when the pitches of every 16th note length are completely matched, 2 points are obtained for semitone deviation, 1 point is given for one note deviation, and 0 points are given for deviations beyond that. To. The similarity is obtained by the following equation (3).

    Similarity = corrected output value × coincidence (3)

  For example, the input pattern of Hamming is “C4C4D4D4E4E4C4C4D4D4D4D4”, and the note sequence pattern of the music being compared is “C4“ D4 ”D4D4E4E4“ C # 4 ”C4D4D4“ F4 ”D4”. The three sounds enclosed by “” (“D4”, “C # 4”, “F4”) are sounds that do not coincide completely. In this case, “D4” is a C4 one-tone deviation, “C # 4” is a C4 semitone deviation, and “F4” is a D4 one-tone or more deviation. 30 ".

  In this way, when the calculation of the degree of similarity with all the music pieces is completed, one or more music lists from the one with the higher degree of similarity are output as the checking result.

  As described above, in the second embodiment, the music search is performed using only the note string corrected using the note chain probability, so that the humming is slightly distorted or affected by noise. But you can search for the correct music.

  In the second embodiment, the music search is performed using only the note string corrected by using the note chain probability. However, the similarity is also obtained for the uncorrected note string, and the similarity calculation is performed. You can use it.

  In the second embodiment, the music database 15 is part of the music search device 1a. However, the present invention is not limited to this mode. For example, a configuration in which one or a plurality of music databases arranged in a remote place via a telephone line or the Internet may be referred to may be used. In this case, the comparison unit 16 may also be installed at a remote place in the same manner as the music database.

  In each of the above embodiments, correction is performed based on the chain probability of “notes”, but the present invention is not limited to this, and the notes may be read as “pitch”.

It is a functional block diagram of the Hamming input device in a 1st embodiment. It is a conceptual diagram which shows the relationship between the acoustic model of speech recognition, and a language model. It is a figure which shows the production | generation (construction) algorithm of a note chain model. It is a figure which shows the operation | movement flowchart of 1st Embodiment. It is a schematic diagram which shows a chroma vector. It is a schematic diagram of a filter. It is a figure which shows an example of a hamming input. It is a frequency distribution figure of a Hamming input. It is a search schematic diagram of a note change candidate. It is a conceptual diagram of hypothesis note sequence output focusing on one frame. It is a figure which shows the hypothetical musical note sequence output from the whole Hamming input. It is a figure which shows the correction | amendment process by a note chain model. It is a figure which shows an example of the calculation result of step S25e. It is a functional block diagram of the music search device in 2nd Embodiment. It is a figure which shows the operation | movement flowchart of 2nd Embodiment. It is a conceptual diagram of similarity calculation.

Explanation of symbols

1 Hamming input device (melody input device)
1a Music searching device 4 Acoustic signal input unit (input means)
6 Feature extraction unit (extraction means)
10 Note chain model storage (database)
11 Input correction unit (correction means)
14 Music search section (search means)

Claims (10)

In a melody input device comprising input means for inputting a melody and extraction means for extracting pitch information from the inputted melody,
A database containing note chain information about the chain of notes;
A melody input device comprising: correction means for correcting pitch information extracted by the extraction means based on note chain information in the database.   The melody input device according to claim 1, wherein the note chain information is created for each musical instrument.   The melody input device according to claim 1, wherein the note chain information is created for each genre of music.   The melody input device according to claim 1, wherein the note chain information is created for each composer.   The melody input device according to claim 1, wherein the note chain information does not take into account information related to a sound length. An input means for inputting a melody;
Extraction means for extracting pitch information from the inputted melody;
A database containing note chain information about the chain of notes;
Correction means for correcting the pitch information extracted by the extraction means based on the note chain information in the database;
A music search apparatus comprising: search means for searching for music having similar pitch information from a plurality of music using the pitch information corrected by the correcting means.   The music search apparatus according to claim 6, wherein the musical note chain information is created for each musical instrument.   The music search apparatus according to claim 6, wherein the musical note chain information is created for each genre of music.   The music search apparatus according to claim 6, wherein the note chain information is created for each composer. 7. The music search apparatus according to claim 6, wherein the musical note chain information does not take into account information related to a sound length.

Images