JP3433818B2 - Music search device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for selecting music that meets conditions by designating music characteristics and music images, and in particular, music retrieval for automatically creating an index required for retrieval. It relates to the device.

[0002]

2. Description of the Related Art Conventionally, in a karaoke machine or the like,
It has been proposed that information such as the song name, singer name, and lyrics of a song is stored in addition to the musical tone information and the song is searched. (For example, JP-A-62-291776 and JP-A-3-2735.
The device described in Japanese Patent Publication No. 85). Further, as a music retrieval using a melody, there is a device described in Japanese Patent Laid-Open No. 2-54300.
This is because when a user sings a melody of a song that he / she wants to retrieve, his / her singing voice is analyzed to extract pitch and length information, and comparison is made with the melody information of a plurality of songs stored in advance,
It is configured to output a song having a melody similar to the input melody.

[0003]

However, in a search using a bibliographic item such as the song title, singer name, and lyrics of a song, accurate information about the song, such as "I want to find a song I've heard before", can be obtained. It's difficult to deal with if you don't have one. In addition, these bibliographic items are not directly related to the musical characteristics such as melody, harmony, and rhythm, and therefore cannot meet the needs for searching using the musical characteristics. As a method of dealing with these, it is conceivable to add an image and musical characteristics of the song in advance, but it is necessary for a person to judge and add such information. Labor is required.

On the other hand, the melody search device described in Japanese Patent Laid-Open No. 2-54300 is effective to some extent when bibliographic items are unknown or when it is desired to directly specify musical characteristics. There is a problem. Generally, in order to sing accurately without accompaniment, it is necessary to listen to the song quite a number of times, and it is difficult to search for a song that is unremembered or never heard. In the case of instrumental music that does not contain vocals, there are many that have complicated melodies that humans cannot sing accurately, and it is difficult to search. Since it is necessary to sing the melody accurately, mental tension is involved, and it is difficult to search in public. It is necessary to store the melody of the music as digital data, but it is difficult to obtain the melody data of many music in a form that can be directly used. In most cases, the data is created manually when creating the database. It is necessary and requires a lot of labor.

In view of this, the present invention automatically creates an index (index) required for a search, and based on the created index, specifies musical characteristics and an image of the song and selects a song that meets the conditions. A search device is provided.

[0006]

The present invention solves the above problems.
In order to do so, the following apparatus is provided. (1) Primary in which is a bibliographic item such as the song title
Means to store the dex and music data of the music
, Time signature classification data, rhythm pattern classification data, and
Frequency of occurrence of chords with a certain pitch difference from the reference tone
At least one of the data is generated and
Generates tempo of music from music data
The generated data is used to show the musical characteristics of the music.
Secondary index that is a secondary index that is an index
And the secondary index created above.
Enter the method of payment and the conditions of the music to be searched from the outside.
Means, each index and the input condition
The means for comparing and the ease of searching according to the result of the comparison.
Means for outputting sound data, and
Music search device. (2) Primary in which is a bibliographic item such as the song title
Means to store the dex and music data of the music
, Time signature classification data, rhythm pattern classification data, and
Frequency of occurrence of chords with a certain pitch difference from the reference tone
At least one of the data is generated and
Generates tempo of music from music data
The generated data is used to show the musical characteristics of the music.
Secondary index that is a secondary index that is an index
And a predetermined line from the secondary index
For a given image of the song based on the shape combination formula
A third-order in which calculates a third-order index showing precision
Dex creating means and the secondary and tertiary inputs created above
Exclude the method of storing the dex and the condition of the music to be searched.
Means for inputting from the section, each index and the input
Depending on the means for comparing the specified conditions and the result of the comparison
Further comprising a means for outputting a musical tone data search Te
Music retrieval device characterized by.

[0007]

The musical tone data such as a CD (compact disc) and the primary index which is a bibliographic item such as a song title are stored in advance in the device. Based on these musical tone data and primary indexes, secondary indexes indicating musical characteristics (objective data that can be expressed on a musical score) such as rhythm information and chord information are extracted and created. Further, based on this secondary index, a tertiary index indicating the image of the music and the situation (subjective / sensitivity feature) in which the music is appropriate is created. Based on the primary index and the automatically created secondary and tertiary indexes, the music is searched based on the musical characteristics and the image of the music.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a music retrieval apparatus according to the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a music search device. (Overall Configuration of Music Retrieval Device) In the figure, 1a and 1b are means for storing music tone data of music and a primary index which is a bibliographic item such as a music title. The musical sound data is a digital or analog audio signal, and a CD (compact disc), together with the primary index,
It is stored on tape, hard disk, or other readable storage medium.

Reference numeral 2 is a means for extracting and creating a secondary index indicating musical characteristics (objective data that can be expressed on a musical score) such as rhythm information and chord information from the musical sound data in the storage means 1a. Reference numeral 3 is a means for creating a tertiary index indicating an image of a music piece and a situation (subjective / sensitive characteristic) suitable for the music piece based on the created secondary index. The secondary and tertiary indexes created by these index creating means 2 and 3 are stored and stored in the storage means 4 and 5. The storage means 4 and 5 for the secondary index and the tertiary index are
It is composed of a readable / writable storage medium such as a hard disk or a magneto-optical disk.

Further, 6 is a condition input means for inputting a condition of a desired music piece. The condition input means 6 is a keyboard,
For example, as shown in FIG. 2, the mouse is used to select an item to be used from the menu corresponding to the primary, secondary, and tertiary indexes.
The inputted conditions are compared by the comparison means 7 with the respective indexes stored and stored in the storage means 1b, 4 and 5 described above. Then, according to the result of the comparison, the musical tone data (the musical piece for which the musical tone data is reproduced) and the respective indexes that match the conditions are output from the search result output means 8. Each means constituting the music search device will be described in detail.

(Structure of Secondary Index Creating Means 2)
FIG. 3 shows a specific configuration of the secondary index creating means 2 which is one of the main parts of the present invention. The tone data stored in the tone data storage means 1a (see FIG. 1) is
In the case of an analog signal, after being converted into digital data by the A / D converter 20, the data for one song is stored in the analysis storage device 21. Then, the tempo and time signature are extracted by the tempo extraction unit 22, the rhythm pattern is extracted by the percussion instrument pattern detection unit 24, and the specific chord is extracted by the specific chord detection unit 28 to create a secondary index.

<Extraction of Tempo and Time Signature by Tempo Extraction Means 22> The tempo extraction means 22 extracts the tempo, time signature, and time signature information of the musical composition and uses it as a secondary index. The music tempo and time signature are extracted as follows. Acoustic power data is obtained from the data stored in the analysis storage device 21, and the differential value x of the acoustic power data is calculated.
(n) {n = 1 to N} is calculated. However, if x (n) is smaller than a certain value, processing for setting x (n) to 0 is performed. Since x (n) has a periodicity and has a property of increasing the correlation with a time difference corresponding to one beat, the calculation of the equation (1) is performed on x (n). Here, when the integer M is changed in the range of N1 or more and N2 or less (however,
N1 and N2 are (integers that satisfy 0 <N1 <N2 <N), and M when B (M) becomes maximum (hereinafter this is referred to as Mmax). further
From equation (2), find the length τ of one beat. Usually, the tempo is expressed by the number of beats per minute, so the tempo TE is calculated using equation (3).

[0013]

[Equation 1]

[Equation 2]

[Equation 3]

Further, in the equation (1), B (M) is calculated when M is set to 2 times and 3 times Mmax, and if B (M) at the time of 2 times is larger than that at the time of 3 times, it is a 4-beat system, If it is the other way around, it will be set to 3 beats. The beat positions of the entire song are detected and used in the subsequent specific chord detection processing. For x (n), check the peak every cycle of Mmax ± α (α is 10% of Mmax),
If there is a peak, use it as the beat position. If it does not exist, repeat the same process until the next peak is found.
During that time, it is assumed that there are beats at equal intervals.

Further, in the time-frequency table creating means 23, the FFT is performed while shifting the observation window by a constant time (for example, several msec) with respect to the data stored in the analyzing storage means 21.
(Discrete Fourier transform) is performed to obtain the power spectrum, and a time frequency table as shown in FIG. 4 is created.
Below, the value of the time-frequency table at time t and frequency f is expressed as S (t, f).

<Rhythm pattern extraction by percussion instrument pattern detecting means 24> Next, percussion instrument pattern detecting means 24
The rhythm pattern (beat) extraction by will be described. Rhythm patterns are considered to be roughly classified according to performance patterns of percussion instruments, so percussion instrument performance information is extracted. It can be said that extracting and using performance information of percussion instruments is more practical than extracting and using performance information of other musical instruments. The rhythm pattern is classified into, for example, “no drum”, “8 beats”, and “16 beats”, but is not limited to these.

In the percussion instrument spectrum data storage means 25, time variation information D (t, f) of a typical power spectrum of a percussion instrument to be detected is stored for each type of percussion instrument, as shown in FIG. In the percussion instrument pattern detection means 24, S (t,
f) and D (t, f) are calculated by Eq. (4) to find the time when the distance H (T) between them at time T becomes smaller than a certain value, and this is the time when the percussion instrument was played. To consider.

[0018]

[Equation 4]

Further, as shown in FIG. 6, the percussion instrument playing time is standardized into a musical note by a unit time which is an integral fraction of the length of one beat. For example, in the case of ordinary song and popular music, performance information can be obtained with sufficient time accuracy if the unit time is about 1/12 of the length of one beat. The above processing is performed for each percussion instrument detected. For ordinary songs and popular music, it is sufficient to detect several kinds of percussion instruments.

As shown in FIG. 7, the various rhythm pattern storage means 26 stores the relation between the performance information of each percussion instrument expressed in the unit time and the overall rhythm pattern. The extracted performance information of a plurality of percussion instruments is compared with the stored performance information, and the closest rhythm pattern is determined to be the rhythm pattern of the music. As an example, the hi-hat is mainly composed of eighth notes and the snare is 2, 4
If it is centered on the beat, it is determined to be "8 beats". If there is no percussion instrument, it is determined that there is no drum.

<Specific Chord Extraction by Specific Chord Detection Unit 28> Next, pitch detection unit 27 and specific chord detection unit 28
The specific chord extraction by will be described. Some chords include
It is considered that there is a characteristic thing that greatly affects the character of the music, and in order to detect such a chord, the specific chord detection means 28 performs the processing shown in FIG.

First, in the pitch detecting means 27, it is considered that a musical interval musical instrument (including a singing voice) is played at a portion where substantially the same frequency continues for a certain time or longer from the time frequency table, and the frequencies of this portion are averaged. After processing, it corresponds to the scale of equal temperament. The time axis corresponds to the obtained beat position. As a result, a pitch map as shown in FIG. 9 is obtained. By the way, most musical pieces use multiple pitched musical instruments, but at the present time, the technique of separating and detecting the performance information of each musical instrument part from the acoustic signal in which multiple pitched musical instruments are mixed is practical. is not. However, to detect something with a certain frequency based on the root of a chord,
It is possible without separating the musical instrument parts, and the musical characteristics are well expressed, which is practical.

In the specific chord detecting means 28, once in 4 beats in the case of 4-beat system, once in 3 beats in the case of 3-beat system, a section corresponding to the length of 1 beat in the pitch map. Inspect the lowest note of. If the pitch of the lowest note has not changed in this section, it is regarded as a valid section and the following processing is performed. If it has changed, the next section is inspected (step 28a in FIG. 8,
28b, 28c). If it is a valid section, the lowest note is regarded as the root note, the pitch difference (frequency) from the lowest pitch is calculated for all the pitches included in that section, and the frequency is compared with prestored frequency data. If so, the chord is specified (steps 28d, 28e, 28f). further,
The appearance frequency of the specific chord is calculated throughout the entire song (step 28g).

As an example of the specific chord, 9,1 for the root note
There are "tension chords" with a frequency of 1, 13 degrees, but the chords to be detected are not limited to these. The tension code is considered to be closely related to the genre and mood of the song, for example, it is often used in jazz. The "tempo" and "specific chord frequency" obtained by the above processing are stored as numerical values, and the "beat" and "rhythm pattern" are stored in the form of a classification code, and the tertiary index creating means 3 and the comparing means 7 (Fig. 1)).

(Structure of tertiary index creating means 3)
Next, the tertiary index creating means 3 (see FIG. 1) will be described. The information of the secondary index represents musical characteristics such as "tempo of quarter note 120 per minute" and "8 beats" (objective data that can be expressed in musical score). Such characteristics may be directly specified. However, this method requires some musical knowledge, so other designation methods may be preferable in some cases. It is considered that there is a certain kind of correlation between the image of the song and the situation suitable for listening to the song and the secondary index created above, and this is used to create the tertiary index. First, decide the item of the tertiary index. For example, subjective and emotional characteristics such as "brightness" and "modernity" as the image of the music and "BGM" and "drive" as the situation of listening to the music can be considered.

Next, for each item, the precision ratio Gi {i = 1 to K: K is the number of tertiary indexes} and the value Ej {j = 1 to L of each item of the secondary index. : L represents the relationship of the number of secondary indexes} by, for example, equation (5), and obtains the weighting coefficient Wij.

[0027]

[Equation 5]

In Wij, a human gives Gi to a part of the stored music, and the relationship with Ej is obtained by a multivariate analysis method. When Wij is calculated, Gi is calculated for all the music stored by the formula (5), and 3
It is stored in the secondary index storage means 5 (see FIG. 1).
Although a linear relationship is used here, a non-linear relationship may be used. Since the secondary and tertiary indexes can be created independently of the storage of the musical tone data and the primary index, it is relatively easy to change the search items after creating the database.

(Music Search Using Primary, Secondary, and Tertiary Indexes) As shown in FIG. 1, the condition input means 6 is composed of a keyboard, a mouse, etc., and is shown in FIG. 2, for example. As described above, the item to be used is selected from the search menu M corresponding to the primary index I ₁ , the secondary index I ₂ , and the tertiary index I ₃ . AND and OR conditions may be specified using multiple items. With regard to the tempo and the like, rough designation such as “fast” or “slow” may be made without directly inputting the numbers. In this case, processing using fuzzy theory may be performed.

It is also possible to input a known music piece as a key. In this case, the comparison is performed using the secondary and tertiary indexes of the music piece of the key. The comparison means 7 compares the input conditions with the primary, secondary, and tertiary index data. The comparison with the primary index is similar to a normal search. For the secondary index, the ones whose numerical values or classification codes of each item are close to each other are candidates. For the third-order index, a song with a high matching rate Gi is selected as a candidate. As the search result, the primary, secondary, and tertiary index information including the song names of one or a plurality of songs that meet the conditions are displayed on the monitor screen, and at the same time, as a sound signal through the speaker, the search result output means 8 Is played.

[0031]

As described above in detail, the music retrieval apparatus according to the present invention has the following features. Musical characteristics (secondary index) and music images (third order)
Since the index mechanisms) can search using allows you to find the music even if it does not have accurate information, spread the use range. When searching using the image of a song
Since users can easily access songs that they do not know by relying on the image, more songs can be actually used than before. Since the index used for searching can be automatically extracted and created from the music data , the construction of the database is greatly reduced compared to the case where the index is added by human judgment. Further, it is possible to easily deal with the change of the search item after the database is created. In addition, to create extract the secondary index indicating the musical features such as rhythm information and chord information (objective data that can be represented in music), and the secondary index to the original, the music images ( Create a third-order index that indicates subjective and emotional characteristics)
In that case , a practical index can be easily obtained. Because I don't directly input the melody,
It can be used for any kind of song, can be used by anyone, and can be done anywhere you search.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of a music search device according to the present invention.

FIG. 2 is an example of a menu screen when a search condition is input.

FIG. 3 is a block diagram of a secondary index creating means.

FIG. 4 is an example of a time frequency table.

FIG. 5 is an example of power spectrum time change information of a percussion instrument.

FIG. 6 is a diagram illustrating extraction of performance information of a percussion instrument.

FIG. 7 is an example of various rhythm patterns.

FIG. 8 is a flowchart of specific chord extraction.

FIG. 9 is an example of a pitch map.

[Explanation of symbols]

1a, 1b means for storing a primary index, 2 secondary index extraction creating means, 3 tertiary index creating means, 4 secondary index storing means, 5 tertiary index storing means, 6 (search condition) input means 7 comparing means 8 search result output means, 22 tempo extraction unit, 24 percussion pattern detecting means, 28 a particular chord detection means M search menu, I ₁ 1-order index, I ₂ 2-order index, I ₃ 3 Secondary index.

─────────────────────────────────────────────────── ─── Continuation of front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) G11B 27/10 G10K 15/04 302 G10H 1/38 G10H 1/40

Claims (4)

(57) [Claims] 1. A primary item that is a bibliographic item such as a song title.
A means for storing an index and a beat classification data, a rhythm pattern
Turn classification data and a specific pitch difference with respect to the reference sound
At least one of the frequency data of the chords
Along with the generation of the music data,
Data representing the po is generated, and each generated data is
A secondary index that is an index that indicates the musical characteristics of a song
A secondary index creating means, a means for storing the created secondary index, a means for externally inputting a condition of a music piece to be searched, and a means for comparing each index with the input condition.
And the musical tone data searched according to the comparison result is output.
Music searching device comprising means that, further comprising: a. 2. A primary item that is a bibliographic item such as a song title.
A means for storing an index and a beat classification data, a rhythm pattern
Turn classification data and a specific pitch difference with respect to the reference sound
At least one of the frequency data of the chords
Along with the generation of the music data,
Data representing the po is generated, and each generated data is
A secondary index that is an index that indicates the musical characteristics of a song
And secondary indexing means to, based from the secondary index to a predetermined linear combination formula
3rd order showing the matching rate of the music to the specified image
Means for creating a tertiary index for calculating a dynamic index
And a procedure for storing the created secondary and tertiary indexes.
Dan, means for inputting the condition of the music piece to be searched from the outside, and a means for comparing each index with the input condition.
And the musical tone data searched according to the comparison result is output.
Music searching device comprising means that, further comprising: a. 3. The secondary index creating means includes means for creating a time-frequency table of musical sound data and percussion instruments of a type closely related to a rhythm pattern of a musical piece.
About the means to store typical spectrum data, and the performance timing of each percussion instrument for each rhythm pattern
Various rhythm pattern storage means, the time-frequency table, and a representative spectrum of the percussion instrument
Music data and the performance timing of each of the above percussion instruments.
Determine the rhythm pattern of the song and
Serial to claim 1 or claim 2 characterized in that it has a percussion pattern detecting means for obtaining data, the
Music search device listed. 4. The secondary index creating means creates means for creating a time-frequency table of musical sound data, and a frequency range table for a predetermined time or more from the time-frequency table.
Sound that detects a portion where the wave number component lasts as a significant pitch
The lowest note that is the reference sound is detected from the output of the pitch detection means and the previous period pitch detection means.
However, the section in which there is a sound that has a specific pitch difference from the reference sound is specified.
Calculate the frequency of occurrence of specific chords, with the constant chords existing section,
Frequency of occurrence of chords with a certain pitch difference from the reference tone
3. A specific chord detecting means for obtaining data, and claim 2.
Music search device listed.