TWI579716B - Two - level phrase search system and method - Google Patents

Description

Two-level phrase search system and method

The present invention relates to a two-level phrase search system and method, and more particularly to a two-level phrase search system and method capable of balancing a fast comparison mechanism and a precision comparison mechanism.

At present, with the popularity of digital music, music data is growing at an alarming rate. Among them, the Musical Instrument Digital Interface (MIDI) is widely used to simplify the expression of music, and has created a huge number of instrument digital interface file database. How to search from it becomes a popular technology.

Traditionally, the use of text search and search can only search for additional annotations of music, and it does not satisfy the user's need to search for music directly through melody. Therefore, many methods of searching for phrases are invented to improve this problem.

Since the production method of musical phrases is not unique, and the sources of search phrases and database phrases may be different, when doing phrase comparison, additional calculations are needed to more accurately measure the similarity between the two phrases. Various musical phrase comparison mechanisms were invented, such as string similarity (Levenshtein distance), longest common subsequence (LCS, longest common Subsequence), linear scaling, dynamic time warping (DTW), etc. How to choose the appropriate comparison mechanism often means the trade-off between computational complexity and accuracy.

Simpler methods such as linear stretching can result in faster response times, but also Rough search results, while more complex methods such as dynamic time correction are used to exchange more accurate search results for a longer period of time.

In view of the above-mentioned problems of the prior art, the object of the present invention is to provide a two-level phrase search system and method for solving and improving the problem of difficulty in selecting between search rate and accuracy when searching for a phrase.

The phrase search system of the present invention comprises a phrase extraction module, a phrase association building module, a phrase database, a first level phrase comparison module and a second level phrase comparison module. The phrase capture module extracts the phrase segment from the symbolic music file (MIDI); the phrase association module calculates the phrase correlation coefficient of the phrase segment; the phrase database stores the information of the phrase segment to be compared and the phrase correlation coefficient; the first level The phrase comparison module filters out the first-level candidate phrase set from the phrase database, and compares the phrase set most relevant to the to-be-matched phrase segment; the second-level phrase comparison module is based on the first-level phrase comparison model. The comparison result of the group and the phrase correlation coefficient of the phrase database select a second level candidate phrase set, and compare the phrase most relevant to the to-be-matched phrase segment from the second level candidate phrase set.

The phrase search method of the present invention comprises the following steps: extracting a phrase segment from a symbolic music file (MIDI) by using a phrase capture module; and calculating a phrase correlation coefficient of the phrase segment by using a phrase association module; storing the waiting ratio in the phrase database Information about the phrase segment and the phrase correlation coefficient; the first level phrase matching module selects the first level candidate phrase set from the phrase database, and compares the phrase set most relevant to the to-be-matched phrase segment; The second level phrase comparison module selects the second level candidate phrase set according to the comparison result of the first level phrase comparison module and the phrase correlation coefficient of the phrase database, and compares the second level candidate phrase set from the second level candidate phrase set. The phrase most relevant to the segment of the phrase to be compared.

In view of the above, a two-level phrase search system and method in accordance with the present invention may have one or more of the following advantages:

1. Correlation coefficient establishment: Pre-analyze the music database and establish the correlation coefficient between music phrases. When combined with the search results of fast search, it can effectively limit the search range of accurate search, and achieve the effect of speed and precision.

2. Quick Search: Provides a quick music phrase search method at the first level, which provides a short search result to the user in a short period of time.

3. Accurate search: Provide accurate music phrase search methods at the second level, and use the results of quick search and pre-calculated correlation coefficients to find accurate search results for users.

10‧‧‧ Phrase Search System

100‧‧‧ phrase extraction module

101‧‧‧ Input audio signal

110‧‧‧Query phrase sequence number sequence

200‧‧‧ phrase association module

210‧‧‧ Phrase data inventory module

220‧‧‧ Phrase Correlation Computation Module

221‧‧‧ phrase correlation coefficient algorithm

300‧‧‧ Phrase Database

310‧‧‧ Phrase Clip Information Sheet

311‧‧‧ music number

312‧‧‧Segment number

313‧‧‧ Phrase segment number sequence

320‧‧‧ Phrase Correlation Coefficient Data Sheet

321‧‧‧Source music number

322‧‧‧Source segment number

323‧‧‧Object music number

324‧‧‧ Target segment number

325‧‧‧Correlation coefficient

400‧‧‧First level phrase comparison module

410‧‧‧First-level candidate phrase collection screening module

411‧‧‧First-level candidate phrase collection screening algorithm

412‧‧‧First Level Candidate Phrase Collection

420‧‧‧First Level Phrase Correlation Coefficient Calculation Module

421‧‧‧The first level of phrase correlation coefficient algorithm

422‧‧‧First-level candidate phrase correlation coefficient set

430‧‧‧First level phrase query result screening module

431‧‧‧The first level of phrase query result screening algorithm

432‧‧‧First level candidate phrase query results

500‧‧‧Second level phrase comparison module

510‧‧‧Query expansion module

511‧‧‧Second Level Candidate Phrase Collection

520‧‧‧Second level phrase correlation coefficient calculation module

521‧‧‧Second level phrase correlation coefficient algorithm

522‧‧‧Second Level Candidate Phrase Correlation Coefficient Set

530‧‧‧Second level phrase query result screening module

531‧‧‧Second level phrase query result screening algorithm

532‧‧‧Second Level Candidate Phrase Search Results

1 is a system architecture diagram of a phrase search system of the present invention.

2 is a schematic diagram of a phrase capture module of the phrase search system of the present invention and an input thereof.

3 is a schematic diagram of a phrase database of the phrase search system of the present invention.

4 is a diagram showing a phrase association building module and a phrase database association of the phrase search system of the present invention.

FIG. 5 is a diagram showing the relationship between the first level phrase comparison module and other data of the phrase search system of the present invention.

FIG. 6 is a diagram showing the second level phrase comparison module and other data association diagrams of the phrase search system of the present invention.

The main components of the phrase search system 10 of the present invention are as shown in FIG. 1 and FIG. 2, which are composed of five system components. The names and main functions are described as follows: the phrase capture module 100 is from the instrument digital interface file (MIDI). Taking a phrase segment; the phrase association building module 200 calculates a correlation coefficient between the two phrases; the phrase database 300 stores the information of the phrase segment to be compared (or the information of the phrase segment) and the phrase correlation coefficient; The first-level phrase comparison module 400 filters out the first-level candidate phrase set from the phrase database, and compares the phrase set most relevant to the query phrase; the second-level phrase comparison module 500 compares the first-level phrase according to the first-level phrase. The result of the module, and the phrase correlation coefficient of the phrase database, selects the second level candidate phrase set. And the phrase most relevant to the query phrase is compared from the second level candidate phrase set.

Referring to FIG. 3, the phrase database 300 of the present invention includes two sets of main data tables, one of which is a phrase segment information data table 310, and the other is a phrase correlation coefficient data table 320. The data field of the phrase segment information table 310 includes a music number 311, a segment number 312, and a phrase segment number sequence 313. The data field of the phrase correlation coefficient data table 320 contains the source music. No. 321, source segment number 322, destination music number 323, destination segment number 324, and correlation coefficient 325.

The phrase association building module 200 includes a phrase material inventory taking module 210, And the phrase correlation calculation module 220. The phrase data inventory retrieval module 210 is responsible for connecting and accessing the phrase segment information data table 310 and the phrase correlation coefficient data table 320 in the modified phrase database 300. The phrase association calculation module 220 extracts any two phrase information by the phrase data inventory module 210, and calculates the correlation coefficient between the phrases by the phrase correlation coefficient algorithm 221, and the source music number, the source phrase number, the destination music number, and the destination phrase. The number and associated coefficient are stored in the phrase correlation coefficient data table 320.

The phrase capture module 100 will input the audio signal 101 or the symbolic pitch time. The sequence 102 is converted into a format in which the sequence number sequence of the phrase segments in the phrase database is stored, and becomes a query phrase segment number sequence 110 for subsequent comparison search.

Referring to FIG. 5, the first level phrase comparison module 400 includes the first level candidate. The selected phrase collection screening module 410, the first level phrase correlation coefficient calculation module 420, and the first level phrase query result screening module 430. The first level candidate phrase set screening module 410 filters the algorithm 411 according to the first level candidate phrase set, and selects the first level candidate phrase set 412 from the phrase segment information table 310. The first level phrase correlation coefficient calculation module 420 sequentially calculates the query phrase segment number sequence 110 and the first level candidate phrase correlation coefficient set 422 of the first level candidate phrase set 412 by the first level phrase correlation coefficient algorithm 421. The first level phrase query result screening module 430 selects the first level candidate phrase query result 432 according to the first level candidate phrase correlation coefficient set 422, and selects the first level candidate phrase query result 432.

Referring to FIG. 6, the second level phrase comparison module 500 includes a query expansion module. The group 510, the second level phrase correlation coefficient calculation module 520 and the second level phrase query result screening module 530. The query expansion module 510 sequentially queries the phrase correlation coefficient data table 320 with the first level candidate phrase query result 432 to filter out the second level candidate phrase set 511. The second level phrase correlation coefficient calculation module 520 sequentially calculates the second layer candidate phrase correlation coefficient set 522 of the query phrase segment number sequence 110 and the second level candidate phrase set 511 by the second level phrase correlation coefficient algorithm 521. The second level phrase query result screening module 530 selects the second level candidate phrase query result 531 according to the second level candidate phrase correlation coefficient set 522, and selects the second level candidate phrase query result 532 as the result of the query. .

The operation mechanism of the phrase search system of the present disclosure will be described below in the drawings.

Step 1: As shown in FIG. 4, the phrase correlation calculation module 220 extracts any two phrase information by the phrase data inventory taking module 210, calculates the correlation coefficient between the phrases by the phrase correlation coefficient algorithm 221, and numbers the source music number and the source phrase number. The destination music number, the destination phrase number, and the correlation coefficient are stored in the phrase correlation coefficient data table 320. The phrase correlation coefficient algorithm 221 can normalize the two phrases using the mean/median, and obtain the distance between the two phrases using the dynamic time corrected pitch contour comparison method, and obtain the correlation coefficient after taking the reciprocal. For example, the sequence of numbers in phrase A is "10, 10, 10, 11, 11", and the sequence of numbers in phrase B is "19, 20, 20, 20, 22, 21". The phrase B is first normalized to phrase B'" 9, 10, 10, 10, 12, 11", and then through the dynamic time correction to calculate the difference between the phrase A and B' is a delete operation and a substitution operation, the distance is 2, the correlation coefficient is 0.5.

Step 2: The phrase capture module 100 converts the input audio signal or symbolic pitch time series into a format of the sequence of the phrase segment digital sequence stored in the phrase database, and becomes a query phrase segment digital sequence 110 for subsequent comparison search. Query phrase segment number sequence 110 is The main sound frequency values obtained by sampling at fixed time intervals, for example, the serial sequence of the central Do that is sampled at intervals of 200 ms for one second is "261.6, 261.6, 261.6, 261.6, 261.6".

Step 3: The first level candidate phrase set screening module 410 filters the algorithm 411 according to the first level candidate phrase set, and selects the first level candidate phrase set 412 from the phrase segment information table 310. The first level candidate phrase set screening algorithm 411 may randomly take out a phrase segment subset from the phrase segment information table 310; the first level candidate phrase set screening algorithm 411 may also use the phrase segment information according to the search condition. A subset of the specific music numbers is obtained in table 310.

Step 4: The first level phrase correlation coefficient calculation module 420 sequentially calculates the query phrase segment number sequence 110 and the first level candidate phrase correlation coefficient set 422 of the first level candidate phrase set 412 by the first level correlation coefficient algorithm 421. The first level phrase correlation coefficient algorithm 421 can normalize the query phrase segment number sequence 110 and the first level candidate phrase using the mean/median, and calculate the Euclidean distance between the two, and take the countdown to become the first One-level candidate phrase correlation coefficient.

Step 5: The first level phrase query result screening module 430 selects the first level candidate phrase query result 431 according to the first level candidate phrase correlation coefficient set 422, and selects the first level candidate phrase query result 432. The first level phrase query result screening algorithm 431 can be sorted according to the correlation coefficient from large to small, and the top ten is obtained as the query result.

Step 6: The query expansion module 510 sequentially queries the phrase correlation coefficient data table 320 with the first level candidate phrase query result 432, and filters out the second level candidate phrase set 512. The second level candidate phrase set 511 screening mode may be a set of one hundred words before all the first level candidate phrase query result 432 has the highest correlation coefficient in the phrase correlation coefficient data table 320.

Step 7: The second level phrase correlation coefficient calculation module 520 sequentially calculates the query phrase segment number sequence 110 and the second level candidate phrase correlation coefficient set 522 of the second level candidate phrase set 511 by the second level phrase correlation coefficient algorithm 521. . The second level correlation coefficient algorithm 521 may be the same as the phrase correlation coefficient algorithm 221.

Step 8: The second level phrase query result screening module 530 selects the second level candidate phrase query result 531 according to the second level candidate phrase correlation coefficient set 522, and selects the second level candidate phrase query result 532 as the current The result of the query. The second level phrase query result screening algorithm 532 can be sorted according to the correlation coefficient from large to small, and the top ten is obtained as the query result.

In summary, the two-level phrase search system and method of the present invention have the advantages of innovation, immediateness, efficiency, and expandability, and are not only innovative in terms of technical ideas, but also can efficiently and quickly and accurately query music. Phrases.

The above is intended to be illustrative only and not limiting. Any equivalent modifications or alterations to the spirit and scope of the invention are intended to be included in the scope of the appended claims.

10‧‧‧ Phrase Search System

100‧‧‧ phrase extraction module

200‧‧‧ phrase association module

300‧‧‧ Phrase Database

400‧‧‧First level phrase comparison module

500‧‧‧Second level phrase comparison module

Claims (10)

A two-level phrase search system includes: a phrase capture module, which retrieves a plurality of phrase segments from a symbolic music file (MIDI); a phrase association module to calculate a phrase correlation coefficient of the phrase segments; a phrase database storing at least one information of the phrase segments to be compared with the phrase correlation coefficients; a first level phrase matching module, filtering a plurality of first level candidate phrase sets from the phrase database, and comparing a set of the phrases most relevant to the pair of to-be-matched phrases; and a second-level phrase comparison module, the result of the comparison according to the first-level phrase comparison module, and the phrase associations of the phrase database The coefficient selects a plurality of second-level candidate phrase sets, and compares one phrase that is most relevant to the to-be-matched phrase segment from the second-level candidate phrase sets. The second-level phrase search system as described in claim 1, wherein the phrase database includes: a phrase segment information data table, wherein the data field includes a music number, a segment number, and a phrase segment number sequence; A phrase correlation coefficient data table, wherein the data field includes a source music number, a source segment number, a destination music number, a destination segment number, and the phrase correlation coefficients. For example, the second-level phrase search system described in claim 2, wherein the phrase association building module comprises: a phrase data inventory taking module, connecting and accessing and modifying the phrase segment information data table in the phrase database And the phrase correlation coefficient data table; and a phrase correlation calculation module, calculating the to-be-compared pairs by a phrase correlation coefficient algorithm according to the information of the to-be-matched phrase segments stored in the phrase data inventory taking module Phrase piece The phrase correlation coefficient of the segment, and storing the source music number, the source phrase number, the destination music number, the destination phrase number, and the phrase correlation coefficients to the phrase correlation coefficient data table. The second-level phrase search system described in claim 2, wherein the phrase capture module converts one of the input audio signals or a symbolic pitch time sequence into a sequence of the phrase segments in the phrase database. The storage format becomes a sequence of query phrase segments. For example, in the second-level phrase search system described in claim 4, the first-level phrase matching module includes: a first-level candidate phrase set screening module, and a screening algorithm according to a first-level candidate phrase set. Extracting the first-level candidate phrase sets from the phrase segment information data table; a first-level phrase correlation coefficient calculation module, sequentially calculating a number sequence of the query phrase segment by a first-level phrase correlation coefficient algorithm And a first level candidate phrase correlation coefficient set of the first level candidate phrase set; and a first level phrase query result screening module, according to the first level candidate phrase correlation coefficient set, querying with a first level phrase The result is a screening algorithm that screens out the results of a first-level candidate phrase query. The second-level phrase search system described in claim 5, wherein the second-level phrase matching module comprises: a query expansion module, and sequentially querying the phrase correlation coefficient by using the first-level candidate phrase query result The data table selects the second-level candidate phrase sets; a second-level phrase correlation coefficient calculation module calculates the number sequence of the query phrase segments and the seconds in sequence by a second-level phrase correlation coefficient algorithm Hierarchical candidate phrase set a second level candidate phrase correlation coefficient set; and a second level phrase query result screening module, according to the second level candidate phrase correlation coefficient set, using a second level phrase query result screening algorithm, screening one The second level candidate phrase query result is used as the phrase. A two-level phrase search method includes: extracting a plurality of phrase segments from a symbolic music file (MIDI) by using a phrase extraction module; and calculating a phrase correlation coefficient of the phrase segments by using a phrase association module; a phrase database stores at least one information of the phrase segments to be compared with the phrase correlation coefficients; and a first level phrase matching module selects a plurality of first level candidate phrase sets from the phrase database, and compares a set of the phrase that is most relevant to the segment of the phrase to be compared; and a comparison result of the second level phrase comparison module according to the first level phrase comparison module, and the phrase association of the phrase database The coefficient selects a plurality of second-level candidate phrase sets, and compares one phrase that is most relevant to the to-be-matched phrase segment from the second-level candidate phrase sets. The method for searching for a two-level phrase according to claim 7 , wherein the phrase database comprises a phrase information data table and a phrase correlation coefficient data table, wherein the data field of the phrase information table includes a music number a segment number and a phrase segment number sequence, wherein the data field of the phrase correlation coefficient data table includes a source music number, a source segment number, a destination music number, a destination segment number, and the phrase correlation coefficients; The method further comprises the steps of: connecting and revising the phrase segment information data table in the phrase database and the phrase correlation coefficient data table by using a phrase data inventory module; The phrase correlation calculation module calculates the phrase correlation coefficient of the to-be-matched phrase segments by using a phrase correlation coefficient algorithm according to the information of the to-be-matched phrase segments stored in the phrase data inventory module. The source music number, the source phrase number, the destination music number, the destination phrase number, and the phrase correlation coefficients are stored in the phrase correlation coefficient data table. The method for searching for a second-level phrase according to claim 8 of the patent application, further comprising the steps of: converting the input audio signal or a symbolic pitch time sequence to the phrase database according to the phrase retrieval module; The storage format of the sequence of digits of the phrase segment becomes a sequence of digits of the query phrase segment. The method for searching a second-level phrase according to claim 9 , wherein the first-level phrase comparison module comprises a first-level candidate phrase collection screening module, a first-level phrase correlation coefficient calculation module, and a first level phrase query result screening module, the second level phrase comparison module comprises a query expansion module, a second level phrase correlation coefficient calculation module and a second level phrase query result screening module; The method further includes the following steps: filtering the algorithm according to a first level candidate phrase set by the first level candidate phrase set screening module, and filtering the first level candidate phrase sets from the phrase segment information table; The first level phrase correlation coefficient calculation module sequentially calculates a query phrase segment number sequence and a first level candidate phrase correlation coefficient set of the first level candidate phrase set by a first level phrase correlation coefficient algorithm; The first level phrase query result screening module is based on the first level candidate phrase correlation coefficient set, and is first Hierarchical phrase query result screening algorithm, screening out a first level a candidate phrase query result; the query expansion module sequentially queries the phrase correlation coefficient data table according to the first hierarchical candidate phrase query result, and selects the second hierarchical candidate phrase set; and calculates the second hierarchical phrase correlation coefficient The group sequentially calculates a query sequence number sequence of the query phrase segment and a second layer candidate phrase correlation coefficient set of the second level candidate phrase set by a second level phrase correlation coefficient algorithm; and query result by the second level phrase The screening module selects a second-level candidate phrase query result screening algorithm according to the second-level candidate phrase correlation coefficient set, and selects a second-level candidate phrase query result as the phrase.