JP6179188B2 - Music analyzer - Google Patents

Description

  The present invention relates to a technique for specifying a played music piece.

  There is a performance recording device disclosed in Patent Document 1 as a device for recording performance information representing the operation contents of a keyboard. In the performance information recording mode, this performance recording device stores performance information, which is generated in time series based on the operation of the keyboard and representing the pitch, and time information in a temporary storage area. When the performance recording device does not perform for a predetermined time, the performance recording device assumes that the performance has ended, and stores a performance recording file in which performance information temporarily stored is collected in an external storage device. In this performance recording apparatus, mark names such as “lesson” and “birthday” can be set for the time information. For example, if a song name is set as a mark name, it is possible to know which song has been played from which point.

JP 2008-233574 A

  According to the performance recording apparatus of Patent Document 1, the user can know what kind of music was played at which time by setting the mark name. However, in this performance recording apparatus, when setting a mark name, the user needs to perform an operation for displaying a setting screen and input the mark name on the operation unit, and such trouble is felt complicated. Some people.

  The present invention has been made under the above-described background, and an object of the present invention is to provide a technique for specifying a played song without bothering the user.

  The present invention includes performance data acquisition means for acquiring performance data indicating a performance, classification means for dividing the performance data into a plurality of sections on a time axis, and music data acquisition means for acquiring a plurality of music data indicating music. And specifying the music data similar to the performance data of the section for each of the plurality of sections based on the performance data sorted by the sorting means and the plurality of music data obtained by the music data obtaining means And means for aggregating the consecutive sections into one section when the sections having the same identification result of the identifying means are continuous in the plurality of sections, and the aggregated section and the aggregation means Provided is a music analysis apparatus having an adding means for adding tag information indicating music data identified as similar for each of the sections that are not performed.

  In the present invention, the sorting means may be configured to sort the performance data by a period in which performance is not performed.

  Moreover, in this invention, it is good also as a structure which has a division means which divides | segments the said performance data for every area which is not aggregated by the said aggregated area and the said aggregation means.

  ADVANTAGE OF THE INVENTION According to this invention, the performed music can be specified, without bothering a user's hand.

The block diagram which showed the hardware constitutions of the music analysis apparatus which concerns on this invention. The block diagram which showed the structure of the function implement | achieved in a music analysis apparatus. The figure for demonstrating a feature-value series. The figure for demonstrating operation | movement of an analysis part. The figure which showed an example of the screen displayed in a music analysis apparatus. The figure which showed the example of a division of performance note string data. The figure for demonstrating the similarity index value X [n].

[First Embodiment]
FIG. 1 is a block diagram showing a hardware configuration of a music analysis apparatus 10 according to an embodiment of the present invention. The music analysis apparatus 10 acquires note string data (hereinafter referred to as performance note string data Q), which is an example of performance data representing a performance performed by the electronic musical instrument 20, from the electronic musical instrument 20, and the acquired musical note string data. It is a computer device that identifies a played music piece by analyzing Q. The music analysis device 10 is a device that can execute a program, such as a personal computer, a smartphone, or a tablet terminal. The electronic musical instrument 20 is, for example, an electronic piano, and outputs performance note string data Q representing the performance according to the performance of the performer. In the present embodiment, the musical note string data Q is data in the MIDI (Musical Instrument Digital Interface: registered trademark) format. When the performer performs a plurality of music pieces in succession, the performance note string data Q includes MIDI data of a plurality of music pieces.

The communication unit 105 has a function of communicating with the electronic musical instrument 20 connected to the communication unit 105. The communication unit 105 acquires the performance note string data Q supplied from the electronic musical instrument 20 and supplies the acquired performance note string data Q to the control unit 101. The display unit 103 includes a liquid crystal display, and displays a screen and various messages for operating the music analysis apparatus 10. The operation unit 104 includes a pointing device that is provided on the surface of the display unit 103 and transmits an image displayed on the display unit 103 and detects a position touched by a finger. The control unit 101 controls each unit according to the detected position and the screen displayed on the display unit 103. When the music analysis device 10 is a personal computer, the user operates the operation unit 104 with a mouse or a keyboard.
The storage unit 102 includes a hard disk device, and stores performance note string data Q supplied from the electronic musical instrument 20. The storage unit 102 stores in advance musical note string data (hereinafter referred to as reference note string data R) of a plurality of musical pieces used when analyzing the musical note string data Q. The reference note string data R is an example of music data representing a music. In the present embodiment, the reference note string data R is also MIDI format data. Note that the reference note string data R includes an identifier (for example, a song name) uniquely assigned to a song, a tempo of the performance of the song, and the like.

  The control unit 101 includes a central processing unit (CPU), a read only memory (ROM), and a random access memory (RAM). When the CPU executes a program stored in the ROM, a function of acquiring performance note string data Q, specifying a song included in the performance note string data Q, a function of displaying the specified song, and the like are realized. .

  FIG. 2 is a block diagram illustrating a configuration of functions realized in the control unit 101. The analysis unit 1001 acquires the performance note string data Q from the communication unit 105 and the reference note string data R from the storage unit 102. That is, the analysis unit 1001 functions as an acquisition unit that acquires note string data. Based on the performance note string data Q and the reference note string data R, the analysis unit 1001 analyzes which reference note string data R the performance note string data Q is similar to, and identifies a played music piece.

  The analysis unit 1001 divides the note string data (performance note string data Q, reference note string data R) into a plurality of unit sections F on the time axis, as shown in FIG. That is, the analysis unit 1001 functions as a sorting unit that sorts the note string data. The unit section F is set to a time length corresponding to one measure of the music, for example. For example, one bar corresponding to the performance tempo specified by the reference note string data R is defined as one unit section F of the reference note string data R. In addition, one measure corresponding to the performance tempo designated by the performer by the operation on the electronic musical instrument 20 is defined as one unit section F of the performance note string data Q. When the performance tempo is not designated, a section corresponding to one measure is set as the unit section F under a predetermined performance tempo and a predetermined time signature. For example, when the performance tempo is assumed to be 120 BPM (Beats Per Minute) and the time signature is assumed to be 4/4 time, the unit interval F is set to 2 seconds.

  The analysis unit 1001 includes a feature amount sequence X (a feature amount sequence XR of the reference note sequence data R and a feature amount sequence XQ of the performance note sequence data Q) that expresses musical features (particularly harmony features) of the note sequence. Is calculated. The feature quantity series XR is calculated for each unit section F of the reference note string data R, and the feature quantity series XQ is calculated for each unit section F of the performance note string data Q.

  The calculation of the feature amount series X will be described with reference to FIG. As shown in FIG. 3, the analysis unit 1001 performs note string data (reference note string data R, performance note string data Q on a coordinate plane in which a time axis (horizontal axis) and a pitch axis (vertical axis) are set. ). As shown in FIG. 3, the feature amount series X (XR, XQ) is a time series of a plurality of feature amounts xm. The feature quantity xm is a 14-dimensional vector in which twelve feature quantities dm [1] to t [12] corresponding to different pitch classes (pitch names), the highest sound TNm, and the lowest sound BNm are arranged. . This pitch class is a set of pitches (note numbers) having a common pitch name. That is, a plurality of pitches whose frequency is a power-of-two relationship (a relationship in which pitch names are different in different octaves) belong to a common pitch class. Features dm [1] to dm [12] for 12 pitch classes corresponding to 12 semitones (C, C #, D, D #, E, F, F #, G, G #, A, A #, B). 12] is calculated.

  Specifically, the feature quantity dm [c] corresponding to the c-th (c = 1 to 12) pitch class is a note belonging to the pitch class among a plurality of notes existing in one unit section F. Is a ratio (dm [c] = τa / τb) between the total value τa of the continuation lengths (tone values) of and the total value τb of the continuation lengths of all notes in the unit interval F. The division by the total value τb is an operation for normalizing the feature quantity dm [c] to a numerical value in the range of 0 to 1 inclusive. For example, as shown in FIG. 3, a note having a pitch G2 (duration 2 seconds) and a note having a pitch G3 (duration 0.2 seconds) having a common pitch class (pitch name G) are included in the unit interval F. (Τa = 2 + 0.2 = 2.2) and the total value τb of the durations of all the notes in the unit section F is 8 seconds, the pitch class (c = 8), the feature quantity dm [8] is 0.275 (= 2.2 / 8).

  The highest note TNm (TN: top note) in the feature amount xm is the pitch (note number) of the highest note among a plurality of notes existing in the m-th unit interval F of the note sequence. Since the pitch G3 is the highest pitch in the unit section F illustrated in FIG. 3, the highest pitch TNm is set to the note number “67” corresponding to the pitch G3. On the other hand, the lowest note BNm (BN: bottom note) in the feature amount xm is the pitch of the lowest note among a plurality of notes existing in the mth unit interval F of the note sequence. Since the pitch C2 is the lowest in the unit section F of FIG. 3, the lowest pitch BNm is set to the note number “48” corresponding to the pitch C2.

  The analysis unit 1001 obtains the similarity of each reference note string data R with respect to the performance note string data Q for each unit section F using the calculated feature amount series X, and creates a similarity matrix. As a method for creating the similarity matrix, the method described in Japanese Patent Application Laid-Open No. 2013-47938, which is the prior application of the present applicant, can be used. For example, when the number of unit intervals F of the performance note string data Q is M and the number of reference note string data R stored in the storage unit 102 is N, as shown in FIG. A row × M column similarity matrix is created.

The analysis unit 1001 applies DP (Dynamic Programming) matching to the calculated similarity matrix and obtains the maximum likelihood path from the first unit section F to the last unit section F, thereby obtaining the unit section F. For each, reference note string data R similar to the performance note string data Q is specified.
Thus, for example, as shown below the similarity matrix in FIG. 4, for each unit section F, it is specified which reference note string data R each unit section F of the performance note string data Q is similar to. Is done. That is, the analysis unit 1001 functions as a specifying unit that specifies the reference note string data R similar to the performance note string data Q. In FIG. 4, for a plurality of reference note string data R, the unit interval F identified as similar to the performance note string data Q is indicated by hatching. For example, FIG. 4 shows that the data in the reference note string data R3 is similar to the data in the third unit interval F from the first unit interval F of the performance note string data Q. Yes.

  When the analysis unit 1001 finishes specifying similar reference note string data R for each unit section F of the performance note string data Q, the analysis unit 1001 divides the performance note string data Q into a plurality of blocks based on the specification result. A tag is assigned to each block.

  Specifically, when similar reference note string data R is the same for consecutive unit sections F, the analysis unit 1001 aggregates MIDI data included in these continuous unit sections F as one section. That is, the analysis unit 1001 functions as an aggregation unit that aggregates the unit sections F. Thereafter, the control unit 101 treats the aggregated sections as one section. For example, as shown in FIG. 4, the first unit section F to the third unit section F of the performance note string data Q are similar to the reference note string data R3 among the plurality of reference note string data R1 to RN. When it is specified that the data is included, the analysis unit 1001 aggregates the MIDI data included in the first unit section F to the third unit section F as one section as shown in the lower part of FIG. Treat aggregated sections as one section. Further, as shown in FIG. 4, when it is specified that the fourth unit interval F and the fifth unit interval F of the performance note string data Q are similar to the reference note string data R2, FIG. As shown in the lower part, the analysis unit 1001 aggregates the MIDI data included in the fourth unit section F and the fifth unit section F as one section, and handles the aggregated section as one section.

  Next, the analysis unit 1001 attaches a tag to each of the MIDI data block that has not been grouped and the MIDI data block that has been grouped. That is, the analysis unit 1001 functions as an adding unit that adds a tag (tag information). In the present embodiment, a song name is assigned as a tag. For example, as shown in FIG. 4, the first unit section F to the third unit section F collected in the performance note string data Q are similar to the reference note string data R3. When the music name included in the reference note string data R3 is “AAA”, the analysis unit 1001 assigns a tag “music name: AAA” to the collected blocks as shown in FIG. To do. The analysis unit 1001 also gives a tag to the section F that is not grouped into the preceding and following sections F.

  In addition, the structure which specifies the music similar to the performance note sequence data Q for every unit area F using the calculated feature-value series X is not limited to the structure mentioned above. For example, the technique proposed in Japanese Patent Application No. 2012-11564, which is the prior application of the present applicant, or the technique disclosed in Japanese Patent Application Laid-Open No. 2013-47938 may be used.

  The display control unit 1002 displays the processing result of the analysis unit 1001 on the display unit 103. FIG. 5 is a diagram illustrating an example of an analysis result displayed on the display unit 103. As shown in FIG. 5, the performance note string data Q is displayed in a band shape with the time axis in the horizontal direction, and each divided block is displayed in a length corresponding to the time length of the block. Further, the elapsed time from the start of reproduction of the performance note string data Q is displayed with respect to the block boundary. This elapsed time is obtained by analyzing the MIDI data. In addition, in each block, a music name that is a tag attached to the block is displayed.

  For example, when the player of the electronic musical instrument 20 practices a plurality of pieces of music in order, and the musical note sequence data Q represents this practice, according to the music analysis apparatus 10 of this embodiment, FIG. As shown, which part of the musical note string data Q corresponds to which musical piece is displayed, the user can easily know the contents of the musical note string data Q without bothering the user.

[Second Embodiment]
Next, a music analysis apparatus according to the second embodiment of the present invention will be described. In the following description, the same components as those in the first embodiment will be omitted as appropriate by using the reference numerals used in the first embodiment.

  In the second embodiment, the processing performed by the analysis unit 1001 is different from the first embodiment. When classifying the performance note string data Q, the analysis unit 1001 detects a period in which the performance is not performed, and classifies the performance note string data Q with the detected period as a boundary. Note that the analysis unit 1001 sets a period in which the period from note-off to note-on in the MIDI data included in the performance note string data Q is equal to or longer than a predetermined period as a period during which no performance is performed.

  FIG. 6 is a diagram showing an example of division of the performance note string data Q. The analysis unit 1001 expands the performance note string data Q on the time axis, and when there are a plurality of periods in which the performance note string data Q is not played, the performance note string data Q is converted into a plurality of pieces as shown in FIG. The musical note string data Q1 to QM are divided. FIG. 6 shows a period during which the band-like portion is played.

  The analysis unit 1001 further divides the performance note string data Q1 to QM into unit intervals F. The analysis unit 1001 makes the positions of the performance note string data Q1 to QM different from each other by a plurality of times on the time axis with respect to the plurality of reference note string data R. Base values x [b] (x [1] to x [B]) are calculated (b = 1 to B).

  FIG. 7 is a diagram showing an operation example when the basic value x [b] is calculated between the performance note string data Q1 and a certain reference note string data R. The analysis unit 1001 calculates a base value x [b] for each case where the performance note string data Q1 is moved (shifted) by one unit interval F with respect to the reference note string data R. That is, for each case where the position of the performance note string data Q1 is different, the analysis unit 1001 performs a target section σ [b] (of K unit sections F corresponding to the performance note string data Q1 in the reference note string data R). Set) is selected sequentially, and the performance note string data Q1 and the target section σ [b] are compared to calculate the basic value x [b].

Here, a specific operation for calculating the basic value x [b] will be described with the performance note string data Q1 as an example. The analysis unit 1001 uses the similarity V [k] (V [1] to V [] for each unit section F corresponding to the target section σ [b] of the performance note string data Q1 and the reference note string data R. K]) is calculated (k = 1 to K). Specifically, the analysis unit 1001 regards each of the performance note string data Q1 and the reference note string data R as a time series (character string) of characters corresponding to each note, and kth in the performance note string data Q1. The degree of similarity V [k] is calculated according to the editing distance (Levenstein distance) between the th unit section F and the kth unit section F in the target section σ [b]. For the calculation of the edit distance, for example, the technique proposed in Japanese Patent Application No. 2012-1115064 which is the prior application of the present applicant is used.
The analysis unit 1001 calculates a base value x [b from the K similarity degrees V [1] to V [K] calculated for different unit sections F of one target section σ [b] in the reference note string data R. ] Is calculated. For example, an average value or a maximum value of K similarity degrees V [1] to V [K] is calculated as the basic value x [b].

  When the analysis unit 1001 calculates the basic value x [b] every time the performance note string data Q1 is moved by one unit interval F with respect to the reference note string data R, the analysis unit 1001 differs in the reference note string data R. The similarity index value X [n] is calculated according to the B basic values x [1] to x [B] corresponding to the target section σ [b]. Specifically, the maximum value among the B basic values x [1] to x [B] (that is, a numerical value indicating the maximum similarity between the reference note string data R and the performance note string data Q1) is similar. It is selected as the index value X [n].

  The analysis unit 1001 obtains the similarity index value X [n] by the above-described operation between the performance note string data Q1 (performance data) and the plurality of reference note string data R stored in the storage unit 102. Calculate. The similarity index value X [n] calculated for each of the plurality of reference note string data R becomes larger as the reference note string data R including a section similar to the note arrangement of the performance note string data Q1. The analysis unit 1001 specifies reference note string data R (music data) similar to the performance note string data Q1 based on the calculated similarity index value X [n]. The analysis unit 1001 extracts the song name included in the reference note string data R having the maximum similarity index value X [n], and assigns the extracted song name as a tag to the performance note string data Q1. .

The analysis unit 1001 calculates the similarity index value X [n] for each of the performance note string data Q2 to QM as well as the performance note string data Q1, and the similarity index value X [n] becomes the maximum value. The music name included in the reference note string data R is assigned as a tag.
When a tag is assigned to each of the performance note string data Q1 to QM, the display control unit 1002 displays the processing result of the analysis unit 1001 on the display unit 103 as in the first embodiment, for example. The analysis unit 1001 combines the MIDI data included in the continuous performance note string data Q1 to QM into one when the similar reference note string data R is the same in the continuous performance note string data Q1 to QM. You may make it collect.

  According to the music analysis device 10 of the second embodiment, which part of the performance note string data Q corresponds to which music is displayed as in the first embodiment, the user can perform the performance without bothering the hand. The contents of the note string data Q can be easily known.

[Modification]
As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, It can implement with another various form. For example, the present invention may be implemented by modifying the above-described embodiment as follows. In addition, you may combine each of embodiment mentioned above and the following modifications.

In the embodiment described above, the music analysis apparatus 10 acquires the performance note string data Q from the electronic musical instrument 20, but is not limited to this configuration. For example, when the music analysis device 10 is a server device connected to a network, the musical note string data Q is sent from the electronic musical instrument 20 to the server device, and the server device performs the performance. The musical note string data Q may be analyzed.
Further, a plurality of reference note string data R (music data) may be stored in the server device 20, and the music analysis device 10 may acquire a plurality of reference note string data R from the server device. In this case, the control unit 101 acquires reference note string data via the communication unit 105. That is, the control unit 101 functions as music data acquisition means for acquiring music data.

In the above-described embodiment, as a result of analysis, the performance note string data Q may be divided into a plurality of blocks, and then the performance note string data Q may be divided into a plurality of files for each divided block. .
Further, in the above-described embodiment, when the performance note string data Q that has been analyzed and has been tagged is reproduced, the tag, that is, the music title may be displayed.

  In the first embodiment described above, the reference note string data R similar to the performance note string data Q is specified by DP matching. However, similar reference note string data R is determined only by the similarity without performing DP matching. May be specified.

  In the embodiment described above, the reference note string data R similar to the performance note string data Q is specified. However, it is possible to specify which part of the reference note string data R is similar. . Further, in the configuration for specifying which parts of the reference note string data R are similar, statistical processing is performed on a plurality of performance note string data Q, and the number of times the performance is performed for each specified part is calculated. You may make it measure and display the frequency | count of performance. Thus, according to the structure which displays the frequency | count of performance, for example, the frequency | count of practice when performance of practice is performed will be displayed, and it can utilize for the learning of performance by analyzing the frequency | count of practice.

  In the second embodiment described above, the performance note string data Q is divided into a plurality of sections Q1 to QM in a period in which the performance is not performed. However, the present invention is not limited to this configuration. For example, a performer's voice may be acquired by a microphone together with MIDI data, and the performance note string data Q may be divided at a certain timing. In addition, when the music analysis device 10 is a smartphone or a tablet terminal, data indicating the timing at which the screen is tapped is included in the performance note string data Q, and the performance note string data Q is divided at the timing at which the screen is tapped. Also good. Further, the player is photographed with a camera of a smartphone or a tablet terminal, and data indicating the timing at which a specific display is photographed is included in the performance note string data Q, and the performance note string data Q is divided at the timing. May be. Alternatively, a specific note string may be determined in advance, and the performance note string data Q may be divided where this note string exists. According to this configuration, the performance note string data Q can be segmented simply by playing a specific note string.

  In the above-described embodiment, each block of the divided performance note string data Q may be scored for performance in comparison with the reference note string data R identified as similar. In the configuration in which the performance note string data Q is scored, the performance note string data Q having a high scoring result may be stored in a server device connected to the network. In the above-described embodiment, the analysis result of the plurality of musical note string data Q is displayed, the divided blocks are edited, the blocks are combined among the plural musical note string data Q, and a new performance is recorded. Note string data Q may be created.

  In the embodiment described above, the content given as a tag is a song name, but is not limited to a song name. For example, various types of information specified along with the specification of the music, such as the name of the music composer, the name of the composer, the composition date of the music, and the genre of the music, may be added as tags. Information (A melody, rust, etc.) indicating which part of the music may be given. Moreover, in the configuration in which the number of performances is measured as in the above-described modification, the number of performances may be assigned as a tag. In the configuration in which scoring is performed as in the above-described modification, the scoring result is assigned as a tag. May be. Further, performance date information may be given as a tag.

  In the embodiment described above, the performance note string data Q and the reference note string data R are MIDI data, but are not limited to MIDI data. For example, it may be a spectrum of a performance sound waveform.

DESCRIPTION OF SYMBOLS 10 ... Music analysis apparatus, 20 ... Electronic musical instrument, 101 ... Control part, 102 ... Memory | storage part, 103 ... Display part, 104 ... Operation part, 105 ... Communication part, 1001 ... Analysis part, 1002 ... Display control part

Claims (3)

Performance data acquisition means for acquiring performance data indicating performance;
Classifying means for dividing the performance data into a plurality of sections on the time axis,
Music data acquisition means for acquiring a plurality of music data indicating music;
A specifying means for specifying music data similar to the performance data of the section for each of the plurality of sections based on the performance data sorted by the sorting means and the plurality of music data obtained by the music data obtaining means. When,
In the plurality of sections, when sections having the same identification result of the specifying unit are continuous, an aggregation unit that aggregates the continuous sections into one section;
A music analysis apparatus comprising: an adding means for adding tag information indicating music data identified as similar to each of the aggregated sections and sections not aggregated by the aggregation means. The music analysis apparatus according to claim 1, wherein the sorting unit sorts the performance data by a period in which performance is not performed. The music analysis apparatus according to claim 1, further comprising a dividing unit that divides the performance data for each of the aggregated sections and sections not aggregated by the aggregation unit.

Images