CN113744764B - Method for obtaining optimal comparison path of performance time value information and score time value information - Google Patents
Method for obtaining optimal comparison path of performance time value information and score time value information Download PDFInfo
- Publication number
- CN113744764B CN113744764B CN202111036222.5A CN202111036222A CN113744764B CN 113744764 B CN113744764 B CN 113744764B CN 202111036222 A CN202111036222 A CN 202111036222A CN 113744764 B CN113744764 B CN 113744764B
- Authority
- CN
- China
- Prior art keywords
- matrix
- path
- information
- time value
- performance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 239000011159 matrix material Substances 0.000 claims abstract description 107
- 230000006870 function Effects 0.000 claims abstract description 23
- 238000006243 chemical reaction Methods 0.000 claims abstract description 16
- 230000008569 process Effects 0.000 claims abstract description 15
- 239000013598 vector Substances 0.000 claims description 47
- 238000001228 spectrum Methods 0.000 claims description 12
- 238000003780 insertion Methods 0.000 claims description 9
- 230000037431 insertion Effects 0.000 claims description 9
- 238000012217 deletion Methods 0.000 claims description 7
- 230000037430 deletion Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000011156 evaluation Methods 0.000 abstract description 56
- 230000033764 rhythmic process Effects 0.000 abstract description 6
- 239000011295 pitch Substances 0.000 description 84
- 238000000605 extraction Methods 0.000 description 16
- 238000004364 calculation method Methods 0.000 description 9
- 238000012549 training Methods 0.000 description 8
- 238000012163 sequencing technique Methods 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000011218 segmentation Effects 0.000 description 5
- 238000010276 construction Methods 0.000 description 4
- 239000000284 extract Substances 0.000 description 4
- 230000008859 change Effects 0.000 description 2
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/90—Pitch determination of speech signals
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/076—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for extraction of timing, tempo; Beat detection
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/90—Pitch determination of speech signals
- G10L2025/906—Pitch tracking
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Auxiliary Devices For Music (AREA)
Abstract
The invention discloses a method for obtaining an optimal comparison path of performance time value information and score time value information, which mainly solves the problems that the prior evaluation system and method in the prior art mechanically require the matching of actual performance data and standard performance data, neglect the non-fluency of the actual performance process, lack the evaluation of fine technical points such as voice region errors, snap, drags and beats, unstable rhythm and the like, and cause lower performance evaluation. The invention obtains the playing sound information and the music score information; then, respectively performing code conversion on the performance sound information and the music score information to generate a matrix; then inputting the matrix into a corresponding comparison function to generate a comparison matrix; then planning a path according to the comparison matrix, and producing an optimal path; then, the matrix generated first is segmented according to the position index of the optimal comparison path; and finally, evaluating according to the evaluation dictionary and outputting an evaluation language. Through the scheme, the aim of detail comment can be achieved.
Description
Technical Field
The invention relates to the technical field of music analysis, in particular to a method for obtaining an optimal comparison path of performance time value information and score time value information.
Background
Currently, performance evaluation methods for musical instruments are generally based on systems for additionally evaluating performance states on electronic musical instruments; the method and system mainly store standard performance data in an electronic musical instrument or electronic musical instrument metadata, and perform performance evaluation by comparing the standard performance data with actual performance data.
The method has a plurality of barriers in musical instrument playing training and playing evaluation suitable for music education, and mainly comprises the following steps: highly dependent on electronic musical instruments or electronic musical instrument accessories, but most of music education adopts non-electronic musical instruments; in the process of performing performance training, a musical instrument learner needs to gradually improve the performance technology, and the requirement of standard performance data is generally difficult to directly reach, while the method generally mechanically requires the matching of actual performance data and standard performance data, neglects the non-fluency of the actual performance process, and thus causes lower performance evaluation; the core purpose of performance training of a learner of a musical instrument is to improve performance technology, and the above evaluation method generally involves only overall evaluation, lacks evaluation of fine technical points such as a voice region error, snap, a trawling, a rhythm instability, and the like, and cannot well assist performance training of the learner of a musical instrument by adopting the above technology.
Disclosure of Invention
The invention aims to provide a method for obtaining an optimal comparison path of performance time value information and score time value information, so as to solve the problem that the conventional evaluation system and method mechanically require the matching of actual performance data and standard performance data, neglect the non-fluency of the actual performance process, lack the evaluation of fine technical points such as voice region errors, snap, drags and beats, unstable rhythm and the like, and cause lower performance evaluation.
In order to solve the problems, the invention provides the following technical scheme:
The method for obtaining the optimal comparison path of the performance pitch information and the melody pitch information comprises the following steps:
(A1) Acquiring pitch information of a performance sound and pitch information of a melody;
(A2) Performing code conversion on the pitch information of the performance sound and the pitch information of the melody respectively to generate a pitch matrix;
(A3) Inputting the pitch matrix of (A2) into a pitch comparison function to generate a pitch comparison matrix;
(A4) And (3) carrying out path planning on the pitch comparison matrix of the step (A3), and generating a pitch minimum score path as an optimal path.
Specifically, the PITCH information of the melody in step (A1) is taken from the information of < NOTE < PITCH < stem > < OCTAVE > > tag in the melody file in the extensible markup format (XML/MusicXML).
Specifically, in step (A2), the pitch information is code-converted into a two-dimensional boolean matrix of 128×n, the column coordinates represent the position index of each of the played notes in the pitch sequence, the row coordinates represent the position indexes of 128 semitone notes of absolute pitches C-1 to G9 in scientific notation, the element values in the matrix are represented by 0 or 1, 0 represents not playing the corresponding notes, and 1 represents playing the corresponding notes.
Specifically, the pitch comparison function in step (A3) is a composite function including a pitch scoring function and a pitch matching function;
the pitch scoring function is:
The pitch matching function is:
wherein, ；
A j represents the column vector for converting the pitch information of the sound into a pitch matrix in step (A2);
x i represents the column vector of converting the cursive pitch information into a pitch matrix in step (A2);
If the expression x i＝Xi*aj is satisfied between the ith column of the music spectrum pitch matrix and the jth column of the music spectrum pitch matrix, judging that the two columns are matched, assigning the (i+1, j+1) position of the scoring matrix as a matching score m 1, otherwise, assigning the (i+1, j+1) position of the scoring matrix as a matching score u 1;
g 1 represents the insertion score of g 1 for each bit of representation moved in the pitch matrix from left to right and top to bottom, introducing an insertion operation.
Specifically, the path planning in the step (A4) is minimum score path dynamic planning based on position indexes, and the specific process is that the position indexes of the music notes and the sound notes are shifted by 1 positive unit integrally through position index transformation, 0 is used for representing insertion or deletion, and a position index (r, c) sequence of the optimal path is generated according to a minimum score index function;
Wherein i is the ith column of the cursive pitch matrix; j is the j-th column of the pitch matrix of the sound; let R sequence be the set of all R; the C sequence is that a set r of all C represents a certain row coordinate or 0 in the matching path, C represents a certain column coordinate or 0 in the matching path, wherein 0 represents a placeholder and is used for representing insertion and deletion; the sequence R represents a matching sequence of sound information and consists of row coordinates and 0; sequence C represents a matching sequence of the cursive information, consisting of column coordinates and 0.
Specifically, the method for obtaining the optimal comparison path of the performance duration information and the score duration information comprises the following steps:
(B1) Acquiring performance sound duration information and melody duration information;
(B2) Performing code conversion on the (B1) performance sound time value information and the music spectrum time value information respectively to generate a time value matrix;
(B3) Inputting the timing value matrix of the step (B2) into a timing value comparison function to generate a timing value comparison matrix;
(B4) And (3) carrying out path planning according to the timing value comparison matrix of the step (B3), and generating a timing value minimum variance path as an optimal path.
Specifically, the music score value information in step (B1) is taken from the < NOTE < DURATION > > tagged information in the music score file in the extensible markup format (XML/MusicXML).
Specifically, in step (B2), the time value information is code-converted into a two-dimensional numerical matrix 1*n, the column coordinates represent the position index of each performance note in the time value sequence, and the element values in the matrix are represented by integer numerical values and represent the number of time frames.
Specifically, the value comparison function in step (B3) is:
Wherein, the matrix of the curvelet time value is Sound duration matrix is/>Wherein y i represents the ith column vector of the curvelet matrix and b j represents the jth column vector of the sound value matrix; dura_s is the duration alignment matrix.
Specifically, the path planning in the step (B4) is a minimum variance path dynamic planning based on the position index, and the specific process is that the alignment matrix of the timing values is traversed by a directed graph, and the path planning from DURA_S (0,0) to DURA_S (0,0) is acquiredCalculating the variance of each path to obtain a minimum variance path, and converting the minimum variance path into a (r, c) sequence based on a position index, wherein 0 represents insertion or deletion, and shifting the whole of the position indexes of the music notes and the sound notes by 1 positive unit through position index conversion;
Where the duration of each position index (i, j) of DURA_S path(i,j) deviates, E (DURA_S path) represents the average of the duration deviations of the entire path, and N represents the number of path positions.
A comparison method of performance sound information and music score information includes the following steps:
(C1) Obtaining an optimal comparison path of the pitch information according to an obtaining method of the optimal comparison path of the performance pitch information and the melody pitch information;
(C2) Obtaining an optimal comparison path of the time value information according to an obtaining method of the optimal comparison path of the performance time value information and the score time value information;
(C3) Obtaining an optimal comparison path according to the comparison paths in the step (C1) (C2);
(C4) Cutting the pitch matrix and the duration matrix of the step (B2) according to the position index of the optimal comparison path of the step (C3) to generate a section, phrase and paragraph matrix;
(C5) Constructing evaluation rules of different levels in advance to generate an evaluation dictionary;
(C6) And (3) generating evaluation words of different matrixes according to the different matrixes generated in the step (C4) and the evaluation dictionary in the step (C5).
Specifically, the evaluation word dictionary in the step (C5) comprises evaluation words with four evaluation levels of pitch, rhythm and fluency, notes, phrases, music pieces and music pieces; the evaluation word includes: region error, snap, drag beat, rhythm instability, temporary change sign error, pitch error, mispronounce, snap, drag beat, rhythm instability, cross-bar incoherence, phrase incoherence.
Specifically, the specific process of step (C3) is: the path sequencing value rank=p×w, P is an attribute sequence, W is a weight sequence, and the path corresponding to the rank maximum value is the optimal path; adding R, C in step (A4) to sequences R and C, respectively, wherein the eight attribute values of attribute sequence P are in turn the number proportion of element 0 in sequence R, the number proportion of element 0 in sequence C, the relative start position of element 0 in sequence R, the relative start position of element 0 in sequence C, the relative end position of element 0 in sequence R, the relative end position of element 0 in sequence C, the maximum continuous number proportion of element 0 in sequence R, the maximum continuous number proportion of element 0 in sequence C, and the weight sequence w=w 1w2w3w4w5w6w7w8.
Specifically, the specific process of step (C4) is: the method comprises the steps of splitting into position indexes of optimal comparison paths generated in step (C3) according to position indexes of a music spectrum file < measurement > tag of extensible markup (XML/musicXML), pre-constructed phrase position indexes and paragraph position indexes, generating splitting marks, and splitting matrixes in step (A2) and step (B2) into sections, phrases and paragraph matrixes respectively according to the splitting marks.
The comparison system of the performance sound information and the music score information comprises an information extraction module, a vector conversion module, a comparison matrix calculation module, a path planning module and a performance evaluation module which are connected in sequence.
Specifically, the information extraction module comprises an audio receiving module and a stored music score; the audio receiving module and the stored music score are respectively connected with the time value information extracting module and the pitch information extracting module;
the vector conversion module comprises a time value vector conversion module and a pitch vector conversion module which are respectively connected with the time value information extraction module and the pitch information extraction module;
The comparison matrix calculation module comprises a time value comparison matrix calculation module and a pitch comparison matrix calculation module which are respectively connected with the time value vector conversion module and the pitch vector conversion module;
the path planning module comprises a sequencing module, a time value minimum variance path planning module and a pitch minimum score path planning module which are respectively connected with the time value comparison matrix calculation module and the pitch comparison matrix calculation module; the pitch minimum score path planning module is sequentially connected with the time value minimum variance path planning module and the sequencing module;
The performance evaluation module comprises a segmentation module and an evaluation module which are connected with the sequencing module;
The sound value information extraction module and the sound height information extraction module are respectively used for extracting sound value information and sound height information of the music score and the sound received by the sound receiving module;
The time value vector conversion module and the sound height vector conversion module are respectively used for converting pitch information and time value information in the melody and the sound into two-dimensional matrixes with corresponding coding formats;
The time value comparison matrix calculation module and the pitch comparison matrix calculation module are respectively used for calculating a pitch comparison matrix and a time value comparison matrix according to the comparison function;
The time value minimum variance path planning module and the pitch minimum score path planning module are used for carrying out path dynamic planning and sorting on the comparison matrix to obtain an optimal path;
the segmentation module is used for segmenting the pitch matrix and the duration matrix into the section, phrase and paragraph matrix.
Specifically, the sorting value in the sorting module is the product of the attribute sequence P and the weight sequence W, where the eight attribute values of the attribute sequence P are in turn the number proportion of the elements 0 in the sequence R, the number proportion of the elements 0 in the sequence C, the relative start position of the elements 0 in the sequence R, the relative start position of the elements 0 in the sequence C, the relative end position of the elements 0 in the sequence R, the relative end position of the elements 0 in the sequence C, the maximum continuous number proportion of the elements 0 in the sequence R, the maximum continuous number proportion of the elements 0 in the sequence C, and the weight sequence w=w 1w2w3w4w5w6w7w8.
A computer device comprises a processor and a memory in which a computer program is stored which, when loaded and executed by the processor, implements the method of obtaining.
A computer readable medium has a computer program stored therein, the computer program being loaded and executed by a processor to implement the obtaining method.
Compared with the prior art, the invention has the following beneficial effects:
(1) The invention breaks away from dependence on electronic musical instruments and electronic musical instrument accessories by means of voice recognition analysis, and enlarges the range of musical instrument performance evaluation; the method fully considers the unsmooth of a musical instrument learner in the performance training process, matches the actual performance data with the standard performance data through a comparison algorithm, and has more accurate and flexible evaluation results; the evaluation word dictionary is constructed, the evaluation level is more abundant, the evaluation of the performance technology is not limited to overall evaluation, such as overall scoring, and the evaluation granularity can reach the evaluation of the performance technology point of a single performance note, so that the performance training is better assisted.
(2) The invention constructs a comparison function converted into a matrix by coding and based on column vectors (position index vectors) and a dynamic programming algorithm based on the position index on the basis of the position index, can be used for realizing d-dimensional (d is more than or equal to 2) information comparison and generating a comparison path based on the position index, and expands from one-dimensional character string comparison to multi-dimensional matrix comparison.
(3) The invention adopts the playing sound as the input source, thereby eliminating the dependence on the electronic musical instrument and the electronic musical instrument elements in the current playing evaluation system; aiming at the problem that the instrument learner frequently has unsmooth problems such as rebound, sound leakage and the like in the performance training process, the comparison concepts such as 'insert' and 'delete' are introduced, and the actual performance of the instrument learner is identified through a comparison algorithm; the invention realizes multi-level and multi-dimensional performance evaluation by constructing evaluation rules and evaluation word dictionaries of five levels of notes, bars, phrases, paragraphs, tracks and the like and different performance dimensions of pitch, time value, smoothness, strength and the like.
Drawings
Fig. 1 is a block diagram of an alignment system.
Fig. 2 is an example of a coding scheme of a pitch vector matrix.
Fig. 3 is an example of a section and phrase cut.
Fig. 4 is an example of an evaluation dictionary.
Detailed Description
The invention will now be further described with reference to the accompanying drawings and examples, embodiments of which include, but are not limited to, the following examples.
As shown in fig. 1 to 4, the audio receiving module of the present invention is configured to receive performance sounds; the invention obtains the playing sound information and the music score information; then, respectively performing code conversion on the performance sound information and the music score information to generate a matrix; then inputting the matrix into a corresponding comparison function to generate a comparison matrix; then planning a path according to the comparison matrix, and producing an optimal path; then, the first generated matrix is segmented according to the position index of the optimal comparison path to produce a bar, phrase and paragraph matrix; finally, evaluating the bar, phrase and paragraph matrix according to the evaluation dictionary and outputting an evaluation language; the specific implementation process is as follows:
The PITCH information extraction module extracts and stores PITCH information in a music spectrum, the PITCH information being taken from < NOTE < PITCH < STEPS > < OCTAVE > > tagged information in a music spectrum file in an extensible markup format (XML/musicXML).
The [ M112 ] pitch information extraction module extracts and stores pitch information in sound, and the pitch information extraction refers to the patent application number as follows: 201910669985.X, patent name: a music analysis data set construction method and an extraction method in a pitch and duration value extraction method based on the music analysis data set construction method are used for obtaining pitch information.
The [ M121 ] value information extraction module extracts and stores value information in a music spectrum, the value information being taken from information of < NOTE < DURATION > > tag in a music spectrum file in an extensible markup format (XML/MusicXML).
The [ M122 ] timing value information extraction module extracts and stores timing value information in sound, and the timing value information extraction refers to the patent application number as follows: 201910669985.X, patent name: a music analysis data set construction method and an extraction method in a pitch and duration extraction method based on the music analysis data set construction method are used for obtaining duration information.
The pitch information in [ M211 ] is converted into a (128 x n 1) two-dimensional Boolean vector matrix. Wherein the column coordinates represent the position index of each performing note in the pitch sequence, the row coordinates represent 128 semitone notes representing absolute pitches C-1 through G9 in scientific notation, the value of each position in the matrix is represented by 0/1, 0 represents not performing the corresponding note, and 1 represents performing the corresponding note.
Converting the pitch information in [ M212 ] into a (128 x n 2) two-dimensional Boolean vector matrix; the two-dimensional boolean vector matrix is represented by [ M211 ].
Converting the time value information in the M221 into a 1*n 1 two-dimensional numerical vector matrix; wherein the column coordinates represent the position index of each performance note in the pitch sequence.
Converting the time value information in the M223 into a 1*n 2 two-dimensional numerical vector matrix; the two-dimensional numerical vector matrix is represented by [ M221 ].
The PITCH score matrix pitch_s is constructed, initialized to a value of 0, and shaped ((n 1+1)*(n2 +1)).
Inputting the matrix in the M211 and the M212, and updating the scoring matrix S obtained by the M311 according to a scoring function; wherein the scoring function is described below.
Setting the pitch matrix of the cursive spectrum asThe pitch matrix of the sound is/>Where x i represents the column vector of the ith column of the spectrum pitch matrix, a j represents the column vector of the jth column of the sound pitch matrix:
The PITCH comparison matrix pitch_m is constructed, initialized to a value of 0 and shaped ((n 1+1)*(n2 +1)).
The scoring matrix of the M314 is taken as input, and the PITCH comparison matrix PITCH_M obtained by the M313 is updated according to a matching function. The matching function is described as follows:
the value comparison matrix DURA_M is constructed [ M321 ], the value is initialized to 0, and the shape is ((n 1+1)*(n2 +1)).
The matrix in the M322 is input, and the timing value comparison matrix DURA_M obtained by updating the M321 according to the comparison function is input. Wherein the alignment function is described as follows:
Setting the matrix of the curvespectral time value as Sound duration matrix is/>Where y i represents the ith column vector of the curvelet matrix and b j represents the jth column vector of the sound value matrix, then: /(I)
Calculating to obtain the best score paths in the pitch comparison matrix of [ M314 ], wherein the number of the best score paths is more than or equal to 1; the calculation rules are described as follows:
traversing elements in a PITCH comparison matrix PITCH_M, generating R and C according to iteration and assignment rules, and adding R and C into sequences R and C respectively, wherein the iteration and assignment rules are as follows, and 0 represents insertion or deletion and is used for representing multi-performance notes or few-performance notes in the actual performance process:
inputting a plurality of best score paths (M411) and a score duration matrix of (M322), calculating duration variance of each path, and selecting a path with the smallest variance as a minimum variance path, wherein the number of the minimum variance paths is more than or equal to 1:
Where the duration of each position index (i, j) of DURA_S path(i,j) deviates, E (DURA_S path) represents the average of the duration deviations of the entire path, and N represents the number of path positions.
The eight attribute values of the multiple minimum variance paths in the sequence [ M421 ] are respectively obtained and marked as P, and the eight attribute values are sequentially the number proportion of the elements 0 in the sequence R, the number proportion of the elements 0 in the sequence C, the relative starting position of the elements 0 in the sequence R, the relative starting position of the elements 0 in the sequence C, the relative ending position of the elements 0 in the sequence R, the relative ending position of the elements 0 in the sequence C, the maximum continuous number proportion of the elements 0 in the sequence R and the maximum continuous number proportion of the elements 0 in the sequence C.
Inputting eight attribute values of [ M431 ] into a sequencing model for sequencing respectively to obtain an optimal path; the sorting model is that the distribution of the element 0 in the optimal path obeys a certain rule, a path sorting value rank=p×w is calculated, and the path corresponding to the maximum rank is the optimal path; w is the weight sequence w=w iw2w3w4w5w6w7w8.
The method comprises the steps of dividing a vector matrix into section vector matrixes according to an optimal path sequence, wherein dividing rules are that section initial note position indexes are extracted from a music spectrum file < MEASURE > tag in an extensible tag format and are matched with position indexes of an optimal comparison path, section dividing tags are generated, and pitch vector matrixes and duration vector matrixes are divided into section pitch vector matrixes and section duration vector matrixes respectively according to the section dividing tags.
Dividing the vector matrix into phrase vector matrixes according to the optimal path sequence, wherein the dividing rule is that the initial note position index of a phrase is extracted from the pre-constructed phrase position index and is matched with the position index of the optimal comparison path, phrase dividing marks are generated, and the pitch vector matrixes and the duration vector matrixes are respectively divided into phrase pitch vector matrixes and phrase duration vector matrixes according to the phrase dividing marks;
the vector matrix is segmented into paragraph vector matrices according to the optimal path sequence, wherein the segmentation rule is that the paragraph starting note position index is extracted from the pre-constructed paragraph position index and matched with the position index of the optimal comparison path, a paragraph segmentation mark is generated, and the pitch vector matrix and the time value vector matrix are segmented into a paragraph pitch vector matrix and a paragraph time value vector matrix respectively according to the paragraph segmentation mark.
And (3) evaluating each data tuple of the step (M521) according to an evaluation rule, and outputting a corresponding position index.
And (M522) matching and selecting the evaluation words from a pre-constructed evaluation word dictionary, and outputting corresponding evaluation words. Wherein the evaluation word dictionary comprises evaluation words with multiple dimensions of pitch, duration, fluency and the like.
The invention acquires performance sound information and standard music score information; respectively encoding and converting the performance sound information and the standard music score information into vector matrixes; respectively inputting the vector matrix into a comparison function to generate a comparison matrix; generating an optimal comparison path by adopting a dynamic programming algorithm, a sequencing model and the like; the evaluation of the performance sound information and the score information includes: constructing an evaluation word dictionary; dividing the optimal comparison path into a multi-level evaluation sequence; and generating evaluation information of different layers of sequences according to the evaluation rule.
The specific process of path planning is that the position indexes of the music notes and the sound notes are shifted by 1 positive unit integrally through position index conversion, and 0 is used for representing insertion or deletion; if the whole is changed from 0 to 8 to 1 to 9, 0 is introduced if insertion or deletion is needed, for example 102 34 500 6 7 8 9 is the result after the change, which represents that a space needs to be inserted after the first position and 2 spaces are inserted after the 5 th position.
The invention adopts the playing sound as the input source, thereby eliminating the dependence on the electronic musical instrument and the electronic musical instrument elements in the current playing evaluation system; aiming at the problem that the instrument learner frequently has unsmooth problems such as rebound, sound leakage and the like in the performance training process, the comparison concepts such as 'insert' and 'delete' are introduced, and the actual performance of the instrument learner is identified through a comparison algorithm; the invention realizes multi-level and multi-dimensional performance evaluation by constructing evaluation rules and evaluation word dictionaries of five levels of notes, bars, phrases, paragraphs, tracks and the like and different performance dimensions of pitch, time value, smoothness, strength and the like.
The present invention can be well implemented according to the above-described embodiments. It should be noted that, based on the above structural design, even if some insubstantial modifications or color-rendering are made on the present invention, the essence of the adopted technical solution is still the same as the present invention, so it should be within the protection scope of the present invention.
Claims (5)
1. The method for obtaining the optimal comparison path of the performance time value information and the score time value information is characterized by comprising the following steps:
(B1) Acquiring performance sound duration information and melody duration information;
(B2) Performing code conversion on the (B1) performance sound time value information and the music spectrum time value information respectively to generate a time value matrix;
(B3) Inputting the timing value matrix of the step (B2) into a timing value comparison function to generate a timing value comparison matrix;
The value comparison function in step (B3) is: dura_s (i+1,j+1)＝bj/yi;
Wherein, the matrix of the curvelet time value is Sound duration matrix is/>Wherein y i represents the ith column vector of the curvelet matrix, and b j represents the jth column vector of the sound value matrix; dura_s is a timing value comparison matrix;
(B4) Performing path planning according to the timing value comparison matrix of the step (B3), and generating a timing value minimum variance path as an optimal path;
The path planning in the step (B4) is the minimum variance path dynamic planning based on the position index, and the specific process is that the directed graph traversal is performed on the value comparison matrix, and the distance from DURA_S (0, 0) to DURA_S (0) is obtained Calculating the variance of each path to obtain a minimum variance path, and converting the minimum variance path into a (r, c) sequence based on a position index, wherein 0 represents insertion or deletion, and shifting the whole of the position indexes of the music notes and the sound notes by 1 positive unit through position index conversion;
wherein the duration of each position index (i, j) of DURA_S path (i, j) deviates, E (DURA_S path) represents the average of the duration deviations of the entire path, and N represents the number of path positions.
2. The obtaining method according to claim 1, wherein the music score value information in the step (B1) is taken from information of < NOTE < DURATION > > flag in a music score file in a scalable flag format.
3. The obtaining method according to claim 1, wherein the time value information is code-converted into a two-dimensional numerical matrix 1*n in step (B2), the column coordinates representing the position index of each performance note in the time value sequence, and the element values in the matrix are represented by integer numbers representing the number of time frames.
4. A computer device comprising a processor and a memory, in which a computer program is stored which, when loaded and executed by the processor, implements the obtaining method according to claims 1-3.
5. A computer readable medium, characterized in that a computer program is stored in the computer readable medium, which computer program is loaded and executed by a processor to implement the obtaining method according to claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111036222.5A CN113744764B (en) | 2019-09-02 | 2019-09-02 | Method for obtaining optimal comparison path of performance time value information and score time value information |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910806968.6A CN110600057B (en) | 2019-09-02 | 2019-09-02 | Method and system for comparing performance sound information with music score information |
| CN202111036222.5A CN113744764B (en) | 2019-09-02 | 2019-09-02 | Method for obtaining optimal comparison path of performance time value information and score time value information |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910806968.6A Division CN110600057B (en) | 2019-09-02 | 2019-09-02 | Method and system for comparing performance sound information with music score information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113744764A CN113744764A (en) | 2021-12-03 |
| CN113744764B true CN113744764B (en) | 2024-04-26 |
Family
ID=68856284
Family Applications (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910806968.6A Active CN110600057B (en) | 2019-09-02 | 2019-09-02 | Method and system for comparing performance sound information with music score information |
| CN202111041068.0A Active CN113763991B (en) | 2019-09-02 | 2019-09-02 | Comparison method and system of performance sound information and music score information |
| CN202111036222.5A Active CN113744764B (en) | 2019-09-02 | 2019-09-02 | Method for obtaining optimal comparison path of performance time value information and score time value information |
Family Applications Before (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910806968.6A Active CN110600057B (en) | 2019-09-02 | 2019-09-02 | Method and system for comparing performance sound information with music score information |
| CN202111041068.0A Active CN113763991B (en) | 2019-09-02 | 2019-09-02 | Comparison method and system of performance sound information and music score information |
Country Status (1)
| Country | Link |
|---|---|
| CN (3) | CN110600057B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110600057B (en) * | 2019-09-02 | 2021-12-10 | 深圳市平均律科技有限公司 | Method and system for comparing performance sound information with music score information |
| CN113643676A (en) * | 2020-04-27 | 2021-11-12 | 汲趣艺术科技(上海)有限公司 | Performance evaluation system |
| CN111554257A (en) * | 2020-05-07 | 2020-08-18 | 南京邮电大学 | Note comparison system of traditional Chinese national musical instrument and use method thereof |
| CN112258932B (en) * | 2020-11-04 | 2022-07-19 | 深圳市平均律科技有限公司 | Auxiliary exercise device, method and system for musical instrument playing |
| CN112836080B (en) * | 2021-02-05 | 2023-09-12 | 小叶子（北京）科技有限公司 | Method and system for searching music score through audio |
| CN114417915A (en) * | 2021-12-29 | 2022-04-29 | 星花怒放(苏州)科技有限公司 | Two-dimensional sequence similarity evaluation system for turning over spectrums |
| CN116070077A (en) * | 2022-12-29 | 2023-05-05 | 江苏理工学院 | Music composing effect evaluation method and device for automatic music composing algorithm |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5952597A (en) * | 1996-10-25 | 1999-09-14 | Timewarp Technologies, Ltd. | Method and apparatus for real-time correlation of a performance to a musical score |
| CN103824565A (en) * | 2014-02-26 | 2014-05-28 | 曾新 | Humming music reading method and system based on music note and duration modeling |
| CN105070298A (en) * | 2015-07-20 | 2015-11-18 | 科大讯飞股份有限公司 | Polyphonic musical instrument scoring method and device |
| JP2016184112A (en) * | 2015-03-26 | 2016-10-20 | ヤマハ株式会社 | Ensemble evaluation apparatus |
| CN106228961A (en) * | 2016-07-21 | 2016-12-14 | 赵洪云 | Play evaluation methodology and device |
| CN107424476A (en) * | 2017-08-01 | 2017-12-01 | 上海破破飞网络科技有限公司 | A kind of instrumental music training mate system |
| CN107767847A (en) * | 2017-09-29 | 2018-03-06 | 小叶子（北京）科技有限公司 | A kind of intelligent piano performance assessment method and system |
| CN110600057A (en) * | 2019-09-02 | 2019-12-20 | 深圳市平均律科技有限公司 | Method and system for comparing performance sound information with music score information |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020116196A1 (en) * | 1998-11-12 | 2002-08-22 | Tran Bao Q. | Speech recognizer |
| KR100658219B1 (en) * | 2001-06-25 | 2006-12-15 | 어뮤즈텍(주) | Method and apparatus for designating expressive performance notes with synchronization information |
| GB2433349B (en) * | 2004-10-22 | 2010-03-17 | In The Chair Pty Ltd | A method and system for assessing a musical performance |
| US7750224B1 (en) * | 2007-08-09 | 2010-07-06 | Neocraft Ltd. | Musical composition user interface representation |
| JP5593608B2 (en) * | 2008-12-05 | 2014-09-24 | ソニー株式会社 | Information processing apparatus, melody line extraction method, baseline extraction method, and program |
| US8575465B2 (en) * | 2009-06-02 | 2013-11-05 | Indian Institute Of Technology, Bombay | System and method for scoring a singing voice |
| CN103871426A (en) * | 2012-12-13 | 2014-06-18 | 上海八方视界网络科技有限公司 | Method and system for comparing similarity between user audio frequency and original audio frequency |
| CN103823867B (en) * | 2014-02-26 | 2017-02-15 | 深圳大学 | Humming type music retrieval method and system based on note modeling |
| CN106228996B (en) * | 2016-07-15 | 2019-08-02 | 黄河科技学院 | Vocality study electron assistant articulatory system |
| CN106340286B (en) * | 2016-09-27 | 2020-05-19 | 华中科技大学 | Universal real-time musical instrument playing evaluation system |
| CN107818796A (en) * | 2017-11-16 | 2018-03-20 | 重庆师范大学 | A kind of music exam assessment method and system |
| CN108711336B (en) * | 2018-04-27 | 2020-05-12 | 山东英才学院 | Piano playing scoring method and device, computer equipment and storage medium |
| CN109448682A (en) * | 2018-12-04 | 2019-03-08 | 无锡市思娱文化科技有限公司 | A kind of system and its application method for realizing that microphone is interacted with jukebox |
| CN110070847B (en) * | 2019-03-28 | 2023-09-26 | 深圳市芒果未来科技有限公司 | Musical tone evaluation method and related products |
-
2019
- 2019-09-02 CN CN201910806968.6A patent/CN110600057B/en active Active
- 2019-09-02 CN CN202111041068.0A patent/CN113763991B/en active Active
- 2019-09-02 CN CN202111036222.5A patent/CN113744764B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5952597A (en) * | 1996-10-25 | 1999-09-14 | Timewarp Technologies, Ltd. | Method and apparatus for real-time correlation of a performance to a musical score |
| CN103824565A (en) * | 2014-02-26 | 2014-05-28 | 曾新 | Humming music reading method and system based on music note and duration modeling |
| JP2016184112A (en) * | 2015-03-26 | 2016-10-20 | ヤマハ株式会社 | Ensemble evaluation apparatus |
| CN105070298A (en) * | 2015-07-20 | 2015-11-18 | 科大讯飞股份有限公司 | Polyphonic musical instrument scoring method and device |
| CN106228961A (en) * | 2016-07-21 | 2016-12-14 | 赵洪云 | Play evaluation methodology and device |
| CN107424476A (en) * | 2017-08-01 | 2017-12-01 | 上海破破飞网络科技有限公司 | A kind of instrumental music training mate system |
| CN107767847A (en) * | 2017-09-29 | 2018-03-06 | 小叶子（北京）科技有限公司 | A kind of intelligent piano performance assessment method and system |
| CN110600057A (en) * | 2019-09-02 | 2019-12-20 | 深圳市平均律科技有限公司 | Method and system for comparing performance sound information with music score information |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113763991A (en) | 2021-12-07 |
| CN110600057A (en) | 2019-12-20 |
| CN113744764A (en) | 2021-12-03 |
| CN113763991B (en) | 2024-04-19 |
| CN110600057B (en) | 2021-12-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113744764B (en) | Method for obtaining optimal comparison path of performance time value information and score time value information | |
| CN110147451B (en) | Dialogue command understanding method based on knowledge graph | |
| CN113591457B (en) | Text error correction method, device, equipment and storage medium | |
| CN107993636B (en) | Recursive neural network-based music score modeling and generating method | |
| WO2019158927A1 (en) | A method of generating music data | |
| CN113010730B (en) | Music file generation method, device, equipment and storage medium | |
| CN115293139A (en) | Training method of voice transcription text error correction model and computer equipment | |
| CN115293138A (en) | Text error correction method and computer equipment | |
| CN115545033A (en) | Chinese field text named entity recognition method fusing vocabulary category representation | |
| CN114021573B (en) | Natural language processing method, device, equipment and readable storage medium | |
| CN110956039A (en) | Text similarity calculation method and device based on multi-dimensional vectorization coding | |
| CN109102006A (en) | A kind of music automark method based on the enhancing of audio frequency characteristics induction information | |
| CN116483337A (en) | API completion method based on prompt learning and data enhancement | |
| CN114021572B (en) | Natural language processing method, device, equipment and readable storage medium | |
| CN114579763A (en) | Character-level confrontation sample generation method for Chinese text classification task | |
| CN114925197A (en) | Deep learning text classification model training method based on topic attention | |
| Miyao et al. | Format of ground truth data used in the evaluation of the results of an optical music recognition system | |
| Wai et al. | Myanmar (Burmese) String Similarity Measures based on Phoneme Similarity | |
| CN110781678B (en) | Text representation method based on matrix form | |
| JP2014191490A (en) | Abbreviation generation device, abbreviation generation method and program | |
| WO2024042650A1 (en) | Training device, training method, and program | |
| KR20190111360A (en) | System, device, and method to generate polyphonic music | |
| US20240013760A1 (en) | Text providing method and text providing device | |
| Moon et al. | Concepts for Computing Patterns in 15th Century Korean Music | |
| Rahal et al. | Separated Feature Learning for Music Composition Using Memory-Based Neural Networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |