WO2022190453A1 - 運指提示装置、訓練装置、運指提示方法および訓練方法 - Google Patents

運指提示装置、訓練装置、運指提示方法および訓練方法 Download PDF

Info

Publication number
WO2022190453A1
WO2022190453A1 PCT/JP2021/040273 JP2021040273W WO2022190453A1 WO 2022190453 A1 WO2022190453 A1 WO 2022190453A1 JP 2021040273 W JP2021040273 W JP 2021040273W WO 2022190453 A1 WO2022190453 A1 WO 2022190453A1
Authority
WO
WIPO (PCT)
Prior art keywords
note
fingering
notes
information
finger
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/040273
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
正博 鈴木
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yamaha Corp
Original Assignee
Yamaha Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yamaha Corp filed Critical Yamaha Corp
Priority to JP2023505094A priority Critical patent/JP7679871B2/ja
Priority to CN202180095314.2A priority patent/CN116940978A/zh
Publication of WO2022190453A1 publication Critical patent/WO2022190453A1/ja
Anticipated expiration legal-status Critical
Priority to JP2025074933A priority patent/JP7841641B2/ja
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B15/00—Teaching music
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10G—REPRESENTATION OF MUSIC; RECORDING MUSIC IN NOTATION FORM; ACCESSORIES FOR MUSIC OR MUSICAL INSTRUMENTS NOT OTHERWISE PROVIDED FOR, e.g. SUPPORTS
    • G10G1/00—Means for the representation of music
    • G10G1/02—Chord or note indicators, fixed or adjustable, for keyboard of fingerboards

Definitions

  • the present invention relates to a fingering presentation device, a training device, a fingering presentation method, and a training method for presenting fingerings for playing a musical instrument.
  • Patent Document 2 describes a fingering determination method for determining the fingering for each note in a note sequence based on a probability model. JP 2013-083845 A JP 2007-241034 A
  • Patent Document 2 the player can recognize the fingering in playing the musical instrument based on the probability model.
  • the probability model there are countless combinations of fingerings, and there is more than one optimum fingering for playing a piece of music. Therefore, it is desirable to present more appropriate fingerings.
  • An object of the present invention is to provide a fingering presentation device, a training device, a fingering presentation method, and a training method capable of presenting appropriate fingerings for playing a musical instrument.
  • a fingering presentation device uses a receiving unit that receives time-series data including a string of notes made up of a plurality of notes, and a trained model to play at least some of the notes included in the string of notes with a musical instrument.
  • an estimating unit for estimating finger information indicating a finger to be used for playing or note information indicating a note to which fingering is to be applied from a string of notes;
  • a training apparatus comprises: a first acquisition unit that acquires input time-series data including a reference note string consisting of a plurality of notes; a second acquisition unit that acquires output finger information indicating a finger used when fingering is performed, or output note information indicating a note to which fingering is to be applied from a reference note string; a builder for building a trained model that has learned the input-output relationship between the output note information.
  • a fingering presentation method accepts time-series data including a string of notes, and uses a trained model to play at least a portion of the notes included in the string of notes with a musical instrument. Fingering information indicating a finger to be used when performing fingering or note information indicating a note to which fingering is to be applied is estimated from a string of notes, and executed by a computer.
  • a training method obtains input time-series data including a reference string of notes, and uses the musical instrument to play at least some of the notes included in the reference string. Acquire the output fingering information indicating the fingering or the output note information indicating the note to which fingering is applied from the reference note string, and determine the input/output relationship between the input time series data and the output fingering information or output note information.
  • a learned and trained model is built and executed by the computer.
  • FIG. 1 is a block diagram showing the configuration of a processing system including a fingering presentation device and a training device according to the first embodiment of the present invention.
  • FIG. 2 is a diagram showing an example of each training data.
  • FIG. 3 is a block diagram showing the configuration of the training device and the fingering presentation device.
  • FIG. 4 shows an example of an auxiliary musical score displayed on the display unit.
  • FIG. 5 is a flow chart showing an example of training processing by the training device of FIG.
  • FIG. 6 is a flowchart showing an example of fingering presentation processing by the fingering presentation device of FIG.
  • FIG. 7 is a diagram showing another example of input time-series data.
  • FIG. 8 is a diagram showing an example of input time-series data in the modified example.
  • FIG. 1 is a block diagram showing the configuration of a processing system including a fingering presentation device and a training device according to the first embodiment of the present invention.
  • FIG. 2 is a diagram showing an example of each training data.
  • FIG. 9 is a diagram showing an example of output finger information in the modified example.
  • FIG. 10 is a diagram showing an example of input time-series data in the second embodiment.
  • FIG. 11 is a diagram showing an example of output finger information according to the third embodiment.
  • FIG. 12 is a flowchart showing an example of fingering presentation processing in the modified example.
  • FIG. 13 is a diagram showing an example of finger information estimated in step S24 of the fingering presentation process.
  • FIG. 1 is a block diagram showing the configuration of a processing system including a fingering presentation device and a training device according to the first embodiment of the present invention.
  • the processing system 100 includes a RAM (random access memory) 110, a ROM (read only memory) 120, a CPU (central processing unit) 130, a storage section 140, an operation section 150 and a display section 160. .
  • RAM random access memory
  • ROM read only memory
  • CPU central processing unit
  • the processing system 100 is implemented by a computer such as a personal computer, tablet terminal, or smart phone.
  • the processing system 100 may be realized by cooperative operation of a plurality of computers connected by a communication path such as Ethernet, or may be realized by an electronic musical instrument such as an electronic piano having performance functions.
  • the RAM 110 , ROM 120 , CPU 130 , storage section 140 , operation section 150 and display section 160 are connected to the bus 170 .
  • the training device 10 and the fingering presentation device 20 are configured by the RAM 110 , the ROM 120 and the CPU 130 .
  • training device 10 and fingering presentation device 20 are configured by common processing system 100, but may be configured by separate processing systems.
  • the RAM 110 consists of, for example, a volatile memory, and is used as a work area for the CPU 130.
  • the ROM 120 is, for example, a non-volatile memory and stores a training program and a fingering presentation program.
  • CPU 130 performs a training process by executing a training program stored in ROM 120 on RAM 110 . Further, the CPU 130 performs fingering presentation processing by executing a fingering presentation program stored in the ROM 120 on the RAM 110 . Details of the training process and the fingering presentation process will be described later.
  • the training program or the fingering presentation program may be stored in the storage unit 140 instead of the ROM 120.
  • the training program or fingering presentation program may be provided in a form stored in a computer-readable storage medium and installed in ROM 120 or storage unit 140 .
  • a training program or a fingering presentation program distributed from a server (including a cloud server) on the network is installed in the ROM 120 or the storage unit 140.
  • the storage unit 140 includes a storage medium such as a hard disk, an optical disk, a magnetic disk, or a memory card, and stores a trained model M and a plurality of training data D.
  • the trained model M or each piece of training data D may not be stored in the storage unit 140, but may be stored in a computer-readable storage medium.
  • the trained model M or respective training data D may be stored on a server on that network.
  • the trained model M is a machine learning model trained to present fingerings when a user of the fingering presentation device 20 (hereinafter referred to as a performer) plays a musical piece. and is constructed using a plurality of training data D.
  • a user of the training device 10 can generate the training data D by operating the operation unit 150 .
  • the training data D is data created based on the playing knowledge or playing style of the reference performer.
  • the reference performer has a relatively high level of skill in playing the piece of music.
  • a reference performer may be the performer's mentor or teacher in the performance of the musical composition.
  • the training data D indicates a set of input time-series data and output instruction information.
  • the input time-series data indicates a reference note string consisting of a plurality of notes.
  • the input time-series data may be image data representing images of musical scores.
  • the output fingering information indicates the fingers of the reference performer to be used when playing each note of the reference note string by the musical instrument, and can be used to present fingerings when playing the reference note string.
  • the output finger information may be a unique number assigned to each finger. In this example, numbers “1" to "5" are given to the thumb, index finger, middle finger, ring finger and little finger, respectively.
  • the input time-series data further includes a reference player identifier that indicates the classification (category) of the reference player who performs the reference note string.
  • the reference player identifier is determined differently for each of at least one of the physical characteristics of the reference player and the style of performance by the reference player.
  • the physical characteristics of the reference player include, for example, the reference player's hand size (finger length), age, gender, or whether the player is an adult or a child.
  • FIG. 2 is a diagram showing an example of each training data D.
  • FIG. 2 shows part of the input time-series data and the output finger information when the reference player plays the piano.
  • input time-series data A includes elements A0 to A16.
  • the element A0 corresponds to a reference performer identifier, and is represented by a different character string for at least one of the reference performer's physical characteristics and the reference performer's style of performance.
  • Elements A1 to A16 correspond to reference note strings.
  • the element A0 is placed at the beginning of the input time-series data A, that is, before the reference note string (elements A1 to A16), but it may be placed at any position in the input time-series data A.
  • the elements A1 to A4 mean that the key of number '66' is pressed and maintained for 13 units of time, and then the key of number '66' is released and maintained for 2 units of time.
  • the output finger information B includes elements B0 to B16 corresponding to the elements A0 to A16 of the input time-series data A, respectively.
  • Element B0 indicates a reference player identifier and is represented by the same character string as element A0.
  • the training data D in FIG. 2 is generated to indicate left hand fingering, the embodiment is not limited to this.
  • the training data D may be generated to indicate the fingerings of the right hand, or may be generated to indicate the fingerings of each of the left and right hands.
  • Elements of the input time-series data A and the output finger information B for indicating the fingering of the right hand may use, for example, "R" instead of the letter "L”.
  • FIG. 3 is a block diagram showing the configuration of the training device 10 and the fingering presentation device 20.
  • the training device 10 includes a first acquisition unit 11, a second acquisition unit 12, and a construction unit 13 as functional units.
  • the functional units of the training device 10 are implemented by the CPU 130 of FIG. 1 executing the training program. At least part of the functional units of the training device 10 may be realized by hardware such as an electronic circuit.
  • the first acquisition unit 11 acquires the input time-series data A from each training data D stored in the storage unit 140 or the like.
  • the second acquisition unit 12 acquires output finger information B from each piece of training data D.
  • the construction unit 13 uses the input time-series data A acquired by the first acquisition unit 11 as an input element and the output finger information B acquired by the second acquisition unit 12 as an output element. Do machine learning. By repeating machine learning for a plurality of pieces of training data D, the building section 13 builds a trained model M that indicates the input/output relationship between the input time-series data A and the output finger information B.
  • the building unit 13 builds the trained model M by training the Transformer, but the embodiment is not limited to this.
  • the construction unit 13 may construct the trained model M by training a machine learning model of another method that handles time series.
  • the trained model M constructed by the construction unit 13 is stored in the storage unit 140, for example.
  • the trained model M constructed by the construction unit 13 may be stored in a server or the like on the network.
  • the fingering presentation device 20 includes a reception unit 21, an estimation unit 22, and a generation unit 23 as functional units.
  • the functional units of the fingering presentation device 20 are implemented by the CPU 130 of FIG. 1 executing the fingering presentation program. At least part of the functional units of the fingering presentation device 20 may be realized by hardware such as an electronic circuit.
  • the reception unit 21 receives time-series data including a string of notes made up of a plurality of notes.
  • the performer can give image data representing an image of the musical score to the reception unit 21 as time-series data.
  • the performer can generate time-series data by operating the operation unit 150 and provide it to the reception unit 21 .
  • the time-series data has the same configuration as the input time-series data A in FIG. 2, and further includes a performer identifier that indicates the classification (category) of the performer who plays the string of notes.
  • the performer identifier is determined differently for each of at least one of the performer's physical characteristics and the performer's performance style.
  • the player's physical characteristics include, for example, the player's hand size, age, gender, or whether the player is an adult or a child.
  • the estimation unit 22 estimates finger information using the trained model M stored in the storage unit 140 or the like.
  • the finger information indicates the player's finger used when playing each note of the note string accepted by the accepting unit 21, and is estimated based on the note string and the player identifier.
  • the finger information may be a unique number given to each finger.
  • the generation unit 23 generates score information based on the musical note sequence of the time-series data received by the reception unit 21 and the finger information estimated by the estimation unit 22 .
  • the display unit 160 displays an auxiliary musical score based on the musical score information generated by the generating unit 23 .
  • FIG. 4 shows an example of an auxiliary musical score displayed on the display unit 160.
  • the supporting musical score indicates the finger information estimated by the estimation unit 22 so as to correspond to each note of the note string received by the reception unit 21 .
  • the finger numbers of one hand are shown as the finger information.
  • a predetermined letter such as "L” is attached near the finger numbers of the left hand, and another letter such as "R” is attached near the finger numbers of the right hand.
  • Predetermined characters may be attached.
  • the finger numbers of the left hand or the corresponding musical notes may be given a predetermined color such as red, and the finger numbers or the corresponding musical notes of the right hand may be given another predetermined color such as blue.
  • FIG. 5 is a flowchart showing an example of training processing by the training device 10 of FIG.
  • the training process in FIG. 5 is performed by CPU 130 in FIG. 1 executing a training program.
  • the first acquisition unit 11 acquires the input time-series data A from each training data D (step S1).
  • the second acquisition unit 12 acquires the output finger information B from each training data D (step S2). Either of steps S1 and S2 may be performed first, or may be performed simultaneously.
  • the constructing unit 13 performs machine learning on each piece of training data D using the input time-series data A obtained in step S1 as an input element and the output finger information B obtained in step S2 as an output element (step S3). Subsequently, the construction unit 13 determines whether or not sufficient machine learning has been performed (step S4). If the machine learning is insufficient, the construction unit 13 returns to step S3. Steps S3 and S4 are repeated while changing the parameters until sufficient machine learning is performed. The number of iterations of machine learning changes according to quality conditions that the trained model M to be constructed should satisfy.
  • the construction unit 13 saves the input/output relationship between the input time-series data A and the output finger information B learned by the machine learning in step S3 as a trained model M (step S5). This completes the training process.
  • FIG. 6 is a flowchart showing an example of fingering presentation processing by the fingering presentation device 20 of FIG.
  • the fingering presentation process of FIG. 6 is performed by the CPU 130 of FIG. 1 executing the fingering presentation program.
  • the receiving unit 21 receives time-series data (step S11).
  • the estimation unit 22 estimates finger information from the time-series data received in step S11 using the trained model M saved in step S5 of the training process (step S12).
  • the generation unit 23 After that, the generation unit 23 generates musical score information based on the note sequence of the time-series data received in step S11 and the finger information estimated in step S12 (step S13). An auxiliary musical score may be displayed on the display unit 160 based on the generated musical score information. This completes the fingering presentation process.
  • the fingering presentation device 20 includes the receiving unit 21 that receives time-series data including a string of notes composed of a plurality of notes, and the trained model M and an estimating unit 22 for estimating finger information indicating a finger to be used when each note in the string of notes is played by the musical instrument.
  • the trained model M is used to estimate appropriate finger information from the temporal flow of multiple notes in the time-series data. This makes it possible to present appropriate fingerings for playing a musical instrument.
  • the trained model M is input between input time-series data A including a reference note string consisting of a plurality of notes and output fingering information B indicating the finger used when playing each note of the reference note string with a musical instrument. It may be a machine learning model that has learned the output relationship. In this case, finger information can be easily estimated from time-series data.
  • the time-series data may further include a player identifier indicating the player who plays the string of notes, and the estimation unit 22 may estimate finger information based on the player identifier. In this case, appropriate finger information can be estimated according to the player.
  • the performer identifier may be determined to correspond to the physical characteristics of the performer. In this case, appropriate finger information can be estimated according to the player's physical characteristics.
  • the performer identifier may be determined so as to correspond to the performance style of the performer. In this case, appropriate finger information can be estimated according to the performance style of the player.
  • the fingering presentation device 20 may further include a generating unit that generates musical score information indicating an auxiliary musical score to which finger information is attached so as to correspond to each note of the note string.
  • a generating unit that generates musical score information indicating an auxiliary musical score to which finger information is attached so as to correspond to each note of the note string.
  • the performer can easily recognize the finger corresponding to each note of the note string by visually recognizing the supporting musical score.
  • the training apparatus 10 includes a first acquisition unit 11 that acquires input time-series data A including a reference note string consisting of a plurality of notes, and a A second acquisition unit 12 that acquires output finger information B indicating a finger to be used, and a construction unit 13 that constructs a trained model M that has learned the input/output relationship between the input time-series data A and the output finger information B. and According to this configuration, a trained model M that has learned the input/output relationship between the input time-series data A and the output finger information B can be easily constructed.
  • the input time-series data A includes reference player identifiers, and the time-series data includes player identifiers, but the embodiment is not limited to this.
  • the input time-series data A may include the reference note string, and may not include the reference player identifier.
  • the time-series data only needs to include a string of musical notes, and does not have to include a player identifier.
  • the input time-series data A and the output finger information B are described on a so-called motion basis, which indicates key depression or key release in the MIDI (Musical Instrument Digital Interface) standard. is not limited to this.
  • the input time-series data A and the output finger information B may be described in other methods.
  • the input time-series data A and the output finger information B may be described on a so-called note basis, which indicates the start position of a note or the length of a note in the MIDI standard. The same applies to time-series data and finger information.
  • FIG. 7 is a diagram showing another example of input time-series data A.
  • FIG. The upper part of FIG. 7 shows input time-series data A(Ax) described on a motion basis.
  • the middle part of FIG. 7 shows the input time-series data A (Ay) described on a note basis.
  • the input time-series data Ax and the input time-series data Ay include the same reference note string (the reference note string in the musical score shown at the bottom of FIG. 7).
  • "bar" and "beat" in the input time-series data Ax, Ay are elements indicating the metrical structure of the reference note string.
  • the length of the input time-series data A is shortened by describing the input time-series data A on a musical note basis. This makes it possible to process longer input time-series data A easily.
  • the output finger information B corresponding to the input time-series data A is obtained by inserting an element indicating the finger number immediately after the element indicating the pitch number in the input time-series data A ("note_ ⁇ "). can be described.
  • the input time-series data A and the output finger information B may be described by a system representing musical scores.
  • the details of the input time-series data A and the output finger information B described by the method of representing musical scores will be described in the following modifications.
  • FIG. 8 is a diagram showing an example of input time-series data A in a modification.
  • the upper part of FIG. 8 shows the input time-series data A (Az) described by the musical score representation method.
  • a musical score represented by the input time-series data A is shown in the lower part of FIG.
  • the input time-series data Az includes multiple elements A0 to A24. Some elements have attributes. Attributes of an element are written behind the element (after the underscore).
  • the element A0 indicates the ratio of notes to which fingering is applied among the notes included in the reference note string.
  • the element A0 is placed before the beginning of the input time-series data Az, but may be placed at any position in the input time-series data Az.
  • the ratio is specified by the "fingerrate" attribute of the element A0.
  • Attribute "5" in this example means a percentage of 100%. The percentage may have a range or be divided into multiple ranges, such as 20-40% or 40-60%.
  • Element A1 indicates a part.
  • the element A1 is arranged immediately after the element A0, but may be arranged at any position in the input time-series data Az.
  • element A1 "R” and “L” indicate right and left hand parts respectively.
  • the element corresponding to the right hand is placed after the "R”.
  • An “L” is placed after it, and an element corresponding to the left hand is placed after the "L”.
  • the "R” and the right-hand corresponding element may be placed after the left-hand corresponding element. If the parts are not distinguished, the input time-series data Az does not include the element A1.
  • Elements A2, A15, and A24 indicate bar lines of the musical score. Therefore, in the example of FIG. 8, the range delimited by “bar” in element A2 and “bar” in element A15 corresponds to the first measure. The range delimited by "bar” in element A15 and “bar” in element A24 corresponds to the second bar.
  • Element A3 indicates the clef of the score.
  • the clef type is specified by the attribute of "clef" in the element A3.
  • the treble clef is specified as the clef by the element A3.
  • the bass clef is specified as the clef by the element A3.
  • Element A4 indicates the time signature of the musical score.
  • the type of time signature is specified by the "time” attribute of the element A4.
  • the attribute is “4/4", so element A4 specifies "4/4" as the time signature.
  • the direction of the note stems in the score is specified by other attributes of "len” in the elements A6, A10, A12, A14, A17, A19, and A21. If the other attribute is “down”, the stem extends downward from the notehead. If the other attribute is “up”, the stem extends up from the note head.
  • the start position, intermediate position and end position of the beam are further attributes of "len” in elements A10, A12 and A14. Specified by “start”, “continue” and “stop” respectively.
  • a rest in the reference note string is specified by “rest” in elements A7 and A22.
  • the note value of the rest is described by the "len” attribute in the elements A8 and A23.
  • elements A5 and A6 indicate note N1
  • elements A7 and A8 indicate rest R1.
  • Elements A9 and A10 represent note N2
  • elements A11 and A12 represent note N3
  • elements A13 and A14 represent note N4.
  • Elements A16 and A17 indicate note N5, and elements A18 and A19 indicate note N6.
  • Elements A20 and A21 indicate note N7, and elements A22 and A23 indicate rest R2.
  • FIG. 9 is a diagram showing an example of output finger information B in a modified example.
  • the upper part of FIG. 9 shows the output finger information B (Bz) written in a musical score representation format.
  • the output finger information Bz corresponds to the input time-series data Az in FIG.
  • the lower part of FIG. 9 shows the musical score represented by the output finger information Bz.
  • the output finger information Bz includes a plurality of elements B0-B24.
  • the output finger information Bz further includes elements B5f, B9f, B11f, B13f, B16f, B18f, and B20f arranged immediately after the elements B5, B9, B11, B13, B16, B18, and B20, respectively.
  • Elements B0 to B24 are the same as elements A0 to A24 of input time-series data Az in FIG. 8, respectively. Therefore, the first acquisition unit 11 in FIG. 3 can acquire the input time-series data Az by deleting the elements B5f, B9f, B11f, B13f, B16f, B18f, and B20f from the output finger information Bz.
  • Elements B5f, B9f, B11f, B13f, B16f, B18f, and B20f respectively indicate the finger numbers used when the notes corresponding to the immediately preceding elements B5, B9, B11, B13, B16, B18, and B20 are played by the musical instrument. show.
  • the "finger" attribute of the elements B5f, B9f, B11f, B13f, B16f, B18f, and B20f designates the finger number. Therefore, the elements B5f, B9f, B11f, B13f, B16f, B18f, and B20f, as shown in the lower part of FIG. 2”, “1”, “3”, “3” and “2” are written on the score respectively.
  • the generation unit 23 may generate a moving image file showing finger movements by animation or the like, based on the finger information estimated by the estimation unit 22 . This makes it possible to visualize finger movements. Generation of such a moving image file may be performed before or after step S13 in the fingering presentation process of FIG. 6, may be performed in parallel with step S13, or may be performed instead of step S13. may
  • the estimating unit 22 selects some of the notes included in the note string to which fingering is to be applied, and the fingering information for that part of the notes. presume. In this case, it is possible to present appropriate fingerings for a beginner-level or intermediate-level player who is higher than the introductory level to play the musical instrument. In this configuration, the output finger information Bz does not include some of the elements B5f, B9f, B11f, B13f, B16f, B18f and B20f.
  • the estimation unit 22 may estimate the note information indicating the note to which fingering is to be applied from the note sequence without estimating the finger information. Details will be described in a third embodiment, which will be described later.
  • Second Embodiment (1) Processing System Regarding a processing system 100 according to a second embodiment, differences from the processing system 100 according to the first embodiment will be described.
  • the first acquisition unit 11 and the second acquisition unit 12 acquire the input time-series data A and the output finger information B of the training data D, respectively.
  • FIG. 10 is a diagram showing an example of input time-series data A in the second embodiment.
  • the upper part of FIG. 10 shows the input time-series data Az described by the musical score representation method.
  • the lower part of FIG. 10 shows the musical score represented by the input time-series data Az.
  • the input time-series data Az includes multiple elements A0 to A24. Elements A0 to A24 in FIG. 10 are the same as elements A0 to A24 in the modification (FIG. 8) of the first embodiment, respectively. Also, the input time-series data Az includes additional elements arranged immediately after parts of the elements A5, A9, A11, A13, A16, A18, and A20 corresponding to musical notes. In the example of FIG. 10, the input time-series data Az further includes elements A5f, A11f, A16f, and A20f arranged immediately after elements A5, A11, A16, and A20, respectively.
  • Elements A5f, A11f, A16f, and A20f are fingering information (hereinafter referred to as basic fingering information) indicating the finger numbers used when playing the notes corresponding to the immediately preceding elements A5, A11, A16, and A20. ).
  • the "finger" attribute of the elements A5f, A11f, A16f, and A20f designates the finger number. Therefore, the elements A5f, A11f, A16f, and A20f, as shown in the lower part of FIG. "2" is written in each musical score.
  • the output finger information Bz in the present embodiment is the same as the output finger information Bz in the modified example (FIG. 9) of the first embodiment. Therefore, the first acquisition unit 11 can acquire the input time-series data Az by randomly deleting some of the elements B5f, B9f, B11f, B13f, B16f, B18f, and B20f from the output finger information Bz. can.
  • the user of the training device 10 can specify the ratio of the elements B5f, B9f, B11f, B13f, B16f, B18f, and B20f to be deleted by operating the operation unit 150 in FIG.
  • the input time-series data Az is obtained by deleting the elements B9f, B13f, and B18f from the output finger information Bz.
  • Elements B5f, B11f, B16f, and B20f that are not deleted remain as elements A5f, A11f, A16f, and A20f, which are basic finger information.
  • the construction unit 13 in FIG. 3 performs machine learning using the input time-series data Az as an input element and the output finger information Bz as an output element. By repeating machine learning for a plurality of training data D, a trained model M representing the input/output relationship between the input time-series data Az and the output finger information Bz is constructed.
  • the reception unit 21 receives the time-series data.
  • the time-series data further includes basic finger information indicating fingers used to play some of the notes included in the note string with the musical instrument.
  • the estimating unit 22 estimates finger information indicating fingers to be used when playing notes included in the note sequence with a musical instrument.
  • the generation unit 23 generates musical score information based on the note sequence and finger information of the time-series data.
  • the generation unit 23 may generate a moving image file showing finger movements by animation or the like. In this case, finger movements can be visualized.
  • the estimating section 22 estimates finger information for all notes included in the sequence of notes in the time-series data, but the embodiment is not limited to this.
  • the estimating unit 22 determines a second proportion of notes included in the note string, which is greater than the first proportion. Fingering information about notes may be estimated. In this case, appropriate fingerings for a beginner or intermediate player playing the instrument can be presented.
  • the output finger information B of the training data D may not include some of the elements B5f, B9f, B11f, B13f, B16f, B18f, and B20f.
  • the output finger information B when the input time-series data Az includes elements A5f, A11f, A16f, and A20f, the output finger information B includes elements B5f, B11f, B16f, and B20f.
  • the output finger information B may not include some of the elements B9f, B13f, and B18f.
  • training data D represents a set of input time-series data A and output note information.
  • the first acquisition unit 11 and the second acquisition unit 12 acquire the input time-series data A and the output note information of the training data D, respectively. Acquisition of output note information is executed instead of step S2 in the sound learning process of FIG.
  • the input time-series data Az in this embodiment is the same as the input time-series data Az in the modified example (FIG. 8) of the first embodiment.
  • the first acquisition unit 11 can acquire the input time-series data Az by deleting the elements C9f, C11f, and C16f from the output note information Cz in FIG. 11, which will be described later.
  • FIG. 11 is a diagram showing an example of output note information C in the third embodiment.
  • the upper part of FIG. 11 shows the output note information C (Cz) described by the musical score representation method.
  • the lower part of FIG. 11 shows the musical score represented by the output note information Cz.
  • the output note information Cz includes a plurality of elements C0-C24. Elements C0 to C24 in FIG. 11 are the same as elements B0 to B24 of the output finger information Bz of the modified example (FIG. 9) of the first embodiment.
  • the output note information Cz also includes additional elements placed immediately after portions of the elements C5, C9, C11, C13, C16, C18, and C20 corresponding to the notes.
  • the attribute of "fingerrate” in element C0 is “2"
  • attribute "2" means a rate of 40%. Therefore, the output note information Cz consists of elements C9f, C11f, C9f, C11f, Further includes C16f.
  • Elements C9f, C11f, and C16f respectively indicate notes corresponding to immediately preceding elements C9, C11, and C16 as notes to which fingering is to be applied from the reference note string.
  • the notes N2, N3 and N5 corresponding to the elements C9, C11 and C16 are identifiably written on the musical score by the elements C9f, C11f and C16f.
  • the construction unit 13 in FIG. 3 performs machine learning with the input time-series data Az as input elements and the output note information Cz as output elements.
  • a trained model M representing the input/output relationship between the input time-series data Az and the output note information Cz is constructed.
  • the reception unit 21 receives the time-series data. Based on the trained model M constructed by the training device 10 and the time-series data received by the receiving unit 21, the estimating unit 22 estimates note information indicating notes to which fingering is to be applied from the string of notes. do. Estimation of note information is executed instead of step S12 in the fingering presentation process of FIG.
  • the generation unit 23 generates musical score information indicating an auxiliary musical score in which the musical notes indicated by the musical note information are displayed in an identifiable manner.
  • the estimation unit 22 uses the first trained model M constructed in the first embodiment and the second trained model M constructed in the present embodiment to Finger information may be estimated that indicates which fingers are to be used to play some of the notes included in the string.
  • FIG. 12 is a flowchart showing an example of fingering presentation processing in the modified example.
  • the reception unit 21 receives time-series data (step S21).
  • the estimation unit 22 estimates intermediate finger information from the time-series data received in step S11 using the first trained model M constructed in the first embodiment (step S22).
  • the intermediate finger information indicates the finger used when playing each note included in the note string with the musical instrument.
  • the estimation unit 22 also uses the second trained model M constructed in the present embodiment to estimate note information from the time-series data received in step S11 (step S23). Either of steps S22 and S23 may be performed first, or may be performed simultaneously.
  • the estimating unit 22 estimates fingering information for notes included in the note string other than the note indicated by the note information estimated in step S23, based on the intermediate fingering information estimated in step S22. (step S24). After that, the generation unit 23 generates musical score information based on the note sequence of the time-series data received in step S21 and the finger information estimated in step S24 (step S25). This completes the fingering presentation process.
  • the intermediate finger information estimated in step S22 has, for example, the same configuration as the output finger information Bz of the modified example (FIG. 9) of the first embodiment.
  • the note information estimated in step S23 has the same structure as the output note information Cz in FIG.
  • FIG. 13 is a diagram showing an example of finger information estimated in step S24 of the fingering presentation process.
  • the upper part of FIG. 13 shows the finger information F (Fz) written in a musical score representation format.
  • the lower part of FIG. 13 shows an auxiliary musical score represented by finger information Fz.
  • the finger information Fz is composed of elements B9f, B11f, and B16f corresponding to the elements C9f, C11f, and C16f indicating the notes to which fingering is to be applied in the note information (see FIG. 11), from the intermediate finger information (see FIG. 9). is estimated by removing
  • finger information Fz includes a plurality of elements F1 to F24. Elements F1 to F24 in FIG. 13 are the same as elements B1 to B24 of the output finger information Bz of the modified example (FIG. 9) of the first embodiment. Also, the finger information Fz includes additional elements placed immediately after parts of the elements F5, F9, F11, F13, F16, F18, and F20 corresponding to the notes. In this example, the finger information Fz further includes elements F5f, F13f, F18f, and F20f arranged immediately after the elements F5, F13, F18, and F20, respectively.
  • Elements F5f, F13f, F18f, and F20f respectively indicate the finger numbers used when playing the notes corresponding to the immediately preceding elements F5, F13, F18, and F20.
  • the "finger" attribute of the elements F5f, F13f, F18f, and F20f designates the finger number. Therefore, the elements F5f, F13f, F18f, and F20f, as shown in the lower part of FIG. and "2" are respectively written on the supporting musical score.
  • the fingering information for some notes is thinned out from the fingering information for all notes included in the note string of the time-series data.
  • appropriate fingerings for a beginner or intermediate player playing the instrument can be presented.
  • fingering information about important notes is thinned out when playing a musical instrument, so beginner-level or intermediate-level players should cultivate appropriate fingering judgment when practicing a musical instrument. can be done.
  • the fingering presentation device 20 includes the generator 23, but the embodiment is not limited to this.
  • the player can create an auxiliary musical score by transcribing the finger information estimated by the estimating section 22 into a desired musical score. Therefore, the fingering presentation device 20 does not have to include the generator 23 .
  • the training data D is trained to estimate finger information when playing the piano, but the embodiment is not limited to this.
  • the training data D may be trained to estimate finger information when performing with other musical instruments such as drums.
  • the user of the fingering presentation device 20 is a performer has been described as an example, but the user of the fingering presentation device 20 may be, for example, a staff member of a music production company. . Also, the machine learning by the training device 10 may be performed in advance by the staff of the music production company.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Auxiliary Devices For Music (AREA)
  • Electrophonic Musical Instruments (AREA)
PCT/JP2021/040273 2021-03-09 2021-11-01 運指提示装置、訓練装置、運指提示方法および訓練方法 Ceased WO2022190453A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2023505094A JP7679871B2 (ja) 2021-03-09 2021-11-01 運指提示装置、訓練装置、運指提示方法および訓練方法
CN202180095314.2A CN116940978A (zh) 2021-03-09 2021-11-01 运指提示装置、训练装置、运指提示方法及训练方法
JP2025074933A JP7841641B2 (ja) 2021-03-09 2025-04-28 運指提示装置、訓練装置、運指提示方法および訓練方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-037170 2021-03-09
JP2021037170 2021-03-09

Publications (1)

Publication Number Publication Date
WO2022190453A1 true WO2022190453A1 (ja) 2022-09-15

Family

ID=83226235

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/040273 Ceased WO2022190453A1 (ja) 2021-03-09 2021-11-01 運指提示装置、訓練装置、運指提示方法および訓練方法

Country Status (3)

Country Link
JP (2) JP7679871B2 (https=)
CN (1) CN116940978A (https=)
WO (1) WO2022190453A1 (https=)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111723938A (zh) * 2020-05-06 2020-09-29 华南理工大学 一种基于指法规则和强化学习的钢琴指法自动生成方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4613817B2 (ja) * 2005-12-27 2011-01-19 カシオ計算機株式会社 運指表示装置、及びプログラム
JP2007241034A (ja) * 2006-03-10 2007-09-20 Univ Of Tokyo 楽器演奏における運指決定方法及びシステム
WO2008147368A1 (en) * 2007-06-01 2008-12-04 Richard William Worrall Method of automated musical instrument finger finding

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111723938A (zh) * 2020-05-06 2020-09-29 华南理工大学 一种基于指法规则和强化学习的钢琴指法自动生成方法

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
HOTTA, KEISUKE: "Study on the presentation of finger movement considering physical characteristics and musical interpretation", SPRING AND AUTUMN MEETING OF THE ACOUSTICAL SOCIETY OF JAPAN, ACOUSTICAL SOCIETY OF JAPAN, JP, 17 September 2010 (2010-09-17) - 17 September 2019 (2019-09-17), JP , pages 46 - 47, XP009540205, ISSN: 1880-7658 *
NAKAMURA, EITA ET AL.: "Fingering estimation by statistical learning method using fingering data and formulation of playing difficultly", IPSJ TECHNICAL REPORT, vol. 2019-MUS-124, no. 12, 20 August 2019 (2019-08-20), pages 1 - 16, XP009539748 *
SAKO, SHINJI ET AL.: "Automatic estimation of violin textbook fingering useful for beginners", IPSJ TECHNICAL REPORT, vol. 2019-MUS-123, no. 22, 15 June 2019 (2019-06-15), pages 1 - 6, XP009539747 *
SHINMURA, YUKA; SEKIZAWA, AKIRA; NAKAJIMA, KATSUTO: "E-011 Fingering Proposal from Music Note for Practicing String Instrument", PROCEEDINGS OF 18TH FORUM ON INFORMATION TECHNOLOGY (FIT2019); SEPTEMBER 3-5, 2019, vol. 18, no. 2, 20 August 2019 (2019-08-20) - 5 September 2019 (2019-09-05), pages 207 - 208, XP009539638 *
WATANABE, JURI ET AL.: "1T-01 Effective fingering estimation for beginner education of violin", PROCEEDINGS OF THE 81ST NATIONAL CONFERENCE OF IPSJ, INFORMATION PROCESSING SOCIETY OF JAPAN, JP, vol. 81, no. 2, 28 February 2019 (2019-02-28), JP, pages 2 - 2-336, XP009539640 *

Also Published As

Publication number Publication date
JP7841641B2 (ja) 2026-04-07
JPWO2022190453A1 (https=) 2022-09-15
JP2025111710A (ja) 2025-07-30
JP7679871B2 (ja) 2025-05-20
CN116940978A (zh) 2023-10-24

Similar Documents

Publication Publication Date Title
US8586849B1 (en) Media system and method of progressive instruction in the playing of a guitar based on user proficiency
Serafin et al. Considerations on the use of virtual and augmented reality technologies in music education
US10748515B2 (en) Enhanced real-time audio generation via cloud-based virtualized orchestra
US10013963B1 (en) Method for providing a melody recording based on user humming melody and apparatus for the same
CN114267318B (zh) Midi音乐文件的生成方法、存储介质和终端
US9734810B2 (en) Automatic harmony generation system
Syukur et al. Immersive and challenging experiences through a virtual reality musical instruments game: an approach to gamelan preservation
WO2023181570A1 (ja) 情報処理方法、情報処理システムおよびプログラム
WO2019167719A1 (ja) 楽曲の演奏を処理するための情報処理方法および装置
JP2025172909A (ja) コード推定装置およびコード推定方法
US20180268731A1 (en) Musical Modification Method
CN112632401B (zh) 推荐装置、信息提供系统、推荐方法及存储介质
JP7679871B2 (ja) 運指提示装置、訓練装置、運指提示方法および訓練方法
US20240087549A1 (en) Musical score creation device, training device, musical score creation method, and training method
KR102490769B1 (ko) 음악적 요소를 이용한 인공지능 기반의 발레동작 평가 방법 및 장치
JP2019109357A (ja) 音楽情報の特徴解析方法及びその装置
CN115712473A (zh) 乐谱展示方法、装置、电子设备及存储介质
JP2017173641A (ja) 楽譜表示制御装置及びプログラム
JP7782754B1 (ja) 情報処理方法およびプログラム
JP7571804B2 (ja) 情報処理システム、電子楽器、情報処理方法および機械学習システム
CN111354327A (zh) 一种辅助弹奏方法、介质及智能钢琴
JP5609520B2 (ja) 演奏評価装置および演奏評価プログラム
TWI696986B (zh) 智能鋼琴訓練系統
Caronongan III Classifying Rhythm Game Chart Difficulty Level Using Hidden Markov Models
JP6632447B2 (ja) 運指決定プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21930299

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202180095314.2

Country of ref document: CN

Ref document number: 2023505094

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21930299

Country of ref document: EP

Kind code of ref document: A1