JP2005249844A - Device and program for performance indication - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make it easy to read a musical score for keying indication with the eye. <P>SOLUTION: A performance indication symbol indicating notes to be played next and a timing symbol indicating performance timing are displayed on a musical score including a plurality of measures. It is decided whether a performance is carried out correctly in the timing indicated with the timing symbol and when it is judged that the performed is carried out correctly, the performance indication symbol is advanced to a position of a note to be played next. Provided is a means of detecting the performance indication advances to a predetermined period, and when it is decided that the performance indication advances to the scheduled period, the currently displayed musical score is gradually rewritten with a musical score following the currently displayed musical score. A performance sound is decided by comparing a power spectrum obtained by subjecting digital data of the performance sound to FFT processing with the power spectrum of a model. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a performance instruction system for a natural musical instrument, and more particularly to a performance instruction device and a performance instruction program that can advance a performance instruction to the next stage when a performance is correctly performed according to performance information.

  For example, as disclosed in Japanese Patent Publication No. 2-705, an electronic musical instrument having a key pressing instruction unit for self-studying the performance of a keyboard instrument is known. In this electronic musical instrument, stored performance information is sequentially read out to a display circuit, and a display lamp provided corresponding to each key of the keyboard is turned on according to this performance information. An operation key by the performer is detected by a key scanning circuit, and when the pitch of the operation key matches the pitch of the performance information indicated by the display lamp, the next performance information is read out to the display circuit.

  However, such a key pressing instruction means can be easily realized with an electronic musical instrument provided with a key scan circuit, but is difficult to realize with a natural musical instrument such as an acoustic piano without a key scan circuit. Although it is conceivable to provide a key scan circuit for a natural musical instrument, the work of retrofitting key switches and display lamps necessary for the key scan circuit for each key is complicated and cannot be performed easily.

  Instead of using a key scan circuit, it is conceivable to detect an operation key by frequency analysis of musical sounds by performance. For example, in the display device described in Japanese Patent Publication No. 1-51200, the pitch of the audio signal determined based on the frequency analysis result of the audio signal is displayed as a musical note on a staff score on the display.

  However, when detecting the pitch of a musical tone that contains a lot of harmonic components such as piano sounds, the pitch may be misjudged by the harmonic component of the key press, that is, the harmonic component, or two chords with an octave interval and (For example, discrimination between a chord of C3 and C4 and a C3 single tone) may not be possible. This is because these harmonic structures are similar to each other.

  In addition, an apparatus for generating a MIDI code corresponding to the performance of a musical instrument that generates a plurality of fundamental sounds and their harmonics simultaneously has been proposed (Japanese Patent No. 2890831). This device is provided with means for extracting waveform data of frequency components corresponding to the fundamental tone from the waveform data of the performance music sound and detecting the envelope data, while corresponding to the envelope waveform for each fundamental tone generated by the instrument. A storage means for storing the fundamental tone envelope data is provided, and the detected envelope data and the stored fundamental tone envelope data are compared with each other by cross-correlation to determine the instrument sound. This apparatus has a very complicated configuration, and in particular, it is difficult to process envelope data comparison and instrument sound determination in real time with software.

Japanese Patent No. 2880647 proposes a device that compares a signal created by sequentially combining various pitches with a synthesizer and an input signal, and makes a musical score. However, in this apparatus, since it is necessary to compare with all combinations of sounds including chords, there is a problem that calculation time becomes enormous.
Japanese Patent Publication No. 2-705 Japanese Patent Publication No. 1-51200 Japanese Patent No. 2890831 Japanese Patent No. 2880647

  As described above, in an acoustic piano that does not have a key scanning means, it is not possible to determine whether or not the key has been depressed as instructed, and it is not possible to determine the accuracy of the key depression in real time from the musical sound input from the microphone. There was a problem that it was not easy.

  Therefore, the applicant stores the power spectrum of each pitch as a single-tone model, compares the power spectrum of the sound played according to the key-pressing instruction with this single-tone model, and determines the distance between the two power spectra. In this case, a method has been proposed in which a key pressing instruction is advanced (Japanese Patent Application No. 2003-134372). This method is simpler to calculate than parameters such as pitch from the performance sound, and can play a musical instrument to be played and create a model based on that sound. Whether or not the performance sound is correct can be determined regardless of the deviation.

  It is conceivable to advance automatic accompaniment according to the performance information together with the key pressing instruction. However, since it takes time to calculate the fast Fourier transform (FFT) for comparing the power spectra, the determination as to whether the performance sound is correct is delayed from the real time. As with the key press instruction, if the accompaniment is stopped at the exact timing when the musical tone is present and the performance is judged, the accompaniment will resume until it is determined that the performer is able to perform correctly even if the performer performs at the correct timing. Since the accompaniment is interrupted, the performance is not smooth.

  Further, when the key depression instruction is performed on the score displayed on the screen, when not all the scores can be displayed on one screen, the score is viewed by scrolling the screen or switching the screen display. There are a method of scrolling or switching the screen from left to right by displaying the score in a single step, a method of scrolling or switching the score displayed in multiple steps from the top to the bottom, and the like.

  In the method in which the key pressing instruction is advanced to the next key pressing timing when the performance is performed according to the key pressing instruction, the scrolling or switching is automatically performed. For example, the screen is gradually scrolled at a speed in time for the performance, or the screen is switched to the next at a timing when the last displayed measure or the last displayed note is played. However, in any of these scrolling methods, the score itself is moved, so once the player takes his eye off the screen, he or she cannot know where he was playing. Even if you don't take your eyes off the screen, the player will have the extra care of watching the moving score in addition to the key press instruction.

  In addition, it is desirable for the performer to be able to wait for a key-pressing instruction with a margin by looking at a note that is a little ahead of the score rather than displaying the note to be pressed immediately before.

  In view of the above problems, an object of the present invention is to provide a performance instruction system for a natural musical instrument that can easily follow an accompaniment and a key pressing instruction and can perform smoothly.

  In order to solve the above-mentioned problems and achieve the object, the present invention displays a score including a plurality of measures, a performance instruction symbol indicating the next note to be played on the score, and a timing symbol indicating the performance timing. Display performance means, performance sound determination means for determining whether or not the musical note indicated by the performance instruction symbol has been correctly played at the timing indicated by the timing symbol, and determination that the performance sound determination means has performed correctly. Sometimes the display control means for advancing the performance instruction symbol to the position of the note to be performed next, and when the display of at least one of the performance timing symbol and the performance instruction symbol proceeds to a predetermined time, it is currently displayed. The first feature is that it has means for rewriting the currently displayed score from the beginning by the score following the score being displayed.

  According to a second aspect of the present invention, the rewriting of the musical score proceeds while maintaining the display position and the rewriting progress position of at least one of the performance timing symbol and the performance instruction symbol at a predetermined interval. And the scheduled interval is determined so that the display coordinates in the horizontal direction of the display position of at least one of the performance timing symbol and the performance instruction symbol and the rewriting progress position coincide with each other. There are features.

  According to the present invention, the musical score is displayed on one screen with a plurality of stages including a plurality of bars above and below, and each predetermined bar is displayed on at least the second and subsequent stages. There is a fourth feature in that any one of the performance timings is set.

  In the present invention, the score is displayed in one stage on one screen, and the predetermined time is when the display of at least one of the performance timing symbol and the performance instruction symbol is advanced to half of one paragraph. There is a fifth feature in a certain point.

  Further, the present invention is that accompaniment means for automatically accompaniing according to scheduled performance information, and that the performance sound determination means can perform correctly even if the timing at which the note indicated by the performance instruction symbol is played exceeds the scheduled time. There is a sixth feature in that there is provided means for temporarily stopping automatic accompaniment when a determination result cannot be obtained.

  Further, according to the present invention, the performance sound judging means performs FFT processing on digital data of the natural musical instrument performance sound performed in accordance with a performance instruction and calculates a power spectrum of the digital data, and each FFT Whether the power spectrum calculated for each processing is within a predetermined threshold within the predetermined threshold of the power spectrum for each pitch of the natural musical instrument sound stored in advance. The seventh feature is that it comprises a comparison means for judging whether or not the performance is correct when the distance is within a predetermined threshold.

  According to the first to fifth features, when the performance instruction has progressed to the middle of the displayed score, the currently displayed score is changed to the first score by the undisplayed score following the currently displayed score. It will be gradually rewritten from the part. When the performer has read the score to the end, the performer only needs to perform a standard eye movement to move the line of sight to the first portion of the score, so there is no need to search for the portion to be played and follow the score. Therefore, it is possible to easily perform the performance according to the performance instruction.

  According to the sixth feature, the automatic accompaniment is not stopped at the timing at which the next note should be played and the determination of the performance sound is not waited. If the performance is performed correctly, the automatic accompaniment continues. You can practice the performance according to the smooth accompaniment.

  According to the seventh feature, it is determined that there has been an attack when the power spectrum increase amount of the performance sound performed in accordance with the performance instruction is greater than or equal to the schedule. Then, when the power spectrum of the musical instrument performance sound matches the one of the pitches included in the performance instruction among the power spectra of the natural musical instrument sounds stored in advance, it is determined that the musical instrument performance sound was played as instructed. The performance instruction is advanced.

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram of a lesson system of an acoustic piano (hereinafter, simply referred to as “piano” unless otherwise distinguished from an electronic piano) according to an embodiment of a musical tone determination apparatus of the present invention. In this lesson system, key press instructions including piano key or pitch instructions and key press timing instructions are displayed based on pre-prepared performance information, and the performer sequentially advances the performance according to the key press instructions. To go. Specific processing consists of a model creation phase and a lesson phase. In the model creation phase, prior to the lesson, the power spectrum of the generated sound of each piano key used in the lesson is stored as a single tone model. In the lesson mode, a display for a key pressing instruction is performed (details of the key pressing instruction will be described later), and the power spectrum of the performance sound performed in accordance with the key pressing instruction is compared with the single-tone model, and the power of both is compared. If the spectrum is within the planned distance, the key pressing instruction is advanced.

  The lesson system shown in FIG. 1 includes a personal computer including a CPU 1, ROM 2, RAM 3, sound source device 4, speaker 5, A / D converter 6, input device (keyboard or mouse) 7, and display device 8. can do. As the display device 8, a known display means for displaying the processing result of the personal computer, such as a liquid crystal display or a cathode ray tube, can be used. An external storage device 9 is connected to the personal computer, and a microphone 10 is connected via the A / D converter 6. The microphone 10 is provided for collecting the sound generated by the piano P, and is preferably arranged in the piano P.

  The CPU 1 includes a volume measuring unit 11, a spectrum creating unit 12, an attack detecting unit 13, a spectrum comparing unit 14, and a key pressing instruction unit 15 as main functions. The volume measuring unit 11 detects the level (volume) of the sound input from the microphone 10. The spectrum creating unit 12 has an FFT function for obtaining a power spectrum from a digital musical tone signal input through the microphone 9 and the A / D converter 6. The power spectrum obtained by the FFT is stored in the RAM 3. The attack detection unit 13 detects an attack based on the detected power spectrum. The spectrum comparison unit 14 compares the distances of the power spectra created by the spectrum creation unit 12 in the model creation phase and the lesson phase, and determines whether or not the performance has been performed according to the key depression instruction. The key pressing instruction unit 15 creates display data for displaying a key pressing instruction on the display device 8 according to the performance information. The display data is created in response to a performance start instruction input from the input device 7 and input to the display device 8.

  FIG. 2 is a format example of performance information used for displaying a key pressing instruction on the display device 8 and performing accompaniment during a lesson. The performance information stores at least event data and timing data for instructing the reading timing of the event data. The event data is composed of note-on data and note-off data including a note number. The timing data is set as time information from the end of one event to the next event occurrence, for example. Event data and timing data are stored according to address progression as shown. This performance information can be stored in the RAM 3 or the external storage device 9.

  FIG. 3 is a flowchart showing the processing of the CPU 1 regarding the model creation phase. In step S <b> 1, a screen for prompting selection of music is displayed on the display device 8. If a song is selected (Yes at step S2), the process proceeds to step S3, and the performance information of the selected song is scanned to detect the range of the song. This is because a single tone model is created in this range. The tone range for creating a single tone model does not have to be in accordance with music selection. For example, the performer may input the range directly, or the screen may be displayed so that the skill level can be selected instead of the music selection (described later with reference to FIG. 5). Further, the musical range of the selected music may be displayed for reference when the performer directly inputs the musical range (described later with reference to FIG. 6).

  In step S4, the pitch of the single tone model to be recorded is determined. First, the lowest pitch in the above range is determined as the recorded pitch. In step S5, the recorded pitch is displayed on the display device 8 as a key pressing instruction for single-tone model recording (referred to as “model key pressing instruction” in distinction from the key pressing instruction in the lesson phase) (described later with reference to FIG. 7). . For the model key pressing instruction, the musical tone data corresponding to the pitch may be created using the sound source device 4 without using the display device 8 and may be generated by the speaker 5. Further, the instruction by pronunciation may be performed together with the display by the display device 8.

  In step S6, the sound volume measurement by the sound volume measuring unit 11 is started. Volume measurement continues until a key release instruction is issued. In step S7, it is determined whether or not the sound volume exceeds a threshold for attack detection. If this determination is affirmative, it is determined that an attack has been detected, and the process proceeds to step S8. In step S8, a power spectrum is calculated by FFT, and the power spectrum and the volume at the time of power spectrum calculation are temporarily stored in the RAM 3.

  In step S9, the display device 8 is instructed to release the key. The key release instruction may be a display on the display device 11 or a sound or voice instruction from the speaker 5. In step S10, it is determined whether or not the volume is equal to or lower than a threshold value for detecting silence. If the volume falls below this threshold, it is determined that the sound is substantially silent, and the process proceeds to step S11.

  In step S11, it is determined whether the maximum value of the volume from the model key press instruction to the key release instruction is within the allowable range. If the maximum volume is not within the allowable range, it is determined that the sound has been played extremely strongly or extremely weakly and is not suitable as a single-tone model, so the process proceeds to step S5 to request a key press again. Rather than judging from the sound volume when it is considered that an attack has been detected, judging whether or not the key-pressing strength is appropriate based on the maximum value of the measured sound volume does not necessarily mean that the sound volume is maximum at the time of the attack. This is because the sound volume normally reaches the maximum value after detecting the attack. Note that the key pressing strength may be determined by the maximum value of the amplitude instead of determining by the volume.

  If it is determined that the key has been correctly pressed, the process proceeds to step S12, and the power spectrum and the volume temporarily stored in step S8 are stored in the RAM 3 or the external storage device 9 in correspondence with the single tone model of the pitch.

  If step S11 is negative, a message may be displayed so that the key-pressing strength is changed and then the process proceeds to step S5. For example, when the volume is higher than the allowable range, “Please play weaker” is displayed, and when the volume is lower than the allowable range, “Play more strongly” is displayed.

  In step S13, it is determined whether or not a single tone model of the final tone (highest tone) determined in step S3 has been created. If no single tone model of the final tone has been created, the pitch is set to 1 in step S14. After increasing the level, go to step S5. In step S14, it is preferable to raise the pitch so that the white key and the black key can be pressed individually separately, instead of raising the pitch to the pitch immediately above according to the scale. This is because if the pitch is raised in order according to the scale, the white key and the black key may be played alternately, and an error in playing a key that does not correspond to the model key pressing instruction is likely to occur.

  If a single tone model is created up to the final sound, step S13 becomes affirmative and the process of this model creation phase is terminated.

  FIG. 4 is a flowchart showing the processing of the CPU 1 regarding the lesson phase. In step S21, the single tone model stored in step S12 is read. Here, the latest memorized single note model or the single note model used in the previous lesson may be automatically read, or the memorized single note model is displayed on the display device 8 and the one selected by the performer is read. But you can. In step S22, a screen for prompting selection of a song to be lesson is displayed on the display device 8. This screen is created as a song list based on lesson song data stored in advance in the RAM 3 or the external storage device 9. Of the stored lesson songs, only the songs corresponding to the skill level of the performer may be displayed.

  In step S23, it is determined whether or not a lesson song is selected. If a song to be lesson is selected (Yes in step S23), the process proceeds to step S24, and the performance information of the selected song is stored in the RAM 3 or the external storage device 9. Read from.

  In step S25, it is determined whether or not all the pitches in the performance information of the selected song are included in the range of the selected single-tone model. If this determination is affirmative, the process proceeds to step S26. If negative, the process returns to step S22 to prompt the user to select a new song. Note that it may be possible to return to the music selection, or to select the single tone model again.

  In step S26, the pitch of the first musical tone of the selected lesson song is set in the RAM 3 as a variable. In step S27, an instruction (key pressing instruction) is given so that the key corresponding to the pitch can be played. The key pressing instruction is performed by displaying a keyboard or a score on the display device 8 as will be described in detail later. In step S28, one corresponding to the pitch of the key depression instruction is read out from the single tone model.

  In step S29, the power spectrum of the input sound from the microphone 10 is calculated by the function of the spectrum creating unit 12. In step S30, it is determined whether or not an attack has already been detected. The normal attack is detected only at the moment when the sound rises, and the power spectrum at the time of the attack detection does not always match the single tone model. Therefore, after the attack is detected once, while the same musical tone is being generated, the second and subsequent attack detections are avoided by the determination in step S30. Initially, the determination in step S30 is negative, and this determination is affirmative after the second time, and the process proceeds to step S31 to perform attack detection. Attack detection is determined by whether or not the power increase in the power spectrum exceeds a predetermined amount. If an attack is detected, step S32 becomes affirmative, an attack detection flag is set in step S33, and the process proceeds to step S34.

  In step S34, the function of the spectrum comparison unit 14 determines whether or not the power spectrum calculated in step S29 matches the single tone model. This determination is made based on the total distance between the power spectrum of the single tone model and the power spectrum calculated in step S29. That is, the mutual distance is calculated for each frequency component, and it is determined whether or not both spectra match depending on whether or not the sum exceeds a threshold value. A simple comparison of both power spectra may be used, or a comparison of power spectra normalized according to the difference in volume between each other may be used. The Euclidean distance should be used. This is because, according to the Euclidean distance, the difference between the two values is squared, and as a result, the difference is emphasized, and it is easy to determine whether the two values match.

  When comparing the power spectra, the amount of power spectrum input from the spectrum creating unit 12 is calculated for each frequency component to determine how much the power spectrum level is lower than the power spectrum level of the single tone model. May be calculated. This is more convenient for instruments such as pianos that have a lot of reverberation.

  If the two power spectra do not match, the process returns to step S29. If the two power spectra match, the process proceeds to step S35 to reset the attack detection flag.

  In step S36, it is determined whether or not the last musical tone of the performance information has been matched with the single tone model. If the comparison processing with the single tone model for the last musical tone is not completed, the pitch of the musical tone data next to the performance information is set as a variable in the RAM 3 in step S37, and the process proceeds to step S27.

  When the comparison processing for the last musical tone is completed, the process proceeds to step S38 to display whether or not to perform the lesson for the same song again, or to proceed to step S39 to select a lesson for another song. Display the screen to let the player choose whether to do it.

  If step S38 is positive, the process proceeds to step S26, and if step S39 is positive, the process proceeds to step S22. If step S36 and step S39 are negative, the lesson phase ends.

  FIG. 5 is a diagram showing a display example on the display device 8 for setting a sound range corresponding to the skill level on the display screen. In FIG. 5, a keyboard graphic K is displayed, and on this keyboard graphic K, “Introductory”, “Beginner”, “Intermediate”, and “Advanced” indicating the skill level, and each skill level correspond to each. The range is displayed with arrows. If any one of these skill levels is selected, the range of the illustrated range corresponding to each level is set. For example, if an instruction is input by placing the cursor on each character indicating the skill level on this screen, the illustrated range corresponding to each level is set as the lesson range.

  FIG. 6 is a diagram showing an example of a screen that is displayed so as to be a reference for input in the range setting. In FIG. 6, among the keys of the displayed keyboard figure K, the key 16 shown in a color different from the white key and the black key is a key corresponding to the pitch included in the selected lesson song. Since it cannot be colored in the figure, the key 16 is given a rough dot pattern to distinguish it from other black keys and white keys. The performer can set the range to include all of these keys 16 with reference to the pitch display. For example, any two keys may be indicated on the display screen, and a range of pitches where these keys are the lowest and highest sounds, respectively, may be set as the lesson range.

  FIG. 7 is a diagram illustrating a display example of a model key pressing instruction. In FIG. 7, the keyboard figure K displayed on the display device 8 indicates the keyboard position to be pressed for creating a single tone model by using a key 17 with a color or pattern so as to be distinguished from other keys. Yes. The performer can press an actual piano key according to the key 17 on the displayed keyboard figure K. Note that the mark 18 shown on the white key in the drawing is for easily recognizing the key pressing position. By attaching a seal or the like to the actual piano key corresponding to the same pitch as the key indicated by the mark 18 prior to the creation of the single tone model, the displayed keyboard figure K and the actual piano keyboard It becomes easy to deal with.

  FIG. 8 is a diagram showing an example of a musical score with a key pressing instruction drawn on the display screen of the display device 8, and is used in the lesson phase. In this example, the score G is displayed in two stages on the display screen 81 of the display device 8. The number of display stages is not limited to two as long as it is plural, and the present invention can also be applied to the case where a score is displayed one by one on one screen as will be described later. The number of bars displayed in one row is not limited to four bars, and the width of the bars can be arbitrarily determined according to the width of the display screen 81. Further, when displaying a score on a plurality of levels, the number of bars to be displayed may be different for each level.

  In FIG. 8, a scroll bar 82 is displayed on the right side of the display screen 81 as an operation unit for displaying a score that is not currently displayed on the display screen 81. On the display screen 81, a score G corresponding to the position of the scroll bar 82 is displayed. That is, in FIG. 8, the first part (eight bars) of the whole score is displayed, and when the scroll bar 82 is moved downward, the currently displayed 8 of the score of the selected lesson song are displayed. The part after the 9th bar appears after the bar. The scroll bar 82 is used when the entire musical score is viewed by operating before starting the performance, but the function is stopped in the lesson mode, that is, when the performance is actually performed. Therefore, in the lesson mode, the display position of the scroll bar 82 is fixed at the initial position, the current position, or deleted.

  On the display screen 81, a key press instruction NP (performance instruction symbol) indicating the position of a note to be played next by a solid vertical line, and an accompaniment instruction SP (timing symbol) indicated by a dotted vertical line indicating the current position of automatic accompaniment ) Is displayed. The initial position of the accompaniment instruction SP is on the left side of the screen 81, and gradually moves to the right side when the key pressing instruction operation is started. The accompaniment instruction SP approaches the key pressing instruction NP together with, for example, a prelude played with a metronome sound until the first key pressing timing of the first measure. The key press timing is when the accompaniment instruction SP approaches the key press instruction NP at a predetermined performance speed and the accompaniment instruction SP overlaps the key press instruction NP. Therefore, the performer plays the first sound on the piano P. If the power spectrum of the performance sound matches the model, the key pressing instruction NP moves to the position of the note to be pressed next. On the other hand, since the accompaniment instruction SP moves at a predetermined performance speed, the accompaniment instruction SP catches up with the key pressing instruction NP at a next key pressing timing. The key pressing instruction proceeds by repeating this operation.

  When the power spectrum of the performance sound does not match the model, the automatic accompaniment and accompaniment instructions wait without moving. The automatic accompaniment standby process will be described later.

  If the score cannot be displayed on one screen, the score at the end of the performance is displayed by rewriting it with a score subsequent to the currently displayed score. In particular, the present embodiment adopts the following score rewriting method for minimizing the player's eye movement. While the last stage of the score displayed on the display screen 81, that is, in the example of FIG. 8, the second stage is being played, the next stage, that is, the third stage not shown in FIG. Is drawn on the display screen 81. The third-stage score is displayed at the position where the first-stage score is currently displayed. That is, the score on the upper level is switched to the score on the next level following the score currently displayed on the lower level.

  For example, when the lower stage performance is started, the rewriting of the score is started from the top part of the upper stage. The rewriting is gradually performed following the movement of the key pressing instruction NP. In other words, when the lower performance is started, the upper score is rewritten, and when the lower performance is finished, the upper score is rewritten. In other words, the rewriting of the score proceeds so that the current position of the key press instruction NP and the rewriting position match the horizontal coordinate (x coordinate) on the screen 81.

  Note that not only the rewriting of the upper stage is started when the lower stage performance is started, but the score of at least the ninth bar is displayed while the lower stage performance is performed, that is, until the end of the eighth bar performance. 81 only needs to be displayed. However, it is preferable that the next musical score is displayed at least before the performance of the last measure in the lower row is started so that the music player can see the musical score in advance.

  Further, the rewrite position and the display position of the key press instruction NP are not limited to coincide with each other in the x coordinate. For example, the rewrite position is pressed so that both maintain a predetermined interval (number of measures). The display position of the instruction NP may be followed. After the lower performance is finished and the display is moved to the newly displayed upper performance, the rewriting position and the display position of the key press instruction NP are maintained at the predetermined interval, and the score continues to the end of the song. Can be rewritten.

  The rewriting of the score may be gradually progressed as described above, or the score of one step may be switched at a time. For example, when the upper stage finishes and the lower stage starts, the upper score can be quickly rewritten with a new score.

  FIG. 9 and FIG. 10 are diagrams showing examples of the key press display while the musical score is being rewritten. In FIG. 9, as indicated by the accompaniment instruction SP, the performance is currently advanced to the seventh bar, and the key pressing instruction NP is at the position of the second note of the seventh bar. An “eraser mark” 83 indicating the score rewriting position is displayed on the upper score so that the display position of the key press instruction NP and the X coordinate coincide. Here, the first measure of the upper score is rewritten with the score of the tenth measure.

  As described above, the musical score may be rewritten every time one performance is finished according to the movement of the key pressing instruction NP as described above, or the musical score is rewritten one measure every time the performance is finished. May be. In the case of rewriting gradually, when the lower score is finished, all the upper score is rewritten.

  Then, by the end of the performance of the stage including the tenth bar, all of the lower score are switched to a new score. When the musical score is rewritten, the lower musical score is rewritten as in the upper musical score, as shown in FIG.

  In this way, the performer only has to follow the display on the screen 81 in the same manner as when performing according to the score written on normal paper. That is, the eyes are moved from the left to the right and the top to the bottom of the display screen 81, the score is traced to the right end of the bottom, and then the eyes are moved to the left again.

  FIG. 11 shows an example in which a score is displayed in one stage. This is used when the vertical width of the display screen is narrow and multistage display is not suitable. In this example, when the performance of the second measure is started, the original first measure is rewritten with the score of the fifth measure. In this way, when the performance of one bar is finished, the bar is immediately rewritten to the corresponding part of the next stage.

  The rewrite start timing of the score is not limited to the example of FIG. In the one-stage display, the rewriting of the score is started when the key pressing instruction NP advances to half of the stage, and the rewriting is advanced while maintaining the interval between the rewriting position and the position of the key pressing instruction NP. May be.

  9 to 11, the rewriting position of the score is based on the position of the key pressing instruction NP, but instead of this key pressing instruction NP, the rewriting of the score is started based on the position of the accompaniment instruction SP. It may be advanced.

  FIG. 12 is a flowchart showing a key pressing instruction process. In step S51, the total performance time is cleared. The total performance time is the time until the actual performance. In step S52, the key press instruction display position of the first note of the song is set. In step S53, the performance is started and the update (increment) of the total performance time is started. In step S54, the key pressing instruction NP is drawn at the position set in step S52.

  In step S55, the performance position, that is, the display position of the accompaniment instruction SP is calculated based on the current total performance time, and the accompaniment instruction SP is displayed at that position. In step S56, the performance sound is compared with the model. Details of this comparison have been described above. In step S57, it is determined whether or not the power spectrum of the performance sound matches the model. If the power spectrum of the performance sound matches the model, the process proceeds to step S58. If not, the process proceeds to step S62.

  In step S62, it is determined whether or not the total performance time exceeds the time corresponding to the position where the key depression instruction NP is present over the scheduled time. If step S62 is negative, the process proceeds to step S55. If it is determined in step S62 that the total performance time exceeds the time corresponding to the position where the key depression instruction NP is present, the update of the total performance time is stopped in step S63, and the process proceeds to step S55.

  If the spectra match, the process proceeds from step S57 to step S58 to determine whether or not the update of the total performance time is stopped. If the update of the total performance time has stopped, the process proceeds to step S59, and the update of the total performance time is resumed. If the update of the total performance time has not stopped, the process proceeds from step S58 to step S60, and it is determined whether or not the key pressing instruction has been completed for all the notes. This determination is initially negative, and the process proceeds to step S61 to advance the display position of the key press instruction NP to the position of the next note, and then proceeds to step S54. If step S60 is positive, the process of this flowchart is terminated.

  FIG. 13 is a flowchart showing processing for displaying a score on a single screen in a plurality of stages. In this example, the musical score is rewritten at intervals so that the display coordinates (x coordinate) in the horizontal direction between the key press instruction NP and the rewriting progress position coincide.

  In step S71, a score for one screen is displayed. For example, an image of a note or a symbol is displayed superimposed on a staff score image. In step S72, the variable Xp is cleared. This variable Xp is used for prefetching performance information and for storing how far the drawing has been performed, that is, for storing the position of the eraser mark 83. In step S73, the variable Bp is cleared. The variable Bp is used to store the key pressing instruction level at the previous drawing in order to correct the value of the variable Xp depending on which stage the key pressing instruction NP was previously issued.

  In step S74, it is determined whether or not the key pressing instruction NP is in the last stage of the song. If the key depression instruction NP is in the last stage, the process of this flowchart is terminated. The process proceeds to step S75 until the key pressing instruction NP proceeds to the final stage.

  In step S75, it is determined whether or not the variable Bp indicates the current key press instruction step, that is, whether or not the step where the key press instruction NP is displayed has not changed from the previous time. If the variable Bp is not the current key pressing instruction stage, that is, if the key pressing instruction stage has advanced, the process proceeds to step S76 and the variable Xp is cleared. As a result, the drawing start position for prefetching performance information is returned to the left end of the screen. If the variable Bp indicates the current key press instruction stage, step S76 is skipped and the process proceeds to step S77.

  In step S77, it is determined whether or not the key pressing instruction NP is located at the lowest level. If this determination is negative, the process proceeds to step S78. In step S78, a note from the coordinate Xp to the same x coordinate value as that of the key press instruction NP is drawn in a stage one level lower than the stage where the key press instruction NP is present.

  If the key pressing instruction NP is located at the lowest level, the process proceeds to step S79, and a note from the coordinate Xp to the same x coordinate value as the key pressing instruction NP is drawn on the uppermost level of the display screen.

  In step S80, the x coordinate of the current key pressing instruction NP is stored in the variable Xp. In step S81, the stage where the key pressing instruction NP is drawn is stored in the variable Bp.

  FIG. 14 is a flowchart for displaying a score one by one on one screen. In this process, the musical score is rewritten at intervals of half the stage from the display of the key press instruction NP. That is, when the key pressing instruction proceeds to the center of the column, the currently displayed score starts to be rewritten from the top. Then, when the key pressing instruction NP returns to the top of the step, the rewriting position, that is, the position of the eraser mark 83 is processed so as to advance to half of the step.

  In step S91, the score for one stage is displayed. In step S92, the variable Xp is cleared. This variable Xp is used to store how much drawing has been performed in order to prefetch performance information. In step S93, the flag F is cleared (set to zero). The flag F indicates where the key pressing instruction NP at the time of the previous drawing is located with respect to the center of the step. In order to correct the variable Xp when the key pressing instruction NP is on the right side of the center of the step. used. In step S94, it is determined whether or not the x coordinate of the current key pressing instruction NP is right from the center of the step. If the x coordinate of the key press instruction NP is not right from the center of the step, zero is set to the flag F in step S95. In step S96, it is determined whether the key pressing instruction NP is the first stage of the song. If there is a key pressing instruction NP at the first level, the score for that level has already been drawn at step S91, and in this case, the process returns to step S94.

  If the x coordinate of the key pressing instruction NP is right from the center of the step, the process proceeds from step S94 to step S97, and it is determined whether or not the flag F is zero. When step S94 is affirmative and step S97 is affirmative, it is determined that the key pressing instruction NP has moved from the center left of the step to the right. When the key pressing instruction NP moves from the center left to the right of the step, the process proceeds to step S98. If the flag F is not zero, it is determined that the key pressing instruction NP has not shifted from the right to the left from the center of the step, and the process proceeds to step S101.

  In step S98, it is determined whether or not the key pressing instruction NP is the first stage of the song. If the key pressing instruction NP is not the first stage of the music, the process proceeds to step S99, and notes from the coordinate Xp to the right end are drawn. When the key pressing instruction NP moves from the left side to the right side of the step, the position of the eraser mark 83, that is, the rewriting position is moved from the right side to the left side of the step, so that the right end from the position of the previous eraser mark 83 (coordinate Xp). This is a process for reliably terminating the drawing up to. In step S100, the variable Xp is cleared and the position of the eraser mark 83 is positioned at the head of the stage. Since the position of the eraser mark 83 is the initial position when the key pressing instruction NP moves to the right of the stage at the first stage of the music, the process jumps from step S98 to step S100.

  In step S101, “1” is set to a flag F indicating that the key pressing instruction NP is on the right side of the stage. In step S102, it is determined whether or not the key pressing instruction NP has reached the last stage of the song. If the key depression instruction NP is at the last stage of the song, there is no musical score to be drawn next, and the process is terminated.

  If the key pressing instruction NP is not the last stage of the music, the process proceeds to step S103, and notes from the coordinate Xp to the x coordinate value obtained by adding half the value of the stage to the x coordinate of the key pressing instruction NP are currently displayed. Draw in a row. In step S104, the half value of the step is added to the x coordinate of the current key press instruction NP and stored in the variable Xp.

  If it is determined in step S96 that the key pressing instruction NP is not the first level, that is, indicates the second or subsequent score, the process proceeds to step S103.

Accompaniment is performed at the same timing as the accompaniment instruction described above. The performance information for accompaniment is supplied from the ROM 2 or the external storage device 9 together with the performance information of the practice music. Performance information for accompaniment is read according to the accompaniment counter. The accompaniment is interrupted when the performance is performed as instructed by the key depression, that is, when step S57 is negative.
Judging whether or not the performance has been performed correctly takes time for detection and comparison of the power spectrum by FFT. Accordingly, if the accompaniment is interrupted when the accompaniment instruction SP overlaps the key press instruction NP, the accompaniment is interrupted for the FFT processing time. In the present embodiment, since the accompaniment counter is not stopped until it is determined that the performance of the accompaniment has not been performed correctly, the accompaniment is not interrupted when the performance is performed correctly in accordance with the key depression instruction.

  Note that whether or not to stop the accompaniment is not limited only to whether or not the performance has been performed correctly. After the accompaniment has progressed to the timing at which the key should be depressed, a counting processing routine for counting the scheduled time is provided, and the accompaniment can be stopped when there is no determination result that the performance has been performed correctly within the scheduled time. If it is determined that the performance has been performed correctly within the scheduled time, the accompaniment is not interrupted.

  Also, if this scheduled time is less than the minimum note length to be lesson, or less than the time to the next note to be played (that is, the step time), the accompaniment will not be playing the next note and the timing of that note The inconvenience that accompaniment progresses earlier can be prevented, and the player does not feel uncomfortable. Furthermore, it is preferable that the scheduled time can be arbitrarily designated by the performer.

  The processes according to the above-described flowcharts and modifications are programmed so as to be executed by the CPU 1 and can be provided by being recorded on a known recording medium such as a CD or a ROM.

  In the above embodiment, a single tone model is created based on the performance sound of the piano P that is actually used for the lesson, and the single tone model and the performance sound of the piano P are compared. As a result, even if the piano P is not tuned correctly, it is determined by comparing the performance sounds of the same piano P, so that it is surely determined whether or not the piano P can be played according to the key pressing instruction.

  However, the present invention is not limited to this, and can also be applied to a case where a single tone model created separately from the performance sound of another piano is read from the external storage device 9 and compared with the power spectrum of the performance sound of the piano P.

  Further, the present invention is not limited to piano key depression determination, and can be applied to performance sound determination of various natural musical instruments such as an organ, an accordion, a harmonica, a recorder, and a guitar.

It is a block diagram which shows the structure of the lesson system of the piano which concerns on one Embodiment of this invention. It is a figure which shows the example of a format of performance information. It is a flowchart which shows the process of CPU regarding a model creation phase. It is a flowchart which shows the process of CPU1 regarding a lesson phase. It is a figure which shows the example of a display on a display apparatus for setting the sound range according to a skill level on a display screen. It is a figure which shows the example of the screen displayed so that it may become reference of the input in the range setting. It is a figure which shows the example of a display of a model key press instruction | indication. It is a figure which shows the example of a score accompanying the key pressing instruction drawn on the display screen. It is a figure which shows the example of the key press display in the middle of the score being rewritten. It is a figure which shows the example of a key press display after a musical score is rewritten. It is a figure which shows the example of the key press instruction | indication which displayed the score in 1 step | paragraph. It is a flowchart which shows a key press instruction | indication process. It is a flowchart which shows the process which displays a score in multiple steps on one screen. It is a flowchart which shows the process which displays a score one step at a time on one screen.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... CPU, 2 ... ROM, 4 ... Sound source device, 5 ... Speaker, 6 ... A / D converter, 7 ... Input device, 8 ... Display device, 9 ... External storage device, 10 ... Microphone, 11 ... Volume measurement part 12 ... Spectrum creation unit, 13 ... Attack detection unit, 14 ... Spectrum comparison unit, 15 ... Key press instruction unit

Claims (18)

Display means for displaying a score including a plurality of measures, a performance instruction symbol indicating a note to be played next on the score, and a timing symbol indicating a performance timing;
A performance sound determining means for determining whether or not the musical note indicated by the performance instruction symbol has been correctly played at the timing indicated by the timing symbol;
Display control means for advancing the performance instruction symbol to the position of a note to be played next when it is determined that the performance sound determination means has performed correctly;
Means for rewriting the currently displayed score from the beginning with the score following the currently displayed score when the display of at least one of the performance timing symbol and the performance instruction symbol has advanced to a predetermined time And a performance instruction device.   2. The performance instruction according to claim 1, wherein the rewriting of the musical score is performed while maintaining a display position and a rewriting progress position of at least one of the performance timing symbol and the performance instruction symbol at a predetermined interval. apparatus.   3. The scheduled interval is determined so that display coordinates in a horizontal direction of a display position of at least one of the performance timing symbol and the performance instruction symbol and a rewriting progress position coincide with each other. The performance instruction device described. The score is displayed on a single screen with a plurality of stages including a plurality of bars arranged vertically.
2. The performance instruction device according to claim 1, wherein the predetermined time is set to a performance timing of any measure displayed in at least the second and subsequent levels. The score is displayed on a single screen,
3. The performance instruction apparatus according to claim 1, wherein the predetermined time is a time when the display of at least one of the performance timing symbol and the performance instruction symbol has advanced to half of the stage. Accompaniment means for automatically accompaniing according to scheduled performance information;
Means for temporarily stopping the automatic accompaniment when the performance sound judging means cannot obtain a judgment result that the music has been correctly performed even if the timing at which the notes indicated by the performance instruction symbols are played exceeds a predetermined time. The performance instruction device according to claim 1, wherein the performance instruction device is provided.   7. The performance instruction apparatus according to claim 6, wherein the scheduled time is set to be less than a minimum note length among notes included in the score. 7. The performance instruction apparatus according to claim 6, wherein the scheduled time is set to be less than a step time until a note to be played next. The performance sound judging means is
FFT means for performing FFT processing on digital data of a natural musical instrument performance sound played in accordance with a performance instruction and calculating a power spectrum of the digital data;
The distance between the power spectrum calculated for each FFT processing and the pitch included in the performance instruction among the power spectra of the pitches of the natural musical instrument sounds stored in advance is within a predetermined threshold. And comparison means for determining whether or not
The performance instruction device according to claim 1, wherein when the distance is within a predetermined threshold, it is determined that the performance has been correctly performed. Display means for displaying a score including a plurality of measures, a performance instruction symbol indicating a note to be played next on the score, and a timing symbol indicating a performance timing;
A performance sound determining means for determining whether or not the musical note indicated by the performance instruction symbol has been correctly played at the timing indicated by the timing symbol;
Display control means for advancing the performance instruction symbol to the position of a note to be played next when it is determined that the performance sound determination means has performed correctly;
Means for rewriting the currently displayed score from the beginning with the score following the currently displayed score when the display of at least one of the performance timing symbol and the performance instruction symbol has advanced to a predetermined time A performance instruction program that causes a computer to function as   11. The performance instruction program according to claim 10, further comprising a program for rewriting the musical score while maintaining a display interval and a rewriting progress position of at least one of the performance timing symbol and the performance instruction symbol at a predetermined interval.   The performance instruction program according to claim 11, wherein the scheduled interval includes a program in which display coordinates in a horizontal direction of a display position and a rewriting progress position of at least one of the performance timing symbol and the performance instruction symbol coincide. The score is displayed on a single screen with a plurality of stages including a plurality of bars arranged vertically.
The performance instruction program according to claim 10, further comprising a program for setting the predetermined time as a performance timing of each measure displayed in at least the second and subsequent levels. The score is displayed on a single screen,
The performance instruction program according to claim 10 or 11, wherein the predetermined time includes a program that is set when display of at least one of the performance timing symbol and the performance instruction symbol has advanced to half of the stage. Accompaniment means for automatically accompaniing according to scheduled performance information;
A computer as a means for temporarily stopping automatic accompaniment when a result of determination that the performance sound determination means has performed correctly cannot be obtained even if the timing at which the note indicated by the performance instruction symbol is played exceeds a scheduled time. The performance instruction program according to any one of claims 10 to 14, which causes the function to function.   The performance instruction program according to claim 15, further comprising a program for setting the scheduled time to be less than the length of a minimum note among the notes included in the score. The performance instruction program according to claim 15, further comprising a program for setting the scheduled time to be less than a step time until a note to be played next. The performance sound judging means is
FFT means for performing FFT processing on digital data of a natural musical instrument performance sound played in accordance with a performance instruction and calculating a power spectrum of the digital data;
The distance between the power spectrum calculated for each FFT processing and the pitch included in the performance instruction among the power spectra of the pitches of the natural musical instrument sounds stored in advance is within a predetermined threshold. Comprising a program that causes a computer to function as a comparison means for determining whether or not
The performance instruction program according to any one of claims 10 to 17, wherein when the distance is within a predetermined threshold, it is determined that the performance has been correctly performed.

Images