JP2005234303A - Performance sound decision apparatus and performance sound decision program and recording medium with the program recorded therein - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce an attack detecting delay of a performance sound of a piano inputted from a microphone in a performance sound decision apparatus, a performance sound decision program and a recording medium which are suitable to detect the attack (rise of a sound of a musical instrument) as basic information of musical sound decision. <P>SOLUTION: A spectrum generating part 12 calculates a power spectrum by performing the FFT processing of digital data of the performance sound of the piano played according to a playing instruction for every FFT window. An attack detecting part 13 decides whether there is the attack of the performance sound or not depending on whether the power of the calculated power spectrum is increased for more than a scheduled value from that of the calculation of the last time. A spectrum comparing part 14 decides whether a distance between the calculated power spectrum and that of a pitch which is included in the playing instruction among power spectra for every pitch in which power spectra calculated after the decision of the presence of the attack are preliminarily stored is near or not. Moreover, the apparatus is equipped with a means which shifts mutual windows so that front and rear FFT windows are partially overlapped with each other in the FFT processing. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a performance sound determination system for a natural musical instrument, and more particularly to a performance sound determination apparatus, a performance sound determination program, and a recording medium suitable for detecting an attack (rising of a musical instrument sound) as basic information for musical sound determination.

  For example, as disclosed in Japanese Patent Publication No. 2-705, an electronic musical instrument having a key pressing instruction unit for self-studying the performance of a keyboard instrument is known. In this electronic musical instrument, stored performance information is sequentially read out to a display circuit, and a display lamp provided corresponding to each key of the keyboard is turned on according to this performance information. An operation key by the performer is detected by a key scanning circuit, and when the pitch of the operation key matches the pitch of the performance information indicated by the display lamp, the next performance information is read out to the display circuit.

  However, such a key pressing instruction means can be easily realized with an electronic musical instrument provided with a key scan circuit, but is difficult to realize with a natural musical instrument such as an acoustic piano without a key scan circuit. Although it is conceivable to provide a key scan circuit for a natural musical instrument, the work of retrofitting key switches and display lamps necessary for the key scan circuit for each key is complicated and cannot be performed easily.

  Instead of using a key scan circuit, it is conceivable to detect an operation key by frequency analysis of musical sounds by performance. For example, in the display device described in Japanese Patent Publication No. 1-51200, the pitch of the audio signal determined based on the frequency analysis result of the audio signal is displayed as a musical note on a staff score on the display.

  However, when detecting the pitch of a musical tone that contains a lot of harmonic components such as piano sounds, the pitch may be misjudged by the harmonic component of the key press, that is, the harmonic component, or two chords with an octave interval and (For example, discrimination between a chord of C3 and C4 and a C3 single tone) may not be possible. This is because these harmonic structures are similar to each other.

  In addition, an apparatus for generating a MIDI code corresponding to the performance of a musical instrument that generates a plurality of fundamental sounds and their harmonics simultaneously has been proposed (Japanese Patent No. 2890831). This device is provided with means for extracting waveform data of frequency components corresponding to the fundamental tone from the waveform data of the performance music sound and detecting the envelope data, while corresponding to the envelope waveform for each fundamental tone generated by the instrument. A storage means for storing the fundamental tone envelope data is provided, and the detected envelope data and the stored fundamental tone envelope data are compared with each other by cross-correlation to determine the instrument sound. This apparatus has a very complicated configuration, and in particular, it is difficult to process envelope data comparison and instrument sound determination in real time with software.

Japanese Patent No. 2880647 proposes a device that compares a signal created by sequentially combining various instrument sounds with various pitches by a synthesizer with an input signal, and recording from musical sounds. However, in this apparatus, since it is necessary to compare with all combinations of sounds including chords, there is a problem that calculation time becomes enormous.
Japanese Patent Publication No. 2-705 Japanese Patent Publication No. 1-51200 Japanese Patent No. 2890831 Japanese Patent No. 2880647

  As described above, in an acoustic piano that does not have a key scanning means, it is not possible to determine whether or not the key has been depressed as instructed, and it is not possible to determine the accuracy of the key depression in real time from the musical sound input from the microphone. There was a problem that it was not easy.

  Therefore, the applicant stores the power spectrum of each pitch as a single-tone model, compares the power spectrum of the sound played according to the key-pressing instruction with this single-tone model, and determines the distance between the two power spectra. In this case, a method has been proposed in which a key pressing instruction is advanced (Japanese Patent Application No. 2003-134372). This method is simpler to calculate than parameters such as pitch from the performance sound, and can play a musical instrument to be played and create a model based on that sound. Whether or not the performance sound is correct can be determined regardless of the deviation.

  In this method, it is preferable to compare the power spectra of the attacks to determine whether or not the performance sound is correct. Attack detection can be performed by performing Fast Fourier Transform (FFT) on the input sound from the microphone, and how much the detected power spectrum is increased with respect to the power spectrum detected last time.

  The FFT is performed for each predetermined FFT window length, and when an attack is not detected within the window time, the FFT is performed again within the next window time. However, since the FFT window time is long, the detection of an attack is delayed if processing is performed by dividing each window length in this way. In particular, when multiple sounds are played quickly like an arpeggio, attack detection is not in time. For example, in the case of the highest sound of an acoustic piano, the duration of the sound is only about 40 milliseconds, but in the case of an FFT of 1024 points at 11025 Hz, the window time is about 93 milliseconds, and the moment of attack cannot be captured.

  In view of the above-described problems, the present invention provides a performance sound determination system for a natural instrument that can reliably detect an attack necessary for determining whether a natural instrument such as a piano has been played as instructed. Objective.

  In order to solve the above-described problems and achieve the object, the present invention calculates the power spectrum of the digital data by performing FFT processing for each FFT window on the digital data of the natural musical instrument performance sound performed according to the performance instruction. FFT means for performing an attack detection means for determining whether or not there is an attack of the natural musical instrument performance sound based on whether or not the power of the power spectrum calculated for each FFT processing has increased more than expected from the previous calculation time; The distance between the power spectrum calculated after it is determined that the attack has occurred and the power spectrum for each pitch of the natural musical instrument sound stored in advance from the pitch included in the performance instruction is scheduled. The comparison means for determining whether or not the threshold value is within the threshold value and the FFT window of the FFT means are set so that the preceding and following FFT windows partially overlap each other. There is a first feature in that and means.

  Further, the present invention displays the performance instruction on a display device of a computer in accordance with performance information of a preselected song, and when the distance is within a predetermined threshold, the next performance is in accordance with the performance information. A second feature is that a key pressing instruction means for displaying an instruction is provided.

  In the present invention, a shift time for determining an overlap period between the FFT windows is set in advance, and the preceding FFT means is processed by the time when processing of the subsequent FFT means and attack detection means is started. A third feature is that the apparatus further includes means for extending the shift time when the processing of the attack detection means does not end.

  Further, the present invention provides a high-speed unit detecting means for detecting a predetermined phrase having a high performance speed based on the performance information in advance, and the attack detecting means for a performance sound of a performance instruction corresponding to the phrase. There is a fourth feature in that the processing by the comparison means is omitted, and the judgment by the comparison means is executed for the input sound after the performance instruction regardless of the presence or absence of the attack.

  According to the first feature, it is determined that there has been an attack when the amount of increase in the power spectrum of the musical instrument sound performed in accordance with the performance instruction is greater than or equal to a predetermined amount. Then, when the power spectrum of the musical instrument performance sound matches the one of the pitches included in the performance instruction among the power spectra of the natural musical instrument sounds stored in advance, it is determined that the musical instrument performance sound was played as instructed. The performance instruction is advanced.

  In particular, since the FFT window is set so as to partially overlap in the attack determination, the calculation related to the next FFT window following the calculation related to the previous FFT window is newly performed by the amount of the length of the FFT window. Even if it is not input, it can be executed if data for the time shifted to partially overlap the FFT window is input after the previous calculation. Therefore, an attack appearing immediately after the preceding FFT window is executed within a short time following the calculation related to the previous FFT window, without waiting for the next input of data corresponding to the FFT window length. Detected in the next FFT window.

  According to the second feature, the performance instruction is displayed on a display device of a computer, and the performance instruction is advanced when the power spectrum of the performance sound substantially matches the power spectrum of the natural musical instrument sound stored in advance. The

  According to the third feature, since the shift time for determining the overlap time of the FFT window can be adjusted according to the capability of the computer, it is possible to determine whether or not there is an attack as fast as possible within the capability of the computer. .

  For performances such as arpeggios, where the performance speed is high, comparing the power spectrum after determining the presence or absence of an attack does not catch up with the player's fast performance. According to the fourth feature, in such a fast performance, it is possible to determine the coincidence of power spectra without determining the presence or absence of an attack.

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram of a lesson system of an acoustic piano (hereinafter, simply referred to as “piano” unless otherwise distinguished from an electronic piano) according to an embodiment of a musical tone determination apparatus of the present invention. In this lesson system, key press instructions including piano key or pitch instructions and key press timing instructions are displayed based on pre-prepared performance information, and the performer sequentially advances the performance according to the key press instructions. To go. Specific processing consists of a model creation phase and a lesson phase. In the model creation phase, prior to the lesson, the power spectrum of the generated sound of each piano key used in the lesson is stored as a single tone model. In the lesson mode, a key press instruction is displayed (the key press instruction will be described later with reference to FIG. 12), and the power spectrum of the performance sound performed according to the key press instruction is compared with the single tone model. If the power spectrum is within the planned distance, the key pressing instruction is advanced.

  The lesson system shown in FIG. 1 includes a personal computer including a CPU 1, ROM 2, RAM 3, sound source device 4, speaker 5, A / D converter 6, input device (keyboard or mouse) 7, and display device 8. can do. As the display device 8, a known display means for displaying the processing result of the personal computer, such as a liquid crystal display or a cathode ray tube, can be used. An external storage device 9 is connected to the personal computer, and a microphone 10 is connected via the A / D converter 6. The microphone 10 is provided for collecting the sound generated by the piano P, and is preferably arranged in the piano P.

  The CPU 1 includes a volume measuring unit 11, a spectrum creating unit 12, an attack detecting unit 13, a spectrum comparing unit 14, and a key pressing instruction unit 15 as main functions. The volume measuring unit 11 detects the level (volume) of the sound input from the microphone 10. The spectrum creating unit 12 has an FFT function for obtaining a power spectrum from a digital musical tone signal input through the microphone 9 and the A / D converter 6. The power spectrum obtained by the FFT is stored in the RAM 3. The attack detection unit 13 detects an attack based on the detected power spectrum. The spectrum comparison unit 14 compares the distances of the power spectra created by the spectrum creation unit 12 in the model creation phase and the lesson phase, and determines whether or not the performance has been performed according to the key depression instruction. The key pressing instruction unit 15 creates display data for displaying a key pressing instruction on the display device 8 according to the performance information. The display data is created in response to a performance start instruction input from the input device 7 and input to the display device 8.

  FIG. 2 is a format example of performance information used for displaying a key pressing instruction on the display device 8 and performing accompaniment during a lesson. The performance information stores at least event data and timing data for instructing the reading timing of the event data. The event data is composed of note-on data and note-off data including a note number. The timing data is set as time information from the end of one event to the next event occurrence, for example. Event data and timing data are stored according to address progression as shown. This performance information can be stored in the RAM 3 or the external storage device 9.

  FIG. 3 is a flowchart showing the processing of the CPU 1 regarding the model creation phase. In step S <b> 1, a screen for prompting selection of music is displayed on the display device 8. If a song is selected (Yes at step S2), the process proceeds to step S3, and the performance information of the selected song is scanned to detect the range of the song. This is because a single tone model is created in this range. The tone range for creating a single tone model does not have to be in accordance with music selection. For example, the performer may input the range directly, or the screen may be displayed so that the level of skill can be selected instead of the music selection (described later with reference to FIG. 9). Further, the musical range of the selected music may be displayed for reference when the performer directly inputs the musical range (described later with reference to FIG. 10).

  In step S4, the pitch of the single tone model to be recorded is determined. First, the lowest pitch in the above range is determined as the recorded pitch. In step S5, the recorded pitch is displayed on the display device 8 as a key pressing instruction for single-tone model recording (referred to as “model key pressing instruction” in distinction from the key pressing instruction in the lesson phase) (described later with reference to FIG. 11). . For the model key pressing instruction, the musical tone data corresponding to the pitch may be created using the sound source device 4 without using the display device 8 and may be generated by the speaker 5. Further, the instruction by pronunciation may be performed together with the display by the display device 8.

  In step S6, the sound volume measurement by the sound volume measuring unit 11 is started. Volume measurement continues until a key release instruction is issued. In step S7, it is determined whether or not the sound volume exceeds a threshold for attack detection. If this determination is affirmative, it is determined that an attack has been detected, and the process proceeds to step S8. In step S8, a power spectrum is calculated by FFT, and the power spectrum and the volume at the time of power spectrum calculation are temporarily stored in the RAM 3.

  In step S9, the display device 8 is instructed to release the key. The key release instruction may be a display on the display device 11 or a sound or voice instruction from the speaker 5. In step S10, it is determined whether or not the volume is equal to or lower than a threshold value for detecting silence. If the volume falls below this threshold, it is determined that the sound is substantially silent, and the process proceeds to step S11.

  In step S11, it is determined whether the maximum value of the volume from the model key press instruction to the key release instruction is within the allowable range. If the maximum volume is not within the allowable range, it is determined that the sound has been played extremely strongly or extremely weakly and is not suitable as a single-tone model, so the process proceeds to step S5 to request a key press again. Rather than judging from the sound volume when it is considered that an attack has been detected, judging whether or not the key-pressing strength is appropriate based on the maximum value of the measured sound volume does not necessarily mean that the sound volume is maximum at the time of the attack. This is because the sound volume normally reaches the maximum value after detecting the attack. Note that the key pressing strength may be determined by the maximum value of the amplitude instead of determining by the volume.

  If it is determined that the key has been correctly pressed, the process proceeds to step S12, and the power spectrum and the volume temporarily stored in step S8 are stored in the RAM 3 or the external storage device 9 in correspondence with the single tone model of the pitch.

  If step S11 is negative, a message may be displayed so that the key-pressing strength is changed and then the process proceeds to step S5. For example, when the volume is higher than the allowable range, “Please play weaker” is displayed, and when the volume is lower than the allowable range, “Play more strongly” is displayed.

  In step S13, it is determined whether or not a single tone model of the final tone (highest tone) determined in step S3 has been created. If no single tone model of the final tone has been created, the pitch is set to 1 in step S14. After increasing the level, go to step S5. In step S14, it is preferable to raise the pitch so that the white key and the black key can be pressed individually separately, instead of raising the pitch to the pitch immediately above according to the scale. This is because if the pitch is raised in order according to the scale, the white key and the black key may be played alternately, and an error in playing a key that does not correspond to the model key pressing instruction is likely to occur.

  If a single tone model is created up to the final sound, step S13 becomes affirmative and the process of this model creation phase is terminated.

  FIG. 4 is a flowchart showing the processing of the CPU 1 regarding the lesson phase. In step S21, the single tone model stored in step S12 is read. Here, the latest memorized single note model or the single note model used in the previous lesson may be automatically read, or the memorized single note model is displayed on the display device 8 and the one selected by the performer is read. But you can. In step S22, a screen for prompting selection of a song to be lesson is displayed on the display device 8. This screen is created as a song list based on lesson song data stored in advance in the RAM 3 or the external storage device 9. Of the stored lesson songs, only the songs corresponding to the skill level of the performer may be displayed.

  In step S23, it is determined whether or not a lesson song is selected. If a song to be lesson is selected (Yes in step S23), the process proceeds to step S24, and the performance information of the selected song is stored in the RAM 3 or the external storage device 9. Read from.

  In step S25, it is determined whether or not all the pitches in the performance information of the selected song are included in the range of the selected single-tone model. If this determination is affirmative, the process proceeds to step S26. If negative, the process returns to step S22 to prompt the user to select a new song. Note that it may be possible to return to the music selection, or to select the single tone model again.

  In step S26, the pitch of the first musical tone of the selected lesson song is set in the RAM 3 as a variable. In step S27, an instruction (key pressing instruction) is given so that the key corresponding to the pitch can be played. The key pressing instruction is performed by displaying a keyboard or a score on the display device 8 as will be described in detail later. In step S28, one corresponding to the pitch of the key depression instruction is read out from the single tone model.

  In step S29, the power spectrum of the input sound from the microphone 10 is calculated by the function of the spectrum creating unit 12. In step S30, it is determined whether or not an attack has already been detected. The normal attack is detected only at the moment when the sound rises, and the power spectrum at the time of the attack detection does not always match the single tone model. Therefore, after the attack is detected once, while the same musical tone is being generated, the second and subsequent attack detections are avoided by the determination in step S30. Initially, the determination in step S30 is negative, the determination is affirmative after the second time, and the process proceeds to step S31 to perform attack detection. Attack detection is determined by whether or not the power increase in the power spectrum exceeds a predetermined amount. If an attack is detected, step S32 becomes affirmative, an attack detection flag is set in step S33, and the process proceeds to step S34.

  In step S34, the function of the spectrum comparison unit 14 determines whether or not the power spectrum calculated in step S29 matches the single tone model. This determination is made based on the total distance between the power spectrum of the single tone model and the power spectrum calculated in step S29. That is, the mutual distance is calculated for each frequency component, and it is determined whether or not both spectra match depending on whether or not the sum exceeds a threshold value. A simple comparison of both power spectra may be used, or a comparison of power spectra normalized according to the difference in volume between each other may be used. The Euclidean distance should be used. This is because, according to the Euclidean distance, the difference between the two values is squared, and as a result, the difference is emphasized, and it is easy to determine whether the two values match.

  When comparing the power spectra, the amount of power spectrum input from the spectrum creating unit 12 is calculated for each frequency component to determine how much the power spectrum level is lower than the power spectrum level of the single tone model. May be calculated. This is more convenient for instruments such as pianos that have a lot of reverberation.

  If the two power spectra do not match, the process returns to step S29. If the two power spectra match, the process proceeds to step S35 to reset the attack detection flag.

  In step S36, it is determined whether or not the last musical tone of the performance information has been matched with the single tone model. If the comparison processing with the single tone model for the last musical tone is not completed, the pitch of the musical tone data next to the performance information is set as a variable in the RAM 3 in step S37, and the process proceeds to step S27.

  When the comparison processing for the last musical tone is completed, the process proceeds to step S38 to display whether or not to perform the lesson for the same song again, or to proceed to step S39 to select a lesson for another song. Display the screen to let the player choose whether to do it.

  If step S38 is positive, the process proceeds to step S26, and if step S39 is positive, the process proceeds to step S22. If step S36 and step S39 are negative, the lesson phase ends.

  Next, processing that can more reliably perform attack detection will be described. FIG. 5 is a diagram showing the input timing of input data and the position of the FFT window. Attack detection is performed on the sound data input from the microphone 10 for each of the FFT windows thus divided.

  FIG. 6 is a diagram illustrating a relationship between input timing of input data and processing timing. For example, when input data is started to be accumulated from timing t0 and the amount necessary for FFT is accumulated at timing t1, calculation is started for the input data, and the calculation is terminated at timing t2 to determine whether or not the attack is an attack. Is made. If no attack is detected, the calculation is started again at timing t3 after waiting for data to be collected between timings t1 to t3, and the calculation is completed at timing t4 to determine whether or not the attack is being performed. . According to this calculation timing, for example, an attack that occurs between timings t1 and t2 is detected at timing t4, that is, at a point that is approximately delayed by the length of the FFT window from the time of the actual attack.

  Therefore, such a delay in attack detection is eliminated as follows. FIG. 7 is a diagram showing the relationship between the input timing of input data that can eliminate the delay in attack detection and the processing timing. In FIG. 7, the FFT processing is started on the data input between the timing t0 and the timing t1 at the timing t1, and the FFT window time immediately before the timing t20 at the time t20 shifted by the time ΔL from the timing t1. FFT processing is started for input data at time T corresponding to. The time ΔL is a preset value, and is set to, for example, a quarter length of the FFT window. The presence or absence of an attack based on this FFT processing is detected at timing t21 after the processing time ΔT has elapsed since the start of calculation. That is, an attack that occurs immediately after timing t1 is detected at timing t21. Similarly, the FFT processing is performed on the input data at the time when the FFT window is sequentially shifted by the predetermined shift time ΔL, such as time 2ΔL, 3ΔL. Therefore, the attack can be detected in a shorter time than the process in which the attack is detected at the timing t4 as in the example described with reference to FIG.

  FIG. 8 is a diagram showing the input timing of the input data and the position of the FFT window according to the processing of FIG. 7 in which the FFT processing is performed by shifting the FFT window to eliminate the attack detection delay.

  If the FFT processing takes time depending on the processing capability of the CPU 1, that is, if the processing time ΔT started from the timing t1 is longer than the shifted time (shift time) ΔL, the shift ΔL time is the same as the processing time ΔT. Alternatively, it is extended to be longer than the processing time ΔT. On the other hand, when the processing time ΔT is longer than the shift time ΔL, the shift ΔL time can be shortened in accordance with the processing time ΔT. Thus, attack detection can be performed as quickly as possible in accordance with the FFT processing time.

  In addition, since an attack can be detected only once in one FFT process, when there is an attack twice in one FFT window, for example, in a fast phrase such as an arpeggio, an attack of a later sound is detected. I can't. In this case, the key pressing instruction does not proceed. Therefore, for such a fast phrase, the attack detection can be skipped and the process can proceed. Whether it is an arpeggio or the like can be detected by providing a processing routine for pre-determining performance information in advance. That is, the performance information is read in advance, and the position (occurrence timing) of a phrase having a high performance speed such as an arpeggio is stored in advance. In the process for the musical sound at the phrase generation timing, if the power spectrum is calculated in step S29 in FIG. 4, the process immediately proceeds to the comparison with the single tone model (step S34) regardless of whether an attack is detected.

  The threshold value for increasing the power spectrum used to determine whether or not there is an attack may be linked according to the volume stored in the model creation phase together with the power spectrum. If an attack is not detected even after repeating the attack detection process (steps S31 and S32), a threshold setting screen may be displayed on the display device 8 to prompt the player to adjust the threshold value.

  After detecting the attack, it is possible to check whether the peak of the next sound to be played is standing in the power spectrum, and to prevent the attack from being detected with an extra sound.

  FIG. 9 is a diagram showing a display example on the display device 8 for setting a sound range corresponding to the skill level on the display screen. In FIG. 9, a keyboard figure K is displayed, and on this keyboard figure K, “Introductory”, “Beginner”, “Intermediate”, and “Advanced” indicating the skill level correspond to each skill level. The range is displayed with arrows. If any one of these skill levels is selected, the range of the illustrated range corresponding to each level is set. For example, if an instruction is input by placing the cursor on each character indicating the skill level on this screen, the illustrated range corresponding to each level is set as the lesson range.

  FIG. 10 is a diagram illustrating an example of a screen that is displayed so as to be a reference for input when setting a sound range. In FIG. 10, among the keys of the displayed keyboard figure K, a key 16 shown in a color different from the white key and the black key is a key corresponding to the pitch included in the selected lesson song. Since it cannot be colored in the figure, the key 16 is given a rough dot pattern to distinguish it from other black keys and white keys. The performer can set the range to include all of these keys 16 with reference to the pitch display. For example, any two keys may be indicated on the display screen, and a range of pitches where these keys are the lowest and highest sounds, respectively, may be set as the lesson range.

  FIG. 11 is a diagram illustrating a display example of a model key pressing instruction. In FIG. 11, the keyboard figure K displayed on the display device 8 indicates the keyboard position to be pressed for creating a single tone model by the key 17 having a color or pattern so that it can be distinguished from other keys. Yes. The performer can press an actual piano key according to the key 17 on the displayed keyboard figure K. Note that the mark 18 shown on the white key in the drawing is for easily recognizing the key pressing position. By attaching a seal or the like to the actual piano key corresponding to the same pitch as the key indicated by the mark 18 prior to the creation of the single tone model, the displayed keyboard figure K and the actual piano keyboard It becomes easy to deal with.

  FIG. 12 is a diagram illustrating an example of a key pressing instruction in the lesson phase. In the figure, the key is pressed by a piano roll, that is, a scroll display method. A keyboard figure K is displayed on the display screen 81 of the display device 8, and a key pressing instruction mark M is displayed above the keyboard figure K. The display screen 81 has a time axis at the top and bottom, and the distance between the key pressing instruction mark M and the keyboard figure K corresponds to the time until the key below the key pressing instruction mark M is pressed. That is, the key below the key pressing instruction mark M closer to the keyboard figure K is a key to be played earlier. The key depression instruction mark M is displayed at a position corresponding to the key to be depressed, with a length corresponding to the note length, and is scrolled downward when the power spectrum of the performance sound and the single tone model match each other. The key to be currently pressed is indicated by a mark m. As indicated by the bar line BL, in this example, the key pressing instructions for two bars are simultaneously displayed on one screen, but the size of the screen to be displayed simultaneously, that is, the range of the performance information to be displayed is arbitrary. A performance instruction apparatus that simultaneously displays a key depression instruction mark and a keyboard figure based on performance information and scrolls the key depression instruction mark is disclosed in, for example, Japanese Patent Application Laid-Open No. 9-305171.

  The display format of the key pressing instruction is not limited to the method of scrolling the mark with respect to the keyboard figure, but may be a score format, that is, a method of scrolling the notes displayed on the staff. In addition, an indicator with LEDs (light emitting diodes) arranged at positions corresponding to each key along the keyboard of the piano P is installed, and the LED corresponding to the key to be pressed is lit to display the key pressing instruction. It may be a thing.

  The processes according to the flowcharts and modifications shown in FIGS. 3 and 4 are programmed so as to be executed by the CPU 1 and can be provided by being recorded on a known recording medium such as a CD or a ROM.

  In the above embodiment, a single tone model is created based on the performance sound of the piano P that is actually used for the lesson, and the single tone model and the performance sound of the piano P are compared. As a result, even if the piano P is not tuned correctly, it is determined by comparing the performance sounds of the same piano P, so that it is surely determined whether or not the piano P can be played according to the key pressing instruction.

  However, the present invention is not limited to this, and can also be applied to a case where a single tone model created separately from the performance sound of another piano is read from the external storage device 9 and compared with the power spectrum of the performance sound of the piano P.

  Further, the present invention is not limited to piano key depression determination, and can be applied to performance sound determination of various natural musical instruments such as an organ, an accordion, a harmonica, a recorder, and a guitar.

It is a block diagram which shows the structure of the lesson system of the piano which concerns on one Embodiment of this invention. It is a figure which shows the example of a format of performance information. It is a flowchart which shows the process of CPU regarding a model creation phase. It is a flowchart which shows the process of CPU1 regarding a lesson phase. It is a figure which shows the relationship between the input timing of input data, and the position of an FFT window. It is a figure which shows the relationship between the input timing of input data, and a processing timing. FIG. 10 is a diagram (part 2) illustrating a relationship between input timing of input data and processing timing that can eliminate a delay in attack detection; It is a figure which shows the relationship between the input timing of the input data which concerns on the example which can eliminate the delay of attack detection, and the position of an FFT window. It is a figure which shows the example of a display on a display apparatus for setting the sound range according to a skill level on a display screen. It is a figure which shows the example of the screen displayed so that it may become reference of the input in the range setting. It is a figure which shows the example of a display of a model key press instruction | indication. It is a figure which shows an example of the key pressing instruction | indication in a lesson phase.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... CPU, 2 ... ROM, 4 ... Sound source device, 5 ... Speaker, 6 ... A / D converter, 7 ... Input device, 8 ... Display device, 9 ... External storage device, 10 ... Microphone, 11 ... Volume measurement part 12 ... Spectrum creation unit, 13 ... Attack detection unit, 14 ... Spectrum comparison unit, 15 ... Key press instruction unit

Claims (9)

FFT means for calculating the power spectrum of the digital data by performing FFT processing for each FFT window on the digital data of the natural musical instrument performance sound played in accordance with the performance instruction;
Attack detection means for determining whether or not there is an attack of the natural musical instrument performance sound based on whether or not the power of the power spectrum calculated for each FFT processing has increased more than expected from the previous calculation time;
The power spectrum calculated after it is determined that the attack has occurred is stored in advance, and the distance from the pitch included in the performance instruction among the power spectra for each pitch of the natural musical instrument sound is scheduled. A comparison means for determining whether the value is within the threshold;
A performance sound judging device comprising: means for shifting the FFT window of the FFT means so that the front and rear FFT windows partially overlap each other.   The key press for displaying the performance instruction on the display device of the computer according to the performance information of the pre-selected song and displaying the next performance instruction according to the performance information when the distance is within a predetermined threshold. 2. The performance sound determination apparatus according to claim 1, further comprising an instruction means.   A shift time for determining an overlap period between the FFT windows is set in advance, and the FFT means and attack detection means of the preceding FFT means and attack detection means are started by the time when processing of the subsequent FFT means and attack detection means is started. The performance sound determination device according to claim 1, further comprising means for extending the shift time when the processing is not completed. Based on the performance information, high-speed part detecting means for detecting a predetermined phrase having a high performance speed in advance;
For the performance sound of the performance instruction corresponding to the phrase, the processing by the attack detection means is omitted,
4. The performance sound determination apparatus according to claim 1, further comprising means for executing the determination by the comparison means for an input sound after a performance instruction regardless of whether there is an attack. FFT means for calculating the power spectrum of the digital data by performing FFT processing for each FFT window on the digital data of the natural musical instrument performance sound played in accordance with the performance instruction;
Attack detection means for determining whether or not there is an attack of the natural musical instrument performance sound based on whether or not the power of the power spectrum calculated for each FFT processing has increased more than expected from the previous calculation time;
The power spectrum calculated after it is determined that the attack has occurred is stored in advance, and the distance from the pitch included in the performance instruction among the power spectra for each pitch of the natural musical instrument sound is scheduled. A comparison means for determining whether the value is within the threshold;
A performance sound determination program for causing a computer to function as means for shifting the FFT windows of the FFT means so that the front and rear FFT windows partially overlap each other.   The key press for displaying the performance instruction on the display device of the computer according to the performance information of the pre-selected song and displaying the next performance instruction according to the performance information when the distance is within a predetermined threshold. 6. The performance sound determination program according to claim 5, further comprising a program for causing a computer to function as the instruction means.   A shift time for determining an overlap period between the FFT windows is set in advance, and the FFT means and attack detection means of the preceding FFT means and attack detection means are started by the time when processing of the subsequent FFT means and attack detection means is started. The performance sound determination program according to claim 5 or 6, further comprising a program for causing a computer to function as means for extending the shift time when processing is not completed. Based on the performance information, high-speed part detecting means for detecting a predetermined phrase having a high performance speed in advance;
For the performance sound of the performance instruction corresponding to the phrase, the processing by the attack detection means is omitted,
The performance sound determination program according to any one of claims 5 to 7, further comprising a program for causing the computer to function as a means for executing the determination by the comparison means on the input sound after the performance instruction regardless of whether or not there is an attack. .   A computer-readable recording medium on which the performance sound determination program according to any one of claims 5 to 8 is recorded.

Images