WO2004088630A1

WO2004088630A1 - Method of reading music and computer-readable recording medium having music-reading program recorded therein

Info

Publication number: WO2004088630A1
Application number: PCT/JP1998/002806
Authority: WO
Inventors: Seiji Nakano
Original assignee: Seiji Nakano
Priority date: 1997-07-09
Filing date: 1998-06-24
Publication date: 2004-10-14

Abstract

A method of reading a music comprising the step of recognizing all signs of a music including musical notes, the step of estimating the drum notation in a drum part of the music based on the data obtained in the above step, the step of assigning a practical sounding instrument to the sign of the recognized drum part according to the drum notation estimated in the above step, and the step of converting the data into the form of music data that can be played and/or displayed. This makes it possible to recognize the notation of the drum music described by various notations and to convert the recognized data into a form that can be played with a minimum amount of operation.

Description

A combination of a score reading method and a score reading program. A readable recording medium.

Technical field

Rice field

The present invention relates to a score reading method for reading drum scores among scores, and a computer-readable recording medium storing a program for reading scores. ·

Background art

If the score is to be recognized mechanically, the staff should be recognized. After recognizing the paragraphs as shown in Fig. 29, the staff should be erased and various methods could be used. ), The notehead of a note such as the white ball w shown in FIG. 4 (b), the suffix t following the notehead shown in FIG. ) Recognizes the flag f and others that follow the suffix.

On the other hand, the notation of the drum set, which is the notation of the drum set, is not fixed due to historical factors of musical instruments and factors in performance.

The most common notation does not exist, so if you only want to recognize a particular notation, you lose the versatility of drum notation recognition.

On the other hand, if various notations are to be recognized, simply recognizing the notation as written cannot identify which drum instrument (drum sound) the musical note is to be played on. I couldn't play the drum score.

Also, when trying to recognize drum staves without being bound by a particular notation as described above, in order to play the recognition result correctly, the symbols (including notes) are actually used. When doing so, you have to specify what kind of drum sound (drum instrument) it is. It is possible for the user to make such a designation, but in that case, the user must be familiar with the notation of the drum score. In addition, a complicated interface is used for the specification, and the specification operation is also complicated, a large number of corrections are required, and the total correction cost is increased.

In addition, even if the notation type is specified before recognizing the drum notation and recognition is performed based on the specified notation type, and correction after recognition is unnecessary, the user is also required to specify the notation type. You will need accurate knowledge.

These problems have hindered conventional music score recognition technology from supporting drum scores.

On the other hand, among the note heads displayed on the drum score, the notes other than the black and white balls, which are the same as in the normal score, are written in thin lines such as (χ) and (◊). Exists. For this reason, bold and fine separation (refer to Japanese Patent Application Laid-Open No. Hei 9-97060), which is usually used for musical note recognition, separates the parts that make up thin lines from image data, Detects and detects thick symbols from the rest of the image, and recognizes symbols that appear on the score.) Ellipse detection (black and white balls used for note heads of ordinary notes using elliptical expressions) Can not be detected.

In addition, there are other problems that cannot be solved by the conventional method, such as tie-joining to noteheads and chords in which two or more noteheads touch vertically. Ordinary scores, such as the position where noteheads are joined to stems depending on the score, or noteheads in the middle of stems that break the stems. There is no problem.

The present invention has been made in view of the above-mentioned problems of the prior art, and a first object of the present invention is to recognize notation of a drum notation described in various notations and to perform data with a small number of operations. It does not provide a configuration that can be converted to a format. A second object of the present invention is to provide a configuration that enables recognition of drum notes in a drum score different from a normal score and that can convert the data into a playable and / or displayable data format. It is. Disclosure of the invention

Therefore, the configuration of the invention according to claim 1 includes a symbol recognition step of recognizing all symbols of a musical score including a note, and an estimation of a drum notation in a drum part of the musical score based on information obtained in the step. A musical notation estimating step of performing the following, a musical instrument assigning step of assigning an actual sounding musical instrument to the recognized symbol of the drum part in accordance with the drum notation estimated in the step, and a performance and Z or a displayable musical score data format. And a data conversion step for conversion. The configuration of the recording medium according to claim 9 provides a score reading program for executing these functions as a computer-readable recording medium. That is, a symbol recognition function for recognizing and recognizing all symbols in a score including notes, a notation estimation function for estimating a drum notation in a drum part of a score based on information obtained by the function, An instrument assignment function for assigning an actual sounding instrument to a recognized drum part symbol according to the drum notation estimated by the function;

A score reading program for executing a data conversion function of converting to a displayable score data format is provided as a configuration recorded on the recording medium.

When recognizing the drum score, it is not possible to uniquely assign the drum sound (drum instrument) from the recognition of the symbol on the score. Therefore, according to the configuration of the present invention, after performing all the symbol recognition of the score, the drum notation is once estimated based on the information, and then the drum sound is assigned, and the performance and / or display can be performed. It is decided to convert to data format.

In the score reading method (recording medium), the symbols in the drum part of the score recognized by the symbol recognition step (function) include a note head of a drum note, a stem, a flag relating to a flag, a drum note, and a drum score. In recognizing the symbol related to the drum note, the relation between the symbol related to the drum note and the drum note is also detected. As described above, accurate reading cannot be made until the score of the drum part has been read until all the symbols on the score have been recognized.Therefore, at least these symbols must be read before estimation. It is necessary to accurately recognize the relationship between recognition and symbols. Of course, drum notation estimation In this case, in addition to the accurate recognition of these symbols and their relations, the recognition of the position of the staff of the dram note is also an important judgment factor, as described later.

When the drum notation is estimated by the notation estimating step (function), at least the position of the staff of the drum head, the type of drum head, and the drum obtained by the symbol recognition step (function). A hi-hat open symbol for a note, a hi-hat close symbol for a drum note, an accent symbol for a drum note, a suffix of a drum note, a flag attached to a drum note, a character string for specifying the drum sound of a note, the drum Determined by the flag of the note. The pitch is determined based on these information including another pitch determined by the type of notehead given the presence of the length of the other drum note determined by the flag.

This is because the display position (pitch) of the staff changes depending on the type of cymbal-type drum instrument and other drum instruments. (However, the position of some instruments is not fixed. There is also a different drum note head (although the same one may be used, some of which have the same drum note head and staff position). Therefore, depending on the position of the drum notehead on the staff, the range of estimation of the drum instrument is narrowed, and the validity of the estimation is increased to some extent depending on the type of drum notehead.

In the cymbal system, if the drum note head is a hollow diamond type (◊), does it refer to a half note or whole note, or to a specific cymbal instrument? May not be immediately determined. Similarly, there are cases where the drum note head is of the (X) type, although the judgment cannot be made immediately.

In these cases, if there is a hi-hat open symbol (°) related to a drum note, it is always a hi-hat, and the note head of the drum note with the symbol may be a hi-hat. However, even if the same notehead does not have this symbol, it may indicate a crash cymbal or a top cymbal, as well as a note hat close.

If accents according to note drum ^(>) (Λ), generally it will be understood to imply the accented on the note, sometimes pointing crash cymbal. In this case, the type of notehead with accents and the It can be judged to some extent from the staff position of the head.

In such a case, if a character string for designating the drum sound of a note is attached to the character string, the character string takes precedence, and other symbols may be estimated therefrom.

Furthermore, if the above-mentioned diamond-shaped notehead has a stem and a flag attached to the stem, it is understood that the flag indicates a note length such as an eighth note or a sixteenth note (drum note flag). Length determined by In that case, the notehead does not refer to a half note or whole note, but to a particular drum instrument, and may also determine the duration of other drum notes ( The other note lengths determined by the flag determine the different note lengths depending on the notehead type.)

Therefore, estimation of the drum notation is performed based on information such as the above-mentioned symbols.

When the actual notation of drum notation for drum instruments other than cymbals is estimated by the above notation estimation process (function), the notehead is between the third staff members, and black balls and other types are detected. If it is, a black ball notehead is a snare drum, (R) or a notehead other than a black ball is a rim shot, and the notehead is between the third staff and the black ball If only note heads are detected, and if a character string representing a rim shot is detected, all black ball heads are rim shots, and other black ball heads are snares. Drum. Snare drum and rim shots will both be shown with noteheads located between the third of the staff. Even in such a case, the snare dram is usually indicated by a notehead with a black dot, and the rim shot may be a separate notehead such as (R), or the rim shot may be specified as a character string with the black dot. Therefore, the above estimation holds.

When performing the actual estimation of drum notation for drum instruments other than cymbals by the above notation estimation step (function), if the notehead is below the first line of the staff and is not a black ball, The drum note is used as a pedal hi-hat.

Normally the pedal hi-hat is written as a notehead (X) between the lower first. In some cases, it may be shifted downward to avoid the force drum. In consideration of the possibility of misrecognition, as described above, notes other than black balls below the first line are pedal hi-hats.

According to the notation estimation step (function), a command for drum instruments other than cymbals can be obtained. When performing the actual estimation of the ram notation, if the notehead is a black ball and the black ball is detected in the range indicated as a bass drum or tom, excluding the third interval, a sound with a bottom row If it is below the pitch, it is used as a bass drum, and the rest is assigned to toms of multiple pitches. If the bottom row is higher than a certain pitch, all are assigned to toms of multiple pitches.

Normally, bass drums and toms are indicated by black dots, bass drums are indicated by the first interval, and toms are indicated by 1 to 4 pitches from the second to fifth lines excluding the third line. However, there is also a notation in which the bass drum is written in the lower part. In such a case, there is a high possibility that the Low tom is written in the first interval (that is, the same pitch as a normal bass drum). In this way, the pitch of the bass drum changes the tom range, so the first step detects the pitch where the black ball exists, including the bass drum pitch. Specifically, as described above, the black ball is detected in the range between the first line and the fifth line below the staff except for the third line, and the black ball is detected. If the lower row is between the lower first interval and the first interval, the drum note indicating the lowest pitch of the pitch where the black ball exists is the bass drum, and the pitch one level higher than it is The drum note that indicates the Low tom, the drum note that indicates the top row of the pitch is the High pitch, and the pitch between the Low tom and the High pitch except for the third interval. Let the drum note be the Mid tom. If the notehead is a black ball and the black ball is detected in the range of the second and fifth lines of the staff excluding the third interval, the drum note indicating the lowest pitch is a low tom, The drum note that indicates the top pitch is the high tom, and the drum note that indicates the pitch between the low to high and the high pitch other than the third interval is the mid tom.

In the above notation estimation step (function), when estimating the drum notation for a cymbal-based drum instrument, the drum note head other than the black ball should be at the staff position of the fourth space or higher. Drum note heads are divided into cases according to one staff position and two staff positions, and the above cases are subdivided according to the type of drum note head, and the duration determined by the flag of the drum note The above-mentioned case division is further subdivided in accordance with another note length determined by the notehead type due to the presence of the duration of the other drum note obtained by the flag, and a conversion table in accordance with these case divisions To create the drum staff head position, the type of drum note head, the hi-hat open symbol for the drum note, the hi-hat close symbol for the drum note, and the accent for the drum note The type of cymbal-based drum instrument to be estimated is specified in the conversion table in accordance with the character string for designating the drum sound of symbols and notes, and the drum notation is estimated based on the conversion table. Do.

Drum notes related to cymbal-type drum instruments are indicated by drum heads other than black balls at staff positions in the fourth and higher spaces. Even in this case, there are two cases: one is indicated by one pitch (drum note head has one staff position), and the other is indicated by two pitches (drum note head has two staff positions). Exists.

Also, depending on the type of cymbal-based drum instrument, it is often written with different drum note heads. For example, half notes and whole notes are likely to be crash cymbals due to the characteristics of musical instruments. Noteheads (◊) have a high probability of being written as half-notes or whole notes in the history of drum staves. When whole notes are present, half notes are likely to be written with the same notehead as whole notes. Cymbal-type drum notes are not likely to be represented by three or more pitches. When two pitches are used, the lower row is the hi-hat, the upper row is the crash cymbal, and the top cymbal. If both the top and bottom cymbals are written in the upper row, the note head of the top cymbal is represented by (X) and the crash cymbal is written in any other way (such as changing the note head). Furthermore, if there is only one kind of sympano (other than hi-hat cymbals), it is a crash cymbal. Those with a note head (〇) are indicated as hi-hat open, and the hi-hat without the sign (°) [or with the sign ( ⁺ )] is hi-hat closed.

Of course, the note length is determined by the flag of the drum note, the other note length determined by the length of the other drum note determined by the flag, and another note length determined by the notehead type. The head may turn out to be different from the initially estimated cymbal instrument. For example, a notehead (◊) exists, and before estimation, the notehead (◊) is treated as a whole note, and a notehead (◊) with a stem is treated as a half note. If there is an eighth note or a sixteenth note with a stem and a flag on (◊), then the notehead (◊) with a stem is actually a quarter note, and therefore a notehead. A note with (◊) may be recognized as a top cymbal.

Therefore, as mentioned above, the drum staff has one staff position and two The above case is further divided according to the type of drum note, and the above-mentioned eighth note and sixteenth note for the note length (頭) determined by the flag of the drum note. Etc.], and the other note lengths determined by the flag (the above eighth note, 16th note, etc.) have different note lengths [note head (◊), etc.]. The above cases are further subdivided according to the quarter note with a suffix at the notehead (◊), and a conversion table is created according to these cases, and the drum notehead Depending on the line position, the type of drum note head, the hi-hat open symbol for drum notes, the hi-hat close symbol for drum notes, the accent symbol for drum notes, and the character string for specifying the drum sound of notes, Converts the estimated cymbal-type drum instrument type Keep identified in Bull. Then, the drum notation is estimated based on the conversion table.

As the final step, the actual pronunciation instrument is assigned to the symbol of the recognized drum part in accordance with the drum notation estimated by the notation estimation step (function). If the value of the conversion table specifies a half note for a drum note recognized as, these drum notes are converted to half notes. This is because the notation changes the note length of each drum note head.

Further, in order to achieve the second object, the configuration of claim 17 is a musical score for reading an image of a musical score, recognizing the musical symbol, and performing and converting the musical symbol into a displayable musical score format. In the reading method, use the stem information to erase the bits at the stem.

(Hereinafter, erasing refers to erasing the bit plane of the image, which is distinguished from erasing, which erases the data itself on the recognition result storage data.) After that, the separated labels around the detected stem candidates are matched with the dictionaries to recognize the drumhead symbols, and then the combining process of the recognized noteheads and candidate stems is performed. And

As mentioned above, note heads of drum staves are written with thin lines such as (X) and (◊) for notes other than black and white balls, and there are various other types as well. It cannot be detected by the thick / fine separation method or ellipse detection method that is usually used for musical note recognition. Therefore, as in the case of ordinary symbol recognition, the separated and extracted labels are used as a dictionary. Recognition is performed by matching. However, because of stems, it is not possible to separate note heads as a single label. In addition, since there are consecutive hooks and chords, it is impossible to create a dictionary that includes stems because the pattern becomes enormous. Therefore, in this configuration, using the stem information, the label of the note head is once separated, and the separated label is matched with the dictionary so that it can be recognized in the same way as ordinary symbol recognition. At this time, the scanhead candidates separated from the noteheads are scanned, and the drum noteheads are recognized with the labels around the notehead candidates as matching targets. Is performed.

The above configuration is the same for a computer-readable recording medium that stores the score reading program according to claim 27. The configuration includes a label separation function for separating note head labels using stem information. A scanning function for scanning the stems, and a recognition function for recognizing drumheads by recognizing noteheads by matching the separated labels around the candidate stems detected by the scanning with a dictionary. It has a program for executing a combining function for combining the recognized notehead and the stem candidate.

In the above configuration, if the matching between the separated label and the dictionary fails, the scan of the candidate stem is continued, the next detected note head label is combined with the separated label, and then the matching with the dictionary is performed. Should be performed again. This means that in the case shown in Fig. 1, erasing the stem does not cause a problem in recognizing the notehead, but in the case shown in Fig. 2, after erasing the stem, the notehead is removed. This is because they are separated into multiple labels. In such a case, the matching that is performed first fails, so after performing label combining and performing matching again, notehead recognition becomes possible. The above configuration is the same for a computer-readable recording medium that stores the score reading program according to claim 28. The configuration includes a matching judgment function for judging the success or failure of the matching between the separation label and the dictionary. If the matching fails, the scan function is continued to scan for stem candidates, and then a label combining function for combining the detected notehead label and the separation label is executed. It further has a program, and after the label combination, causes the recognition function to perform the matching between the combined label and the dictionary again.

The above-described label combining and re-matching processing performed after dictionary matching fails. Even if it is performed, the notehead label itself that was detected and used for the combining process next should be matched with the dictionary, and if the matching result has a higher priority, the recognition result should be given priority. .

The above-described configuration is the same for a computer-readable recording medium that stores the score reading program of claim 29, and the configuration includes the notehead label itself detected and used for the combining process next. The dictionary is matched by the recognition function, and if the matching result has a higher priority, the recognition result is given priority.

In the score reading method according to claim 18, label combining is repeatedly performed, and priorities are given to the respective recognition results recognized and matched during the combining, and the recognition results are determined in descending order of priority. Good. During label merging, the shape may be the same as another notehead before it becomes a recognition rectangle that includes all the noteheads that should be recognized, so if matching is performed, priority should be given to the recognition results . If the recognition result has a low priority, the ID and rectangle are saved, and the next label scan is performed. If an ID with a higher priority is not found, this is given priority. If the recognition target label formed during label combination is recognized by a different ID from the intended one, put it in the dictionary as reject travel, and if it is recognized, it will be recognized as recognition failure. You can also.

The above configuration is the same for a computer-readable recording medium that records the score reading program according to claim 30, and the configuration is such that the note head recognition and the labenole combination are repeatedly performed. A program for executing a priority addition function for assigning a priority to each recognition result recognized by matching during the note head recognition, wherein the recognition function sorts the recognition results in descending order of priority. It will be corrected.

In the above configuration, label separation cannot be performed when a tie is in contact with the note head. For this reason, it is preferable to first erase the ties, then separate the labels, and recognize the drum notes including the noteheads in contact with the ties.

The above-described configuration is the same for a computer-readable recording medium that stores the score reading program of claim 31. The program further includes a program for executing a tie erasing function for erasing a touched tie. After the erasing, the label separating function performs label separation, and the recognition function recognizes a notehead with which the tie is in contact.

As shown in Fig. 3, when instruments other than a normal drum set are represented on the same staff, there may be chords where (X) contacts each other. In such a case, it is difficult to set the recognition rectangle by separating each notehead before matching with the dictionary. For this reason, in the claims 22 configuration, the combined noteheads whose noteheads touch each other up and down are registered in a dictionary, and the dictionary is matched with the combined noteheads to recognize the combined noteheads. It was decided to do.

The above configuration is the same for a computer readable recording medium that records the score reading program of claim 32. The configuration is such that the combined noteheads whose noteheads are in contact with each other up and down are registered in the dictionary. The recognition function matches the dictionary and the combined notehead to recognize the combined notehead.

As in the past, tremolo symbols as shown in Fig. 4 can be recognized by matching the dictionary while combining labels around stem candidates. Especially in the case of drum music, there are relatively many tremolo mouth symbols. Because the tremolo mouth symbol is almost the same thickness and spacing as the hook, if no countermeasures are taken, the flag will be counted in the same way as the hook, and short notes such as 32 notes will be counted. Misunderstood. Therefore, in the configuration of the present application, the tremolo mouth symbol is registered in the dictionary as shown in FIG. 5, the tremolo mouth symbol is recognized, the symbol bit is deleted, and the tremolo symbol is displayed including the tremolo symbol. Recognition of other drum notes was performed.

The above configuration is the same for a computer-readable recording medium storing the musical score reading program according to claim 33. The configuration is such that tremolo symbols are registered in a dictionary, and the tremolo symbol is registered by the recognition function. In addition to recognizing the symbol, the program further has a program for executing a symbol erasing function for erasing the bits of the tremolo symbol. After the erasure, another drum note displayed including the tremolo mouth symbol is recognized. did.

In the configuration of the present invention, after the note head is recognized, the Although the head and stem are combined, special processing is required for drum staves. One of them is caused by the positional relationship between note heads and stems. In other words, noteheads other than black balls may have a stem at the center of the notehead, as shown in Fig. 2. In addition, note heads are rarely attached to both sides of the stem.

Recognition of notes is done by combining noteheads and stemheads, in which the first notehead at the end of the stem is combined and then the middle notehead.

In the case of normal noteheads (black ball, white ball), the ball may be attached to the left and right of the stem. In addition, the head ball has a fixed position in the X direction (horizontal direction) (for example, if the stem is upward, the first note head is attached only to the left of the stem). Therefore, when combining the first ball, the positional relationship between the stem and the notehead in the X direction is limited to the left and right. At this time, the end of the stem (either upward or downward) is determined by the midpoint CC in the Y direction (vertical direction) of the stem, as shown in Fig. 6. Determined by the positional relationship of the Y coordinate of the head center CY. As shown in the figure, whether or not the ball is the first ball is determined by the distance between the end ST of the suffix candidate and the center CY of the notehead, or the relationship between the upper and lower ends of the notehead ellipse. . However, in the case of a drum notehead, when viewed in the X direction, the stem may be at the center of the notehead, so even in the case of the first ball, as shown in Fig. 7 (a) and (b) , Left and right are not limited (both can be combined). After this, the flag on the side with the stem of the notehead is checked after combining the stemheads to prevent the overlapping of stems, but in the case of drum noteheads, the actual Check the fixed direction regardless of the position of the stem and the center of the notehead in the X direction (if the stem is upward, check the right side). Note that this check flag is stored as indicating the attribute of the data in the stored data on the notehead side in order to prevent overlapping of the stems when combining the notehead with the stem candidates.

In this case, as a configuration of the present application, each notehead is provided with a flag for specifying the position of the link with the stem candidate depending on whether the stem candidate to be combined is upward or downward. In the process of combining the stems with the stem candidates, the stems specified by the flag are not allowed to be combined at positions other than the combining position.

Another reason that special combining processing must be performed in the case of drum staves is the presence of noteheads that separate stems as shown in Figs. 9 (a) and 9 (b). Figure In an example like this, the leading black dot in the stem should be recognized as an eighth note, not a quarter note. As shown in the figure, most of the noteheads that divide stems are black balls at the beginning, and there is one notehead that separates them. Therefore, in this configuration, first, the black ball is not targeted, and as shown in Fig. 10, the stems that separate the stems are combined with the candidate stems above and below the stems. After recognizing a drum note with the opposite stem that shares the note head, the note on the right side of the figure deletes the downward note (instead of erasing the bit plane of the image as in the case of erasing, the recognition result). The process of erasing the data on the stored data itself), and combining the upper and lower stems as candidate stems (right side in the figure) as shown in Fig. 9 (c), and finally the remaining noteheads (black) Ball).

The above configuration is the same for a computer-readable recording medium that stores the score reading program according to claim 34. The configuration includes a note head that separates stems by the combining function and upper and lower note heads. A down note deletion function for deleting notes with down stems, which is executed after recognizing by the recognition function a drum note having a reverse stalk that shares the note head by performing the combination with the stem candidates. It has an execution program with a stem combining function for combining upper and lower stems, and the combined one is used as a stem candidate to be combined with the remaining noteheads by the combining function.

Yet another reason for the special combining process for drum staves is that, in order to recognize the notehead, if the hi-hat open (°) is near the stem, it is mistaken for a notehead. It is possible that As a countermeasure, if you do not want to attach downward stems to the 頭 note head, you can check the flag on the left side, and no stem will be attached to this check side. The checked side of the flag is not allowed to combine stem candidates.)

Also, the hi-hat closed (+) symbol shown in Fig. 11 and the tuplet number are mistaken for a notehead, and when this is combined as a notehead at the beginning of a stem, the stem direction is completely reversed. It will be recognized as a different note. As a countermeasure, these symbols ゃ Tuplet numbers may be registered in the dictionary as reject travel, but the stem direction of drum staves is different from that of normal music, and it can be limited to some extent. Is above the staff and the lower stem is not below the staff, In other words, when the upper stem is above the fifth line and the lower stem is above the first line, the stem is effective as a countermeasure against false recognition of upward and open hats (°)]. If the bottom end is below the staff and the top of the stem is not above the staff, that is, if the bottom of the stem is below the first line and the top of the stem is below the fifth line, The tail was determined to be downward, and the first notehead and the candidate were joined.

The above configuration is the same for a computer-readable recording medium storing the score reading program according to claim 35, wherein the upper end of the stem is above the staff and the lower end of the stem. If the stem is not below the staff, the stem is upward, and if the bottom of the stem is below the staff and the upper stem is not above the staff, the stem is downward. The program further includes a program for executing a stem direction determination function for determining that the first notehead and the stem candidate are combined by the combining function after the determination is performed.

However, when the above-described recognition process of the noteheads that divide the stems and the combining process of these stemheads are performed, the configuration for determining the stem direction described above is implemented, as shown in FIG. It is not possible to recognize a configuration in which a downward stem is connected to a common notehead. (If the upper end of the stem is above the staff and the lower end of the stem is not below the staff, the upward stem is not recognized. To be judged). Therefore, in the configuration of the present application, normalization in the direction of stems is performed after performing the above-described processing for noteheads that divide stems. That is, at the time of recognizing a drum note including a note having a note head that separates a stem, the leading note head of these drum notes is combined with a candidate for a stem to form a drum note having a note head that separates a stem. After recognizing the drum note with the opposite stem that shares the notehead, delete the note with the lower stem, and leave the combination of the upper and lower stems as candidate stems. In addition to the first note head, the stem is turned up and the other drum notes are raised if the upper stem is above the staff and the lower stem is not below the staff. If the lower end of the stem is below the staff and the upper end of the stem is not above the staff, it is determined that the stem is downward, and the candidate for stem is combined with the remaining note heads. For those that do not meet the condition of, after returning to the original stem candidates (normalization processing for stem direction), The stem candidates are combined with the remaining note heads.

The above configuration is a combination recording a score reading program according to claim 36. The same applies to a recording medium that can read data, and the configuration is such that when recognizing a drum note including a note having a note head that divides a suffix, the first note head of the drum note is combined with the first note head of the drum note by the combining function. For a drum note having a note head that divides a stem by performing a process of combining stem candidates, the above-mentioned downward note is used after recognizing a drum note of the opposite stem that shares the note head by the recognition function. The delete function deletes notes with lower stems, and combines the upper and lower stems with the stem combination function as a candidate for stems. For other drum notes, the stem direction determination function allows the stem to be directed upward when the upper stem is above the staff and the lower stem is not below the staff. The lower stem is If the stem is below the line and the upper end of the stem is not above the staff, it is determined that the stem is downward, and in addition to combining the candidate stem with the remaining noteheads by the combining function, Those which do not meet these conditions further have a program for executing a note candidate return function for returning to the original stem candidate. The remaining noteheads are to be combined.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is an explanatory diagram showing a state of a notehead label separation process of a drum note, FIG. 2 is an explanatory diagram showing a state where a notehead is divided into a plurality of portions by a stem erasing process, and FIG. Fig. 4 is an explanatory diagram showing an example of a musical score including notes with their heads touching up and down, Fig. 4 is an explanatory diagram showing an example of a musical score including a tremolo symbol, and Fig. 5 is an explanatory diagram showing an example of an image for a tremolo symbol dictionary. FIG. 6 is an explanatory view showing a positional relationship determination method in the case of combining noteheads and stems, FIG. 7 is an explanatory view showing the state of connecting noteheads and stems in a drum staff, and FIG. Fig. 9 is an explanatory diagram showing the check state of the flag at the time of combining the head and stem, Fig. 9 is an explanatory diagram showing an example in which the stem is separated by the head, and Fig. 10 is Explanatory diagram showing an example of the note recognition process for note heads to be divided, Fig. 11 shows an example of a musical score including hi-hat closed and tuplet numeral symbols FIG. 12 is an explanatory diagram showing an example of a configuration in which a downward stem is connected to a common note head, and FIG. 13 is a computer-readable recording recording a score reading program according to the present invention. Score reading that operates by reading the medium in the external storage device FIG. 14 is a block diagram showing the main processing of the CPU, FIG. 15 is an explanatory diagram showing the notehead to be recognized, and FIG. 16 is a diagram showing the result of recognition. Fig. 17 is an explanatory diagram showing the method of specifying the note for writing the recognition result when there is a specific symbol in Fig. 17. Fig. 17 is an explanatory diagram showing a standard drum notation pattern. An explanatory diagram showing the state of no parentheses, FIG. 19 is an explanatory diagram showing an example in which a notehead to be integrated into one notehead type crosses a staff, and FIG. Fig. 21 is an explanatory diagram showing an example in which a notehead to be integrated into one notehead type intersects a stem, and Fig. 21 is an explanatory diagram showing the form of a label to be recognized in the case of the intersection. Fig. 2 is a flowchart showing details of the matching process including label combining process, Fig. 23 is the same FIG. 24 is a flowchart showing details of the matching process including the label combining process. FIG. 24 is a flowchart showing details of the matching process also including the label combining process. FIG. 25 is an actual flowchart of the matching process including the label combining process. An explanatory diagram showing an example, FIG. 26 is a flowchart showing a flow of the rectangle matching process, FIG. 27 is an explanatory diagram showing a tie recognition process for a notehead label in contact with a tie, and FIG. Explanatory diagram showing the state in which the hi-hat open of the paragraph has entered the recognition rectangle, Fig. 29 is an explanatory diagram of the paragraph on the score, and Fig. 30 is the black ball and white ball head of the note to be recognized. It is explanatory drawing which shows a suffix and a flag.

BEST MODE FOR CARRYING OUT THE INVENTION

(Embodiment 1)

An embodiment of the present invention will be described below with reference to the accompanying drawings. FIG. 13 shows a computer-readable recording medium storing a musical score reading program according to the present invention, which is read and operated by an external storage device (or a CD-ROM drive or the like) such as a flexible disk drive FDD 5 described later. FIG. 1 is a block diagram showing the configuration of an embodiment of a musical score reading device. This device is obtained by adding a scanner and a MIDI interface circuit to the configuration of an electronic computer such as a personal computer. The CPU 1 is a central processing unit that controls the entire score reading apparatus based on a program stored in ROM 2 or RAM 3. In addition, the CPU 1 Built-in timer circuit for interrupt. The RAM 3 is used as an image data buffer, a work area, and the like in addition to the program area. The hard disk drive HDD4 and the flexible disk drive FDD5 store programs, image data, performance data, and the like. The CRT 6 displays video information output from the CRT interface circuit 7 based on the control of the CPU 1, and the information input from the keyboard 8 is input to the CPU 1 via the keyboard interface circuit 9. It is. The printer 10 prints the print information output from the printer interface circuit 11 under the control of the CPU 1.

The scanner 12 optically scans (printed) music, for example, and converts it into binary, grayscale, or color image data. The scanner 12 can be any type such as a flatbed type, a handy type, or a feeder type. Types of scanners can be used. The image information read by the scanner 12 is taken into the RAM 3 or the HDD 4 via the scanner interface circuit 13. The MDI interface circuit 14 is a circuit that transmits and receives MIDI data to and from an external MIDI device such as a sound source module. The bus 15 connects each circuit in the musical score reader and exchanges various data, programs, addresses, and the like. In addition, a pointing device such as a mouse or a serial interface circuit such as RS232C may be provided.

FIG. 14 is a flowchart showing the main processing of the CPU 1. In step S 1, the image of the musical score is loaded into the RAM 3 by the scanner 12. Images are captured as binary image data. In step S2, image quality smoothing processing such as graphic fusion is performed to reduce blurring and dot noise. In step S3, an image quality check process, which will be described later, is performed to obtain resolution and density information and to obtain reference data for staff detection in a later stage. In the music score recognition process, if the resolution and density are out of the predetermined range, the recognition rate will be reduced. Therefore, in step S3, it is checked whether or not these values are within the predetermined range. Is locked. In step S4, it is determined whether or not the check result in step S3 is image quality OK. If the result is not 〇K, the process returns to step S1 to change the resolution and density and re-acquire. Do. In step S5, Perform staff recognition. In the staff recognition process, a staff scanning start position detection process and a staff shift amount detection process are performed. In step S6, a paragraph recognition process is performed. This process is roughly divided into paragraph recognition and bracket recognition. In the paragraph recognition process, staffs are detected in the entire image, a set of staffs whose left ends are at approximately the same location between the staffs is searched, and it is checked whether the ends of the staffs are connected by black pixels. And recognize the step. In step S7, the recognition result of the paragraph is displayed, and the user is checked whether or not the paragraph recognition result is correct, and it is determined whether or not it is OK. If the result is not Ο ステップ, the step is performed. Proceeding to S8, the paragraph recognition result is corrected. If the staff recognition fails, subsequent processing cannot be performed, so it is necessary to change the resolution and density and import the image again. Therefore, in step S7, the staff recognition result is displayed first, and the user is judged whether or not the staff is correct. If the line is correctly recognized, the paragraph recognition result may be displayed and checked.

In step S9, the user selects a drum part of each paragraph. Then, in step S1◦, the normal part performs normal music score recognition, and the drum part performs drum score recognition. After the recognition of symbols on all pages, the drum part in one song is scanned twice (twice noraping) in Steps Sll to 13 so that the drum notation is Estimate and assign instruments according to the notation. This configuration does not include a modified interface for the drum notation estimation result. This is because if the recognition result is wrong, the recognized score symbol can be corrected. Finally, in step S14, the assigned musical instrument is converted into a playable musical score data format in which the pitch and pitch are specified.

Hereinafter, steps S9 to S14 will be described in detail. In step S9, the user performs the drum part setting at the same time as checking the paragraph recognition or correcting the paragraph recognition result. This is done by displaying the results of the recognized staff and paragraphs and having the staff check the drum staves. The part information of the data that saves the paragraph recognition results Set a flag to determine if

Based on the flag information that has been set, the recognition unit classifies normal music score recognition and drum score recognition, and in step S10, performs symbol recognition on the drum score. The differences between normal score recognition and drum score recognition are as follows. Drum notation recognition skips the recognition of symbols that do not exist in the drum notation. (However, in order to take measures against misconfiguration, misrecognition, spelling mistakes, etc., recognition itself is performed and only output may be limited.) In drum notation recognition, we add the unique symbol 'unique string recognition to the drum notation, while removing the recognition of unnecessary strings in the drum notation. Also, recognition of drumheads other than black and white balls is added. On the other hand, the probability of appearance of symbols on drum staves · Change the threshold value corresponding to the appearance form (for example, since the appearance probability of slurs is low, it is possible to speed up by making the threshold value stricter during recognition, etc.) Change the threshold for recognition of normal notes (the number of black ball chords is small, the appearance rate of white ball is small, etc.) Therefore, the threshold value of black ball should be strict, or in some cases, white ball should not be recognized. In this configuration, white balls are not recognized.)

The recognition of the drum head is performed as follows. Black dots, which are the same as normal scores, in the note head of a drum score, are separated by thick and thin separation processing performed by erasing staff lines and the like, and ellipse detection processing for recognizing black dots is performed in the same way as normal scores. Do. In this configuration, as described above, recognition is not performed because the probability of occurrence of white balls in a drum score is low. On the other hand, note heads of drum staves other than these are drawn with thin lines such as (x, O), and there are various other types. It cannot be detected by the ellipse detection processing. Therefore, for whole notes, the detected labels (rectangles of an appropriate size to recognize the target symbol) must be matched with the dictionary, just like normal symbols (for example, rests). Performed by Recognition of other noteheads is realized by separating the labels with the stems (provisioned as stems) and matching the dictionary with the labels around the stems. The recognition of the drum head will be described in detail in a section of a seventh embodiment described later. The noteheads to be recognized are as shown in Fig. 15. There are other drum note heads, but if this level is supported, general drum staves can be almost supported. It is also possible to change the type of notehead to be recognized depending on where the notehead is on the staff (pitch). It is valid [for example, note recognition (R) exists only in the third period, and such a recognition is not performed in this configuration]. For noteheads other than black ball (white ball), the recognized notehead type is left in the symbol storage data. That is, the note head type is written in the storage area dru mN ote in the recognition symbol storage data.

In recognizing drum scores, it is also important to recognize symbols related to drum notes. For example, hi-hat cymbals sometimes express open and close (or other half-open) by writing (°) or ( ⁺ ) above the note. There is also a notation that accents (>) are cymbals (crashes) and others are hi-hat cymbals. Therefore, it is necessary to detect the degree of these factors. Therefore, for drum notation, (°) and (+) are additionally recognized when recognizing symbols. If (°), (+), (>), etc. are present in the recognition result, as shown in Fig. 16, the center of the notehead is added to the rectangle B to which the appropriate width at the top, bottom, left and right of this symbol is added. Write the value corresponding to each symbol type to the note where C or stem end point D exists. In the case of normal drum staves, the notes affected by these symbols are only those written at the highest pitch, and there is no overlap between the accent symbol that expresses the crash cymbal and the hi-hat symbol. The noteheads of these notes are limited to those with (X) noteheads. Therefore, it is conceivable that the note to be involved is the highest pitch, and that the hi-hat and the axent delete the storage area as a flag at the same position. However, considering the possibility of misrecognition and simplifying the processing, it is possible to write separate flags for all notes in the same band. In this configuration, a value corresponding to each symbol is written to all notes other than the black ball in the same band. However, in this case, it is necessary to consider the priority of detection of the degree of engagement. In other words, when hi-hat open or close is involved, it is not necessary to consider the accent. Therefore, it is necessary to detect the change of hi-hat symbols first, and then detect the change of accent marks, so that once a change is detected, the note cannot be overwritten. The standard pattern described later does not support hi-hat half open, hi-hat quarter open, etc. , And may be changed to quotes. (Put a value in the free area of drumNote) Open, close, etc. are written with priority over accents.

When the symbol recognition is completed as described above, the drum notation is estimated in the next steps SI1 to S12. In this configuration, data is created in the form of a standard drum notation pattern in order to estimate the corresponding notation by the operation of fitting, display the recognition result, and perform. The above fit is achieved by scanning the drum part in one song twice. As a result of the fitting, a flag representing the actual instrument is further written in drumNote. By referring to this flag, data is created in the format of drum standard pattern. Before describing the above-mentioned fitting process, the drum notation and the drum standard pattern will be described below.

There are many different notations for drum notation. Some of the notations are very special, but they are infrequent and not included. The most problematic in drum notation is the notation of sympa noles (such as hihattoshinbanore, crash-simpanore, top-symbanolle). Other musical instruments have a slightly different pitch. However, the number of pitches (pitch number) changes (in the case of two, three, or four…). Cymbal notation is roughly divided into two types. In other words, there is a case where one pitch is expressed and another case where the hi-hat and the rest are divided into two (or more) pitches. In each type, the type of musical instrument can be divided into two types: one with a different notehead type, and the other with or without an accent. In some cases, additional noteheads are used to represent half notes and whole notes. In addition, the type of instrument may be specified by a character string.In addition, the instrument may be specified by a character string at the beginning of the song, the area may be divided by a character string, or the character string may be specified for all notes. In some cases.

As described above, the score reading apparatus of this configuration has a drum score standard pattern for displaying the recognized result and performing as shown in FIG. Supported instruments are crash cymbal (sometimes called side cymbal), top cymbal (sometimes called ride cymbal), hi-hat open, hi-hat close, snare drum, rim shot, high tom, mid tom, Low toms, bass drums and pedal no-hats. The result of symbol recognition must be applied to this standard pattern. There are also cases where instruments other than the standard patterns are supported (cowbells, chimes, gongs, etc.). It is necessary to take appropriate measures.

The procedure for applying a drum instrument other than a cymbal system will be described below.

Snare drums and rim shots may appear between the third lines of the staff. The snare drum is usually represented by a black ball, and the rim shot may be a separate note head such as (R), or the rim shot may be specified as a character string without changing the black ball. Therefore, in the first loop of S11 of the two scans, the black ball between the third is a snare drum, and the noteheads other than the black ball are rim shots. If there is a notehead other than a black ball, the logical value r i mO t h e r E X i st is set to true. Also, a character string representing the rim shot is searched, and if it exists, the logical value rimSt rEXi st is set to true. In the second loop of S13, if the logical value rimStrExiSt is true and rimOthereExiSt is false, all the notes in the third are rim shots.

Usually the pedal hi-hat is written as a notehead (X) between the lower first. However, there are also cases where it is shifted downward to avoid the bass drum. Therefore, in consideration of the possibility of erroneous recognition, notes other than the black ball below the first line are used as pedal noise. Pedal hi-hat can be set in one loop.

Normally, bass drums and toms are indicated by black dots, bass drums are indicated by the first interval, and toms are indicated by 1 to 4 pitches from the second to fifth lines excluding the third line. However, there is also a notation in which the bass drum is written in the lower part. In such a case, there is a high possibility that the LOW tom is written between the _lth (that is, the same pitch as a normal bass drum). As described above, the tom range changes depending on the pitch of the bass drum, and therefore, the pitch including the pitch of the bass drum is detected in the first loop. However, in this configuration, pitches lower than the lower first interval are deleted as erroneous recognition. For this reason, in the first loop of S11, the pitch at which the black ball exists is two steps, the lowest (minTom) and one step above (minTom2nd), the highest pitch (max Tom) Is detected, and a black ball note with a pitch lower than the lower first interval is deleted. At this time, the maximum pitch of the tom is usually no more than the fifth line, but in this configuration, the detection range is extended to the second interval in consideration of the possibility of misrecognition and special notation. From this information, the pitch of the bass drum and the pitch of the tom are detected as follows. That is, if minTom is lower than the first interval, the pitch of the bass drum (height Bass Drum) is set to minTom, and the pitch of the low tom (height Let LoTom) be minTom2nd. Otherwise, the pitch of the bass drum is in the first interval, in this case, if minTom is in the first interval, the low tom pitch is minTom2nd, otherwise, the low tom is Low. Tom's pitch is min Tom. The pitch of the high tom (height Hi Tom) is max Tom. In the second loop of S13, the actual settings (confirmation) of the bass drum and tom are made. That is, if it is a black ball and the pitch is height Bass Drum, a bass drum, if it is a black ball and the pitch is height LoTom, a low tom, and if it is a black ball and the pitch is height Hi If it is Tom, it is a High tom and a black ball, and the pitch is between heightLoTom and heightHiTom. In the standard pattern, there are three types of toms, so for four or more toms, the middle is assigned to one type of tom.

The procedure for applying a cymbal-based drum instrument will be described below. As described above, there are various notations for cymbal-based drum instruments, and it is not possible to perform fitting using only one loop. Therefore, the fit is determined by running twice (two loops) the recognition results of all drum parts in all pages and all paragraphs. In the first loop, in the range corresponding to the cymbal, information such as the number of noteheads at each pitch is obtained. Before the second loop, a conversion table for drum notation fitting is set from the obtained information based on the drum fitting conditions. In the second loop, use the conversion table to set the instrument that actually plays each note.

In the process of fitting to the standard drum notation pattern, a reasonable estimation is made from various notations using prerequisite conditions. The preconditions are as follows. Half notes and whole notes are likely to be crash cymbals due to the characteristics of the instrument. Therefore, in this configuration, it is possible to estimate the notation assuming that the half note and the whole note are crash cymbals. Noteheads (◊) are more likely to be written as half-notes or whole notes in the history of drum staves. Therefore, assigning note heads (◊) to half notes and whole notes preferentially makes the notation estimation more appropriate. If whole notes are present, half notes are likely to be written with the same notehead as whole notes. Cymbal-type drum notes are not likely to be represented by three or more pitches.If they are represented by two pitches, the lower row is a hi-hat, the upper row is a crash cymbal, and the top It is a member. If both the cymbal and top cymbal are written in the upper row, the notehead of the top cymbal is represented by (X), and the crash cymbal is written in other ways (such as changing the notehead). Furthermore, if there is only one type of cymbal (other than the hi-hat cymbal), the crash cymbal whose _d- note head is (〇) is indicated as hi-hat open, and the symbol (°) is not related. Is associated with the sign ( ⁺ ).] A hi-hat is a hi-hat closed.

The estimation process of cymbal-based drum instruments by applying the process to the drum standard pattern will be described in detail below.

In the first loop of S11, the following information is acquired.

i) the number of each notehead at each pitch

)) Whether each notehead at each pitch has a note shorter than an eighth note

iii) Percentage of each notehead with accent at each pitch

iv) Set logical value to determine whether note (符) note exists

Here, the process of opening the note with the notehead (〇) in the hi-hat open and the process of replacing the notehead (△) and the notehead (C) with the top cymbal are performed here.

In the first loop of S11, the number of noteheads at each pitch was determined, but in the second loop preprocessing of S12, this information table was scanned from the lowest pitch, Find the pitch of the cymbal. The first pitch found is the first pitch, the second one found is the second pitch, and so on. However, if the number of notes at the first pitch is extremely small, there is a high possibility that a note is erroneously recognized. Therefore, the note may be deleted and the cymbal pitch detection result may be shifted. This process is especially effective when the pitch of the cymbal is three or more steps.

Next, a drum fitting conversion table is created. The conversion table can write the type of instrument assigned when the note was in that state, for the two pitches, for the elements classified according to the type of note head and the relation of accent etc. So that

There are different types of note states, such as CT—PEKE—NU N [not involved with notehead (X)], CT—PEKE—CLS [( ⁺ ) is concerned with notehead (X)], etc. become. The type of instrument assignment is DN HH CLOSE Rose), DN_S I DEHALF (half note of a crash cymbal). Notations are classified according to the number of pitches of the detected cymbals, and instruments are assigned to the conversion table according to each type. The classification is, for example, as shown in Table 1 below.

【table 1】

The most appropriate instrument is assigned to the conversion table based on the above processing assumptions.

In other words, if it is type 1, it is reasonable to assume that all are hi-hats. Of course, due to the preconditions, hi-hat is normally closed, and if (°) is involved, hi-hat is open.

For type 2, it is reasonable to use crash cymbal half notes (whole notes) except for the notehead (X). In this case, noteheads (X) with accents are considered classical cymbals. Conversion table CT— PEKE— ACC [note head (X) and accented], DN- SI DE (crash. Cymbal), conversion table other than note head (X) DN— SI DEHALF (crash cymbal) 2 minutes, whole note). In Type 23, the lower row is all hi-hat cymbals, and the upper row is as follows.

The notehead (X) is the top cymbal.

Also, if there is a whole note and the same notehead as the whole note exists other than a whole note, the whole note is a whole note of the crash cymbal, a note with the same notehead as the whole note is a half note of the crash cymbal, A note head different from the whole note (the other) is a crash cymbal. If a whole note is present and the same notehead as the whole note does not exist except for the whole note, or if there is no whole note, the notehead (◊) will crash the notehead (◊) if there is a cymbal cymbal. The other is a cymbal cymbal quarter note, and if there is no notehead (◊), the smaller number is a cymbal cymbal quarter note and the other is a crash cymbal quarter note.

If a whole note does not exist and a notehead (◊) is present, the notehead (◊) is a crash cymba-nore quarter note and the other is a crash cymba-nore quarter note.

The above notation is the most probable, but for example, in Type 2, other than the notehead X is a quarter note of the crash cymbal, and the notehead (X) with an accent is a hihat. obtain.

After the preprocessing of the second loop is completed, instruments are assigned in the second loop of S13. That is, while scanning the recognition result, the musical instruments on the conversion table are assigned according to the relation between the noteheads and symbols of each pitch. If the conversion table value specifies a half note such as DN_S I DEHALF, and if it is originally recognized as a quarter note before estimation, it converts the note ID to a half note. If the conversion table is specified to be deleted (DN—MATCH I NGNONE), the recognition symbol is deleted (ID = ID—DELETED).

During the above scanning, it is also possible to assign the instrument on the conversion tape by recognizing the designation of the instrument by the character string. The character string of the force cymbal is "ridecym", "sidecym", "t.cno", " c. c. ", etc., the strings are similar to each other, and the risk of misidentification is high. Therefore, it is not possible to dictionary all the strings written on the drum notation and reflect all the recognition results. Therefore, it is reasonable to limit the dictionary of character strings to only those with a low probability of misrecognition and limit the effect of the character strings. If the character string is only "top" and the effect is specified with a crash cymbal of 2 minutes and no whole note, the top cymbal where the crash cymbal is usually set and the hi-hat cymbal will be (°) ( ⁺ ) If there is no specification, there are only two cases where the top cymbal is the place where the hi-hat cymbal is normally set. If both of these two cases apply, the one with the higher probability of notation should be used.

Other note heads (△) ゃ Recognize note heads (C). These are usually cowbells and cups and are not present in the standard pattern. Therefore, it is conceivable that the conversion process deletes the conversion table by specifying deletion. Also, even if percussion such as tampaline is written on the drum notation, special processing is required if it is not supported by the fitting process, such as the (X) notehead on the third line. Otherwise, the data will be deleted from the data. However, simply deleting such notes may not be able to achieve the rhythm, so they can be replaced with top cymbals. Similarly, simply deleting a note may not be able to achieve the rhythm, so instead of deleting the note, it is conceivable to replace it with a rest of the same duration.

After the musical instruments have been assigned in this manner, in step S14, the musical score data format that can be played and / or displayed based on the pitches and durations of the notes of the drum notation that are evident from the musical score recognition described above. Convert to

By recognizing the notation of the drum notation as described above, it becomes possible to recognize and perform the drum notation without burdening the user. Also, users who do not have knowledge of drum scores can use them. In addition to the configuration of the above-described embodiment, the following other embodiment configuration is also conceivable.

(Embodiment 2)

If a standard pattern is provided for pasting notes and other symbols to the drum score, and if it is not possible to do anything other than pasting in the standard pattern format, it will be necessary to paste with a notation different from commercial music score, Correction of the recognition result cannot be based on the original score. Provide an interface that allows any notation as in the score, and when performing, estimate the notation using the above-mentioned fitting algorithm and identify the instrument to be played. Then, the recognition result should be corrected so that it is in the same state as the original score.

(Embodiment 3)

In this configuration, a character string is used to support notation that changes the timbre (drum instrument) in the middle of a song. Of particular concern with this notation are changes to snare drum and rim shots, changes to crash sympanole and top cymbals, and changes to noise notes and top sympa.

Processing is divided into two types: snare drum / rim shot and other cases (cymbals). Then, it is determined whether or not each of the timbres is actually changed in the middle of the music by the character string. In the case of a cymbal, you must determine which change. In this embodiment, a snare drum, a classic cymbal, a hi-hat cymbal, and the like are added to the character string to be recognized in addition to the rimshot and the top cymbal in order to realize the reflection of the character string in a time series. In addition, a means to recognize the range specification such as parentheses attached to the character string is added. In this case, the hot spot that becomes the reference for time series sorting performed later is the starting point (left end) of the parentheses. The parenthesized range specification is recognized by distinguishing between the case where there is a terminal as shown in Fig. 18 (a) and the case where there is no terminal as shown in Fig. 18 (b). In the case of a character string with a terminator, another symbol data with an ID for terminating the character string with the right end of the parenthesis as a hot spot is added so that it can be used as a reference for time-series scanning. It also recognizes terminal brackets as shown in Fig. 18 (c), in case that the end of the range specification by the parentheses of the character string is described separately. In this case, the reference hot spot for chronological ordering is the right end of the parentheses. Furthermore, in the case of a character string without a range, the hot spot may be shifted slightly forward (left) in chronological order because the note may be shifted backward (right) from the note.

The results recognized in this way are arranged in chronological order.

The process of changing the tone of the snare drum rim shot is performed as follows. First, the recognition result is scanned from the top in time series. The initial value of the tone storage data snare Or Rim is DN—SNA RE. If there is a third black ball, write the value of snare Orim to dru mN ote. A string representing a snare drum or If there is a character string representing the rim shot, write DN-SNARE or DN-RIM in snare Or Rim. When the terminal ID is detected, the value is rewritten to a value opposite to the current snare_or_Rim. Then, when there is a repeated bar line end or the like, snare Or Rim may be changed. On the other hand, when the tone specification other than the start position between the third is only one of the character string representing the snare drum and the character string representing the rim shot, and the case where both character strings are alternately present, However, it is conceivable to change the processing. In other words, if both character strings exist, the end does not need to be considered.

The process of changing the tone color of a cymbal is performed as follows. For top cymbals, crash cymbals, and hi-hats, there are cases where the clash cymbal and top cymbal are exchanged with a character string, and where the hi-hat cymbal and top cymbal are exchanged. At the same time, you must determine which instrument to change.

The following processing is performed after the step of assigning instruments to the conversion table according to each type (for example, type 23 shown in Table 1 above) at the time of notation estimation (see the description following Table 1). Execute (however, do not process the character string representing the top cymbal). The result of this processing is referred to as “first estimation result”.

First, if the cymbal has two pitches, there is no exchange between the hi-hat and the top cymbal. If the first estimation results show that both the cymbal and the top cymbal already exist, the instrument is not exchanged using a character string. Even when the cymbal has one pitch, if both the crash cymbal and the top cymbal are present in the first estimation result, it can be estimated that the hihat cymbal and the top cymbal are exchanged. Otherwise, run the recognition result in time series once, count the number of notes of the hi-hat cymbal and the crash cymbal in the first estimation result in the area specified by the character string representing the top cymbal, If there are many hi-hat cymbals, it is determined that the hi-hat cymbal and the top cymbal are exchanged. Otherwise, it is determined that the crash cymbal and the top cymbal are exchanged. In this judgment, considering that the number of hi-hat cymbals is basically larger than the number of crash cymbals, the number of hi-hat cymbals and crash cymbals is not May be determined based on a threshold value.

Once the exchange method has been determined, perform another time-series scan and change the instrument in the same way as for the snare drum and rim shot. In the case of exchanging the hi-hat cymbal and the top cymbal, the one that was regarded as the hi-hat cymbal in the first estimation result is exchanged for the top cymbal. In the case of exchanging a crash cymbal with a top cymbal, the crash cymbal determined in the first estimation result is exchanged for a top cymbal.

(Embodiment 4) "The user does not specify a drum part, but automatically recognizes whether or not it is a drum part. In a score, a part name is often described as a character string to the left of the staff. By recognizing this character string, the drum part can be recognized automatically.In addition to the note recognition, in addition to black and white balls, drum heads can also be recognized, and after recognition, the drum part can be recognized according to the number of drum heads. In this case, a threshold value is set for the number of drum note heads, taking into account the possibility of misrecognition and the case where clapping and the like are usually indicated by (X). By letting the user specify whether or not a drum part exists, it is useless when there is no drum notation, such as recognition of the character string to the left of the staff and recognition of the drum head in normal music. Processing and risk of misrecognition It can be avoided. Also if it is specified that the drum part is present, it is also possible to simply specify the lowermost part of the paragraph and drum notation.

(Embodiment 5)

The assignment of the crash cymbal and the top cymbal varies depending on the tempo of the song. When the tempo of a song is slow, the tone of the song is often quiet, and in such a case, it is common to use a top cymbal in a situation where a crash cymbal is normally used and to perform a soft accent. is there. However, this is rarely specified in character strings. Therefore, such a situation is estimated from the tempo. In other words, if the tempo is slower than a certain value and the notation is assigned to the instrument, and there is no top sympanole assigned, the part that would normally be the crash cymbal is the top cymbal. The tempo can be determined because the tempo symbol is recognized. (Embodiment 6)

To improve the accuracy of character string recognition, especially character string recognition especially for drum music recognition. For example, "ridecym" and "sidecym" are significantly different in meaning due to the difference of one character. Therefore, as an example, when "idecym" is recognized, 7 out of 8 characters are matched, so that a successful recognition of the character string is simply performed to drop the character string in the dictionary in the earlier order. Then there is a problem. Therefore, the positions of the characters to be noted in the character string are stored in the dictionary. If the recognition of the character at the above character position has failed, a process that does not reduce the character string recognition result is performed, and the character with the highest matching degree among the characters that may exist at that position is detected. , Increase the precision (for example, compare the degree of matching between r and s, and if s is high, change to "sidecym".

(Embodiment 7)

Here, embodiments of the invention described in claims 17 to 36 will be described. In the drum note head recognition according to the first embodiment described above, the black ball, which is the same as the normal score, is used for the thick and thin separation processing performed by erasing the staff lines and the like and the black ball portion recognition as in the normal score. As described above, the recognition is not performed because the detection is performed by the ellipse detection process and the probability of white balls appearing in the drum music is low. On the other hand, note heads of drum staves other than black ball 'white ball' are drawn with thin lines such as (X, O), and there are various other types. In addition, it cannot be detected by the thick / fine separation processing and the ellipse detection processing. Therefore, after removing the stems, the notehead labels (rectangles set to recognize the target symbol) are separated,

This is done by scanning (provisioned as stems) and matching the detected labels around the stem candidates with the dictionaries.尾 Since noteheads may be broken into multiple labels due to stem erasure, if matching fails, further scan the stem candidates and continue with the next detected notehead label and the separation label. After performing the join processing with, matching with the dictionary is performed again. In addition, there are unique drum recognition and combining processes, which will be described later. In addition, Dora Since the number of chords of the black ball in the musical score (when there is contact with the black ball) is small, note heads can also be detected by matching by label separation described above, instead of the ellipse detection processing described above. The notehead to be recognized is as shown in FIG. 15 as described above.

Hereinafter, the details of the method of recognizing the score of a drum score in the configuration of the present score recognition apparatus will be described. As a prerequisite for this recognition, a score image read by the scanner 12 is acquired at one dot and one byte, and one bit is used as a plane (bitSrc) of the original image.

(1) Normal note detection method (other than white ball)

Normal note detection (white ball detection) excluding white ball is

① Thick and fine separation processing

Copy the original image plane (bitSrc) to two other bitplanes (bitTic, bitDrm) for normal and drum notes. Perform staff erasure processing on the two copied bit planes. However, staff elimination in the original image plane (bitSrc) is not performed. Scans the score image horizontally, eliminating short runs from one of the copied bitplanes (bitTic). It also scans vertically, eliminating short runs. The remaining parts make up the thick components on the music score (black ball head, hook, etc.).

② Stem detection

Detects fine vertical lines using short run bits erased by horizontal scanning.

③ Note head detection

If a thick part exists while scanning around the above vertical thin line (hereinafter referred to as stem candidates), the boundary line of this thick part is detected, coordinates are voted at certain intervals, and the elliptic equation is calculated. If this elliptic equation is valid as a notehead, add it to the ellipse storage data ( _ep ).

④ Stem couplingRecognition

It is determined whether or not the stem candidate and the ellipse are in a proper positional relationship as a note, and if appropriate, are combined with the stem, and then added to the recognition result as a note. It is performed in the procedure as follows. The recognized black ball is deleted from the other copied bit plane (bit D rm) so that it is not subject to the next drum note recognition. Since small notes have a high risk of misidentification, it is better to take measures such as not recognizing them or deleting them.

(2) Drum note recognition basic processing

① Note head label separation

The left and right label bits of the stem candidates of the drum note bit plane (bitDrm) are erased with a certain width as shown in Fig.1. Then the label of the drum note head can be separated from the stem. However, in some cases, notehead labels may be split into multiple labels, as shown in Figure 2. Therefore, recognition is performed while performing label combining as described later.

② Matching

Detect labels on bit-planes (bitDrm) as they scan around the tails. The detected label rectangle is spread slightly to the left and right, and the plane (bitSrc) in this rectangle is targeted for dictionary matching. Spreading the rectangle slightly to the left and right is a process to reduce stem noise. That is, since the distance between the detected label and the stem is short, if the rectangle is left as it is, some of the stems may or may not enter the recognition target, and the matching with the dictionary will be greatly changed. The rectangle is expanded so that stems are included in the recognition target. The reason why the target bit plane is set to bitSrc instead of bitDrm is to prevent the state of staff elimination from affecting matching.

Then, bitSrc in the rectangle (label) is matched with the dictionary. Any matching method may be used.

For the dictionary for matching, multiple IDs are prepared for one notehead type by the combination of noteheads, stems, and staves. After matching, ID integration processing is performed. That is, as shown in Fig. 19 and Fig. 20, when the notehead intersects with the staff and intersects with the stem, these are combined and more than one ID is assigned to one notehead type. Be prepared. At this time, the recognition target label (label matching the dictionary) is as shown in Fig. 21. Match these with the dictionary as separate IDs, and finally Merge into a single ID, notehead (X). These patterns may be used as one ID from the beginning, but then the number of registration patterns for one ID becomes enormous. Therefore, as described above, if the classification is made based on the intersection with the staff and the intersection with the suffix, the number of patterns per ID is reduced, and it becomes easier to organize the dictionary.

Whether or not matching is to be performed is determined based on the size of the rectangle. For example, a rectangle whose height is clearly smaller than the notehead does not need to be matched, so the process proceeds to the label combining process described later. If the width is wide, a tie may be connected to the notehead, so the tie elimination process described later is performed. If this process fails and the rectangle remains large, the matching fails from the beginning and the label binding process is reset.

③ Ravenore coupling

Matches the note head dictionary with the label. If no match is found (ID—MA TCHINGNONE), label scanning is continued, then the label is detected and combined, and re-recognition (matching the dictionary again) is performed. . However, matching is performed once with the second label itself used for the combination, and if the priority of the ID of the matching result is the highest, this is prioritized. It also resets the maximum size (d x t) of label merging and resets label merging (to enable label merging), and also sets the vertical distance (dy t) at which new merging starts. Furthermore, if the priority of the ID of the recognition result is the highest, the rectangle combining is terminated at that point, and the rectangle and distance are initialized to perform a new rectangle combining (the above two reset processings are performed). Do).

During label combining, priorities are given to recognition results because the shape may be the same as another notehead before it becomes a recognition rectangle that includes all the noteheads that should be originally recognized. If the recognition result has a low priority, the ID and rectangle are saved and the next label scan is performed. If no higher priority ID is found, this is given priority.

If the recognition target label created during label combining is recognized by a different ID from the intended one, put it in the dictionary as reciprocal travel, and if the recognition result becomes this, recognize it as recognition failure. You can also. There are special cases where the matching result is special, such as a combination of a notehead and a tremolo symbol that represents a combination of noteheads. If these are recognized, processing is performed accordingly, as described later. When matching with the dictionary is completed, data is added to the notehead data (ep). ④ Stem couplingRecognition

In the same way as for the black ball, the combining process with the stem candidate is performed. However, special processing may be required for drum note heads. This will be described later. For noteheads other than black ball (white ball), the recognized notehead type is left in the symbol storage data. That is, the note head type is written in the storage area dRumNote in the recognition symbol storage data.

Details of the matching process including the label combining process are shown in flowcharts in FIGS. 22 to 24. The process shown in the figure is to search for bit D rm while scanning around the vertical line of a stem candidate with a certain width dw, and if found, apply a painting function to it and apply a rectangle (rect Cur: (Enclosing rectangle) is detected. While judging the size of the rectangle, as shown in Fig. 25 (a) and (b), matching is performed only with rect Curr, and depending on the result, the rectangle is merged with the previous rectangle, and the merged rectangle (rec tMr g : Rectangle that connects labels).

In step S21, the value of default is entered into the coordinates of rec tMr g and rect Cur, and the coordinates of the rectangle are initialized, and the stored notehead ID is also initialized. In step S22, the y value of the highest point of the stem is put into yBef. In step S23, y is looped from the top to the bottom of the stem candidates. Then, in step S24, it is determined whether or not each coordinate of recMrg is a default value, and if it is the default value, the process jumps to step S29 described later. If this value is not the default value, go to step S25. If there is some distance where there is no dot at that time, it must be reset for the next scan. Therefore, in step S25, it is determined whether or not the absolute value of the difference between the value of the point y under investigation and the value of the uppermost point of the stem has exceeded the value of dyt. The process jumps to step 29, but if it does, it is determined in step S26 whether or not the stored notehead ID is in the initial state. If it is in the initial state, the process proceeds to step S28 described later. On the other hand, if not in the initial state, in step S27, notehead data is created using the saved notehead ID and the rectangle. In step S28, a default value is set for each coordinate of rec tM rg, and the stored note ID is also initialized. A series of processing from S25 to S28 is performed only when the coordinates of rec tMr g are not in the initial state (See S24). After this processing is completed, or when it is determined in S25 that the value is not exceeded, in Step S29, the periphery of the vertical line of the stem and tail is scanned with a certain width dw. In step S30, it is determined whether or not the bit bit Drm exists at the coordinates (x, y). If not, the process jumps to step S48, which will be described later, and ends the X-direction scanning loop. In S30, if the same bit exists, the painting function is applied to this bit in step S31 to detect a second label (rectangle) (rect Cur). In step S32, it is determined whether the found second label rectangle is larger than the maximum size (dxt) of the label combination, and if it is determined to be larger, tie elimination processing described later is performed in step S33. In step S34, it is determined whether or not the tie elimination process has succeeded. If the tie elimination process has failed, the process jumps to step S42, and the bottom y coordinate of rect Curr is set to yB ef. If it is determined in S34 that the tie elimination process has been successful, and if it is determined in S32 that the found second label rectangle is smaller than the above dXt, in step S35, the second label rectangle is determined. It is determined whether or not the label is large enough for matching. If the label is small, the process jumps to step S48 to be described later and ends the X-direction scanning loop. If it is determined in S35 that the size is sufficient, then in step S36, matching is first performed once only with the second label (rectCur). In step S37, it is determined whether or not the priority of the ID is the maximum. If it is determined that the priority is the maximum, the process jumps to step S42, where the bottom y coordinate of rect Cur is set to yB ef, and In S46, notehead data is created. If it is determined in S37 that the priority is not the maximum, it is determined in step S38 whether or not the coordinates of the label rec tMrg to be combined match the default value. If not, step S39 is performed. In, the combining process of the first label and the second label is performed, and if they match, the combining process is not performed in step S40, and the shape as it is is set as a merge rectangle (rec tMr g). In step S41, the bottom y coordinate of the above rec tMr g is set to the above y B ef. Next, in step S43, it is determined whether or not rec tMr g is large enough for matching. If it is determined that rec tMr g is not sufficient, the process jumps to step S48 to be described later and ends the loop of the X-direction running. I do. Judged that the size is large enough in S43 Then, in step S44, matching with the dictionary of rec tMr g is performed. In step S45, it is determined whether or not the priority of the ID is the maximum. If it is determined that the priority of the ID is the maximum, the step The process jumps to S46 to create notehead data using the saved notehead ID and the rectangle. If the priority is not the highest, the process jumps to step S48 to terminate the X-direction scanning loop. After the above step S46, in step S47, a process of setting a default value to each coordinate of rec tMr g (end of label combination) is performed, and a stored note ID is initialized. If it is determined in S45 that the priority is not the highest, and if the label combination end processing is performed in S47, the rape of the X-direction scanning is ended in step S48. Further, in step S49, the loop of scanning in the y direction is ended. Thereafter, in step S50, it is determined whether or not the remaining stored notehead ID is in the initial state, and if it is in the initial state, the process ends. If it is not the initial state, notehead data is created in step S51.

FIG. 26 shows the flow of the rectangular matching process in steps S36 and S44 in FIG. In step S52, matching with the dictionary is performed, and the processing result is input to the ID. In step S53, it is determined whether or not the matching has succeeded. If it is determined that the matching has failed (ID—MATCH I NGNONE), the process ends. If it is determined in S53 that no matching failure has occurred, it is determined in step S54 whether or not the matching result is reject travel, and if it is determined that reject travel, the process ends. If it is determined in step S54 that it is not a reject travel, then in step S55, it is determined whether or not the matching result is a combining notehead. If it is determined that the matching result is a combining notehead, in step S56, Note ID and rectangle are saved. On the other hand, if it is determined in S55 that the character is not a head, then in step S57, it is determined whether or not the matching result is a tremolo mouth symbol.If it is determined that the symbol is a tremolo symbol, in step S58, The tremolo thick portion is erased. On the other hand, if it is determined in S57 that the symbol is not a tremolo symbol, the notehead ID and the rectangle are stored in step S59. After finishing the processing of S56, S58 and S59, the entire processing is finished.

(3) Tie erasure processing If the width of the label is wider than the notehead as described above, it is considered that the tie is connected to the notehead, and the label cannot be separated, so the tie elimination performed in step S32 and below in Fig. 23 Processing will be performed. As shown in Fig. 27, scanning is performed vertically from the left and right ends of the rectangle in the opposite direction, and one run is found, and the X width where the run length is less than a certain threshold value dyt 2 is obtained. If it is greater than or equal to a certain threshold value dXt2, it can be determined that this is a notehead. In that case, the run length will be longer than a certain length dyt 2 (the part of the tie in contact with the notehead will have a run length longer than dyt 2) or the second and subsequent runs (while scanning the run) The run that is found separately in the vertical direction) is used as a tie, and up to that point is erased.

(4) Combination notehead processing

There are few chords with noteheads in contact with drum staves, and there is no problem with contact between black balls or between black balls and other noteheads, but when instruments other than ordinary drum sets are written on the same staff As shown in FIG. 3, there may be a chord in which (X) contacts each other. In such a case, it is difficult to set the recognition rectangle by separating each notehead before matching with the dictionary. For this reason, the combination of note heads whose note heads touch each other up and down is registered in the dictionary, and when this dictionary and the combination note head match (see step S55 in FIG. 26), Data is added as many as the number of noteheads linked to the notehead data (see step S56 in the figure). Since such a label has a long shape in the vertical direction, the ID of the dictionary can be changed according to the vertical width of the label to reduce the rate of misperception and recognition. It is also effective to add a feature value by size to the matching itself.

(5) Tremolo symbol processing

By following the procedure described so far and combining the labels around the suffix and matching the dictionaries, the tremolo symbol as shown in Fig. 4 can also be recognized. Especially in the case of drum music, there are relatively many tremolo symbols. Since the tremolo symbol has almost the same thickness and spacing as the hook, if no countermeasures are taken, the flag will be counted in the same way as the hook, and it will be mistaken for a short note such as a 32nd note. Resulting in. Therefore, the tremolo symbol is registered in the dictionary as shown in Fig. 5. However, in this configuration, the symbol pasting interface does not support tremolo symbols. If a match is found (see step S57 in Fig. 26), the thick part is erased (see step S58 in Fig. 26) to take measures against flag count failure when detecting consecutive hooks. (6) Note-stem combining process

In this configuration, the process of combining noteheads and stems is performed after recognition of noteheads as in the case of normal music score recognition. Is required.

① Positional relationship between note heads and stems

One is that noteheads other than black balls may have a stem at the center of the notehead, as shown in Fig. 2. In addition, note heads are rarely attached to both sides of the stem. In such a case, the first notehead of the upward stem or the downward stem is combined, and then the middle notehead is combined. In this case, if this is not a black ball or a white ball, either the center of the ellipse or the stem coordination flag is checked according to the positional relationship between the center of the ellipse and the X coordinate of the stem, and the head of the stem is checked. Regarding the notehead of, it is determined that the combination is impossible based on the positional relationship in the X direction. However, in the case of a drum notehead (other than a black ball), when combining the first notehead, Decide whether to check left or right depending on whether you are above or below. That is, if it is below the midpoint, the right side of the note head is checked regardless of the positional relationship in the X direction. The middle noteheads determine the check direction depending on whether the stems are facing up or down. The reason why the notehead combination flag is used in this way is to avoid combining stems by disabling the combination of stem candidates on the side where the flag is checked.

② Noteheads that divide stems

Another reason that special combining processing must be performed in the case of drum staves is the presence of noteheads that separate stems as shown in Figs. 9 (a) and 9 (b). As shown in the figure, most of the noteheads that divide stems are black balls at the beginning, and there is one notehead that separates them. Therefore, in the present configuration, first, the black ball is not targeted, and as shown in FIG. 10 (a), the note head dividing the stem and the upper and lower note heads are combined. Then, a drum note having the opposite stem direction and sharing the note head as shown in FIG. Then, the stem-coupling process is performed. However, in this case, the stem length in the state before concatenation becomes short, so the shortest length of the stem length combined with the notehead is The value must be kept low. Before the stem concatenation process, the note with the downward stem on the right side of Fig. 10 (b) is deleted, and the stem candidates are changed to those combining the upper and lower stems as shown in Fig. 10 (c). Yes (right side of the figure). Then, when note-stem concatenation is also performed on black balls, the stem candidates combined in the above process and black balls are connected. '

③ 〇 Note head and hi-hat open

Yet another reason for the special combining process for drum staves is that, in order to recognize the notehead, if the hi-hat open (°) is near the stem, it is mistaken for a notehead. It is possible that As a countermeasure against this, in this configuration, the 〇 note head checks the left side of the left and right stem combination flags in advance. In this way, the 頭 note head has no downward stem. However, as shown in Fig. 28, a recognition rectangle is set for each part in one image (indicated by the wavy line in the figure). May enter. Considering the danger of the hi-hat open in the next paragraph entering the recognition rectangle and the possibility of using the hi-hat open (°) for the pedal noisy hat below the first line, The lower 〇 mark head may check the flag on the right instead of the left.

④ Hi-hat closed, countermeasures for mistaken tuplet numbers

In addition, the hi-hat closed (+) symbol shown in Fig. 11 ゃ The tuplet number is mistaken for a note head, and when this is combined as the head of a stem, the stem direction is completely different. It will be recognized as a note. As a countermeasure, these symbols and tuplet numbers may be registered in the dictionary as rigid travel. However, since the stem direction of drum staves is different from that of regular music, it can be limited to some extent. Is above the fifth line and the lower end of the stem is above the first line, the stem is also effective as a countermeasure against false recognition of hat-open (°)], and the lower end of the stem is above the first line. When the lower stem and the upper end of the stem are below the fifth line, it is determined that the stem is downward, and the first note head is combined with the candidate stem.

However, when the above-described recognition process of the noteheads that divide the stems and the combining process of these stemheads are performed, the configuration for determining the stem direction described above is implemented, as shown in FIG. It is not possible to recognize a configuration in which a downward stem is connected to a common note head. If is above the fifth line and the lower stem is above the first line, it is determined to be an upward stem.) Therefore, in the present configuration, after performing the above-described processing for the notehead that divides the stem, normalization in the stem direction is performed. That is, at the time of recognizing a drum note including a note having a note head that divides a stem, the leading note head of these drum notes and a stem candidate are combined, and a drum note having a note head that divides a stem is recognized. After recognizing drum notes with opposite stems sharing the same notehead, deleting notes with lower stems, and combining the upper and lower stems as stem candidates, the remaining first noteheads are used. And for other drum notes, if the upper stem is above the fifth line and the lower stem is above the first line, the stem is upward, the lower stem is below the first line and If the upper end of the stem is below the fifth line, the stem is determined to be downward, and the candidate stem is combined with the rest of the stems. After returning to the candidate (stem direction normalization), the candidate is combined with the remaining note heads. It is set to be.

削減 Reduction of false recognition by ratio of thick part on stem

Even in normal scores, if the proportion of thick parts on stem candidates was high, it was regarded as a misidentification of stems and excluded from note head combining. In the case of a drum score, since the number of chords of a black ball is small, the threshold can be lowered to reduce the probability of misrecognition. Especially in the case of drum staves, note head labels are divided and matched, so the labels around the vertical lines detected on symbols that are not actually notes are matched while dividing and joining. You may be mistaken as a head. Therefore, the above processing becomes effective. However, care must be taken when processing noteheads that split stems, since there are short catches. In order to solve this, the note with the lower stem on the right side of Fig. 10 (b) is deleted and the stem candidates are combined with the upper and lower stems as shown in Fig. 10 (c). (Right side of the figure), and then perform note-stem concatenation on black balls as well, detect the proportion of the thick part on the stem, and delete the note if it is higher than the threshold. Good luck,

In addition, in the case of misidentification of a note, considering that the possibility of misidentification to a continuous hook is low and that the proportion of the thick portion of the note where the continuous hook exists is relatively large, only the thick portion without the continuous hook is used. It is also effective to check by the ratio of minutes and to make the threshold for checking the ratio of the thick part of those without hooks stricter than the one with hooks. In the configuration of the present embodiment described above, in the same manner as in the normal score, in the drum score, a thick and thin separation process performed by erasing staff lines and the like and an ellipse detection process for recognizing a black ball portion are performed. The same black ball is detected. The noteheads of drum staves other than the black ball and the white ball with low appearance probability (no recognition) are drawn with thin lines such as (X, ◊), and there are various other types. However, these cannot be detected by the thick and thin separation processing and the ellipse detection processing as in the case of black ball detection. This is done by matching the detected label around the tail candidate with the dictionary. However, noteheads may be split into multiple labels due to stem erasure, so if matching fails, further scanning of stems candidates is continued and the next detected notehead label and the separated label After performing the join processing of, matching with the dictionary is performed again. In addition, since there is a unique process for recognizing and combining drum staves, each process is performed as described above. According to the configuration of the invention described in claims 1 to 16 described above in detail, since the drum notation is recognized while estimating the notation of the drum notation described in various notations, a complicated It is possible to convert to a data format that can be played with few operations without using an interface and without burdening the user. Also, users who do not have knowledge of drum scores can use them.

Further, according to the configuration of the invention described in claims 17 to 36, it is possible to recognize drum notes in a drum score different from a normal score, and to convert to a data format that can be played and played or displayed. Will be able to In particular, in recognition of noteheads in drum staves, erasing stems and separating notehead labels, scanning candidate stems, and matching detected labels around stem candidates with the dictionary In addition, note endings may cause the note head to be divided into multiple labels, so if matching fails, scanning of stem suffix candidates is continued and the next detection is performed. By performing the matching process between the notehead label and the separation label and then matching the dictionary again, accurate recognition of the drum note becomes possible. Industrial applicability

As described above, the configuration of the computer-readable recording medium that records the score reading method and the score reading program according to the present invention is useful when reading a score, especially a drum score, mechanically. It is suitable for creating data for karaoke performances from musical scores, or for performing part of a machine performance instead of human performance in studio recording or live performance.

Claims

The scope of the claims

1. a symbol recognition step of recognizing all symbols of a musical score including a note, and a notation estimating step of estimating a drum notation in a drum part of the musical score based on information obtained in the step; In accordance with the drum notation estimated in the step, an instrument assigning step of assigning an actual sounding instrument to the recognized symbol of the drum part, and a data conversion step of converting into a performance and / or a displayable score data format are executed. A score reading method characterized by this.

2. The score reading method according to claim 1, wherein the symbols in the drum part of the score recognized in the symbol recognition step include a note head of a drum note, a stem, a flag, a symbol relating to a drum note, and a drum score. 2. The method according to claim 1, further comprising the step of recognizing the symbol relating to the drum note, and detecting the relationship between the symbol relating to the drum note and the drum note together. How to read music.

3. In the score reading method according to claims 1 or 2, at least when the drum notation is estimated in the notation estimating step, at least a staff position of a drum note head and a drum obtained by a symbol recognition function. Note head type, hi-hat open symbol for drum notes, hi-hat close symbol for drum notes, accent symbols for drum notes, stems of drum notes, flags attached to drum notes, drum notes for notes These information, including a character string for the following, a pitch determined by the flag of the drum note, and another pitch determined by the type of notehead given the length of the other drum note determined by the flag 3. The score reading method according to claim 1, wherein the score is read based on the score.

4. The score reading method according to any one of claims 1 to 3, wherein the notation estimation step estimates the drum notation for a drum instrument other than a cymbal system. If black balls and other types are detected, black balls are used as snare drums, (R) or black balls are used as rim shots, and noteheads are between the third staff lines and only black balls are detected. Character string representing the rim shot 4. The musical score reading method according to claim 1, wherein, if issued, the black ball specified by the character string is a rim shot, and the other black balls are a snare drum.

5. In the score reading method as set forth in claims 1 to 3, wherein the notation estimation step includes estimating a drum notation for a drum instrument other than a cymbal system, the notehead is below the first line of the staff. 4. The score reading method according to claim 1, wherein the drum note is a pedal hi-hat if it is not a black ball.

6. The score reading method according to any one of claims 1 to 5, wherein in the notation estimation step, the drum notation for drum instruments other than cymbals is estimated, and the head is a black ball and the third interval is excluded. If the black ball is detected in the range described as a bass drum or tom, if the lowest row is below a certain pitch, it is used as a bass drum, and the rest is assigned to multiple pitch toms, and the bottom row is 6. The score reading method according to claim 1, wherein when the pitch is higher than a certain pitch, all the pitches are assigned to toms having a plurality of pitches.

7. The score reading method according to any one of claims 1 to 3, wherein, in the notation estimation step, the drum notation of the cymbal-based drum musical instrument is estimated to have drum heads other than black balls in the fourth or longer interval. Provided that the staff is at the staff position, the cases are divided according to the case where the staff of the drum notehead is one and two, and the above cases are further subdivided according to the type of the notehead of the drum. In addition to the duration determined by the flag of the drum note, and the duration of the other drum note determined by the flag, the above case division is further subdivided according to the different duration determined by the notehead type. A conversion table is created according to these cases, and the staff positions of the drum note heads, the type of drum note heads, the hi-hat open symbol for drum notes, and the hi-hat close for drum notes are closed. The estimated cymbal-type drum instrument type is specified in the conversion table according to the symbol, the accent symbol related to the drum note, and the character string for designating the drum sound of the note, and based on the conversion table. 4. The score reading method according to claim 1, wherein a drum notation is estimated.

8. The score reading method according to claim 7, wherein the notation estimation step is performed. When assigning the actual pronunciation instrument to the recognized drum part symbol in accordance with the estimated drum notation, the value of the conversion table for the drum note recognized as a quarter note before the estimation is set to 2 minutes. 8. The score reading method according to claim 7, wherein when a note is specified, the drum note is converted into a half note.

9. A computer-readable recording medium storing a score reading program, a symbol recognition function for recognizing all symbols of a score including musical notes, and a drum part of the score based on information obtained by the function. Notation estimation function for estimating the drum notation in, an instrument assignment function for assigning the actual pronunciation instrument to the recognized drum part symbol in accordance with the drum notation estimated by the function, performance and Z or display A computer-readable recording medium that stores a score reading program for executing a data conversion function for converting the data into a simple score data format.

10. A computer-readable recording medium recording the score reading program according to claim 9, wherein the symbols in the drum part of the score recognized by the symbol recognition function are: It includes a symbol related to a drum note and a character string characteristic of a drum score, and in recognizing the symbol related to the drum note, the relationship between the symbol related to the drum note and the drum note is also detected. A computer-readable recording medium storing the musical score reading program according to claim 9.

11. In a computer-readable recording medium on which a score reading program according to claims 9 to 10 is recorded, when the notation estimating function is used to estimate the drum notation, it is obtained by a symbol recognition function. , At least the staff position of the drum note head, the type of drum note head, the hi-hat open symbol for the drum note, the hi-hat close symbol for the drum note, the accent symbol for the drum note, the suffix of the drum note, the drum A flag attached to the note, a character string for designating the drum sound of the note, a note length determined by the flag of the drum note, and a note length determined by the type of notehead given the length of the other drum note determined by the flag The score reading program according to any one of claims 9 to 10, characterized in that the score reading program is performed based on such information including the pitch of the music. Computer readable recording medium.

12. A computer-readable recording medium on which the musical score reading program according to claims 9 to 11 is recorded, wherein the notation estimation function estimates a drum notation for a drum instrument other than a symphonore. If the notehead is between the third staff and a black ball and other types are detected, the black ball is a snare dram, (R) or a non-black ball is a rim shot, and the notehead is a staff. If only the black ball is detected and a character string representing the rim shot is detected, the black ball specified by the character string is set as the rim shot, and the other black balls are detected. A computer-readable recording medium recording a score reading program according to any one of claims 9 to 11, wherein the program is a snare program.

13. A computer-readable recording medium on which the score reading program according to claims 9 to 11 is recorded, wherein the notation estimation function estimates a drum notation for a drum instrument other than a symphonore. The musical score reading program according to any one of claims 9 to 11, wherein the drum note is a pedal hi-hat if the note head is below the first line of the staff and is not a black ball. Computer-readable recording medium.

14. In a computer-readable recording medium recording the score reading program according to claims 9 to 13, wherein the notation estimation function estimates a drum notation for a drum instrument other than a sympanore-based drum instrument. If the head is a black ball, and the black ball is detected in the range indicated as a tom, excluding the third period, and a tom, if the bottom row is below a certain pitch, it is regarded as a bass drum, The score reading program according to any one of claims 9 to 13, wherein the remainder is assigned to toms of a plurality of pitches, and if the lowest row is higher than a certain pitch, all of the notes are assigned to toms of a plurality of pitches. A computer-readable recording medium on which is recorded.

15. In a computer-readable recording medium on which the musical score reading program according to claims 9 to 11 is recorded, when the notation estimation function estimates a drum notation for a cymbal-based drum instrument, the black notation is used. Provided that the drum note head other than the ball is at the staff position at or above the fourth interval, the drum note head is divided into one staff position and two staff members, and the drum note head The above case division is subdivided according to the type, and the pitch determined by the flag of the drum note, and the pitch determined by the flag Given the length of the other drum notes, the above case is further subdivided according to the different note length determined by the notehead type, and a conversion table is created according to these cases. In addition, the staff position of the drum note head, the type of the drum note head, the hi-hat open symbol related to the drum note, the hi-hat closed symbol related to the drum note, the accent symbol related to the drum note, and the designation of the drum sound of the note The type of cymbal-based drum instrument to be estimated is specified in a conversion table according to a character string used for the estimation, and the drum notation is estimated based on the conversion table.

A computer-readable recording medium that records the score reading program according to any one of items 9 to 11.

16. A computer-readable recording medium on which the score reading program according to claim 15 is recorded, wherein a symbol of the recognized drum part is actually added to the symbol of the recognized drum part in accordance with the drum notation estimated by the notation estimation function. When assigning pronunciation instruments, if the conversion table value specifies a half note for a drum note that was recognized as a quarter note before estimation, these drum notes are converted to half notes. A computer-readable recording medium storing the musical score reading program according to claim 15, which is converted.

17 7. In a score reading method of reading an image of a score and recognizing the musical symbol and converting it into a musical form that can be played and / or displayed, a note head label is separated using stem information, and a stem candidate is used. A music label reading method comprising matching a separation label around the stem candidate detected by scanning with a dictionary, recognizing a drum head symbol, and performing a stem-note head combining process. .

18. In the score reading method according to claim 17, when matching between the separation label and the dictionary fails, scanning of stem candidates is continued and the next detected note head label and the separation label are compared. 18. The score reading method according to claim 17, wherein after performing the combining process, matching with the dictionary is performed again.

19. The score reading method according to claim 18, wherein the notehead label itself detected next and used in the combining process is also matched with the dictionary, and if the matching result has a higher priority, the The score reading method according to claim 18, wherein the recognition result is prioritized.

20. In the score reading method according to claim 18, the label combining is repeatedly performed, and priorities are given to respective recognition results recognized and matched during the combining, and the recognition is performed in descending order of priority. 19. The score reading method according to claim 18, wherein the result is determined.

21. In the score reading method as set forth in claims 17 to 20, the tie in contact with the note head is erased to perform label separation, and the drum note including the note head in contact with the tie is recognized. The score reading method according to any one of claims 17 to 20, wherein the score reading method is performed.

22. In the score reading method according to claims 17 to 21, the combined noteheads whose noteheads touch each other up and down are registered in a dictionary, and the dictionary is matched with the combined noteheads. The musical score reading method according to any one of claims 17 to 21, wherein a combined notehead is recognized.

23. In the score reading method according to claims 17 to 22, the tremolo symbol is registered in a dictionary, and the tremolo mouth symbol is recognized and deleted, and then displayed including the tremolo mouth symbol. The score reading method according to any one of claims 17 to 22, wherein the other drum notes that have been read are recognized.

24. In the score reading method according to claims 17 to 23, when recognizing a drum note having a note head that divides a stem, a note head that divides a stem and candidate stems above and below it are recognized. After recognizing drum notes with opposite stems that share the note head by combining them, delete notes with lower stems and combine upper and lower stems as candidates for stems. The score reading method according to any one of claims 17 to 23, wherein the score reading method is combined with a note head.

25. In the score reading method according to claims 17 to 23, when the upper end of the stem is above the staff and the lower end of the stem is not below the staff, the stem is directed upward. If the lower stem is below the staff and the upper stem is not above the staff, the stem is determined to be downward and the first note head is combined with the candidate stem. The musical score reading method according to any one of claims 17 to 23, wherein the musical score is read.

26. In the score reading method according to claim 24, when a drum note including a note having a note head that divides a stem is recognized, a leading note head of these drum notes is recognized. For a drum note having a notehead that divides a stem by performing the process of combining stems, a note with a stem that is downward is deleted after recognizing a drum note with an opposite stem that shares the notehead. Then, the combination of the upper and lower stems is combined with the remaining first noteheads as stem candidates, and for other drum notes, the upper end of the stem is above the staff and the lower end of the stem is If the stem is not below the staff, the stem is upward, if the bottom of the stem is below the staff and the upper stem is not above the staff, the stem is downward. Is determined, the candidate is combined with the remaining noteheads, and those that do not meet these conditions are returned to the original candidate, and then the candidate is combined with the remaining noteheads. The score reading method according to claim 24, wherein the score is read.

27. On a computer-readable recording medium that stores a score reading program that reads an image of a score, recognizes the music symbol, and converts it into a playable and / or displayable score format, it uses the stem information to add noteheads. A label separation function for separating the labels of the characters, a scanning function for scanning the stem candidates, and matching the separation labels detected around the candidate stems detected by the scanning with the dictionary to recognize the note head. A computer-readable recording medium that stores a score reading program for executing a recognition function of recognizing a drum notehead and a combining function of combining a recognized notehead and a stem candidate.

28. In a computer-readable recording medium recording the score reading program according to claim 27, a matching judgment function for judging success or failure of matching between the separation label and the dictionary; The program further includes a program that causes the scan function to continue scanning for stem candidates, and executes a label combining function of performing a combining process of the next detected note head label and the separation label. 28. A computer-readable recording medium storing a musical score reading program according to claim 27, wherein a matching function is performed again on the combined label and the dictionary for the recognition function.

29. In a computer-readable recording medium recording the score reading program according to claim 28, the note head label itself detected and used next for the combining process is also converted to a dictionary by the recognition function. Matching is performed, and if the matching result has a higher priority, the recognition result is given priority. A computer-readable recording medium storing the musical score reading program according to claim 28.

30. In a recording medium readable by a computer recording the music score reading program according to claim 28, the notehead recognition and the label combination are repeatedly performed during the notehead recognition while the notehead recognition and label combination are repeatedly performed. It further comprises a program for executing a priority adding function for assigning priority to each of the recognized recognition results, wherein the recognition function corrects the recognition results in descending order of priority. A computer-readable recording medium recording the musical score reading program according to claim 28.

31. A computer-readable recording medium on which the score reading program according to claims 27 to 30 is recorded, further comprising: a program for executing a type erasing function for erasing a tie contacting a notehead, The musical score reading program according to any one of claims 27 to 30, wherein label separation is performed by a label separation function after the erasure, and the note head to which the tie is in contact is recognized by the recognition function. A computer-readable recording medium on which the data is recorded.

32. In a computer-readable recording medium on which a score reading program according to claims 27 to 31 is recorded, a combined note head in which note heads vertically contact each other is registered in a dictionary, and the recognition is performed. 31. A computer-readable recording recording a score reading program according to any one of claims 27 to 31, wherein the dictionary is matched with the combined notehead by a function to recognize the combined notehead. Medium.

33. In a computer-readable recording medium storing the score reading program according to claims 27 to 32, a tremolo symbol is registered in a dictionary, and the tremolo symbol is recognized by the recognition function. 27. A program according to claim 27, further comprising a program for executing a symbol erasure function for erasing said tremolo symbol, and performing recognition of another drum note displayed including the tremolo mouth symbol after erasure. 33. A computer-readable recording medium on which the musical score reading program according to any one of to 32 is recorded.

34. In a computer-readable recording medium having recorded the score reading program according to claims 27 to 33, a driver having a note head for dividing a stem. At the time of recognizing a note, a notehead that divides a stem by the combining function is combined with a candidate for a stem above and below the notehead, and a drum note having a notehead opposite to that sharing the notehead is recognized by the recognition function. It has an execution program of a down note deletion function that deletes notes with lower stems and a stem combination function that combines upper and lower stems, which are executed later. 34. A computer-readable recording medium storing a musical score reading program according to claim 27, wherein the remaining noteheads are combined by the combining function as candidates.

35. In a computer-readable recording medium storing the score reading program according to claims 27 to 33, the upper end of the stem is above the staff and the lower end of the stem is below the staff. If not, the stem is upward, and if the lower stem is below the staff and the upper stem is not above the staff, the stem is judged to be downward. The method according to any one of claims 27 to 33, further comprising a program for executing a function, wherein after performing the determination, the leading notehead and the stem candidate are combined by the combining function. A computer-readable recording medium that stores a score reading program.

36. In a computer-readable recording medium recording the score reading program according to claim 34, when a drum note including a note having a note head that divides a stem is recognized, the combining function is used to recognize these notes. For a drum note having a note head that divides a stem by performing a process of combining the first note head and the stem candidate of the drum note, the above-mentioned recognition function is used to generate a note note in the opposite direction that shares the note head. After recognition, the downward note deletion function deletes notes with lower stems, and the upper and lower stems combined by the stem combination function are used as stem candidates, and further by the combination function. In addition to combining with the rest of the first notehead, for the other drum notes, the above-mentioned stem-steering function determines that the upper end of the stem is above the staff and the lower end of the stem is below the staff. Absent In this case, if the stem is upward, the lower end of the stem is below the staff and the upper end of the stem is not above the staff, it is determined that the stem is downward and the combining function determines that the stem is downward. In addition to combining the stem candidates with the rest of the noteheads, those that do not meet these conditions also have a program to perform a note candidate recovery function that restores the original stem candidates, After returning to the tail candidate, the combining function links the stem candidate with the remaining noteheads. A non-transitory computer-readable recording medium storing the musical score reading program according to claim 34.