JP3607065B2 - Music score recognition method and computer readable recording medium having recorded music score recognition program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a score recognition method and a computer-readable recording medium on which a score recognition program is recorded.
[0002]
[Prior art]
In order to input characters with a scanner or the like to recognize characters and symbols included in these images, it is necessary to adjust the resolution of the image to be captured, as well as to adjust the inclination of the document that significantly affects the recognition rate. In the case of capturing a binarized image, it is necessary to determine a binary image by setting a binarization threshold value.
[0003]
[Problems to be solved by the invention]
It is relatively easy to make the document tilt and capture resolution optimal, but since the binarization threshold depends on the recognition algorithm, if the recognition algorithm is not known, Setting to the optimal value is difficult. Further, depending on the display resolution, there are many cases where the binarization state cannot be sufficiently visually checked.
Therefore, since the binarization threshold value set manually is not appropriate, the recognition performance cannot often be sufficiently obtained.
Further, after the recognition is performed, if it is found that the cause of the low recognition rate is the binarization threshold value, if the image capture is performed again from the beginning, the processing man-hours increase and the processing takes time. .
[0004]
The present invention was devised in view of the above problems of the prior art, and it is possible to automatically set a threshold value for image binarization that has a significant effect on the recognition rate to a value optimal for recognition. It is intended to provide a possible configuration.
[0005]
[Means for Solving the Problems]
Therefore, the configuration of claim 1 of the present application is executed by including a score information reading means, a threshold setting means, and an image reading means, reading a score, and automatically reading the read score data by computer processing. In a score recognition method for recognizing music symbols and creating data for performance and / or score display, the score information reading means reads the score as a gradation image, and the threshold value setting means reads the score. The score image According to a predetermined binarization threshold A binarization step; Similarly, the threshold setting means obtains the ratio of the staff width to the staff interval of the binarized score image, determines the image state from the ratio, and thereby calculates and sets the next binarization threshold Sequentially changing the threshold value to obtain a final threshold value from a threshold band when the ratio of the staff width to the staff interval becomes a target ratio; and By setting means, finally Basically performing the steps of binarizing the read musical score image with a calculated threshold value and recognizing music symbols from the binarized musical score image by the image capturing means. Characteristic.
The configuration of claim 3 provides the above procedure as a recording medium. The specific configuration reads a score, automatically recognizes a music symbol by computer processing from the read score data, and performs a performance. And / or a computer-readable recording medium storing a score recognition program for creating data for displaying a score, and by executing the program, score information reading means, threshold setting means, and image capture Means is implemented on the computer, the score information reading means reads the score as a gradation image, and the threshold value setting means reads the score image read According to a predetermined binarization threshold A binarization step; Similarly, the threshold setting means obtains the ratio of the staff width to the staff interval of the binarized score image, determines the image state from the ratio, and thereby calculates and sets the next binarization threshold Sequentially changing the threshold value to obtain a final threshold value from a threshold band when the ratio of the staff width to the staff interval becomes a target ratio; and By setting means, finally A score for executing the step of binarizing the read score image with a threshold value calculated and the step of recognizing a musical symbol from the binarized score image by the image capturing means. A computer-readable recording medium having a recognition program recorded thereon.
[0006]
In these configurations, the binarization threshold is set based on the score information. Includes a 5-line recognition process to determine if the threshold is appropriate, Therefore, the setting by the trial and error of the operator is not necessary, and the optimum threshold value is automatically set and recognized. As a result, the overall recognition rate can be improved. In that case, Used in the 5-line recognition process As musical score information, as shown in FIG. 1, a staff interval H and a staff line width t are obtained, and a threshold value at which this ratio is optimal is detected. The reason will be described below. When the binarization threshold is changed, the density of the binary image changes and the thickness of the line changes. Therefore, the line width of the staff of the score image changes depending on the binarization threshold. Various music scores tone Imported as an image, binarized with various threshold values, the value obtained by normalizing the line width of the staff by the staff interval for the one with the highest recognition rate (the staff line relative to the staff interval) When the ratio of width takes a substantially constant value, a binary score image can be created with a substantially reasonable recognition rate by setting a binarization threshold value so as to approach this value. When the ratio of the line width of the staff to the staff interval at the optimum threshold changes depending on the resolution, etc. (If the resolution is low, the higher the ratio of the staff width to the staff interval, the higher the recognition rate, On the contrary, when the resolution is high, the opposite is true). For the staff interval, the target value may be changed by a linear function or table conversion.
[0007]
In addition, if the line width and interval of the staff are different for each paragraph or part, it is not possible to set an appropriate threshold after all by simply calculating the interval of the staff and the line width of the staff. Become. Therefore, the configuration of claim 2 divides the image into a plurality of blocks (for example, divided into paragraphs, parts, or staffs, and further divided in the horizontal direction), and each block is binarized using score information. The threshold value is set and the score image is captured as a binary image. That is, a musical score information reading unit, a threshold setting unit, and an image capturing unit are executed to read a musical score, and the musical score data that has been read is automatically recognized by computer processing to perform a performance. And / or in a score recognition method for creating data for displaying a score, a step of reading a score as a gradation image by the score information reading means, and a score image read by the threshold setting means According to a predetermined binarization threshold By the step of binarization and the threshold value setting means, The binarized musical score image is divided into a plurality of blocks, the divided musical score image is binarized by a predetermined binarization threshold value for each block, and the staff corresponding to the staff interval of the binarized musical score image The ratio of the line width is obtained for each block, the image state is determined from the ratio, and the next binarization threshold value is calculated and set for each block, and the threshold value is sequentially changed to correspond to the staff interval. A step of obtaining a final threshold value for each block from a threshold band when the staff width ratio becomes a target ratio, and finally, for each block by the threshold setting means The score image that has been read with the threshold value calculated For each block The binarization step and the image capture means With different binarization threshold for each block And a step of recognizing a musical symbol from the binarized musical score image.
[0008]
Similarly, the configuration of claim 4 provides the above procedure as a computer-readable recording medium on which a score recognition program is recorded. The specific configuration reads the score, and the read score data is stored in the computer. A computer-readable recording medium storing a score recognition program for automatically recognizing music symbols by processing and creating data for performance and / or score display, and by executing the program, score information reading means And a threshold setting means and an image capturing means are realized on the computer, the score information reading means reads the score as a gradation image, and the score read by the threshold setting means Image According to a predetermined binarization threshold By the step of binarization and the threshold value setting means, The binarized musical score image is divided into a plurality of blocks, the divided musical score image is binarized by a predetermined binarization threshold value for each block, and the staff corresponding to the staff interval of the binarized musical score image The ratio of the line width is obtained for each block, the image state is determined from the ratio, and the next binarization threshold value is calculated and set for each block, and the threshold value is sequentially changed to correspond to the staff interval. A step of obtaining a final threshold value for each block from a threshold band when the staff width ratio becomes a target ratio, and finally, for each block by the threshold setting means The score image that has been read with the threshold value calculated For each block The binarization step and the image capture means With different binarization threshold for each block The recording medium includes a score recognition program for executing a step of recognizing a music symbol from a binarized score image. Also in this configuration, it goes without saying that the staff interval and the line width of the staff are suitable as the musical score information.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
(Example 1)
Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. FIG. 2 shows a musical score which is read and operated by an external storage device (or a CD-ROM drive or the like) such as a flexible disk drive FDD5 described later. It is a block diagram which shows the Example structure of a recognition apparatus. In this apparatus, a scanner and a MIDI interface circuit are added to the configuration of an electronic computer such as a personal computer. The CPU 1 is a central processing unit that controls the overall score recognition apparatus based on a program stored in the ROM 2 or RAM 3. A timer circuit for interrupting the CPU 1 at a predetermined cycle set in advance is incorporated. The RAM 3 is used as an image data buffer, a work area, etc. in addition to a program area. The hard disk device HDD4 and the flexible disk drive FDD5 store programs, image data, performance data, and the like. The CRT 6 displays video information output from the CRT interface circuit 7 based on the control of the CPU 1, and information input from the keyboard 8 is taken into the CPU 1 through the keyboard interface circuit 9. The printer 10 prints the print information output from the printer interface circuit 11 based on the control of the CPU 1.
[0010]
The scanner 12 optically scans a (printed) musical score, for example, and converts a grayscale or color image into a single color. tone This is to convert to attached data (however, binary images can be taken in), and any type of scanner such as flatbed type, handy type, feeder type, etc. can be used. Image information read by the scanner 12 is taken into the RAM 3 or the HDD 4 via the scanner interface circuit 13. The MIDI interface circuit 14 is a circuit that transmits and receives MIDI data to and from an external MIDI device such as a sound module. The bus 15 connects each circuit in the score recognition apparatus and exchanges various data, programs, addresses, and the like. In addition, a pointing device such as a mouse or a serial interface circuit such as RS232C may be provided.
[0011]
FIG. 3 is a flowchart showing the main processing of the CPU 1. In step SI, an image of a musical score (in which the staff width and staff spacing are also taken in as musical score information) is fetched into the RAM 3 by the scanner 12. Single color image tone Import as an attached image. Of course, an image file saved on a hard disk or the like in such a format may be read. In the case of a color image, monochrome processing is performed by processing according to the image format. tone It can be converted into an attached image. If the image imported from the scanner or read as a file is a binary image, it cannot be automatically binarized, so it can be recognized as it is or for automatic binarization. tone Import again as an attached image. When the storage capacity of the RAM 3 into which the musical score image is taken is insufficient, the image can be partially processed after being partially stored in the RAM 3. In this configuration, one pixel is 8 bits 256. tone However, the number of bits is not limited to this. Also tone Is described below, assuming that the larger numerical value is white and the smaller numerical value is black.
[0012]
In step SII, automatic binarization processing (threshold setting processing) is performed. This automatic binarization process will be described later. In step SIII, a binarized image is captured, and a score image is captured as a binarized image based on the set binarization threshold.
[0013]
FIG. 4 shows the main process steps of the automatic binarization process. First, in step S1, a threshold value (thres) is detected by a bisection method. It is possible to detect the target line width and staff interval in all 256 stages and obtain the target threshold value. However, since there is a problem with the execution speed, the optimum threshold value is obtained by the bisection method. It was to be. This process will be described later.
[0014]
As mentioned above, 256 tone If so, the target threshold value can be obtained in about eight dichotomy loops. However, in this embodiment, since the staff interval and the line width of the staff are obtained by integers, the ratio of the line width of the staff to the staff interval changes stepwise, and the target interval is less than 8 times. In some cases, a threshold can be detected. In this state, the threshold detection processing for automatic binarization may be terminated, but if the band with the same value as the target ratio is wide, it is more accurate to set the threshold as the midpoint of the band. The threshold can be obtained. Therefore, in step S3, when there is a band whose ratio is the same as the target value, an upper limit and a lower limit of the band are obtained, and the middle point is set as a threshold value (thres). Step S2 is a process that is performed to determine whether or not there is a band whose ratio is the same as the target value. The value of the threshold determination process (ret) is a threshold value. It is determined whether or not the determination end code [T_OK (value when equal to the target ratio DEST_PER)] is reached. Step S3 is a process for obtaining the upper limit and the lower limit of the band and setting the middle point as a threshold value (thres) as described above. That is, in step S30, the band upper limit (thres2) is detected, in step S31, the band lower limit (thres3) is detected, and in step S32, a midpoint between the upper limit (thres2) and the lower limit (thres3) is obtained, and the threshold is set. Set as a value (thres). The detection of the upper limit and the lower limit can also be performed by a bisection method as described later. If the ratio is changed sufficiently continuously, for example, by detecting the line width and staff interval of the staff in real numbers, the detection processing of the upper limit and the lower limit of the band becomes unnecessary. Thereafter, as described above, in step SIII, a binarized image is captured, and a score image is captured as a binarized image based on the set binarization threshold.
[0015]
FIG. 5 shows a procedure for detecting a threshold (thres) by the bisection method in step S1. In step S100, initial setting is performed. That is, the maximum threshold value (thresMax) tone Maximum value 255 and again to minimum threshold (thresMin) tone The minimum value 0 is set, and 256 is set to the counter (ct) that counts the number of times the threshold detection processing loop is repeated by the bisection method, and the maximum value (retMax) of the threshold judgment processing result and the same result are set. Is set to the minimum value (retMin) of T_NOT indicating that the staff interval and the line width of the staff could not be detected.
[0016]
In step S101, an intermediate value between the thresMax and thresMin is set as a threshold value (thres). In step S102, the threshold value (thres) is determined later, and the result is set as ret. As will be described later, the determination processing result is T_OK (output result when the threshold value is determined to be equal to the target ratio DEST_PER), and T_THIN (the threshold value is determined to be smaller than the target ratio DEST_PER). , That is, the output result when the line width of the staff is determined to be thin), T_THICK (when the threshold value is determined to be greater than the target ratio DEST_PER, that is, the line width of the staff is thick) Output result when it is determined), T_NOT (output result when it is determined that the staff interval and the line width of the staff cannot be detected).
[0017]
In step S103, it is determined whether or not the determination processing result (ret) is T_NOT. If it is determined that the determination result is not T_NOT, that is, if the staff interval and the line width of the staff can be detected, the process proceeds to step S104. Then, it is determined whether or not the determination processing result (ret) is T_OK. If this process result (ret) is T_OK, the process exits from the loop and the threshold value detection process by the bisection method is terminated (that is, the determination in step S2 in FIG. 4 is YES, and the process proceeds to step S3). On the other hand, if the processing result (ret) is not T_OK, the process proceeds to step S105, and it is determined whether or not the processing result (ret) is T_THICK. If this processing result is T_THICK (when the staff width is thick), the threshold value (thres) is set to the maximum threshold value (thresMax) in step S107, and the threshold value is recalculated in the calculation formula of step S101. In this case, the maximum threshold value (thresMax) can be changed to a smaller side (the threshold value is reset so that the line width of the staff is closer to the narrow side). On the other hand, if the processing result is not T_THICK (when the staff width is thin), the threshold value (thres) is set to the minimum threshold value (thresMin) in step S106, and the threshold value is set in the calculation formula of step S101. At the time of recalculation, the minimum threshold value (thresMin) can be changed to a larger side (the threshold value is reset so that the line width of the staff is closer to the thicker side).
[0018]
In step S103, when it is determined that the determination processing result (ret) is T_NOT, that is, when the staff interval and the line width of the staff cannot be detected, the threshold value fluctuates to the darker side or the thinner side. Is determined by a threshold overflow check in step S109 and subsequent steps, and based on this, a bisection process is performed. However, since the threshold overflow check is a process with high processing cost, the threshold judgment results at both ends are stored as the minimum value (retMin) and the maximum value (retMax) of the result. In step S108, it is determined whether or not retMax and retMin are T_NOT. If both are T_NOT, a threshold value overflow check is performed in step S109. In step S110, it is determined whether the threshold value is shifted to the dark side or the light side. If the threshold value overflows to the black side, the threshold value (thres) is set to the maximum value (thresMax) in step S112. ) And the maximum value (retMax) of the threshold determination result is T_NOT. On the contrary, if it overflows to the white side, in step S111, the threshold value (thres) is set to the minimum value (thresMin), and the minimum value (retMin) of the threshold determination result becomes T_NOT.
[0019]
In the above step S108, if it is determined that one of retMax or retMin is not T_NOT, the threshold overflow check is invoked by performing a bisection process so as to approach the non-T_NOT side in step S113 and subsequent steps. The number of times can be reduced. In step S113, it is determined whether retMax or retMin is not T_NOT. If retMax is not T_NOT, in step S114, the threshold value (thres) is set to its minimum value (thresMin), and the threshold determination result The minimum value (retMin) is T_NOT. If retMin is not T_NOT, in step S115, the threshold value (thres) is set to its maximum value (thresMax), and the maximum value (retMax) of the threshold determination result becomes T_NOT.
[0020]
Unless the threshold judgment result (ret) is T_OK, the number of counters (ct) is reduced to half in step S117, and the above processing is looped until the number of counters (ct) becomes 1 (step S116). Repeat with. In this way, the start band is changed from thresMin to thresMax, the region is narrowed by the bisection method, and the final determination result becomes the binarization threshold value (thres) (reference step S1).
[0021]
After the binarized threshold value is detected as described above, as described above, in step S2, when the threshold value determination ends with ret = T_OK, in step S3, where in the T_OK band It is necessary to decide whether to use a threshold value. FIG. 6 shows a procedure for detecting the upper band limit (thres2) in step S30 when such a determination is made.
[0022]
First, in step S300, initial setting is performed. That is, the maximum threshold value (thresMax) is set as it is as the maximum threshold value (thresMax2) in this processing, and the threshold value (thres) obtained by the above processing is set as the minimum threshold value (thresMin2) in this processing. At the same time, a value (+1) obtained by subtracting the minimum threshold value (thresMin2) from the maximum threshold value (thresMax2) is set in a counter (ct) that counts the number of times the threshold detection processing loop is divided by the bisection method.
[0023]
In step S301, an intermediate value between thresMax2 and thresMin2 is set as a threshold value (thres2). In step S302, the threshold value (thres2) is determined later, and the result is set as ret. As will be described later, the determination processing result is T_OK (output result when the threshold value is determined to be equal to the target ratio DEST_PER), and T_THIN (the threshold value is determined to be smaller than the target ratio DEST_PER). , That is, the output result when the line width of the staff is determined to be thin), T_THICK (when the threshold value is determined to be greater than the target ratio DEST_PER, that is, the line width of the staff is thick) Output result when it is determined), T_NOT (output result when it is determined that the staff interval and the line width of the staff cannot be detected).
[0024]
In step S303, it is determined whether or not the determination result (ret) is T_OK. If it is determined that the determination result (ret) is T_OK, that is, if the determination result (ret) is equal to the target ratio (DEST_PER). In step S305, the threshold value (thres2) is set as the minimum threshold value (thresMin2). If it is determined in step S303 that the determination result (ret) is not T_OK, the threshold value (thres2) is set as the maximum threshold value (thresMax2). Thereafter, the number of counters (ct) is reduced to half in step S307, and the above processing is repeated in a loop until the number of counters (ct) becomes 1 (step S306). In this way, the detection flow at the upper limit of the band when the ratio of the staff width to the staff interval has a certain band, the threshold from thres to thresMax at the time when thres = T_OK is obtained by the bisection method. A search is performed to detect a boundary between a region that is T_OK and a region that is not. That is, the start band is changed from thresMin2 to thresMax2, and the region is narrowed by the bisection method. The result determined last is thres2 (reference step S30).
[0025]
The detection of the lower limit of the band in step S31 is performed in the same flow, and the result of the lower limit is thres3. For this, a search from thres to thresMin is performed by a bisection method, and a boundary between a region that is T_OK and a region that is not T_OK is detected, and the processing procedure is the same as the detection of the upper limit of the band, and is omitted. . As described above, in step S32, the middle point of thres2 and thres3 obtained as described above is set as a threshold value (thres).
[0026]
FIG. 7 shows a process flow of threshold determination in step S102 of FIG. 5 and step S302 of FIG. First, in step S1020, the staff interval (blank) and the line width (thick) of the staff are detected. That is, as shown in FIG. 8, the score is scanned in the vertical direction, and the background and the figure are separated depending on whether the value is larger or smaller than the threshold values (thres, thres2, thres3), and the background is shown in FIG. And a histogram of each run length in the figure is created. Then, the center of gravity of the value of a certain band in the vicinity of the subscript having the maximum value of the background, the figure, and each histogram or the subscript having the maximum value is defined as the staff interval (blank) and the line width (thick) of the staff. . Since the blank is actually the run length of the background, when calculating the ratio of the line width of the staff to the staff interval, (blank + thick) is applied as the staff interval (thick / 2 at both ends of the blank). ). Further, it is not necessary to scan the image in the vertical direction at all positions on the abscissa X. In order to improve the processing speed, as shown in FIG.
[0027]
If blank = 0 at this time of detection (step S1021), detection of the staff interval has failed (cannot be detected as either black or white), so the determination result (ret) is set to T_NOT in step S1028. The process ends. In addition to this, detection failure of the staff interval and staff line width may be added even when thick = 0, thick> blank, or when thick or blank exceeds a certain range. good.
[0028]
In step S1022 after this determination, the ratio (per) of the line width (thick) of the staff to the staff interval (blank + thick) is obtained. Of course, in order to obtain a more accurate value or improve the correspondence with the recognition rate, the ratio (per) may be obtained by a mathematical expression different from the flow. In step S1023, it is determined whether the ratio (per) is equal to the target ratio (DEST_PER). If it is determined that the ratio is equal, in step S1027, the determination result (ret) is set to T_OK. If it is determined in S1024 that it is greater than that, the determination result (ret) is set to T_THICK in step S1025, and if it is determined that it is smaller than that, the determination result (ret) is determined to be T_THIN in step S1026. To end the determination.
[0029]
The threshold value overflow check in step S109 in FIG. 5 refers to determining a state of being substantially black or white when binarization is performed with the threshold value. Therefore, the check can be performed by counting the pixel values that are larger or smaller than the threshold value and determining this number. Of course, it is not necessary to measure all the pixels of the image, and it is sufficient to check at the X position when detecting the staff interval and the line width of the staff. Since accuracy is not required, the X position may be further reduced for speeding up. At the time of overflow check, it is assumed that it overflowed to either black or white, and it overflowed to black when the black count number becomes more than half of the check pixel count (the total number of pixels at the X position where the overflow check is performed) If the determination is completed and the process is terminated, and the black count is less than half of the number of check pixels, it may be determined that it has overflowed to white.
[0030]
(Example 2)
In the first embodiment, the staff interval detected in the entire image and the line width of the staff are used as reference values for determining the threshold value. This is because the run length of the part other than the staff is also reflected. Due to the influence of titles, pictures, shadows of the folds of the score, etc., the peaks in the histogram deviate from the correct staff spacing and staff width, and the staff spacing and It may be different from the line width. Therefore, staff recognition is performed on an image once binarized with a rough threshold value, and the threshold detection processing of the subsequent bisection method is limited to the periphery of the staff.
[0031]
Also in this configuration, an image with a single color gradation (stored in one pixel 8-bit format) stored in the RAM 3 is binarized with an appropriate binarization threshold in the same manner as in the above embodiment. The staff recognition is not so much affected by the binarization threshold, so the temporary threshold is simply set to the center value of the gradation. The staff interval and the line width of the staff are detected with this threshold, and if this is within the permissible range of staff recognition, the staff is detected with this threshold. That is, the image of the staff portion is scanned in the vertical direction, and divided into a line portion and a blank portion depending on whether or not it is larger than the threshold value, and respective histograms are created. Based on these histograms, the staff interval and the line width of the staff are recognized. If it is not within the allowable range, a value that falls within the allowable range may be further searched by the bisection method. However, the staff recognition is stopped, and the staff interval and the staves of the entire image are stopped as in the above embodiment. You may switch to the method of detecting a line width and setting a threshold value.
[0032]
The staff recognition may be performed on a temporary binary image created separately from the gradation image area in the RAM 3. That is, the image of the staff portion is scanned in the vertical direction, and divided into a line portion and a blank portion based on 0 or 1, and respective histograms are created. Based on these histograms, the staff interval and the line width of the staff are recognized. However, in order to reduce the storage area, a temporary binarized image may be overwritten on the least significant bit of the image with gradation without creating another area (if the gradation is about 256, the least significant bit Because changes do not significantly affect threshold detection).
[0033]
Further, the staff may be directly recognized from the gradation-added image based on the obtained temporary binarization threshold value. Furthermore, the recognition process may be terminated when one staff is detected. As shown in FIG. 11, a rectangle (dotted line portion) surrounding the detected staff is set. The left and right ends are the end points of the staff. The added line is often written with a slightly thicker line than the staff, and the width between the added lines is often wider than the staff. Therefore, the upper and lower sides of the rectangle may be a region not including the added line.
[0034]
A binarized threshold value is detected by the same method as in the first embodiment. However, the processing area is limited to the above rectangle. As a result, a more accurate staff interval and staff line width can be obtained, and since the rectangle is small, the subsequent processing steps are reduced. Further, since staff recognition has already been performed, the staff interval is detected at this point, and at the time of bisection processing, it is possible to further increase the speed by detecting only the line width of the staff.
[0035]
In this embodiment, the threshold value selection method which is a normal image processing method such as a mode method or a differential histogram may be used to determine the binarization threshold value for provisional binarization. Further, the threshold judgment area is not a rectangle (rectangle), but may be a parallelogram area connecting the start and end positions of the staff, or the staff shift detected by staff recognition as shown in FIG. It may be a strip-shaped region in consideration of the amount. In addition, in order to accurately detect the line width of the staff, not only the detection of the staff interval and the line width of the staff, but also the first detected staff when the threshold value is detected by the bisection method. In addition, it is possible to add a process of scanning the figure of the image binarized with the threshold value up and down and taking the average value of the run lengths of the portions that are sure to be the staff.
[0036]
(Example 3)
It is sufficient in most cases to determine one binarization threshold in one musical score image. However, this may not be sufficient. A particularly prominent example is when there are staffs with different widths. In such a case, if binarization is performed with the same threshold value, the threshold value of the staff that has not become a reference is deviated from the optimum value, and the recognition rate decreases. Therefore, in this embodiment, staffs are detected by the same method as in the previous embodiment (all staffs are detected), and a rectangle surrounding each staff is detected. Recognition of brackets is also performed, and a rectangle for each part may be used as shown in FIG. These rectangles are rectangles for determining threshold values, and the binarized threshold values are determined within the rectangles by means similar to those in the first embodiment.
[0037]
In the binarization processing after the threshold value is determined, binarization is performed with the threshold value in the rectangle, but between the rectangles, as shown in FIG.
{Circle around (1)} When binarization is performed with the middle value of the threshold values of the upper and lower rectangular areas, a place where no figure exists is searched from the midpoint of the two areas and separated at the detected position.
{Circle around (2)} Similarly, when binarization is performed, a region where no figure is present is obtained and separated at the portion over the widest region.
(3) Interpolate a threshold value between two rectangular areas by various methods.
{Circle around (4)} Two threshold areas are separated in the middle of two rectangular areas.
It binarizes by the method of etc.
[0038]
FIG. 13 shows an example of binarization in threshold value determination. FIG. 2B shows the score image that has been read, and the dotted rectangle is the threshold value determination rectangle. FIG. 9A shows the state of the determined threshold value, and the threshold value determined in each rectangle is used for the sections b and f, and the edges of the images of a and c are used. The same threshold is used for the section b as the section b, the same threshold as the section f is used for the section e, and the threshold value of the two areas is linear for the section d. Interpolate and use.
[0039]
Further, by changing the threshold value, it is possible to deal with partial printing fading. This can be realized by dividing into a plurality of rectangles instead of only one rectangle in the horizontal direction as shown in FIG. 13 (the same threshold value in the horizontal direction).
[0040]
The threshold judgment area is not a rectangle (rectangle), but may be a parallelogram area connecting the start and end positions of the staff, as described above, or consider the amount of staff shift detected by staff recognition. It may be a strip-shaped area.
[0041]
【The invention's effect】
According to the configuration of the present invention described in detail above, it is possible to obtain a binarized image that is optimal for recognition using score information, and to improve the recognition rate. In addition, there is no need to set an optimum threshold value when scanning an image, and work efficiency is improved.
[Brief description of the drawings]
FIG. 1 is an explanatory diagram showing the line width and staff interval of a staff.
FIG. 2 is a block diagram showing the configuration of an embodiment of a score recognition apparatus that operates by reading a computer-readable recording medium that records a score recognition program according to claim 4 of the present invention with an external storage device;
FIG. 3 is a flowchart showing main processing of a CPU.
FIG. 4 is a flowchart showing main processing steps of automatic binarization processing.
FIG. 5 is a flowchart showing a threshold detection procedure by a bisection method.
FIG. 6 is a flowchart showing a procedure for detecting the upper limit of the band when determining which of the T_OK bands is to be used as the threshold when the threshold determination ends with T_OK.
FIG. 7 is a flowchart showing a processing flow of threshold determination.
FIG. 8 is an explanatory diagram showing a state in which a background is separated from a figure depending on whether the score is scanned in the vertical direction and the value is larger or smaller than a threshold value.
FIG. 9 is a graph showing histograms of run lengths for line widths and staff intervals on the staff.
FIG. 10 is an explanatory diagram illustrating a state in which scanning in the vertical direction of an image is performed at certain intervals.
FIG. 11 is an explanatory diagram showing a setting state of a rectangle surrounding a staff set for threshold value determination.
FIG. 12 is an explanatory diagram showing an example of a strip shape set as a threshold determination region.
FIG. 13 is an explanatory diagram showing an example of binarization processing in threshold determination.
[Explanation of symbols]
1 CPU
2 ROM
3 RAM
4 Hard disk devices
5 Flexible disk drive
6 CRT
7 CRT interface circuit
8 Keyboard
9 Keyboard interface circuit
10 Printer
11 Printer interface circuit
12 Scanner
13 Scanner interface circuit
14 MIDI interface circuit
15 bus

Claims (4)

A musical score information reading means, a threshold value setting means, and an image capturing means are executed to read a musical score, automatically recognize a music symbol by computer processing from the read musical score data, and perform performance and / or Or, in a score recognition method for creating data for score display,
A step of reading a score with a gradation image by the score information reading means;
Binarizing the score image read by the threshold setting means with a predetermined binarization threshold ;
Similarly, the threshold setting means obtains the ratio of the staff width to the staff interval of the binarized musical score image, determines the image state from the ratio, and thereby calculates and sets the next binarization threshold The threshold value is sequentially changed to obtain a final threshold value from the threshold band when the ratio of the staff width to the staff interval becomes the target ratio;
A step of binarizing the read musical score image with a threshold value finally calculated and set by the threshold value setting means ;
And a step of recognizing a music symbol from the binarized musical score image by the image capturing means. A musical score information reading means, a threshold value setting means, and an image capturing means are executed to read a musical score, automatically recognize a music symbol by computer processing from the read musical score data, and perform performance and / or Or, in a score recognition method for creating data for score display,
A step of reading a score with a gradation image by the score information reading means;
Binarizing the score image read by the threshold setting means with a predetermined binarization threshold ;
Similarly, the binarized score image is divided into a plurality of blocks by the threshold setting means, and the divided score image is binarized by a predetermined binarization threshold value for each block. The ratio of the staff width to the staff spacing of the image is obtained for each block, the image state is determined from the ratio, and the next binarization threshold is calculated and set for each block, and the threshold is sequentially set. Changing, and obtaining a final threshold value for each block from the threshold band when the ratio of the staff width to the staff interval becomes the target ratio;
Binarizing the read musical score image for each block with a threshold value finally calculated and set for each block by the threshold setting means ;
And a step of recognizing a music symbol from a score image binarized with a different binarization threshold for each block by the image capturing means. A computer-readable recording medium storing a score recognition program for reading a score, automatically recognizing a music symbol by computer processing from the read score data, and creating data for performance and / or score display By executing the program, a score information reading unit, a threshold setting unit, and an image capturing unit are realized on the computer.
A step of reading the score as a gradation image by the score information reading means;
Binarizing the score image read by the threshold value setting means with a predetermined binarization threshold value ;
Similarly, the threshold setting means obtains the ratio of the staff width to the staff interval of the binarized score image, determines the image state from the ratio, and thereby calculates and sets the next binarization threshold The threshold value is sequentially changed to obtain a final threshold value from the threshold band when the ratio of the staff width to the staff interval becomes the target ratio;
A step of binarizing the read musical score image with a threshold value finally calculated and set by the threshold value setting means ;
A computer-readable recording medium on which a score recognition program for executing the step of recognizing a music symbol from the binarized score image by the image capturing means is recorded. A computer-readable recording medium storing a score recognition program for reading a score, automatically recognizing a music symbol by computer processing from the read score data, and creating data for performance and / or score display By executing the program, a score information reading unit, a threshold setting unit, and an image capturing unit are realized on the computer.
A step of reading the score as a gradation image by the score information reading means;
Binarizing the score image read by the threshold value setting means with a predetermined binarization threshold value ;
Similarly, the binarized musical score image is divided into a plurality of blocks by the threshold setting means, and the divided musical score image is binarized by a predetermined binarization threshold for each block. The ratio of the staff width to the staff interval of the image is obtained for each block, the image state is determined from the ratio, and the next binarization threshold is calculated and set for each block, and the threshold is sequentially set. A step of obtaining a final threshold value for each block from a threshold band when a ratio of the staff width to the staff interval becomes a target ratio by changing,
Binarizing the read musical score image for each block with a threshold value finally calculated and set for each block by the threshold setting means ;
A computer-readable recording medium on which a score recognition program for executing a step of recognizing a music symbol from a score image binarized with a different binarization threshold for each block by the image capturing means is recorded. .

Images