JP4803797B2 - Music score recognition apparatus and music score recognition program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To more adequately recognize lyrics which are written in musical scores. <P>SOLUTION: A musical score recognition device includes; a musical score image collection part 12 which collects musical score images indicating musical scores which express musical compositions from an external device 5; a musical score recognition part 13 which recognizes the respective images of a plurality of figures, which form the musical score images, as a plurality of marks; a lyric rectangle extraction part 14 which extracts, from the musical score images, lyric rectangles which surround figures, which are not recognized as the musical score marks forming the musical scores, among the plurality of figures; and a lyric recognition part 16 which recognizes the images of lyric figures, which are surrounded by the lyric rectangles, as characters. <P>COPYRIGHT: (C)2007,JPO&amp;INPIT

Description

The present invention relates to a score recognition device and a score recognition program, and more particularly to a score recognition device and a score recognition program for recognizing a score image as a score.

  2. Description of the Related Art Music score recognition software is known that can capture music with an image scanner and edit, print, and play it on a personal computer. As the score recognition software, “Score Maker” manufactured by Kawai Gakki Seisakusho is exemplified. The score recognition software can also recognize the lyrics described in the score. The score recognition software reduces false recognition by improving the recognition accuracy of individual symbols. An example of a method for improving the recognition accuracy is to provide a character string dictionary and sequentially check the character string against the character string dictionary after character recognition. Lyrics described in the score are expressed mostly in hiragana, especially in Japanese, and there is no character separation, making it difficult to determine the phrase. For this reason, it is difficult to improve the recognition rate of the lyrics using a character string dictionary. Furthermore, when the lyrics are expressed in alphabet, various languages (eg, English, French, German, Italian, Latin) are considered, and a string dictionary corresponding to all of these languages is prepared. Is difficult. There is a demand for a score recognition device that more appropriately recognizes lyrics described in a score.

  In Japanese Patent No. 3454307, when a similarity between two arbitrary melodies is determined, a comparison method applying a human-specific auditory characteristic (the ability to memorize a sound and associate it with a subsequent sound = memory reverberation effect) is used. A melody similarity determination method is disclosed that enables rational and multi-faceted similarity determination that can provide a result in accordance with a sense. The melody similarity determination method is a melody similarity determination method for a song that determines the similarity of the determination melody to the verification melody by comparing the verification melody stored in advance in the melody database with the determination melody. A first step of creating a pitch permutation perception score and a time-division perception score from the original music of the music subject to the determination melody, a pitch permutation comparison is performed by collation based on the pitch permutation perception score, and a predetermined sound A second step of extracting the collation melody exceeding a column coincidence rate as a comparison collation melody, and a pitch time coincidence rate based on a pitch time comparison with respect to the comparison collation melody by collation based on the time-division auditory score; A third step of calculating a pronunciation point matching rate based on the pronunciation point comparison and adding the information to the comparison target melody; the pitch sequence matching rate; and the pitch time matching rate And a fourth step of calculating a similarity rate between the determination melody and the comparison target matching melody based on the pronunciation point matching rate, and adding information to the comparison target matching melody; , Pitch time match rate, pronunciation point match rate, and a certain tendency appearing between the comparison number of sounds classified by type, the tone string match rate of the comparison target melody, the pitch time match rate, A fifth step of performing type matching with the pronunciation point matching rate, and adding information to the comparison target matching melody as a matching pattern as a matching pattern; and for the comparison target matching melody, the tone string matching rate, the pitch time A determination process is performed using the matching rate, the pronunciation point matching rate, the similarity rate, and the similar pattern as determination elements, and the determination result is added as determination evaluation data to the comparison target verification melody, as an evaluation target verification melody And a sixth step of leaving, is characterized by determining the similarity between the determined melody and the verification melody based on the determination evaluation data.

  Japanese Patent No. 3466894 provides a configuration for performing character-type character discrimination processing for music symbols composed of mixed italic characters and standard characters on a musical score, thereby highly recognizing music symbols containing these characters. A music score recognition method is disclosed in which a rate is obtained. The musical score recognition method is a musical score recognition method that reads a musical score image, recognizes the musical symbol, and creates data for performance or musical score display. On the other hand, the label is extracted and recognized by pattern matching, and then the inclination of the estimated regression line of these symbols on the label is detected, and whether or not the font of the symbol is italic due to the difference in the inclination. It is characterized by distinguishing.

Japanese Patent No. 3454307 Japanese Patent No. 3466894

An object of the present invention is to provide a score recognition device and a score recognition program for more appropriately recognizing lyrics described in a score.
Another object of the present invention is to provide a score recognition apparatus and a score recognition program for more appropriately recognizing lyrics described in a score without changing the accuracy of optically reading a printed matter.

  In the following, means for solving the problems will be described using the reference numerals used in the best modes and embodiments for carrying out the invention in parentheses. This reference numeral is added to clarify the correspondence between the description of the claims and the description of the best mode for carrying out the invention / example, and is described in the claims. It should not be used to interpret the technical scope of the invention.

  A score recognition apparatus (1) according to the present invention includes a score image collecting unit (12) that collects a score image representing a score representing a musical piece from an external device (5), and a plurality of figures forming the score image as a plurality of symbols. A score recognition unit (13) that recognizes an image, a lyrics rectangle extraction unit (14) that extracts a lyric rectangle surrounding a figure that is not recognized by a score symbol that forms a score among a plurality of figures, It is preferable to include a lyric recognition unit (16) that recognizes a lyric figure surrounded by a lyric rectangle as a character.

  The score recognition apparatus (1) according to the present invention preferably further includes a character type table (21) for associating a plurality of types with a plurality of character sets. At this time, the lyrics recognizing unit (16) searches for a type corresponding to the character set including the most characters whose lyrics figure is recognized by the score recognizing unit (13) from a plurality of types, and the plurality of character sets. The lyric figure is image-recognized preferentially to characters belonging to the character set corresponding to the type.

  A musical score recognition apparatus (1) according to the present invention includes a lyrics number rectangle extracting unit that extracts a lyrics number rectangle surrounding a plurality of lyrics number figures arranged on a straight line parallel to a staff among the lyrics figures. (15) is preferably further provided. At this time, the lyrics recognizing unit (16) recognizes only the figure surrounded by the lyric number rectangle among the plurality of figures as characters.

  The lyric recognition unit (16), when the first figure surrounded by the first lyric number rectangle in the lyric number rectangle is unrecognizable as a character, the first lyric number in the lyric number rectangle It is preferable that the first graphic is recognized as a character recognized from the second graphic whose horizontal position coincides with the first graphic among the graphics surrounded by the second lyric number rectangle adjacent to the rectangle.

  The lyric recognition unit (16) is a case where the first figure surrounded by the first lyric number rectangle in the lyric number rectangle cannot be recognized as a character, and the first lyric number in the lyric number rectangle When the figure surrounded by the second lyric number rectangle adjacent to the rectangle includes two or more figures whose horizontal positions coincide with the first figure, the first is located at the horizontal position of the boundary between the two or more figures. It is preferable to recognize each of two or more figures obtained by cutting the figure as characters.

  The lyrics number rectangle extraction unit (15) preferably selects an invalid lyrics number rectangle satisfying a predetermined condition from the lyrics number rectangle. At this time, the lyrics recognizing unit (16) does not recognize the figure surrounded by the invalid lyrics number lyrics rectangle among the plurality of figures as characters.

  The condition is that among the figures surrounded by the invalid lyric number rectangle, there are more than a predetermined percentage of figures whose horizontal positions do not correspond to the horizontal positions of the notes in the musical score symbols. Alternatively, the condition is that the invalid lyrics number rectangle is adjacent to a predetermined symbol. Alternatively, the condition is that a plurality of characters recognized from a plurality of figures surrounded by the invalid lyric number rectangle do not include characters belonging to one type at a ratio of a predetermined value or more. Alternatively, the condition is that a character string recognized from a plurality of figures surrounded by an invalid lyrics number rectangle includes a predetermined number of character strings.

  The score recognition program according to the present invention is a computer program executed by a score recognition device (1) that is a computer, and as a part of the computer program, a score image indicating a score representing music is collected from an external device (5). A musical score image collecting unit (12) for performing image recognition, a musical score recognition unit (13) for recognizing a plurality of graphics forming a musical score image as a plurality of symbols, and a graphic not recognized by a musical score symbol forming a musical score among the plurality of graphics. It is preferable to include a lyric rectangle extracting unit (14) for extracting a lyric rectangle from a score image and a lyric recognizing unit (16) for recognizing a lyric figure surrounded by the lyric rectangle among a plurality of figures as characters.

  The score recognition program according to the present invention preferably further includes a character type table (21) for associating a plurality of types with a plurality of character sets. At this time, the lyrics recognizing unit (16) searches for a type corresponding to the character set including the most characters whose lyrics figure is recognized by the score recognizing unit (13) from a plurality of types, and the plurality of character sets. The lyric figure is image-recognized preferentially to characters belonging to the character set corresponding to the type.

  The score recognition program according to the present invention extracts a lyric number rectangle extracting unit (15) that extracts a lyric number rectangle surrounding a plurality of lyric number figures arranged on a straight line parallel to a staff among lyric figures. Is preferably further provided. At this time, the lyrics recognizing unit (16) recognizes only the figure surrounded by the lyric number rectangle among the plurality of figures as characters.

  The lyric recognition unit (16), when the first figure surrounded by the first lyric number rectangle in the lyric number rectangle is unrecognizable as a character, the first lyric number in the lyric number rectangle It is preferable that the first graphic is recognized as a character recognized from the second graphic whose horizontal position coincides with the first graphic among the graphics surrounded by the second lyric number rectangle adjacent to the rectangle.

  The lyric recognition unit (16) is a case where the first figure surrounded by the first lyric number rectangle in the lyric number rectangle cannot be recognized as a character, and the first lyric number in the lyric number rectangle When the figure surrounded by the second lyric number rectangle adjacent to the rectangle includes two or more figures whose horizontal positions coincide with the first figure, the first is located at the horizontal position of the boundary between the two or more figures. It is preferable to recognize each of two or more figures obtained by cutting the figure as characters.

  The lyrics number rectangle extraction unit (15) preferably selects an invalid lyrics number rectangle satisfying a predetermined condition from the lyrics number rectangle. At this time, the lyrics recognizing unit (16) does not recognize the figure surrounded by the invalid lyrics number lyrics rectangle among the plurality of figures as characters.

  The condition is that among the figures surrounded by the invalid lyric number rectangle, there are more than a predetermined percentage of figures whose horizontal positions do not correspond to the horizontal positions of the notes in the musical score symbols. Alternatively, the condition is that the invalid lyrics number rectangle is adjacent to a predetermined symbol. Alternatively, the condition is that there exists a type to which characters of a predetermined ratio or more among a plurality of characters recognized from a plurality of figures surrounded by invalid lyric number rectangles belong. Alternatively, the condition is that a character string recognized from a plurality of figures surrounded by an invalid lyrics number rectangle includes a predetermined number of character strings.

  The score recognition apparatus and score recognition program according to the present invention can more appropriately recognize the lyrics described in the score.

  An embodiment of an electronic musical instrument according to the present invention will be described with reference to the drawings. In the musical score recognition apparatus 1, the musical score recognition apparatus main body 2 includes a scanner 5 and an output device 7 as shown in FIG. 1. The scanner 5 captures a printed matter on which a score representing music is printed, generates a score image of the score, and outputs the score image to the score recognition apparatus body 2. Examples of the output device 7 include a display and a printer. The display includes a display surface, and displays a screen generated by the score recognition apparatus main body 2 on the display surface. The printer prints the screen generated by the score recognition apparatus body 2 on paper.

  The score recognition device main body 2 is a computer, and includes a CPU, a storage device, and an input device (not shown). The CPU executes a computer program installed in the score recognition apparatus main body 2 and controls the storage device, the input device, the scanner 5 and the output device 7. The storage device records the computer program and temporarily records information generated by the CPU. The input device outputs information input by operation by the user to the CPU.

  The score recognition apparatus main body 2 includes a character type database 11, a score image collection unit 12, a score recognition unit 13, a lyrics rectangle extraction unit 14, a lyrics number rectangle extraction unit 15, a lyrics recognition unit 16, and a score output unit 17 as computer programs. And are installed. The character type database 11 records a character type table indicating the types of characters expressing lyrics in a storage device so as to be searchable by other computer programs.

  The musical score image collecting unit 12 collects the musical score images generated by the scanner 5 from the scanner 5. The score recognition unit 13 generates a score based on the score image collected by the score image collection unit 12. That is, the score recognition unit 13 extracts a plurality of figures from the score image collected by the score image collection unit 12, and assigns a figure ID to each of the plurality of figures. The score recognition unit 13 further compares each of the plurality of figures with each of the plurality of symbols, and recognizes the figure as the most similar symbol of the plurality of symbols. The symbol includes a musical score symbol representing a musical sound to be played and characters representing a lyrics. Examples of the musical score symbols include staffs, notes, rests, tempo symbols, and slurs. Examples of the characters include hiragana, katakana, kanji, and alphabet. The score recognition unit 13 further recognizes the plurality of graphics as a character string when the plurality of graphics recognized by the characters are arranged adjacent to each other.

  The lyrics rectangle extraction unit 14 extracts a lyrics rectangle from the score image collected by the score image collection unit 12 based on the symbols recognized by the score recognition unit 13. The lyric number rectangle extracting unit 15 extracts the lyric number rectangle from the lyric rectangle extracted by the lyric rectangle extracting unit 14 based on the symbols recognized by the score recognizing unit 13. The lyrics recognizing unit 16 recognizes the figure surrounded by the lyrics number rectangle extracted by the lyrics number rectangle extracting unit 15 as characters.

  The score output unit 17 outputs the score generated by the score recognition unit 13 and the lyrics recognition unit 16 to the output device 7. That is, a screen showing the score is created and displayed on a display, and the screen showing the score is printed on paper using a printer.

  FIG. 2 shows an example of a part of the score image collected by the score image collecting unit 12. The score image shows a score expressing a chorus, and a staff figure 31 and lyrics figures 33-1 to 33-2 are described. The staff figure 31 is formed of a plurality of figures, and the plurality of figures indicate a staff, a note, a rest, and the like. Each of the lyric figures 33-1 to 33-2 is formed of a plurality of character figures. The plurality of character figures indicate characters indicating lyrics. The plurality of character figures are arranged on a straight line parallel to the staff above or below the staff. The plurality of character figures are described in a plurality of lines when the chorus has a plurality of numbers, that is, they are arranged on a plurality of parallel lines parallel to the staff. At this time, the score recognition unit 13 recognizes the figure of the staff figure 31 as a staff, a note, a rest, and the like, and recognizes the character figures of the lyrics figures 33-1 to 33-2 as characters.

  FIG. 2 further shows the lyrics rectangle extracted by the lyrics rectangle extraction unit 14. The lyric rectangles 34-1 to 34-2 are rectangles formed by sides parallel to the staff and sides perpendicular to the staff, and the score recognition unit 13 recognizes the staff, notes, rests, and the like. The figure recognized by the score recognition unit 13 is enclosed without surrounding the figure to be displayed. In addition, the lyrics rectangles 34-1 to 34-2 are substantially the same length as the staff, the horizontal position of the right end substantially coincides with the horizontal position of the right end of the staff, and the horizontal position of the left end is the left end of the staff. It almost coincides with the horizontal position of. That is, the lyrics rectangle extraction unit 14 does not enclose a figure recognized by the score recognition unit 13 as a staff, a note, a rest, or the like, encloses a figure recognized by the score recognition unit 13 as a character and its length. Are formed to be substantially the same, the lyrics rectangles 34-1 to 34-2 are formed.

  FIG. 3 shows another example of a part of the score image collected by the score image collection unit 12. The score image shows a score expressing a chorus, and a grand staff figure 41 is described. The grand staff figure 41 is composed of a treble staff chart figure 42-1 and a bass staff chart figure 42-2. The treble part staff figure 42-1 and the bass part staff figure 42-2 indicate a staff, a note, a rest, and the like, respectively. The musical score image further includes lyrics graphics 43-1 to 43-2. Each of the lyric figures 43-1 to 43-2 is formed of a plurality of character figures. The plurality of character figures indicate characters indicating lyrics. The plurality of character figures are arranged on a straight line parallel to the staff above or below the staff. At this time, the score recognition unit 13 recognizes the treble staff chart figure 42-1 and the bass staff chart figure 42-2 as a staff, a note, a rest, and the like, and the character figures of the lyrics figures 43-1 to 43-2 are recognized. Recognize characters.

  FIG. 3 further shows the lyrics rectangle extracted by the lyrics rectangle extraction unit 14. The lyrics rectangles 44-1 to 44-2 are rectangles formed by a side parallel to the staff and a side perpendicular to the staff, and the score recognition unit 13 recognizes the staff, notes, rests, and the like. The lyrics rectangle extraction unit 14 encloses the figure recognized by the score recognition unit 13 without enclosing the figure to be displayed. The lyrics rectangles 44-1 to 44-2 are substantially the same in length as the staff, the horizontal position of the right end substantially coincides with the horizontal position of the right end of the staff, and the horizontal position of the left end is the left end of the staff. It almost coincides with the horizontal position of. That is, the lyrics rectangle extraction unit 14 does not enclose a figure recognized by the score recognition unit 13 as a staff, a note, a rest, or the like, encloses a figure recognized by the score recognition unit 13 as a character and its length. Are formed so that the lyric rectangles 44-1 to 44-2 are substantially the same.

  FIG. 4 shows still another example of a part of the musical score image collected by the musical score image collecting unit 12. Lyrics described in a score expressing a choral song are generally written in one line when a common lyrics is sung by multiple numbers of lyrics, and different lyrics are sung by multiple numbers of lyrics. Sometimes written in multiple lines. Further, when the lyrics are described in a plurality of lines, the song number is described at the head. The lyrics further include one line of lyrics that are commonly sung in the plurality of numbers and multiple lines of lyrics that are sung differently in the plurality of numbers in one paragraph. Parentheses are written at the boundary between the lyrics and the multi-line lyrics. Each of the plurality of characters representing the lyrics is written immediately below the note written on the staff, and the horizontal position substantially coincides with the horizontal position of the note.

  For this reason, the score image includes a staff figure 51, note figures 58-1 to 58-8, and a lyrics figure 52. A staff figure 51 shows a staff. The note graphics 58-1 to 58-8 respectively indicate a plurality of notes described in the staff. Each of the lyric figures 52 includes a plurality of character figures 57-1 to 57-17. The plurality of character figures indicate characters indicating lyrics. The character graphics 57-1 to 57-17 are arranged on a straight line parallel to the staff above or below the staff. When the chorus has a plurality of numbers, the character figures 57-1 to 57-17 are described in a plurality of lines, that is, arranged on a plurality of parallel lines parallel to the staff. The lyric graphic 52 further includes a parenthesis graphic 54 indicating a parenthesis.

  Each of the character figures 57-1 to 57-17 has a horizontal position substantially corresponding to any one of the musical note figures 58-1 to 58-8. That is, the horizontal position of the character graphic 57-1 substantially matches the horizontal position of the note graphic 58-1. The horizontal position of the character graphic 57-2 substantially matches the horizontal position of the note graphic 58-2. The horizontal position of the character graphic 57-3 substantially matches the horizontal position of the note graphic 58-3. The horizontal position of the character graphic 57-4 substantially matches the horizontal position of the note graphic 58-4. The horizontal position of the character graphic 57-6 substantially matches the horizontal position of the note graphic 58-5. The horizontal position of the character graphic 57-7 substantially coincides with the horizontal position of the note graphic 58-6. The horizontal position of the character graphic 57-8 substantially matches the horizontal position of the note graphic 58-7. The horizontal position of the character graphic 57-9 substantially coincides with the horizontal position of the note graphic 58-8. The horizontal position of the character graphic 57-11 substantially coincides with the horizontal position of the note graphic 58-5. The horizontal position of the character graphic 57-12 substantially matches the horizontal position of the note graphic 58-6. The horizontal position of the character graphic 57-13 substantially coincides with the horizontal position of the note graphic 58-7. The horizontal position of the character graphic 57-14 substantially coincides with the horizontal position of the note graphic 58-8.

  At this time, the score recognition unit 13 recognizes the figure of the staff figure 51 as a staff, a note, a rest, and the like, and recognizes the character figures 57-1 to 57-17 as characters. The score recognition unit 13 further recognizes the note graphics 58-1 to 58-8 as musical notes and recognizes the parenthesis graphic 54 as parentheses.

  FIG. 4 further shows the lyrics rectangle extracted by the lyrics rectangle extraction unit 14. The lyrics rectangle 53 is a rectangle formed by a side parallel to the staff and a side perpendicular to the staff, and a figure recognized by the score recognition unit 13 as a staff, a note, a rest, a slur, or the like. Without being enclosed, the lyrics rectangle extraction unit 14 encloses a figure recognized by the score recognition unit 13 as a character. Furthermore, the lyrics rectangle 53 is substantially the same length as the staff, the lateral position of the right end substantially coincides with the lateral position of the right end of the staff, and the lateral position of the left end is substantially equal to the lateral position of the left end of the staff. I'm doing it. That is, the lyrics rectangle extraction unit 14 does not enclose a figure recognized by the score recognition unit 13 as a staff, a note, a rest, or the like, encloses a figure recognized by the score recognition unit 13 as a character and its length. The lyric rectangles 53 are formed so that are substantially the same.

  The lyric rectangle 53 is divided into a plurality of divided lyric rectangles 55-1 to 55-2 with the parenthesis graphic 54 as a boundary. FIG. 4 further shows the lyrics number rectangle extracted by the lyrics number rectangle extraction unit 15. The lyrics number rectangles 56-1 to 56-3 are each surrounded by one of the divided lyrics rectangles 55-1 to 55-2, and are composed of a side parallel to the staff and a side perpendicular to the staff. A rectangle to be formed. The lyrics number rectangles 56-1 to 56-3 are further substantially the same in length as the divided lyrics rectangles 55-1 to 55-2, respectively, and the horizontal position at the right end is the divided lyrics rectangles 55-1 to 55-. 2 substantially coincides with the horizontal position of the right end, and the horizontal position of the left end substantially coincides with the horizontal position of the left end of the divided lyrics rectangles 55-1 to 55-2. The lyric number rectangles 56-1 to 56-3 further enclose figures recognized by the score recognition unit 13 as characters. The figures respectively surrounded by the lyric number rectangles 56-1 to 56-3 are not arranged in a plurality of lines, but are arranged on one straight line parallel to the staff.

  That is, the lyric number rectangle extracting unit 15 divides the lyric rectangle 53 extracted by the lyric rectangle extracting unit 14 into the divided lyric rectangles 55-1 to 55-2 by the parenthesis figure 54 recognized by the score recognition unit 13 in parentheses. To do. The lyric number rectangle extracting unit 15 is further surrounded by any of the divided lyric rectangles 55-1 to 55-2, and the lyric number rectangle 56-1 so as to surround the character figures arranged on one straight line. ~ 56-3. The lyrics number rectangle extraction unit 15 further deletes the lyrics number rectangle when the lyrics number rectangle is recognized as a predetermined symbol and arranged near the figure. An example of the predetermined symbol is a tempo symbol. The lyric number rectangle extracting unit 15 further deletes the lyric number rectangle when the characters recognized from the character figure surrounded by the lyric number rectangle include a predetermined number or more. As the specific character, a circled alphabet “C” indicating “Copyright”, “Edit”, “Song”, “Lyrics”, “Poetry” copyright is exemplified. As the predetermined number, one is exemplified. The lyric number rectangle extracting unit 15 further deletes the lyric number rectangle when the character string recognized from the character figure surrounded by the lyric number rectangle includes a predetermined number of character strings. As the specific character string, “arranged” and “copyright” are exemplified. As the predetermined number, one is exemplified. The lyrics number rectangle extraction unit 15 further refers to the character type table 21 and determines the ratio of characters belonging to one type having a plurality of characters recognized from the character figure surrounded by the lyrics number rectangle to a predetermined value or more. When it is not included, delete the lyrics number rectangle. The lyric number rectangle extracting unit 15 further deletes the lyric number rectangle when a plurality of characters recognized from the character figure surrounded by the lyric number rectangle include kanji at a ratio equal to or higher than a predetermined value. To do. At this time, the lyric number rectangle extracting unit 15 counts the number of voids in a graphic that is not recognized by characters among the graphics surrounded by the lyric number rectangle, and counts a graphic whose number is a predetermined value or more as a kanji. . The lyric number rectangle extraction unit 15 further deletes the lyric number rectangle when a plurality of character figures surrounded by the lyric number rectangle do not coincide with the note shape.

  FIG. 5 shows a character type table recorded by the character type database 11. The character type table 21 associates a type 22, a character set 23, a range 24, and a correction method 25 with each other. Type 22 indicates the type of character expressing the lyrics. Examples of the type include hiragana, katakana, alphabets, alphabets to which symbols are added, ligatures, and italic alphabets. Examples of symbols added to the alphabet include umlauts, axants, and cedillas. The character set 23 shows a plurality of characters belonging to the type indicated by the type 22.

  The range 24 indicates a range in which the character recognized by the character set 23 can be included in the characters recognized from the graphic surrounded by the plurality of lyrics number rectangles extracted by the lyrics number rectangle extraction unit 15. . The correction method 25 is a range in which a range 24 indicates a ratio in which characters indicated by the character set 23 are included in characters recognized from a figure surrounded by a plurality of lyrics number rectangles extracted by the lyrics number rectangle extraction unit 15. Included in the plurality of lyrics number rectangles extracted by the lyrics number rectangle extraction unit 15, other than the characters recognized from the figure recognized other than the characters indicated by the character set 23. The correction method to change to the character of is shown.

  For example, the correction method is such that when a figure surrounded by a plurality of lyric number rectangles extracted by the lyric number rectangle extracting unit 15 is recognized by the alphabet as 80% or more, characters other than the alphabet are converted to alphabets. To change to. The correction method is to extract a lyric number rectangle when a figure surrounded by a plurality of lyric number rectangles extracted by the lyric number rectangle extracting unit 15 is recognized as hiragana (katakana) by 90% or more. This indicates that the character recognized from the figure surrounded by the plurality of lyrics number rectangles extracted by the unit 15 is not changed. The correction method is a case in which a figure surrounded by a plurality of lyric number rectangles extracted by the lyric number rectangle extracting unit 15 is recognized by Hiragana (Katakana) as 60% or less, and one letter is included in the alphabet. When hiragana (Katakana) is sandwiched, the one-character Hiragana (Katakana) is changed to alphabet, and when Hiragana (Katakana) is sandwiched with one to two alphabets, the one-letter alphabet is changed to alphabet To change to. In the correction method, a figure surrounded by a plurality of lyrics number rectangles extracted by the lyrics number rectangle extraction unit 15 is often recognized as Hiragana, and a figure adjacent to the figure recognized as Katakana is Hiragana. The figure shows that the figure recognized as the Katakana is changed to Hiragana. In the correction method, a figure surrounded by a plurality of lyrics number rectangles extracted by the lyrics number rectangle extraction unit 15 is often recognized as a katakana character, and a figure adjacent to the figure recognized as a hiragana character is a katakana character. The figure shows that the figure recognized by the hiragana character is changed to a katakana character.

  At this time, the lyrics recognizing unit 16 refers to the character type table 21 and searches the character set 23 for the character set to which the most recognized characters are recognized by the musical score recognition unit 13. Search for the type corresponding to the specified character set. The lyrics recognizing unit 16 further refers to the character type table 21 and includes characters indicated by the character set 23 in the characters recognized from the figure surrounded by the lyrics number rectangle extracted by the lyrics number rectangle extracting unit 15. When the ratio to be included is included in the range indicated by the range 24, a character other than the character indicated by the character set 23 is changed to another character using a correction method corresponding to the character set 23 and the range 24. To preferentially recognize characters belonging to the character set.

  For example, the lyrics recognizing unit 16 re-recognizes the figure recognized as a character other than the alphabet when the figure surrounded by the lyrics number rectangle is 80% or more. The lyric recognition unit 16 re-recognizes characters recognized from the figure surrounded by the lyrics number rectangle when the ratio of the figure surrounded by the lyrics number rectangle is 90% or more. do not do. The lyric recognition unit 16 recognizes a single hiragana character (Katakana) when one character hiragana (Katakana) is sandwiched between alphabets when the percentage recognized by the hiragana character (Katakana) is 60% or less. The recognized figure is re-recognized as an alphabet, and when a one- or two-letter alphabet is sandwiched between hiragana (katakana), the figure recognized as the one-letter alphabet is re-recognized as an alphabet. Next, the lyrics recognizing unit 16 has a figure recognized by the katakana character when the characters recognized by the score recognizing unit 13 have many hiragana characters and a graphic adjacent to the graphic recognized by the katakana character is recognized by the hiragana character. Is recognized again as Hiragana. For example, the lyrics recognizing unit 16 changes the figure recognized as “F” in Katakana to “He” in Hiragana, changes the figure recognized as “Hi” in Katakana to “To” in Hiragana, Change the figure recognized by "Li" to Hiragana "Ri".

  The lyrics recognizing unit 16 measures the sizes of a plurality of figures surrounded by the single lyrics number rectangle. The lyrics recognizing unit 16 recognizes small characters used for hiragana or katakana in hiragana or katakana based on the size, or discriminates between uppercase and lowercase letters of the alphabet. The lyric recognizing unit 16 also has one lyric number when there is a figure that is not recognized by the score recognizing unit 13 among a plurality of figures surrounded by one of the lyric number rectangles 56-1 to 56-3. Measure the size of multiple figures surrounded by several rectangles. When the size of the unrecognized graphic is more than twice the size of the recognized graphic, the lyrics recognition unit 16 is adjacent to the other lyrics number rectangle in the vertical direction. In addition, among the figures surrounded by other lyrics number rectangles, the unrecognized figure is divided at the boundary between a plurality of figures that coincide with the horizontal position of the unrecognized figure, and the divided figure is recognized as a character. When the size of the unrecognized graphic is more than twice the size of the recognized graphic, the lyric recognition unit 16 is not adjacent to the other lyric number rectangle in the vertical direction. Then, the unrecognized figure is cut at the contact position (cutting position) with the smallest contact portion, and the cut figure is recognized as a character.

  When the lyrics recognizing unit 16 recognizes a plurality of figures surrounded by a plurality of lyrics number rectangles arranged vertically as characters, the figure arranged at the left end of the lyrics number rectangle is preferentially recognized as a number. For example, when the figure at the left end of the lyrics number rectangle is recognized as the alphabet “l” by the score recognition unit 13, the lyrics recognition unit 16 changes the figure to the number “1”.

  The lyric recognition unit 16 recognizes, as a character, a plurality of figures surrounded by a plurality of lyric number rectangles arranged vertically, and when one figure surrounded by one of the plurality of lyric number rectangles is not recognized as a character. Then, a plurality of character strings respectively recognized from the figures surrounded by the plurality of lyrics number rectangles are compared. The lyric recognition unit 16 converts the unrecognized figure into a character that is recognized from the same figure in the horizontal position surrounded by other lyric number rectangles when the plurality of character strings match a predetermined ratio or more. recognize. At this time, the lyrics recognizing unit 16 recognizes the recognition when the unrecognized figure is arranged at the left end of the lyrics number rectangle and the figure at the left end of the other lyrics number rectangle is recognized as a number. Unrecognized figures are not recognized.

  When recognizing a figure as a hiragana or katakana character, the lyric recognition unit 16 measures the complexity of a part corresponding to the position of the character where the dakuten or semi-dakuten is attached. Such measurement of complexity is well known, and can be measured based on, for example, the presence frequency of the contour of a graphic described in a certain area. When the complexity is equal to or higher than a predetermined value, the lyrics recognizing unit 16 determines the presence / absence of a gap in the part. The lyric recognizing unit 16 adds a cloud point to the character when there is no space, and a semi-cloud point to the character when there is the space.

  FIG. 6 shows the operation of the score recognition apparatus 1. The user images the printed matter on which the score is printed using the scanner 5. The score recognition apparatus 1 collects score images of the score from the scanner 5 (step S1). The score recognition apparatus 1 recognizes the score image as a score (step S2). That is, the score recognition apparatus 1 extracts a plurality of figures from the score image, and assigns a figure ID to each of the figures. The score recognition apparatus 1 further compares each of the plurality of figures with each of the plurality of symbols, and recognizes the figure as the most similar symbol among the plurality of symbols. The symbol includes a musical score symbol representing a musical sound to be played and characters representing a lyrics. Examples of the musical score symbols include staffs, notes, rests, tempo symbols, and slurs. Examples of the characters include hiragana, katakana, kanji, and alphabet. The musical score recognition apparatus 1 further recognizes a plurality of graphics as a character string when a plurality of graphics recognized as characters are arranged adjacent to each other.

  The score recognition device 1 extracts a lyric rectangle from the score image based on the recognized symbol (step S3). That is, the score recognition device 1 does not enclose a figure recognized as a staff, a note, a rest, etc., surrounds a figure recognized as a character, and sets a lyric rectangle so that the length is substantially the same as the staff. Form.

  The musical score recognition apparatus 1 divides the lyrics rectangle into a plurality of divided lyrics rectangles by a figure recognized by parentheses. The score recognizing device 1 further forms a lyrics number rectangle so as to surround a character figure surrounded by one of the divided lyrics rectangles and arranged on one straight line (step S4). At this time, the score recognizing device 1 deletes the lyric number rectangle when the lyric number rectangle is recognized as a predetermined symbol and arranged near the figure. An example of the predetermined symbol is a tempo symbol. The musical score recognition apparatus 1 further deletes the lyrics number rectangle when the characters recognized from the character figure surrounded by the lyrics number rectangle include a predetermined number of characters or more. As the specific character, a circled alphabet “C” indicating “Copyright”, “Edit”, “Song”, “Lyrics”, “Poetry” copyright is exemplified. The musical score recognition apparatus 1 further deletes the lyrics number rectangle when the character string recognized from the character figure surrounded by the lyrics number rectangle includes a predetermined number of character strings. The musical score recognition apparatus 1 further refers to the character type table 21 and includes a plurality of characters recognized from the character figure surrounded by the lyric number rectangle at a ratio equal to or higher than a predetermined value. When not, delete the lyrics number rectangle. The musical score recognition apparatus 1 further deletes the lyric number rectangle when a plurality of characters recognized from the character figure surrounded by the lyric number rectangle include kanji at a ratio equal to or higher than a predetermined value. At this time, the score recognizing device 1 counts the number of voids in a figure that is not recognized by characters among the figures surrounded by the lyric number rectangle, and counts figures whose number is a predetermined value or more as kanji. The musical score recognition apparatus 1 further deletes the lyric number rectangle when a plurality of character figures surrounded by the lyric number rectangle have a graphic pattern whose horizontal position does not match the note graphic at a predetermined ratio or more.

  The musical score recognition apparatus 1 recognizes the figure surrounded by the lyric number rectangle as a character (step S5). That is, the score recognition device 1 refers to the character type table 21 and searches the character set 23 for the character set to which the most recognized characters belong. The musical score recognition apparatus 1 further refers to the character type table 21 and within the range indicated by the range 24 the ratio of the characters indicated by the character set 23 to the characters recognized from the graphic surrounded by the lyric number rectangle. When included, the character other than the character indicated by the character set 23 is changed to another character by using a correction method corresponding to the character set 23 and the range 24 to give priority to the characters belonging to the character set. To recognize. For example, the musical score recognition apparatus 1 changes a figure recognized as a katakana to a hiragana when there are many hiragana characters in the recognized characters and a figure adjacent to the figure recognized as a katakana is recognized as a hiragana. And recognize. For example, the score recognizing apparatus 1 recognizes a figure recognized as “F” in Katakana as “He” in Hiragana, recognizes a figure recognized as “Hi” in Katakana as “To” in Hiragana, The figure recognized as “Li” is recognized as “Ri” in Hiragana.

  The musical score recognition apparatus 1 measures the sizes of a plurality of figures surrounded by the single lyric number rectangle. The musical score recognition apparatus 1 recognizes small characters used for hiragana or katakana in hiragana or katakana based on the size, or discriminates between uppercase and lowercase letters of the alphabet. When the size of the unrecognized figure is more than twice the size of the recognized figure, the score recognition apparatus 1 has one lyric number rectangle vertically adjacent to another lyric number rectangle. In addition, among the figures surrounded by other lyrics number rectangles, the unrecognized figure is divided at the boundary between a plurality of figures that coincide with the horizontal position of the unrecognized figure, and the divided figure is recognized as a character. When the size of the unrecognized figure is more than twice the size of the recognized figure, the score recognition apparatus 1 has one lyric number rectangle not vertically adjacent to another lyric number rectangle Then, the unrecognized figure is cut at the contact position (cutting position) with the smallest contact portion, and the cut figure is recognized as a character.

  When recognizing a plurality of figures surrounded by a plurality of lyric number rectangles arranged vertically as a character, the score recognizing apparatus 1 recognizes a figure arranged at the left end of the lyric number rectangle preferentially as a number. For example, when the figure at the left end of the lyric number rectangle is recognized as the alphabet “l”, the score recognition apparatus 1 changes the figure to the number “1”.

  The musical score recognition apparatus 1 recognizes, as a character, a plurality of figures surrounded by a plurality of lyrics number rectangles lined up and down, and one figure surrounded by one of the plurality of lyrics number rectangles is not recognized as a character. Then, a plurality of character strings respectively recognized from the figures surrounded by the plurality of lyrics number rectangles are compared. When the plurality of character strings match a predetermined ratio or more, the score recognizing device 1 surrounds the unrecognized figure with other lyric number rectangles and is recognized as a character recognized from the same figure in the horizontal position. recognize. At this time, the score recognition apparatus 1 recognizes the recognition when the unrecognized figure is arranged at the left end of the lyrics number rectangle and the figure at the left end of the other lyrics number rectangle is recognized as a number. Unrecognized figures are not recognized.

  When recognizing a figure as a hiragana or katakana character, the score recognizing apparatus 1 measures the complexity of a part corresponding to the position where the dakuten or semi-dakuten of the figure is attached. The score recognition device 1 determines whether or not there is a gap in the part when the complexity is a predetermined value or more. The score recognizing device 1 adds a cloud point to the character when there is no space, and adds a semi-cloudy point to the character when the space exists.

  According to such an operation, the score recognizing apparatus 1 can more appropriately recognize the lyrics described in the score without changing the accuracy of the process of optically reading the printed matter. According to such an operation, the score recognizing device 1 can recognize the lyrics more appropriately without using the character string dictionary.

FIG. 1 is a block diagram showing an embodiment of a score recognition apparatus according to the present invention. FIG. 2 is a diagram illustrating an example of a lyrics rectangle. FIG. 3 is a diagram illustrating another example of the lyrics rectangle. FIG. 4 is a diagram illustrating an example of a lyrics number rectangle. FIG. 5 shows a character type table. FIG. 6 is a flowchart showing the operation of the score recognition apparatus according to the present invention.

Explanation of symbols

1: Music score recognition device 5: Scanner 7: Output device 11: Character type database 12: Music score image collection unit 13: Music score recognition unit 14: Lyric rectangle extraction unit 15: Lyric number rectangle extraction unit 16: Lyric recognition unit 17: Music score output Part 21: Character type table 22: Type 23: Character set 24: Range 23: Correction method 31: Staff figure 33-1 to 33-2: Lyric figure 34-1 to 34-2: Lyric rectangle 41: Grand staff figure 42- 1: Treble staff chart figure 42-2: Bass staff chart figure 43-1 to 43-2: Lyric figure 44-1 to 44-2: Lyric rectangle 51: Staff figure 52: Lyric figure 53: Lyric rectangle 54: Bracket figure 55-1 to 55-2: divided lyrics rectangles 56-1 to 56-3: lyrics number rectangles 57-1 to 57-17: character figures 58-1 to 58-8: note figures

Claims (20)

A score image collection unit that collects a score image indicating a score representing music from an external device;
A score recognition unit for recognizing a plurality of figures forming the score image as a plurality of symbols,
A lyric rectangle extracting unit that extracts a lyric rectangle surrounding a figure that is not recognized by a musical score symbol that forms the musical score of the plurality of figures from the musical score image;
A lyric recognition unit that recognizes a lyric figure surrounded by the lyric rectangle of the plurality of figures as a character ;
A character type table associating multiple types with multiple character sets;
The lyric recognition unit searches for a type corresponding to a character set including the most characters from which the lyric graphic is recognized by the score recognition unit among the plurality of types, and corresponds to the type of the plurality of character sets. Recognize images of the lyric figures preferentially to characters belonging to the character set
Music notation recognition device. In claim 1,
A lyric number rectangle extracting unit for extracting a lyric number rectangle surrounding a plurality of lyric number figures arranged on a straight line of the lyric figure from the inside of the lyric rectangle;
The lyric recognition unit is a score recognition device that recognizes only a figure surrounded by the lyric number rectangle among the plurality of figures as characters . A score image collection unit that collects a score image indicating a score representing music from an external device ;
A score recognition unit for recognizing a plurality of figures forming the score image as a plurality of symbols,
A lyric rectangle extracting unit that extracts a lyric rectangle surrounding a figure that is not recognized by a musical score symbol that forms the musical score of the plurality of figures from the musical score image;
A lyric recognition unit that recognizes a lyric figure surrounded by the lyric rectangle of the plurality of figures as a character;
The lyrics and ingredients Bei the lyrics numbering number rectangle extracting unit for extracting lyrics numbered number rectangular from the interior of the lyrics rectangle surrounding a plurality of lyrics numbering the number of figures aligned on a straight line of the figure,
The lyric recognition unit recognizes only a figure surrounded by the lyric number rectangle among the plurality of figures as characters. In claim 2 or claim 3,
The lyric recognition unit is configured to recognize the first graphic number in the lyric number rectangle when the first figure surrounded by the first lyric number rectangle in the lyric number rectangle cannot be recognized as a character. A musical score recognition apparatus for recognizing a first figure as a character recognized from a second figure whose horizontal position is coincident with the first figure among figures surrounded by a second lyrics number rectangle adjacent to the number rectangle. In claim 2 or claim 3,
The lyric recognition unit is a case where the first figure surrounded by the first lyric number rectangle in the lyric number rectangle cannot be recognized as a character, and the first lyric number in the lyric number rectangle When the figure surrounded by the second lyric number rectangle adjacent to the number rectangle includes two or more figures whose horizontal positions coincide with the first figure, the second position is located at the horizontal position of the boundary between the two or more figures. A musical score recognition device that recognizes two or more figures obtained by cutting one figure as characters. In claim 2 or claim 3,
The lyrics number rectangle extraction unit selects an invalid lyrics number rectangle that satisfies a predetermined condition from the lyrics number rectangle,
The lyric recognition unit does not recognize a figure surrounded by the invalid lyric number lyrics rectangle among the plurality of figures as a character. In claim 6,
The condition is that a figure whose horizontal position does not correspond to a horizontal position of a note among the musical score symbols among the figures surrounded by the invalid lyrics number rectangle is a score recognition apparatus. In claim 6,
The condition is that the invalid lyrics number rectangle is adjacent to a predetermined symbol. In claim 6,
The condition is that a plurality of characters recognized from a plurality of figures surrounded by the invalid lyric number rectangle do not include characters belonging to one type at a ratio of a predetermined value or more. In claim 6,
The condition is that a character string recognized from a plurality of figures surrounded by the invalid lyric number rectangle includes a predetermined number of character strings in a predetermined number or more. A score image collection unit that collects a score image indicating a score representing music from an external device;
A score recognition unit for recognizing a plurality of figures forming the score image as a plurality of symbols,
A lyric rectangle extracting unit that extracts a lyric rectangle surrounding a figure that is not recognized by a musical score symbol that forms the musical score of the plurality of figures from the musical score image;
A lyric recognition unit that recognizes a lyric figure surrounded by the lyric rectangle of the plurality of figures as a character ;
A score recognition program for causing a computer to implement a character type table that associates a plurality of types with a plurality of character sets.
The lyric recognition unit searches for a type corresponding to a character set including the most characters from which the lyric graphic is recognized by the score recognition unit among the plurality of types, and corresponds to the type of the plurality of character sets. Recognize images of the lyric figures preferentially to characters belonging to the character set
Music notation recognition program. In claim 11,
A score recognition program for causing the computer to further implement a lyric number rectangle extracting unit that extracts a lyric number rectangle surrounding a plurality of lyric number figures arranged on a straight line among the lyric figures from the inside of the lyric rectangle. Yes,
The lyric recognition unit is a score recognition program that recognizes only a figure surrounded by the lyric number rectangle among the plurality of figures as characters . A score image collection unit that collects a score image indicating a score representing music from an external device ;
A score recognition unit for recognizing a plurality of figures forming the score image as a plurality of symbols,
A lyric rectangle extracting unit that extracts a lyric rectangle surrounding a figure that is not recognized by a musical score symbol that forms the musical score of the plurality of figures from the musical score image;
A lyric recognition unit that recognizes a lyric figure surrounded by the lyric rectangle of the plurality of figures as a character;
A score recognition program for causing a computer to realize a lyrics number rectangle extraction unit that extracts a lyrics number rectangle surrounding a plurality of lyrics number figures arranged on a straight line of the lyrics figure from the inside of the lyrics rectangle . ,
The lyric recognition unit recognizes only a figure surrounded by the lyric number rectangle among the plurality of figures as characters. In any one of claims 12 or claim 13,
The lyric recognition unit is configured to recognize the first graphic number in the lyric number rectangle when the first figure surrounded by the first lyric number rectangle in the lyric number rectangle cannot be recognized as a character. A musical score recognition program for recognizing a first figure as a character recognized from a second figure whose horizontal position matches the first figure among figures surrounded by a second lyric number rectangle adjacent to the number rectangle. In any one of claims 12 or claim 13,
The lyric recognition unit is a case where the first figure surrounded by the first lyric number rectangle in the lyric number rectangle cannot be recognized as a character, and the first lyric number in the lyric number rectangle When the figure surrounded by the second lyric number rectangle adjacent to the number rectangle includes two or more figures whose horizontal positions coincide with the first figure, the second position is located at the horizontal position of the boundary between the two or more figures. A musical score recognition program that recognizes two or more figures cut from one figure as characters. In any one of claims 12 or claim 13,
The lyrics number rectangle extraction unit selects an invalid lyrics number rectangle that satisfies a predetermined condition from the lyrics number rectangle,
The lyric recognition unit does not recognize a figure surrounded by the invalid lyric number lyrics rectangle among the plurality of figures as characters. In claim 16,
The condition is that a figure whose horizontal position does not correspond to a horizontal position of a note among the musical score symbols is greater than or equal to a predetermined ratio among the figures surrounded by the invalid lyrics number rectangle. In claim 16,
The condition is that the invalid lyric number rectangle is adjacent to a predetermined symbol. In claim 16,
The condition is that a plurality of characters recognized from a plurality of figures surrounded by the invalid lyric number rectangle do not include characters belonging to one type at a ratio of a predetermined value or more. In claim 16,
The condition is that a character string recognized from a plurality of figures surrounded by the invalid lyric number rectangle includes a predetermined character string in a predetermined number or more.

Images