JP5162873B2 - Pen-type musical score input device, music playback system, and music playback program - Google Patents

Description

The present invention relates to a pen-type musical score input device , a music playback system, and a music playback program .

  Patent Document 1 describes an apparatus that can input musical notes, rests, and other musical score symbols by writing simplified predetermined handwritten figures on a touch tablet.

JP 2003-302967 A (Claims, Abstract)

  By the way, in the technique shown in patent document 1, since a user needs to memorize | store a handwritten figure, there exists a problem that it takes time until it gets used to input operation.

  In addition, since the touch tablet requires a certain area, the size of the device itself increases to some extent. For this reason, there is also a problem that operability and portability are low when a musical score is input by placing such a device beside a musical instrument or when a musical score is input outdoors.

  Furthermore, when printing an input musical score on a printing paper, for example, it is necessary to perform a printing procedure by connecting to a printer or the like, so that there is a problem that the operation becomes complicated.

The present invention has been made based on the above circumstances, and an object of the present invention is to provide a pen-type score symbol input device , a music playback system, and a music playback program that have high operability and portability. .

In order to achieve the above-described object, the pen-type musical score input apparatus of the present invention includes reading means for reading position information of a musical score symbol written on a staff score of a print medium on which position information indicating a position on the medium is printed. , A writing pressure detection means for detecting a writing pressure applied to the pen tip, a recognition means for recognizing a musical score symbol by referring to position information read by the reading means, and a musical score symbol obtained by the recognition means can be reproduced. And pressure information detected by the pressure detection means based on the position information where the hit is detected when detecting that the predetermined area of the print medium is hit by the pen tip , A tempo determining means for detecting a tapping interval and determining a tempo based on the detected interval, and a reproducing means for reproducing music data based on the tempo.

A music playback system according to another aspect of the invention is a music playback system including a pen-type device and an external device capable of communicating with the pen-type device, and the pen-type device is printed with position information indicating a position on a medium. A reading means for reading the position information of the musical score symbol written on the staff of the printed medium, a writing pressure detecting means for detecting the writing pressure applied to the pen tip, and a brush detected by the position information and the writing pressure detecting means. Communication means for transmitting pressure information to an external device, and the external device receives position information and writing pressure information of a musical notation symbol written on a print medium from a pen-type device, and position information. , A recognition means for recognizing a musical score symbol written on the print medium, a conversion means for converting the musical score symbol obtained by the recognition means into reproducible music data, and a predetermined area of the print medium by the pen tip the beating has been that When detecting, detecting an interval between beating with reference to the pen pressure information pressure detecting means brush based on the position information detected its beating, and tempo determining means for determining a tempo based on the detected distance, the music data Reproducing means for reproducing based on the tempo.

According to another aspect of the present invention, there is provided a music reproduction program that receives, from a pen-type device, position information on a staff score written on a print medium and writing pressure information from a pen-type device, and refers to the position information for printing. A recognition step for recognizing a musical notation symbol written on the medium, a conversion step for converting the musical notation symbol obtained by the recognition step into reproducible music data, and a predetermined area of the print medium being struck by the pen-type device By detecting the tapping interval by referring to the detection information of the pen pressure detecting means for detecting the pen pressure applied to the pen tip of the pen type device based on the position information where the tapping is detected, This is a program for executing a tempo determination step for determining a tempo based on a detected interval and a reproduction step for reproducing music data based on the tempo.

According to the present invention, it is possible to provide a pen-type musical score input device , a music playback system, and a music playback program that have high operability and portability.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing an external appearance of a pen-type musical score input device 10 according to an embodiment of the present invention. As shown in FIG. 1, the pen-type musical score input device 10 includes a housing 11 having a substantially cylindrical shape, for example. A part of the bottom of the housing 11 is provided with a nib 12 made of a writing member such as graphite, a ballpoint nib, or a fountain pen, which is a pencil core. Note that one end of the nib 12 is in contact with a writing pressure sensor 21 described later inside the housing 11. An infrared LED (Light Emitting Diode) 13 as a part of a reading unit for reading an optical pattern printed on a corresponding paper 30 to be described later and a reading unit are also provided near the pen tip 12 at the bottom of the housing 11. A C-MOS (Complementary Metal-Oxide Semiconductor) camera 14 is provided as a unit.

  A playback button 15 that is operated when playing back music data generated from an input musical score symbol is provided below the side surface of the housing 11 (bottom of the drawing). In addition, an LCD (Liquid Crystal Display) 16 as a display unit for confirming an input musical score symbol is provided at the upper part (upper part of the drawing) of the side surface part. A speaker 17 for outputting music data as sound is provided on the upper portion of the LCD 16. At the top of the housing 11, for example, an antenna 18 for communicating with a personal computer or other devices is provided. The antenna 18 is connected to a communication unit 25 described later.

  Note that the user can write arbitrary information on the corresponding paper 30 by holding the pen 16 in contact with the corresponding paper 30 with the thumb positioned at the intermediate portion of the LCD 16 and the playback button 15 of the housing 11. it can. At this time, the locus on the corresponding paper 30 by the pen tip 12 is based on an optical pattern (pattern printed on the corresponding paper 30) read by the infrared LED 13 and the C-MOS camera 14 as will be described in detail later. Acquired, the corresponding musical score symbol is recognized and displayed on the LCD 16. The user can determine from the information displayed on the LCD 16 whether the recognition result is correct. Further, by operating the play button 15 with a thumb or the like, the music data generated from the input musical score symbol can be output from the speaker 17 as a sound and confirmed.

  FIG. 2 is a block diagram showing an example of the electrical configuration of the pen-type musical score input device 10 shown in FIG. As shown in this figure, the pen-type musical score input device 10 includes an infrared LED 13, a C-MOS camera 14, a playback button 15, an LCD 16, a speaker 17, a CPU (Central Processing Unit) 20, a writing pressure sensor 21, and an image processing unit. 22, a memory 23, a timer 24, a communication unit 25, and a sound source unit 26 are main components.

  Here, the infrared LED 13 irradiates the corresponding paper 30 as a print medium with infrared rays based on the control of the CPU 20. FIG. 3 is a diagram for explaining the details of the optical pattern printed on the corresponding paper 30. As shown in this figure, the corresponding paper 30 is printed with dots that absorb infrared rays at intervals of 0.3 mm, as shown in the enlarged center, together with the staff. These dots are printed with a predetermined amount of deviation from the center of the grid in any of the upper, lower, left and right directions, as further enlarged and shown on the right side of the drawing. Since 6 × 6 = 36 dots are gathered to form one optical pattern, and each dot has four combinations, one optical pattern of 1.8 mm × 1.8 mm size has four 36 There are combinations of rides. One optical pattern includes position information indicating the position on the corresponding paper 30, and the position of the pen tip 12 on the corresponding paper 30 can be detected by detecting the position information.

  FIG. 4 is a diagram showing a partial cross-sectional structure of the corresponding paper 30. As shown in this figure, the dot pattern shown in FIG. 3 is printed on the paper 30a serving as the base of the corresponding paper 30 by the dot ink 30b that absorbs infrared rays. On top of this, a pattern such as a staff is printed with normal ink 30c that transmits infrared rays. Of the infrared light emitted from the infrared LED 13, the portion irradiated to the dot ink 30 b is absorbed and does not enter the C-MOS camera 14. Further, the portion irradiated with the normal ink 30 c is transmitted and reflected by the paper 30 a and is incident on the C-MOS camera 14. Thereby, infrared rays having an intensity distribution according to the optical pattern are incident on the C-MOS camera 14.

  The image processing unit 22 extracts image information (position information) included in the optical pattern by inputting image data corresponding to the optical pattern of infrared rays incident on the C-MOS camera 14 and performing image processing. , Supplied to the CPU 20.

  The writing pressure sensor 21 as writing pressure detecting means is constituted by, for example, a pressure sensitive sensor whose resistance value changes according to the applied pressure, and the pen tip 12 is brought into contact with the corresponding paper 30 to write predetermined contents. Detects the pen pressure at the time of reading.

  The memory 23 as a storage unit is configured by a flash memory, for example, and stores a basic program executed by the CPU 20. The memory 23 temporarily stores a program or data that is being calculated when the CPU 20 executes the calculation process. Further, the memory 23 adds the time information supplied from the timer 24 to the position information supplied from the image processing unit 22 and stores it as read data.

  The timer 24 generates time information and supplies it to the CPU 20. The CPU 20 stores the time information supplied from the timer 24 as read data in the memory 23 in association with the position information supplied from the image processing unit 22. In addition, you may make it produce | generate the elapsed time after the power supply of the pen-type musical score symbol input device 10 was turned on instead of time.

  The communication unit 25 serving as a transfer unit exchanges information with another device (not shown) via the antenna 18 based on a standard such as Bluetooth. Instead of wireless communication, for example, a cable such as a USB (Universal Serial Bus) may be used for wired connection with other devices to exchange information.

  The sound source unit 26 has a function of reproducing MIDI (Musical Instruments Digital Interface) data as music data. After the MIDI data supplied from the CPU 20 is converted into a corresponding audio signal, it is amplified by an amplifier (not shown). And supplied to the speaker 17.

  FIG. 5 is realized by the cooperation of the CPU 20 as hardware and the software when a program (not shown) as software stored in the memory 23 shown in FIG. 2 is executed. An example of a processing module is shown. Note that the program stored in the memory 23 may be shipped in a preinstalled state or installed after the user purchases it. Alternatively, only a part of the product may be shipped in a preinstalled state, and another part may be installed later. When installing later, for example, it may be installed from a storage medium such as a CD-ROM (Compact Disk Read Only Memory), or may be downloaded and installed from a network such as the Internet.

  As shown in FIG. 5, the functional blocks realized when the program is executed include a trajectory reproduction module 50, an analysis module 51, a symbol recognition module 52, a musical notation symbol DB (Data Base) 53, a core module 54, There are a display processing module 55, a MIDI data conversion module 56, a MIDI data conversion DB 57, a MIDI data storage module 58, and an output processing module 59.

  Here, the trajectory reproduction module 50 acquires the read data stored in the memory 23, reproduces the trajectory drawn on the corresponding paper 30 by the pen tip 12 of the pen-type musical score input device 10, and the symbol recognition module. 52.

  The analysis module 51 is a module for analyzing the position on the corresponding paper 30 on which the musical score symbol analyzed by the symbol recognition module 52 is drawn.

  The symbol recognition module 52 as a recognition means is written on the corresponding sheet 30 by comparing the data including the trajectory reproduced by the trajectory reproduction module 50 with the score symbol pattern stored in the score symbol DB 53. The type of musical score symbol is recognized, and the recognition result is supplied to the core module 54.

  DB 53 for musical notation includes notes (quarter notes, etc.), rests (quarter rests, etc.), change symbols (sharp, etc.), clef symbols (g clef, etc.), time signatures (quarter quarter, etc.) In addition, pattern information and the like regarding strength symbols (forte, etc.), velocity symbols (adagio, etc.), and other musical score symbols are stored and used when the symbol recognition module 52 recognizes the symbols.

  The core module 54 serving as a tempo determination unit is a module serving as a core, and has a function of controlling each module. Further, it also has a function of specifying a score symbol included in the read data based on the analysis result of the symbol recognition module 52, the score symbol DB 53, the analysis module 51, and the like.

  The display processing module 55 performs a drawing process based on the information supplied from the core module 54 and displays the obtained image on the LCD 17.

  The MIDI data conversion module 56 as conversion means converts the musical score symbol supplied from the core module 54 into corresponding MIDI data with reference to the MIDI data conversion DB 57 and stores it in the MIDI data storage module 58.

  The MIDI data conversion DB 57 stores information for converting musical score symbols to MIDI data.

  The MIDI data storage module 58 stores the MIDI data obtained by the conversion by the MIDI data conversion module 56. As will be described later, the MIDI data storage module 58 also stores position information indicating the position on the corresponding sheet 30 on which a musical score symbol corresponding to each piece of information constituting the MIDI data is described.

  The output processing module 59 as a reproducing means acquires the MIDI data stored in the MIDI data storage module 58, converts it into an audio signal, and drives the speaker 17.

  Next, the operation of the embodiment of the present invention will be described.

  FIG. 6 is an example of processing executed when writing a predetermined musical score symbol on the corresponding paper 30 by the pen-type musical score input device 10 according to the embodiment of the present invention. When this process is started, the following steps are executed.

  Step S10: The CPU 20 refers to the output of the writing pressure sensor 21 and determines whether or not an input has been made by the pen-type musical score input device 10. Specifically, when the user grips the pen-type musical score input device 10 and brings the pen tip 12 into contact with the corresponding paper 30 shown in FIG. Therefore, the CPU 20 determines that the writing pressure sensor 21 has detected, and proceeds to step S11.

  When the user holds the pen-type musical score input device 10 and writes some information on the corresponding paper 30 shown in FIG. 7, a trace is engraved on the corresponding paper 30 by the pen tip 12. At this time, infrared rays are emitted from the infrared LED 13 and applied to the vicinity of the tip of the nib 12 of the corresponding paper 30. As shown in FIG. 7, the staff 30 and other information are printed on the corresponding paper 30. However, since this is normal ink as shown in FIG. 4, it transmits infrared rays. On the other hand, the optical pattern printed with dot ink reflects infrared rays. Infrared light reflected by the optical pattern is incident on the C-MOS camera 14, and the light intensity pattern is converted into a corresponding image signal.

  The image signal output from the C-MOS camera 14 is supplied to the image processing unit 22, and an optical pattern as a set of 6 × 6 dots is converted into position information (information indicating a position on the paper surface) to the CPU 20. Is output. The CPU 20 adds the time information supplied from the timer 24 to the position information and stores it in the memory 23 as read data. Since the operation as described above is repeated at a predetermined frequency during a period when the pen pressure is detected by the pen pressure sensor 21, information as shown in FIG. In this example, as position information, values indicating the x and y coordinates of the corresponding sheet 30 and time information indicating hours, minutes, and seconds are stored in association with each other. A series of information that is continuous in position or time is stored as a first trajectory and a second trajectory. Each of the first trajectory and the second trajectory usually corresponds to one musical score symbol. In addition, as a method of dividing the plurality of input positional information for each trajectory, for example, when the acquired time is opened for 2 seconds or more with reference to time information, the trajectories are different trajectories. Can be determined. Further, with reference to the position information, when the distance is, for example, 100 or more on the coordinates, it can be determined that the paths are different. It is also possible to make a determination based on both position and time.

  Step S11: The analysis module 51 acquires the above-described read data stored in the memory 23. Specifically, the analysis module 51 acquires the position information shown in the first line shown in FIG. 8, for example.

  Step S12: The analysis module 51 refers to the position information, determines whether or not the position input by the pen-type musical score input device 10 is the tempo input unit 31 shown in the upper right of FIG. If it is determined that the unit 31 is selected, the process proceeds to step S13. Otherwise, the process proceeds to step S14. Here, the tempo input unit 31 is a part for inputting the tempo of the music by hitting the part at a desired tempo with the pen tip 12 of the pen-type musical score input device 10. As a method for determining whether or not the tempo input unit 31 has been struck, a range of coordinates of the tempo input unit 31 (for example, coordinates of two opposing vertices) is stored in the musical score symbol DB 53. When the position information is included in the range of the coordinates, it can be determined that it is in the tempo input unit 31.

  Step S13: The analysis module 51 acquires the read data (data input from the tempo input unit 31) stored in the memory 23, and calculates the tempo from the time information. Specifically, when the tempo input unit 31 is struck, the position information is read at the timing when the writing pressure sensor 21 is pressed (the timing when the pen tip 12 contacts the corresponding paper 30), and the memory 23 together with the time information. Stored in The analysis module 51 acquires time information of position information belonging to the region (rectangular region) of the tempo input unit 31, calculates a difference between adjacent time information, and obtains an average value. For example, when the average value of the differences between the time information is 1 second, the tempo of the music is determined to be a tempo at which a quarter note is beaten 60 times per minute. When the calculation is completed, the process proceeds to step S23, and the recognition result is displayed on the LCD 16 as described later.

  Step S14: If it is determined in step S12 that the input is not from the tempo input unit 31, the trajectory reproduction module 50 acquires the read data from the memory 23 and reproduces the trajectory. Specifically, the trajectory is reproduced by acquiring a series of position information shown in FIG. 8 and connecting the positions specified by the respective position information with line segments. For example, in the case of the first trajectory in FIG. 8, the position corresponding to the position information (125, 256) on the first line and the position corresponding to the position information (126, 258) on the second line are mutually. Connected by line segments, the trajectory is reproduced. Similarly, the positions corresponding to all the position information are connected by line segments, and the trajectory is reproduced. In the example of FIG. 8, only position information and time information are stored, but writing pressure information may be added thereto, and the thickness of the line segment of the locus may be changed according to the writing pressure. That is, when the writing pressure is high, the line segment is thickened, and when the writing pressure is low, the line segment is thinned. The trajectory reproduced in this way is supplied to the symbol recognition module 52.

  Step S15: The symbol recognition module 52 compares the trajectory supplied from the trajectory reproduction module 50 with the information indicating the pattern of each musical score symbol stored in the musical score symbol DB 53 (by pattern matching processing). The musical score symbol indicated by the locus is specified. For example, the musical score symbol DB 53 stores a plurality of templates representing musical score symbols, and the symbol recognition module 52 calculates the correlation between the trajectory and each template and corresponds to the template having the highest correlation. It is determined that the symbol is a corresponding symbol. The core module 54 is supplied with information indicating the identified symbol.

  Step S16: The core module 54 determines whether or not the recognized symbol is a musical note. As a result, if it is determined that it is a note, the process proceeds to step S18, and otherwise, the process proceeds to step S17. For example, in the corresponding paper 30 shown in FIG. 7, if a quarter note is input, the process proceeds to step S18, and if any other symbol is input, the process proceeds to step S17.

  Step S17: The core module 54 supplies the information of the recognition result in step S15 to the MIDI data conversion module 56. The MIDI data conversion module 56 refers to the MIDI data conversion DB 57 and converts the musical score symbol specified by the recognition into corresponding MIDI data.

  Specifically, for example, when “Piano” is described in the program change unit 32 shown in an enlarged view at the bottom of FIG. 7, the MIDI data conversion module 56 displays the tone of the channel corresponding to the staff. MIDI data for changing to a piano is generated. As an example, “C0H 00H” (channel 0 is changed to a tone of 00 (piano)) is generated. In addition, when a treble clef is described in the sound input unit 33, the pitch of the note is determined as a treble that is 5 degrees higher than the central sound of the second line of the staff. When “4/4” is described in the time signature input unit 34, it is determined that the time signature of a note input thereafter becomes a quarter time signature.

  Step S18: If the identified symbol is a note, the core module 54 sends a command to the analysis module 51 to detect the position of the center of gravity of the note head. Specifically, as shown in FIG. 9, the analysis module 51 detects the note head 72 of the note 70 constituted by the note stem 71, the note head 72, the note tail 73, and the like, and detects the detected note head 72. The position of the center of gravity 80 (the position on the corresponding paper 30) is obtained. As a method of obtaining the center of gravity, for example, the average coordinate of the x coordinate and y coordinate of the dots (trajectory) constituting the note head 72 may be obtained. The position information of the center of gravity of the note head 72 obtained in this way is supplied to the core module 54.

  Step S19: The core module 54 determines the pitch of the note based on the position of the center of gravity 80 of the note head 72 of the note 70 obtained in step S18 and the symbol input to the note input unit 33. When a change symbol (sharp, flat, etc.) is written near the note 70, the pitch is determined in consideration of this.

  Step S20: The core module 54 detects the velocity of the note. For example, as shown in an enlarged view at the bottom of FIG. 7, when a symbol indicating strength (mf (mesoforte) in this example) is written in the vicinity of the note, this is detected.

  Step S21: The core module 54 supplies the detected information to the MIDI data conversion module 56 to convert the notes into corresponding MIDI information. Specifically, the MIDI data conversion module 56 generates data as shown in FIG. The generated data is stored in the MIDI data storage module 58.

  In the example of FIG. 10, L1 indicates a measure (Measure), L2 indicates a beat (Beat), L3 indicates a tick (Type), Type indicates an event type, Value1 indicates a pitch, Value 2 indicates the length of the sound, and Value 3 indicates the intensity of the sound. For example, the data on the first line corresponds to the first note (F of a quarter note) shown in FIG. Since this data is the sound of the first beat of the first measure, L1 is 0001, L2 is 01, and L3 is 000. Since the event generates a sound, Type is Note. Furthermore, since the sound of F3 is output by ff for the quarter note, Value1 to Value3 are F3 [65], [00: 430], and ff [100], respectively.

  Step S22: The core module 54 generates note position data based on the position information of the center of gravity of the note detected in step S18. Specifically, the core module 54 generates information as shown in FIG. The generated information is stored in the MIDI data storage module 58.

  In the example of FIG. 10B, information indicating the position of the center of gravity of the note head of each note (information indicating the position of the corresponding paper 30) is stored. For example, in the case of the information on the first row, it is indicated that the center of gravity position of the note head of the first note shown in FIG. 7 is (32, 50). Note that the MIDI data shown in FIG. 10A and the position information shown in FIG. 10B have a one-to-one relationship.

  Step S23: The core module 54 supplies the recognition result information to the display processing module 55 and causes the LCD 17 to display the recognition result. For example, when the first note shown in FIG. 7 is written on the corresponding paper 30 and recognized, information such as “quarter note fa” is displayed. At this time, a sound corresponding to the recognized note may be output from the speaker 17 through the output processing module 59.

  Step S24: The core module 54 determines whether or not the input of the musical score symbol has been completed. If it is determined that the input has not been completed, the core module 54 returns to step S10 and repeats the same processing. Exit. For example, if the playback button 15 is operated or a power switch (not shown) is turned off, the process is terminated.

  Through the above processing, when the user holds the pen-type musical score input device 10 and enters musical score symbols on the corresponding paper 30, the locus of the pen tip 12 is recorded on the corresponding paper 30 and the pen tip 12 The optical pattern existing at the position where the lens has passed is read by the C-MOS camera 14. From the read optical pattern, the trajectory is reproduced by the trajectory reproduction module 50 and the type of the symbol is recognized by the symbol recognition module 53. Then, the data is converted into corresponding MIDI data by the MIDI data conversion module 56 and stored in the MIDI data storage module 58.

  Note that the music data (MIDI data) input as described above can be transferred to another device (for example, a personal computer) via the communication unit 25. Then, the other device that has received the transfer can display the MIDI data on the display device or reproduce it by the audio system.

  Next, operations for reproducing the music data input as described above and for reproducing the sound of the newly written note will be described.

  FIG. 11 is a flowchart for explaining an example of the operation when the music data stored in the MIDI data storage module 58 is reproduced and when the sound of a newly written note is reproduced. When this flowchart is started, the following steps are executed.

  Step S30: The core module 54 determines whether or not the play button 15 has been operated. If it is determined that the play button 15 has been operated, the process proceeds to step S31, and otherwise the same processing is repeated.

  Step S31: The core module 54 determines whether or not the detection by the writing pressure sensor 21 has been performed. If there is a detection, the process proceeds to step S32. Otherwise, the process returns to step S30 and is the same as described above. Repeat the process. Specifically, when the output value of the writing pressure sensor 21 is equal to or higher than a predetermined value, the core module 54 determines that the writing pressure has been detected and proceeds to step S32.

  Step S32: The analysis module 51 acquires the above-described read data stored in the memory 23.

  Step S33: The analysis module 51 determines whether or not the inside of the reproduction bar 36 shown in FIG. 7 has been traced by the pen tip 12. If within the reproduction bar 36, the analysis module 51 proceeds to step S34. Proceed to step S38. More specifically, the score symbol DB 53 stores coordinate information of two vertices facing each reproduction bar 36 printed on the corresponding paper 30 shown in FIG. By comparing the coordinate information with the position information of the read data, if the read data is within the range of any reproduction bar 36, the process proceeds to step S34; otherwise, the process proceeds to step S38.

  Even when the playback bar 36 is not included in the range, the playback bar 36 may be determined to have been traced if it is included within the predetermined range from the playback bar 36. . This is because it is difficult to accurately trace the reproduction bar 36, and thus it is allowed to protrude from the reproduction bar 36 to some extent.

  Step S34: The core module 54 obtains position information included in the read data supplied from the analysis module 51.

  Step S35: The core module 54 acquires MIDI data corresponding to the traced position from the MIDI data storage module 58 via the MIDI data conversion module 56. More specifically, the core module 54 acquires position information indicating the current position of the pen tip 12 from the analysis module 51 and extracts the value of the y coordinate. And the range of the y-coordinate value of the note corresponding to the extracted y-coordinate value is specified. For example, in the example of FIG. 7, the playback bar 36 at the top is for playing the notes written on the staff on the top, so the y-coordinate of the notes to be played back A range of values is specified. Next, the core module 54 extracts the value of the x coordinate from the position information indicating the current position of the pen tip 12. Then, the core module 54 searches for position information belonging to the y-coordinate value range and having the closest x-coordinate distance from the position information on the center of gravity of the note shown in FIG. For example, if the acquired position information of the pen tip 12 is (40, 60), the y-coordinate value 60 is referenced with reference to the y-coordinate value 60, and the range of the y-coordinate value of the target note is 48 to 58. Is identified. Next, the position information of the center of gravity having a value closest to 40 which is the value of the x coordinate is searched. In the example of FIG. 10B, the position information (42, 50) of the second row is specified because it is the closest, and the MIDI data of the second row of FIG. 10A corresponding to this is specified as the MIDI data storage module 58. Obtained from.

  Step S36: The core module 54 supplies the acquired MIDI data to the output processing module 59 via the MIDI data conversion module 56. Thereby, the MIDI data acquired in step S35 is reproduced and outputted as sound from the speaker 17.

  Step S37: The core module 54 determines whether or not to end the processing. If the processing is not ended, the core module 54 returns to step S34 to repeat the same processing, and otherwise ends the processing. Specifically, the process ends when the input of the read data is not performed for a predetermined time (for example, 3 seconds) or longer, otherwise the process ends, and the process returns to step S34 and the same process is repeated. .

  By repeating the processing of step S34 to step S37, MIDI data corresponding to a note existing on the staff corresponding to the reproduction bar 36 and closest to the x coordinate of the nib 12 is sequentially reproduced. Therefore, for example, when the user traces the predetermined reproduction bar 36 with the pen tip 12, the music is played according to the position of the pen tip 12.

  In the above processing, the music is played according to the movement of the pen tip 12, but it is difficult for the user to move the pen-type musical score input device 10 at a constant speed. When a part of the playback bar 36 is designated, all the notes existing on the corresponding staff are played at a predetermined tempo, or traced at a speed faster than the predetermined tempo. In such a case, reproduction may be performed at a predetermined tempo, and reproduction may be performed at a speed slower than the predetermined tempo.

  Step S38: The locus reproduction module 50 acquires the read data from the memory 23 and reproduces the locus. Specifically, the trajectory is reproduced by acquiring a series of position information shown in FIG. 8 and connecting the positions specified by the respective position information with line segments.

  Step S39: The symbol recognition module 52 compares the trajectory supplied from the trajectory reproduction module 50 with information indicating the pattern of each musical score symbol stored in the musical score symbol DB 53 (by pattern matching processing). The musical score symbol (in this case, the type of note) indicated by the locus is specified. For example, the musical score symbol DB 53 stores a plurality of templates representing musical score symbols, and the symbol recognition module 52 calculates the correlation between the trajectory and each template and corresponds to the template having the highest correlation. It is determined that the symbol is a corresponding symbol. The identified symbol is supplied to the core module 54.

  Step S40: The core module 54 sends a command to the analysis module 51 to detect the center of gravity position of the note head. Specifically, the analysis module 51 detects the note head 72 of the note 70 and obtains the position of the center of gravity 80 of the detected note head 72 (position on the corresponding paper 30). In addition, as a method of calculating | requiring a gravity center, as mentioned above, what is necessary is just to obtain | require the coordinate of the average value of the x coordinate of the dot (trajectory) which comprises the note head 72, and y coordinate. The position information of the center of gravity of the note head 72 obtained in this way is supplied to the core module 54.

  Step S41: The core module 54 determines the pitch of the note based on the position of the center of gravity 80 of the note head 72 of the note 70 obtained in step S18 and the symbol input to the note input unit 33. When a change symbol (sharp, flat, etc.) is written near the note 70, the pitch is determined in consideration of this.

  Step S42: The core module 54 detects the velocity of the note. For example, as shown in an enlarged view at the bottom of FIG. 7, when a symbol indicating strength (mf (mesoforte) in this example) is written in the vicinity of the note, this is detected.

  Step S43: The core module 54 supplies the detected information to the MIDI data conversion module 56 to convert the notes into corresponding MIDI information.

  Step S44: The core module 54 supplies the acquired MIDI data to the output processing module 59 via the MIDI data conversion module 56. As a result, MIDI data corresponding to the musical notes written on the corresponding paper 30 is reproduced and output as sound from the speaker 17. Note that the processing from step S38 to step S44 may be repeated until an instruction to end is given.

  As described above, according to the embodiment of the present invention, by writing a musical score symbol on the corresponding paper 30 by the pen-type musical score input device 10, the trajectory of the pen tip 12 is displayed on the corresponding paper 30. While drawing, the music data corresponding to the written symbol can be stored in the memory 23. For this reason, since it is not necessary to memorize a new input symbol other than the musical score symbol, the convenience of the user can be improved.

  Further, the information written on the corresponding paper 30 by the pen tip 12 can be stored in the same manner as the information written on the normal paper. Therefore, the trouble of connecting the apparatus to a printer and newly printing on paper can be saved.

  Further, according to the embodiment of the present invention, the recognition result of the musical score symbol written on the corresponding paper 30 is displayed on the LCD 16 for confirmation, so that it is possible to determine whether or not the recognition result is correct. Become. Furthermore, since the recognition result is reproduced as sound, it is possible to determine whether the recognition result is correct or not by sound.

  In addition, according to the embodiment of the present invention, the tempo can be input by tapping the tempo input unit 31, so that more sensual input can be performed as compared with the case of inputting numbers. it can. User operability can be improved.

  Further, according to the embodiment of the present invention, the musical instrument data type can be specified by writing information indicating the musical instrument type in the program change unit 32. Thereby, user-friendly input can be realized.

  Further, according to the embodiment of the present invention, the corresponding music data is reproduced by tracing the reproduction bar 36 with the pen tip 12. For example, a part of the inputted music is reproduced and confirmed. can do. Also, by adjusting the moving speed of the pen tip 12, the music can be played back slowly or quickly, so that the music can be played back at a speed according to the user's purpose.

  Further, according to the embodiment of the present invention, the input music data can be transferred to another device by the communication unit 25, so that the music data can also be used by another device.

  The above-described embodiments are preferred examples of the present invention, but the present invention is not limited to these, and various modifications and changes can be made without departing from the scope of the present invention. is there.

  For example, in the above embodiment, all the functional blocks shown in FIG. 2 are built in the housing 11, but for example, a part of them is built in the housing 11, and other parts are in other devices. It may be built in. As an example, the pen tip 12, the infrared LED 13, the C-MOS camera 14, and the writing pressure sensor 21 are built in the housing 11, and the others are built in other housings. May be connected wirelessly or by wire. According to such a mounting method, since the weight of the pen-type musical score input device 10 can be reduced, the operability can be improved and the power consumption can be reduced. The usage time of 10 batteries can be extended.

  Moreover, although the case where the inputted musical score symbol is deleted or changed is not described in the above embodiment, the inputted musical symbol can be deleted or changed. As a specific method, for example, a button for performing deletion or change is provided on the housing 11, or a command unit for performing deletion or change is provided on a part of the corresponding paper 30. Then, when these buttons are operated or the target musical score symbol is designated after the command portion is designated by the pen tip 12, the core module 54 is positioned at the position shown in FIG. The target musical score symbol is specified based on the information. In the case of deletion, the MIDI data corresponding to the musical score symbol is deleted. On the other hand, in the case of correction, it waits for a new musical score symbol to be substituted for the musical score symbol, and when it is entered, the corresponding music data is rewritten. Then, the user erases the old symbol written on the corresponding paper 30 with an eraser or the like, or erases the old symbol by entering, for example, a diagonal line. According to such a method, it is possible to delete or change a written symbol.

  It should be noted that an eraser is provided in the vicinity of the pen tip 12 at the bottom of the pen-type musical score input device 10, and the eraser is also provided with a writing pressure sensor, and if the writing pressure sensor is detected, the musical score symbol is erased. May be determined, and the corresponding music data may be deleted based on the position information detected by the C-MOS camera 14 or the like.

  In the above embodiment, the case where there is only one corresponding sheet 30 has been described. However, for example, a plurality of corresponding sheets 30 may be bound to form a notebook. In such a case, for example, page information as well as position information may be stored in the optical pattern so that a written page can be specified. Alternatively, a large coordinate system capable of covering all pages may be prepared, and a part of the coordinate system may be assigned to each page. For example, if “fine”, which is a musical score symbol indicating the end of a song, is entered, it is determined that the song is at the end, and otherwise it is determined that the song continues across pages. To do. According to such a method, it is possible to input a long musical piece exceeding one page.

  In the above embodiment, the input musical score symbol is converted into MIDI data. However, it may be converted into music data other than this.

  In the above embodiment, when the playback bar 36 is traced after the playback button 15 is operated, the corresponding music data is played back. For example, when the playback bar 36 is traced, The corresponding music data may be played back regardless of whether or not the playback button 15 is operated. Alternatively, when the play button 15 is operated, all input music pieces may be played from the beginning.

  Further, in the above embodiment, the playback bar 36 is provided, and the music is played when the playback bar 36 is traced. For example, the playback bar 36 is not provided, and the playback bar 36 is provided. The corresponding music data may be reproduced when the is traced.

  In the above embodiment, when the recognition result is displayed on the LCD 16, it is displayed as character information. However, it may be displayed as image information. For example, a staff may be displayed on the LCD 16 and a musical score symbol indicating a recognition result may be displayed on the staff in an overlapping manner. When the input music data is reproduced, a plurality of musical score symbols may be displayed on the LCD 16 and the musical score symbol being reproduced may be highlighted to clearly indicate the reproduction position.

  The present invention can be applied to, for example, a portable musical score input device.

It is a figure which shows the external appearance of the pen-type score symbol input device which concerns on embodiment of this invention. It is a block diagram which shows the electrical structural example of the pen-type score symbol input device which concerns on embodiment of this invention. It is a figure explaining the optical pattern printed on the corresponding paper. It is a figure explaining the reading method of the section shape and optical pattern of corresponding paper. It is a figure which shows the functional block implement | achieved when the program is performed in embodiment shown in FIG. 3 is a flowchart for explaining a flow of processing executed when writing is performed on a corresponding sheet in the embodiment shown in FIG. 2. It is a figure for demonstrating the detail of corresponding | compatible paper. It is a figure which shows an example of reading data. It is a figure explaining the flow of the process which determines the gravity center of a note. It is a figure explaining MIDI data and position information. FIG. 3 is a flowchart for explaining a flow of processing executed when music data is reproduced or a newly entered musical score symbol is reproduced in the embodiment shown in FIG. 2.

Explanation of symbols

10 Pen-type musical symbol input device 12 Pen tip 13 Infrared LED (part of reading means)
14 C-MOS camera (part of reading means)
16 LCD (display means)
21 Writing pressure sensor (writing pressure detection means)
23 Memory (storage means)
25 Communication section (transfer means)
30 Compatible paper (printing media)
52 Symbol recognition module (recognition means)
54 Core module (tempo determination means)
56 MIDI data conversion module (conversion means)
59 Output processing module (reproducing means)

Claims (3)

Reading means for reading position information of a musical score symbol written on a staff score of a print medium on which position information indicating a position on the medium is printed;
A pen pressure detecting means for detecting a pen pressure applied to the pen tip;
Recognizing means for recognizing the musical score symbol with reference to the position information read by the reading means;
Conversion means for converting the musical score symbol obtained by the recognition means into reproducible music data;
When it is detected that a predetermined area of the print medium has been struck by the pen tip, referring to the writing pressure information detected by the writing pressure detection means based on the position information where the hit is detected, Tempo determination means for detecting an interval and determining a tempo based on the detected interval;
Playback means for playing back the music data based on the tempo;
A pen-type musical score input device. In a music playback system comprising a pen-type device and an external device capable of communicating with the pen-type device,
The pen-type device is
Reading means for reading position information of a musical score symbol written on a staff score of a print medium on which position information indicating a position on the medium is printed;
A pen pressure detecting means for detecting a pen pressure applied to the pen tip;
Communication means for transmitting the position information and the writing pressure information detected by the writing pressure detection means to the external device;
The external device is
Communication means for receiving the position information and the writing pressure information of a musical score symbol written on a print medium from the pen-type device;
Recognizing means for recognizing a musical score symbol written on the print medium with reference to the position information;
Conversion means for converting the musical score symbol obtained by the recognition means into reproducible music data;
When it is detected that a predetermined area of the print medium has been struck by the pen tip, the tapping interval is detected by referring to the pen pressure information of the pen pressure detecting means based on the position information at which the tapping has been detected. Tempo determination means for determining a tempo based on the detected interval;
Playback means for playing back the music data based on the tempo;
A music playback system comprising: On the computer,
Receiving from the pen-type device position information on the staff score written on the print medium and writing pressure information;
Recognizing a musical score symbol written on the print medium with reference to the position information;
A conversion step of converting the musical score symbol obtained by the recognition step into reproducible music data;
Brush said pen device when tested knowledge that a predetermined region of the print medium is beaten, detects the pen pressure applied to the pen tip of the pen-type device based on the position information detected the beating A tempo determination step of determining a tempo based on the detected interval by detecting the interval of hitting with reference to detection information of a pressure detection means;
A reproduction step of reproducing the music data based on the tempo;
A music playback program for running.

Images