JP2022187914A - Music playing toy using portable terminal device - Google Patents

Abstract

To provide a music playing toy which detects rotational speed of a rotation gesture by the rotation gesture on a touch panel by one contact point of an electronic device such as a portable terminal device, determines a playing position of a musical score inputted from digital musical score input means on the basis of the detected rotational speed, and plays a music instrument on the basis of the determined playing position or to provide a music playing toy which detects a virtual radius of rotation, determines a sound volume on the basis of the detected virtual radius of rotation and plays the music instrument on the basis of the detected sound volume.SOLUTION: Detection means for detecting rotational speed of a rotation gesture on a touch panel by one contact point of an electronic device such as a portable terminal device and determination means for determining a playing position of a musical score on the basis of the detected rotational speed are provided, and a music instrument is played on the basis of the determined playing position, or detection means for detecting a virtual radius of rotation of the rotation gesture and determination means for determining a sound volume on the basis of the detected virtual radius of rotation are provided, and the music instrument is played on the basis of the determined sound volume.SELECTED DRAWING: Figure 2

Description

TECHNICAL FIELD The present invention relates to a musical playing toy using a mobile terminal device such as a smart phone.

Conventionally, as shown in Patent Document 1, for example, lines corresponding to musical notes are printed and recorded on roll paper or the like to form a musical score, and the movement of the roll paper or the like is photographed, recorded, or created from digital moving image data by other methods. By providing an image input means for continuously cutting out still images and inputting them, and an image processing means for detecting musical notes recorded by image processing from the input still images, correspondence is made based on the detected musical notes. There is known a music playing toy that imitates a self-playing piano that plays the musical score by producing the sound of the notes played.

According to the same document, the image input to the image input means is not limited to the moving image input from the camera, and there is a scene of winding up a roll paper on which musical notes etc. are printed, and a roll paper of an actual self-playing piano. It is said that means for inputting moving images in MP4 format or the like obtained by video-recording how the tape is wound up during a performance, or similar moving images created electronically may be used.

Further, according to Patent Documents 2, 3, and 4, a musical score recorded on roll paper for an automatic piano and a piano roll that electronically simulates the state of performance are converted into separately prepared digital musical score data, such as MIDI data. It discloses the ability to play music based on

Furthermore, Japanese Patent Application Laid-Open No. 2002-200000 discloses a technique for receiving exercise information according to the movement of an operator transmitted from an operation terminal and editing existing digital musical score data based on the received exercise information. Although digital musical score data is called performance data in Patent Document 5, it will be called digital musical score data (abbreviated musical score data) here in order to compare with musical score images that will appear later. Digital musical score images and digital musical score data are collectively referred to as digital musical scores.

On the other hand, Patent Literatures 6 and 7 disclose techniques for recognizing as a gesture an operation of touching and moving a touch panel of a mobile terminal device with a plurality of fingers. A technology is disclosed that recognizes a gesture event as a gesture event, sends the recognized information to a browser, and dynamically rotates an element of a web page.

Further, in Patent Document 8, as a method of recognizing rotation of a finger on a touch panel, a virtual circle is set on the position coordinates, and among the center coordinates of the virtual circle, the radius of the virtual circle, and the rotation angle of the virtual circle, Techniques for calculating at least one change over time as an operation signal have been disclosed.

Here, as an input image for the apparatus disclosed in Patent Document 1, consider a still image of an electronically produced musical score, which is not disclosed in the document. This is an electronic image in which the musical score recorded on the roll paper disclosed in Patent Document 1 is saved in a file format such as PNG or JPEG. By using this musical score image, the same musical score image is cut out while sequentially shifting the reading position at regular time intervals in the same way as a still image is cut out from the camera or moving image data disclosed in Patent Document 1, and then the cut out image is displayed as an image. The score image can be played by passing it to the processing means and performing the same processing as disclosed in Patent Document 1. FIG.

At this time, if the position at which the still image is cut out as the performance position of the musical score is determined based on the rotation speed obtained by the rotation gesture on the touch panel of the mobile terminal device, the rotation speed depends on the rotation speed of the rotation gesture. It is possible to change the performance speed in the shape.

In addition, if the virtual diameter of the rotation circle is recognized when recognizing the rotation of the finger on the touch panel as a gesture, and the sound of the musical note is made to correspond to the volume of the performance, the sound of the musical note is played. It is possible to adjust the volume during a performance, which could not be realized with a device.

Furthermore, if the above-mentioned digital musical score data (MIDI data, etc.) is used instead of the musical score image as the musical score, a simpler apparatus that does not require image processing means can be realized.

However, Patent Documents 2, 3, 4, and 5 do not disclose a technique for changing the actual performance speed in real time. However, it does not disclose a method for determining the performance position of the musical score for performance of the musical score.

Further, in Patent Document 8, an operation using two or more fingers without determining the center of the circle, and an operation with one finger after determining the center of the circle in advance, the radius of the virtual circle and A method for obtaining the rotation angle of the virtual circle is disclosed, but a method for obtaining the diameter of the virtual circle by manipulating one finger without predetermining the center of the circle is not disclosed.

Patent application 2020-210850 Japanese Patent Application Laid-Open No. 2001-51586 JP 2007-279470 JP 2015-069151 JP 2004-053930 JP 2019-57295 JP 2016-201122 Patent No. 6058118

The problem to be solved by the present invention is to detect the rotation speed of the gesture from the rotation operation on the touch panel that is recognized as a gesture event by one contact point in an electronic device such as a mobile terminal device, and to detect the rotation speed from the detected rotation speed. To provide a practical method and apparatus for determining a performance position of a musical score for performance and playing the same musical score based on the determined performance position.

Another problem to be solved by the present invention is that, in an electronic device such as a mobile terminal device, from a rotation operation on a touch panel that is recognized as a gesture event by one contact point, the virtual diameter and determining the volume of the performance according to the length of the performance, thereby providing a practical method and device capable of adjusting the volume by gestures during the performance.

The present invention comprises a musical score reading means for reading a digitally recorded musical score to be used in a performance, a gesture detecting means for recognizing a turning gesture on a touch panel by a single contact point, and the gesture detecting means at regular time intervals. Rotational speed detection means for determining the rotation speed of the gesture from the obtained touch position coordinates on the two-dimensional coordinates, and a performance position for determining the performance position of the musical score from the gesture rotation speed detected by the rotation speed detection means. The most important feature is that a determination means is provided and the musical score is played based on the determined playing position.

Further, the present invention provides gesture detection means for recognizing a rotation gesture on a touch panel by one contact point, and the diameter of a virtual circle of rotation is determined from touch position coordinates on two-dimensional coordinates obtained from the gesture detection means at regular time intervals. A diameter detecting means for obtaining the diameter and a volume determining means for determining the volume of the performance from the obtained diameter of the virtual circle are provided, and the performance is performed at the volume based on the determination of the volume determining means.

According to the present invention, a performance position determination means is provided for determining a performance position of a musical score based on the speed of rotation of the rotation gesture by one contact point on the touch panel. It is possible to change the playing speed of the musical score according to the rotation speed of the drum, and the advantage that it is possible to provide the same function as a conventional musical playing toy equipped with a conventional physical hand-cranked device with only an electronic device such as a mobile terminal device. There is

Further, according to the present invention, there are provided diameter detection means for obtaining the diameter of a virtual circle of rotation of a rotation gesture by one contact point on the touch panel, and volume determination for determining the volume of the performance from the obtained diameter of the virtual circle. By providing the means and changing the volume of the performance based on the determination of the volume determining means, it is possible to freely change the volume during the performance by means of gestures, which is an advantage of being able to provide a function not found in conventional music playing toys. There is

FIG. 1 is an explanatory diagram showing the first implementation method. (Example 1) FIG. 2 is an explanatory diagram showing the configuration of hardware and software in the first embodiment. FIG. 3 is a flowchart showing software operations in the first embodiment. FIG. 4 is an explanatory diagram 1 for explaining contents of gestures in the first embodiment. FIG. 5 is an explanatory diagram 2 for explaining contents of gestures in the first embodiment. FIG. 6 is an explanatory diagram for explaining a general musical score. FIG. 7 is an explanatory diagram for explaining a digital musical score image according to the first embodiment. FIG. 8 is an explanatory diagram showing the second implementation method. (Example 2)

The purpose of realizing a music playing toy that simulates a conventional automatic piano with only electronic devices such as mobile terminal devices is to input digital musical scores from a digital musical score input means and gesture by one touch point on the touch panel of the mobile terminal. By detecting the rotation speed of the gesture, determining the performance position of the same score from the detected rotation speed, and playing the same score based on the determined performance position, the score is played according to the rotation speed of the gesture. It was realized by the method of playing. In addition, for the purpose of realizing volume control by gestures when playing the same music playing toy, the diameter of the virtual circle of rotation by gesture by one touch point on the touch panel was obtained, and the volume of the performance was calculated from the diameter of the virtual circle obtained. This was achieved by determining the

FIG. 1 is a general view of the first embodiment of the device of the present invention, wherein 100 is a smart phone main body, and 101 is an image output section using a liquid crystal panel. The liquid crystal panel is a touch panel, and also serves as an information input section 102 operated by the user. The part indicated by the dashed frame of 1011 is read from the musical score input means in the main body of the smartphone, and the part necessary for display is cut out or drawn, and the movement of the musical score according to the progress of the performance is displayed in real time. In the sheet music image display section displayed in , the portion of the glass window through which the rolling paper can be seen as the roll paper is being played with the performance of a conventional automatic playing piano is simulated. Note that the score image displayed in this embodiment moves in the direction of the arrow indicated by 302 as the performance progresses. A dashed line 1020 is a virtual timing line indicating a performance position, and indicates that a sound is produced when a note is on that line. 2001, 2002 and 2003 are musical notes of the score image displayed on the liquid crystal panel 101. FIG. Marker lines 2010 and 2011 are recorded at both ends of the musical score image displayed on the same panel 101 for position detection of notes. In this embodiment, the marker lines are used in the image processing means, which will be described later, but they are not essential for generating notes using the digital musical score. The portion indicated by the dashed frame of 1012 is the keyboard display section. When there is a note at the performance position, the white and black keys are grayed out to indicate that the key corresponding to the note has been pressed. change and display. Also, if the note that was at the playing position has moved and disappeared over time, the colors of the white and black keys are changed to their original colors to indicate that the key corresponding to the note has been released. indicate. Reference numeral 300 denotes a single finger for performing a gesture operation on the liquid crystal panel 101, and 301 denotes an arrow simulating a rotational gesture by the finger 300. FIG. When the finger 300 is rotated in the direction of the arrow 301, the musical score moves in the direction of the arrow 302 according to the rotation speed, and when a note comes on the imaginary timing line 1020 which is the performance position, the sound is produced and the timing is adjusted. A performance is realized by erasing the sound when the note deviates from the line 1020 .

FIG. 2 is a configuration diagram showing the hardware and software of the embodiment of the device of the present invention. Reference numeral 1 denotes a CPU (Central Processing Unit), which implements the present invention by performing various processes and issuing commands to various means according to a program stored in a nonvolatile memory (not shown). A score image input means 2 is used by the CPU 1 to read a score image from a score storage means (not shown) in the main body of the apparatus composed of a non-volatile memory or from an external website or the like. . It should be noted that the musical score image stored in the musical score storage means can also be created from musical score data such as MIDI. The details are omitted. Gesture detection means 7 recognizes the aforementioned rotation gesture by the finger 300 on the touch panel 102 and detects the position coordinates (x, y) of the finger 300 on the touch panel 102 at regular time intervals. Based on the detected position coordinates (x, y) of the finger 300, the CPU 1 calculates the rotation speed of the gesture by the rotation speed detection means 8 configured by the software program, and determines the performance position similarly configured by the software program. The performance position of the musical score is determined by the means 10 . Further, from the detected position coordinates (x, y) of the finger 300, the CPU 1 calculates the diameter of the virtual circle of rotation caused by the gesture by the diameter detection means 9 of the virtual circle of rotation constituted by the software program. The volume of the performance is determined by the volume determining means 11 composed of. Reference numeral 3 denotes an image output means composed of a liquid crystal panel 101 or the like. , the image is trimmed to a size necessary for display in , and is transferred to the image output means 3 , and the image output means 3 displays the musical score image transferred from the CPU 1 on the musical score image display section 1011 . The CPU 1 also cuts out a portion corresponding to the same performance position from the musical score image read from the musical score image input means 2 and sends it to the image processing means 4. , and detect whether there is a note at the specified position. The image processing means 4 that detects whether or not there is a note returns the information to the CPU 1, and the CPU 1 stores the state of the keyboard corresponding to the note at the timing when the previous still image was delivered from the keyboard state storage means 5. If it is determined that the previous state of the keyboard is "unpressed" and that there is a note in the currently passed still image, it is determined that the sound of the corresponding note will be played, and the same If it is determined that the state of the keyboard at the previous processing timing is "pressed" and that there is no note at present, it is determined that the sound of the corresponding note is stopped, and the contents of the keyboard state storage means 5 be updated to the latest state. Based on the determination, the CPU 1 instructs the sound output means 6 to produce or stop the sound of the corresponding note. At this time, when the sound of a musical note is to be produced, an instruction is given together with the volume determined by the volume determining means 11. - 特許庁A sound output means 6 has a function of outputting or stopping the sound of a corresponding note with a tone color set in advance according to an instruction from the CPU 1 at a specified volume, thereby playing music according to the musical score image. In addition, the CPU 1 instructs the image output means 3 to change the color of the corresponding keyboard according to the decision to play or stop the sound of the corresponding note. Change the color of the corresponding keyboard.

FIG. 3 is a flow chart showing the operation of the software of the same embodiment of the device of the present invention. First, the CPU 1 waits for a gesture event to be detected by the gesture detection means 7 in S1. In the case of this embodiment, when the gesture operation by the finger 300 starts, an event occurs at an interval of 1/60 second, and the position where the finger 300 first touches the panel is the origin O: (x, y)=(0 , 0), and thereafter, relative coordinates from the origin are detected each time until the finger 300 leaves the panel. If the gesture is detected, the process advances to S2, and the rotation speed and the diameter of the virtual circle due to the gesture are obtained by the rotation speed detection means 8 and the diameter detection means 9 of the virtual circle of rotation, respectively.

Here, a method of obtaining the rotation speed and the diameter of the virtual circle by the speed detection means 8 and the diameter detection means 9 of the virtual circle of rotation from the position coordinates of the finger 300 on the panel due to the gesture event will be described. FIG. 4 shows the coordinate system on the panel and feature points in the gesture operation. ), with the positive x-axis on the right and the positive y-axis on the bottom. Dh and Dv represent the distances between the feature points Plc and Prc and between the feature points Ptc and Pbc used to obtain the diameter of the virtual circle of rotation, respectively.

FIG. 5 is a transition graph of the coordinates (x, y) when the finger 300 is rotated by gestures over time, detected by the gesture detection means 7 . The horizontal axis is the number of occurrence events (0 to 119), the vertical axis is coordinate values, and the solid and broken lines indicate the x and y values at the time of event occurrence, respectively.

で求められる。 From these, the rotation speed V can be obtained by taking the coordinate values of the event that occurred most recently as (X, Y) and the coordinate values of the event that occurred immediately before that as (Xp, Yp).

is required.

により仮想円の直径Dが求められる。 On the other hand, from FIGS. 4 and 5, the diameter of the virtual circle of rotation is the feature point (in the figure, each upward black-and-white triangles Plc, Ptc) and feature points whose slope changes from positive to negative (represented by downward-pointing black-and-white triangles Prc, Pbc in the figure). The distance between them roughly corresponds to the diameter of the virtual circle of rotation. Assuming that these are Dh and Dv, respectively, these two values are generally different, so the diameter of the virtual circle is the average of both. Therefore, let the coordinate values in Plc be (Xlc, Ylc), the coordinate values in Prc be (Xrc, Yrc), the coordinate values in Pbc be (Xbc, Ybc), the coordinate values in Ptc be (Xtc, Ytc), and the virtual circle If D is the diameter,

The diameter D of the virtual circle is obtained by

The above calculations are performed by the rotation speed detection means 8 and the diameter detection means 9 of the virtual circle of rotation, respectively. A performance position and volume are determined by the position determination means 10 and the volume determination means 11 .

Here, a musical score image representing a musical score digitally will be described. 6 and 7 are diagrams for explaining how a general musical score is expressed and played as a digital musical score image. What is shown in FIG. 6 is a general score, and what is indicated by 2004, 2005, 2006, 2007, and 2008 in the same score are quarter notes of different scales. A symbol indicated by 400 is a speed symbol, which indicates that a quarter note should be played at a speed of 80 times per minute. FIG. 7 shows the beginning of a musical score image obtained by converting the musical score shown in FIG. 6 into a digital image. Its coordinates are indicated by 401 in FIG. The notes included in the circle are indicated as 2004, 2005, 2006, 2007, 2008. Marker lines 2010 and 2011 are recorded at both ends of the digital musical score image for position detection of notes. In this embodiment, the number of pixels in the width of the score image is 384 pixels, the width of the note is 2 pixels, the gap between adjacent notes is 1 pixel, and 85 notes (= 7 octaves [1 octave is 12 octaves including semitones)]. sound]+1 sound), the space between the marker lines 2010 and 2011 is 256 pixels=(2+1)×85+1 (+1 is the gap between the marker lines). In this embodiment, the marker lines 2010 and 2011 are arranged on the musical score image in order to detect the positions of the notes. If so, the marker lines 2010 and 2011 are not essential.

Here, the quarter note 2004 in FIG. 6 is positioned three octaves above the lowest note in this embodiment, so it corresponds to the 36th note (=12 notes×3 octaves) counted from the lowest note. Therefore, there is an interval of 109 pixels=(2+1)×36+1 from the marker line 2010 in FIG. Similarly, quarter notes 2005, 2006, 2007 and marker line 2010 are spaced 115, 121, 124 pixels, respectively.

The length of the quarter notes 2004, 2005, 2006, 2007, and 2008 in FIG. 7 in the y-axis direction corresponds to the length of the quarter note. The distance to the beginning of quarter note 2005 is 180 pixels. Similarly, the distances between 2005 and 2006, 2006 and 2007, and 2007 and 2008 are also 180 pixels. Although not shown here, an eighth note has a length of 90 pixels, a sixteenth note has a length of 45 pixels, and a half note has a length of 360 pixels.

となる［ＲＯＵＮＤ（）は小数点以下四捨五入を表す］。以上により、速度Ｖのときに破線の進む距離であるピクセル数Ｎが求まるので、前回のジェスチャー・イベント発生時の演奏位置にピクセル数Ｎを加えて新たな演奏位置とすればよい。このような方法でＣＰＵ１はＳ３において演奏位置決定手段１０により演奏位置を決定する。 The performance of the musical score in FIG. 6 progresses from left to right as indicated by arrow 500 according to the musical score. On the other hand, the musical score image proceeds from bottom to top as indicated by an arrow 500 in FIG. A dashed line 501 in FIGS. 6 and 7 indicates the performance position, and if this dashed line is advanced in the direction of the arrow 500 at regular time intervals, the performance on the musical score and the musical score image will progress. Therefore, the distance that the dashed line 501 advances is determined according to the rotation speed at the timing when the gesture event occurs (1/60 second interval in this embodiment). For example, in the case of this embodiment, the speed symbol 400 in FIG. 6 indicates that the quarter note is played at a speed of 80 times per minute. Pixels (=180 pixels×80 times) should be advanced. Since the gesture event occurs at an interval of 1/60 second, it suffices to advance 4 pixels (=14400 pixels/60 seconds/60 times) for each occurrence of the gesture event. On the other hand, since it is known that the rotation speed V obtained in step S2 has an average value of 15.0 as a result of actual measurement, a constant If C is obtained, 4 pixels=15.0×C, then C=4/15.0=0.2667.
Therefore, if the number of pixels advanced at the speed of rotation V is N,

[ROUND( ) represents rounding to the nearest whole number]. As described above, the number of pixels N, which is the distance that the dashed line advances at the velocity V, can be obtained. Therefore, a new performance position can be obtained by adding the number of pixels N to the performance position at the time when the previous gesture event occurred. In this manner, the CPU 1 determines the performance position by the performance position determining means 10 in S3.

となる。以上により音量を決定する事ができるので、ＣＰＵ１はＳ３において音量決定手段１１により音量を決定する。 On the other hand, volume determination by the volume determining means 11 is as follows. In this embodiment, the diameter D of the virtual circle detected by the diameter detecting means 9 of the virtual circle of rotation is found to fall within the range of about 0 to 236 as a result of actual measurement. Therefore, the detected value is clipped between 0 and 236 and scaled to the volume range that can be specified for the sound output means 5 . In the case of this embodiment, the values are from 0 to 127, so if the volume to be specified to the sound output means 5 is Vol, then

becomes. Since the volume can be determined as described above, the CPU 1 determines the volume by the volume determining means 11 in S3.

After determining the performance position and volume in step S3, the process advances to step S4 to trim the musical score image so that it can be displayed on the musical score display section 1011 on the liquid crystal panel 101 based on the performance position. For example, if the performance position is the position indicated by the dashed line 501 in FIG. Crop to display. The trimmed musical score image is delivered to the image output means 3, and the CPU 1 instructs the musical score display section 1011 of the liquid crystal panel 101 to display the same image. Further, in S5, the CPU 1 cuts out a portion necessary for image processing based on the performance position obtained in S3 from the musical score image. In this embodiment, the portion necessary for image processing is the one-line image on the dashed line 501 indicating the playing position in FIG. At S6, the CPU 1 transfers the musical score image cut out at S5 to the image processing means 4, and instructs the image processing means 4 to detect notes. At S7, the image processing means 4 specifies the positions of the notes from the same clipped musical score image delivered, detects whether or not there is a note at the specified positions, and returns the result to the CPU1. At S8, the CPU 1 determines whether the note detected from the same digital musical score image and the keyboard corresponding to the same note determined from the previously input digital musical score image stored in the keyboard state storage means 5 is pressed. Whether the keyboard is newly pressed or released is determined based on the information storing the state of absence, and the state is updated and stored in the keyboard state storage means 5.例文帳に追加In S9, based on the determination of whether the keyboard is newly pressed or released in S8, the sound output means 6 is instructed to output or stop the sound of the note corresponding to the same keyboard with the volume determined in S3. At S10, the CPU 1 instructs the image output means 3 to change the color of the corresponding keyboard based on the determination at S8 whether the keyboard is newly pressed or released. Since the series of processing due to the occurrence of the gesture event is completed as described above, the process returns to S1 again and waits for the gesture event to be detected by the gesture detection means 7 .

The content of the image processing for detecting notes from the digital musical score image by the image processing means 4 in this embodiment is as described in Japanese Patent Application No. 2020-210850. In this embodiment, the marker lines 2010 and 2011 are arranged on the digital musical score image in order to obtain the x-coordinate position of the note. It can detect musical notes from images. However, in this embodiment, unlike the configuration of inputting an actually photographed image as described in Japanese Patent Application No. 2020-210850, a digitally created score image is input, so camera shake during photographing is suppressed. Also, there is no need to consider the fluctuation of the roll paper during hand-cranked performance. Therefore, as already described, if the positions of the notes on the x-axis of the musical score image are determined in advance, the marker lines 2010 and 2011 are not essential, and the brightness of the pixels at the predetermined positions is determined in advance. Equivalent processing can be realized by checking the level and the difference in brightness level with adjacent pixels.

As described above, according to this embodiment, there are the musical score input means for inputting the musical score image, the gesture detection means for recognizing the rotation gesture by one contact point on the touch panel, and the rotation speed of the detected gesture for obtaining the rotation speed. and performance position determination means for determining the performance position of the musical score image from the rotation speed of the gesture detected by the rotation speed detection means, and the digital musical score is generated based on the determined performance position. By adopting the playing method, a musical playing toy simulating a conventional automatic playing piano can be realized only by an electronic device such as a portable terminal device.

The present invention also provides gesture detection means for recognizing a rotation gesture by a single contact point on a touch panel, virtual rotation diameter detection means for obtaining the diameter of the virtual circle of rotation of the detected gesture, and detection. A conventional self-playing piano is simulated by providing volume determination means for determining the volume of notes from the diameter of the virtual circle, and playing the notes by changing the volume of the notes to be played based on the determination of the volume determination means. We have made it possible to realize functions that could not be realized with music playing toys.

In this embodiment, the rotation gesture with one contact point using the finger 300 on the touch panel is described, but a gesture with one contact point using a touch pen or the like may be used as long as the gesture can be recognized.

FIG. 8 shows a second embodiment of the present invention. In the first embodiment, a musical score image is input from the musical score image input means 2, but in this embodiment digital musical score data such as MIDI is used instead of the musical score image. is changed to the musical score data input means 12 for inputting .

Since this embodiment does not have a musical score image input means, the musical score image displayed on the musical score display section 1011 is based on the performance position determined by the performance position determining means 9 and the musical score data input from the musical score data input means 12. An image for display on the musical score display section 1011 is created by the image generating means 14 and displayed, but details thereof are omitted here.

Further, in the first embodiment, a portion corresponding to the performance position is cut out from the musical score image read out from the musical score image input means 2 and sent to the image processing means 4. The position of the note is specified by the image processing means 4, and the specified position is However, in this embodiment, it is detected whether or not there is a note between the musical score data input from the musical score data input means 12 and the performance position determined by the performance position determining means 10. The musical score data analysis means 13 for directly detecting the musical notes determines the musical notes to be played.

As shown in Patent Document 2, musical score data generally holds data in the form of which note is played or stopped at which performance position. For example, in the case of MIDI data, groups called tracks hold data in the form of repetitions of delta time (duration referred to in Patent Document 2) and events (note events referred to in Patent Document 2). Time is the time obtained by the length of a quarter note separately specified and the time unit, and represents the elapsed time from the delta time specified one before, and the event is the scale, volume, and It specifies whether it is on or off. Therefore, when the performance position is determined by the performance position determination means 10, the CPU 1 inputs the score data from the score data input means 12 by the score data analysis means 12 constituted by the software program, analyzes it, It is possible to determine notes that have changed from the performance position at the time of event occurrence to the current performance position. If the changed note can be determined, the sound output means 6 is instructed to output or stop the sound along with the volume determined by the volume determining means 11, thereby realizing a function equivalent to that of the first embodiment. .

According to this embodiment, the musical score image input means 2 is replaced by the musical score data input means 12, and the musical score data is used instead of the image data for the musical score for performance, so that the determination of musical notes by image processing is possible. Since this is not performed, a music playing toy can be realized more easily.

By detecting the speed of rotation by electrical signals instead of gestures to determine the performance position and playing the musical score based on the determined performance position, not only portable terminal devices but also personal computer devices, or single units It can also be applied to music playing toys that operate with

1 CPU (central processing unit)
2 musical score image input means 3 image output means 4 image processing means 5 keyboard state storage means 6 sound output means 7 gesture detection means 8 rotation speed detection means 9 virtual circle diameter detection means 10 performance position determination means 11 volume determination means 12 score data input means 13 score data analysis means 14 image generation means

Claims (4)

digital musical score input means; gesture detection means for recognizing a rotation operation on the touch panel by one contact point as a gesture; means for detecting the rotation speed of the gesture from the detected gesture; A musical playing toy comprising a means for determining a playing position of the musical score from a musical score and a detected rotation speed, and playing a musical note by determining notes to be played based on the determined playing position. 2. The digital musical score is read as a digital musical score image from the musical score image input means based on the determined performance position, and the musical score is determined by the image processing means and played. A musical playing toy as described in . The digital musical score is read from the musical score data input means as digital musical score data, and based on the data and the determined performance position, the musical score data analysis means determines notes to be played and performs the musical score. The musical playing toy according to claim 1. Gesture detection means for recognizing a rotation operation on the touch panel by one contact point as a gesture, means for detecting the diameter of a virtual circle of the gesture from the detected gesture, and determining the volume of the note from the diameter of the detected virtual circle. A music performance toy characterized by providing a volume determining means for playing a musical instrument by changing the volume of notes to be played based on the determination of the volume determining means.

Images