KR101524279B1 - Rgb-d camera based piano education system and method - Google Patents

Rgb-d camera based piano education system and method Download PDF

Info

Publication number
KR101524279B1
KR101524279B1 KR1020140033664A KR20140033664A KR101524279B1 KR 101524279 B1 KR101524279 B1 KR 101524279B1 KR 1020140033664 A KR1020140033664 A KR 1020140033664A KR 20140033664 A KR20140033664 A KR 20140033664A KR 101524279 B1 KR101524279 B1 KR 101524279B1
Authority
KR
South Korea
Prior art keywords
depth
key
keyboard
pressed
rgb
Prior art date
Application number
KR1020140033664A
Other languages
Korean (ko)
Inventor
김준호
노승민
황재인
Original Assignee
국민대학교산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 국민대학교산학협력단 filed Critical 국민대학교산학협력단
Priority to KR1020140033664A priority Critical patent/KR101524279B1/en
Application granted granted Critical
Publication of KR101524279B1 publication Critical patent/KR101524279B1/en

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B15/00—Teaching music
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances
    • G09B5/02—Electrically-operated educational appliances with visual presentation of the material to be studied, e.g. using film strip

Abstract

The present invention relates to a piano education system, and more particularly to a piano education system which detects a depression depth of a keyboard by a fingering of a user and confirms the position of the keyboard, The present invention relates to a piano education system using an RGB-D camera and a method thereof.
A piano education system using an RGB-D camera according to the present invention includes a camera for capturing a plurality of color (RGB) images and depth images of a plurality of keys; A detector for detecting a position and a depth of each key by using the captured color (RGB) image and the depth image; A setting unit configured to set an initial position and a depth of the keyboard using the detected positions and depths of the keys; A determining unit for determining whether a key is pressed by comparing an initial position and a depth of the set keyboard with a depth and a depth of the keyboard when the user depresses the keyboard; And a display for displaying the determined result.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a piano education system using RGB-D cameras,

The present invention relates to a piano education system, and more particularly to a piano education system which detects a depression depth of a keyboard by a fingering of a user and confirms the position of the keyboard, The present invention relates to a piano education system using an RGB-D camera and a method thereof.

In modern society, the piano is a familiar medium and has become a musical instrument in everyday life. As the piano becomes more popular, there is an increasing interest in playing piano, and many people want to learn because they can develop their sensitivity, insight, imagination, and experience their artistic expression, thereby increasing their level of self-expression.

However, the piano is recognized as an instrument that is difficult to learn in that it handles many keys simultaneously. Especially, for the education of the piano, it is necessary to understand it according to the learner 's situation and to do it repeatedly so that it does not lose interest, and various musical experiences can be done.

Several systems have been studied for the purpose of increasing the efficiency of piano education. Especially, it is mainly used to show musical score through PC monitor screen output by using a MIDI (Musical Instrument Digital Interface) keyboard and to learn the piano through the input of the MIDI keyboard. However, since the acoustic touch of the keyboard of the acoustic piano I can not feel the feeling. Conversely, traditional acoustic pianos rely on MIDI inputs or digital signals to enter, practice, and train the songs being played.

Therefore, existing piano education methods do not exceed the limits of piano education in order to cultivate the actual piano playing ability.

Accordingly, the present invention has been made to solve the above problems of the conventional art, and it is an object of the present invention to photograph a key depressed by a user through a camera and detect the depth of the depressed key to find its position.

It is another object of the present invention to provide a piano education system by only recognizing a piano key depression without inputting a digital signal even in an acoustic piano.

It is another object of the present invention to provide a piano education system capable of real time displaying a result of pressing a key by a user together with a pre-stored music score according to a flow of a music piece.

According to an aspect of the present invention, there is provided a piano education system using an RGB-D camera, including: a camera for capturing a plurality of color (RGB) images and depth images of a plurality of keys; A detector for detecting a position and a depth of each key on the captured color (RGB) image and the depth image; A setting unit configured to set an initial position and a depth of the keyboard using the detected position and depth of each key; A determining unit for determining whether a key is pressed by comparing an initial position and a depth of the set keyboard with a depth and a depth of the keyboard when the user depresses the keyboard; And a display for displaying the determined result.

In the present invention, the setting unit may set one or more patches at predetermined intervals on each of the keyboards, and the detecting unit may be configured to set a predetermined number of patches And detects the depth of the patch (patch).

In the present invention, when the depth of the patch is detected, the detection unit may detect a depth of the patch, which is not covered by the finger, Multiply the weight for the length ratio up to the patch corresponding to one of the patches.

In the present invention, the setting unit sets the entire musical scale position of the keyboard by finding a key of another octave in a predetermined interval from the pressed key, based on the position of one or more keys pressed by the user.

In the present invention, when the depression of the key is determined, the display displays the scale and the tempo of the pressed key along with the pre-stored score according to the flow of the tune.

In the present invention, the display indicates whether the musical scale and beat of the pressed key and the musical scale and beat on the musical score match.

According to another aspect of the present invention, there is provided a method of teaching a piano using an RGB-D camera, comprising: capturing a color (RGB) image and a depth image of a plurality of keys; Detecting a position and a depth of each key by the detected color (RGB) image and the depth image; Setting the initial position and depth of the keyboard using the position and the depth of each key detected by the setting unit; Determining whether the key is pressed or not by comparing the initial position and depth of the set keyboard with the depth and depth of the keyboard when the user depresses the key; And displaying the determined result through a display; .

In the present invention, the step of setting the initial position and the depth of the keyboard may further include a step of setting a key of another octave by a predetermined interval from the pressed key, based on the position of one or more keys pressed by the user Setting a total scale position of the search key; .

In the present invention, it is preferable that the step of determining whether or not the key is depressed is characterized in that the detection unit sets at least one patch on each of the keys at a predetermined interval, and when a specific key is pressed by the user, Detecting a depth of a patch not covered by the finger; .

In the present invention, it is preferable that the step of determining whether or not the key is pressed comprises the steps of: when a predetermined key is depressed by a user, a patch of a patch of the depressed key, And multiplying the weight of the length ratio up to the patch corresponding to one of the uncovered patches of the whole length.

The piano education system and method using the RGB-D camera according to the present invention have the following effects.

According to the present invention, since an acoustic piano does not receive a separate MIDI signal input and recognizes the depression of the keyboard through the camera, it has an advantage that it can naturally learn the touch feeling of the keyboard and improve a great skill.

In addition, according to the present invention, it is possible to perform interesting learning and efficient learning by displaying the result of pressing the key by the user in real time together with the music score stored so as to allow the user to learn by himself / herself.

1 is a block diagram of a piano education system using an RGB-D camera according to the present invention.
FIG. 2 is a diagram illustrating an example of a patch bundle line setting of a keyboard according to an embodiment of the present invention. FIG.
3 is an exemplary view illustrating position detection of a black key according to an embodiment of the present invention;
4 is a diagram illustrating an example of setting the overall scale position of a key according to an embodiment of the present invention;
5 is a diagram illustrating an overall position correction of a key according to an embodiment of the present invention.
FIG. 6 illustrates an example of recognition of a patch bundle according to a user's key fingering position in an embodiment of the present invention; FIG.
FIG. 7 is an exemplary view illustrating application of a weight of a key according to an embodiment of the present invention; FIG.
FIG. 8 is an exemplary view showing pre-stored score and actual performance information display in an embodiment of the present invention; FIG.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

1 is a block diagram of a piano education system using an RGB-D camera according to the present invention.

Referring to FIG. 1, the piano education system using the RGB-D camera of the present invention includes a camera 10, a control module 20, a piano 30, and a display 40.

The piano 30 is an acoustic piano. In the embodiment of the present invention, an unplugged piano is used. The piano 30 may use one of a grand piano and an upright piano composed of a plurality of black and white keys 31.

The camera 10 takes a picture of the key 31 of the piano 30. The camera 10 is mounted on the keyboard 31 using a separate stand (not shown) at a predetermined height in a direction perpendicular to the keyboard 31 so as to photograph the keyboard 31 in addition to the main body including the incense plate of the piano 30 .

The camera 10 photographs a color (RGB: red-green-blue) image and a depth image of a plurality of keys 31 of the piano 30. The RGB-D camera for acquiring the color (RGB) image and the depth image may include a Depth sensor integrally included in the RGB camera or a separate depth camera. A typical camera obtains only RGB images, but the camera 10 used in the embodiment of the present invention uses a camera capable of obtaining depth information.

In order to photograph the depth image, SwissRanger 4000, PMD [vision] CamCube, D-IMager, and Kinect of Microsoft may be used according to design conditions.

The RGB (red-green-blue) image and the depth image of the key 31 photographed by the camera 10 are separately used for detecting the position and depth of each key May be stored and updated in a storage device (not shown)

1, the control module 20 includes a detection unit 201, a determination unit 202, and a setting unit 203.

The detection unit 201 detects the position and depth of each key by using the captured color (RGB) image and the depth image.

More specifically, a color (RGB) image and a depth image captured through the camera 10 are subjected to a process of detecting pixel information through each photographed image. (RGB) image and a depth image are captured before detecting the pixel information. That is, given the position and calibration information between the depth sensor and the RGB camera, the sensor matching is performed by projecting the three-dimensional point. Thereby correcting the parallax and the field of view (FOV) of each camera or sensor that captures the color (RGB) image and the depth image. Since the sensor matching process is a conventional technique, a description thereof will be omitted in the present invention. The detection unit 201 forms a depth map using the detected position and depth of each key. The depth map may include pixel information and depth information corresponding to the entire area of the key 31.

The setting unit 203 sets the initial position and depth of the keyboard 21 using the detected position and depth of each key.

FIG. 2 is an exemplary view illustrating a patch bundle line setting of a keyboard according to an embodiment of the present invention.

2A and 2B, the setting unit 203 processes the color (RGB) image photographed through the camera 10 and processes the image to set the initial position of the key 31. First, a black key line is searched out of the entire area of the key 31 using two-dimensional pixel data extracted through a color (RGB) image. That is, the entire area of the black key having the color (RGB) distinguishable from the white key is retrieved according to a certain line by using the inherent pixel data of the black key. If the camera 10 and the piano 30 are not located in parallel with each other as shown in FIG. 2 (c), the color (RGB) image is extracted The entire area of the black key is re-searched by rotating a predetermined line formed using the two-dimensional pixel data. The searching process can be performed until a correct black key is searched.

When the entire area of the black key is searched, the position of the black key is searched in the depth map.

3 is an exemplary view illustrating the position detection of a black key according to an embodiment of the present invention. As shown in FIG. 3 (a), the positions of the black keys are represented by different depths. However, the numerical values appearing in the depth map generated through the depth image are irregular patterns due to the noise, but the depths of the white and black keys are generally discriminated to be distinguishable .

As shown in FIG. 3B, the setting unit 203 distinguishes a point having a low depth and a point having a high depth from the values in the depth map, Set the position of the black key.

When the position of each black key is set, the setting unit 203 finds a key of another octave of a predetermined interval from the pressed key on the basis of the position of one or more keys pressed by the user, ) Is set.

4 is a diagram illustrating an example of setting the overall scale position of a keyboard according to an embodiment of the present invention. 4A and 4B, when the user presses the lowest C key and the highest C key on the key 31, the user divides the interval of the two keys by the total octave number set in advance. In general, a piano with 88 keys is composed of 7 octaves, so that the interval between the lowest C key and the highest C key is divided by 5 octaves. The interval between one octave interval for setting the interval between octaves, that is, the interval between C, D, E, F, G, A and B keys is stored. When the interval between the seven octaves is set, the entire scale position of the keyboard 31 is set by setting a scale between the octaves.

When the entire musical scale position of the keyboard 31 is set, the setting unit 203 sets one or more patches on the respective keys at predetermined intervals. The patch is for detecting a change in depth of a key 31 pressed by the user at the time of playing the piano. The patch is not physically attached, but sets an area to be matched to each position of the key board to be photographed by the camera 10. In the embodiment of the present invention, seven patches are set for white keys and four patches are set for black keys. Each of the patches has a width and a length of n * n pixels in a square shape within a range not exceeding the width and length of each key.

The patches are arranged using a depth map as in the case of setting the positions of the black keys. FIG. 5 is an exemplary view showing the overall position correction of the keyboard according to an embodiment of the present invention. As shown in FIG. 5 (a), the relative position of the black key and the white key varies with the distance from the camera 10, and the position at which the patch is arranged is set by the setting unit 203 The keyboard may be out of position. Referring to FIG. 5B, a depth map is used as in the case of setting the entire scale position of the keyboard 31 when the patch is set. In the depth map, The scale may be erroneously set on the key A corresponding to the position of X in the middle. At this time, the setting unit 203 corrects the scale with the key B corresponding to the position of Y, which is estimated as the position of the black key, not the X among the alpha regions of the depth map (Depth Map). Accordingly, by rearranging the remaining keys based on the position of the B key that is found correctly as shown in FIG. 5 (c), the erroneous position of the patch is corrected.

The detecting unit 201 detects a depth of a patch not covered by a user's finger among the set patches when a specific key is pressed by the user. The positions of the areas where the keys 31 are pressed are different according to the flow of the music or the performance technique of the user.

FIG. 6 is a diagram illustrating recognition of a patch bundle according to a user's key fingering position in an embodiment of the present invention. FIG. According to Fig. 6 (a), when the white key is completely pressed, the patches of the pressed keys are detected at depths of 10, 9, 8, 7, and 6 mm, respectively. The determination unit 202 compares the depth of the keyboard with the depth of the predetermined key 31 and determines whether the depth of the key is greater than the depth of the predetermined key 31 . The maximum depression depth of the keyboard for determining the depression can be set to 60 to 70%. For example, in the case of a keyboard having a maximum depression depth (Depth) of 11 mm, when the key pressed by an actual user is 7 mm or more, the determination unit 202 can determine that the key is pressed.

6 (b), when the outer two patches of the patches of the white keyboard are covered by the fingers, the camera 10 sets the depth of the obscured patches as The detection unit 201 can not detect the depth of the keyboard. In the above case, when the number of patches that are not obstructed to detect the depth of the remaining patches other than the obscured patches is small, the depth of the patches is small, Judgment can be difficult.

To this end, the detecting unit 201 multiplies a patch not covered by the finger when a depth of a patch is detected by a weight corresponding to a length ratio of the key. That is, the Depth of the key pressed by the finger is calculated by multiplying the depth of the patch not covered with the finger by the weight.

FIG. 7 is a diagram illustrating an application of a weight of a key according to an embodiment of the present invention. Referring to FIG. 7 (a), in the case of a white keyboard, the depression of the keyboard has a maximum of 11 mm when it is normally pressed. If at least one of the patches of the keyboard is covered with a finger and is not detected, the depth of the patch corresponding to X is detected, and the depth of the patch up to the X patch Depth of the depressed key is calculated using the key length A The interval between the patches and the length of the patch use predetermined values. If X is detected as 7 mm, the Depth of the pressed key is calculated by multiplying 7 mm by the weight of the length ratio up to the X patch (Patch) of the entire length of the keyboard. In an embodiment of the present invention, when the vertical length of the keyboard is 150 mm, the depth of the X patch (patch) is 7 mm, and the distance A to the X patch (Patch) is 100 mm, the weight is multiplied by X * 150 / It can be seen that the Depth of the pressed key is 10.5 mm.

As shown in Fig. 7 (b), the depression of the pressed key can be obtained by using the entire keyboard length of 80 mm and the length B up to the Y patch (patch) in the black keyboard as well.

When the depression of the keyboard is judged, the display 40 displays the scale and the tempo of the pressed key along with the pre-stored score according to the flow of the tune.

The display 40 displays actual music information such as a musical score stored in the upper body of the piano 30 or a musical instrument and a pre-stored musical score and a musical scale, a beat, and a note when the user plays the piano. The display 40 may be configured in a PC form with a single device or control module 20. In the present invention, a portable display which is easily removable or mountable is used to easily move to another acoustic piano.

FIG. 8 is a view illustrating an example of a pre-stored score and actual performance information in an embodiment of the present invention. Referring to FIG. 8 (a), a score is obtained through a MIDI file or a score downloaded on another medium is displayed on the display 40. The score includes information for a player to play, such as a tempo, a note, and a beat, and is sequentially displayed according to a predetermined time or flow. In addition, it is possible to visually confirm whether or not the key actually pressed is displayed by displaying the key currently played by the player in real time.

As shown in FIG. 8 (b), an event is generated according to whether or not the score displayed on the display 40 matches the performance played by the user. In order to display the current music flow, a rectangular bar is displayed on the currently played note, and a message such as Perfect, Good, Miss is displayed on the display 40 according to the accuracy of the performance. According to the design conditions, voice guidance can be provided besides the message.

Hereinafter, an embodiment of a piano education method using the RGB-D camera having the above-described configuration will be described with reference to FIG.

First, the camera captures color (RGB) images and depth images of a plurality of keys (S10). Next, the detection unit 201 detects the position and depth of the key by the captured color (RGB) image and the depth image (S11). When the depth is detected, the setting unit 203 sets the initial position and depth of the keyboard using the detected position and depth of the key (S12). At this time, in order to set the position of the keyboard, that is, the initial position and depth of the keyboard, the setting unit 203 sets the position of the key by pressing the key You can set the overall scale position of the keyboard by looking for other octave keys. When the setting is completed, the determining unit 202 compares the initial position and depth of the set keyboard with the position and depth of the keyboard when the user depresses the keyboard, thereby determining whether the key is pressed S13). For this determination, the detection unit 201 sets one or more patches at predetermined intervals on each of the keys, and when the specific key is pressed by the user, the key is hidden by the finger among the patches of the pressed key And detects the depth of a non-patch (patch). In addition, when the setting unit 203 determines that a certain key is pressed by the user, the patch (not covered by the finger) among the patches of the pressed key, (Patch) corresponding to one of the patches (patch) is multiplied by the weight of the ratio. If it is determined that the key is depressed, the display 40 displays the determined result (S14).

While the invention has been shown and described with reference to certain preferred embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments. Those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope of the appended claims, The genius will be so self-evident. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

10: camera 20: control module
30: Piano 31: Keyboard
40: Display 201: Detector
202: determination unit 203: setting unit

Claims (10)

A camera for capturing a plurality of color (RGB) images and depth images of a plurality of keys;
A detector for detecting a position and a depth of each key on the captured color (RGB) image and the depth image;
A setting unit configured to set an initial position and a depth of the keyboard using the detected position and depth of each key;
A determining unit for determining whether a key is pressed by comparing an initial position and a depth of the set keyboard with a depth and a depth of the keyboard when the user depresses the keyboard; And
And a display for displaying the determined result,
Wherein the setting unit sets one or more patches at predetermined intervals on each of the keys, and the detecting unit detects a patch that is not obscured by a finger among the set patches when a specific key is pressed by the user, Wherein the depth detection unit detects the depth of the piano by using the RGB-D camera. delete The apparatus according to claim 1,
When a depth of the patch is detected, a patch which is not covered by the finger among the set patches is added to one of the uncovered patches of the entire length of the keyboard And a weight for a length ratio up to a corresponding patch is multiplied. The apparatus according to claim 1,
Wherein the control unit sets the entire musical tone position of the keyboard by finding a key of another octave of a predetermined interval from the pressed key on the basis of the position of one or more keys pressed by the user, . 5. The display of claim 4,
Wherein the scale and the beat of the pressed key are displayed together with the pre-stored score according to the flow of the tune when the depression of the key is determined. 6. The display of claim 5,
And displays whether or not the scale and beat of the depressed key are matched with the scale and beat of the score. Photographing a color (RGB) image and a depth image of a plurality of keys by a camera;
Detecting a position and a depth of each key by the detected color (RGB) image and the depth image;
Setting the initial position and depth of the keyboard using the position and the depth of each key detected by the setting unit;
Determining whether the key is pressed or not by comparing the initial position and depth of the set keyboard with the depth and depth of the keyboard when the user depresses the key; And
Displaying the determined result through a display; Lt; / RTI >
Wherein the step of determining whether the key is pressed comprises:
When a specific key is depressed for one or more patches set at predetermined intervals on the respective keys by the user, the detection unit detects the depth of the patches not covered by the fingers of the pressed patches ) Of the piano is detected. 8. The method of claim 7,
Wherein setting the initial position and depth of the keyboard comprises:
Setting the entire musical scale position of the keyboard by finding a key of another octave of a predetermined interval from the pressed key, based on the position of at least one key pressed by the user; The method of teaching a piano using an RGB-D camera. delete 8. The method of claim 7,
Wherein the step of determining whether the key is pressed comprises:
Wherein when the setting section is pressed by a user with a specific key, a patch which is not covered by a finger among the patches of the pressed key is pressed to one of the uncovered patches of the entire length of the keyboard And multiplying a weight value of a length ratio up to a corresponding patch by a weight of the corresponding pitch.
KR1020140033664A 2014-03-21 2014-03-21 Rgb-d camera based piano education system and method KR101524279B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020140033664A KR101524279B1 (en) 2014-03-21 2014-03-21 Rgb-d camera based piano education system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020140033664A KR101524279B1 (en) 2014-03-21 2014-03-21 Rgb-d camera based piano education system and method

Publications (1)

Publication Number Publication Date
KR101524279B1 true KR101524279B1 (en) 2015-06-01

Family

ID=53490716

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020140033664A KR101524279B1 (en) 2014-03-21 2014-03-21 Rgb-d camera based piano education system and method

Country Status (1)

Country Link
KR (1) KR101524279B1 (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017121049A1 (en) * 2016-01-15 2017-07-20 Findpiano Information Technology (Shanghai) Co., Ltd. Piano system and operating method thereof
CN106981233A (en) * 2016-01-15 2017-07-25 易弹信息科技（上海）有限公司 A kind of intelligent piano large-size screen monitors show tutoring system
CN107464474A (en) * 2017-09-05 2017-12-12 上海智凌信息技术有限公司 A kind of detection method and device of piano playing fingering
CN109035963A (en) * 2018-07-20 2018-12-18 广州市雅迪数码科技有限公司 The method and device of piano long-distance education
CN109344794A (en) * 2018-10-19 2019-02-15 深圳市微蓝智能科技有限公司 A kind of piano performance methods of marking, device and computer storage medium
CN110517555A (en) * 2019-07-19 2019-11-29 森兰信息科技（上海）有限公司 Method, system, storage medium and the device of synchronization class state based on piano
KR20200022251A (en) * 2018-08-22 2020-03-03 이윤정 Method for producing and providing music contents and apparatus therefor
CN110991800A (en) * 2019-10-31 2020-04-10 安徽踏极智能科技有限公司 Piano teaching quality evaluation system
CN111312015A (en) * 2019-10-31 2020-06-19 安徽踏极智能科技有限公司 Transmission system based on piano teaching information
CN111695499A (en) * 2020-06-09 2020-09-22 小叶子（北京）科技有限公司 Key identification method and device, electronic equipment and storage medium
CN112883804A (en) * 2021-01-21 2021-06-01 小叶子（北京）科技有限公司 Error correction method and device for hand motion during musical instrument playing and electronic equipment
KR102374443B1 (en) * 2020-11-19 2022-03-14 임성민 Assist apparatus for education of playing keyboard

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20120034486A (en) * 2010-10-01 2012-04-12 김부전 Keyboard apparatus for music lesson
JP2012220303A (en) * 2011-04-07 2012-11-12 Nec Corp Gripped-article position recognition device, gripped-article position recognition method, and gripped-article position recognition program
KR20130067856A (en) * 2011-12-14 2013-06-25 한국전자통신연구원 Apparatus and method for performing virtual musical instrument on the basis of finger-motion
KR101307732B1 (en) * 2012-05-23 2013-09-11 동신대학교산학협력단 The method for 3d rhythm action games

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20120034486A (en) * 2010-10-01 2012-04-12 김부전 Keyboard apparatus for music lesson
JP2012220303A (en) * 2011-04-07 2012-11-12 Nec Corp Gripped-article position recognition device, gripped-article position recognition method, and gripped-article position recognition program
KR20130067856A (en) * 2011-12-14 2013-06-25 한국전자통신연구원 Apparatus and method for performing virtual musical instrument on the basis of finger-motion
KR101307732B1 (en) * 2012-05-23 2013-09-11 동신대학교산학협력단 The method for 3d rhythm action games

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10950137B2 (en) 2016-01-15 2021-03-16 Sunland Information Technology Co., Ltd. Smart piano system
US10657943B2 (en) 2016-01-15 2020-05-19 Sunland Information Technology Co., Ltd. Systems and methods for calibrating a musical device
US11328618B2 (en) 2016-01-15 2022-05-10 Sunland Information Technology Co., Ltd. Systems and methods for calibrating a musical device
WO2017121049A1 (en) * 2016-01-15 2017-07-20 Findpiano Information Technology (Shanghai) Co., Ltd. Piano system and operating method thereof
CN106981233A (en) * 2016-01-15 2017-07-25 易弹信息科技（上海）有限公司 A kind of intelligent piano large-size screen monitors show tutoring system
US10600399B2 (en) 2016-01-15 2020-03-24 Sunland Information Technology Co., Ltd. Smart piano system
CN106981233B (en) * 2016-01-15 2020-06-12 森兰信息科技（上海）有限公司 Intelligent piano large screen display teaching system
US10636402B2 (en) 2016-01-15 2020-04-28 Sunland Information Technology Co., Ltd. Systems and methods for automatic calibration of musical devices
CN107464474A (en) * 2017-09-05 2017-12-12 上海智凌信息技术有限公司 A kind of detection method and device of piano playing fingering
CN109035963A (en) * 2018-07-20 2018-12-18 广州市雅迪数码科技有限公司 The method and device of piano long-distance education
KR20200022251A (en) * 2018-08-22 2020-03-03 이윤정 Method for producing and providing music contents and apparatus therefor
KR102116311B1 (en) 2018-08-22 2020-05-28 이윤정 Method for producing and providing music contents and apparatus therefor
CN109344794B (en) * 2018-10-19 2022-04-19 深圳市微蓝智能科技有限公司 Piano playing scoring method and device and computer storage medium
CN109344794A (en) * 2018-10-19 2019-02-15 深圳市微蓝智能科技有限公司 A kind of piano performance methods of marking, device and computer storage medium
CN110517555A (en) * 2019-07-19 2019-11-29 森兰信息科技（上海）有限公司 Method, system, storage medium and the device of synchronization class state based on piano
CN110991800A (en) * 2019-10-31 2020-04-10 安徽踏极智能科技有限公司 Piano teaching quality evaluation system
CN111312015A (en) * 2019-10-31 2020-06-19 安徽踏极智能科技有限公司 Transmission system based on piano teaching information
CN111695499B (en) * 2020-06-09 2024-04-16 小叶子（北京）科技有限公司 Key identification method and device, electronic equipment and storage medium
CN111695499A (en) * 2020-06-09 2020-09-22 小叶子（北京）科技有限公司 Key identification method and device, electronic equipment and storage medium
KR102374443B1 (en) * 2020-11-19 2022-03-14 임성민 Assist apparatus for education of playing keyboard
CN112883804A (en) * 2021-01-21 2021-06-01 小叶子（北京）科技有限公司 Error correction method and device for hand motion during musical instrument playing and electronic equipment
CN112883804B (en) * 2021-01-21 2024-03-15 小叶子（北京）科技有限公司 Error correction method and device for musical instrument playing hand actions and electronic equipment

Similar Documents

Publication Publication Date Title
KR101524279B1 (en) Rgb-d camera based piano education system and method
US10825432B2 (en) Smart detecting and feedback system for smart piano
US7582825B2 (en) Method and apparatus for keyboard instrument learning
US10446048B2 (en) Learning assistant system capable of indicating piano fingering
US11087636B2 (en) Lights-guided piano learning and teaching apparatus, and method
CN111259863B (en) Playing hand type detection/display method, medium, piano, terminal and server
CN111695499B (en) Key identification method and device, electronic equipment and storage medium
CN106952510B (en) Pitch calibrator
US8744136B2 (en) Sound reproduction method and device
CN113286183B (en) Method, device and equipment for synchronously displaying performance process and music score and storage medium
JPWO2019049294A1 (en) Code information extraction device, code information extraction method and code information extraction program
US20130284000A1 (en) Music note position detection apparatus, electronic musical instrument, music note position detection method and storage medium
WO2019087331A1 (en) Reference position detection device and reference position detection method
US20130106689A1 (en) Methods of operating systems having optical input devices
CN114677431A (en) Piano fingering identification method and computer readable storage medium
KR20180130432A (en) Method for Performing Korean Charactor Sound of Drum Music Editing and Apparatus for Converting Music Performance File Composd of Korean Charactor Sound of Drum Music Editing Thereof
KR20130058384A (en) Portable drum playing system
JPH06332443A (en) Score recognizing device
Rho et al. Automatic piano tutoring system using consumer-level depth camera
KR100659488B1 (en) the mathod of give marks by voice recognition
WO2023032422A1 (en) Processing method, program, and processing device
JP3229939B2 (en) Media converter
JP3221331U (en) Animation picture type portable playing practice device
JP6782491B2 (en) Musical tone generator, musical tone generator and program
JP2007156370A (en) Electronic musical instrument using real image displayed in space as input index

Legal Events

Date Code Title Description
E701 Decision to grant or registration of patent right
GRNT Written decision to grant
FPAY Annual fee payment

Payment date: 20180406

Year of fee payment: 4