CN113255470A - Multi-mode piano partner training system and method based on hand posture estimation - Google Patents
Multi-mode piano partner training system and method based on hand posture estimation Download PDFInfo
- Publication number
- CN113255470A CN113255470A CN202110492931.8A CN202110492931A CN113255470A CN 113255470 A CN113255470 A CN 113255470A CN 202110492931 A CN202110492931 A CN 202110492931A CN 113255470 A CN113255470 A CN 113255470A
- Authority
- CN
- China
- Prior art keywords
- information
- wrong
- piano
- gesture
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000012549 training Methods 0.000 title claims abstract description 30
- 210000002517 zygapophyseal joint Anatomy 0.000 claims abstract description 45
- 230000008569 process Effects 0.000 claims abstract description 34
- 239000012634 fragment Substances 0.000 claims abstract description 15
- 238000012545 processing Methods 0.000 claims description 22
- 238000012937 correction Methods 0.000 claims description 10
- 238000011156 evaluation Methods 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 3
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 238000013480 data collection Methods 0.000 claims description 3
- 230000008030 elimination Effects 0.000 claims description 3
- 238000003379 elimination reaction Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 210000000056 organ Anatomy 0.000 claims description 3
- 239000013598 vector Substances 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 4
- 239000000284 extract Substances 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 210000001145 finger joint Anatomy 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000002478 hand joint Anatomy 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Acoustics & Sound (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Signal Processing (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Electrophonic Musical Instruments (AREA)
- User Interface Of Digital Computer (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a multi-mode piano partner training system and method based on hand posture estimation. The system comprises: the data acquisition module is used for acquiring gesture information and musical note information in the playing process; the data identification module is used for identifying the facet joint points in the gesture information according to the musical instrument tone information and a preset algorithm and sending the facet joint points to the data comparison module; the data comparison module is used for comparing the facet joint recognition result with a standard database to obtain wrong gesture information and wrong piano tone fragment information in the playing process; and the result display module is used for marking the wrong gesture information and the wrong music fragment information on the music score and feeding back the wrong gesture information and the wrong music fragment information to the user in a multi-mode form. The invention extracts and identifies the gesture characteristics in multiple aspects through multiple algorithms, thereby effectively extracting the time characteristic information and the space characteristic information of the video frame, greatly reducing the computational power consumption by receiving the sound through the microphone, and improving the piano partner training effect through facet joint positioning.
Description
Technical Field
The invention relates to the technical field of piano partner training, in particular to a multi-mode piano partner training system and method based on hand posture estimation.
Background
The piano practice plays an extremely important role in improving the understanding and mastering of the piano by students. Traditional piano practice is often repeated by students alone, but no scientific instruction usually easily results in wrong muscle memory, resulting in a great deal of time wasted for correcting errors in class returns. Therefore, some piano intelligent training products appear in the market, and aim to solve the problems encountered by piano learners during autonomous training. However, the products in the same field on the market only pay attention to whether the played music is correct or not, but neglect the extremely important evaluation of the playing gesture.
The existing intelligent piano partner training system uses an audio recognition technology to compare the playing audio of a user with a music score and give the evaluation of rhythm and intonation. The disadvantages of this approach are:
1. the piano tones are identified only by using an audio identification technology, and the requirement on environmental tones is high. If the user is in a noisy use environment, the high-decibel environmental sound will greatly reduce the accuracy of audio analysis, resulting in evaluation errors.
2. Evaluation mode was single rigid. Using audio analysis can only confirm that the user's notes performed correctly, but not whether the user's style performed correctly, e.g., the user may use an incorrect hand to play the full song or play a recording of the performance to get rating feedback.
Therefore, the existing intelligent piano training system is not comprehensive in training unmanned guidance and training service evaluation standard and is a technical problem to be solved urgently.
Disclosure of Invention
In order to solve the technical problem that the practice is not fully guided by people and the practice service evaluation standard is not comprehensive, the invention provides a multi-mode piano practice accompanying system and method based on hand posture estimation, and the system and method are used for improving the autonomous practice efficiency of a user.
In order to achieve the purpose, the multi-mode piano training accompanying system based on hand posture estimation comprises a data acquisition module, a data identification module, a data comparison module and a result display module;
the data acquisition module is used for acquiring gesture information and musical note information in the playing process, aligning the two groups of information and sending the aligned information to the data identification module;
the data identification module is used for identifying the facet joint points in the gesture information according to the musical instrument tone information and a preset algorithm to obtain facet joint point identification results, and sending the facet joint point identification results to the data comparison module;
the data comparison module is used for comparing the facet joint recognition result with a standard database to obtain wrong gesture information and wrong musical note fragment information in the playing process, and sending the wrong gesture information and the wrong musical note fragment information to the result display module;
and the result display module is used for marking the wrong gesture information and the wrong music piece information on the music score to obtain a marking result and feeding the marking result back to the user in a multi-mode form.
Preferably, the standard database is composed of a mass music library and a fingering library.
Preferably, the data acquisition module comprises: the wide-angle camera is used for collecting gesture information in the playing process, and the microphone array is used for collecting organ tone information in the playing process.
Preferably, the data identification module and the data comparison module are located in a cloud background, and the preset algorithm includes: gesture recognition algorithms, audio recognition algorithms and image processing algorithms.
Preferably, the gesture recognition algorithm is specifically: a ResNet residual network and a dual-channel convolutional neural network are adopted, the content of rapid change in gesture information is concerned, the position and direction vectors of all the facet joint points in a predicted image capture motion information, the positions of the facet joint points are used for generating a heat map, and the heat map is used as a signal supervision training process to realize facet joint point identification and semantic understanding.
Preferably, the audio recognition algorithm is specifically: based on the microphone array, the sound information is subjected to filtering processing and windowing processing, a sound interval is determined, and the hand position during playing is assisted to be judged.
Preferably, the image processing algorithm is specifically: distortion elimination processing is carried out firstly, then binarization processing is carried out on the gesture image, and finally noise is removed by adopting image opening operation.
Preferably, the multimodal form refers to: marking the wrong gesture information and the wrong music segment information on the music score, repeatedly checking the wrong segment in the wrong position 10s by clicking the wrong label, displaying the correct gesture and the correct music at the same time to obtain a correction evaluation prompt, and displaying or not displaying the label of the facet joint in the playing process by selecting a display mode.
Preferably, the display mode includes: the device comprises an advanced mode and a common mode, wherein the advanced mode displays the facet joint point marks in the playing process, and the common mode does not display the facet joint point marks in the playing process.
In addition, in order to achieve the above object, the present invention also provides a multi-modal piano training method based on hand pose estimation, comprising the steps of:
the gesture information and the piano tone information in the playing process are collected through the data collection module, and the two sets of information are aligned and then sent to the data identification module;
identifying the facet joint points in the gesture information through the data identification module according to the musical instrument tone information and a preset algorithm to obtain facet joint point identification results, and sending the facet joint point identification results to the data comparison module;
comparing the facet joint recognition result with a standard database through the data comparison module to obtain wrong gesture information and wrong piano tone fragment information in the playing process; sending the wrong gesture information and the wrong musical instrument sound fragment information to the result display module;
and marking the wrong gesture information and the wrong music piece information on the music score through the result display module to obtain a marking result, and feeding the marking result back to the user in a multi-mode form.
Preferably, the preset algorithm includes a gesture recognition algorithm, an audio recognition algorithm and an image processing algorithm.
The invention has the beneficial effects that: the invention processes continuous video frames through a gesture recognition algorithm and an image processing algorithm, and performs feature extraction from three aspects of hand contours, finger joint point distribution structures and hand motion characteristics, thereby effectively extracting time feature information and space feature information of the video frames, receiving sound by a microphone, primarily judging the range of keys by an audio recognition algorithm, reducing the key screening range, greatly reducing the computational power consumption, improving the speed and accuracy of hand positioning, and improving the piano partner training effect through facet joint positioning.
Drawings
The invention will be further described with reference to the accompanying drawings and examples, in which:
FIG. 1 is a block diagram of a multi-modal piano training system based on hand pose estimation according to the present invention;
FIG. 2 is a software interface diagram of a multi-modal piano rehearsal system based on hand pose estimation according to the present invention;
FIG. 3 is a diagram of an exercise initiation interface of the present invention;
FIG. 4 is a schematic view of facet joint identification according to the present invention;
FIG. 5 is a schematic view of the present invention for sound correction;
FIG. 6 is a flow chart of a multi-modal piano partner training method based on hand pose estimation in accordance with the present invention.
Detailed Description
For a more clear understanding of the technical features, objects and effects of the present invention, embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a block diagram illustrating a multi-modal piano training partner system based on hand pose estimation according to the present invention;
in this embodiment, a multimode piano training mate system based on hand gesture is estimated includes: the device comprises a data acquisition module, a data identification module, a data comparison module and a result display module;
the data acquisition module, the data identification module, the data comparison module and the result display module are sequentially connected;
the data acquisition module is used for acquiring gesture information and musical note information in the playing process, aligning the two groups of information and sending the aligned information to the data identification module;
the data identification module is used for identifying the facet joint points in the gesture information according to the musical instrument tone information and a preset algorithm to obtain facet joint point identification results, and sending the facet joint point identification results to the data comparison module;
the data comparison module is used for comparing the facet joint recognition result with a standard database to obtain wrong gesture information and wrong musical note fragment information in the playing process, and sending the wrong gesture information and the wrong musical note fragment information to the result display module;
and the result display module is used for marking the wrong gesture information and the wrong music piece information on the music score to obtain a marking result and feeding the marking result back to the user in a multi-mode form.
As an optional implementation, the data acquisition module includes: the wide-angle camera is used for collecting gesture information in the playing process, and the microphone array is used for collecting organ tone information in the playing process.
In this embodiment, the data recognition module and the data comparison module are located in a cloud background, and the preset algorithm includes a gesture recognition algorithm, an audio recognition algorithm and an image processing algorithm.
In this embodiment, the gesture recognition algorithm specifically includes: a ResNet residual network and a dual-channel convolutional neural network are adopted, the content of rapid change in gesture information is concerned, the position and direction vectors of all small joint points in images are predicted, motion information is captured, the positions of the small joint points are used for generating heat maps, the heat maps are used as a signal supervision training process, and small joint point identification and semantic understanding are achieved.
In this embodiment, the audio recognition algorithm specifically includes: based on the microphone array, the sound information is subjected to filtering processing and windowing processing, a sound interval is determined, and the hand position during playing is assisted to be judged.
In this embodiment, the image processing algorithm specifically includes: distortion elimination processing is carried out firstly, then binarization processing is carried out on the gesture image, and finally noise is removed by adopting image opening operation.
Referring to fig. 2 and 3, on the software interface, the annotated error information is displayed in a multi-modal form: marking the wrong gesture information and the wrong music segment information on the music score, repeatedly checking the wrong segment in the wrong position 10s by clicking the wrong label, displaying the correct gesture and the correct music at the same time to obtain a correction evaluation prompt, and displaying or not displaying the label of the facet joint in the playing process by selecting a display mode.
In this embodiment, the display mode includes: the device comprises an advanced mode and a common mode, wherein the advanced mode displays the facet joint point marks in the playing process, and the common mode does not display the facet joint point marks in the playing process.
When the user clicks the advanced mode, the facet joint labels in the playing process are displayed on the interface.
In the embodiment, functionally, the multi-mode piano training system based on hand posture estimation is divided into a fingering error correction module and a piano tone error correction module.
A fingering error correction module: supported by facet joint recognition technology: the wide-angle camera at the top of the piano is used for capturing different gestures, marks of each finger joint are acquired, a user refers to the graph 4, then whether the action of a piano exerciser is standard or not is judged according to the neural network model, when wrong gestures are captured, the system can make judgment, whether each gesture of the piano exerciser is correct or not can be accurately obtained after exercise is finished, a piano exercising report is generated, the credibility and the accuracy of the piano exercising report are greatly increased by combining with the microphone receiving system, and the purpose of helping the piano exerciser to improve the level is achieved.
And a musical instrument tone error correction module: the sound of each angle is acquired by receiving sound through a microphone array arranged around the piano, different musical scale tones are acquired through computer analysis and recognition, whether musical scale errors occur during the playing of a player is judged through comparison of music scores in a database, and error segments are automatically marked, and the method is referred to fig. 5.
The key identification technology is used as an intersection point of two modules and aims to realize identification and calibration of 88 keys of the piano, and the technology is mainly divided into two steps of image preprocessing and key positioning. As a characteristic function in the invention, the key recognition function can position each key to the maximum extent, so that the system can recognize the piano tones more accurately, and can also be matched with gesture recognition to mark the coordinates of the keys, thereby correcting wrong hand shapes of beginners better.
The invention relies on a large amount of gesture data sets to train the gesture model, greatly improving the accuracy and speed of model recognition; and the piano tone error correction and the fingering error correction are finished by means of the massive song library and the fingering library.
In addition, referring to fig. 6, based on the multi-modal piano training accompanying system based on hand posture estimation, the present embodiment further provides a multi-modal piano training accompanying method based on hand posture estimation, including the following steps:
s1, acquiring gesture information and piano tone information in the playing process through the data acquisition module, aligning the two sets of information and sending the aligned information to the data identification module;
s2, identifying the facet joint points in the gesture information through the data identification module according to the piano tone information and a preset algorithm to obtain facet joint point identification results, and sending the facet joint point identification results to the data comparison module;
s3, comparing the facet joint recognition result with a standard database through the data comparison module to obtain wrong gesture information and wrong sound fragment information in the playing process, and sending the wrong gesture information and the wrong sound fragment information to the result display module;
and S4, marking the wrong gesture information and the wrong music piece information on a music score through the result display module to obtain a marking result, and feeding the marking result back to the user in a multi-mode form.
According to the invention, continuous video frames are processed through a gesture recognition algorithm and an image processing algorithm, and feature extraction is carried out from three aspects of hand contours, hand joint distribution structures and hand motion characteristics, so that time feature information and space feature information of the video frames are effectively extracted, the microphone is used for receiving sound, the audio recognition algorithm is used for carrying out preliminary judgment on the sound range where the keys are located, the key screening range is narrowed, the computational power consumption is greatly reduced, the speed and accuracy of hand positioning are improved, and the piano partner training effect is improved through facet joint positioning.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (10)
1. A multi-modal piano training system based on hand pose estimation, the multi-modal piano training system comprising: the device comprises a data acquisition module, a data identification module, a data comparison module and a result display module;
the data acquisition module is used for acquiring gesture information and musical note information in the playing process, aligning the two sets of information and sending the aligned information to the data identification module;
the data identification module is used for identifying the facet joint points in the gesture information according to the musical instrument tone information and a preset algorithm to obtain facet joint point identification results, and sending the facet joint point identification results to the data comparison module;
the data comparison module is used for comparing the facet joint recognition result with a standard database to obtain wrong gesture information and wrong musical note fragment information in the playing process, and sending the wrong gesture information and the wrong musical note fragment information to the result display module;
and the result display module is used for marking the wrong gesture information and the wrong music piece information on a music score to obtain a marking result and feeding the marking result back to a user in a multi-mode form.
2. The multi-modal piano sparring system based on hand pose estimation of claim 1, wherein said data collection module comprises: the wide-angle camera is used for collecting gesture information in the playing process, and the microphone array is used for collecting organ tone information in the playing process.
3. The multi-modal piano training accompanying system based on hand posture estimation as claimed in claim 1, wherein said data recognition module and said data comparison module are located in a cloud background, and said predetermined algorithm comprises: gesture recognition algorithms, audio recognition algorithms and image processing algorithms.
4. The multi-modal piano sparring system based on hand pose estimation of claim 3, wherein said gesture recognition algorithm is specifically: a ResNet residual network and a dual-channel convolutional neural network are adopted, the content of rapid change in gesture information is concerned, the position and direction vectors of all small joint points in images are predicted, motion information is captured, the positions of the small joint points are used for generating heat maps, the heat maps are used as a signal supervision training process, and small joint point identification and semantic understanding are achieved.
5. The multi-modal piano sparring system based on hand pose estimation of claim 3, wherein said audio recognition algorithm is specifically: based on the microphone array, the sound information is subjected to filtering processing and windowing processing, a sound interval is determined, and the hand position during playing is assisted to be judged.
6. The multi-modal piano sparring system based on hand pose estimation of claim 3, wherein said image processing algorithm is specifically: distortion elimination processing is carried out firstly, then binarization processing is carried out on the gesture image, and finally noise is removed by adopting image opening operation.
7. The multi-modal piano sparring system based on hand pose estimation of claim 1, wherein said multi-modal form is: marking the wrong gesture information and the wrong music segment information on the music score, repeatedly checking the wrong segment in the wrong position 10s by clicking the wrong label, displaying the correct gesture and the correct music at the same time to obtain a correction evaluation prompt, and displaying or not displaying the label of the facet joint in the playing process by selecting a display mode.
8. The multi-modal piano sparring system based on hand pose estimation of claim 7, wherein said display modes comprise: the device comprises an advanced mode and a common mode, wherein the advanced mode displays the facet joint point marks in the playing process, and the common mode does not display the facet joint point marks in the playing process.
9. A multi-mode piano partner training method based on hand posture estimation is characterized by comprising the following steps:
the gesture information and the piano tone information in the playing process are collected through the data collection module, and the two sets of information are aligned and then sent to the data identification module;
identifying the facet joint points in the gesture information through the data identification module according to the musical instrument tone information and a preset algorithm to obtain facet joint point identification results, and sending the facet joint point identification results to the data comparison module;
comparing the facet joint recognition result with a standard database through the data comparison module to obtain wrong gesture information and wrong musical note fragment information in the playing process, and sending the wrong gesture information and the wrong musical note fragment information to the result display module;
and marking the wrong gesture information and the wrong music piece information on the music score through the result display module to obtain a marking result, and feeding the marking result back to the user in a multi-mode form.
10. The multi-modal piano sparring method based on hand pose estimation of claim 9, wherein said predetermined algorithms comprise gesture recognition algorithms, audio recognition algorithms and image processing algorithms.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110492931.8A CN113255470B (en) | 2021-05-06 | 2021-05-06 | Multi-mode piano accompany training system and method based on hand gesture estimation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110492931.8A CN113255470B (en) | 2021-05-06 | 2021-05-06 | Multi-mode piano accompany training system and method based on hand gesture estimation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113255470A true CN113255470A (en) | 2021-08-13 |
| CN113255470B CN113255470B (en) | 2024-04-02 |
Family
ID=77223830
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110492931.8A Active CN113255470B (en) | 2021-05-06 | 2021-05-06 | Multi-mode piano accompany training system and method based on hand gesture estimation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113255470B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117863175A (en) * | 2023-12-25 | 2024-04-12 | 之江实验室 | Offline evaluation system and method for playing piano robot |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017037342A1 (en) * | 2015-09-04 | 2017-03-09 | Pianorobot Oy | System for teaching a user to play a musical instrument from musical notation via virtual exercises and a method thereof |
| CN108074439A (en) * | 2016-11-18 | 2018-05-25 | 北京酷我科技有限公司 | A kind of piano training mate equipment and piano |
| CN110210547A (en) * | 2019-05-27 | 2019-09-06 | 南京航空航天大学 | Piano playing gesture identification method based on inertia gloves |
| CN111191627A (en) * | 2020-01-06 | 2020-05-22 | 浙江工业大学 | Method for improving accuracy of dynamic gesture motion recognition under multiple viewpoints |
| CN111259863A (en) * | 2020-03-03 | 2020-06-09 | 森兰信息科技（上海）有限公司 | Method for detecting/displaying playing hand type, medium, piano, terminal and server |
-
2021
- 2021-05-06 CN CN202110492931.8A patent/CN113255470B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017037342A1 (en) * | 2015-09-04 | 2017-03-09 | Pianorobot Oy | System for teaching a user to play a musical instrument from musical notation via virtual exercises and a method thereof |
| CN108074439A (en) * | 2016-11-18 | 2018-05-25 | 北京酷我科技有限公司 | A kind of piano training mate equipment and piano |
| CN110210547A (en) * | 2019-05-27 | 2019-09-06 | 南京航空航天大学 | Piano playing gesture identification method based on inertia gloves |
| CN111191627A (en) * | 2020-01-06 | 2020-05-22 | 浙江工业大学 | Method for improving accuracy of dynamic gesture motion recognition under multiple viewpoints |
| CN111259863A (en) * | 2020-03-03 | 2020-06-09 | 森兰信息科技（上海）有限公司 | Method for detecting/displaying playing hand type, medium, piano, terminal and server |
Non-Patent Citations (1)
| Title |
|---|
| 黄承承;: "钢琴演奏自动评估系统开发与设计", 自动化技术与应用, no. 09 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117863175A (en) * | 2023-12-25 | 2024-04-12 | 之江实验室 | Offline evaluation system and method for playing piano robot |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113255470B (en) | 2024-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106340286B (en) | Universal real-time musical instrument playing evaluation system | |
| CN106485984B (en) | Intelligent teaching method and device for piano | |
| Fan et al. | Example-based automatic music-driven conventional dance motion synthesis | |
| CN102110399B (en) | A kind of assist the method for explanation, device and system thereof | |
| Johnson et al. | Detecting hand posture in piano playing using depth data | |
| CN109669661A (en) | A kind of control method and electronic equipment of dictation progress | |
| JP2000502479A (en) | Online Handwritten Character Recognition Method and Apparatus Based on Feature Vectors Using Aggregation Observation Extracted from Time Series Frames | |
| CN110490173B (en) | Intelligent action scoring system based on 3D somatosensory model | |
| WO2022052941A1 (en) | Intelligent identification method and system for giving assistance with piano teaching, and intelligent piano training method and system | |
| CN112614489A (en) | User pronunciation accuracy evaluation method and device and electronic equipment | |
| CN113052138A (en) | Intelligent contrast correction method for dance and movement actions | |
| CN110310528A (en) | A kind of paper cloud interaction language teaching system and method | |
| CN112488047A (en) | Piano fingering intelligent identification method | |
| CN113674565A (en) | Teaching system and method for piano teaching | |
| CN105513580A (en) | Keyboard musical instrument playing note identification system based on auxiliary camera | |
| Lee et al. | Observing pianist accuracy and form with computer vision | |
| CN111079501B (en) | Character recognition method and electronic equipment | |
| CN111968675A (en) | Stringed instrument note comparison system based on hand recognition and use method thereof | |
| CN113255470B (en) | Multi-mode piano accompany training system and method based on hand gesture estimation | |
| CN117151548A (en) | Music online learning method and system based on hand motion judgment | |
| CN116386424A (en) | Method, device and computer readable storage medium for music teaching | |
| CN114170868A (en) | Intelligent piano training method and system | |
| Kerdvibulvech et al. | Real-time guitar chord estimation by stereo cameras for supporting guitarists | |
| CN114677431A (en) | Piano fingering identification method and computer readable storage medium | |
| CN113327482A (en) | String instrument playing teaching method and system based on video monitoring |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |