CN113255470B - Multi-mode piano accompany training system and method based on hand gesture estimation - Google Patents
Multi-mode piano accompany training system and method based on hand gesture estimation Download PDFInfo
- Publication number
- CN113255470B CN113255470B CN202110492931.8A CN202110492931A CN113255470B CN 113255470 B CN113255470 B CN 113255470B CN 202110492931 A CN202110492931 A CN 202110492931A CN 113255470 B CN113255470 B CN 113255470B
- Authority
- CN
- China
- Prior art keywords
- information
- musical instrument
- gesture
- wrong
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000012549 training Methods 0.000 title claims abstract description 34
- 239000012634 fragment Substances 0.000 claims abstract description 22
- 238000001228 spectrum Methods 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims description 18
- 238000012937 correction Methods 0.000 claims description 11
- 238000011156 evaluation Methods 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 4
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 239000013598 vector Substances 0.000 claims description 3
- 238000013480 data collection Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 4
- 210000002517 zygapophyseal joint Anatomy 0.000 abstract description 4
- 238000004364 calculation method Methods 0.000 abstract description 3
- 238000000605 extraction Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 210000001145 finger joint Anatomy 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 210000002478 hand joint Anatomy 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000033764 rhythmic process Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Acoustics & Sound (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Signal Processing (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Electrophonic Musical Instruments (AREA)
- User Interface Of Digital Computer (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a multimode piano partner training system and method based on hand gesture estimation. The system comprises: the data acquisition module acquires gesture information and musical instrument information in the playing process; the data identification module is used for identifying small nodes in the gesture information according to the musical instrument sound information and a preset algorithm and sending the small nodes to the data comparison module; the data comparison module is used for comparing the small joint point identification result with a standard database to obtain error gesture information and error musical instrument sound fragment information in the playing process; and the result display module is used for marking the wrong gesture information and the wrong musical instrument sound fragment information on the musical instrument spectrum and feeding back the wrong gesture information and the wrong musical instrument sound fragment information to a user in a multi-mode. According to the invention, gesture feature extraction and recognition are carried out through multiple algorithms in multiple aspects, so that time feature information and space feature information of a video frame are effectively extracted, meanwhile, calculation power consumption is greatly reduced by means of microphone reception, and piano training effect is improved through facet joint positioning.
Description
Technical Field
The invention relates to the technical field of piano cosmesis, in particular to a multimode piano cosmesis system and method based on hand gesture estimation.
Background
The piano practice has extremely important functions in improving the knowledge and grasp of students on the piano. Traditional piano practice is often repeated by students alone, but without scientific guidance, wrong muscle memory is often easily formed, resulting in a great deal of time wasted for correcting errors in lessons. For this reason, some piano intelligence accompany the training products have appeared in the market, aim at solving the problem that the piano learner encountered when independently practising. However, these products in the same field on the market tend to only pay attention to whether the played musical instrument is correct or not, but neglect the very important evaluation of the playing gesture.
The existing intelligent piano partner training system uses an audio recognition technology to compare the performance audio of a user with a music score and give out evaluation of rhythm and tone. The drawbacks of this approach are:
1. the method only uses the audio recognition technology to recognize the musical instrument sound, and has higher requirements on the environmental sound. If the user is in a noisy use environment, the high decibel ambient sound will greatly reduce the accuracy of the audio analysis, causing an evaluation error.
2. The evaluation mode is single and dead. The use of audio analysis only confirms that the user's note is playing correctly, but does not confirm whether the user's manner of performance is correct, for example, the user may use an incorrect hand-type performance music score or play a performance recording to get evaluation feedback.
Therefore, the existing intelligent piano training accompanying system is used for training unmanned guidance, and training accompanying service evaluation standards are incomplete, so that the technical problem to be solved is urgent.
Disclosure of Invention
The invention provides a multimode piano partner training system and method based on hand gesture estimation, which are used for improving the autonomous piano training efficiency of a user.
In order to achieve the above purpose, the multimode piano training partner system based on hand gesture estimation of the invention comprises a data acquisition module, a data identification module, a data comparison module and a result display module;
the data acquisition module is used for acquiring gesture information and musical instrument information in the playing process, aligning the two groups of information and then sending the two groups of information to the data identification module;
the data identification module is used for identifying small joint points in the gesture information according to the musical instrument sound information and a preset algorithm, obtaining small joint point identification results and sending the small joint point identification results to the data comparison module;
the data comparison module is used for comparing the small joint point identification result with a standard database to obtain error gesture information and error musical instrument sound fragment information in the playing process, and sending the error gesture information and the error musical instrument sound fragment information to the result display module;
and the result display module is used for marking the wrong gesture information and the wrong musical instrument sound fragment information on a musical instrument spectrum, obtaining a marking result and feeding back the marking result to a user in a multi-mode.
Preferably, the standard database is composed of a mass library and a fingering library.
Preferably, the data acquisition module comprises: the device comprises a wide-angle camera and a microphone array, wherein the wide-angle camera is used for collecting gesture information in the playing process, and the microphone array is used for collecting musical instrument sound information in the playing process.
Preferably, the data identification module and the data comparison module are located in a cloud background, and the preset algorithm includes: gesture recognition algorithms, audio recognition algorithms, and image processing algorithms.
Preferably, the gesture recognition algorithm specifically comprises: the ResNet residual network and the double-channel convolutional neural network are adopted, the content of rapid change in gesture information is focused, the position and direction vectors of all small nodes in the image are predicted to capture motion information, the positions of the small nodes are used for generating a heat map, and the heat map is used as a signal supervision training process to realize small joint point identification and semantic understanding.
Preferably, the audio recognition algorithm specifically comprises: based on the microphone array, the musical instrument information is subjected to filtering processing and windowing processing, a sound interval is determined, and the hand position during playing is assisted in judgment.
Preferably, the image processing algorithm specifically comprises: firstly, carrying out de-distortion treatment, then carrying out binarization treatment on the gesture image, and finally adopting image open operation to remove noise points.
Preferably, the multi-modal form refers to: marking the wrong gesture information and the wrong musical instrument sound segment information on a musical instrument spectrum, repeatedly checking the wrong segment in the wrong position 10s by clicking the wrong marking, simultaneously displaying the correct gesture and the correct musical instrument sound, obtaining a correction evaluation prompt, and displaying or not displaying the marking of the small joint point in the playing process by selecting a display mode.
Preferably, the display mode includes: the high-level mode and the common mode, wherein the high-level mode displays the small joint point marks in the playing process, and the common mode does not display the small joint point marks in the playing process.
In addition, in order to achieve the above object, the present invention also provides a multimode piano training method based on hand gesture estimation, comprising the following steps:
the gesture information and the musical instrument information in the playing process are collected through the data collection module, and the two groups of information are aligned and then sent to the data identification module;
identifying small joint points in the gesture information according to the musical instrument sound information and a preset algorithm through the data identification module, obtaining small joint point identification results, and sending the small joint point identification results to the data comparison module;
comparing the small joint point identification result with a standard database through the data comparison module to obtain error gesture information and error musical instrument sound fragment information in the playing process; the error gesture information and the error musical instrument sound fragment information are sent to the result display module;
and marking the wrong gesture information and the wrong musical instrument sound fragment information on a musical instrument spectrum through the result display module to obtain a marking result, and feeding back the marking result to a user through a multi-mode form.
Preferably, the preset algorithm comprises a gesture recognition algorithm, an audio recognition algorithm and an image processing algorithm.
The beneficial effects of the invention are as follows: according to the invention, continuous video frames are processed through a gesture recognition algorithm and an image processing algorithm, and feature extraction is carried out from three aspects of hand outline, finger joint point distribution structure and hand motion characteristics, so that time feature information and space feature information of the video frames are effectively extracted, the voice domain where the keys are positioned is primarily judged through a microphone radio and an audio recognition algorithm, the key screening range is shortened, the calculation power consumption is greatly reduced, the hand positioning speed and accuracy are improved, and the piano training effect is improved through small joint positioning.
Drawings
The invention will be further described with reference to the accompanying drawings and examples, in which:
fig. 1 is a block diagram of a multi-modal piano training system based on hand pose estimation according to the present invention;
FIG. 2 is a software interface diagram of a multimodal piano training system based on hand pose estimation of the present invention;
FIG. 3 is a diagram of a start exercise interface of the present invention;
FIG. 4 is a schematic illustration of the present invention's facet joint recognition;
FIG. 5 is a schematic diagram of tone correction according to the present invention;
fig. 6 is a flowchart of a multi-modal piano training method based on hand pose estimation according to the present invention.
Detailed Description
For a clearer understanding of technical features, objects and effects of the present invention, a detailed description of embodiments of the present invention will be made with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a block diagram of a multi-mode piano training system based on hand gesture estimation according to the present invention;
in this embodiment, a multimode piano training partner system based on hand gesture estimation includes: the device comprises a data acquisition module, a data identification module, a data comparison module and a result display module;
the data acquisition module, the data identification module, the data comparison module and the result display module are sequentially connected;
the data acquisition module is used for acquiring gesture information and musical instrument information in the playing process, aligning the two groups of information and then sending the two groups of information to the data identification module;
the data identification module is used for identifying small joint points in the gesture information according to the musical instrument sound information and a preset algorithm, obtaining small joint point identification results and sending the small joint point identification results to the data comparison module;
the data comparison module is used for comparing the small joint point identification result with a standard database to obtain error gesture information and error musical instrument sound fragment information in the playing process, and sending the error gesture information and the error musical instrument sound fragment information to the result display module;
and the result display module is used for marking the wrong gesture information and the wrong musical instrument sound fragment information on a musical instrument spectrum, obtaining a marking result and feeding back the marking result to a user in a multi-mode.
As an alternative embodiment, the data acquisition module includes: the device comprises a wide-angle camera and a microphone array, wherein the wide-angle camera is used for collecting gesture information in the playing process, and the microphone array is used for collecting musical instrument sound information in the playing process.
In this embodiment, the data recognition module and the data comparison module are located in the cloud background, and the preset algorithm includes a gesture recognition algorithm, an audio recognition algorithm and an image processing algorithm.
In this embodiment, the gesture recognition algorithm specifically includes: the ResNet residual network and the double-channel convolutional neural network are adopted, the content of rapid change in gesture information is focused, the position and direction vectors of all small nodes in the image are predicted, motion information is captured, the positions of the small nodes are used for generating a heat map, and the heat map is used as a signal supervision training process to realize small node identification and semantic understanding.
In this embodiment, the audio recognition algorithm specifically includes: based on the microphone array, the musical instrument information is subjected to filtering processing and windowing processing, a sound interval is determined, and the hand position during playing is assisted in judgment.
In this embodiment, the image processing algorithm specifically includes: firstly, carrying out de-distortion treatment, then carrying out binarization treatment on the gesture image, and finally adopting image open operation to remove noise points.
Referring to fig. 2 and 3, error information of the annotation is displayed in a multi-modal form on the software interface: marking the wrong gesture information and the wrong musical instrument sound segment information on a musical instrument spectrum, repeatedly checking the wrong segment in the wrong position 10s by clicking the wrong marking, simultaneously displaying the correct gesture and the correct musical instrument sound, obtaining a correction evaluation prompt, and displaying or not displaying the marking of the small joint point in the playing process by selecting a display mode.
In this embodiment, the display mode includes: the high-level mode and the common mode, wherein the high-level mode displays the small joint point marks in the playing process, and the common mode does not display the small joint point marks in the playing process.
When the user clicks the advanced mode, the facet joint labels in the playing process are displayed on the interface.
In this embodiment, functionally, the multimode piano training system based on hand gesture estimation of the present invention is divided into a fingering error correction module and a musical instrument sound error correction module.
Finger error correction module: supported by facet joint recognition technology: the method comprises the steps of capturing different gestures by using a wide-angle camera at the top of a piano, acquiring the marks of each finger joint, referring to fig. 4, judging whether the action of a piano exerciser is standard or not according to a neural network model, judging by a system when the wrong gestures are captured, accurately judging whether each gesture of a piano exerciser is correct or not after the exercise is finished, generating a piano exercise report, and combining the piano exercise report with a microphone radio system to greatly increase the credibility and accuracy of the piano exercise report, thereby achieving the purpose of helping the piano exerciser to improve the level.
And the musical instrument sound error correction module: the sound of each angle is obtained through the sound receiving of the microphone array arranged around the piano, different musical scale tones are obtained through the analysis and recognition of the computer, whether the player plays a musical scale error is judged through the comparison of the musical scores in the database, and error fragments are automatically marked, and refer to fig. 5.
The key identification technology is used as the intersection point of two large modules and aims at realizing the identification and calibration of the 88 keys of the piano, and the technology mainly comprises two steps of image preprocessing and key positioning. As the characteristic function in the invention, the key recognition function can position each key to the greatest extent, so that the system can recognize the musical instrument more accurately, can be matched with gesture recognition, marks the key coordinates and corrects the wrong hand shape of a beginner better.
According to the gesture model training method, gesture model training is carried out by means of a large number of gesture data sets, and the accuracy and speed of model identification are greatly improved; and completing the tone error correction and fingering error correction by means of a mass library of music and fingering library.
In addition, referring to fig. 6, based on the above-mentioned multimode piano training system based on hand gesture estimation, the embodiment further provides a multimode piano training method based on hand gesture estimation, which comprises the following steps:
s1, acquiring gesture information and musical instrument information in a playing process through a data acquisition module, aligning the two groups of information and then sending the two groups of information to a data identification module;
s2, recognizing a small joint point in the gesture information through the data recognition module according to the musical instrument sound information and a preset algorithm, obtaining a small joint point recognition result, and sending the small joint point recognition result to the data comparison module;
s3, comparing the small joint point identification result with a standard database through the data comparison module to obtain error gesture information and error musical instrument sound fragment information in the playing process, and sending the error gesture information and the error musical instrument sound fragment information to the result display module;
and S4, marking the wrong gesture information and the wrong musical instrument sound fragment information on a musical instrument spectrum through the result display module to obtain a marking result, and feeding back the marking result to a user through a multi-mode form.
According to the invention, continuous video frames are processed through a gesture recognition algorithm and an image processing algorithm, and feature extraction is carried out from three aspects of hand outline, hand joint distribution structure and hand motion characteristics, so that time feature information and space feature information of the video frames are effectively extracted, the voice domain where the keys are positioned is primarily judged through a microphone radio and an audio recognition algorithm, the key screening range is reduced, the calculation power consumption is greatly reduced, the hand positioning speed and accuracy are improved, and the piano training effect is improved through small joint positioning.
The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.
Claims (5)
1. Multimode piano training partner system based on hand gesture estimation, characterized in that the multimode piano training partner system comprises: the device comprises a data acquisition module, a data identification module, a data comparison module and a result display module;
the data acquisition module is used for acquiring gesture information and musical instrument information in the playing process, aligning the two groups of information and then sending the two groups of information to the data identification module;
the data identification module is used for identifying small joint points in the gesture information according to the musical instrument sound information and a preset algorithm, obtaining small joint point identification results and sending the small joint point identification results to the data comparison module;
the data comparison module is used for comparing the small joint point identification result with a standard database to obtain error gesture information and error musical instrument sound fragment information in the playing process, and sending the error gesture information and the error musical instrument sound fragment information to the result display module;
the result display module is used for marking the wrong gesture information and wrong musical instrument sound fragment information on a musical instrument spectrum to obtain a marking result, and feeding back the marking result to a user in a multi-mode;
the data identification module and the data comparison module are located in the cloud background, and the preset algorithm comprises: gesture recognition algorithms, audio recognition algorithms, and image processing algorithms;
the audio recognition algorithm specifically comprises the following steps: based on the microphone array, carrying out filtering processing and windowing processing on the musical instrument information, determining a sound interval, and assisting in judging the hand position during playing;
the multi-modal form refers to: marking the wrong gesture information and wrong musical instrument sound segment information on a musical instrument spectrum, repeatedly checking wrong segments in wrong positions for 10s by clicking the wrong marking, simultaneously displaying correct gestures and correct musical instrument sounds, obtaining correction evaluation prompts, and displaying or not displaying small joint points in the playing process by selecting a display mode;
the display mode includes: the high-level mode and the common mode, wherein the high-level mode displays the small joint point marks in the playing process, and the common mode does not display the small joint point marks in the playing process.
2. The multi-modal piano training system based on hand pose estimation of claim 1, wherein the data acquisition module comprises: the device comprises a wide-angle camera and a microphone array, wherein the wide-angle camera is used for collecting gesture information in the playing process, and the microphone array is used for collecting musical instrument sound information in the playing process.
3. The multi-modal piano training system based on hand pose estimation of claim 1, wherein the gesture recognition algorithm is specifically: the ResNet residual network and the double-channel convolutional neural network are adopted, the content of rapid change in gesture information is focused, the position and direction vectors of all small nodes in the image are predicted, motion information is captured, the positions of the small nodes are used for generating a heat map, and the heat map is used as a signal supervision training process to realize small node identification and semantic understanding.
4. The multi-modal piano training system based on hand pose estimation of claim 1, wherein the image processing algorithm is specifically: firstly, carrying out de-distortion treatment, then carrying out binarization treatment on the gesture image, and finally adopting image open operation to remove noise points.
5. The multimode piano training partner method based on hand gesture estimation is characterized by comprising the following steps of:
the gesture information and the musical instrument information in the playing process are collected through the data collection module, and the two groups of information are aligned and then sent to the data identification module;
identifying small joint points in the gesture information according to the musical instrument sound information and a preset algorithm through the data identification module, obtaining small joint point identification results, and sending the small joint point identification results to a data comparison module;
comparing the small joint point identification result with a standard database through the data comparison module to obtain error gesture information and error musical instrument sound fragment information in the playing process, and sending the error gesture information and the error musical instrument sound fragment information to a result display module;
marking the wrong gesture information and wrong musical instrument sound fragment information on a musical instrument spectrum through the result display module to obtain a marking result, and feeding back the marking result to a user through a multi-mode form;
the specific steps of the multi-modal form include: marking the wrong gesture information and wrong musical instrument sound segment information on a musical instrument spectrum, repeatedly checking wrong segments in wrong positions for 10s by clicking the wrong marking, simultaneously displaying correct gestures and correct musical instrument sounds, obtaining correction evaluation prompts, and displaying or not displaying small joint points in the playing process by selecting a display mode;
the display mode includes: a high-level mode and a normal mode, wherein the high-level mode displays the small joint point marks in the playing process, and the normal mode does not display the small joint point marks in the playing process;
the preset algorithm comprises a gesture recognition algorithm, an audio recognition algorithm and an image processing algorithm, and the specific implementation steps of the audio recognition algorithm comprise: based on the microphone array, the musical instrument information is subjected to filtering processing and windowing processing, a sound interval is determined, and the hand position during playing is assisted in judgment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110492931.8A CN113255470B (en) | 2021-05-06 | 2021-05-06 | Multi-mode piano accompany training system and method based on hand gesture estimation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110492931.8A CN113255470B (en) | 2021-05-06 | 2021-05-06 | Multi-mode piano accompany training system and method based on hand gesture estimation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113255470A CN113255470A (en) | 2021-08-13 |
| CN113255470B true CN113255470B (en) | 2024-04-02 |
Family
ID=77223830
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110492931.8A Active CN113255470B (en) | 2021-05-06 | 2021-05-06 | Multi-mode piano accompany training system and method based on hand gesture estimation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113255470B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117863175A (en) * | 2023-12-25 | 2024-04-12 | 之江实验室 | Offline evaluation system and method for playing piano robot |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017037342A1 (en) * | 2015-09-04 | 2017-03-09 | Pianorobot Oy | System for teaching a user to play a musical instrument from musical notation via virtual exercises and a method thereof |
| CN108074439A (en) * | 2016-11-18 | 2018-05-25 | 北京酷我科技有限公司 | A kind of piano training mate equipment and piano |
| CN110210547A (en) * | 2019-05-27 | 2019-09-06 | 南京航空航天大学 | Piano playing gesture identification method based on inertia gloves |
| CN111191627A (en) * | 2020-01-06 | 2020-05-22 | 浙江工业大学 | Method for improving accuracy of dynamic gesture motion recognition under multiple viewpoints |
| CN111259863A (en) * | 2020-03-03 | 2020-06-09 | 森兰信息科技（上海）有限公司 | Method for detecting/displaying playing hand type, medium, piano, terminal and server |
-
2021
- 2021-05-06 CN CN202110492931.8A patent/CN113255470B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017037342A1 (en) * | 2015-09-04 | 2017-03-09 | Pianorobot Oy | System for teaching a user to play a musical instrument from musical notation via virtual exercises and a method thereof |
| CN108074439A (en) * | 2016-11-18 | 2018-05-25 | 北京酷我科技有限公司 | A kind of piano training mate equipment and piano |
| CN110210547A (en) * | 2019-05-27 | 2019-09-06 | 南京航空航天大学 | Piano playing gesture identification method based on inertia gloves |
| CN111191627A (en) * | 2020-01-06 | 2020-05-22 | 浙江工业大学 | Method for improving accuracy of dynamic gesture motion recognition under multiple viewpoints |
| CN111259863A (en) * | 2020-03-03 | 2020-06-09 | 森兰信息科技（上海）有限公司 | Method for detecting/displaying playing hand type, medium, piano, terminal and server |
Non-Patent Citations (1)
| Title |
|---|
| 钢琴演奏自动评估系统开发与设计;黄承承;;自动化技术与应用(第09期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113255470A (en) | 2021-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106340286B (en) | Universal real-time musical instrument playing evaluation system | |
| CN106485984B (en) | Intelligent teaching method and device for piano | |
| CN103150940B (en) | A kind of keyboard instrument teaching system and teaching method | |
| CN102110399B (en) | A kind of assist the method for explanation, device and system thereof | |
| CN107918821A (en) | Teachers ' classroom teaching process analysis method and system based on artificial intelligence technology | |
| CN109669661A (en) | Control method of dictation progress and electronic equipment | |
| CN110085068A (en) | Learning tutoring method and device based on image recognition | |
| Johnson et al. | Detecting hand posture in piano playing using depth data | |
| CN112614489A (en) | User pronunciation accuracy evaluation method and device and electronic equipment | |
| CN113255470B (en) | Multi-mode piano accompany training system and method based on hand gesture estimation | |
| CN111695499B (en) | Key identification method and device, electronic equipment and storage medium | |
| CN116109455B (en) | Language teaching auxiliary system based on artificial intelligence | |
| CN111951629A (en) | Pronunciation correction system, method, medium and computing device | |
| CN106960609A (en) | A kind of intelligent music teaching device and its application method | |
| CN111079501B (en) | Character recognition method and electronic equipment | |
| CN111968675A (en) | Stringed instrument note comparison system based on hand recognition and use method thereof | |
| CN112151194A (en) | Fitness training monitoring system and method, storage medium and electronic equipment | |
| KR101840014B1 (en) | Apparatus for recognizing dancing motion based music and method for operating the same | |
| CN117151548A (en) | Music online learning method and system based on hand motion judgment | |
| CN113286193A (en) | Intelligent music accompanying method and system for children based on deep learning of computer | |
| CN111554318B (en) | Method for realizing mobile phone terminal pronunciation visualization system | |
| CN115482397A (en) | Action scoring system | |
| CN112967736A (en) | Pronunciation quality detection method, system, mobile terminal and storage medium | |
| CN101458927A (en) | Method and system for generating and detecting confusion sound | |
| CN114170868A (en) | Intelligent piano training method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |