CN109522959A - A kind of music score identification classification and play control method - Google Patents

A kind of music score identification classification and play control method Download PDF

Info

Publication number
CN109522959A
CN109522959A CN201811375423.6A CN201811375423A CN109522959A CN 109522959 A CN109522959 A CN 109522959A CN 201811375423 A CN201811375423 A CN 201811375423A CN 109522959 A CN109522959 A CN 109522959A
Authority
CN
China
Prior art keywords
note
image
music score
score
identification
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811375423.6A
Other languages
Chinese (zh)
Inventor
严复刚
于福航
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin University of Science and Technology
Original Assignee
Harbin University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin University of Science and Technology filed Critical Harbin University of Science and Technology
Priority to CN201811375423.6A priority Critical patent/CN109522959A/en
Publication of CN109522959A publication Critical patent/CN109522959A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/23Clustering techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/12Computing arrangements based on biological models using genetic models
    • G06N3/126Evolutionary algorithms, e.g. genetic algorithms or genetic programming
    • G06T5/70

Abstract

Musical score image recognition methods disclosed by the invention and performance control method, including, it obtains music score to be processed: first passing through the useless point on k nearest neighbor algorithm removal music score, then the edge detection based on Gauss-Laplace operator is carried out to music score and obtains music score content;The position that happy symbol is obtained with LSD algorithm, judges music score classification with linear position and relative distance;Classified by music score classifier, sorted music score is then subjected to image segmentation, obtains the image of independent note;Image will be obtained and divide surface layer and inner layer, surface layer identifies note, inner layer identifies slur, table inner layer is inputted into convolutional neural networks respectively, the result after identification is combined again, obtain completely happy symbol, and note is associated with corresponding motor control feeding, the target realizing digital control platform autonomous classification music score and playing the musical instrument.The features such as music score recognition method and device of invention have identification range wide compared to other existing congenic methods, accuracy of identification and high robustness.

Description

A kind of music score identification classification and play control method
Technical field
The present invention relates to technical field of machine vision, in particular to a kind of musical score image identification and classification method.
Background technique
Machine vision, target will be ingested by machine vision product by, which referring to, is converted into picture signal, sends to dedicated Image processing system obtains the shape information of target subject, according to the information such as pixel distribution and brightness, color, is transformed into number Change signal, picture system carries out various operations to these signals to extract clarification of objective, and then controls according to the result of differentiation Make the device action at scene.
Existing musical score image recognition methods can only identify specific music score (staff or numbered musical notation is such as individually identified) mostly, Some image-recognizing methods are also based on traditional computer recognition methods, can not accomplish quickly to accurately identify, some is also needed to knowledge Other music score makes height Standardization Requirement, be unfavorable for using.
Summary of the invention
The present invention is intended to provide a kind of musical score image identification classifies and plays control method, current music score identification can solve The problems such as precision is not high and music score identification specification is excessive.
In order to achieve the above object, the embodiment of the invention discloses a kind of, musical score image identification classification and playing control method, wrap It includes: obtaining numbered musical notation or staff image to be processed, and be used for k nearest neighbor algorithm and carry out the pretreatment of image to remove image On useless point;Then using the edge detection method for being based on Gauss-Laplace operator (LOG), inclination school is carried out to image Just and obtain the image 1 after image filtering, enhancing, detection and positioning;Identification classification is carried out to the music score in image: by adopting Note locations are obtained with the LSD algorithm in straight-line detection, and image 1 is denoised by straight line cluster and straight line screening, so It inputs in music score classifier afterwards and the relative distance the straight line in image 1 is compared with preset threshold value, judgement is found pleasure in The classification of spectrum, i.e. staff or numbered musical notation;Sorted staff or numbered musical notation are inputted to the something lost of trained different parameters respectively It is every to obtain that propagation algorithm (because the note of staff and numbered musical notation is of different sizes, therefore training parameter is slightly different) carries out image segmentation The image 2 of a independent note;Image 2 is divided into surface layer and inner layer, is separately input in trained convolutional neural networks, table Layer identification note, inner layer identify slur, after neural network recognization by after identification note and slur form it is complete Note obtains the note of standard;The note in note library is matched by the standard note of acquisition, and by corresponding to standard note Numerical control music platform control instruction corresponding to note in note library recalls, and is played according to target instruction target word.
Preferably, pretreatment musical score image to be processed filtered for the first time, comprising: use k nearest neighbor algorithm first (KNN) edge-smoothing filtering is carried out to the image got, then using the method for spot (Blob) detection, identifies the company by note Sound legato symbol, staccato staccato symbol, the slurs such as attachment.
Preferably, the musical score image recognition methods, which is characterized in that the music score of identification is used and is based on LOG operator Edge detection method, to image carry out slant correction and obtain through image filtering, enhancing, detection and positioning, obtain image 1, Including:
A) LOG operator is selected:
B) LOG is selected often to use template:
Preferably, the note locations in the every row of music score are obtained by using the LSD algorithm in straight-line detection, and passes through straight line Cluster denoises the straight line in image 1 with straight line screening, and linear distance shortest in the image of acquisition 1 is inputted music score In classifier, according to threshold decision music score classification preset in classifier, i.e. staff or numbered musical notation.
Preferably, the staff classified or numbered musical notation image are inputted respectively and carries out image point in trained genetic algorithm It cuts to obtain each complete independent note image, is denoted as image 2.
Preferably, the image of acquisition 2 is divided into surface layer and inner layer, surface layer is note main body, and inner layer is slur, rest Deng modification note, then surface layer and inner layer are inputted respectively in preset convolutional neural networks and identified, will identify that table inner layer sound Symbol is combined into complete note, as target note.
Preferably, by the good note control unit of target note difference compiling after matching, include in this note control unit Current of electric corresponding to each note, the state modulators information such as lead screw amount of feeding, so that the music score for completing music platform is played Movement.
As seen from the above technical solution, the music score (staff and numbered musical notation) to be processed to acquisition: the embodiment of the present invention is first led to It crosses k nearest neighbor (KNN) algorithm and removes useless point on musical score image to be processed, then analyze and music score to be processed is carried out based on height This-edge detection method of Laplace operator (LOG operator) carries out image slant correction and obtains music score content;Using straight line LSD algorithm in detection obtains position of the happy symbol in music score, then straight by straight line cluster and straight line screening removal erroneous judgement Line, and music score classification is judged with linear position and relative distance, i.e., (linear distance is big and each apart from difference for staff or numbered musical notation Fluctuate small for numbered musical notation) classified by music score classifier, then sorted music score is carried out by genetic algorithm (GA) Image segmentation, to obtain the image of each independent note;It will obtain image layered (surface layer and inner layer), surface layer identifies note, Inner layer identifies slur, table inner layer is inputted convolutional neural networks identification, then the result after identification is combined respectively, can obtained Complete happy symbol, and note is associated with corresponding motor control feeding, realization digital control platform autonomous classification music score and plays the musical instrument Target.The music score recognition method and device of invention have identification range wide compared to other existing congenic methods, The features such as accuracy of identification and high robustness.Certainly, it implements any of the products of the present invention or method must be not necessarily required to reach simultaneously All the above advantage.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, below to embodiment or existing Attached drawing needed in technical description is briefly described, it should be apparent that, drawings discussed below is only the present invention Some embodiments for those of ordinary skill in the art without creative efforts, can also basis These attached drawings obtain other attached drawings.
Fig. 1 is the structural schematic diagram in a kind of embodiment of music score identification classification of the present invention and performance control device;
Fig. 2 is the flow chart in a kind of embodiment of music score identification classification of the present invention and performance control method;
Fig. 3 is the key technology flow chart in a kind of embodiment of music score identification classification of the present invention and performance control method;
Fig. 4 is straight-line detection and music score classification original in a kind of embodiment of music score identification classification of the present invention and performance control method Reason figure;
Fig. 5 is the convolutional neural networks structure chart in a kind of embodiment of music score identification classification of the present invention and performance control method.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other Embodiment shall fall within the protection scope of the present invention.
The first embodiment of music score recognition method of the present invention, as shown in Figure 1, comprising the following steps: step 1: the pleasure of acquisition Spectrogram picture;Step 2: first passing through k nearest neighbor (KNN) algorithm and remove useless point on musical score image to be processed, then analyze to be processed Music score carry out the edge detection method based on Gauss-Laplace operator (LOG operator) and image slant correction and obtain Music score content;Step 3: the edge detection method based on LOG operator being used to the music score of identification, slant correction is carried out simultaneously to image It obtains through image filtering, enhancing, detection and positioning;Step 4: being classified by music score classifier, then by sorted pleasure Spectrum carries out image segmentation by genetic algorithm (GA), to obtain the image of each independent note;Step 5: image point will be obtained Layer (surface layer and inner layer), surface layer identify that note, inner layer identify slur, and table inner layer is inputted convolutional neural networks identification respectively, The result after identification is combined again, completely happy symbol can be obtained;Step 6: note is associated with corresponding motor control feeding, The target realizing digital control platform autonomous classification music score and playing the musical instrument.
A kind of structure of embodiment of musical score image identification device of the present invention, as shown in figure XX, comprising: image collection module, Obtain musical score image to be processed;Music score preprocessing module is found pleasure in by KNN filtering and the edge detection method of LOG operator The marginal information and basic content of spectrogram picture;Music score categorization module obtains music score using the line detection method based on LSD algorithm Linear position, then will in the music score classifier of happy symbol straight line relative distance input preset threshold, judge music score for staff still Numbered musical notation, and inputted in different genetic algorithms respectively according to classification, so that segmentation obtains each independent note;Score note is known Other module, image layered (surface layer and the inner layer) that categorization module is obtained, surface layer identify note, and inner layer identifies slur, by table Inner layer inputs convolutional neural networks identification respectively, then the surface layer inner layer result after identification is combined, and obtains the note of standard;It is happy Spectrum plays module, matches the note in note library by the standard note of acquisition, and by corresponding to standard note in note library Note corresponding to numerical control music platform control instruction recall, played according to target instruction target word.

Claims (5)

1. a kind of musical score image recognition methods, it is characterised in that: obtain numbered musical notation or staff image to be processed, and be used for K nearest neighbor algorithm carries out the pretreatment of image to remove the useless point on image;Then using based on Gauss-Laplace operator (LOG) edge detection method carries out slant correction to image and obtains the figure after image filtering, enhancing, detection and positioning As 1;Identification classification is carried out to the music score in image: obtaining note locations by using the LSD algorithm in straight-line detection, and passes through Straight line cluster denoises image 1 with straight line screening, then inputs in music score classifier between opposite the straight line in image 1 Distance is compared with preset threshold value, and judgement obtains the classification of music score, i.e. staff or numbered musical notation;By sorted staff or The genetic algorithm that numbered musical notation inputs trained different parameters respectively (because the note of staff and numbered musical notation is of different sizes, therefore is trained Parameter is slightly different) image segmentation is carried out to obtain the image 2 of each independent note;Image 2 is divided into surface layer and inner layer, point It is not input in trained convolutional neural networks, surface layer identifies that note, inner layer identify slur, by neural network recognization Afterwards by the note and the complete note of slur composition after identification, the note of standard is obtained;It is matched by the standard note of acquisition Decode the note in library, and by corresponding to standard note numerical control music platform control instruction corresponding to the note in note library It recalls, is played according to target instruction target word.
2. musical score image recognition methods according to claim 1, which is characterized in that musical score image to be processed, comprising: Edge-smoothing filtering, then the side using spot (Blob) detection are carried out to the image got using k nearest neighbor algorithm (KNN) first Method identifies that the liaison by note is continued playing and accords with that staccato staccato symbol, the slurs such as attachment use the music score of identification and are based on LOG operator Edge detection method, to image carry out slant correction and obtain through image filtering, enhancing, detection and positioning.
3. musical score image recognition methods according to claim 1, which is characterized in that calculated using LSD the image 1 detected Method carries out straight-line detection, cluster and screening, and linear position and linear position relative distance are inputted in music score classifier, according to Preset threshold decision music score classification in classifier inputs trained staff or numbered musical notation for sorted music score respectively In genetic algorithm, image segmentation is carried out to music score by genetic algorithm, to obtain the image of each independent note.
4. musical score image recognition methods according to claim 1, which is characterized in that each of will divide independent note Image is respectively divided into surface layer and inner layer, is input in trained convolutional neural networks, and surface layer identifies that note, inner layer identification connect Note, by the note and the complete note of slur composition after identification, as target note after neural network recognization.
5. a kind of musical score image identification device, which is characterized in that including image collection module obtains musical score image to be processed; Music score preprocessing module obtains the marginal information and base of musical score image by KNN filtering and the edge detection method of LOG operator This content;Music score categorization module obtains music score linear position using the line detection method based on LSD algorithm, then by happy Fu Zhi Line relative distance inputs in the music score classifier of preset threshold, judges that music score for staff or numbered musical notation, and is distinguished according to classification It inputs in different genetic algorithms, so that segmentation obtains each independent note;Score note identification module, categorization module is obtained Image layered (surface layer and the inner layer) obtained, surface layer identify that note, inner layer identify slur, table inner layer inputted convolutional Neural respectively Network Recognition, then the surface layer inner layer result after identification is combined, obtain the note of standard;Music score plays module, passes through acquisition Standard note matching note library in note, and by corresponding to standard note numerical control sound corresponding to the note in note library Leping platform control instruction recalls, and is played according to target instruction target word.
CN201811375423.6A 2018-11-19 2018-11-19 A kind of music score identification classification and play control method Pending CN109522959A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811375423.6A CN109522959A (en) 2018-11-19 2018-11-19 A kind of music score identification classification and play control method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811375423.6A CN109522959A (en) 2018-11-19 2018-11-19 A kind of music score identification classification and play control method

Publications (1)

Publication Number Publication Date
CN109522959A true CN109522959A (en) 2019-03-26

Family

ID=65776632

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811375423.6A Pending CN109522959A (en) 2018-11-19 2018-11-19 A kind of music score identification classification and play control method

Country Status (1)

Country Link
CN (1) CN109522959A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110299049A (en) * 2019-06-17 2019-10-01 韶关市启之信息技术有限公司 A kind of intelligence of electronic music shows method
CN110598581A (en) * 2019-08-25 2019-12-20 南京理工大学 Optical music score recognition method based on convolutional neural network
CN111104964A (en) * 2019-11-22 2020-05-05 北京永航科技有限公司 Music and action matching method, equipment and computer storage medium
CN111104869A (en) * 2019-11-26 2020-05-05 杭州电子科技大学 Method for digitizing work-ruler spectrum capable of identifying content of small characters
CN111275043A (en) * 2020-01-22 2020-06-12 西北师范大学 Paper numbered musical notation electronization play device based on PCNN handles
CN111274891A (en) * 2020-01-14 2020-06-12 成都嗨翻屋科技有限公司 Method and system for extracting pitches and corresponding lyrics for numbered musical notation images
CN112183658A (en) * 2020-10-14 2021-01-05 小叶子(北京)科技有限公司 Music score identification method and device, electronic equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090202106A1 (en) * 2008-02-12 2009-08-13 Tae-Hwa Hong Method for recognizing music score image with automatic accompaniment in mobile device
US20120160077A1 (en) * 2010-12-27 2012-06-28 Kabushiki Kaisha Kawai Gakki Seisakusho Musical score recognition device and computer-readable storage medium
CN102663423A (en) * 2012-03-28 2012-09-12 北京航空航天大学 Method for automatic recognition and playing of numbered musical notation image
CN103310769A (en) * 2012-03-14 2013-09-18 卡西欧计算机株式会社 Musical performance device,and method for controlling musical performance device
CN104036227A (en) * 2013-03-05 2014-09-10 联想移动通信科技有限公司 Electronic music score generating method and mobile terminal

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090202106A1 (en) * 2008-02-12 2009-08-13 Tae-Hwa Hong Method for recognizing music score image with automatic accompaniment in mobile device
US20120160077A1 (en) * 2010-12-27 2012-06-28 Kabushiki Kaisha Kawai Gakki Seisakusho Musical score recognition device and computer-readable storage medium
CN103310769A (en) * 2012-03-14 2013-09-18 卡西欧计算机株式会社 Musical performance device,and method for controlling musical performance device
CN102663423A (en) * 2012-03-28 2012-09-12 北京航空航天大学 Method for automatic recognition and playing of numbered musical notation image
CN104036227A (en) * 2013-03-05 2014-09-10 联想移动通信科技有限公司 Electronic music score generating method and mobile terminal

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
APURVA A MEHTA ET AL: "Optical music notes recognition for printed piano music score sheet", 《2015 INTERNATIONAL CONFERENCE ON COMPUTER COMMUNICATION AND INFORMATICS (ICCCI)》 *
王育峰 等: "乐谱识别中音符分割技术的研究", 《中国科技论文在线》 *

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110299049A (en) * 2019-06-17 2019-10-01 韶关市启之信息技术有限公司 A kind of intelligence of electronic music shows method
CN110299049B (en) * 2019-06-17 2021-12-17 韶关市启之信息技术有限公司 Intelligent display method of electronic music score
CN110598581A (en) * 2019-08-25 2019-12-20 南京理工大学 Optical music score recognition method based on convolutional neural network
CN110598581B (en) * 2019-08-25 2022-09-27 南京理工大学 Optical music score recognition method based on convolutional neural network
CN111104964A (en) * 2019-11-22 2020-05-05 北京永航科技有限公司 Music and action matching method, equipment and computer storage medium
CN111104964B (en) * 2019-11-22 2023-10-17 北京永航科技有限公司 Method, equipment and computer storage medium for matching music with action
CN111104869A (en) * 2019-11-26 2020-05-05 杭州电子科技大学 Method for digitizing work-ruler spectrum capable of identifying content of small characters
CN111274891A (en) * 2020-01-14 2020-06-12 成都嗨翻屋科技有限公司 Method and system for extracting pitches and corresponding lyrics for numbered musical notation images
CN111275043A (en) * 2020-01-22 2020-06-12 西北师范大学 Paper numbered musical notation electronization play device based on PCNN handles
CN111275043B (en) * 2020-01-22 2021-08-20 西北师范大学 Paper numbered musical notation electronization play device based on PCNN handles
CN112183658A (en) * 2020-10-14 2021-01-05 小叶子(北京)科技有限公司 Music score identification method and device, electronic equipment and storage medium
CN112183658B (en) * 2020-10-14 2024-01-26 小叶子(北京)科技有限公司 Music score identification method and device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
CN109522959A (en) A kind of music score identification classification and play control method
CN115082683B (en) Injection molding defect detection method based on image processing
US10835930B2 (en) Sorting system
CN110032938B (en) Tibetan recognition method and device and electronic equipment
CN109886998A (en) Multi-object tracking method, device, computer installation and computer storage medium
CN111144490A (en) Fine granularity identification method based on alternative knowledge distillation strategy
CN109712162B (en) Cable character defect detection method and device based on projection histogram difference
CN109815967A (en) CNN ship seakeeping system and method based on Fusion Features
CN106297755B (en) Electronic equipment and identification method for music score image identification
WO2015131468A1 (en) Method and system for estimating fingerprint pose
CN116205919A (en) Hardware part production quality detection method and system based on artificial intelligence
CN112528975A (en) Industrial quality inspection method, device and computer readable storage medium
CN108846831A (en) The steel strip surface defect classification method combined based on statistical nature and characteristics of image
CN111898594A (en) Road surface terrain recognition method and device based on suspension vibration signal and storage medium
CN107103608A (en) A kind of conspicuousness detection method based on region candidate samples selection
Chen et al. Automatic sorting of fresh tea leaves using vision‐based recognition method
JP2010071826A (en) Teacher data preparation method, and image sorting method and image sorter
JP7377159B2 (en) Individual identification system, individual identification program, and recording medium
CN110364224B (en) Chromosome split phase positioning and sequencing method
JP6425468B2 (en) Teacher data creation support method, image classification method, teacher data creation support device and image classification device
Lee et al. SPIN: a method of skeleton-based polarity identification for neurons
Kiratiratanapruk et al. Segmentation algoritm for touching round grain image
CN113470017A (en) Power transmission line safety early warning detection method, device and equipment based on artificial intelligence
CN106845523B (en) A kind of data processing method blending geodata and sensor signal data
CN110334644B (en) Gesture recognition method based on machine vision

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20190326