CN106297755A - A kind of electronic equipment for musical score image identification and recognition methods - Google Patents
A kind of electronic equipment for musical score image identification and recognition methods Download PDFInfo
- Publication number
- CN106297755A CN106297755A CN201610859907.2A CN201610859907A CN106297755A CN 106297755 A CN106297755 A CN 106297755A CN 201610859907 A CN201610859907 A CN 201610859907A CN 106297755 A CN106297755 A CN 106297755A
- Authority
- CN
- China
- Prior art keywords
- note
- image
- head
- sound
- circuit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/32—Constructional details
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2220/00—Input/output interfacing specifically adapted for electrophonic musical tools or instruments
- G10H2220/155—User input interfaces for electrophonic musical instruments
- G10H2220/441—Image sensing, i.e. capturing images or optical patterns for musical purposes or musical control purposes
- G10H2220/455—Camera input, e.g. analyzing pictures from a video camera and using the analysis results as control data
Abstract
A kind of electronic equipment for musical score image identification disclosed by the invention and recognition methods, including housing, sound-generating element, the mainboard being arranged in housing and the image scanning parts being arranged on housing first ends；Governor circuit and the sound card circuit electrically connected respectively and power circuit it is provided with governor circuit on mainboard；Obtain pending staff image by photographic head and pass to governor circuit；Pending staff image is identified by governor circuit, identifies each complete note；Governor circuit, according to the complete note identified, sends corresponding audio digital signal, to sound card circuit, sound card circuit, the audio digital signal received is converted into playable analogue signal, pass to sound-generating element and play out；Present in present device solution prior art, image capture module separates with identification module, awkward problem.Method uses note grader to carry out note identification with convolutional neural networks cascade, has recognition speed fast, the advantage that accuracy of identification is high.
Description
Technical field
The present invention relates to image identification technical field, particularly to a kind of electronic equipment for musical score image identification and knowledge
Other method.
Background technology
Image recognition, refers to utilize computer to process image, analyze and understand, to identify various different mode
Target and the technology to picture.
Musical score image identification equipment of the prior art, including image capture module and computer, image capture module with
Take pictures or scan the mode of music score and gather the view data of music score, be input in computer, by the identification mould in computer
The view data collected is analyzed identifying by block.
But, apply above-mentioned musical score image identification equipment, there is techniques below problem: image capture module with identify mould
Block separates, and needs to rely on computer operation, and work process is longer, the convenience that impact uses.
Musical score image recognition methods of the prior art, is mostly based on traditional computer visible sensation method, at accuracy of identification and
It not very good in recognition speed, it is impossible to accomplish fast accurate identification, even need music score to be identified is made high metric
Generalized requirement, is unfavorable for the use of everyday scenes.
Summary of the invention
The purpose of the embodiment of the present invention is to provide a kind of electronic equipment for musical score image identification and recognition methods, can
Separate with identification module with solution musical score image identification equipment image capture module of the prior art, use inconvenience, and existing
Musical score image recognition methods accuracy of identification in technology and the undesirable problem of recognition speed.
For reaching above-mentioned purpose, the embodiment of the invention discloses, a kind of electronic equipment for musical score image identification, including
Housing, sound-generating element, the mainboard being arranged in housing and be arranged on the image scanning parts of described housing first ends；
Governor circuit and the sound card circuit electrically connected respectively and power circuit it is provided with governor circuit on described mainboard；
Described image scanning parts include scanning roller and being arranged on the photographic head above scanning roller, and described scanning is rolled
Wheel and photographic head all electrically connect with described governor circuit；The musical score image of shooting is sent to governor circuit by described photographic head to be carried out
Process；
Described sound-generating element is connected with described sound card circuit, and the acoustical signal sent by governor circuit sends sound；
Described power circuit electrically connects as its power supply respectively with described scanning roller, photographic head and sound-generating element；
The second end of described housing is provided with battery flat and hatchcover, and battery flat is connected with the power circuit on mainboard.
Preferably, described housing is lip pencil housing；Described image scanning parts are arranged on the first end of lip pencil housing；
Described sound-generating element is arranged on above described image scanning parts, and described image scanning parts and sound-generating element make
One end is formed as nib shape；
Described mainboard is the position of close nib in being arranged on lip pencil housing；
At least 2 mainboard mounting posts it are provided with in described lip pencil housing；Described mainboard is installed by described at least 2 mainboards
Post is fixed in lip pencil housing.
Preferably, the second end of described lip pencil housing is provided with battery flat and hatchcover, battery flat and the power supply on mainboard
Circuit is connected.
Preferably, the second end of described lip pencil housing is provided with circumscripted power line, circumscripted power line and the electricity on mainboard
Source circuit is connected.
The embodiment of the invention also discloses, a kind of musical score image recognition methods, including,
Obtain pending staff image by photographic head and pass to governor circuit；
Pending staff image is identified by governor circuit, identifies each complete note；
Governor circuit, according to the complete note identified, sends corresponding audio digital signal to sound card circuit, sound card electricity
The audio digital signal received is converted into playable analogue signal by road, passes to sound-generating element and plays out；
Pending staff image is identified by described governor circuit, including,
Use edge detection method to depict the marginal information of image pending staff image, then examined by straight line
Survey method detects five line position coordinates；
Use the note grader preset, pending staff image is carried out note locating segmentation, obtains each complete
Whole note position in the picture；
Use preset convolutional neural networks to segmentation obtain note symbol head be identified, it is judged that be solid symbol head or
Hollow symbol head, and obtain according with the position of head；
The five line position coordinates, the relative position of each complete note that obtain according to described, it is solid symbol head or hollow
Symbol head and the position of symbol head, identify each complete note.
Preferably, the training process of described note grader, including:
Set up positive sample data set and negative sample data set, in data set includes position data and the posting of posting
The view data of staff image, positive sample data set is the view data including complete note, and negative sample data set is bag
Include except the view data that is likely to occur of remaining music score in addition to complete note；
Extract the channel characteristics of each sample in positive sample data set and negative sample data set, train note grader.
Preferably, described carries out note locating segmentation to pending staff image, including,
Pending staff image randomly selects several candidate's posting, one by one Scan orientation frames, to each
The channel characteristics described in image zooming-out in posting, is input to the channel characteristics of extraction in note grader, it is judged that location
Image in frame is positive sample or is negative sample, and the complete note that positive sample is judged in music score, negative sample is judged to music score
Background is given up, thus obtains the complete note in pending staff image, the position of posting in comparison note grader
Data obtain each complete note position in the picture.
Preferably, the training process of described convolutional neural networks, including,
Set up note symbol head data set, including solid symbol head, hollow symbol head and three kinds of categorical datas of background；
Build convolutional neural networks, including 2 convolutional layers, 2 down-sampling layers and 1 full articulamentum；
The symbol head view data accorded with by note in head data set is input in convolutional neural networks, completes training.
Preferably, the described note symbol head using convolutional neural networks to obtain segmentation is identified, including,
The complete note obtained by note locating segmentation, is input in convolutional neural networks, by according with head data with note
Data Comparison in collection, draws it is solid symbol head, hollow symbol head or background, gives up background, simultaneously comparison note symbol head data
The position data of the symbol head in collection, determines the position according with head in complete note.
Preferably, described pending staff image, particularly as follows: carry out denoising, contrast increasing to staff image
By force, gray processing, minimizing noise or the even process of uneven illumination, the bianry image obtained.
As seen from the above technical solutions, electronic equipment embodiment of the present invention is by by sound-generating element, mainboard and image
Sweep unit is fully integrated within one device, significantly improves the portability of product, solves figure present in prior art
As acquisition module separates with identification module, awkward problem.
Recognition methods embodiment of the present invention, uses edge detection method to depict image pending staff image
Marginal information, then detect five line position coordinates by line detection method；Use the note grader preset, to pending
Staff image carries out note locating segmentation, obtains each complete note position in the picture；Use the convolutional Neural preset
The note symbol head that segmentation is obtained by network is identified, it is judged that is solid symbol head or hollow symbol head, and obtains according with the position of head；
The five line position coordinates, the relative position of each complete note that obtain according to described, it is solid symbol head or hollow symbol head and symbol
The position of head, identifies each complete note.Compared to traditional computer visible sensation method, the present invention uses note grader and volume
Long-pending neutral net cascade carries out note identification, has recognition speed fast, the advantage that accuracy of identification is high.
Certainly, arbitrary product or the method for implementing the present invention must be not necessarily required to reach all the above excellent simultaneously
Point.
Accompanying drawing explanation
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
In having technology to describe, the required accompanying drawing used is briefly described, it should be apparent that, the accompanying drawing in describing below is only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, it is also possible to
Other accompanying drawing is obtained according to these accompanying drawings.
Fig. 1 is a kind of example structure schematic diagram of electronic equipment of the present invention；
Fig. 2 is the circuit diagram of mainboard in electronic equipment embodiment of the present invention；
Fig. 3 is the control principle drawing of mainboard in electronic equipment embodiment of the present invention；
Fig. 4 is the flow chart of the first embodiment of music score recognition method of the present invention；
Fig. 5 is that in the first embodiment of recognition methods of the present invention, pending staff image is identified by governor circuit
Flow chart；
Fig. 6 is that in recognition methods the second embodiment of the present invention, pending staff image is identified by governor circuit
Flow chart；
Fig. 7 is monolateral edge detection method schematic diagram in music score recognition method the second embodiment of the present invention；
Fig. 8 is the design sketch of five line position coordinate measurements in music score recognition method the second embodiment of the present invention；
Fig. 9 is the training process schematic of note grader in music score recognition method the second embodiment of the present invention；
Figure 10 is positive sample data set and the sample of negative sample data set in music score recognition method the second embodiment of the present invention
This schematic diagram；
Figure 11 is the flow chart of note locating segmentation in music score recognition method the second embodiment of the present invention；
Figure 12 is the design sketch of note locating segmentation in music score recognition method the second embodiment of the present invention；
Figure 13 is the training process schematic of convolutional neural networks in music score recognition method the second embodiment of the present invention；
Figure 14 is convolutional neural networks structure chart in music score recognition method the second embodiment of the present invention；
Figure 15 is the flow chart of note symbol head identification in music score recognition method the second embodiment of the present invention；
In figure, 1. hatchcover, 2. battery flat, 3. mainboard, 4. photographic head, 5. scanning roller, 6. mainboard mounting post, 7. pars stridulans
Part, 8.LED light compensating lamp.
Detailed description of the invention
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Describe, it is clear that described embodiment is only a part of embodiment of the present invention rather than whole embodiments wholely.Based on
Embodiment in the present invention, it is every other that those of ordinary skill in the art are obtained under not making creative work premise
Embodiment, broadly falls into the scope of protection of the invention.
The present invention is for the structure of a kind of embodiment of the electronic equipment of musical score image identification, as it is shown in figure 1, housing is pen
Shape housing, image scanning parts are arranged on the first end of lip pencil housing, and sound-generating element 7 is arranged on above image scanning parts,
Image scanning parts and sound-generating element 7 make first end be formed as nib shape；Image scanning parts include scanning roller 5 and setting
Put the photographic head 4 above scanning roller 5.
Mainboard 3 is the position of close nib in being arranged on lip pencil housing.At least 2 mainboard mounting posts it are provided with in lip pencil housing
6, mainboard 3 is fixed in lip pencil housing by least 2 mainboard mounting posts 6.As in figure 2 it is shown, be provided with governor circuit on mainboard 3
And the sound card circuit that electrically connects with governor circuit respectively and power circuit；Scanning roller 5 and photographic head 4 are all electrically connected with governor circuit
Connect；The musical score image of shooting is sent to governor circuit and processes by photographic head 4；Sound-generating element 7 is connected with sound card circuit, by main
The acoustical signal that control circuit sends sends sound.
The second end of lip pencil housing is provided with battery flat 2 and hatchcover 1, battery flat 2 and the power circuit phase on mainboard 3
Even.Should be noted that and battery flat 2 and hatchcover 1 are set, it is therefore an objective to power to the power circuit on mainboard 3, it is also possible to select it
His structure is used for powering, such as: the second end at lip pencil housing is arranged on circumscripted power line, circumscripted power line and mainboard 3
Power circuit is connected.
Preferably, photographic head 4 is additionally provided with LED light supplement lamp 8, for photographic head 4 light filling.
Preferably, sound-generating element 7 is speaker.Should be noted that sound-generating element 7 fills for sounding of the prior art
Put, it is therefore an objective to complete the function of sounding.
Preferably, photographic head 4 uses cmos image sensor OV7620 to realize；Governor circuit uses microprocessor Argus3
Chip realizes.As it is shown on figure 3, microprocessor Argus3 chip embedded ARM9TDMI core, one cache of core Embedded,
Individual special RAM and various abundant application interface, support the forms such as SPAM, FLASH, and provide video processing engine and image
Processor.
Preferably, it is provided with the protection set being flexibly connected with lip pencil housing in the outside of image scanning parts, protection set
Shape matches with pointed shape, is used for protecting photographic head 4.
The first embodiment of musical score image recognition methods of the present invention, as shown in Figure 4, including,
Step 101: obtain pending staff image by photographic head and pass to governor circuit；
Step 102: pending staff image is identified by governor circuit, identifies each complete note；
Step 103: governor circuit, according to the complete note identified, sends corresponding audio digital signal to sound card electricity
Road, the audio digital signal received is converted into playable analogue signal by sound card circuit, passes to sound-generating element and plays out；
Pending staff image is identified by described governor circuit, as it is shown in figure 5, include,
Step 1021: pending staff image uses edge detection method depict the marginal information of image, then
Five line position coordinates are detected by line detection method；
Step 1022: use the note grader preset, pending staff image is carried out note locating segmentation,
To each complete note position in the picture；
Step 1023: the note symbol head using the convolutional neural networks preset to obtain segmentation is identified, it is judged that be real
Heart symbol head or hollow symbol head, and obtain according with the position of head；
Step 1024: the five line position coordinates, the relative position of each complete note that obtain according to described, be solid symbol head
Or hollow symbol head and the position of symbol head, identify each complete note.
The second embodiment of musical score image recognition methods of the present invention, as shown in Figure 6, with the first embodiment of recognition methods
Difference be, pending staff image is identified by described governor circuit, including,
Step 2021: the staff image obtained is carried out denoising, contrast enhancing, gray processing, minimizing noise or illumination
Uneven process, obtains bianry image；
Step 2022: the bianry image obtained uses monolateral edge detection method depict the marginal information of image, then
Five line position coordinates are detected by hough line detection method；
Step 2023: use the note grader preset, the bianry image obtained is carried out note locating segmentation, obtains every
Individual complete note position in the picture；
Step 2024: the note symbol head using the convolutional neural networks preset to obtain segmentation is identified, it is judged that be real
Heart symbol head or hollow symbol head, and obtain according with the position of head；
Step 2025: the five line position coordinates, the relative position of each complete note that obtain according to described, be solid symbol head
Or hollow symbol head and the position of symbol head, identify each complete note.
Other steps in the second embodiment of musical score image recognition methods of the present invention refer to the first embodiment, this
Place repeats no more.
Preferably, monolateral edge detection method described in step 2022 in recognition methods the second embodiment of the present invention, bag
Include:
A) select Sobel operator, obtain respectively in horizontal direction and Grad in vertical direction:
Horizontal gradient: sx=(a2+2a3+a4)-(a0+2a7+a6)
Vertical gradient: sy=(a0+2a1+a2)-(a6+2a5+a4)
Amplitude:
Sobel template:
Wherein, a0-a7Represent 8 neighborhood territory pixel points；
B) use non-maxima suppression that the Grad in horizontal direction and in vertical direction is suppressed, the most only retain every
The point of the maximum on individual direction gradient straight line, the value of remaining point is all set to 0；
C) use adaptive threshold method to obtain the size of threshold value to be placed in each region, use this threshold value as whether
The condition of adjoining edge limits, and depicts the marginal information of image.
In order to better illustrate the beneficial effect of monolateral edge detection method, below by traditional canny side edge detection
The monolateral edge detection method that method and the present invention use does a comparative illustration:
1) traditional canny edge detection method step includes:
A) by asking for the single order local derviation of each pixel in image and calculating gradient direction and amplitude, thus show that each point exists
Amplitude on different directions, during can relate to different operator templates, such as Robert operator, Prewitt operator etc.；
B) gradient magnitude carrying out non-extreme value suppression, the element value in image gradient amplitude matrix is the biggest, in explanatory diagram picture
The Grad of this point is the biggest, but is not enough to determine that this point is exactly marginal point, therefore needs to find pixel pole point-blank
Value, is set to 0 by the gray value corresponding to non-extreme point, so can weed out the point of most non-edge；
C) by the detection of dual threshold algorithm and adjoining edge, select two threshold values, obtain an edge image according to high threshold.
In high threshold image, boundary chain being connected into profile, when arriving the end points of profile, algorithm can be sought in 8 value neighborhood points of breakpoint
Look for the point meeting Low threshold, collect new edge further according to this point, until whole image border closes, form whole edge graph
Picture.
2) the monolateral edge detection method step that the present invention uses includes:
A) change the template operator that original canny algorithm is conventional, and then select Sobel operator (a0-a7Represent 8 neighborhoods
Pixel), obtain respectively in horizontal direction and Grad in vertical direction；
Horizontal gradient: sx=(a2+2a3+a4)-(a0+2a7+a6)
Vertical gradient: sy=(a0+2a1+a2)-(a6+2a5+a4)
Amplitude:
Sobel template:
B) equally the Grad on each direction is suppressed, but due to it is desirable that the edge of linear one-sided, so
Need to change suppressing method, suppress to change non-maxima suppression into by the non-extreme value in former method, the most only retain each direction gradient
The point of the maximum on straight line, the value of remaining point is all set to 0, as it is shown in fig. 7, using the region of (3*3) as comparison block, respectively will
Non-maximum point, compared with (1,5) (2,6) (3,7) (4,8), is set to 0 by center pixel；
C) use adaptive threshold method to obtain the size of threshold value to be placed in each region, use this threshold value as whether
The condition of adjoining edge limits, and the method has been used for reference the mode in self-adaption binaryzation, decreased the factors such as illumination the most simultaneously
Impact on zones of different.
Should be noted that described adaptive threshold method, for common method of the prior art.
Through above-mentioned contrast, during tradition canny method detection, find that bilateral edge, impact location effect all occur in every five lines
Really, the present invention uses non-maxima suppression only to retain the monolateral extreme value of gradient, adds adaptive threshold condition so that five lines are preferable
Present monolateral edge；
Should be noted that the hough line detection method in step 2022, be conventional straight-line detection of the prior art
Method, it is possible to detect five line position coordinates according to the marginal information of the image obtained, as shown in Figure 8, in the present embodiment five
The design sketch of line spectrum location.
Preferably, the training process of step 2023 note grader in recognition methods the second embodiment of the present invention, such as Fig. 9
Shown in, including:
Step 301: setting up positive sample data set and negative sample data set, as shown in Figure 10, data set includes posting
Position data and posting in the view data of staff image, positive sample data set is the picture number including complete note
According to, negative sample data set be include except the view data that is likely to occur of remaining music score in addition to complete note；
Step 302: extract the channel characteristics of each sample in positive sample data set and negative sample data set, training note divides
Class device.
Should be noted that negative sample herein can be the note image of incompleteness, staff image, music score background image
Deng, but it is not limited to the above-mentioned image enumerated.
Preferably, the channel characteristics of each sample, including, gray scale and color, linear filtering, nonlinear transformation, pointwise become
Change, histogram of gradients.Should be noted that described 5 kind channel characteristics, for integrating channel feature of the prior art, definition
It is explained as follows:
Gray scale and color: gray scale is a kind of simple passage, and LUV color space is also three conventional passages；
Linear filtering: utilize linear transformation to obtain passage, as carried out convolution by the Gabor filter of image Yu different directions
The passage obtained, each passage comprises the marginal information of different directions, thus obtains the texture of image different scale
Information；
Nonlinear transformation: calculate image gradient amplitude, captures edge strength information；Capture edge gradient information, gradient is then
Not only comprise edge strength but also comprise edge direction, for coloured picture, then need to calculate gradient respectively at 3 passages, and by right
Answer the peak response of 3 gradients of position as last output；Binary image, is carried out with two different threshold values respectively by image
Binaryzation；
Pointwise converts: any pixel in passage can be changed as post processing by any one function.As logical
Cross Log computing, local multiplication operator exp (∑ can be obtainedilog(xi))=∏ixi, similar, each pixel is calculated p time
Power can be used for solving extended mean value；
Histogram of gradients: be a weighted histogram, its bin index is by the direction calculating of gradient out, and
Its weights are then to be calculated by the amplitude of gradient and come, say, that passage here is to be calculated such that Qθ(x, y)=G
(x, y) * 1 [Θ (x, y)=θ], here G (x, y) and Θ (x y) is the gradient magnitude of representative image respectively and quantifies gradient side
To, meanwhile carry out the fuzzy of different scale, the gradient information of different scale can be calculated.Additionally, by means of gradient
Amplitude information, is normalized the rectangular histogram calculated, and the method is similar to HOG feature.
Preferably, posting is rectangular block posting, determines the size of posting according to five distance between centers of tracks interval, location
Frame height height and width width calculates according to formula respectively:
Height=5*interval；Width=2.5*interval.
Preferably, pending staff image is carried out in step 2023 by recognition methods the second embodiment of the present invention
Note locating segmentation, as shown in figure 11, including,
Binary map to be identified randomly selects several candidate's posting, one by one Scan orientation frames, to each location
The channel characteristics described in image zooming-out in frame, is input to the channel characteristics of extraction in note grader, it is judged that in posting
Image be positive sample or for negative sample, the complete note that positive sample is judged in music score, negative sample is judged to music score background
Giving up, thus obtain the complete note in binary map to be identified, in comparison note grader, the position data of posting obtains
Each complete note position in the picture, as shown in figure 12.
The present embodiment randomly selects 2000 candidate's postings.
Preferably, the training of the convolutional neural networks in step 2024 in recognition methods the second embodiment of the present invention
Journey, as shown in figure 13, including,
Step 401: set up note symbol head data set, including solid symbol head, hollow symbol head and three kinds of categorical datas of background；
Step 402: as shown in figure 14, builds convolutional neural networks, including 2 convolutional layers, 2 down-sampling layers and 1 complete
Articulamentum；
Step 403: the symbol head view data accorded with by note in head data set is input in convolutional neural networks, completes instruction
Practice.
Note symbol head data set in the present embodiment includes 2000 solid symbol heads, 1500 hollow symbol heads and 4000 back ofs the body
Scape image.
The present embodiment uses caffe framework convolutional neural networks, caffe framework be one clear, readable high, quickly
Degree of depth learning framework, its model structure is simple, parameter is less, and (notebook, mobile phone etc.) has only to realize letter in many environments
Single convolution and the full feedforward network that connects can carry out note identification, it is not necessary to configuration caffe environment, very convenient letter separately
Single.
Preferably, in recognition methods the second embodiment of the present invention employing convolutional neural networks in step 2025 to segmentation
The note symbol head obtained is identified, as shown in figure 15, including,
The complete note obtained by note locating segmentation, is input in convolutional neural networks, by according with head data with note
Data Comparison in collection, draws it is solid symbol head, hollow symbol head or background, gives up background, simultaneously comparison note symbol head data
The position data of the symbol head in collection, determines the position according with head in complete note.
In actual application, can generate, according to the note information identified, the electronic music that can play, play out.
Using above-mentioned the second embodiment to carry out note identification, hardware is Samsung galaxy S3, and CPU tests, note
Recognition speed has reached 500fps, and accuracy rate is 98.71%.
It should be noted that in this article, the relational terms of such as first and second or the like is used merely to a reality
Body or operation separate with another entity or operating space, and deposit between not necessarily requiring or imply these entities or operating
Relation or order in any this reality.And, term " includes ", " comprising " or its any other variant are intended to
Comprising of nonexcludability, so that include that the process of a series of key element, method, article or equipment not only include that those are wanted
Element, but also include other key elements being not expressly set out, or also include for this process, method, article or equipment
Intrinsic key element.In the case of there is no more restriction, statement " including ... " key element limited, it is not excluded that
Including process, method, article or the equipment of described key element there is also other identical element.
Each embodiment in this specification all uses relevant mode to describe, identical similar portion between each embodiment
Dividing and see mutually, what each embodiment stressed is the difference with other embodiments.
The foregoing is only presently preferred embodiments of the present invention, be not intended to limit protection scope of the present invention.All
Any modification, equivalent substitution and improvement etc. made within the spirit and principles in the present invention, are all contained in protection scope of the present invention
In.
Claims (10)
1. the electronic equipment for musical score image identification, it is characterised in that include housing, sound-generating element, be arranged on housing
In mainboard and be arranged on the image scanning parts of described housing first ends；
Governor circuit and the sound card circuit electrically connected respectively and power circuit it is provided with governor circuit on described mainboard；
Described image scanning parts include scan roller and be arranged on scanning roller above photographic head, described scanning roller and
Photographic head all electrically connects with described governor circuit；The musical score image of shooting is sent at governor circuit by described photographic head
Reason；
Described sound-generating element is connected with described sound card circuit, and the acoustical signal sent by governor circuit sends sound；
Described power circuit electrically connects as its power supply respectively with described scanning roller, photographic head and sound-generating element；
The second end of described housing is provided with battery flat and hatchcover, and battery flat is connected with the power circuit on mainboard.
Electronic equipment for musical score image identification the most according to claim 1, it is characterised in that described housing is lip pencil
Housing；Described image scanning parts are arranged on the first end of lip pencil housing；
Described sound-generating element is arranged on above described image scanning parts, and described image scanning parts and sound-generating element make the first end
Portion is formed as nib shape；
Described mainboard is the position of close nib in being arranged on lip pencil housing；
At least 2 mainboard mounting posts it are provided with in described lip pencil housing；Described mainboard is solid by described at least 2 mainboard mounting posts
It is scheduled in lip pencil housing.
Electronic equipment for musical score image identification the most according to claim 2, it is characterised in that described lip pencil housing
The second end is provided with battery flat and hatchcover, and battery flat is connected with the power circuit on mainboard.
Electronic equipment for musical score image identification the most according to claim 2, it is characterised in that described lip pencil housing
The second end is provided with circumscripted power line, and circumscripted power line is connected with the power circuit on mainboard.
5. a musical score image recognition methods based on electronic equipment described in claim 1-4 any one, it is characterised in that bag
Include,
Obtain pending staff image by photographic head and pass to governor circuit；
Pending staff image is identified by governor circuit, identifies each complete note；
Governor circuit is according to the complete note identified, and the audio digital signal of transmission correspondence is to sound card circuit, and sound card circuit will
The audio digital signal received is converted into playable analogue signal, passes to sound-generating element and plays out；
Pending staff image is identified by described governor circuit, including,
Pending staff image use edge detection method depict the marginal information of image, then by straight-line detection side
Method detects five line position coordinates；
Use the note grader preset, pending staff image is carried out note locating segmentation, obtains each complete sound
Symbol position in the picture；
The note symbol head using the convolutional neural networks preset to obtain segmentation is identified, it is judged that be solid symbol head or hollow
Symbol head, and obtain according with the position of head；
The five line position coordinates, the relative position of each complete note that obtain according to described, it is solid symbol head or hollow symbol head
And the position of symbol head, identify each complete note.
Musical score image recognition methods the most according to claim 5, it is characterised in that the training of described note grader
Journey, including:
Setting up positive sample data set and negative sample data set, data set includes in the position data of posting and posting five
The view data of line spectrum image, positive sample data set is the view data including complete note, and negative sample data set is for including removing
Cross the view data that remaining music score outside complete note is likely to occur；
Extract the channel characteristics of each sample in positive sample data set and negative sample data set, train note grader.
Musical score image recognition methods the most according to claim 6, it is characterised in that described to pending staff figure
As carrying out note locating segmentation, including,
Pending staff image randomly selects several candidate's posting, one by one Scan orientation frames, to each location
The channel characteristics described in image zooming-out in frame, is input to the channel characteristics of extraction in note grader, it is judged that in posting
Image be positive sample or for negative sample, the complete note that positive sample is judged in music score, negative sample is judged to music score background
Give up, thus obtain the complete note in pending staff image, the position data of posting in comparison note grader
Obtain each complete note position in the picture.
Musical score image recognition methods the most according to claim 5, it is characterised in that the training of described convolutional neural networks
Process, including,
Set up note symbol head data set, including solid symbol head, hollow symbol head and three kinds of categorical datas of background；
Build convolutional neural networks, including 2 convolutional layers, 2 down-sampling layers and 1 full articulamentum；
The symbol head view data accorded with by note in head data set is input in convolutional neural networks, completes training.
Musical score image recognition methods the most according to claim 8, it is characterised in that described employing convolutional neural networks pair
The note symbol head that segmentation obtains is identified, including,
The complete note obtained by note locating segmentation, is input in convolutional neural networks, by according with in head data set with note
Data Comparison, draw it is solid symbol head, hollow symbol head or background, give up background, simultaneously comparison note symbol head data set in
Symbol head position data, determine the position according with head in complete note.
Musical score image recognition methods the most according to claim 5, it is characterised in that described pending staff figure
Picture, particularly as follows: staff image to be carried out the process that denoising, contrast enhancing, gray processing, minimizing noise or uneven illumination are even,
The bianry image obtained.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610859907.2A CN106297755B (en) | 2016-09-28 | 2016-09-28 | Electronic equipment and identification method for music score image identification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610859907.2A CN106297755B (en) | 2016-09-28 | 2016-09-28 | Electronic equipment and identification method for music score image identification |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106297755A true CN106297755A (en) | 2017-01-04 |
| CN106297755B CN106297755B (en) | 2023-06-13 |
Family
ID=57715584
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610859907.2A Active CN106297755B (en) | 2016-09-28 | 2016-09-28 | Electronic equipment and identification method for music score image identification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106297755B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107945780A (en) * | 2017-11-23 | 2018-04-20 | 北京物灵智能科技有限公司 | A kind of instrument playing method and device based on computer vision |
| CN108665888A (en) * | 2018-05-11 | 2018-10-16 | 西安石油大学 | A kind of system and method that written symbol, image are converted into audio data |
| CN108766463A (en) * | 2018-04-28 | 2018-11-06 | 平安科技（深圳）有限公司 | Electronic device, the music playing style recognition methods based on deep learning and storage medium |
| CN110796146A (en) * | 2019-10-11 | 2020-02-14 | 上海上湖信息技术有限公司 | Bank card number identification method, model training method and device |
| CN112133264A (en) * | 2020-08-31 | 2020-12-25 | 广东工业大学 | Music score recognition method and device |
| CN112925944A (en) * | 2021-03-10 | 2021-06-08 | 上海妙克信息科技有限公司 | Music score identification method, terminal equipment and computer readable storage medium |
| CN113076967A (en) * | 2020-12-08 | 2021-07-06 | 无锡乐骐科技有限公司 | Image and audio-based music score dual-recognition system |
| CN115019600A (en) * | 2022-01-17 | 2022-09-06 | 滁州职业技术学院 | Music staff recognizer and music recognition method thereof |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0997060A (en) * | 1995-09-29 | 1997-04-08 | Kawai Musical Instr Mfg Co Ltd | Musical score recognition device |
| CN1283832A (en) * | 1999-08-10 | 2001-02-14 | 曾平蔚 | Optical scan method and device for reading music score |
| JP2003242439A (en) * | 2003-02-07 | 2003-08-29 | Kawai Musical Instr Mfg Co Ltd | Musical score recognizing device |
| CN103646247A (en) * | 2013-09-26 | 2014-03-19 | 惠州学院 | Music score recognition method |
| CN105022993A (en) * | 2015-06-30 | 2015-11-04 | 北京邮电大学 | Stave playing system based on image recognition technology |
| CN206097909U (en) * | 2016-09-28 | 2017-04-12 | 北京邮电大学 | A electronic equipment for music book image recognition |
-
2016
- 2016-09-28 CN CN201610859907.2A patent/CN106297755B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0997060A (en) * | 1995-09-29 | 1997-04-08 | Kawai Musical Instr Mfg Co Ltd | Musical score recognition device |
| CN1283832A (en) * | 1999-08-10 | 2001-02-14 | 曾平蔚 | Optical scan method and device for reading music score |
| JP2003242439A (en) * | 2003-02-07 | 2003-08-29 | Kawai Musical Instr Mfg Co Ltd | Musical score recognizing device |
| CN103646247A (en) * | 2013-09-26 | 2014-03-19 | 惠州学院 | Music score recognition method |
| CN105022993A (en) * | 2015-06-30 | 2015-11-04 | 北京邮电大学 | Stave playing system based on image recognition technology |
| CN206097909U (en) * | 2016-09-28 | 2017-04-12 | 北京邮电大学 | A electronic equipment for music book image recognition |
Non-Patent Citations (1)
| Title |
|---|
| 刘晓翔等: "乐谱识别中音符结构分析方法" * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107945780A (en) * | 2017-11-23 | 2018-04-20 | 北京物灵智能科技有限公司 | A kind of instrument playing method and device based on computer vision |
| CN108766463A (en) * | 2018-04-28 | 2018-11-06 | 平安科技（深圳）有限公司 | Electronic device, the music playing style recognition methods based on deep learning and storage medium |
| WO2019205383A1 (en) * | 2018-04-28 | 2019-10-31 | 平安科技（深圳）有限公司 | Electronic device, deep learning-based music performance style identification method, and storage medium |
| CN108665888A (en) * | 2018-05-11 | 2018-10-16 | 西安石油大学 | A kind of system and method that written symbol, image are converted into audio data |
| CN110796146A (en) * | 2019-10-11 | 2020-02-14 | 上海上湖信息技术有限公司 | Bank card number identification method, model training method and device |
| CN112133264A (en) * | 2020-08-31 | 2020-12-25 | 广东工业大学 | Music score recognition method and device |
| CN112133264B (en) * | 2020-08-31 | 2023-09-22 | 广东工业大学 | Music score recognition method and device |
| CN113076967A (en) * | 2020-12-08 | 2021-07-06 | 无锡乐骐科技有限公司 | Image and audio-based music score dual-recognition system |
| CN112925944A (en) * | 2021-03-10 | 2021-06-08 | 上海妙克信息科技有限公司 | Music score identification method, terminal equipment and computer readable storage medium |
| CN115019600A (en) * | 2022-01-17 | 2022-09-06 | 滁州职业技术学院 | Music staff recognizer and music recognition method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106297755B (en) | 2023-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106297755A (en) | A kind of electronic equipment for musical score image identification and recognition methods | |
| CN106446952B (en) | A kind of musical score image recognition methods and device | |
| WO2020151489A1 (en) | Living body detection method based on facial recognition, and electronic device and storage medium | |
| CN106874826A (en) | Face key point-tracking method and device | |
| US8750573B2 (en) | Hand gesture detection | |
| CN103164692B (en) | A kind of special vehicle instrument automatic identification system based on computer vision and method | |
| CN104794479B (en) | This Chinese detection method of natural scene picture based on the transformation of local stroke width | |
| CN104202547B (en) | Method, projection interactive approach and its system of target object are extracted in projected picture | |
| CN112686812B (en) | Bank card inclination correction detection method and device, readable storage medium and terminal | |
| US20120027252A1 (en) | Hand gesture detection | |
| CN103955499B (en) | A kind of visual experience Enhancement Method based on instant computing and dynamic tracing | |
| CN104166841A (en) | Rapid detection identification method for specified pedestrian or vehicle in video monitoring network | |
| CN106056064A (en) | Face recognition method and face recognition device | |
| CN106934351B (en) | Gesture recognition method and device and electronic equipment | |
| CN102713938A (en) | Scale space normalization technique for improved feature detection in uniform and non-uniform illumination changes | |
| CN109871845B (en) | Certificate image extraction method and terminal equipment | |
| CN104281839A (en) | Body posture identification method and device | |
| CN110378946A (en) | Depth map processing method, device and electronic equipment | |
| CN103034838A (en) | Special vehicle instrument type identification and calibration method based on image characteristics | |
| CN106204658A (en) | Moving image tracking and device | |
| CN103345644A (en) | Method and device for detecting online-training targets | |
| Su et al. | A novel forgery detection algorithm for video foreground removal | |
| CN103218603A (en) | Face automatic labeling method and system | |
| CN107767358B (en) | Method and device for determining ambiguity of object in image | |
| CN110400338A (en) | Depth map processing method, device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |