JPH03249799A - Sheet music recognizer - Google Patents

Abstract

PURPOSE:To input the contents of a sheet music as musical performance information with a high recognition rate by analyzing a sheet music symbol by using a neural network. CONSTITUTION:In a ROM 3, a control program is stored, and to a RAM 4, various registers for processing an inputted sheet music graphic are set. A neural network 5 is provided with an input layer for receiving the data of a normalized sheet music graphic to a binary matrix, and an output layer consisting of seven pieces of neurons corresponding to seven kinds of sheet music symbols. In this case, based on binarized data of an input graphic received by the input layer, the neural network infers which sheet music symbol this input graphic is, and is learned in advance so that the output layer neuron corresponding to the sheet music symbol concerned outputs a large value. In such a way, the sheet music graphic can be recognized with a high recognition rate.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of Industrial Application This invention relates to a musical score recognition device that recognizes the contents written on musical scores.

M) Conventional technology automatic performance devices have been put into practical use. These devices are equipped with means for storing information (performance information) related to musical instrument performance, and by reading out the performance information stored in this means, it is possible to automatically perform an electronic musical instrument or the like. Performance data has conventionally been input to this automatic performance device using a real-time method, a step method, or the like. The real-time method is a method in which the musical instrument is actually played and the performance information is written. Furthermore, the step method is a method in which the pitch and length of each note are entered numerically.

(C) Problems to be Solved by the Invention However, such conventional systems have the following drawbacks. The real-time method has the disadvantage that the number of people who can input data is limited because it is impossible to input data unless one can actually play the instrument, and the content of the input information is influenced by the skill of the player. On the other hand, the step method has the disadvantage that it takes time to input the pitch and length of each note of the musical score.

An object of the present invention is to provide a musical score recognition device that can input performance information with a higher recognition rate for the contents of musical scores by analyzing musical score symbols using a neural network.

(d) Means for Solving the Problems The first invention of this application includes an input means for inputting musical score figures, a normalization means for converting the input musical score figures into a fixed binary matrix, and a normalization means for converting the input musical score figures into a fixed binary matrix. The present invention is characterized in that it is provided with a neural network that receives the binary matrix outputted by the means as input data and analyzes musical score symbols that match the musical score figure.

A second invention of this application includes an input means for inputting musical score figures, a normalization means for converting the input musical score figures into a fixed binary matrix, and a binary matrix outputted by the normalization means as input data. a neural network that analyzes musical score symbols that match the musical score shapes, a pitch analysis means that analyzes pitches from the positions of the musical score shapes, and performance information based on the analyzed musical score symbols and pitches. The present invention is characterized in that it includes a means for outputting.

(e) Effect of the Invention The musical score recognition device of the present invention converts musical score figures input from the input means into a fixed binary matrix. As a binary matrix, for example, a 16×1 unmarked portion is binarized. The matrix binary data generated in this manner is input to a neural network having an input layer consisting of the same number of neurons as dots in this matrix. As described above, this neural network has the same number of input layers as dots in the matrix, an intermediate layer consisting of a predetermined number of neurons, and
It has an output layer consisting of the same number of neurons as musical score symbols to be recognized. Musical score symbols recognized by this neural network include, for example, circles for musical notes, flags, and figures for rests. This neural network infers which music score symbol this input shape is based on the binarized data of the input shape received by the input layer, and instructs the output layer neuron corresponding to the corresponding score symbol to output a large value. has been learned in advance. By determining which output layer neuron has a larger output, it is possible to recognize which symbol a handwritten musical score figure corresponds to.

Furthermore, in the second invention of this application, a means is provided for analyzing the pitch of the analyzed musical score symbol (note) based on the position IF (mainly the upper and lower positions) in which the musical score figure is written. This means includes means for determining the pitch by determining the center of gravity of the musical score figure and determining the position of the center of gravity on the staff (preset). Performance information can be obtained from the input musical score figure by comprehensively determining the pitch determined by this means and the musical score symbols analyzed based on the neural network.

(f) Embodiment FIG. 1 is a block diagram of a control section of a musical score recognition apparatus which is an embodiment of the present invention. This device is controlled by a CPU 1, which is connected to each circuit section via a bus 2. Bus 2 has ROM3, RAM4. Neural network 5. Reading device 6. keyboard? , CRT 8 and MIDI interface 9 are connected. R
The OM3 stores a control program as shown in a flowchart to be described later, and the RAM4 has various registers set therein for processing input musical score figures.

The neural network 5 has an input layer that receives musical score figure data normalized to a binary matrix, and an output layer that is made up of seven neurons corresponding to seven types of musical score symbols (see FIG. 3). An intermediate layer composed of an appropriate number of neurons is formed between the input layer and the output layer. The neurons in the input layer, middle layer, and output layer are connected by synaptic connections, and 7
Each output layer neuron is recognized as a symbol of a musical score figure. Here, the seven types of musical notation symbols are whole notes,
White circles representing half notes, black circles representing quarter notes, etc., flags representing eighth notes, flags for downward notes, quarter rests, eighth rests, whole rests, half rests, etc. It is a black square.

Further, the reading device 8 used as an input device can be any device as long as it is a means for inputting figures. For example, a mouse or digitizer used for inputting musical score figures by hand, an image scanner for reading pre-drawn (printed) musical scores, etc. can be used.

Further, the MIDI interface 9 is connected to a sound source device for producing sounds of the read musical score, a sequencer for accumulating and storing performance information, and the like.

The functions and operations of this musical score recognition device will be explained with reference to the explanatory diagrams of FIGS. 2 and 3 and the flowchart of FIG. 4. In this embodiment, a mouse and a digitizer will be used as an example of the reading device 6. When a figure is input by inputting the coordinates from the pen-on to the pen-off of the musical score from the reading device 6 (nl), the maximum, minimum x, y coordinates of the figure (the maximum value, the minimum value of the X coordinate of the figure, and (maximum, minimum value of y coordinate) (n2; see Figure 2), divide the rectangle surrounded by these coordinate lines into 16x16 matrices (n3) - Therefore, depending on the size of the figure, each one of the matrices The size of ×1 square also differs.

The density is detected for each square, and a square with a high density is assumed to be a square on which a figure is drawn (black dot), and 1 is stored in the corresponding storage area (n4). The binarized and interpolated 256 (16x16) matrix data is output to the neural network 5 to infer which symbol this figure represents (n5: see Figure 3)
. The neural network 5 performs inference based on this matrix data, and outputs the likely degree of each symbol from each neural network in the output layer.

The CPUI receives this inference result (n6) and determines which musical score symbol the drawn figure is (nl). - For boat fishing, the symbol is determined to correspond to the output layer neuron with the largest output, but in addition to this, the symbol is determined to be valid based on the arrangement of symbols and the rules for writing musical scores. For example, notes and rests that exceed the number of beats are not written in one measure, and notes with white circles generally do not have flags representing notes below eighth notes. It will be done. If it is determined that the symbol of the read note is a musical note, the position where the note (black circle, white circle) is drawn is detected (n9), and the pitch is determined based on this (nlo). . In this case, if the note is written above or below the staff, the pitch is determined by also determining the number of separate lines between the note and the staff. Note that the position of the note is determined as the center point of the black circle and the white circle, but the center point may be an intermediate value between the maximum coordinate and the minimum coordinate, or the center of gravity may be used.

Based on the above data, event data based on MIDI code is generated (nil). MIDI this data
It outputs to the sound source device or sequencer via the interface (n12), and if it is determined that the figure read at nl is a rest, it proceeds directly from n8 to nil.

As described above, the performance information can be analyzed from the musical score.

In the above embodiment, the matrix is 16×16, but the size may be arbitrarily determined. However, if the matrix is made too large, there is a risk that the effect of blurring (trembling) of handwritten strokes will increase and the recognition rate will decrease, and if the matrix is made too small, the pattern may be distorted. That is, if the pattern is made small enough not to be distorted, it is possible to construct a high-speed neural network that has a high recognition rate, is easy to learn, and is fast.

In addition, in the above embodiment, a handwriting input device such as a mouse was used as the reading device, so the switch is turned on and off for each stroke (the action of drawing an individual figure such as a circle or vertical bar), making it easy to identify the figure. When a pre-drawn musical score is read and recognized using a scanner or the like, an algorithm is required to separate individual figures.

(g) Effects of the invention As described above, according to the musical score recognition device of the present invention, input musical score figures are analyzed using a neural network, which is superior to conventional algorithms such as stroke analysis. Musical score figures can be recognized with a high recognition rate. Therefore, you can easily input performance information by handwriting the score using a mouse, etc., or inputting performance information by reading the score directly. Input eliminates the need to enter data. As a result, even those who cannot play musical instruments can easily input performance information.

[Brief explanation of drawings]

FIG. 1 is a block diagram of a control unit of a musical score recognition device that is an embodiment of the present invention, FIGS. 2 and 3 are diagrams illustrating a method of analyzing symbols based on figures read by the musical score recognition device, FIG. 4 is a flowchart showing the operation of the musical score recognition device. 5-neural network, 6-reading device.

Claims (2)

[Claims] (1) An input means for inputting a musical score figure; a normalizing means for converting the input musical score figure into a fixed binary matrix; A musical score recognition device comprising: a neural network that analyzes musical score symbols that match a figure; (2) input means for inputting musical score figures; normalizing means for converting the input musical score figures into a constant binary matrix; and receiving the binary matrix outputted by the normalizing means as input data, a neural network that analyzes musical score symbols that match a shape; a pitch analysis means that analyzes pitch from the position of the musical score shape; and a means that outputs performance information based on the analyzed musical score symbol and pitch. A musical score recognition device characterized by: