GB2197742A - Indentifying instrumental sounds - Google Patents

Abstract

Method and apparatus for indentifying instrumental sounds, especially for teaching the playing of musical instruments. A microphone (2) attached to the instrument (1) provides signals which are squared in a signal processor (3) and applied to a computer (4). The squared signals strobe clock pulses to provide frequency counts related to pitch. These are processed to obtain best estimate values and applied to a state table to recognise notes and breaks. Attack and decay periods of notes are recognised and allowed for. Instead of a musical instrument, a cyclically operating machine or an electric motor may be monitored using the microphone (2). <IMAGE>

Description

METHOD OF AND APPARATUS FOR IDENTIFYING INSTRUMENTAL SOUNDS.

The invention relates to the identification of instrumental sounds, and in its preferred form is applicable to apparatus for teaching the playing of musical instruments. However, it is not limited to the recognition of musical notes, but may be applied, for example, to the analysis of the sounds produced by machinery, such as an electric motor, to give early warning of overload or incipient failure.

In identifying the notes produced by a musical instrument two main problems arise. In the first place, the notes of musical instruments (and, of course, of other equipment) do not consist of a pure tone, but contain a substantial harmonic content. As a result it may be difficult to determine from the wave form of the note the frequency which would be recognised by the ear as being the pitch of the note. A second problem is that the harmonic composition of the note may change considerably through its duration. For example, in stringed or percussion instruments there will be initial transients which die away fairly rapidly, and in wind instruments such as a recorder, the note may start with a predominating second harmonic content, so that the apparent pitch is doubled for the first few hundreths of a second of each note.

These problems have not hitherto been satisfactorily solved. In particular, the recognition of pitch has hitherto usually been carried out by Fourier analysis, which requires either complex and expensive hardware, or computer programmes requiring substantial amounts of memory.

The present invention provides a simple and reliable method of identifying an instrumental note, and also apparatus for carrying out such a method. The apparatus may be designed for use in teaching the playing of an instrument, and may consist of a simple signal processor attachable to a micro computer of modest memory size programmed to analyse the sounds of the instrument being played and to produce a display indicating the played notes, or showing how far they correspond to, or deviate from, a musical score.

The method of the invention comprises the steps of: (a) Converting the sound to be analysed to an electrical signal.

(bY Processing the signal to provide a square wave whose period is related to the pitch of the iritial sound.

(c) Applying the square wave signal to gate clock pulses so as to obtain a succession of counts characteristic of the pitch of the initial sound, (d) Determining the pitch of the sound and the duration of tone sections during which the pitch count remains substantially constant, (e) Addressing a stored state table with the pitch and duration of the tone sections to identify notes and pauses between notes, and (f) Producing output signals representing the pitch and duration of the notes so identified.

Preferably, the pitch count is processed to obtain a 'best' value. This may be done by combining a number of successive counts into a group, determining the range and mean of each group, and then determining the best value in accordance with the following rules: (i) if any count within the group exceeds a preset upper limit the pitch is set to an arbitrary value representing silience; (ii) if the range is within a preset maximum the pitch count is taken as the mean of the group; (iii) if the range lies outside the maximum the median is compared with a moving average of previous pitch counts, and if the difference lies below a preset limit the median is taken as the best estimate; if the median differs from the moving average by more than the preset linear the pitch counts are examined for a periodic recurring pattern, and the total count of one period of the pattern is compared with the moving average; if the difference is within a preset limit the total count of the period is taken as the best estimate.

(iv) in any other case the moving average is taken as the best estimate.

There will be occasions when no best estimate can be determined. This will happen when the signal source is silent, or is producing random noise, and in such cases it may be convenient to set the pitch arbitrarily to zero.

The moving average is computed by taking the previous pitch average, multiplying it by a constant, K, adding the most recently determined pitch, and dividing the sum by (K+I). The constant K determines the response time of the averaging process.

To identify the pitch change which determines the end of a tone section, the difference between the current pitch average and the previous pitch average may be taken as representing the rate of change of pitch, and may be compared with a constant representing a maximum allowed rate. If rcls tnaxinun is exceeded, the current tone section is ended.

A preferred form of apparatus according to the invention comprises equipment for teaching the playing of a musical instrument. The musical instrument is provided with a microphone coupled to a signal processing circuit which is attached to the interface of a personal computer such as the BBC model B computer. The computer is provided with programmes for determining the best estimate of the pitch count from the signal provided by the processing circuit, and also contains the state table. It may also contain a further stored table corresponding to the score of a piece of music to be played, and means for generating a display indicating whether the instrument has been played correctly, that is to say in accordance with the music represented by the stored table.

Music teaching apparatus according to the invention will now be described by way of example with reference to the accompanying drawings, in which:- Figure 1 is a schematic diagram of part of the hardware employed in carrying out the invention; Figure 2 is a circuit diagram of a signal processor; Figure 3 is an overall flow chart showing the routine for recogisicg notes; Figure 4 is a detail flow chart of the procedure for determining the best pitch estimate of a note; and Figure 5 shows the contents of the state table stored in the computer for the purpose of identifying notes.

The apparatus to be described is designed for teaching young children to play the recorder, this being the instrument most commonly taught in schools. It is described as being used with a BBC micro-computer, but the computer to be used is a matter for the choice of the programmer and the invention may be carried using almost any home computer or personal computer, or from a timesharing terminal on a main-frame computer.

As shown in Figure 1, a recorder 1 carries a plastic clip 2 bearing an electret microphone.

This is connected to a signal processor unit 3 which plugs directly into the input port of a BBC micro-computer 4. The micro-computer is programmed to identify the notes played, as will be described in more detail, and to produce an appropriate display on a VDU scree (not shown).

Figure 2 is a circuit diagram of a circuit suitable for the signal processor unit. It comprises a microphone input with correction for the microphore characteristics, a low pass filter, two stages of amplification and a Schmitt trigger circuit producing a square wave output to be fed into the computer.

Figure 3 is a flow chart of the procedure for identifying a note played on the recorder shown in Figure 1. The square wave input from the signal processing unit strobes a series of micro-second clock pulses to obtain counts related to the pitch of the note, and these are stored in a buffer. The stored pitch counts are then processed to determine a best estimate of the pitch count, and consecutive pitch estimates are compared to determine whether they agree within preset limits, and to detect their changes. Pitch count groups of constant pitch are combined to form note sections whose pitch and duration are stored in a second buffer.

These stored pitch and duration data of the tone sect ions are then applied to a state table which derives the pitch and duration of the notes represented by the tone sections and the durations of rests or longer pauses between notes. The information so derived is stored, and may be displayed in suitable form or it may be compared with a further stored table representing expected values of note duration and pitch as derived from a musical score, the results of the comparison being displayed in suitable form so that the learner is informed whether or not he or she is playing correctly.

Figure 4 is a flow sheet of the procedure for obtaining a best estimate of the pitch count and duration. A set of 8 consecutive counts is taken (10) and each is compared in turn with a preset constant t/break (11). If any one of the counts exceeds this value, it is assumed that a relatively long period of silence has occurred representing a break between notes and the pitch value is arbitrarily set to zero (12).

If all of the set of counts lie below the limit then their range and mean are calculated (13) and the range is compared with a preset maximum limit (14). If the range lies within this limit the pitch is calculated from the mean (15). If the range lies outside the limit, then the median is calculated (16) and the pitch is calculated from the median (17). This calculated pitch is compared with the current time average (18) to determine whether it lies within a preset limit. If it is within limits the value calculated from the median is taken as the pitch. If it is outside the limit the consecutive counts are examined to determine w h e t Th e r they show a recurrir.g patterr, arc the pitch is calculated from the overall count of one repeating period of this pattern (19).This calculated pitch is then compared with the time average pitch, and if it lies within limits it is taken as the new value of the pitch (20). If it lies outside limits then the current average pitch is taken as the new pitch value (21). The pitch value so obtained is continously averaged with a previous average value to form a continous moving average, unless the pitch value is zero, in which case the average is set to zero. This average value is used in steps 18,20 and 21 described above. The current pitch is also compared with the average value to determine the rate of change of pitch. While the deviation from average remains within a preset limit, the duration of the note is summed at 22. When the rate of change exceeds the limit, then the previous average pitch value and the summed duration are stored as representing the pitch and duration of a tone section.

The pitch and duration values of the tone sections are then applied to a stored state table, the contents of which are shown in Figure 5. The function of this table is to combine the tone sections into complete musical notes. A typical note is assumed to consist of four sections: (a) Break : consisting of long time intervals. The gap between notes.

(b) Attack : the start of a note; an onset transient.

(c) Note : the main body of the note.

(d) Decay : the termination of the note which might be prolonged, for example by reverberant room acoustics.

Each tone section derived from the previous processing is defined by two variables, its timeaveraged pitch, p and its duration d.

The state table determines whether or not a tone section should be added into a note by comparing it with its immediate predecessors.

At any time, the state table as shown in Figure 2 will have a current state determined by previous tone sections. This current state will be either break, attack, note or decay.

(a) Pitch comparison When a tone section is received its pitch p is compared with the current state pitch P.

The comparison will lead to one of the following results: (i) Z : pitch p is zero (i) N : pitch p is 'near' to the current state pitch P. This is judged as follows: pitch p is defied to be ear if I (p - P)l *f p where the factor, f may be chosen in the range 10-30 depending upon instrument characteristics.

(iii) F : pitch is 'far' from the current state pitch P.

The length of time for which a state has existed is important. The total state duration is the sum of the duration of previous entries to the state (D) plus the current duration d. This duration is compared with three empiricallydetermined constants.

Results, as shown in Figure 2, are described by the letters G, D, A and S, defined as follows: (i) G : total duration D+d is greater than or equal to a predetermined time interval : the maximum gap allowed within a note (a gap being a short absence of signal) (200-700 ms) (ii) D : D+d = maximum decay duration (120-550 ms) (iii) A : D+d = maximum attack duration (40 ms) (iv) S : D+d maximum attack duration (40 ms) Referring now to Figure 5, the first three columns indicate the current state, the pitch copariso results and the duration comparison results.

The fourth column indicates whether or not a note of pitch P or a break between notes (each of total duration shown in brackets) should be reported (the pitch and duration being stored in a rotary buffer).

Note that D' is the duration of the previous state and D is the duration of the current state.

Column. 5 defines the new state which is the result of the comparisons. As before, it may be either break, attack, note or decay.

The final three columns define the updated values of D (current state duration), D' (the previous state duration) and P (the current state pitch). If no entry is shown in any of these columns for any particular state then no change has been made to the state variables.

On receipt of a new pair of tone section p and d parameters, the new state parameters P and D become current and the pitch and duration comparisons are made as before.

Comments (a) Break State Total duration of state = D The duration of the new event = d The end of a break is only signaled if a o- zero pitch p is recorded and if this lasts for a significant length of tine. he cannot distinguis "near" and "far" pitch comparisons since the break pitch is defined to be zero. For this reason N and F are grouped together.

(b) Attack State Total duration of state = D + d D' will be the duration of the previous break.

Short durations will lead to a continued attack state. Long periods of tone will initiate the note state whether the pitch comparison is "near" or far".

(c) Note State Total duration of state = D + d When in the note state, any new input will end the state (with attack, decay, break or possibly a new note). For this reason durations do not combine cu.mulatively into the note state.

(d) Decay State Total duration of state = D + d D' is the duration of the preceding note.

The table stays in decay state until there is a big pitch jump or (d+D) becomes longer than the maximum decay length. Gaps in the tone will be tolerated in decay if the total time is small enough.

The values of pitch and duration of the notes reported from the state table are stored in a buffer and may be displayed directly, preferably in a pictorial format, or they may compared with a further stored table representing desired pitch durations and rests in accordance with a musical score of the piece which the user is attempting to play. The results of the comparisons may then be displayed in suitable form.

The same principles may be applied to functions other than the recognition of musical notes. For example, the functioning of machinery may be monitored by means of a suitably positioned microphone. The state table would have to contain entries enabling it to recognise normal conditions of operation and abnormal or fault conditions. For example, in the case of an electric motor, the sound in normal operation might be a note whose pitch depended on the loading of the motor. A note of pitch outside a prescribed range might indicate either overloading or a runaway condition due to loss of load. By taking account of the duration of the sound at various pitches, excessive periods of operation under heavy load could be detected.

Similarly, in the case of machinery operating in a repeated cycle, the cycle of sounds corresponding to normal operation could be programmed into the 5 t e table ar.d sigr.ificar.t deviatiors i n pitch or tining could be recognised.

Claims (7)

CLAIMS:

1. A method of identifying instrumental sounds comprising the steps of: a. converting the sound to an electrical signal; b. processing the signal to provide a square wave whose period is related to the pitch of the sound; c. applying the square wave signal to gate a clock pulse signal so as to obtain a sucession of counts characteristic of the pitch of the sound; d. determining from the pitch counts so obtained, the pitch of the sound and the duration of tone sections during which the pitch count remains substantially constant; e. addressing a stored state table with the pitch and duration of the tone sections to identify notes and pauses between notes; and f. producing output signals representing the pitch and duration of the notes so identified.

2. A method according to Claim 1 in which successive pitch counts are grouped and processed to obtain an improved estimate of the pitch count.

3. A method according to Claim 2 in which the groups of pitch counts are processed according to the following rules: (i) if any count within the group exceeds a preset upper limit the pitch is set to an arbitrary value representing silence; (ii) if the range is within a preset maximum the pitch count is taken as the mean of the group; (iii) if the range lies outside the maximum the median is compared with a moving average of previous pitch counts, and if the difference lies below a preset limit the median is taken as the best estimate; if the median differs from the moving average by more than the preset limit the pitch counts are examined for a periodic recurring patter, and the total count of one period of the pattern is compared with the moving average; if the difference is within a preset limit the total count of the period is taken as the best estimate.

(iv) in any other case the moving average is taken as the best estimate.

4. A method according to any preceding claim employing a stored state table storing data related to the four following states respectively: (i) silence, (ii) an initial phase at the commencement of sounding of a note, (iii) the sounding of a note, (iv) a period of sound decay following the cessation of a note.

5. A method according to any preceding claim in which the preset limits in the pitch estimate determining algorithmn and the initial phase maximum duration of the state table are chosen with reference to the characteristics of a descant recorder.

6. Apparatus comprising a microphone, a signal processor connected to the microphone and arranged to produce an output of square wave form whose period is related to the pitch of the sounds received by the microphone, and a data processor storing data processing algorithmns, and a state table arranged to process the signals received from the signal processor by a method according to any preceding claim and produce a display representing the pitch and duration of the notes.

7. Apparatus according to Claim 6 in which the processor further stores a table representing the pitch and duration of notes to be played according to a specified musical score and produces a display indicating whether the notes played are in accordance with the stored data.