WO1990015405A1

WO1990015405A1 - Method for producing a code sequence, particularly a note code sequence

Info

Publication number: WO1990015405A1
Application number: PCT/FI1990/000151
Authority: WO
Inventors: Teuvo Kohonen
Original assignee: Teuvo Kohonen
Priority date: 1989-06-06
Filing date: 1990-06-05
Publication date: 1990-12-13
Also published as: AU5673990A; JPH04505970A; FI84670B; FI892764A; FI892764A0; JP3290652B2; FI84670C

Abstract

The invention relates to a method of producing a digital code sequence, particularly a note code sequence from a finite number of different code types, each representing one or more quantized properties of, for instance, a predetermined note, wherein new codes are generated one at a time after the code sequence on the basis of the existing codes of the sequence. The present invention utilizes rule propositions produced on the basis of mutual local equivalencies between symbols occurring in an example material. A new code is attempted to be produced first on the basis of the code produced last and then more and more new codes are tested to find on the basis of them a rule proposition which unambiguously gives the code sequence of the new code.

Description

Method for producing a code sequence, particularly a note code sequence

The invention relates generally to a method of forming a digital code sequence from a finite number of different code types, wherein new codes are gen¬ erated one at a time after the code sequence on the basis of the existing codes of the sequence. More precisely, the method is intended for automatically composing computer music, each code representing one or more quantized properties of a predetermined note.

Musical productions that sound agreeable but lack the form of an independent work of art are often used as background music in films, plays and other presentations. The quantitative need of this kind of music may be considerable. Quiet music is also used extensively in shops and other public premises to entertain customers and to create a desired atmosphere. One way of producing such background music is to use an electric device generating so- called synthesized music. Such devices comprise one or more electronic musical instruments or syn¬ thesizers and an automatic device producing control signals for them. One way of producing such control code se¬ quences and signals is to use so-called artificial intelligence programs utilizing heuristically rules based on musical expertise. The present invention, however, is not concerned with this kind of expert methods but with a device forming the required rules automatically on the basis of example material and producing new code sequences automatically with the aid of these rules.

One prior art device producing control signals is based on Markov processes, in which each note (pitch, duration) is treated as a single stochastic state in a sequence of states. If example material, that is, note material, is given, the probability Pr of a state S_j_ in the sequence is

S_i_₂,-.) when the preceding states in the sequence are S_i_₁, ~^±-2 ~ ^e'^tc _' Three preceding states are often suffi¬ cient to achieve a satisfactory outcome in music based on Markov processes. New music is generated by probability functions stored in the memory, starting from a key sequence to which is added a successor state having the highest probability on the basis of the probability function Pr and, e.g., the last three notes or states in the sequence. The sequence so in¬ creased is used as a new key sequence so that the process generates endlessly note code material and control signals for electronic musical instruments or synthesizers. Moreover, additional operations or rules are needed to produce typical musical struc¬ tures from melody parts. This prior art method of generating note codes requires large amounts of training material to form conditional probability density functions. In addition, synthesized music produced as described above does not usually comprise any surprise element and is monotonous, since each note has the same value in a stochastic process, whereas the same is not true with the properties of natural music.

The object of the present invention is to provide a method which increases melodic variation and avoids or alleviates certain problems associated with the prior art.

This is achieved by means of a method according to claim 1.

The method of the invention utilizes the prin- ciple of dynamically expanding context in the produc- tion of a continuous sequence of codes. This principle has previously been applied in speech re¬ cognition (see [1] Dynamically expanding context, with application to the correction of symbol strings in the recognition of continuous speech, Teuvo Koho- nen, Proceedings of the Eighth International Con¬ ference on Pattern Recognition, October 27-31, 1986, Paris, France (IEEE Computer Society) p. 1148-1151. The present method differs from speech recognition mainly in that in the last-mentioned the method is used primarily for correcting codes whereas the present method creates continuously new stochastic sequences of codes.

Similarly as in Markov processes, a code in a sequence of codes is defined in the present method on the basis of codes immediately preceding it. The present invention, however, uses discrete "gram¬ matical" rules in which the length of the contents of the premises of the rule propositions, that is, the number of required preceding codes, is a dynamic parameter which is defined on the basis of discre¬ pancies occurring in the example sequences when the rule propositions are being formed from the example sequences. In other words, if two or more rule pro- positions have the same premises but different con¬ sequences, that is, a new code, during the production of the rule propositions, these rule propositions are indicated to be invalid, and the length of their pre¬ mises is increased until unambiguous or valid rule propositions are found. However, all such shorter, mutually discrepant invalid rule propositions are also maintained and formed into a tree structure as described below; the method of dynamically expanding context is to a very great extent based on the util- ization of this structure. As the above-mentioned rules are produced mechanically on the basis of local equivalences between symbols occurring in the example material, the production of rules does not, for in¬ stance, require music theoretical analysis based on expertise on the example music material.

Correspondingly, when the rule propositions are utilized to generate a new code after a sequence of codes, the code generated last in the code sequence is first compared with the rule propositions in a search table, then the two last codes are compared, etc., until an equivalence is found with the premises of a valid rule proposition, whereby the code in¬ dicated by the consequence of this rule proposition can be added last in the sequence of codes. The above-mentioned tree structure enables systematic comparisons. This results in an "optimal" sequence of codes which "stylistically" attempts to follow the rules produced on the basis of the example sequences. When the method is applied as such to produce a sequence of note codes, the produced music is in the desired style but may still contain rather long copied portions of the example material.

Variety and surprising changes can be produced in the sequence of codes by using random choice at least intermittently. In other words, after a valid rule proposition has been found, it is replaced ran¬ domly with an invalid rule proposition associated with it and having the same premises as the found rule proposition decreased with a random number of codes.

A code produced automatically by means of the method of the invention can be utilized for the control of electronic musical instruments or syn¬ thesizers either directly or converted into suitable control signals complying with the MIDI standard, for instance.

The invention will now be described in greater detail by means of embodiments and with reference to the attached drawing, in which Figure 1 illustrates a code structure applic¬ able in the method of the present invention;

Figure 2 shows a search table to be used in the method of the invention; and

Figure 3 illustrates a tree structure formed by interconnected rule propositions.

In the preferred embodiment of the method of the invention, individual code types represent notes which are here chosen to represent the smallest used musical units; alternatively, the codes may represent other quantities which can be represented by quantized states. In the present invention, a note is described by two or more quantized properties of a tone, such tone pitch and duration. Figure 1 illustrates one preferred 16-bit code structure in which the seven least significant bits represent tone pitch k which may thus have 128 different values of which one may indicate a rest. The seven following bits represent tone duration p which may also have 128 different values. Finally, the two most significant bits represent the beat phase of the note, that is, the position of the note in a time or time section. In the four-four music used in the present example it may thus have four different states. Figure 2 illustrates the structure of the search table to be used in the method of the inven¬ tion, which search table is stored in the memory. The search table consists of rule propositions each one of which comprises premises X, a consequence Y and a discrepancy flag Z. The state 0 of the flag Z indicates a valid proposition, and the state 1 indicates an invalid proposition.

The principles of the production of rule propositions for the search table of the invention are discussed generally in the above-mentioned article [1] . The invention applies a special case of the procedure described in the article, in which only the preceding codes on the left side of the code to be treated are taken into account when producing rule propositions.

The procedure can be illustrated by a simple example sequence in which letters represent code types

ABCDEFG...IKFH...LEFJ... If one now attempts to deduce a subsequent code solely on the basis of, for instance, the code F (which occurs several times), a threefold discrepancy will occur. The code could be followed by any of the codes G, H or J. If the symbol preceding the code F is included in the contents of the premises in an attempt to increase the precision of the code patterns, there still exists a twofold discrepancy: the combination EF could be followed by G or J. The combination KF, however, already forms a valid rule proposition at its point, giving as an outcome the unambiguous code H. In cases where F is preceded by E, two symbols preceding F can be included in the premises, whereby all discrepancies can be solved. For these points, two valid rule propositions can be formed. In one of them the premises are DEF and the consequence is G while in the other the premises are LEF and the consequence is J.

Invalid rules produced during the production of rule propositions are not, however, destroyed because they are needed both in the construction of the rule tree to be described below and in the production of new codes in the sequence of codes. Instead, each produced rule is indicated to be valid or invalid by the above-mentioned discrepancy flag Z. Rule proposi- tions are similarly sought separately for each code in the example sequence. In this way a search table is formed which contains valid and invalid rule pro¬ positions with premises of varying length.

The information structure of the rule proposi- tions stored in the search table is illustrated in Figure 3 by a graphical representation interconnect¬ ing the rules. For each predetermined code (such as F) there is a tree the root of which is formed by a rule proposition with premises containing this par- ticular code only. If the example sequences contain discrepancies, at least two branches extend from the root. The branches lead to nodes in which the premises of the rule proposition contain some other code in addition to the code F. The consequences of the branches are written beside the branches. The last nodes of the tree, representing leaves, cor¬ respond to the final valid rules, whereas all the other nodes correspond to invalid rules.

The following discussion deals with the genera- tion of a new symbol to a code sequence. Assume that the initial sequence is CDEF. A key sequence growing by degrees and produced on the basis of one or more codes produced last in the initial sequence is utilized in the search of a new code. Initially the key sequence is always formed by the last code of the initial sequence, in this particular case by F. When the initial sequence is now compared with the search table of Figure 2, it is found that a rule proposi¬ tion with this kind of premises has the state 1 of the discrepancy flag, so it is invalid. Thereafter the code E preceding the last code F in the initial sequence is added to the beginning of the key sequence, so that the key sequence now has the length of two codes, being EF. When comparing with the search table of Figure 2, it is to be seen that this search too leads to an invalid rule. The third last code D in the initial sequence is then added to the key sequence, whereby the key sequence becomes DEF. The search performed by this key sequence gives the valid rule in the memory position 13, and the code G indicated by the consequence of this rule is added to the end of the initial sequence. This increased code sequence is now used as a new initial sequence, whereby the first key sequence in the generation of a new code contains the code G. The length of the key sequence is again increased until a valid rule is found.

The above-described basic method easily results in a code sequence which forms copies of example material portions and may even start to repeat it¬ self.

For this reason, the code indicated by the found valid rule proposition is not always selected as the new code in the preferred embodiment of the invention. Instead, a kind of random choice is used in which the extent of the changes is adjustable. The generation of such new, partially random sequences can be illustrated by means of the preceding example and Figure 3. Assume again that the initial sequence is CDEF, and a valid rule proposition is searched in accordance with the above example. The search thereby begins from the root F of Figure 3 and follows the branches until an equivalent leaf, in this particular case DEF, is found. In the above example, the code G indicated by the found leaf DEF was selected as a new code. On the path from the root to the leaf, however, there are possibly several nodes which contain in¬ valid propositions giving various alternative new codes. In the present method the last key sequence, by which the valid rule proposition was found, is shortened at random at the most by a predetermined number of codes, and one of the invalid rule proposi¬ tions having premises equivalent to the shortened key sequence is selected as a new code. In other words, at the most a limited number of steps are taken at random to return from the leaf of the rule tree. In this way, random variation is produced in the obtained code sequence. In the case of a note sequence, for instance, this gives the produced music variety and surprising changes while it still con¬ forms to certain rules.

As mentioned above, both the key sequence and the premises of the rule propositions are in the pre¬ ferred embodiment of the invention formed solely by basic codes. Even though the music obtained by using this kind of method for producing note code sequences gives a feeling of musical continuity, typical Western music favors melodic arrangements of a still greater harmony. This object is achieved, e.g., by another em¬ bodiment of the invention, in which e.g. only the two last symbols in the key sequence and in the premises of the rule proposition consist of absolute codes while the preceding symbols stand for information of higher level, each representing a different combina¬ tion formed by a group of at least two codes. In musical terms, the higher-level symbols may re¬ present, e.g., chords which best describe the melody sequences and which are formed by the notes of pre- ceding times, half times, quadruple times, etc., and in which the order of the notes of the combination may be arbitrary. In place of musical chords, it is also possible to use other clusters of note com¬ binations (histograms) occurring in the example material. In this particular embodiment the premises of the rule proposition are formed, e.g, by the last two note codes, as described above; the example sequence preceding them is, however, analyzed in groups of two or more codes which are compared with a preformed library of higher-level symbols. When one of the code combinations is recognized as a certain symbol, this symbol is included in the premises of the rule proposition on the left side of the first two codes. In the same way, more higher-level symbols can be added to the left end of the premises. The rule proposition may thus contain one or more such high-level symbols or none. As a melody may contain notes which do not belong to the chord or to the histograms corresponding to the clusters, the re- cognition of the symbols in a short code sequence has to be based on approximating pattern recognition techniques.

A new code sequence is produced on the basis of these rule propositions in such a way that the two shortest forms of key sequences are, for instance, formed as described above, and the following forms of the key sequences are formed by recognizing combina¬ tions formed by groups of two or more preceding codes in the initial sequence and by replacing them with higher-level symbols equivalent to them or to com¬ binations closest to them.

This embodiment may also utilize the above- mentioned random choice.

When polyphonic music is produced by the method described above, the first note code sequence, cor- responding to the main tone or melody, . is formed first on the above-mentioned way utilizing a first search table; the first note code sequence is then used as an initial sequence, and one or more addi- tional code sequences, each corresponding to one accompanying tone, are formed by means of one or more additional search tables, respectively. The addi¬ tional tables contain separate rule propositions for each accompanying tune. The method of the invention is intended par¬ ticularly for the production of note code information in digital form for the control of electronic musical instruments or synthesizers or other such devices. The produced note codes can be converted into control signals complying with the MIDI standard, and these signals are further applied to the above-mentioned devices. The abbreviation MIDI stands for Musical Instrument Digital Interface and is a standard inter¬ face through which synthesizers, rhythm machines, computers, etc., can be linked together. Information on MIDI standards can be found, e.g., from [2] MIDI 1.0 specification, Document No, MIDI-1.0, August 1983, International MIDI Association.

The figures and the description related to them are only intended to illustrate the present inven¬ tion. In its details, the method of the invention may vary within the scope of the attached claims.

Claims

Claims :

1. A method of producing a digital code sequence, particularly a note code sequence from a finite number of different code types, each re¬ presenting one or more quantized properties of, for instance, a predetermined note, wherein new codes are generated one at a time after the code sequence on the basis of the existing codes of the sequence, c h a r a c t e r i z e d in that the generation of each new code comprises at least the steps of a) forming a number of key sequences increasing by degrees in length on the basis of codes produced last in the code sequence, the first and shortest key sequence being formed on the basis of the code produced last in the code sequence and the following ones on the basis of the number of codes produced last, said number increasing by steps of one or more codes; b) comparing the key sequences during their formation to a search table containing valid rule propositions each with different premises and invalid rule propositions of which at least two rule proposi¬ tions have equivalent premises but different con- sequences, until at least an approximate equivalence is found between the key sequence and the premises of a valid rule proposition; and c) selecting as a new code the code indicated by the consequence of the found valid rule proposi- tion or the code indicated by the consequence of a rule proposition selected at random from a group of invalid rule propositions having premises at least approximately equivalent to a key sequence shorter than the final key sequence by no more than a pre- determined number of codes.

2. A method according to claim 1, c h a r a c ¬ t e r i z e d in that the codes of the code sequence are used directly as a key sequence, starting from a key sequence comprising the code produced last in the code sequence, and adding preceding codes in the code sequence one at a time in front of the key sequence until equivalence is found between the premises of a valid rule proposition and the key sequence.

3. A method according to claim 1, c h a r a c - t e r i z e d in that the first and shortest form of the key sequence is formed on the basis of the last code in the code sequence, the possible subsequent key sequence forms on the basis of two or more last codes and all the following forms by adding symbols of higher level in front of the preceding code sequences by degrees, the higher-level symbols being formed by recognizing combinations formed by groups of two of more preceding codes in the code sequence and replacing them with predetermined higher-level symbols equivalent to them and to combinations closest to them in the key sequence, and that in at least some of the rule propositions the search por¬ tion is formed by one or more code terms and one or more higher-level symbol term.

4. A method according to claim 3, c h a r a c ¬ t e r i z e d in that each higher-level symbol re¬ presents a chord formed by notes occurring in a time section, one time or several times or sequences formed by such chords.

5. A method according to claim 3 or 4, c h a r a c t e r i z e d in that the recognition of the code combinations as predetermined higher-level symbols is carried out by a statistic accumulation method or some other statistic approximating pattern recognition method.

6. A method according to any of the preceding claims, c h a r a c t e r i z e d in that at least one additional code sequence is generated, such as a note code sequence representing a tone accompanying a melody, by using the key sequence increasing by degrees and formed on the basis of the codes of the first code sequence, and at least one additional search table.

7. A method according to any of the preceding claims, c h a r a c t e r i z e d in that if a key sequence equivalent to the premises of a valid rule proposition in the search table or additional search table is not found when generating a code sequence or an additional code sequence, a random number of codes preceding this situation is extracted from the first code sequence and the generation of the code sequence is continued on the basis of the remaining codes.

8. A method according to any of the preceding claims, c h a r a c t e r i z e d in that the code sequences are converted into control signals complying with the MIDI standard for controlling electronic musical instruments or synthesizers.