WO2009085054A1 - System and method for adaptive melodic segmentation and motivic identification - Google Patents

System and method for adaptive melodic segmentation and motivic identification

Info

Publication number
WO2009085054A1
WO2009085054A1 PCT/US2007/089225 US2007089225W WO2009085054A1 WO 2009085054 A1 WO2009085054 A1 WO 2009085054A1 US 2007089225 W US2007089225 W US 2007089225W WO 2009085054 A1 WO2009085054 A1 WO 2009085054A1
Authority
WO
Grant status
Application
Patent type
Prior art keywords
segment
getvalue
system
println
out
Prior art date
Application number
PCT/US2007/089225
Other languages
French (fr)
Inventor
Gregory Winston Wilder
Original Assignee
Orpheus Media Research, Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10HELECTROPHONIC MUSICAL INSTRUMENTS
    • G10H1/00Details of electrophonic musical instruments
    • G10H1/0008Associated control or indicating means
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10HELECTROPHONIC MUSICAL INSTRUMENTS
    • G10H2210/00Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/031Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
    • G10H2210/061Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for extraction of musical phrases, isolation of musically relevant segments, e.g. musical thumbnail generation, or for temporal structure analysis of a musical piece, e.g. determination of the movement sequence of a musical work

Abstract

The present invention comprises a system and method, modeled on research observations in human perception and cognition, capable of accurately segmenting primarily (although not exclusively) melodic input in performance data and encoded digital audio data, and mining the results for defining motives within the input data.

Description

SYSTEM AND METHOD FOR

ADAPTIVE MELODIC SEGMENTATION AND MOTIVIC IDENTIFICATION

Summary of the Invention

The present invention is a computer-implemented method and system for the analysis of musical information. Music is an informational form comprised of acoustic energy (sound) or informational representations of sound (such as musical notation or MIDI datastream) that conveys characteristics such as pitch (including melody and harmony), rhythm (and its characteristics such as tempo, meter, and articulation), dynamics (a characteristic of amplitude and perceptual loudness), structure, and the sonic qualities of timbre and texture. Musical compositions are purposeful arrangements of musical elements. Because music may be highly complex, varying over time in many simultaneous dimensions, there exists a need to characterize musical information so that it may be indexed, retrieved, compared, and otherwise automatically processed. The present invention provides a system and method for doing- so that considers the perceptual impact of music on a human listener, as well as the objective physical characteristics of musical compositions. The present invention comprises methods, modeled on research observations in human perception and cognition, capable of accurately segmenting primarily (although not exclusively) melodic input and mining the results for defining motives using context-aware search strategies. These results may then be employed to describe fundamental structures and unique identity characteristics of any musical input, regardless of style or genre. Background of the Invention

Musical melodies consist, at the least, of hierarchal grouped patterns of changing pitches and durations. Because music is an abstract language, parsing its grammatical constructs require the application of expanded semiotic and Gestalt principals. In particular, the algorithmic discretization of musical data is necessary for successful automated analysis and forms the basis for the present invention. Melodic Construction and Analysis

Term Definitions

Phrase: a section of music that is relatively self contained and coherent over a medium time scale. A rough analogy between musical phrases and the linguistic phrase can be made, comparing the lowest phrase level to clauses and the highest to a complete sentence.

Melody: a series of linear musical events in succession that can be perceived as a single (Gestalt) entity. Most specifically this includes patterns of changing pitches and durations, while most generally it includes any interacting patterns of changing events or quality. Melodies often consist of one or more musical phrases or motives, and are usually repeated throughout a work in various forms.

Prototypical Melody: generalization to which elements of information represented in the actual melody may be perceived as relevant.

Motive: the smallest identifiable musical element (melodic, harmonic, or rhythmic) characteristic of a composition. A motive may be of any size, though it is most commonly regarded as the shortest subdivision of a theme or phrase that maintains a discrete identity. For example, consider Beethoven's Fifth Symphony (Opus 67 in C minor, first movement) in which the pattern of three short notes followed by one long note is present throughout. Musical Hierarchies

Consider the graphic representation of musical form using the first line of Mary Had a Little Lamb shown in Fig. 1. The arcs connect two passages that contain the same sequence of notes, (after, Martin Wattenberg, "Arc Diagrams: Visualizing Structure in Strings," infovis, p. 110, 2002 IEEE Symposium on Information Visualization (Info Vis 2002), 2002.) Using this technique to graphically represent J.S. Bach's Minuet in G Major, shown in Fig. 2, a more elaborate (and potentially more interesting) series of hierarchical patterns emerges. (Wattenberg, 2002)

The internal structure of musical compositions is understood hierarchically; phrases often contain melodies, which are in turn composed of one or more motives. Phrases may also combine to form periods in addition to larger sections of music. Each hierarchical level provides essential information during analysis; smaller units tend to convey composition-specific identity characteristics while the formal design of larger sections allow general classification based on style and genre.

During the 1960s, composer and theorist Edward Cone devised the concept of hypermeter, a large scale metric structure consisting of hypermeasures and hyperbeats. Hyperunits describe patterns of strong and weak emphasis not notated in the musical score, but that are perceived by listeners and performers as "extended" levels of hierarchical formal organization. (Krebs, Harald (2005). "Hypermeter and Hypermetric Irregularity in the Songs of Josephine Lang.", in Deborah Stein (ed.),: Engaging Music: Essays in Music Analysis. New York: Oxford University Press.)

Further hierarchical approaches to musical analysis were introduced by theorist Heinrich Schenker in the 1930s, and later expanded by Salzer, Schachter. and others. By the 1980s, these views formed the foundation of "Schenkerian Analysis Techniques" and is one of the primary analytical methods practiced by music theorists today. Semiotic and Cognitive Considerations Music Semiology

With the exception of certain codes (rule-driven semiotic systems which suggest a choice of signifiers and their collocation to transmit intended meanings), music is an abstracted language that lacks specific instances and definitions with which to communicate concrete ideas. Because musical information is encoded in varying modalities (e.g. written and aural), the understanding of its defining grammatical principles is best illuminated through the study of music semiology, a branch of semiotics developed by musicologists Nattiez, Flatten, Monelle, and others.

Composer/musicologist Fred Lerdahl and linguist Ray Jackendoff have attempted to codify the cognitive structures (or "mental representations") a listener develops in order to acquire the musical grammar necessary to understand a particular musical idiom, and also to identify areas of human musical capacity that are limited by our general cognitive functions. These investigations led the authors to conclude that musical discretization, or segmentation, is necessary for cognitive perception and understanding, thus making discretization the basis for their work on pitch space analysis and cognitive constraints in human processing of musical grammar. (Lerdhal, F., Jackendoff, R. A Generative Theory of Tonal Music. MIT Press, Cambridge, MA. (1983); Jackendoff, R.& Lerdahl, F., "The Human Music Capacity: What is it and what's special about it?," Cognition, 100, 3372 (2006).) For these reasons, the process of musical analysis often involves reducing a piece to relatively simpler and smaller parts. This process of discretization is generally considered necessary for music to become accessible to analysis. (Nattiez, JeanJacques. Music and Discourse: Toward a Semiology of Music. (Musicologie generale et semiologue, 1987). Translated by Carolyn Abbate (1990).) Gestalt and the Implication Realization Cognition Model

The founding principles of Gestalt perception suggest that humans tend to mentally arrange experiences in a manner that is regular, orderly, symmetric, and simple. Cognitive psychologists have defined "Gestalt Laws" which allow us to predict the interpretation of sensation. Of particular interest to musical cognition research is the Law of Closure, which states that the mind may experience elements it does not directly perceive in order to complete an expected figure.

Eugene Narmour's Implication-Realization Model (Narmour, E. The Analysis and Cognition of Basic Melodic Structures: The Implication-Realization Model. Chicago:

University of Chicago Press. (1990); Narmour, E. The Analysis and Cognition of Melodic Complexity: The Implication-Realization Model. Chicago: University of Chicago Press. (1992)) is a detailed formalization based on Leonard Meyer's work on applied Gestalt psychology principles with regard to musical expectation. (Meyer, Leonard B. Emotion and Meaning in Music. Chicago: Chicago University Press. (1956)) This theory focuses on implicative intervals that set up expectations for certain realizations to follow. Narmour's model is one of the most significant modern theories of melodic expectation, providing specific detail regarding the expectations created by various melodic structures. Analysis and Cognition of Basic Melodic Structures: The Implication

Realization Model begins with two general claims. The first is given by "two universal formal hypotheses" describing what listeners expect. The process of melody perception is based on "the realization or denial" of these hypotheses (1990): 1) A+A → A (hearing two similar items yields an expectation of repetition)

2) A+B → C (hearing two different items yields an expected change) The second claim is that the "forms" above function to provide either closure or nonclosure. Narmour goes on to describe five melodic archetypes in accordance with his theory: 1) process [P] or iteration (duplication) [D] (A+A without closure)

2) reversal [R] (A+B with closure)

3) registral return [A+B+A] (exact or nearly exact return to same pitch)

4) dyad (two implicative items, as in 1 and 2, without a realization)

5) monad (one element which does not yield an implication) Central to the discussion is direction of melodic motion and size of intervals between pairs of pitches. [P] refers to motion in the same registral direction combined with similar intervallic motion (two small intervals or two large intervals). [D] refers to identical intervallic motion with lateral registral direction. [R] refers to changing intervallic motion (large to relatively smaller) with different registral directions.

P, D, and R only account for cases where registral direction and intervalic motion are working in unison to satisfy the implications. When one of these two factors is denied, there are more possibilities; the five archetypal derivatives:

1) intervallic process [IP]: small interval to similar small interval, different registral directions 2) registral process [VP]: small to large inteval, same registral direction

3) intervallic reversal [IR]: large interval to small interval, same registral direction

4) registral reversal [VR] : large interval to larger interval, different registral direction

5) intervallic duplication [ID]: small interval to identical small interval, different registral directions

Narmour contends that these eight symbols reference either a "prospective" or "retrospective" dimension and are therefore representative of generally available cognitive musical structures: "As symbological tokens, all sixteen prospective and retrospective letters purport to represent the listener's encoding of many of the basic structures of melody." (1990) Data Representation

The difficulties in accurately representing music for transmission and analysis have plagued musicians since sounds were first notated. Musical representation differs from generalized linguistic techniques in that it involves a unique combination of features among human activities: a strict and continuous time constraint on an output that is generated by a continuous stream of coded instructions. Additionally, it remains difficult (even for human experts) to consistently determine which musical elements are most important when transcribing musical performances. Past approaches have tended to favor perceived "foreground" parameters which are easiest to notate, while neglecting similarly important aspects of musical expression that are more difficult to capture or define. These challenges require a multidimensional representation system capable of measuring the amount of raw and relative change in simultaneous attribute dimensions and signifiers. Pattern Variation and Relevance

Once an adequate method of data collection and representation has been implemented, it remains problematic to reliably discover and compare potentially related musical ideas due to their various presentations and functions within a given work. Past models have attempted to directly extract significant patterns from raw musical material only to be overwhelmed with the volume of results, most of which may be unimportant. Flexible, context-based judgments are required to determine the prototypical structure and the analytical relevance of musical ideas, a task not well suited to standard heuristic techniques. Semantic Interpretation Issues

While the encoding of music shares certain characteristics with linguistic and grammar studies, research clearly demonstrates that many aspects of human musical capacity are interlinked with other more general cognitive functions. This observation, along with the semiotic nature of musical languages, requires a system capable of rendering adaptive solutions to largely self-defined data sets. Idiomatic Relational Grammar

A generative grammar is a set of rules or principles that recursively "specify" or "generate" the well- formed expressions of a natural language. Semiotic codes create a transformational grammar that renders rule-based approaches very weak. Even if idiomatic grammar rules could be found to provide a robust approach to musical data mining and analysis, it remains that individual pieces of music are fundamentally created from (and therefore shaped by) unique motivic ideas. This observation leads to the debate surrounding the definition of creativity and its origins.

Data Mining Within Creative Models

Creativity has been defined as "the initialization of connections between two or more multifaceted things, ideas, or phenomena hitherto not otherwise considered actively connected." (Cope, David. Computer Models of Musical Creativity. Cambridge, Mass.: MIT Press, 2005.) These inconspicuous and generally unpredictable connections create data characteristics that are often responsible for the most interesting (and arguably influential) musical works. Effectively interpreting this broad landscape requires any analyst (human or otherwise) to draw on contextual experience while maintaining a flexible approach. Prior Art Approaches to Algorithmic Musical Data Mining

Musical analysis generally involves reducing a piece to relatively smaller and simpler parts. This process of discretization, or segmentation, is necessary for the implementation of an algorithmic approach to significant pattern discovery. Melodic Segmentation

Prior art approaches have tended toward the application of complicated rule sets that rely on assumptions about specific style and language conventions. Overall, these approaches demonstrate four points of failure:

1) Rule based segmentation tends to create internal conflicts in real world application scenarios. Dependable musical analysis requires the awareness of contextual data trends when making segmentation boundary decisions.

2) Even if these conflicts are resolved appropriately, the assumptions required to design the original rule base necessarily limit the analysis process with regard to style and genre. 3) Certain implementations of rule based discretization systems require preprocessing of the input data to provide consistency within the samples. While this may make data processing more straightforward, it alters the original input, thus destroying the integrity of the data, making the results unreliable. 4) Grammatical rules may be useful in describing detailed analysis observations and outlining stylistic conventions, but these rules on their own do not provide the necessary knowledge base required to recreate an example resembling the original subject. This strongly suggests that no matter how complex a system of strict rules may become, it cannot adequately describe the transformational grammar at work in musical contexts. (By way of example: undergraduate music theory students are often taught part writing and counterpoint using rules drawn from "expert" analysis and observation, however they are rarely able to produce results that rival the models upon which these rules are based.) Gestalt Segmentation (Tenney, J., Polansky, L., "Temporal Gestalt Perception," Music Journal of Music Theory," Vol. 24, Issue 2, 1980. (pp. 205-241))

This prior art method relies on a single change indicator that presumes the inverse of proximity and similarity upon which grouping preference rule systems are based. When elements exceed a certain threshold of total (Gestalt) change, a boundary is formed. While correct in predicting the application of Gestalt principals, this system remains inflexible in that it relies on a single indicator of change and a predetermined threshold value.

GTTM Grouping Preference Rules (Lerdahl and Jackendoff, 1983) Musician Fred Lerdahl and linguist Ray Jackendoff attempted to codify the cognitive structures (or "mental representations") a listener develops in order to acquire the musical grammar necessary to understand a particular musical idiom. 1) GPR 1 (size) Avoid small grouping segments. The smaller, the less preferable. 2) GPR 2 (proximity) Given nl, n2, n3, n4; n2n3 may be group boundary if:

1. attack point interval between n2n3> nln2 && n3n4 OR

2. time between end of n2 and attack point of n3 > end of n3 to attack point of n4.

3) GPR 3 (change) Given nl, n2, n3, n4; n2n3 may be group boundary if: 1. pitch interval between n2n3> nln2&& n3n4 OR

2. dynamic interval of change between n2n3> nln2&& n3n4 OR

3. articulation duration between n2n3> nln2&& n3n4 OR

4. length of n2 != n3 && length of (nl+n2) = (n3+n4)

4) GPR 4 (intensification) When groupings from GPR 2&3 become pronounced, they may be split into higher level groups.

5) GPR 5 (symmetry) Grouping two parts of equal length.

6) GPR 6 (parallelism) Similar segments are preferably seen as parallel.

7) GPR 7 (timespan and prolongation stability) Large scale groupings that allow the greatest stability of the groupings within it. While they provide a valuable guide for the application of Gestalt principals and music cognition research to melodic segmentation, algorithmic implementations of the GPRs routinely lead to internal rule conflicts.

Structure Grouping (Berry, Wallace. Structural Functions in Music. New York: Dover Publications. 1987; and Cambouropoulos, E. (1997). Musical Rhythm: A Formal Model for Determining Local Boundaries, Accents and Meter in a Melodic Surface, in M. Leman (Ed.), Music, Gestalt, and Computing: Studies in Cognitive and Systematic Musicology (pp. 277-293). Berlin: Springer- Verlag.)

This technique is an extension of Gestalt Segmentation based on Lerdahl and Jackendoff s GPR 3 and Tenny and Polansky's research, that applies a preestablished threshold to the following criteria: tempo, register shift (pitch), approach (pitch), duration, articulation, timbre, and texture density. Recognizing the need to employ threshold tests to multiple attributes is an improvement on previous designs; however, this system remains insensitive to data tendencies and is therefore successful in only a limited number of cases. The Cognition of Basic Musical Structures (Temperley, David. The Cognition of Basic Musical Structures. Cambridge, MA: MIT Press. 2001)

This theory consists of six preference rule systems (conceptually similar to the GTTM), each containing "wellformedness" rules that define a class of structural descriptions that specify an optimal application for the given input. The six grammatical attributes analyzed are: meter, phrasing, counterpoint, harmony, key and pitch. Temperley' s approach requires event onset quantization (based on an arbitrary 35ms threshold) which alters (and therefore destroys) the integrity of the input data. In addition, algorithmic implementation of several of the proposed rule systems is impossible due to the fact that the descriptions are inadequate or incomplete. By way of example: phrase structure preference rule (PSPR) 2 claims that ideal melodic phrases should contain approximately 8 note events, which is an unjustified assumption based on one specific musical style. Automatic Generation of Grouping Structure (Hamanaka, M., Hirata, K. & Tojo, S., "ATTA: Automatic Time-Span Tree Analyzer Based on Extended GTTM", in Proceedings of the Sixth International Conference on Music

Information Retrieval, ISMIR 2005, 358-365.)

As previously discussed, direct application of the GTTM suffers from frequent rule conflicts. The authors of this study introduced adjustable parameters, in addition to a basic weighting process that allows for priority among the GPR.

Recognizing the faults of the inflexible rule-based GPR algorithms is a step in the right direction, however, this attempt fails to include procedures that allow for continuous context-based parameter adjustment; changes are made at the beginning of the process, but the parameters fail to fully adapt and comply to the input data. The result is clearly an improvement on the GTTM, but remains inflexible nonetheless.

Pattern Analysis in Music Data

Most historical approaches have attempted to mine musical patterns from low-dimension string representations; often without any preprocessing whatsoever. This has resulted in one of three common points of failure:

1) Applying heuristic search techniques to strings of musical data produces an overwhelming number of results; most of which are unimportant in terms of cognitive perception. Musical grammar naturally contains similar patterns throughout, but determining which of these have analytical value remains a significant challenge.

2) Some approaches attempt to filter results based on pattern frequency or length, however this still ignores the greater context considerations described within the largely self-defined musical data set.

3) In nearly every case, the difficulty of identifying musical parallelism remains unaddressed. Empirical research (Deliege, L, "Prototype effects in music listening: An empirical approach to the notion of imprint," Music Perception, 18, 2001. (pp. 371-407)) strongly suggests that beginnings of patterns play a crucial role in cognitive pattern recognition. This requires either preprocessing segmentation or a post-processing filtering algorithm capable of reliably identifying pattern start points so that beginning similarity can be analyzed.

Interactive Music Systems: Machine listening and Composing (Rowe, Robert. Interactive music systems: Machine listening and composing. Cambridge, MA: MIT Press. 1993.) Rowe' s approach rates each pattern occurrence based on the frequency with which the pattern is encountered. While frequency of pattern occurrence is an important factor in determining pattern relevance, this system ignores contextual issues and phrase parallelism (GPR 6). Music Indexing with Extracted Melody (Shih, H.H., S. S. Narayanan, and CC. Jay Kuo, "Automatic Main Melody Extraction from MIDI Files with a Modified LempelZiv Algorithm," Proc. of Intl. Symposium on Intelligent Multimedia, Video and Speech Processing, 2001.)

The disclosed method is a dictionary approach to repetitive melodic pattern extraction. Segmentation is based solely on tempo, meter, and bar divisions read from score. After basic extraction using a modified Lempel Ziv 78 compression method, the data is pruned to remove non-repeating patterns. Search and pruning processes are repeated until dictionary converges. Relying on the metric placement of musical events to determine hierarchal relevance can be misleading - this is especially true for complex music and most "Classical" literature composed after 1800. While this approach may work with some examples, musical phrasing often functions "outside" the bar.

FlExPat: Flexible Extraction of Sequential Patterns (Rolland, Pierre Yves, "Discovering patterns in musical sequences," Journal of New Music Research, 1999. (pp. 334-350); Rolland, Pierre Yves, "FlExPat: Flexible extraction of sequential patterns," Proceedings of the IEEE International Conference on Data Mining (IEEE ICDM'Ol). (pp. 481-488) San Jose, CA. 2001.)

This method identifies all melodic passage pairs that are significantly similar (based on a similarity threshold set in advance), extracts the patterns, and orders them according to frequency of occurrence and pattern length. The heavy combinatorial computation required is carried out using dynamic programming concepts. The use of euclidean distance-based dynamic programming techniques is an important advance toward increasing computational efficiency; however, this approach generates many unimportant results and does not take into account contextual issues and the importance of phrase parallelism (GPR 6). Finding Approximate Repeating Patterns from Sequence Data (J. L. Hsu, C C. Liu, and Arbee L. P. Chen, "Discovering Nontrivial Repeating Patterns in Music Data," Proceedings of IEEE Transactions on Multimedia, pp: 311-325, 2001.) This method is an application of feature extraction from music data to search for approximate repeating patterns. "Cut" and "Pattern Join" operators are applied to assist in sequential data search. This approach fails to introduce continuity issues raised through examination of midlevel and global context trends. Musical Parallelism and Melodic Segmentation: A Computational Approach (Cambouropoulos, E., "Musical Parallelism and Melodic Segmentation: A Computational Approach." Music Perception 23(3):249-269. (2006)) According to this method, discovered patterns are used as a means to determine probable segmentation points of a given melody. Relevant patterns are defined in terms of frequency of occurrence and length of pattern. The special status of non-overlapping, immediately repeating patterns is examined. All patterns merge into a single "pattern" segmentation profile that signifies points within the surface most likely to be perceived as segment boundaries. Requiring discovered patterns to be non-overlapping allows Cambouropoulos to introduce elements of context consideration into his process. However, by attempting to produce segmentation results using initial pattern searches, the process runs contrary to firmly established understandings of music cognition: namely the need for surface discretization for music to become accessible to algorithmic analysis. (Nattiez 1990)

In the patent literature, U.S. Patent No. 6,747,201 to Birmingham, et al. teaches a method using an exhaustive search for all potential patterns in a musical work, which are then filtered and rated by perceptual significance. U.S. Patent No. 7,227,072 to Weare discloses a system and method for processing audio recordings to determine similarity between audio data sets. Component such as harmonic, rhythmic and melodic input are generated and arbitrarily reduced in dimensionality to six by a mapper using two-dimensional feature maps generated by a trainer. The method disclosed produces results completely different from a melodic segmentation approach which requires the separation of polyphonic input into monophonic lines in order to develop a catalog of relational change (delta) between individual attributes (pitch, rhythm, articulation, dynamics) of individual musical events. Moreover, without knowing the full data set used by the trainer, however, the method cannot be defined, and its results cannot be repeated. Finally, U.S. Patent 7,206,775 to Kaiser, et al. discloses a music playlist generator based on genre "classification" (both human and automated) of media. No classification method is disclosed, and the patent teaches that there are no automated processes known that are capable of producing adequate results without human intervention in the processing method.

DETAILED DESCRIPTION OF THE INVENTION Data Formatting and Representation

Musical data is represented indirectly within the system of the present invention as a series of note event attribute changes. Both manual (performance data such as MIDI or score and the like) and auditory (encoded audio in the form of AIF, FLAC, MP3, MP4, and the like) input streams are used to build a comprehensive picture of the data models. Manual input supplies detailed information while auditory streams provide a simulation of the actual human listening experience. A user determined "style tag" may optionally be provided along with the model data for purposes of categorization and software training. This approach is based on current cognition models and is similar to the way humans acquire and process novel information. In this manner, associated identifiers and style awareness are developed over time and exposure to data streams. Manual (MIDI/SCORE) Models

Working with MIDI and score data allows in the present invention permits:

1) the high level of precision necessary for detailed analysis,

2) instrument-specific controller information, and

3) the ability compare specific performance data with perceived auditory data.

Global MetaStructure

According to the present invention, the data provided comprises: phrase structure, measure and tempo information, section identifiers, stylistic attributes, exact pitch, onset, offset, velocity, as well as note density for both micro (measure) and macro (phrase/section) groupings. Tracking includes translating controller data into stylistically context aware performance attributes.

Stylistic Performance Implications

By further comparing the analysis output with the calculated tempo grid, a specific analysis of stylistic character can occur. The exacting nature of this data format makes it especially (although not exclusively) suited to the segmentation analysis techniques described herein. Auditory Models

Working directly with auditory input allows the present invention to provide: 1) the modeling of human perception enhancements (and limitations),

2) realistic analysis of polyphonic textures (i.e. alberti bass),

3) and the potential to detect subtle performance variations (timbre, tempo). The following is a list of core issues along with their respective solutions specific to auditory model processing in the present invention. Equal Loudness (Fletcher-Munson) Contour Filtering

Human aural sensitivity varies with frequency. Software listeners filter input to compensate for this natural phenomenon and ensure relevant model analysis. First documented by Fletcher and Munson in 1933 (and refined by Robinson and Dadson in 1956), an equal loudness contour is the measure of sound pressure, over the frequency spectrum, for which a listener perceives a constant loudness. Aspects of implementing this filtering process have been described by Berry Vercoe (MIT), David Robinson and others. Frequency Tracking

The present invention employs spectral pitch tracking process using Csound's PVSPITCH opcode (Alan Ocinneide 2005.

(http://sourceforge.net/projects/csound/)) to determine localized frequency fundamentals. The pitch detection algorithm implemented by PVSPITCH is based upon J. F. Schouten's hypothesis that the brain times intervals between the beats of unresolved harmonics of a complex sound in order to find the pitch. The output of PVSPITCH is captured and stored at predetermined intervals (10ms) and analyzed for pattern correlations. Additionally, the results of PVSPITCH can be directly applied to an oscillator and audibly compared with the original signal. RMS and Pulse/Beat Tracking (Tempo Extraction)

RMS (root mean square: the statistical measure of the magnitude of a varying quantity) of the input signal is calculated to determine perceived signal strength and then examined for amplitude periodicity via the RMS Csound opcode. While beat/tempo tracking is not currently necessary for the auditory segmentation analysis process, RMS is calculated in attempt to detect changes in event onset and offset data. Csound' s TEMPEST opcode has been implemented for beat/tempo extraction. TEMPEST passes auditory input through a lowpass filter and places the residue in a short term memory buffer (attenuated over time) where it is analyzed for periodicity using a form of autocorrelation. The resulting period output is expressed as an estimated tempo (BPM). This result is also used internally to make predictions about future amplitude patterns, which are placed in a buffer adjacent to that of the input. The two adjacent buffers can be periodically displayed, and the predicted values optionally mixed with the incoming signal to simulate expectation. Timbre/Partial Tracking

The present invention employs a form of Instantaneous Frequency Distribution (IFD) analysis (Toshihiko Abe, Takao Kobayashi, Satoshi Imai, "Harmonics Estimation Based on Instantaneous Frequency and Its Application to Pitch Determination of Speech," IEICE TRANSACTIONS on Information and Systems Vol.E78-D No.9 pp.l 188-1194, 1995.) originally developed to accomplish spoken language pitch estimation in noisy environments. Implemented via Csound' s PVSIFD opcode (Lazzarini, 2005. (http://sourceforge.net/projects/csound/)) which performs an instantaneous frequency magnitude and phase analysis, using the short time Fourier transform (STFT) and IFD. The opcode generates two PV signals - one contains amplitude and frequency data (similar to PVSANAL) while the other contains amplitude and unwrapped phase information. Stylistic Performance Implications

By further comparing the frequency tracking output with the inferred tempo grid, a generalized stylistic tempo map may optionally be induced. Additionally, it may be useful to compare the placement of note event start points with the inferred tempo grid. Consistent discrepancies likely indicate the presence of a unique style identifier. Process Flow

Referring now to Fig. 3 there is shown a schematic process flow of the method of the present invention. The method begins by loading a data set representative of music into a computer memory. The method proceeds, as detailed herein, to identify at least one subset of the loaded data set representative of melody, and then to identify at least one subset of the melody data subset that is representative of motive. After such identification, post-processing steps as detailed herein (not shown) may be employed. Data Representation Attribute Formatting pitch: MIDI note number (0127) onset: absolute time offset: absolute time velocity: 0127 (MIDI) Delta Observations

Input data is represented indirectly within the system of the present invention as a series of change functions which provide pure abstraction of the musical material and ensures context aware analysis. For example: the relationship of three consecutive note events (NEs) (actually, it's the descriptive attributes that are of interest) are represented and compared using two normalized data points that describe the delta change between the NE data. Adaptive Melodic Segmentation

Calculations Between Consecutive Note Events (NE)

Property Definitions: pitch, velocity, onset, offset [double] length (calculated as offset onset) [double] current_pitch_to_next_pitch [double] current length to next length [double] current onset to next onset [double] current offset to next onset [double] current velocity to next velocity [double] Pseudocode:

Set current attribute to next attribute (pitch, onset, length, and velocity) [double] if (NEn > NEn+ 1) then {NEn+1 / NEn} else (NEn / NEn+ 1}

Case Specific Calculations

Pitch Contour is the quality necessary to maintain melodic specificity with regard to the delta pitch attribute.

Property Definitions

LSL (long/short/long length profile) [boolean] pitch contour (melodic direction) [boolean] delta_pitch_contour (change of melodic direction) [boolean]

Pseudocode: Set pitch contour [boolean] and delta pitch contour [boolean] if (NEn < NEn+ 1) while (NEn++ < NE(n+l)++) then {pitch contour to NEn+ 1 = UP} set delta_pitch_contour found = true if (NEn > NEn+ 1) while (NEn++ > NE(n+l)++) then {pitch_contour to NEn+ 1 = DOWN} set delta_pitch_contour found = true if (NEn == NEn+ 1) while (NEn++ == NE(n+l)++) then {pitch_contour to NEn+ 1 = SAME} set delta_pitch_contour found = true Java Code

// Case Specific — Pitch Contour

NoteEventLystltr previous = new

NoteEventLystltr ( this .getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . getSegmentNoteEventLyst ( ) . get ( 1-1 ) ) ;

// start at beginning-1 of NoteEventLyst current = new

NoteEventLystltr (this .getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) .getSegmentNoteEventLyst ( ) .get( 1) ) ; // start at beginning of NoteEventLyst next = new

NoteEventLystltr (this .getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . getSegmentNoteEventLyst ( ) . get ( 1+1 ) ) ; // start at beginning+1 of NoteEventLyst // scan NoteEvents and set Contour while ( !next.atEnd( ) ) { // Pitch Contour "Up" if ( !next.atEnd( ) &&

(current . getNoteEvent ( ) . get_Pitch( ) < next .getNoteEvent ( ) .get_Pitch( ) ) ) { current .getNoteEvent ( ) . set_pitch_contour_to_n ext_note( "U" ) ; assignment_counter++; // keep track of contour assignments

} // Pitch Contour "Down" if ( !next.atEnd( ) &&

(current . getNoteEvent ( ) . get_Pitch( ) > next .getNoteEvent ( ) .get_Pitch( ) ) ) { current . getNoteEvent ( ) . set_pitch_contour_to_n ext_note( "D" ) ; assignment_counter++; // keep track of contour assignments

} // Pitch Contour "Same" if ( !next.atEnd( ) &&

(current . getNoteEvent ( ) . get_Pitch( ) == next .getNoteEvent ( ) .get_Pitch( ) ) ) { current .getNoteEvent ( ) . set_pitch_contour_to_n ext_note ( "S" ) ; assignment_counter++; // keep track of contour assignments

} previous . advance ( ) ; next . advance ( ) ; current . advance ( ) ; } Long Short Long (LSL) Profile assists in identifying segment boundaries. Property Definitions

LSL (long/short/long length profile) [boolean]

Pseudocode: Set long short long note length (for all NEs) [boolean] if (NEn > NEn+ 1 < NEn+2) then {set NEn+2.LSL = true} Java Code

// Case Specific — Long Length current = new

NoteEventLystltr(this .getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s )

. getValue ( ) . getSegmentNoteEventLyst ( ) . get (I)); // start at beginning of NoteEventLyst next = new

NoteEventLystltr(this .getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s )

. getValue ( ) . getSegmentNoteEventLyst ( ) . get ( 1+1 ) ) ;

// start at beginning+1 of NoteEventLyst NoteEventLystltr twoAhead = new

NoteEventLystltr(this .getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s )

. getValue ( ) . getSegmentNoteEventLyst ( ) . get ( 1+2 ) ) ;

// start at beginning+2 of NoteEventLyst // scan NoteEvents and set LSL while (!twoAhead. atEnd( )) { if ( (next .getNoteEvent ( ) .get_Length( ) > current .getNoteEvent ( ) .get_Length( ) ) &&

( next . getNoteEvent ( ) . get_Length( ) > twoAhead. getNoteEvent ( ) .get_Length( ) ) ) { twoAhead. getNoteEvent ( ) .set_deltalonglength(t rue ) ;

} next . advance ( ) ; current . advance ( ) ; twoAhead. advance ( ) ; } Offset/Onset Overlap accounts for possible NE overlap in offset/onset calculations. (This step is particularly necessary for performance input.)

Pseudocode: Set offset to next onset [double] if (NEn+ 1.onset < NEn.offset) then {set offset to next onset = 0} // account for overlap else {set offset to next onset = NEn+ 1.onset NEn. offset}

Delta Calculations

Delta values represent amount of change between (NEn, NEn+ 1) and (NEn+ 1, Nen+2) and are used to conduct primary data calculations. This represents a significant process advantage in that it allows for the contextually aware attribute layers to align with key identifying characteristics of the original input. Property Definitions delta_pitch_to_next_pitch [double] delta length to next length [double] delta onset to next onset [double] delta offset to next onset [double] delta velocity to next velocity [double]

Pseudocode: Set delta attribute to next attribute (pitch, onset, length, and velocity) [double] setdelta=l( abs(NEn NEn+

I))

Pseudocode: Set delta offset/onset to next offset/onset [double] if (NEn == 0 or NEn+ 1 == 0) then {set delta offset/onset to next offset/onset = 0} else if (NEn > NEn+ 1) then {delta = NEn+ 1 / NEn} else {delta = NEn / NEn+ 1 }

Java Code

// Delta Calculations NoteEventLystltr current = new

NoteEventLystltr ( this .getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s )

. getValue ( ) . getSegmentNoteEventLyst ( ) . get (I)); // start at beginning of NoteEventLyst

NoteEventLystltr next = new

NoteEventLystltr (this .getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s )

. getValue ( ) . getSegmentNoteEventLyst ( ) . get ( 1+1 ) ) ; // start at beginning+1 of NoteEventLyst

NoteEventLystltr twoAhead = new

NoteEventLystltr (this .getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s )

. getValue ( ) . getSegmentNoteEventLyst ( ) . get ( 1+2 ) ) ; // start at beginning+2 of NoteEventLyst while ( !next.atEnd( ) ) { // Offset to Onset if

( (next .getNoteEvent ( ) . get_current_offset_to_next_o nset( ) == 0 | | current .getNoteEvent ( ) . get_current_offset_to_next_ onset ( ) == 0) ) { current .getNoteEvent ( ) . set_delta_offset_to_ne xt_onset (0.0); } else if

(next .getNoteEvent ( ) . get_current_offset_to_next_on set () / current .getNoteEvent ( ) . get_current_offset_to_next_ onset ( ) >= 1 ) { current .getNoteEvent ( ) . set_delta_offset_to_ne xt_onset( (current .getNoteEvent ( ) . get_current_offse t_to_next_onset ( ) / next .getNoteEvent ( ) . get_current_offset_to_next_ons et ( ) ) ) ;

} else { current .getNoteEvent ( ) . set_delta_offset_to_ne xt_onset ( ( next . getNoteEvent ( ) . get_current_offset_t o_next_onset ( ) / current .getNoteEvent ( ) . get_current_offset_to_next_ onset ( ) ) ) ;

}

// Onset to Onset current . getNoteEvent ( ) . set_delta_onset_to_nex t_onset(l -

(Math . abs ( next . getNoteEvent ( ) . get_current_onset_to _next_onset ( ) - current . getNoteEvent ( ) . get_current_onset_to_next_o nset ()))); if (next .current .getNext( ) .getNext( ) == null) { current .getNoteEvent ( ) . set_delta_onset_to_nex t_onset (0.0); }

// Pitch to Pitch current .getNoteEvent ( ) . set_delta_pitch_to_nex t_pitch(l -

(Math . abs ( next . getNoteEvent ( ) . get_current_pitch_to _next_pitch( ) - current . getNoteEvent ( ) . get_current_pitch_to_next_p itch()))); if (next .current .getNext( ) .getNext( ) == null) { current .getNoteEvent ( ) . set_delta_pitch_to_nex t_pitch (0.0);

}

// System. out. println ("*** Pitch Delta Calculation Result: " + current .getNoteEvent ( ) .get_delta_pitch_to_next_pit ch());

// Velocity to Velocity current .getNoteEvent ( ) . set_delta_vel_to_next_ vel(l - (Math. abs (next .getNoteEvent ( ) .get_current_vel_to_n ext_vel ( ) - current .getNoteEvent ( ) .get_current_vel_to_next_vel

O))); if (next .current .getNext( ) .getNext( ) == null) { current .getNoteEvent ( ) . set_delta_vel_to_next_ vel(O.O) ;

}

// Length to Length current . getNoteEvent ( ) . set_delta_length_to_ne xt_length( 1 -

(Math. abs ( next . getNoteEvent ( ) . get_current_length_t o_next_length( ) - current .getNoteEvent ( ) . get_current_length_to_next_ lengthO))); if (next .current .getNext( ) .getNext( ) == null) { current .getNoteEvent ( ) . set_delta_length_to_ne xt_length(0.0) ; }

// Pitch Contour if ( !twoAhead.atEnd( ) && current . getNoteEvent ( ) . get_pitch_contour_to_next_n ote() == "U") { if

( next . getNoteEvent ( ) . get_pitch_contour_to_next_not e() == "U") { next .getNoteEvent ( ) . set_deltapitchcontour(tru e);

} } else if ( !twoAhead.atEnd( ) && current . getNoteEvent ( ) . get_pitch_contour_to_next_n ote() == "D") { if

( next . getNoteEvent ( ) . get_pitch_contour_to_next_not e() == "D") { next .getNoteEvent ( ) . set_deltapitchcontour(tru e);

} } else if ( !twoAhead.atEnd( ) && current . getNoteEvent ( ) . get_pitch_contour_to_next_n ote() == "S") { if

( next . getNoteEvent ( ) . get_pitch_contour_to_next_not e() == "S") { next .getNoteEvent ( ) . set_deltapitchcontour(tru e);

} } else { next . getNoteEvent ( ) . set_deltapitchcontour ( fal se) ;

} assignment_counter++; twoAhead. advance ( ) ; current . advance ( ) ; next . advance ( ) ; } Adaptive Thresholds

Threshold Generation is an automatic procedure to establish statistically relevant threshold points for each NE attribute and allow for the creation of boundary candidates. After ensuring the adaptation process begins with a threshold candidate below the lower boundary, this method establishes an appropriate incremental value to be applied to the threshold candidate until the result is within boundary limits. This approach maintains a close link between the threshold and the input data. (NOTE: In extreme cases where the attribute data remains consistently static, the system may be unable to adapt an appropriate threshold. When this happens, the attribute in question does not influence boundary weighing. Property Definitions pitch threshold [double] length threshold [double] velocity threshold [double] onset to onset threshold [double] offset to onset threshold [double] mean = total delta change / total NEs [double] standard deviation (using mean) [double] std multiplier = 1 [double] divisor = 1 (pitch, onset, velocity) 100 (length) [int] divisor multiplier = 1 (pitch, onset, velocity) 10 (length) [int] success_multiplier = 4 (pitch, onset, velocity) 2 (length) [int] increment = (1 mean)/ divisor [double] lower boundary = lower bound of acceptable data points (15%) [double] upper boundary = upper bound of acceptable data points (45%) [double] previous success = number of NEs below the threshold (before adaptation) [double] successful events = number of NEs below the threshold [double]

Pseudocode: Set attribute threshold (pitch, onset, length, and velocity) [double]

FIRST PASS ONLY: set threshold to (std multiplier * standard deviation) test threshold against all NEs if (successful_events > total_NEs * lower_boundary) then {set std multiplier = std multiplier 0.1} else {set threshold to standard deviation} set threshold to (increment * standard deviation) set previous_success to successful_events test threshold against all NEs

(re)set successful_events based on "new" threshold if (successful_events >= previous_success * success_multiplier) then {set divisor = (successful_events - previous_success) * divisor_multiplier} else {set increment to (1 - mean)/divisor}

ALL SUCCESSIVE PASSES (NOT TO EXCEED 1000): test threshold against all NEs if (successful events >= total NEs * lower boundary && <= upper boundary) then {set threshold to threshold} else if (threshold < 1)

{set threshold to threshold + (increment * standard_deviation) and test against all NEs} else {set threshold to null} // unable to determine

Pseudocode: Set offset/onset threshold [double] set threshold to 0.0175

Java Code public double thresholdEstimtationOffset ( ) { System. out .println( ) ;

System. out .println( "*** Starting Adaptive Threshold Estimation (Offset)"); for ( int vl=l; vl <= this . getCompleteVoiceLayerLyst ( ) . size ( ) ; vl++ ) { for (int s=l; s <= this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . size ( ) ; s++ ) { double threshold = 0.0175; int success_counter = 0, pass = 0; // pass through candidate list to count number of successful events (screen output) NoteEventLystltr success_scanner = new

NoteEventLystltr (this .getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . getSegmentNoteEventLyst ( ) . get (I)); / / start at beginning of NoteEventLyst while ( ! success_scanner . atEnd2 ( ) ) { pass++; if ( (pass > 2) &&

( success_scanner .getNoteEvent ( ) . get_delta_offset_t o_next_onset ( ) <= threshold) &&

( success_scanner. getNoteEvent ( ) . get_delta_offset_t o_next_onset ( ) > O)) { if

( ( success_scanner . current . getPrev ( ) .getPrev( ) .getv alue( ) .get_delta_offset_to_next_onset( ) != 0) && ( success_scanner . current . getPrev ( ) . getValue ( ) .get_ delta_offset_to_next_onset( ) != O)) { success_scanner .current .getNext( ) . getValue ( ) . set_deltaspace ( true ) ; success_counter++ ;

} // success_counter++;

} success_scanner. advance ( ) ;

} // report success and store the result

System. out .println( " Offset to Onset Events Passing Threshold: " + success_counter) ; this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getv alue( ) . setThresholdOffToOnPass(success_counter ) ; // display a final message

System. out .println( "*** Completed Adaptive Threshold Estimation (Offset)"); this . complete_voice_layer_lyst . get ( vl ) . getVal ue( ) . setFoundOffToOnThreshold( true) ; return 0.0175; } } return 0.0175; } public double thresholdEstimtationOnset ( ) { System. out .println( ) ;

System. out .println( "*** Starting Adaptive Threshold Estimation (Onset)"); for (int vl=l; vl <= this .getCompleteVoiceLayerLyst (). size( ); vl++) { for (int s=l; s <= this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) .getCompleteSegmentLyst ( ) . size( ) ; s++) { // Onset to Onset (manual adjustments based on initial data range sensitivity) int success_counter = 0, divisor = 1, divisor_multiplier = 1; // increment divisor bias necessary for attributes with wide data spread (pitch, vel, etc. = 1, length could be 10 or even 100) int success_multiplier = 4; // determines how fast the "scaling up" of the increment adaptation will be (larger is faster) double total = 0.0, mean = 0.0, increment = 0.0, std = 0.0, threshold = 0.0, multiplier = 1.0; // determines initial multiplier for std double lower_bound_events_passing = lower_boundary + 0.05; // lower percentage boundary (target) for number of NEs passing the threshold double upper_bound_events_passing = upper_boundary + 0.05; // upper percentage boundary (target) for number of NEs passing the threshold int segmentSize =

( this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) .size( ) - 2); int pass_counter = 0; for (int p=0; p < 100 * segmentSize; p++) { // do not allow more than 100 * segmentSize passes pass_counter++; if (p == 0) { // calculate the initial standard deviation total = this . onsetThresholdTotal ( this . getCompleteVoiceLaye rLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) ) ; mean = this . thresholdMean ( this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s) . getValue ( ) , total); increment = this .thresholdlncrement (mean, divisor) ; std = this . onsetThresholdStandardDeviation ( this . getCompl eteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getComplete SegmentLyst ( ) . get ( s ) . getValue ( ) , total , mean, increment) ;

// store the std value in the threshold profile this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue( ) . setOnsetStandardDeviation( std) ;

System. out . println ( "

Onset to Onset Standard Deviation Results: " + std) ;

} else {

// adjust the threshold for all subsequent passes threshold = threshold + (std * increment);

} if (p == 0) { // if neccessary, adapt the starting value to fall below the lower boundary value boolean success = false; while (! success) { success_counter = 0; threshold = (multiplier * std) ;

NoteEventLystltr scanner = new NoteEventLystltr

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) . get ( 1 ) ) ; // start at beginning of NoteEventLyst

// pass through candidate list to count number of successful events while (! scanner. atEnd( )) { if

( scanner.getNoteEvent ( ) .get_delta_onset_to_next_on set( ) <= threshold) { success_counter++;

} scanner. advance ( ) ;

} // if the number of successful events is higher than the lower boundary percentage, adjust the threshold (via the multiplier) and try again if ( success_counter > (segmentSize * lower_bound_events_passing) ) { multiplier = multiplier - 0.1;

}

// continue once the initial number of events have dipped below the lower boundary else {success = true; }

} } if (p == 0) { // if neccessary, adapt the increment value int previous_success = 0, current_success = 0; success_counter = 1; // reset the success counter (allowing a *real* value for the first calculation) boolean success = false; while (! success) { threshold = threshold + ( std * increment); // recalculate the threshold

NoteEventLystltr scanner = new NoteEventLystltr (this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) .get ( 1 ) ) ; // start at beginning of NoteEventLyst previous_success = success_counter; // store the previous result for comparison with the next pass

// pass through candidate list to count number of successful events while (! scanner.atEnd( )) { if

( scanner.getNoteEvent ( ) .get_delta_onset_to_next_on set () <= threshold) { success_counter++;

} scanner. advance ( ) ;

}

// update the new number of candidates current_success = success_counter; if (current_success >= (previous_success * success_multiplier) ) { divisor = ( (current_success - previous_success ) * divisor_multiplier) ; } else { increment = ((I - mean) / divisor); success = true;

} } }

// continue process with the adapted starting point and increment adjustment

NoteEventLystltr scanner = new NoteEventLystltr

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) .get( 1) ) ; // start at beginning of NoteEventLyst int ne_counter = 0 ; while (! scanner.atEnd( )) { if

( scanner.getNoteEvent ( ) .get_delta_onset_to_next_on set() <= threshold) { if

( scanner.getNoteEvent ( ) .get_delta_onset_to_next_on set () > this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdOnToOnMax( ) ) { this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue ( ) . setThresholdOnToOnMax( scanner. getNoteEvent ( ) . get_delta_onset_to_next_onset ( ) ) ;

} if

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdOnToOnMin( ) == 0 ) { this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue( ) . setThresholdOnToOnMin( scanner.getNoteEvent (

) . get_delta_onset_to_next_onset ( ) ) ; } else if

( scanner.getNoteEvent ( ) .get_delta_onset_to_next_on set() < this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue (

) .getThresholdOnToOnMin( ) && scanner.getNoteEvent ( ) .get_delta_onset_to_next_ons et() > 0) { this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getv alue( ) . setThresholdOnToOnMin( scanner .getNoteEvent ( ) . get_delta_onset_to_next_onset ( ) ) ; } ne_counter++; // count the number of NE ' s that pass the threshold

} scanner . advance ( ) ;

}

// threshold = 1.1; // TODO:

KILL ONSET — IF ONSET FAILS, THE ANSWERS BECOME CORRECT !!!!!!!!!!! // bail if threshold grows too large without passing a suitable number of NEs if (threshold > 1) {

System. out . println ( "

Voice Layer unable to generate an acceptable threshold profile — threshold output " + threshold + " after " + pass_counter + " pass (es ) " ) ;

// display a final message System. out. println( "***

Completed Adaptive Threshold Estimation (Onset)"); return 0.0; } // test for number of events within boundary range if ((ne_counter >=

(segmentSize * lower_bound_events_passing) ) && (ne_counter <= (segmentSize * upper_bound_events_passing) ) ) {

System. out . println ( "

Onset to Onset Adaptive Threshold MAX: " + this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) .getThresholdOnToOnMax( ) ) ; System. out . println ( "

Onset to Onset Adaptive Threshold MIN: " + this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) .getThresholdOnToOnMin( ) ) ;

// pass through candidate list to count number of successful events (screen output only) success_counter = 0;

NoteEventLystltr success_scanner = new NoteEventLystltr (this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) . get ( 1 ) ) ; // start at beginning of NoteEventLyst while ( ! success_scanner. atEnd2 ( ) ) { if

( success_scanner.getNoteEvent ( ) .get_delta_onset_to _next_onset ( ) <= threshold) { success_scanner.current .getNext( ) . getValue ( ) . set_deltaattack(true) ; success_counter++;

} success_scanner. advance ( ) ;

}

// report success and store the result

System.out .println( " Onset to Onset Events Passing Threshold: " + success_counter) ;

System.out .println( "

Onset to Onset Threshold Results: " + threshold + " after " + pass_counter + " pass(es)"); this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue( ) .setThresholdOnToOnPass(success_counter) ; // display a final message

System.out .println( "*** Completed Adaptive Threshold Estimation (Onset)"); this .complete_voice_layer_lyst . get (vl ) . getVal ue( ) . setFoundOnToOnThreshold(true) ; return threshold; } } }

} // final chance to bail return 0.0;

} public double thresholdEstimtationPitch( ) { System. out . println ( ) ;

System. out .println( "*** Starting Adaptive Threshold Estimation (Pitch)"); for ( int vl=l; vl <= this .getCompleteVoiceLayerLyst ( ) . size( ) ; vl++) { for (int s=l; s <= this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getCompleteSegmentLyst ( ) . size( ) ; s++) {

// Pitch to Pitch (manual adjustments based on initial data range sensitivity) int success_counter = 0, divisor = 1, divisor_multiplier = 1; // increment divisor bias necessary for attributes with wide data spread (pitch, vel, etc. = 1, length could be 10 or even 100) int success_multiplier = 4; // determines how fast the "scaling up" of the increment adaptation will be (larger is faster) double total = 0.0, mean = 0.0, increment = 0.0, std = 0.0, threshold = 0.0, multiplier = 1.0; // determines initial multiplier for std double lower_bound_events_passing = lower_boundary; // lower percentage boundary (target) for number of NEs passing the threshold double upper_bound_events_passing = upper_boundary; // upper percentage boundary (target) for number of NEs passing the threshold int segmentSize = ( this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) . size ( ) - 2 ) ; int pass_counter = 0; for (int p=0; p < 10000 * segmentSize; p++) { // do not allow more than 10000 passes pass_counter++; if (p == 0) { total = this . pitchThresholdTotal ( this . getCompleteVoiceLaye rLyst( ) .get(vl) .getValue( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) ) ; mean = this . thresholdMean ( this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) , total ) ; increment = this .threshold-Increment (mean, divisor) ; std = this . pitchThresholdStandardDeviation ( this . getCompl eteVoiceLayerLyst ( ) .get(vl) . getValue ( ) .getComplete SegmentLyst ( ) . get ( s ) . getValue ( ) , total , mean , increment) ;

System. out . println ( " Pitch to Pitch Standard Deviation Results: "

+ std) ;

} else {

// adjust the threshold for all subsequent passes threshold = threshold +

(std * increment);

} if (p == 0) { boolean success = false; while (! success) { success_counter = 0; threshold = (multiplier * std) ;

NoteEventLystltr scanner = new NoteEventLystltr

( this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) .get( 1) ) ; // start at beginning of NoteEventLyst

// pass through candidate list to count number of successful events while (! scanner. atEnd( )) { if

( scanner .getNoteEvent ( ) .get_delta_pitch_to_next_pi tch( ) <= threshold) { success counter++; scanner.advance ( ) ;

} // if the number of successful events is higher than the lower boundary percentage, adjust the threshold (via the multiplier) and try again if ( success_counter > (segmentSize * lower_bound_events_passing) ) { multiplier = multiplier - 0.1;

}

// continue once the initial number of events have dipped below the lower boundary else {success = true; }

} } if (p == 0) { int previous_success = 0, current_success = 0; success_counter = 1; // reset the success counter (allowing a *real* value for the first calculation) boolean success = false; while (! success) { threshold = threshold + ( std * increment); // recalculate the threshold

NoteEventLystltr scanner = new NoteEventLystltr (this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) .get ( 1 ) ) ; // start at beginning of NoteEventLyst previous_success = success_counter; // store the previous result for comparison with the next pass

// pass through candidate list to count number of successful events while (! scanner.atEnd( )) { if

( scanner. getNoteEvent ( ) .get_delta_pitch_to_next_pi tch() <= threshold) { success_counter++;

} scanner. advance ( ) ;

}

// update the new number of candidates current_success = success_counter; if (current_success >= (previous_success * success_multiplier) ) { divisor = ( (current_success - previous_success ) * divisor_multiplier) ;

} else { increment = ((I - mean) / divisor); success = true; } } } // continue process with the adapted starting point and increment adjustment

NoteEventLystltr scanner = new NoteEventLystltr

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) .get ( 1 ) ) ; // start at beginning of NoteEventLyst int ne_counter = 0 ; while (! scanner. atEnd( )) { if

( scanner.getNoteEvent ( ) .get_delta_pitch_to_next_pi tch( ) <= threshold) { if ( scanner. getNoteEvent ( ) . get_delta_pitch_to_next_pi tch() > this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdPitchMax( ) ) { this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue ( ) . setThresholdPitchMax( scanner. getNoteEvent ( ) •get_delta_pitch_to_next_pitch( ) ) ;

} if

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdPitchMin( ) == 0 ) { this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue( ) . setThresholdPitchMin( scanner.getNoteEvent ( )

•get_delta_pitch_to_next_pitch( ) ) ; } else if

( scanner.getNoteEvent ( ) .get_delta_pitch_to_next_pi tch() < this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue (

) .getThresholdPitchMin( ) && scanner.getNoteEvent ( ) .get_delta_pitch_to_next_pit ch() > 0) { this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue( ) . setThresholdPitchMin( scanner.getNoteEvent ( ) .get_delta_pitch_to_next_pitch( ) ) ;

} ne_counter++; // count the number of NE ' s that pass the threshold

} scanner. advance ( ); }

// bail if threshold grows too large without passing a suitable number of NEs if (threshold > 1) { System.out .println( "

Voice Layer unable to generate an acceptable threshold profile -- threshold output " + threshold + " after " + pass_counter + " pass (es ) " ) ; // display a final message

System.out .println( "*** Completed Adaptive Threshold Estimation (Pitch)"); return 0.0; } if ( ( ne_counter >=

( segmentSize * lower_bound_events_passing) ) && (ne_counter <= (segmentSize * upper_bound_events_passing) ) ) { System. out . println ( "

Pitch to Pitch Adaptive Threshold MAX: " + this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) .getThresholdPitchMax( ) ) ;

System. out . println ( " Pitch to Pitch Adaptive Threshold MIN: " + this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) .getThresholdPitchMin( ) ) ;

// pass through candidate list to count number of successful events (screen output) success_counter = 0; NoteEventLystltr success_scanner = new NoteEventLystltr ( this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) .get ( 1 ) ) ; // start at beginning of NoteEventLyst while ( ! success_scanner . atEnd2 ( ) ) { if

( success_scanner .getNoteEvent ( ) .get_delta_pitch_to _next_pitch( ) <= threshold) { success_scanner .current .getNext( ) . getValue ( ) . set_deltapitch(true) ; success_counter++ ;

} success_scanner. advance ( ) ;

}

// report success and store the result

System. out . println ( " Pitch to Pitch Events Passing Threshold: " + success_counter ) ;

System. out . println ( "

Pitch to Pitch Threshold Results: " + threshold + " after " + pass_counter + " pass(es)"); this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue( ) .setThresholdPitchPass(success_counter) ; // display a final message

System.out .println( "*** Completed Adaptive Threshold Estimation (Pitch)"); this .complete_voice_layer_lyst . get (vl ) . getVal ue( ) . setFoundPitchThreshold(true) ; return threshold; } } }

}

// final chance to bail return 0.0;

} public double thresholdEstimtationVelocity( ) { System.out .println( ) ;

System.out .println( "*** Starting Adaptive Threshold Estimation (Velocity)"); for ( int vl=l; vl <= this. getCompleteVoiceLayerLyst ( ) .size( ) ; vl++) { for (int s=l; s <= this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . size ( ) ; s++) {

// Velocity to Velocity (manual adjustments based on initial data range sensitivity) int success_counter = 0, divisor = 1, divisor_multiplier = 1; // increment divisor bias necessary for attributes with wide data spread (pitch, vel, etc. = 1, length could be 10 or even 100) int success_multiplier = 4; // determines how fast the "scaling up" of the increment adaptation will be (larger is faster) double total = 0.0, mean = 0.0, increment = 0.0, std = 0.0, threshold = 0.0, multiplier = 1.0; // determines initial multiplier for std double lower_bound_events_passing = lower_boundary; // lower percentage boundary

(target) for number of NEs passing the threshold double upper_bound_events_passing = upper_boundary; // upper percentage boundary (target) for number of NEs passing the threshold int segmentSize = (this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) . size ( ) - 2 ) ; int pass_counter = 0; for (int p=0; p < 1000 * segmentSize; p++) { // do not allow more than 1000 passes pass_counter++; if (p == 0) { total = this .velocityThresholdTotal (this . getCompleteVoiceL ayerLyst( ) .get(vl) . getValue ( ) .getCompleteSegmentLy st ( ) . get ( s ) . getValue ( ) ) ; mean = this .thresholdMean(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) , total ) ; increment = this .thresholdlncrement (mean, divisor) ; std = this .velocityThresholdStandardDeviation(this . getCo mpleteVoiceLayerLyst ( ) .get(vl) . getValue ( ) .getCompl eteSegmentLyst ( ) . get ( s ) . getValue ( ) , total , mean, increment) ;

System.out .println( " Velocity to Velocity Standard Deviation

Results: " + std) ;

} else {

// adjust the threshold for all subsequent passes threshold = threshold +

(std * increment);

} if (p == 0) { boolean success = false; while (! success) { success_counter = 0; threshold = (multiplier * std) ;

NoteEventLystltr scanner = new NoteEventLystltr

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) . get ( 1 ) ) ; // start at beginning of NoteEventLyst // pass through candidate list to count number of successful events while (! scanner.atEnd( )) { if

( scanner.getNoteEvent ( ) .get_delta_vel_to_next_vel( ) <= threshold) { success_counter++; } scanner.advance ( ) ;

} // if the number of successful events is higher than the lower boundary percentage, adjust the threshold (via the multiplier) and try again if ( success_counter > (segmentSize * lower_bound_events_passing) ) { multiplier = multiplier - 0.1;

}

// continue once the initial number of events have dipped below the lower boundary else {success = true; }

} } if (p == 0) { int previous_success = 0, current_success = 0; success_counter = 1; // reset the success counter (allowing a *real* value for the first calculation) boolean success = false; while (! success) { threshold = threshold + ( std * increment); // recalculate the threshold

NoteEventLystltr scanner = new NoteEventLystltr

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) . get ( 1 ) ) ; // start at beginning of NoteEventLyst previous_success = success_counter; // store the previous result for comparison with the next pass

// pass through candidate list to count number of successful events while ( ! scanner. atEnd( ) ) { if ( scanner. getNoteEvent ( ) . get_delta_vel_to_next_vel ( ) <= threshold) { success_counter++;

} scanner.advance ( ) ;

}

// update the new number of candidates current_success = success_counter; if (current_success

>= (previous_success * success_multiplier) ) { divisor =

( (current_success - previous_success ) * divisor_multiplier) ;

} else { increment = ((I

- mean) / divisor); success = true; } } } // continue process with the adapted starting point and increment adjustment

NoteEventLystltr scanner = new NoteEventLystltr (this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) .get ( 1 ) ) ; // start at beginning of NoteEventLyst int ne_counter = 0 ; while (! scanner.atEnd( )) { if

( scanner.getNoteEvent ( ) .get_delta_vel_to_next_vel( ) <= threshold) { if

( scanner.getNoteEvent ( ) .get_delta_vel_to_next_vel(

) > this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdVelocityMax( ) ) { this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue( ) . setThresholdVelocityMax( scanner.getNoteEven t( ) .get_delta_vel_to_next_vel( ) ) ;

} if

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdVelocityMin( ) == 0 ) { this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue ( ) . setThresholdVelocityMin( scanner. getNoteEven t( ) . get_delta_vel_to_next_vel ( ) ) ;

} else if ( scanner.getNoteEvent ( ) .get_delta_vel_to_next_vel(

) < this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdVelocityMin( ) && scanner.getNoteEvent ( ) .get_delta_vel_to_next_vel( ) > 0) { this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue( ) . setThresholdVelocityMin( scanner.getNoteEven t( ) . get_delta_vel_to_next_vel ( ) ) ;

} ne_counter++; // count the number of NE ' s that pass the threshold

} scanner. advance ( ); }

// bail if threshold grows too large without passing a suitable number of NEs if (threshold > 1) {

System. out . println ( "

Voice Layer unable to generate an acceptable threshold profile -- threshold output " + threshold + " after " + pass_counter + " pass(es)"); this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue ( ) . setThresholdVelocityMin (0.0); this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue ( ) . setThresholdVelocityMax (0.0);

// display a final message

System. out .println ( "*** Completed Adaptive Threshold Estimation (Velocity) " ) ; return 0.0;

} if ((ne_counter >= (segmentSize * lower_bound_events_passing) ) && (ne_counter <= (segmentSize * upper_bound_events_passing) ) ) {

System. out . println ( "

Velocity to Velocity Adaptive Threshold MAX: " + this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdVelocityMax( ) ) ;

System. out . println ( "

Velocity to Velocity Adaptive Threshold MIN: " + this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdVelocityMin( ) ) ;

// pass through candidate list to count number of successful events (screen output ) success_counter = 0; NoteEventLystltr success_scanner = new NoteEventLystltr ( this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) . get ( 1 ) ) ; // start at beginning of NoteEventLyst while ( ! success_scanner. atEnd2 ( ) ) { if

( success_scanner.getNoteEvent ( ) .get_delta_vel_to_n ext_vel() <= threshold) { success_scanner.current .getNext( ) .getValue( ) . set_deltavel (true ) ; success_counter++;

} success_scanner. advance ( ) ;

}

// report success and store the result

System.out .println( " Velocity to Velocity Events Passing

Threshold: " + success_counter) ;

System.out .println( "

Velocity to Velocity Threshold Results: " + threshold + " after " + pass_counter + " pass(es)"); this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue( ) .setThresholdVelocityPass(success_counter) ; // display a final message

System.out .println( "***

Completed Adaptive Threshold Estimation (Velocity) " ) ; this .complete_voice_layer_lyst . get (vl ) . getVal ue( ) . setFoundVelThreshold(true) ; return threshold;

} }

} }

// final chance to bail return 0.0; } public double thresholdEstimtationLength( ) { System. out .println( ) ;

System. out .println( "*** Starting Adaptive Threshold Estimation (Length)"); for ( int vl=l; vl <= this.getCompleteVoiceLayerLyst( ) .size( ) ; vl++) { for (int s=l; s <= this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . size ( ) ; s++ ) {

// Length to Length (manual adjustments based on initial data range sensitivity) int success_counter = 0, divisor =

100, divisor_multiplier = 10; // increment divisor bias necessary for attributes with wide data spread (pitch, vel, etc. = 1, length could be 10 or even 100) int success_multiplier = 2; // determines how fast the "scaling up" of the increment adaptation will be (larger is faster) double total = 0.0, mean = 0.0, increment = 0.0, std = 0.0, threshold = 0.0, multiplier = 1.0; // determines initial multiplier for std double lower_bound_events_passing = lower_boundary; // lower percentage boundary

(target) for number of NEs passing the threshold double upper_bound_events_passing = upper_boundary ; // upper percentage boundary (target) for number of NEs passing the threshold int segmentSize =

( this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) . size ( ) - 2 ) ; int pass_counter = 0; for (int p=0; p < 1000 * segmentSize; p++) { // do not allow more than 1000 passes pass_counter++ ; if (p == 0) { total = this . lengthThresholdTotal ( this . getCompleteVoiceLay erLyst( ) .get(vl) . getValue ( ) .getCompleteSegmentLyst ( ) . get (S). getValue ( ) ) ; mean = this . thresholdMean ( this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) , total ) ; increment = this . threshold-Increment (mean, divisor) ; std = this . lengthThresholdStandardDeviation ( this . getComp leteVoiceLayerLyst ( ) .get(vl) . getValue ( ) .getComplet eSegmentLyst ( ) . get ( s ) . getValue ( ) , total , mean , increment) ; // store the std value in the threshold profile this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getv alue( ) . setLengthStandardDeviation( std) ; System. out . println ( "

Length to Length Standard Deviation Results: " + std);

} else {

// adjust the threshold for all subsequent passes threshold = threshold + (std * increment);

} if (p == 0) { boolean success = false; while (! success) { success_counter = 0; threshold = (multiplier * std) ; NoteEventLystltr scanner = new NoteEventLystltr

( this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) .get ( 1 ) ) ; // start at beginning of NoteEventLyst

// pass through candidate list to count number of successful events while (! scanner .atEnd( )) { if

( scanner .getNoteEvent ( ) .get_delta_length_to_next_l ength() <= threshold) { success counter++; scanner. advance ( ) ;

}

// if the number of successful events is higher than the lower boundary percentage, adjust the threshold (via the multiplier) and try again if (success_counter

> (segmentSize * lower_bound_events_passing) ) { multiplier = multiplier - 0.1;

} // continue once the initial number of events have dipped below the lower boundary else {success = true; } }

} if (p == 0) { int previous_success = 0, current_success = 0; success_counter = 1; // reset the success counter (allowing a *real* value for the first calculation) boolean success = false; while (! success) { threshold = threshold + ( std * increment); // recalculate the threshold

NoteEventLystltr scanner = new NoteEventLystltr

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) .get( 1) ) ; // start at beginning of NoteEventLyst previous_success = success_counter; // store the previous result for comparison with the next pass

// pass through candidate list to count number of successful events while (! scanner.atEnd( )) { if

( scanner. getNoteEvent ( ) .get_delta_length_to_next_l ength() <= threshold) { success_counter++;

} scanner. advance ( ) ;

}

// update the new number of candidates current_success = success_counter; if (current_success >= (previous_success * success_multiplier) ) { divisor = ( (current_success - previous_success ) * divisor_multiplier) ;

} else { increment = ((I - mean) / divisor); success = true; } } } // continue process with the adapted starting point and increment adjustment

NoteEventLystltr scanner = new NoteEventLystltr

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) .get ( 1 ) ) ; // start at beginning of NoteEventLyst int ne_counter = 0 ; while (! scanner. atEnd( )) { if

( scanner.getNoteEvent ( ) .get_delta_length_to_next_l ength() <= threshold) { if ( scanner. getNoteEvent ( ) . get_delta_length_to_next_l ength() > this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdLengthMax( ) ) { this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue ( ) . setThresholdLengthMax( scanner. getNoteEvent ( ) .get_delta_length_to_next_length( ) ) ;

} if

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdLengthMin( ) == 0 ) { this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue( ) . setThresholdLengthMin( scanner.getNoteEvent (

) .get_delta_length_to_next_length( ) ) ; } else if

( scanner.getNoteEvent ( ) .get_delta_length_to_next_l ength() < this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue (

) .getThresholdLengthMin( ) && scanner.getNoteEvent ( ) .get_delta_length_to_next_le ngth() > 0) { this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue( ) . setThresholdLengthMin( scanner.getNoteEvent ( ) .get_delta_length_to_next_length( ) ) ;

} ne_counter++; // count the number of NE ' s that pass the threshold

} scanner. advance ( ); }

// bail if threshold grows too large without passing a suitable number of NEs if (threshold > 1) { System.out .println( "

Voice Layer unable to generate an acceptable threshold profile -- threshold output " + threshold + " after " + pass_counter + " pass (es ) " ) ; // display a final message

System.out .println( "***

Completed Adaptive Threshold Estimation (Length) " ) ; return 0.0;

} if ( ( ne_counter >=

( segmentSize * lower_bound_events_passing) ) && (ne_counter <= (segmentSize * upper_bound_events_passing) ) ) { System. out . println ( "

Length to Length Adaptive Threshold MAX: " + this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) .getThresholdLengthMax( ) ) ;

System. out . println ( " Length to Length Adaptive Threshold MIN: " + this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) .getThresholdLengthMin( ) ) ;

// pass through candidate list to count number of successful events (screen output) success_counter = 0; NoteEventLystltr success_scanner = new NoteEventLystltr ( this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . gets egmentNoteEventLyst ( ) .get ( 1 ) ) ; // start at beginning of NoteEventLyst while ( ! success_scanner . atEnd2 ( ) ) { if

( success_scanner .getNoteEvent ( ) .get_delta_length_t o_next_length( ) <= threshold) { success_scanner .current .getNext( ) . getValue ( ) . set_deltalength(true) ; success_counter++ ;

} success_scanner. advance ( ) ;

}

// report success and store the result

System. out . println ( " Length to Length Events Passing Threshold: "

+ success_counter ) ;

System. out . println ( "

Length to Length Threshold Results: " + threshold + " after " + pass_counter + " pass(es)"); this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue( ) .setThresholdLengthPass(success_counter) ; // display a final message

System. out .println( "***

Completed Adaptive Threshold Estimation (Length) " ) ; this . complete_voice_layer_lyst . get (vl ) . getVal ue( ) . setFoundLengthThreshold( true) ; return threshold;

} }

} }

// final chance to bail return 0.0; }

Max and Min Delta Threshold Change

Having adapted relevant thresholds in the previous stage, this method searches for maximum and minimum results that pass the threshold and stores them.

Property Definitions pitch max [double] pitch min [double] off to on max [double] off to on min [double] on to on max [double] on to on min [double] length max [double] length min [double] vel max [double] vel min [double] Weighting Factors

Attribute thresholds are applied and boundary candidates are identified if their delta value falls below this threshold. A bonus system is employed to produce better (more context aware) decision making. For example, as pitch contour remains constant, equity is accumulated and then spent (as a weighting bonus) when a change is detected. This bonus "equity" is only applied to the result if delta_pitch passes the adaptive threshold value. Property Definitions pitch_range_percentage = (pitch max pitch_min)/100 [double] onset_range_percentage = (on to on max on_to_on_min)/100 [double] length_range_percentage = (length max length_min)/100 [double] deltaattack = false (from onset to onset) [boolean] deltapitch = false [boolean] deltapitchcontour = false [boolean] contour equity = 0 [double] deltalength = false [boolean] deltavel = false [boolean] deltalonglength = false [boolean] store[] [array of doubles] weight counter = 4 [int] equity counter = 0 [int] booster [double] weighting (confidence value; 0 = definite, 1 = not boundary) [double]

Pseudocode: Apply weighting factor based upon its placement within delta threshold range. FOR ALL NEs: if (NEn. deltapitch = true) if (pitch max = pitch min) then {store[0] = 1} else {store[0] = 1 (

NEnI delta_pitch_change_to_next_pitchpitch_ min) /

(pitch_range_percentage * 0.01)} if (NEn.deltapitchcontour = true) if (pitchcontour = UP or DOWN) then {contour equity = contour equity + (NEn.delta_pitch_to_next_pitch * 0.75)} if (pitchcontour = S AME) then {contour equity = contour equity + 0.025} then {store[0] = store[0] * (1 +

(contour equity / equity counter)} then {weight_counter} else {store[0] = 0} if (NEn.deltaattack = true) if (onto onmax = onto onmin) then {store[l] = 1} else {store[l] = 1 (

NEnI delta attack change to next attack attack_ min) /

(attack_range_percentage * 0.01)} then {weight_counter} else {store[l] = 0} if (NEn.deltalength = true) if (lengthmax = lengthmin) then {store[2] = 1} else {store[2] = (1 ( NEnI delta length change to next length length_ min) /

(length_range_percentage * 0.01)} if (NEn.deltalonglength = true) then {store[2] = store[2] * 1.25} then {weight_counter} else {store[2] = 0} if (NEn.deltaspace = true) then {booster = booster + 0.75} if (NEn H NEnI. delta offset to next onset = 0 && NEn.deltaattack = true) then {booster = booster + 0.25} if (NEn.deltavel = true) then {booster = booster + 0.15} if (weight counter != 0) then {weighting = 1 ( store[0] / weight_counter + [1] / weight_counter + [2] / weight_counter) + booster)} if (weighting < 0) then {weighting = 0}

Java Code public void weightCalculations( ) { System.out .println( ) ;

System.out.println("*** Starting Weight Calculations" ) ; for (int vl=l; vl <= this .getCompleteVoiceLayerLyst ( ) . size( ) ; vl++) { for (int s=l; s <= this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getCompleteSegmentLyst( ) . size( ) ; s++) { // Weight Calculations NoteEventLystltr previous = new

NoteEventLystltr(this .getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . getSegmentNoteEventLyst ( ) . get (I)); // start at beginning of NoteEventLyst

NoteEventLystltr scanner = new

NoteEventLystltr(this .getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . getSegmentNoteEventLyst ( ) . get ( 1+1 ) ) ; // start at beginning+1 of NoteEventLyst double totalweight; double pitch_range_percentage =

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdPitchMax( ) - this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdPitchMin( ) ) / 100; double onset_range_percentage =

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdOnToOnMax( ) - this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdOnToOnMin( ) ) / 100; double length_range_percentage =

(this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdLengthMax( ) - this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdLengthMin( ) ) / 100; double[] result = new double[3]; while (! scanner. atEnd( )) { result[0] = 0; result[l] = 0; result [2] = 0; totalweight = 1; int counter = 4 ; double booster = 0; double contour_equity = 0.0; int equity_counter = 0 ; if ( scanner. getNoteEvent ( ) . get_deltapitch( ) ) { if (this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdPitchMax( ) - this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdPitchMin( ) == 0) {result[0] = 1.0;} // in case max and min are equal else { result[0] = previous .getNoteEvent ( ) .get_delta_pitch_to_next_pi tch() - this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getThresholdPitchMin( ) ; result[0] = 1 - ((result[0] / pitch_range_percentage) * 0.01);

} if ( scanner .getNoteEvent () .get_deltapitchcontour ()) {

// LEGACY ERROR: these two "original" lines should not create new NoteEvents and have been replaced with the following line (NOV 21st) // NoteEvent previous_check = new NoteEvent();

// previous_check = scanner. getValue ( ) .getPrev( ) . getValue ( ) ;

NoteEventLystltr previous_check = new

NoteEventLystltr ( scanner .getValue ( ) .getPrev( ) ) ;

// create new scanner to check for past contour results

NoteEventLystltr scanner2 = new

NoteEventLystltr ( scanner .getValue ( ) ) ; scanner2. deAdvance ( ) ; scanner2. deAdvance ( ) ;

// for the first time through if

( scanner2. getNoteEvent ( ) . get_pitch_contour_to_next _note() == "D" | | scanner2.getNoteEvent ( ) . get_pitch_contour_to_next_ note () == "U") { contour_equity = contour_equity + ( scanner2. getNoteEvent ( ) . get_delta_pitch_to_next_p itch( ) * 0.5); // reducing average delta value by 1/2 for more reasonable bonus amount

//

System. out. println (" Delta Pitch to Pitch is: " + scanner2. getNoteEvent ( ) . get_delta_pitch_to_next_pi tch()); //

System. out .println( " Delta Pitch Change Bonus: " + contour_equity ) ; equity_counter++;

} else { contour_equity = contour_equity + 0.15; // TODO ORIG = 0.25

// System. out .println( " Same to Same Bonus: " + contour_equity ) ; equity_counter++ ;

} while

( scanner2.getValue( ) != this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue (

) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . getSe gmentNoteEventLyst ( ) . get ( 0 ) && previous_check.getNoteEvent ( ) . get_pitch_contour_to

_next_note ( ) == scanner2.getNoteEvent ( ) . get_pitch_contour_to_next_ note()) { if ( scanner2. getNoteEvent ( ) . get_pitch_contour_to_next

_note() == "S") { contour_equity = contour_equity + 0.15; // TODO ORIG = 0.25 //

System. out .println( " Same to Same Bonus: " + contour_equity ) ; equity_counter++ ; } if

( scanner2. getNoteEvent ( ) . get_pitch_contour_to_next _note( ) == "D" I I scanner2.getNoteEvent ( ) . get_pitch_contour_to_next_ note () == "U") { contour_equity = contour_equity +

( scanner2. getNoteEvent ( ) . get_delta_pitch_to_next_p itch( ) * 0.5); // reducing average delta value by 1/2 for more reasonable bonus amount //

System. out. println (" Delta Pitch to Pitch is: " + scanner2.getNoteEvent ( ) .get_delta_pitch_to_next_pi tch()); //

System. out .println (" Delta Pitch Change Bonus: " + contour_equity ) ; equity_counter++ ; } scanner2. deAdvance ( ) ;

} result[0] = (result[0] * (1 + (contour_equity / equity_counter ) ) ) ;

//

System. out .println ( "Equity Counter is: " + equity_counter ) ; //

System. out .println ( "Contour Bonus is: " + (1 + (contour_equity / equity_counter ) ) ) ; contour_equity =

0.0; // reset the contour equity } counter--;

} else {result[0] = 0;} if (scanner. getNoteEvent ( ) .get_deltaattack( ) ) { if

( this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) .getThresholdOnToOnMax( ) - this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) .getThresholdOnToOnMin( ) == 0) {result[l] = 1;} // in case max and min are equal else { result[l] = previous .getNoteEvent ( ) .get_delta_onset_to_next_on set () - this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) .getThresholdOnToOnMin( ) ; result[l] = 1 -

((result[l] / onset_range_percentage) * 0.01) ; } counter--; } else {result[l] = 0;} if

( scanner . getNoteEvent ( ) . get_deltalength ( ) ) { if

( this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) .getThresholdLengthMax( ) - this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) .getThresholdLengthMin( ) == 0) {result [2] = 1;} // in case max and min are equal else { result [2] = previous. getNoteEvent ( ) .get_delta_length_to_next_l ength( ) - this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) .getThresholdLengthMin( ) ; result [2] = 1 - ((result [2] / length_range_percentage) * 0.01);

} if

( scanner .getNoteEvent ( ) .get_deltalonglength( ) ) { result [2] = (result [2] * 1.5 ) ; } // TODO ORIG = 1.25 counter--; } else {result [2] = 0;} if (counter != 0) { if

( scanner .getNoteEvent () .get_deltavel ()) {booster = booster + 0.15;} if

( ( scanner .getNoteEvent ( ) . get_delta_offset_to_next_ onset ( ) == 0.0) | |

( scanner .getValue ( ) .getPrev( ) . getValue ( ) .get_delta _offset_to_next_onset( ) == 0.0)) { if

( scanner .getNoteEvent ( ) .get_deltaattack( ) ) {booster = booster + 0.25;}

} if

( (scanner. getNoteEvent ( ) .get_deltaspace( ) ) ) {booster = booster + 0.5;} // TODO ORIG = 0.75 totalweight = 1 -

(((result[0] / counter) + (result[l] / counter) + (result [2] / counter)) + booster ); if (totalweight < 0) {totalweight = 0;}

} scanner . getNoteEvent ( ) . set_weight ( totalweight

); scanner .advance ( ) ; previous . advance ( ) ;

} // display the calculation results

// this . showWeightCalculations (vl , s);

} } System. out .println( "*** Completed Weight

Calculations" ) ; }

Boundary Identification Examine weighting results (confidence value) and apply a context based adaptive algorithm (using a standard deviation derived threshold) to set definitive boundary points by searching for the lowest (most confident) weightings.

Property Definitions mean = total_weighting / total_NEs standard deviation (using mean) boundary [boolean] weighting [double]

Pseudocode: Define boundaries.

FOR ALL NEs: if NEn+ 1. weighting <= NEn. weighting if NEn. weighting < mean ( standard deviation * 0.80) then {boundary = true}

Java Code public void boundaryOperations ( ) { System.out .println( ) ;

System.out .println( "*** Starting Boundary Operations" ) ; for (int vl=l; vl <= this .getCompleteVoiceLayerLyst ( ) . size( ) ; vl++) { for (int s=l; s <= this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getCompleteSegmentLyst ( ) . size( ) ; s++) { // Boundary Operations int counter = 0; // to keep track of number of Note Events (not 1.0) evaluated double total_weight = 0.0; int total_counter = 0; // to keep track of total NEs present

NoteEventLystltr scannerl = new

NoteEventLystltr(this .getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . getSegmentNoteEventLyst ( ) . get (I)); // start at beginning of NoteEventLyst scannerl . advance ( ) ; // necessary to get max/min to calculate our weighted mean while (! scannerl .atEnd( )) { total_weight = total_weight + scannerl .getNoteEvent ( ) .get_weight( ) ; scannerl . advance ( ) ; total_counter++;

} doublet] std_array = new doublettotal_counter] ;

NoteEventLystltr scanner2 = new

NoteEventLystltr(this .getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) .getSegmentNoteEventLyst ( ) .get( 1) ) ; // start at beginning of NoteEventLyst scanner2. advance ( ) ; for (int a=0; a < (total_counter) ; a++) { std_array[a] = scanner2. getNoteEvent ( ) . get_weight ( ) ; scanner2. advance ( ) ;

}

// calculate weighted mean for threshold double weighted_mean = 0.0; weighted_mean = total_weight/total_counter ; double std = 0.0; for ( int b=0; b < ( total_counter ) ; b++) { double v =

Math.abs ( std_array [b] - weighted_mean) ; std = std + (v*v) ;

} std = (std/total_counter) ; std = Math. sqrt( std) ;

/*

System. out . println ( " Total Weight (" + total_weight + ")/No. Cases (" + total_counter + ") = Weighted Mean: " + weighted_mean) ;

System. out . println ( " Standard Deviation: " + std);

*/ double boundary_threshold = weighted_mean - (std * 0.80); // TODO ORIG = weighted_mean - (std * 0.80) this . complete_voice_layer_lyst . get ( vl ) . getVal ue( ) .setBoundaryThreshold(boundary_threshold) ; // store mastery boundary threshold

NoteEventLystltr scanner3 = new

NoteEventLystltr ( this .getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( s ) . getValue ( ) . getSegmentNoteEventLyst ( ) . get (I)); Il start at beginning of NoteEventLyst scanner3. getNoteEvent ( ) . set_boundary ( true ) ; // set first note event in piece as a START boundary scanner3. advance ( ) ; while ( ! scanner3.atEnd( ) ) { if

( scanner3.getNoteEvent () .get_weight( ) == 1 && !scanner3.atEnd( ) ) { counter++; scanner3. advance ( ) ;

} else { while ( ! scanner3.atEnd2 ( )

&& ( scanner3. getValue ( ) . getNext ( ) . getValue ( ) . get_weig ht() <= scanner3.getNoteEvent ( ) .get_weight ( ) ) ) { // while we are getting lower weighting value in each succesive note event counter++; scanner3. advance ( ) ;

} if ( (counter > 1) &&

( scanner3.current . getValue ( ) . get_weight ( ) < boundary_threshold) ) { scanner3. getNoteEvent ( ) . set_boundary(true ) ;

// scanner3. getValue ( ) . getNext ( ) . getValue ( ) . set_bound ary(true); // ! scanner3.atEnd2 ( ) counter = 0 ;

} else if

( ! scanner3.atEnd( ) ) { // move through LAST events in piece counter++; scanner3. advance ( ) ; } } }

// display the calculation results // this . showBoundaryOperations (vl , s, boundary_threshold) ;

} }

System.out .println( "*** Completed Boundary Operations" ) ;

} public void setSegments( ) { System.out .println( ) ;

System.out .println( "*** Creating Segments"); for ( int vl=l; vl <= this . getCompleteVoiceLayerLyst ( ) . size ( ) ; vl++) {

// Set Segments -- build new segments based on boudary markers

// add each new segment after the current complete list (starting with 2)

// this will create a duplicate set of NEs (312 will become 624) // once the operation has been confirmed (312 did in fact become 624) remove the first segment

NoteEventLystltr scanner = new NoteEventLystltr ( this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( 1 ) . getValue ( ) . getSegmentNoteEventLyst ( ) . get (I)); / / start at beginning of NoteEventLyst (hard coded for 1 Segment with 1 NoteEventLyst int ne_counter = 0; while (! scanner .atEnd2 ()) { if ( scanner .getNoteEvent ( ) . get_boundary ( ) == true) {

NoteEventLyst NE_LYST = new NoteEventLyst ( ) ; // create new NoteEventLyst

// add the initial event NoteEvent ne_input = scanner. getNoteEvent ( ) ;

NE_LYST . addTail ( ne_input ) ; ne_counter++; scanner . advance ( ); // advance scanner to read events within the segment

// read events within the segment while

( scanner .getNoteEvent () . get_boundary ( ) == false) { ne_input = scanner. getNoteEvent ( ) ; NE_LYST . addTail ( ne_input ) ; ne_counter++ ; scanner .advance ( ) ;

}

// display NE add results // System. out. println ("

NE_LYST contains " + NE_LYST. size( ) + " note events" ) ;

// now stick the NE_LYST into a new Segment Segment SEG_LYST = new

Segment (NE_LYST, false); this . getCompleteVoiceLayerLyst ( ) . get ( vl ) . getv alue( ) . getCompleteSegmentLyst ( ) .addTail (SEG_LYST) ; // System. out. println ("

SEG LYST contains " + this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getCompleteSegmentLyst ( ) . size( ) + " segment ( s ) " ) ;

// now get the data out // System.out.println(" SEG contains " + SEG_LYSthis.getSegmentSize( ) + " note event ( s ) " ) ;

} } // wrap-up

// System.out .println( ) ; // System.out .println( "*** Finalizing Segment Creation");

// add the final event to the last segment

NoteEvent last_ne = scanner.getNoteEvent ( ) ; this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue ( ) . getCompleteSegmentLyst ( ) . get (this . getComple teVoiceLayerLyst ( ) .get(vl) . getValue ( ) .getCompleteS egmentLyst( ) .size( ) ) . getValue ( ) .getSegmentNoteEven tLyst ( ) . addTail ( last_ne ) ; ne_counter++; // System.out.println(" final NE added" ) ;

// now get the data out // System.out .println(" final SEG now contains " + this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) .getCompleteSegmentLyst ( ) .get (this .getCompleteVoi ceLayerLyst( ) .get(vl) . getValue ( ) .getCompleteSegmen tLyst ( ) .size( ) ) . getValue ( ) .getSegmentNoteEventLyst ().size() + " note event (s)"); if (ne_counter != this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( 1 ) . getValue ( ) . getSe gmentSize( ) ) {

// System.out .println( "*** Segment Assignment ERROR Detected: Number of original events does NOT match the number of assigned events" ) ;

} else {

// System.out.println( "*** Total of " + ne_counter + " NEs assigned" ) ;

} // remove the first segmment this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getv alue ( ) . getCompleteSegmentLyst ( ) . remove ( 1 ) ; // System. out .println( " first segment removed" ) ;

// final output message

// System. out .println( "*** Number of NEs in first segment: " + this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ) . getCompleteSegmentLyst ( ) . get ( 1 ) . getValue ( ) . getSe gmentSize( ) ) ;

// System. out. println( "*** Total of " + this . getCompleteVoiceLayerLyst ( ) . get (vl ) . getValue ( ). getCompleteSegmentLyst () .size () + " segments created (Voice Layer: " + vl + " ) " ) ;

}

System. out .println( "*** Completed Creating Segments" ) ; }

Motive Identification

Variation Matrix Processing

This method creates a Euclidean based distance matrix variant that searches for attribute patterns (exact repetition and related variations) while ignoring differences in sample size. The comparison of similar attribute patterns allows the system to determine the extent to which events within identified boundaries share common properties. Rejecting the sample size factor supports variation searches within identified boundaries; a prerequisite for segment ballooning. This "variation matrix" method ("VM") is critical throughout the motive identification process. Java Code (pitch attribute only) public double Minimum (double a, double b, double c) { double min = a; if (b < min) {min = b;} if (c < min) {min = c;} return min;

}

/****************************** VARIATION MATRIX public double varMatrix(VoiceLayer vl, Segment s, Segment t, int type) {

/* varMatrix Type Key:

0 = Pitch 1 = Length

2 = Onset

*/

NoteEventLystltr it_source = new

NoteEventLystltr( s . getSegmentNoteEventLyst( ) . get( 1 )); // start at beginning of Segment NoteEventLyst

NoteEventLystltr it_target = new

NoteEventLystltr(t . getSegmentNoteEventLyst ( ) . get ( 1 )); // start at beginning of Segment NoteEventLyst int SegmentDiff = Math.abs(s.getSegmentSize( ) - t.getSegmentSize( ) ) ;

// define arrays to hold candidates segments double[] sourcearray = new double [ s . getSegmentSize ( ) ] ; doublet] targetarray = new double [t . getSegmentSize ( ) ] ;

// populate source array for (int a=0; a < sourcearray. length; a++) { switch (type) { case 0: sourcearray[a] = it_source . getNoteEvent ( ) . get_delta_pitch_to_next_p itch(); break; case 1: sourcearray[a] = it_source. getNoteEvent ( ) .get_delta_length_to_next_ lengthO; break; case 2: sourcearray[a] = it_source. getNoteEvent ( ) .get_delta_onset_to_next_o nset( ) ; break;

} it_source . advance ( ) ;

}

// populate target array for (int b=0; b < targetarray. length; b++) { switch (type) { case 0: targetarray[b] = it_target. getNoteEvent ( ) .get_delta_pitch_to_next_p itch(); break; case 1: targetarray [b] = it_target .getNoteEvent ( ) .get_delta_length_to_next_ length ( ) ; break; case 2: targetarray [b] = it_target .getNoteEvent ( ) .get_delta_onset_to_next_o nset ( ) ; break;

} it_target . advance ( ) ;

} double d[ ] [ ] ; int i; // iterates through s int j; // iterates through t int n = s .getSegmentSize( ) ; // length of s int m = t .getSegmentSize( ) ; // length of t double s_i; // ith position of sourcearray double t_j ; // jth position of targetarray double cost = 0.0; // cost double std = 0.0; // standard deviation double similarity_allowance = 0.0; // for length and onset

// initialize the matrix d = new double [ n+1 ] [m+1 ]; for (i = 0; i <= n; i++) { d[i][0] = i;

} for (j = 0; j <= m; j++) { d[O][j] = j; }

// display temporary results in the terminal window

// System. out. println( ) ;

// System. out .println( "Building Variation Matrix:");

// System. out .println( ) ; if (type == 1) { std = vl.getLengthStandardDeviation( ) ;

} if (type == 2 ) { std = vl .getOnsetStandardDeviation( ) ;

} for (i=l; i <= n; i++) { s_i = sourcearray [ i-1 ]; // set input source for (j=l; j <= m; j++) { t_j = targetarray [ j-1 ] ; // set input source if (type == 1 I I type == 2 ) { similarity_allowance = Math . abs ( ( sourcearray [ i-1 ] -targetarray [ j-1 ] ) ) ;

} if ((s_i == t_j) Il (similarity_allowance < std) ) { cost = 0; // if the candidates are same, there is no cost

// System. out .println( "Cost set to 0 " ) ;

} else { // add 1 to actual distance to get cost cost = 1 + Math . abs ( ( sourcearray [ i-1 ] -targetarray [ j -1 ] ) ) ;

// System. out .println( "Data subtraction result " + Math. abs (( s_i - t_j ) ) ) ;

// System. out .println( "Cost set to " + cost) ;

}

// find path of least resistance d[i][j] = Minimum (d[i-l ] [ j ]+l, d[i][j-l]+l, d[i-l][j-l] + cost);

//d[i][j] = d[i-l][j-l] + cost; } } // display our matrix

// for (int e=0; e <= n; e++) {

// for (int f=0; f <= m; f++) { // floor output (display) // System. out. print ( (Math. floor(d[e] [f] * 1000.000)/ 1000.000) + "\t");

// }

// System. out. println( ) ;

// } // System. out. println( ) ;

// System. out .println( "Variation Matrix Output: " + (d[n][m] - SegmentDiff )); return (d[n][m] - SegmentDiff );

//return (d[n][m]); } public double contourVarMatrix ( Segment s, Segment t) {

NoteEventLystltr it_source = new

NoteEventLystltr ( s . getSegmentNoteEventLyst ( ) . get ( 1 )),' // start at beginning of Segment NoteEventLyst

NoteEventLystltr it_target = new

NoteEventLystltr ( t . getSegmentNoteEventLyst ( ) . get ( 1 ) ) ; // start at beginning of Segment NoteEventLyst int SegmentDiff = Math.abs(s.getSegmentSize( ) - t.getSegmentSize( ) ) ;

// define arrays to hold candidates segments String [ ] sourcearray = new String [ s. getSegmentSize( ) ] ;

String [ ] targetarray = new String [ t . getSegmentSize ( ) ] ;

// populate source array for (int i=0; i < sourcearray . length; i++) { sourcearray [i] = it_source . getNoteEvent ( ) . get_pitch_contour_to_next _note ( ) ; it_source . advance ( ) ;

}

// populate target array for (int i=0; i < targetarray . length; i++) { targetarray [i] = it_target . getNoteEvent ( ) . get_pitch_contour_to_next _note ( ) ; it_target . advance ( ) ;

} double d[ ] [ ] ; int n; // length of s int m; // length of t int i; // iterates through s int j; // iterates through t String s_i; // ith position of sourcearray

String t_j ; // jth position of targetarray double cost; // cost n = s. getSegmentSize ( ) ; m = t . getSegmentSize ( ) ; // initialize the matrix d = new double [ n+1 ] [m+1 ]; for (i = 0; i <= n; i++) { d[i][0] = i;

} for (j = 0; j <= m; j++) { d[O][j] = j; }

// display temporary results in the terminal window

// System. out .println( ) ; // System. out .println( "Building Variation

Matrix : " ) ;

// System. out .println( ) ; for (i = 1; i <= n; i++) { s_i = sourcearray [ i-1 ] ; // set input source for (j = 1; j <= m; j++) { t_j = targetarray [ j-1 ] ; // set input source if (s_i == t_j) { cost = 0; // if the candidates are same, there is no cost

// System. out .println( "Cost set to 0 " ) ;

} else {

// add 1 to actual distance to get cost cost = 1;

// System. out .println( "Data subtraction result " + Math.abs ( ( s_i - t_j ) ) ) ;

// System. out .println( "Cost set to " + cost) ;

}

// find path of least resistance d[i][j] = Minimum (d[i-l ] [ j ]+l, d[i][j-l]+l, d[i-l][j-l] + cost);

//d[i][j] = d[i-l][j-l] + cost; } } // display our matrix for (i = 0; i <= n; i++) { for (j = 0; j <= m; j++) {

// floor output (display) // System. out. print( (Math. floor(d[i] [j] * 1000.000)/ 1000.000) + "\t");

}

// System. out. println( ) ;

} // System. out. println( ) ; // System. out .println( "Variation Matrix Output: " + (d[n][m] - SegmentDiff )); return (d[n][m] - SegmentDiff ); // return (d[n][m]); }

Similarity Ballooning

Searches current segments for inter-segment attribute uniformity and attempts to combine similar consecutive candidates (based on attribute VM comparisons) to create larger, thematically related sections. (Thematically related sections are defined as multi-segment collections containing variation patterns between neighboring NE delta values.) The goal of similarity ballooning is to reduce the overall number of segments by combining thematically similar units to form the largest possible units of internally related motivic material, thus strengthening system understanding of midlevel musical form. Segment Similarity

For each segment, determine pitch, pitch contour, and length similarity without regard to sample size.

Property Definitions primary_segment [segment] secondary_segment [segment] segment_to_test [segment] test_target [segment] voice_layer = current voice layer combine_segments(segment, segment) [segment] vm_pitch(segment, segment) [double] vm_contour(segment, segment) [double] vm_length(segment, segment, voice_layer) [double]

Pseudocode: Define segments. test_target = combine_segments (secondary_segment and segment_to_test) if (vm_pitch(primary_segment, test_target) < 1.5) then {if vm_contour(primary_segment, test_target) < 2} then {if vm_length(primary_segment, test_target, voice_layer) < 0} then {similarity = true} else {similarity = false}

Java Code

public boolean areSegmentsSimilar(VoiceLayer vl, Segment primary, Segment secondary) {

VariationMatrix Matrix = new VariationMatrix( ) ;

// if segments return PITCH similarity of less than 1.5 double pitch_test = Matrix.varMatrix(vl, primary, secondary, 0); if (pitch_test < 1.5) { // was 1.5

System.out .println( " *** Passed Pitch Similarity with: " + pitch_test) ;

// if segments return CONTOUR similarity of less than 2 double contour_test =

Matrix.contourVarMatrix(primary, secondary) ; if (contour_test < 2.0) { // was 2.0

System.out .println( " *** Passed Contour Similarity with: " + contour_test ) ;

// if segments return LENGTH similarity of less than 0 double length_test = Matrix.varMatrix(vl, primary, secondary, 1); if (length_test == 0.0) {

System.out .println( " *** Passed Length Similarity with: " + length_test ) ; return true;

} else {

System. out .println( " *• Failed Length Similarity with: " + length_test ) ; }

} else {

System. out .println( " **** Failed Contour Similarity with: " + contour_test) ;

} } else {

System. out .println( " **** Failed Pitch Similarity with: " + pitch_test ) ;

} return false ;

}

Combine Segments

Add the contents of two adjacent segments, returning a single, larger segment.

Property Definitions a_target [segment] a_target_NE [NE] b_target [segment] b_target_NE [NE] combined segment [segment]

Pseudocode: Combine two adjacent segments, iterate target_a {a_target_NE + combined_segment} iterate target_b {b_target_NE + combined_segment} return {combined segment}

Java Code public Segment combineSegments (Segment a, Segment b) {

// System.out.println(" *** Attempting to Combine Segments"); // System.out .println(" Segment A contains:

" + a.getSegmentSize( ) + " events");

// System.out .println(" Segment B contains:

" + b.getSegmentSize( ) + " events"); // start with new segment

Segment combine = new Segment ( ) ; // System.out .println(" Combined Segment (pre-process) contains: " + combine. getSegmentSize () + " Note Events"); // prepare to scan through a and b

NoteEventLystltr a_scanner = new

NoteEventLystltr( a. getSegmentNoteEventLyst ( ) . get ( 1 ) ) ; // start at beginning of Segment NoteEventLyst

NoteEventLystltr b_scanner = new NoteEventLystltr(b. getSegmentNoteEventLyst ( ) . get ( 1 ) ) ; // start at beginning of Segment NoteEventLyst

// System.out .println(" Attempting segment combination... " ) ; // start with NEs from segment a while ( !a_scanner.atEnd( ) ) { combine . getSegmentNoteEventLyst ( ) . addTail ( a_s canner.getNoteEvent ( ) ) ; a_scanner. advance ( ) ;

}

// System.out .println(" Combined Segment (A only) contains: " + combine. getSegmentSize () + " Note Events" ) ; // append NEs from segment b while ( !b_scanner.atEnd( ) ) { combine. getSegmentNoteEventLyst ( ) . addTail (b_s canner.getNoteEvent ( ) ) ; b_scanner. advance ( ) ;

}

// System.out .println(" Combined Segment (final) contains: " + combine. getSegmentSize () + " Note Events" ) ; // System.out.println(" ***

Combine Segments Complete"); return combine;

Large Segment Ballooning This method compares selected attributes of segments larger than the median segment size for similarity using VM. If candidates pass as similar, the system attempts to "balloon" the smallest candidate by combining it with its smallest neighbor. (NOTE: by first attempting combination using the smaller candidates, the process is made more efficient. If a tie occurs between the neighbors or the candidates themselves, either one may be chosen for initial comparison provided the alternative is immediately considered as well.) VM attribute comparison is once again conducted on the newly ballooned pair. This process is repeated until all candidates have been successfully expanded to their largest potential size while maintaining context-based attribute similarity.

Property Definitions number of segments = total number of segments [int] median segment size = median segment size [int] primary_segment = largest untested segment candidate [segment] secondary_segment = second largest untested segment candidate [segment] current right neighbor = right neighbor of current segment candidate [segment] current left neighbor = left neighbor of current segment candidate [segment] balloon candidate = potential balloon candidate [segment]

Matrix. vm_pitch = VM pitch attribute comparison of primary segment and secondary segment [double]

Matrix.vm contour = VM pitch contour attribute comparison of primary segment and secondary segment [double] Matrix.vm_length = VM length (offsetonset) comparison of primary segment and secondary segment [double] segment_similarity (original_segment, segment_to_test) combine_segments (a_target, b_target)

Pseudocode: Build thematically related sections by combining segments that pass selected attribute VM comparisons. // calculate median segment size if (number_of_segments%2 ==1) {median_segment_size = segment_list /2)} else {median_segment_size = ((number_of_segments/2)l) + (number_of_segments/2)) 12)} FOR ALL SEGMENTS LARGER THAN median segment size: if (Matrix.vm_pitch < 1.5) and (Matrix.vm contour < 2) and (Matrix.vm length ==

0){ if (primary_segment > secondary_segment) or (primary_segment == secondary_segment) { if (current left neighbor > current right neighbor) { balloon candidate = combine segments (secondary segment, current right neighbor)

} if (current left neighbor < current right neighbor) { balloon candidate = combine segments (secondary segment, current left neighbor)

} segment_similarity (primary_segment, balloon_candidate) // test the ballooned candidate if (segment_similarity == true) {update segment_list and rerun method} if (segment_similarity == false) {rerun method starting with next largest candidate}

} if (primary_segment < secondary_segment) { if (current left neighbor > current right neighbor) { balloon candidate = combine segments (primary segment, current right neighbor)

} if (current left neighbor < current right neighbor) { balloon candidate = combine segments (primary segment, current left neighbor) } segment_similarity (secondary_segment, balloon_candidate) // test the ballooned candidate if (segment similarity == true) {update segment list and rerun method} if (segment_similarity == false) {rerun method starting with next largest candidate} }

}

Java Code

public void largeSegmentIteration( ) { System.out .println( ) ;

System.out .println( "*** Starting Large Segment Ballooning Method"); // debug only

VariationMatrix Matrix = new VariationMatrix( ) ; boolean process_complete = false;

// cycle through primary candidates SegmentLystNode primaryNode = new SegmentLystNode ( ) ; while ( !process_complete && primaryNode != null && !process_complete) { // necessary to stop calling getLargestSegmentAboveMedian( )

// System.out.println(" *** Start of Primary Loop" ) ;

// display clean segment list SegmentLystltr clean_seg_scanner = new SegmentLystltr(this .complete_segment_lyst . get (I)); // start at beginning of SegmentLyst System.out .println( ) ;

System.out .println( "Current Segments (" + this.complete_segment_lyst. size () + "):"); int seg_pass = 0; while ( !clean_seg_scanner.atEnd( ) ) { seg_pass++ ;

System.out .print ( " | S" + seg_pass + " ( " + clean_seg_scanner.getSegment ( ) .getSegmentSize( ) + ")\t"); clean_seg_scanner. advance ( ) ;

}

System.out .println( ) ; // get the primrary candidate primaryNode = getLargestSegmentAboveMedian(this .getMedianSize( ) , 0); if (primaryNode == null) { System.out .println( " End of

Primary Candidates (IN MAIN METHOD — Median: " + this.getMedianSize( ) + ")");

// set all LargeBalloonPass flags to true to force stop

SegmentLystltr segment_scanner = new SegmentLystltr

(this.getCompleteSegmentLyst( ) .get( 1) ) ; // start at the beginning of the SegmentLyst while ( ! segment_scanner.atEnd( ) ) { segment_scanner.current .getValue( ) . setLargeBa lloonPrimary(true) ; segment_scanner.current . getValue ( ) . setLargeBa lloonSecondary(true) ; segment_scanner. advance ( ) ;

} process_complete = true; } else if ( !process_complete) {

// find primary candidate index SegmentLystltr primary_seg_scanner = new

SegmentLystltr(this .complete_segment_lyst . get (I)); // start at beginning of SegmentLyst int primary_seg_index = 1; // counter needs to start on 1 because the scanner will bail out one Node early while (primary_seg_scanner.getData( ) != primaryNode) { primary_seg_index++; primary_seg_scanner. advance ( ) ; }

// primary candidate has been selected

System. out .println( ) ; System. out . println ( " * Primary S" + primary_seg_index + "(" + this . complete_segment_lyst . get ( primary_seg_index ) . getValue ( ) . getSegmentSize ( ) + " ) " ) ;

// get the secondary candidate SegmentLystNode secondaryNode = new SegmentLystNode ( ) ; while (secondaryNode != null) { // necessary to stop calling getLargestSegmentAboveMedian( )

// System. out. println (" *** Start of Secondary Loop" ) ; secondaryNode = getLargestSegmentAboveMedian( this .getMedianSize( ) , i); if (secondaryNode == null) {

System. out . println ( " End of Secondary Candidates (IN MAIN METHOD -

- Median: " + this .getMedianSize( ) + " ) " ) ;

// clear secondary and balloon candidate flags SegmentLystltr segment_scanner = new SegmentLystltr (this .getCompleteSegmentLyst () .get ( 1 )); // start at the beginning of the SegmentLyst while ( !segment_scanner.atEnd( ) ) { segment_scanner .current .getValue ( ) . setLargeBa lloonSecondary ( false ) ; segment_scanner . advance ( ) ;

}

// restart with next primary candidate process_complete = true; largeSegmentIteration( ) ; } else {

// find secondary candidate index

SegmentLystltr secondary_seg_scanner = new

SegmentLystltr(this .complete_segment_lyst . get (I)); // start at beginning of SegmentLyst int secondary_seg_index =

1; // counter needs to start on 1 because the scanner will bail out one Node early while

( secondary_seg_scanner.getData( ) != secondaryNode ) { secondary_seg_index++; secondary_seg_scanner. advance ( ) ;

} // secondary candidate has been selected

System.out .println( "

* Secondary S" + secondary_seg_index + "(" + this .complete_segment_lyst . get ( secondary_seg_index ) . getValue ( ) . getSegmentSize ( ) + " ) " ) ;

// check to be sure they are not the same if (primaryNode == secondaryNode ) {

System.out .println( "

* Primary and Secondary Candidates are the same -- skipping ballooning" ) ; } else {

// test candidates for pitch, contour, and length similarity if ( (Matrix.varMatrix(this , primaryNode . getValue ( ) , secondaryNode. getValue ( ) , 0) < 1.5) &&

(Matrix.contourVarMatrix(primaryNode. getValue ( ) , secondaryNode. getValue ( ) ) < 2.0) && (Matrix.varMatrix(this , primaryNode . getValue ( ) , secondaryNode. getValue ( ) , 1) == 0.0)) { System.out .println( " ** Candidates have passed first round comparison tests"); System.out.println(" ** Secondary S" + secondary_seg_index + " ( " + this .complete_segment_lyst . get ( secondary_seg_index ) .getValue( ) .getSegmentSize( ) + " ) passes first round comparison tests");

// test the primary and secondary candidates boolean match =

Matrix. areSegmentsSimilar(this , primaryNode . getValue ( ) , secondaryNode . getValue ( ) ) ; if (match == true) {

System.out .println( " *** MATCH FOUND" ) ;

//

BALLOONING BEGINS HERE if ( !process_complete) {

// determine ballooning type int type = ballooningTypeSelect (primaryNode, primary_seg_index, secondaryNode, secondary_seg_index) ;

//

System.out .println(" *** Type: " + type); if (type != 0 ) { // first ballooning bail

// set indexes based on type int candidate_index = 0, balloon_index = 0; // reset indexes switch (type) { case 1: // System.out .println( " *** Ballooning Candidate (PR) contains: " + this .complete_segment_lyst . get (primary_seg_index+l ) .getValue( ) .getSegmentSize( ) + " note events"); candidate_index = primary_seg_index; balloon_index = primary_seg_index+l ; break;

case 2: // System.out .println( " *** Ballooning Candidate (PL) contains: " + this .complete_segment_lyst . get (primary_seg_index- 1 ) .getValue( ) .getSegmentSize( ) + " note events"); candidate_index = primary_seg_index; balloon_index = primary_seg_index-l ; break;

case 3: // System.out .println( " *** Ballooning Candidate (SR) contains: " + this .complete_segment_lyst . get ( secondary_seg_index +1 ) .getValue( ) .getSegmentSize( ) + " note events"); candidate_index = secondary_seg_index; balloon_index = secondary_seg_index+l ; break;

case 4: // System.out .println( " *** Ballooning Candidate (SL) contains: " + this .complete_segment_lyst . get ( secondary_seg_index -1 ) .getValue( ) .getSegmentSize( ) + " note events"); candidate_index = secondary_seg_index; balloon_index = secondary_seg_index-l ; break;

}

// balloon candidate report if (type != 0) {

System.out .println( " *** Ballooning Candidate type: " + type + " size: " + this .complete_segment_lyst . get (balloon_index) . getv alue( ) .getSegmentSize( ) ) ;

}

// proceed with candidate_index and balloon_index

Segment balloon_candidate = new Segment(); if (balloon_index > candidate_index) { // maintain correct combining order (RIGHT) balloon_candidate =

Matrix.combineSegments (this .complete_segment_lyst . get (candidate_index) .getValue( ) , this .complete_segment_lyst . get (balloon_index) . getv alueO);

System.out .println( " *** Ballooned Candidate contains : " + balloon_candidate.getSegmentSize( ) + " note events" ) ;

} if (balloon_index < candidate_index) { // maintain correct combining order (LEFT) balloon_candidate =

Matrix.combineSegments (this .complete_segment_lyst . get (balloon_index) . getValue ( ) , this .complete_segment_lyst . get (candidate_index) . ge tValue( ) ) ;

System.out .println( " *** Ballooned Candidate contains : " + balloon_candidate.getSegmentSize( ) + " note events" ) ;

};

// test the new candidate boolean balloon_match = false; switch (type) { case 1: System.out.println(" *** Testing

Secondary S" + secondary_seg_index + "(" + secondaryNode.getValue( ) .getSegmentSize( ) + " ) against the Ballooned Candidate ( " + balloon_candidate.getSegmentSize( ) + ")"); balloon_match =

Matrix. areSegmentsSimilar(this , secondaryNode . getValue ( ) , balloon_candidate ) ; break;

case 2 : System.out .println(" *** Testing

Secondary S" + secondary_seg_index + "(" + secondaryNode. getValue ( ) .getSegmentSize( ) + " ) against the Ballooned Candidate ( " + balloon_candidate.getSegmentSize( ) + ")"); balloon_match =

Matrix. areSegmentsSimilar(this , secondaryNode . getValue ( ) , balloon_candidate ) ; break; case 3 :

System.out .println( " *** Testing Primary S" + primary_seg_index + "(" + primaryNode.getValue( ) .getSegmentSize( ) + " ) against the Ballooned Candidate ( " + balloon_candidate . getSegmentSize ( ) + " ) " ) ; balloon_match =

Matrix. areSegmentsSimilar(this , primaryNode . getValue ( ) , balloon_candidate ) ; break;

case 4 :

System.out .println( " *** Testing Primary S" + primary_seg_index + "(" + primaryNode. getValue ( ) . getSegmentSize ( ) + " ) against the Ballooned Candidate ( " + balloon_candidate . getSegmentSize ( ) + " ) " ) ; balloon_match =

Matrix. areSegmentsSimilar(this , primaryNode . getValue ( ) , balloon_candidate ) ; break;

}

if ( !balloon_match && balloon_index != 0) { // if this attempt fails, flag the neighbor and try again

System.out .println( " ***** Ballooning FAILED ( first time ) " ) ;

// balloon match failed, try the "other" candidate switch (type) { case 1 : if (primary_seg_index != 1 && primaryNode.getPrev( ) != secondaryNode ) {type 2;} else {type = 0;} break;

case 2 : if (primary_seg_index+l <= this .getCompleteSegmentLyst ( ) .size( ) && primaryNode . getNext ( ) != secondaryNode) {type i;} else {type = 0;} break;

case 3: if ( secondary_seg_index != 1 && secondaryNode. getPrev() != primaryNode) {type 4;} else {type = 0;} break;

case 4 : if (secondary_seg_index+l <= this .getCompleteSegmentLyst ( ) .size( ) && secondaryNode . getNext () != primaryNode) {type 3;} else {type = 0;} break; if ( type ! = 0 ) {

// set indexes based on type candidate_index = 0; balloon_index = 0;

// reset indexes switch (type) { case 1: // System.out .println( " ***

Ballooning Candidate (PR) contains: " + this .complete_segment_lyst . get (primary_seg_index+l ) .getValue( ) .getSegmentSize( ) + " note events"); candidate_index = primary_seg_index; balloon_index = primary_seg_index+l ; break;

case 2: // System.out .println( " *** Ballooning Candidate (PL) contains: " + this .complete_segment_lyst . get (primary_seg_index- 1 ) .getValue( ) .getSegmentSize( ) + " note events"); candidate_index = primary_seg_index; balloon_index = primary_seg_index- i; break;

case 3: // System.out .println( " *** Ballooning Candidate (SR) contains: " + this .complete_segment_lyst . get ( secondary_seg_index +1 ) .getValue( ) .getSegmentSize( ) + " note events"); candidate_index = secondary_seg_index; balloon_index = secondary_seg_index+l ; break;

case 4: // System.out .println( " *** Ballooning Candidate (SL) contains: " + this .complete_segment_lyst . get ( secondary_seg_index -1 ) .getValue( ) .getSegmentSize( ) + " note events"); candidate_index = secondary_seg_index; balloon_index = secondary_seg_index-l; break;

}

// balloon candidate report if (type != 0) {

System.out .println( " ***

Ballooning Candidate type: " + type + " size: " + this .complete_segment_lyst . get (balloon_index) . getv alue( ) .getSegmentSize( ) ) ;

}

// proceed with candidate_index and balloon_index balloon_candidate = new Segment(); if (balloon_index > candidate_index) {

// maintain correct combining order (RIGHT) balloon_candidate =

Matrix . combineSegments ( this . complete_segment_lyst . get (candidate_index) .getValue( ) , this . complete_segment_lyst . get ( balloon_index ) . getv alueO);

System. out .println( " *** Ballooned Candidate contains: " + balloon_candidate.getSegmentSize( ) + " note events" ) ;

} if (balloon_index < candidate_index) {

// maintain correct combining order (LEFT) balloon_candidate =

Matrix . combineSegments ( this . complete_segment_lyst . get ( balloon_index ) . getValue ( ) , this . complete_segment_lyst . get ( candidate_index ) . ge tValue( ) ) ;

System. out .println( " *** Ballooned Candidate contains: " + balloon_candidate.getSegmentSize( ) + " note events" ) ;

};

// test the new candidate balloon_match = false; switch (type) { case 1: System.out.println(" ***

Testing Secondary S" + secondary_seg_index + " ( " + secondaryNode. getValue ( ) .getSegmentSize( ) + " ) against the Ballooned Candidate ( " + balloon_candidate.getSegmentSize( ) + ")"); balloon match = Matrix. areSegmentsSimilar(this , secondaryNode . getValue ( ) , balloon_candidate ) break;

case 2 :

System.out .println( " *** Testing Secondary S" + secondary_seg_index +

" ( " + secondaryNode. getValue ( ) .getSegmentSize( ) + " ) against the Ballooned Candidate ( " + balloon_candidate . getSegmentSize ( ) + " ) " ) ; balloon_match =

Matrix. areSegmentsSimilar(this , secondaryNode . getValue ( ) , balloon_candidate ) ; break;

case 3 :

System.out .println( " *** Testing Primary S" + primary_seg_index + "("

+ primaryNode. getValue ( ) . getSegmentSize ( ) + " ) against the Ballooned Candidate ( " + balloon_candidate . getSegmentSize ( ) + " ) " ) ; balloon_match =

Matrix. areSegmentsSimilar(this , primaryNode . getValue ( ) , balloon_candidate ) ; break;

case 4 :

System.out .println( " *** Testing Primary S" + primary_seg_index + "("

+ primaryNode. getValue ( ) . getSegmentSize ( ) + " ) against the Ballooned Candidate ( " + balloon_candidate . getSegmentSize ( ) + " ) " ) ; balloon match = Matrix. areSegmentsSimilar(this , primaryNode . getValue ( ) , balloon_candidate ) ; break;

} if ( !balloon_match && balloon_index != 0) { // if this attempt fails, flag the neighbor and try again

System.out .println( " ***** Ballooning FAILED (second time)");

// second try success

} else if (balloon_match == true) { System.out.println(" *****

Ballooning SUCCESS! (second time)");

// assign the balloon_candidate switch (type) { case 1: case 3:

this.getCompleteSegmentLyst( ) .add(candidate_i ndex-1, balloon_candidate) ;

this .getCompleteSegmentLyst ( ) . remove (candidat e_index+l ) ;

this .getCompleteSegmentLyst ( ) .remove (candidat e_index+l ) ; break; case 2 : case 4 :

this .getCompleteSegmentLyst ( ) .add(candidate_i ndex-2, balloon_candidate) ;

this .getCompleteSegmentLyst ( ) . remove (candidat e_index+l ) ;

this . getCompleteSegmentLyst ( ) . remove (candidat e_index+0 ) ; break; }

// reset all flags so we can start over...

SegmentLystltr segment_scanner = new SegmentLystltr

(this. getCompleteSegmentLyst ( ) .get( 1) ) ; // start at the beginning of the SegmentLyst int scan_pass = 0; while ( !segment_scanner.atEnd( ) ) { scan_pass++;

segment_scanner.current .getValue( ) . setLargeBa lloonPrimary( false ) ;

segment_scanner.current .getValue( ) .setLargeBa lloonSecondary( false ) ; segment_scanner. advance ( ) ; // successful ballooning has finished, call the method to start again largeSegmentIteration( ) ; }

} else {

System.out .println( " ***** Ballooning SKIPPED (second time)");

}

// original balloon success

} else if (balloon_match == true) {

System.out .println( " ***** Ballooning SUCCESS! (first time)");

// assign the balloon_candidate switch (type) { case 1 : case 3 :

this .getCompleteSegmentLyst ( ) .add(candidate_i ndex-1, balloon_candidate) ;

this .getCompleteSegmentLyst ( ) . remove (candidat e_index+l ) ;

this . getCompleteSegmentLyst ( ) . remove (candidat e_index+l ) ; break;

case 2 : case 4 :

this . getCompleteSegmentLyst ( ) . add(candidate_i ndex-2, balloon_candidate) ;

this .getCompleteSegmentLyst ( ) . remove (candidat e_index+l) ;

this. getCompleteSegmentLyst ( ) .remove (candidat e_index+0 ) ; break;

// reset all flags so we can start over...

SegmentLystltr segment_scanner = new SegmentLystltr (this. getCompleteSegmentLyst ( ) .get( 1) ) ; // start at the beginning of the SegmentLyst int scan_pass = 0; while ( !segment_scanner.atEnd( ) ) { scan_pass++;

segment_scanner.current . getValue ( ) . setLargeBa lloonPrimary( false ) ;

segment_scanner.current .getValue ( ) .setLargeBa lloonSecondary( false ); segment_scanner. advance ( )

// successful ballooning has finished, call the method to start again largeSegmentIteration( ) ;

} // end of ballooning success report

} // first ballooning bail

} // end of ballooning if loop

} // end of BALLOONING } // end initial candidate test for pitch, contour, and length similarity loop

} // end initial checks for same nodes } // end of secondary process

(else)

} // end of secondary while loop } // end of primary process (else) } // end of primary while loop "PROCESS COMPLETE LOOP"

// display a final message System. out .println( "*** Completed Large Segment Ballooning" ) ;

} // end of method

// external method called during Large Segment

Ballooning public SegmentLystNode getLargestSegmentAboveMedian ( double medianSize , int type) {

// scan for the largest segment above the median size SegmentLystltr segment_scanner = new

SegmentLystltr (this .getCompleteSegmentLyst () .get ( 1 )); // start at the beginning of the SegmentLyst

Segment largestSegment = new Segment(); // used during scanning, and is discarded to prevent data corruption int scan_pass = 1, largestlndex = 0; while ( !segment_scanner.atEnd( ) ) {

// test to see that segment has not been flagged, is larger than the median, and is larger than previous discoveries switch (type) { case 0: if ( ! segment_scanner.getSegment ( ) .getLargeBalloonPrim ary( ) && segment_scanner.getSegment ( ) .getSegmentSize( ) >= medianSize && segment_scanner.getSegment ( ) .getSegmentSize( ) > largestSegment. getSegmentSize( ) ) { largestSegment = segment_scanner.getSegment ( ) ; largestlndex = scan_pass;

} break; case 1 : if

( !segment_scanner. getSegment ( ) .getLargeBalloonSeco ndary( ) && segment_scanner. getSegment ( ) .getSegmentSize( ) >= medianSize && segment_scanner.getSegment ( ) .getSegmentSize( ) > largestSegment. getSegmentSize( ) ) { largestSegment = segment_scanner. getSegment ( ) ; largestlndex = scan_pass;

} break;

} scan_pass++; // increment counter to keep track of Segment index number segment_scanner. advance ( ) ;

}

// if a result was found, flag successful candidate if (largestlndex != 0) { switch (type) { case 0 : this . getCompleteSegmentLyst ( ) . get ( largestlnde x) . getValue ( ) . setLargeBalloonPrimary(true ) ; break; case 1 : this . getCompleteSegmentLyst ( ) . get ( largestlnde x). getValue ( ) . setLargeBalloonSecondary(true ) ; break;

}

// if no result was found, reset secondary flags and return null } else {

// reset LargeBalloonPass flags (secondary only) segment_scanner = new SegmentLystltr (this .getCompleteSegmentLyst () .get ( 1 )); // start at the beginning of the SegmentLyst while ( ! segment_scanner.atEnd( ) ) { segment_scanner.current .getValue ( ) . setLargeBa lloonSecondary( false ) ; segment_scanner. advance ( ) ;

}

System.out .println( " End of Secondary Candidates (balloon flags reset)"); return null; }

// return the successful candidate index return this . getCompleteSegmentLyst ( ) . get ( largestlndex) ;

}

// external method called during Large Segment

Ballooning public int ballooningTypeSelect (SegmentLystNode primaryNode, int primary_seg_index, SegmentLystNode secondaryNode, int secondary_seg_index) { /* Type Key:

1 = Primrary Right (PR)

2 = Primrary Left (PL) 3 = Secondary Right (SR)

4 = Secondary Left (SL) */ int type = 0 ; if (primaryNode.getValue( ) .getSegmentSize( ) < secondaryNode.getValue( ) .getSegmentSize( ) ) { // primary segment is smaller int min = secondaryNode.getValue( ) .getSegmentSize( ) ; if (primary_seg_index+l <= this .getCompleteSegmentLyst ( ) .size( ) && primaryNode . getNext ( ) != secondaryNode ) { // if PR exists and does not overlap, set as min min = primaryNode . getNext ( ) .getValue( ) .getSegmentSize( ) ; // begin with the primary right neighbor System.out.println(" *** PR: "

+ min) ; type = 1;

} if (primary_seg_index != 1 && primaryNode. getPrev( ) != secondaryNode && primaryNode. getPrev( ) .getValue( ) .getSegmentSize( ) < min) { // if PL exists, does not overlap, and is smaller than previous candidates, set as min min = primaryNode . getPrev( ) . getValue ( ) . getSegmentSize ( ) ;

System.out.println(" *** PL: " + min) ; type = 2;

} } if ( secondaryNode . getValue ( ) . getSegmentSize ( ) <= primaryNode. getValue ( ) . getSegmentSize ( ) ) { // secondary segment is smaller (or equal) int min = primaryNode . getValue ( ) . getSegmentSize ( ) ; if ( secondary_seg_index+l <= this .getCompleteSegmentLyst ( ) .size( ) && secondaryNode . getNext () != primaryNode) { // if SR exists and does not overlap, set as min min = secondaryNode . getNext ( ) . getValue ( ) . getSegmentSize ( );

System.out.println(" *** SR: " + min) ; type = 3 ;

} if ( secondary_seg_index != 1 && secondaryNode.getPrev( ) != primaryNode && secondaryNode.getPrev( ) .getValue( ) .getSegmentSize( ) < min) { // if SL exists, does not overlap, and is smaller than previous candidates, set as min min = secondaryNode.getPrev( ) .getValue( ) .getSegmentSize(

);

System. out. println (" *** SL: " + min) ; type = 4 ; } } return type ; }

Small Segment Ballooning

Same as large segment ballooning however, only candidates smaller than the median segment size are considered.

Thematic Segment Finalization

Split Point Candidates

Tidyup method that searches for uncharacteristically large offset/onset gaps between consecutive NEs within currently defined segment boundaries. As before, this method adapts the required judgment criteria from general data trends. First, standard deviation is calculated based on the inter-quartile mean to provide a statistical measure of central tendency. Gap candidates are then selected if they lie more than 4 standard deviations outside the inter-quartile mean. Once a potential gap candidate has been identified, the method calculates mean-based standard deviation for the NE gaps within the localized segment. If the original candidate lies outside 2 standard deviations of the inter-segment mean, the gap is identified as a split point.

Property Definitions total [double] iq_mean (interquartile mean) [double] std (standard deviation using interquartile mean) [double] calcarray = new double[get_complete_note_event_list().get_number_of_note_events()] [array of doubles] event counter [int] quartile = get_complete_note_event_list() . get_number_of_note_events()/4.0 [double] modifier [double] fractional low [double] fractional high [double] Java Code public void segmentFinalization(SegmentLyst s) {

System.out .println( ) ; System.out .println( "*** Starting Segment

Finalization" ) ; // debug only double total=0.0; double iq_mean = 0; // interquartile mean for central tendency double std = 0; doublet] calcarray = new double [this . numberOfNoteEvents ] ; int event_counter = 0 ; double quartile = this .numberOfNoteEvents/4.0 ;

// if ( (int)quartile == 0) {quartile = 1.0;} // needed for segments with fewer than 4 NEs double modifier = 0.0; double fractional_low = 0.0; double fractional_high = 0.0; int split_event_index_left = 0; int split_event_index_right = 0; for (int i = 1; i < s.size()+l; i++) {

// System.out.println( "*** Output for Segment " + i + " : " ) ;

Segment current_seg = s . get ( i ) . getValue ( ) ; for (int p = 0; p < current_seg.getSegmentSize( ) ; p++) { if (p+1 != s.size( ) ) { calcarray[p] = current_seg.getSegmentNoteEventLyst( ) .get (p+1) .get Value ( ) . get_current_offset_to_next_onset ( ) ; event_counter++; total= total + calcarray[p] ; // addition for std totals

} else { calcarray[p] = current_seg. getSegmentNoteEventLyst ( ) . get (p+1 ) . get Value ( ) . get_current_offset_to_next_onset ( ) ; event_counter++;

} // end of IF that calculates gaps } // end of FOR loop that scans the events of each segment

} // end of FOR loop that moves through the segments

System.out .println( " Counted " + event_counter + " events (total should be " + this.numberOfNoteEvents + " ) " ) ;

Java.util .Arrays . sort (calcarray); // sort the NEs Offset to next Onset

// calculate the outer fractional values in case modifier is needed for ( int n=0; n < this.numberOfNoteEvents; n++) {

// System.out.print (" Offset = " + calcarray[n] ) ; if ((n+1) >= (int)quartile && (n+1) <= this.numberOfNoteEvents - ( int )quartile) { // add only those values that fall within the second and third quartiles

} if ((n+1) == (int)quartile) { // get the lower of the two fractional values to be used as modifiers fractional_low = calcarray[n] ;

} if ((n+1) == ( (int)quartile*3) ) { // get the upper of the two fractional values to be used as modifiers fractional_high = calcarray[n] ; } } // grab the modifier multiplier if (this.number0fNoteEvents%4 != 0) { modifier = quartile - ( int )quartile; // System.out.println(" Modifier = " + modifier) ;

} // calculate the IQM if (modifier != 0) { iq_mean = (total + modifier* ( fractional_low+fractional_high) ) / (quarti le*2); } else { iq_mean = total/ (quartile*2 );

}

// System.out.println(" IQM = " + iq_mean); // Standard Deviation Calculation (based on IQM) for (int j=0; j <= (quartile*2 ) ; j++) { double v = Math.abs(calcarray[ j ] - iq_mean) ; std = std + (v*v) ; }

System.out .println(" Quartile x 2 (half the NEs): " + quartile*2) ; std = (std/(quartile*2) ) ; std = Math. sqrt( std) ; System.out.println(" IQM Standard

Deviation: " + std); for (int i=l; i <= this . getCompleteSegmentLyst ( ) . size ( ) ; i++) {

// System.out .println(" * Output for Segment " + i + " : " ) ;

Segment current_seg = s . get ( i ) . getValue ( ) ; double seg_std = interSegmentOffsetStandardDeviation(current_seg) ; // report result

System.out .println( ) ;

System.out .println(" Standard Gap Deviation for Segment " + i + ": " + seg_std) ; current_seg. setStandardGapDeviation( seg_std) ;

// compare event gaps with the STD double candidate = 0.0; for (int k = 0; k < current_seg.getSegmentSize( ) ; k++) { if (k+1 != current_seg.getSegmentSize( ) ) { if

( current_seg . getSegmentNoteEventLyst ( ) . get ( k+1 ) . ge tValue( ) . get_current_offset_to_next_onset ( ) > std*4) { // looks for internal gaps outside the 4 standard deviations of all offset to onset values

System. out .println( ) ; System. out . println ( "

** Segment " + i + ": SPACE BETWEEN EVENT " + (k+1) + " AND " + (k+2) + " is " + current_seg . getSegmentNoteEventLyst ( ) . get ( k+1 ) . get Value ( ) .get_current_offset_to_next_onset ( ) + " and may be a split candidate!"); candidate = current_seg . getSegmentNoteEventLyst ( ) . get ( k+1 ) . get Value ( ) . get_current_offset_to_next_onset ( ) ; split_event_index_left = k+1; split_event_index_right = k+2; // // debug display only

// for (int p = 0; p <

(current_seg.getSegmentSize( ) ) ; p++) { // if (p+1 != current_seg.getSegmentSize( ) ) {

// System. out .println ("

SPACE BETWEEN THIS AND THE NEXT EVENT: " +

( double ) current_seg . getSegmentNoteEventLyst ( ) . get ( p+1 ) . getValue ( ) . get_current_offset_to_next_onset ( )

);

// } else {

// System. out .println ("

** Final Event: data ignored"); //

System. out .println ( ) ;

// } // end of IF that calculates gaps

// } // end of FOR loop that scans the events of each segment

// evaluate the candidate against the standard gap deviation if (candidate >

( current_seg . getStandardGapDeviation ( ) *2 ) ) { System. out . println ( "

*** Candidate: " + candidate + " is larger than std*2: " + (current_seg.getStandardGapDeviation( )*2) ) ;

System. out . println ( "

*** SUCCESS: appears to be a reasonable gap- based split point!");

// determine type of finalization operation required if

( split_event_index_left == 1) {

System. out .println (" *** Split candidate appears the first NE in the segment");

// move it to the previous segment if (i == 0) {

// check for start of the Segment list

System. out .println (" **** Unable to move split candidate -- previous segment doesn't exist (should be checked to see if it should be its own single event segment? ) " ) ;

} else {

System. out .println (" **** Moving candidate to the previous segment ..."); s .get ( i-

1 ) . getValue ( ) . getSegmentNoteEventLyst ( ) . addTail ( cu rrent_seg . getSegmentNoteEventLyst ( ) . get ( 1 ) . getValu e ( ) ) ; current_seg. getSegmentNoteEventLyst ( ) . remove ( i);

System. out .println (" **** Move complete. Previous S" + (i-1) + "(" + s.get(i-

1) . getValue ( ) .getSegmentSize( ) + ") | Candidate S" + (i) + "(" + s.get(i) . getValue ( ) .getSegmentSize( ) + ")");

} } else if

( split_event_index_right == current_seg.getSegmentSize( ) ) {

System. out .println (" *** Split candidate appears the last NE in the segment"); // move it to the following segment if (i+1 > current_seg.getSegmentSize( ) ) { // check for end of segment list

System.out .println( " **** Unable to move split candidate — following segment doesn't exist (should be checked to see if it should be its own single event segment? ) " ) ;

} else {

System.out .println( " **** Moving candidate to the following segment ..."); s . get ( i+1 ) . getValue ( ) . getSegmentNoteEventLyst ( ) .addHead(current_seg. getSegmentNoteEventLyst ( ) .g et(current_seg.getSegmentSize( ) ) . getValue ( ) ) ; current_seg. getSegmentNoteEventLyst ( ) . remove ( current_seg.getSegmentSize( ) ) ;

System.out .println( " **** Move complete. Candidate S" + (i) + "(" + s.get(i) . getValue ( ) .getSegmentSize( ) + ") | Following S" + (i+1) + "(" + s. get (i+1) . getValue ( ) .getSegmentSize( ) + " ) " ) ;

}

} else {

System.out .println( " *** Split candidate appears to be in the middle of the segment");

VariationMatrix

Matrix = new VariationMatrix( ) ; Segment firstHalf = new Segment(); double medianSize = this.calculateMedian( ) ;

// create temporary segment from NEs up to and including first split candidate for ( int n=l; n <= split_event_index_left; n++) { firstHalf . getSegmentNoteEventLyst ( ) . addTail (c urrent_seg.getSegmentNoteEventLyst ( ) .get(n) .getVal ue ( ) ) ;

}

// System. out .println( " *** First Half segment size: " + firstHalf .getSegmentSize( ) ) ;

// combine temporary segment with previous segment if (i != 1) { // check for start of segment list

Segment firstHalfPrevious = new Segment(); firstHalfPrevious = Matrix. combineSegments ( s .get ( i-1 ) .getValue( ) , firstHalf ) ;

// check inter-segment consistency double firstHalfPrevious_std = interSegmentOffsetStandardDeviation ( firstHalfPrevi ous ) ;

// report first half segment std result

System. out .println( ) ;

System. out .println( " Standard Gap Deviation for First Half Previous Segment: " + firstHalfPrevious_std) ;

// scan for gaps boolean gap_found = interSegmentGapScan( firstHalfPrevious, std, firstHalfPrevious_std) ;

// report gap discovery result if

( !gap_found) {

System. out .println( " NO GAP discovered in First Half Previous Segment -- checking size for testing" ) ;

// if newly combined segment is smaller than (or equal to) the median size, test against previous small ballooning candidates if

( firstHalfPrevious .getSegmentSize( ) <= medianSize) {

System.out .println( " First Half Previous Segment (" + firstHalfPrevious. getSegmentSize( ) + " ) is smaller than median ( " + medianSize + " ) -- testing against other small segments for strict similarity" ) ;

SegmentLystltr segment_scanner = new SegmentLystltr (this.getCompleteSegmentLyst( ) .get( 1) ) ; // start at the beginning of the SegmentLyst boolean match = false; int match_count = 0 ; while ( ! segment_scanner.atEnd( ) ) { if ( segment_scanner.current .getValue( ) != current_seg && segment_scanner.getData( ) .getValue( ) .getSegmentSiz e() <= medianSize) { match = Matrix. areSegmentsSimilar(this, segment_scanner.getSegment( ) , firstHalfPrevious ); if (match) {match_count++; }

} if ( segment_scanner.current .getValue( ) == current_seg) {

System.out .println( " Same Segment SKIPPED");

} segment_scanner. advance ( ) ;

} if (match_count != 0) {

System. out .println( " " + match_count + " MATCH(ES) FOUND! (First Half Previous Segment Small)") ;

System. out .println( " First Half NEs will be moved to the previous segment");

// combine firstHalf with the previous segment for ( int 1=1; 1 <= split_event_index_left ; 1++) { s .get ( i- 1 ) . getValue ( ) . getSegmentNoteEventLyst ( ) . addTail ( cu rrent_seg . getSegmentNoteEventLyst ( ) . get ( 1 ) . getValu e ( ) ) ;

}

// remove the firstHalf NEs from the current_seg for (int 1=1; 1 <= split_event_index_left ; 1++) {

current_seg . getSegmentNoteEventLyst ( ) . remove ( i);

} System. out .println( " **** Move complete.

Previous S" + (i-1) + "(" + s.get(i- 1) . getValue ( ) .getSegmentSize( ) + ") | Candidate S" + (i) + "(" + s.get(i) . getValue ( ) .getSegmentSize( ) + ")"); }

// if newly combined segment is larger than (or equal to) the median size, test against previous large ballooning candidates if

( firstHalfPrevious .getSegmentSize( ) > medianSize)

{ System.out .println( " First Half Previous

Segment (" + firstHalfPrevious. getSegmentSize( ) + " ) is larger than median ( " + medianSize + " ) -- testing against other large segments for strict similarity" ) ;

SegmentLystltr segment_scanner = new SegmentLystltr

(this.getCompleteSegmentLyst( ) .get( 1) ) ; // start at the beginning of the SegmentLyst boolean match = false; int match_count = 0 ; while ( ! segment_scanner.atEnd( ) ) { if ( segment_scanner.current .getValue( ) != current_seg && segment_scanner.getData( ) .getValue( ) .getSegmentSiz e() > medianSize) { match = Matrix. areSegmentsSimilar(this, segment_scanner. getSegment ( ) , firstHalfPrevious ) ; if (match) {match_count++; }

} if ( segment_scanner.current .getValue( ) == current_seg) {

System.out .println( " Same Segment SKIPPED" ) ; } segment_scanner. advance ( ) ;

} if (match_count != 0) {

System. out .println( " " + match_count + " MATCH(ES) FOUND! (First Half Previous Segment Large)");

System. out. println (" First Half NEs will be moved to the previous segment"); // combine firstHalf with the previous segment for ( int 1=1; 1 <= split_event_index_left ; 1++) { s .get ( i-

1 ) . getValue ( ) . getSegmentNoteEventLyst ( ) . addTail ( cu rrent_seg . getSegmentNoteEventLyst ( ) . get ( 1 ) . getValu e ( ) ) ;

}

// remove the firstHalf NEs from the current_seg for (int r=l; r <= split_event_index_left ; r++) {

current_seg. getSegmentNoteEventLyst ( ) . remove ( i); }

System. out .println (" **** Move complete. Previous S" + (i-1) + "(" + s.get(i- 1) . getValue ( ) .getSegmentSize( ) + ") | Candidate S" + (i) + "(" + s.get(i) . getValue ( ) .getSegmentSize( ) + ")"); }

} else {

System. out. println (" GAP DISCOVERED in First Half Previous Segment -- testing as stand alone segment");

// discard "First Half Previous" and test First Half Segment as "stand alone" candidate

// if

First Half Segment is smaller than (or equal to) the median size, test against previous small ballooning candidates if ( firstHalf .getSegmentSize( ) <= medianSize) {

System. out .println (" First Half Segment (" + firstHalf .getSegmentSize( ) + ") is smaller than median ( " + medianSize + " ) -- testing against other small segments for strict similarity" ) ; SegmentLystltr segment_scanner = new

SegmentLystltr

(this .getCompleteSegmentLyst () .get ( 1 )); // start at the beginning of the SegmentLyst boolean match = false; int match_count = 0 ; while ( ! segment_scanner .atEnd( ) ) { if

( segment_scanner .getData( ) .getValue( ) .getSegmentSi ze() <= medianSize) { match = Matrix. areSegmentsSimilar ( this, segment_scanner . getSegment ( ) , firstHalf ) ; if (match) {match_count++; } } segment_scanner. advance ( ) ;

} if (match_count != 0) { // should be != 0

System. out .println( " " + match_count + " MATCH(ES) FOUND! (First Half Segment Small)");

System. out .println( " First Half Segment should stand alone -- to be inserted into segment list");

// insert firstHalf into segment list s.add(i, firstHalf);

// remove the firstHalf NEs from the current_seg for ( int r=l; r <= split_event_index_left ; r++) {

current_seg . getSegmentNoteEventLyst ( ) . remove ( i);

} System. out. println (" **** insert complete. Previous S" + (i-1) + "(" + s.get(i- 1) .getValue( ) .getSegmentSize( ) + ") | Candidate S" + (i) + "(" + s.get(i) .getValue( ) .getSegmentSize( ) + " ) I Total Number of Segments : " + this . complete_segment_lyst . size ( ) ) ;

}

} else if ( firstHalf .getSegmentSize( ) > medianSize) { // if First Half Segment is larger than (or equal to) the median size, test against previous large ballooning candidates

System. out .println( " First Half Segment (" + firstHalf .getSegmentSize( ) + ") is larger than median ( " + medianSize + " ) -- testing against other large segments for strict similarity" ) ; SegmentLystltr segment_scanner = new

SegmentLystltr

(this .getCompleteSegmentLyst () .get ( 1 )); // start at the beginning of the SegmentLyst boolean match = false; int match_count = 0 ; while ( ! segment_scanner .atEnd( ) ) { if

( segment_scanner .getData( ) .getValue( ) .getSegmentSi ze() > medianSize) { match = Matrix. areSegmentsSimilar ( this, segment_scanner . getSegment ( ) , firstHalf ) ; if (match) {match_count++; } } segment_scanner. advance ( ) ;

} if (match_count != 0) { // should be != 0

System. out .println( " " + match_count + " MATCH(ES) FOUND! (First Half Segment Large)");

System. out .println( " First Half Segment should stand alone -- to be inserted into segment list"); // insert firstHalf into segment list s.add(i, firstHalf)

// remove the firstHalf NEs from the current_seg for ( int r=l; r <= split_event_index_left; r++) {

current_seg.getSegmentNoteEventLyst ( ) . remove ( i);

}

System.out .println( " **** Insert complete. Previous S" + (i-1) + "(" + s.get(i- 1) .getValue( ) .getSegmentSize( ) + ") | Candidate S" + (i) + "(" + s.get(i) .getValue( ) .getSegmentSize( ) + " ) I Total Number of Segments : " + this.complete_segment_lyst. size ( ) ) ; }

} } } else { System.out.println(" **** Unable to combine mid-split candidate -- previous segment doesn't exist");

} } } else {

System.out .println( "

*** Candidate: " + candidate + " is smaller than std*2: " +

(current_seg.getStandardGapDeviation( )*2) ) ; System.out .println( "

*** FAILED: to pass gap-based split point criteria" ) ;

} } }

} } // end of FOR loop that moves through the segments

// display a final message System. out .println( "*** Completed Segment Finalization" ) ; }

Boundary Split

If split point result occurs with a single NE on either side, the gap isolated NE is removed from the current segment and added to the closest neighbor. Mid-Segment Split

Otherwise, NE combination adjustments on each side of the split point are tested to find a "best fit" resolution. NEs to the left of the midsegment split are combined with the left neighbor segment and tested against all remaining segments for multiple attribute similarity using the variation matrix method. If no reasonable match is found, the same procedure occurs with NEs to the right of the midsegment split. New segments are created as necessary to accommodate groupings that don't match any of the remaining segments. Motive and Variation Data Mining

Using a sliding ballooning window data scan method, the system searches within each thematic segment (beginning with the largest) for internal motivic repetition or variation patterns. Repetition and variation is determined using our variation matrix comparison method (pitch and pitch contour attributes). As previously noted, studies in music cognition strongly suggest that beginnings of patterns play a critical role in determining pattern recognition. For this reason, the motive discovery windowing process begins at the start of each thematic segment and slides forward from there. The motive identification process occurs within individual segments only.

This final data mining is successful because it relies heavily upon the robust results achieved by the adaptive segmentation and ballooning processes described above. It is the combination of these two processes (adaptive segmentation and context- aware formal discovery) that allows the windowed scan to reliably identify musically valuable motivic information.

Property Definitions pass counter = 0 [int] balloon_pass = 0 [int] primary_window [array of NE attribute values] target_window [array of NE attribute values] primary number of events [int] primary_window_position = 0 [int] target_window_position = primary_window_position + 3 [int]

Pseudocode: Identify motive matches using a ballooning window data scanning technique.

FOR ALL SEGMENTS LARGER THAN 5 (FROM LARGEST TO SMALLEST): for (primary_number_of_events5)

{ primary_window[0] = pitch_to_next_pitch(NEprimary_window_position) primary_window[l] = pitch_to_next_pitch(NEprimary_window_position+l) if (primary_window[0] == primary_window[l]) {primary_window_position++} else { target_window[0] = pitch_to_next_pitch(NEtarget_window_position+pass_counter) target_window[l] = pitch_to_next_pitch(NEtarget_window_position+ 1 +pass_counter) while (primary_window == target_window) { primary_window[l+balloon_pass] = pitch_to_next_pitch(NEprimary_window_position+ 1 +balloon_pass) target_window[l+balloon_pass] = pitch_to_next_pitch(NEtarget_window_position+l+pass_counter+balloon_pass) balloon_pass++

} if (balloon_pass > 0 ) {return motive}

} primary_window_position++ reset balloon_pass

} Java Code double d [ ] [ ] ; int n = 2; // size of source window (delta values) int m = 2; // size of target window (delta values) double current_comparison = 0.0; double previous_comparison = 0.0;

// define arrays to hold candidates segments double[] sourcearray = new double [n]; doublet] targetarray = new double [m] ; int match_count = 0 ; boolean primary_comparison_same = false; for (int i = 1; i < s .get_number_of_segments ( )+l ; i++) { // control segment advancement segment primary = s . indexreturn( il ) . getData( ) ; int pass = 0; // count number of passes for (int a = 0; a <

( s . get_segment_at_index ( i ) . get_number_of_note_even ts ( ) 5 ) ; a++) { // control window slide advancement match_count = 0; // reset the match counter

// only consider segments with more than 5 NEs if

((s. get_segment_at_index ( i ) . get_number_of_note_eve nts( ) > 5) && (pass+1 < s . get_segment_at_index ( i ) . get_number_of_note_event s())) { for (int p = 0; p < n; p++) { sourcearray [p] = primary .get_segment_note_events_list( ) .indexreturn (p+pass) .getData( ) .get_current_pitch_to_next_pitch

O;

} previous_comparison = 0.0; // reset the previous comparison data for (int r = 0; r < n; r++) { current_comparison = sourcearray [r ] ; // check primary array for duplication at the beginning

(repeated notes/changes) if (current_comparison == previous_comparison)

{primary_comparison_same = true;} previous_comparison = current_comparison; // update current comparison

System. out. print ("NE" + (r+pass+1) + "" + (r+pass+2) + ": ") ;

System. out .print ( sourcearray [r] + ", ");

} if (primary_comparison_same == true)

{System. out .println( "Primary values are the same skipping analysis" ) ; } else {System. out .println( "Primary values are the different continuing analysis" ) ; } int round = 0 ;

// check that we don't search beyond the segment end, and that the source data isn't the same while ((round+pass < s . get_segment_at_index ( i ) . get_number_of_note_event s()5)

&& (primary_comparison_same == false)) { targetarray [ 0 ] = primary .get_segment_note_events_list( ) .indexreturn

( 3+round+pass ) .getData( ) . get_current_pitch_to_next _pitch(); targetarray [ 1 ] = primary .get_segment_note_events_list( ) .indexreturn

( 4+round+pass ) .getData( ) . get_current_pitch_to_next

_pitch ( ) ; // local implementation of Variation Matrix int k; // iterates through s int j; // iterates through t double s_k; // ith position of sourcearray double t_j ; // jth position of targetarray double cost; // cost d = new double [ n+1 ] [m+1 ]; for (k = 0; k <= n; k++) {d[k][0] = k;} for (j = 0; j <= m; j++) {d[O][j] = j ; } for (k = 1; k <= n; k++) { s_k = sourcearray [kl ] ; // set the input source for (j = 1; j <= m; j++) { t_j = targetarray [ jl ] ; // set the input source if ( s_k == t_j ) {cost = 0; // if the candidates are the same, then there is no cost} else {cost = 1 + Math. abs( (sourcearray [kl ] targetarray [ ji]));}

// find the path of least resistance d[k][j] = Minimum (d[kl][ j]+l, d[k][jl]+ 1, d[kl][

Jl]

+ cost) ; }

} int SegmentDiff = Math.abs(nm) ;

// balloon the candidates if exact match is found if (d[n][m] SegmentDiff == 0.0) { int balloon_pass = 1; boolean balloon_continue = true; doublet] balloon_source_array = new double [ s . get_segment_at_index ( i ) . get_number_of_not e_events ( ) ] ; doublet] balloon_target_array = new double [ s . get_segment_at_index ( i ) . get_number_of_not e_events ( ) ] ; while (balloon_continue == true) { // master ballooning control balloon_source_array [ 0 ] = sourcearray [ 0 ]; balloon_source_array [ 1 ] = sourcearray [ 1 ]; balloon_target_array[0] = targetarray [ 0 ]; balloon_target_array [ 1 ] = targetarray [ 1 ]; if ( (4+round+pass+2+balloon_pass) <= s . get_segment_at_index ( i ) . get_number_of_note_event s( ) &&

(balloon_pass+pass+3) <

( 4+round+pass+l+balloon_pass ) ) { // check for end of segment and primary collision with target balloon_source_array [ l+balloon_pass ] = primary .get_segment_note_events_list( ) . indexreturn

( l+pass+balloon_pass ) . getData( ) .get_current_pitch_to_next_pitch( ) ; balloon_target_array[ l+balloon_pass] = primary .get_segment_note_events_list( ) .indexreturn

( 4+round+pass+balloon_pass ) . getData( ) . get_current_pitch_to_next_pitch ( ) ;

// be sure last two target candidates are not same as the first two primary candidates if ( (balloon_target_array [ ( l+m+balloon_pass)2 ]

!= balloon_source_array [ 0 ] ) &&

( balloon_target_array [ ( l+m+balloon_pass ) 1 ]

!= balloon_source_array [ 1 ] ) ) { // run local match test d = new double [ n+l+balloon_pass ] [m+l+balloon_pass ] ; for (k = 0; k <= n+balloon_pass; k++) {d[k][0] = k;} for (j = 0; j <= m+balloon_pass; j++) {d[O][j] = j;} for (k = 1; k <= n+balloon_pass; k++) { s_k = balloon_source_array[kl] ;

// set the input source for (j = 1; j <= m+balloon_pass; j++) { t_j = balloon_target_array [ jl ] ;

// set the input source if (s_k == t_j ) {cost = 0; // if the candidates are the same, then there is no cost} else {cost = 1 +

Math . abs ( ( balloon_source_array [ kl ] balloon_ target_array [ j 1 ] ) ) ; }

// find the path of least resistance d[k][j] = Minimum (d[kl][ j]+l, d[k][jl]+

1, d[kl][

Jl]

+ cost) ; }

}

SegmentDiff = Math. abs ( (n+balloon_pass) ( m+balloon_pass) ) ; if (d[n+balloon_pass ] [m+balloon_pass ] SegmentDiff == 0.0) {

System. out .println( " Ballooning Successful!"); match_count++; balloon_continue = true; } else {

System. out .println( " Ballooning Aborted Candidates to not match" ) ;

//primary_starting_position = 0; balloon_continue = false;

}

} else { System. out .println( " Ballooning Aborted Repeat of Motive Detected" ) ; //primary_starting_position = 0; balloon_continue = false;

} } else {

System. out .println( " Ballooning Aborted End of Segment or Segment Collision Detected");

//primary_starting_position = 0; balloon_continue = false; } // end of nested match ballooning (nested for data check) balloon_pass++;

}

} round++;

}

} else if

( s . get_segment_at_index ( i ) . get_number_of_note_even ts() < 5 ) { System. out .println( " Contains " + s . get_segment_at_index ( i ) . get_number_of_note_event s() + " note events skipping analysis" ) ;

} else { System. out .println( " End of Segment Detected");

}

System. out .printIn (match_count + " matches found ! " ) ; primary_comparison_same = false; // reset the primary comparison value pass++; } }

Discovered motivic patterns can be stored and compared against the remaining candidates to determine its prototypical form and made available for further application specific processing. Optional Operation Post-Processing

For certain post-processing applications, it may be necessary for model data to exist in two forms:

1) Style Tagged: Data initially provided to the system is tagged with a predetermined style association for purposes of categorization and software training. This approach is similar to the way humans acquire and process novel information; or 2) Analysis-Based Classification: Groupings are inferred once the appropriate amount of input data is present. Algorithms parse the data looking for relationships between the various input streams and identify relevant connections. The result expands and enhances the useful style repertoire and maintains an approach similar to human-based induction. Auditory Specific Processing

The frequency analysis process is to be tested on exposed (separated) audio layers with the aim of detecting pitch and timber changes relative to a known tempo/beat grid.

Median Filters

Nonlinear digital filtering used to remove noise from the input data stream. Results are stored for further analysis.

Frequency Analysis Median Filters are applied to the Frequency Tracking output at predetermined intervals (for example, 50ms) to search for areas where the analysis results are within a range of 70 cents (0.7 semitones). (NOTE: In terms of octave point decimal notation, one semitone is a difference of 0.08333...) Timbre Analysis IFD is applied to detect the presence of specific partials. Predefined bands check for changes in harmonic content over time and determine when significant change has occurred. Results are provided as an indicator value and stored for further stylistic analysis.

Segment Function Assignment Function analysis may be used to build larger phrase-based musical forms based on previously analyzed models. Initially these models are added as manual input, but eventually become integral to the system's comparative reading of the analysis data.

Function Analysis Vertical and approach interval tensions are combined with representations of duration and metric emphasis. Measurable units are applied to these attributes in order to allow for analysis computation. Phrases may be defined and a grouping average determined.

Automated Style Classification Additional classification relationships are identified once the necessary data is present. This approach expands system applications by suggestion musically appropriate substitutions when alternative solutions are desired. This discovered relationship demonstrates resonance between the input data and the inductive association necessary to create connections. Context Development

When possible, auditory and manual analysis and classification data are combined to create a comprehensive picture of musical style characteristics.

Industrial Application One application of the system and method disclosed herein is in the quantification of substantial similarity between or among a plurality of musical data sets. Such quantification would be useful in judicial proceedings where copyright infringement is alleged, and there exists a need for testimony regarding the similarities between the accused musical work or performance and one or more of the plaintiffs musical works and/or performances. Heretofore, expert musicologists have provided expert testimony based on artistic qualitative measures of similarity. Using the method and system of the present invention, however, will permit quantitative demonstrations of similarities in a wide range of characteristics of the music, allowing a high degree of certainty about copying, influence, and the like.

While the invention has been described in its preferred embodiments, it is to be understood that the words which have been used are words of description rather than of limitation and that changes may be made within the purview of the appended claims without departing from the true scope and spirit of the invention in its broader aspects. Rather, various modifications may be made in the details within the scope and range of equivalents of the claims and without departing from the spirit of the invention. The inventor further requires that the scope accorded his claims be in accordance with the broadest possible construction available under the law as it exists on the date of filing hereof (and of the application from which this application obtains priority,) and that no narrowing of the scope of the appended claims be allowed due to subsequent changes in procedure, regulation or law, as such a narrowing would constitute an ex post facto adjudication, and a taking without due process or just compensation.

Claims

I claim as my invention: 1 . In a computer system, a method for characterizing a data set representative of music comprising: a. identifying within the data set at least one subset of data representing melody; b. identifying within said subset of data representing melody at least one subset of data representing at least one motive. 2. The method of Claim 1 wherein the data set representative of music comprises encoded digital audio. 3. The method of Claim 1 wherein the data set representative of music comprises encoded performance information. 4. The method of Claim 1 wherein the data set representative of music comprises both encoded performance information and encoded digital audio. 5. The method of Claim 1 wherein the data set representative of music comprises information representative of one or more of the following characteristics: phrase structure, measure information, tempo information, section identifiers, stylistic attributes, exact pitch, onset, offset, velocity, and note density. 6. The method of Claim I wherein the motive is identified using adaptive melodic segmentation. 7. The method of Claim 1 wherein the step of identifying the subset of data representing melody comprises: a. identifying note events within the data set; b. determining at least one attribute for each note event, the attributes being selected from among pitch, onset, length and velocity; and c. determining the attributes of difference between consecutive note events. 8. The method of claim 7 wherein the step of identifying note events is optionally preceded by at least one preprocessing step selected from the group of: equal loudness contour filtering, spectral pitch tracking, tempo extraction, instantaneous frequency distribution analysis, and style identification. 9. The method of Claim 7 wherein the step of identify the subset of data representing melody additionally comprises: a. determining pitch contour between each note event pair and determining note length for each note event. 10. The method of Claim 9 further comprising the step of segmenting the subset of data representing melody into motive segments. I I . The method of Claim 10 wherein the segmentation is performed by a method comprising: a. defining thresholds of change of the attributes or contours; b. weighting the defined thresholds; and c. determining boundaries within the subset of data representing melody; d. identifying motive segments between the boundaries. 12. The method of Claim 1 1 wherein consecutive motive segments are combined into a single, larger motive segment by similarity ballooning. 13. A computer-implemented system for characterizing a data set representative of music comprising: a. means for identifying within the data set at least one subset of data representing melody; and b. means for identifying within said subset of data representing melody at least one subset of data representing at least one motive. 14. A method for quantitatively assessing the similarities between or among a plurality of data sets representative of music wherein one or more accused data sets represents an alleged infringement of copyright in one or more proprietary data sets comprising the steps of: a. Processing each of the data sets to identify the motives therein by adaptive melodic segmentation; and b. Comparing the motives identified to establish the degree of similarity between or among the data sets.
PCT/US2007/089225 2007-12-31 2007-12-31 System and method for adaptive melodic segmentation and motivic identification WO2009085054A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2007/089225 WO2009085054A1 (en) 2007-12-31 2007-12-31 System and method for adaptive melodic segmentation and motivic identification

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
CA 2707621 CA2707621A1 (en) 2007-12-31 2007-12-31 System and method for adaptive melodic segmentation and motivic identification
GB201009632A GB201009632D0 (en) 2007-12-31 2007-12-31 System and method for adaptive melodic seqmentation and motivic identification
PCT/US2007/089225 WO2009085054A1 (en) 2007-12-31 2007-12-31 System and method for adaptive melodic segmentation and motivic identification
US12777448 US8084677B2 (en) 2007-12-31 2010-05-11 System and method for adaptive melodic segmentation and motivic identification
US13336581 US20120144978A1 (en) 2007-12-31 2011-12-23 System and Method For Adaptive Melodic Segmentation and Motivic Identification

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US12777448 Continuation US8084677B2 (en) 2007-12-31 2010-05-11 System and method for adaptive melodic segmentation and motivic identification

Publications (1)

Publication Number Publication Date
WO2009085054A1 true true WO2009085054A1 (en) 2009-07-09

Family

ID=40824582

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2007/089225 WO2009085054A1 (en) 2007-12-31 2007-12-31 System and method for adaptive melodic segmentation and motivic identification

Country Status (3)

Country Link
CA (1) CA2707621A1 (en)
GB (1) GB201009632D0 (en)
WO (1) WO2009085054A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6574594B2 (en) * 2000-11-03 2003-06-03 International Business Machines Corporation System for monitoring broadcast audio content
US20060190450A1 (en) * 2003-09-23 2006-08-24 Predixis Corporation Audio fingerprinting system and method
US20070113724A1 (en) * 2005-11-24 2007-05-24 Samsung Electronics Co., Ltd. Method, medium, and system summarizing music content

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6574594B2 (en) * 2000-11-03 2003-06-03 International Business Machines Corporation System for monitoring broadcast audio content
US20060190450A1 (en) * 2003-09-23 2006-08-24 Predixis Corporation Audio fingerprinting system and method
US20070113724A1 (en) * 2005-11-24 2007-05-24 Samsung Electronics Co., Ltd. Method, medium, and system summarizing music content

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
'Proceedings of the International Conference on Music Perception and Cognition 2002', UNIVERSITY OF NEW SOUTH WALES, SYDNEY, AUSTRALIA article WEYDE, T: 'Integrating Segmentation and Similarity in Melodic Analysis', pages 240 - 243 *

Also Published As

Publication number Publication date Type
GB2468080A (en) 2010-08-25 application
GB201009632D0 (en) 2010-07-21 grant
CA2707621A1 (en) 2009-07-09 application

Similar Documents

Publication Publication Date Title
Gouyon et al. A review of automatic rhythm description systems
Mauch et al. Simultaneous estimation of chords and musical context from audio
Dubnov et al. Using machine-learning methods for musical style modeling
Brossier Automatic annotation of musical audio for interactive applications
US20030023421A1 (en) Music database searching
Ellis Prediction-driven computational auditory scene analysis
Clarke Empirical methods in the study of performance
Bod Memory-based models of melodic analysis: Challenging the Gestalt principles
Pearce The construction and evaluation of statistical models of melodic structure in music perception and composition
US20060065102A1 (en) Summarizing digital audio data
Humphrey et al. Moving Beyond Feature Design: Deep Architectures and Automatic Feature Learning in Music Informatics.
Mauch Automatic chord transcription from audio using computational models of musical context
Dixon et al. Towards Characterisation of Music via Rhythmic Patterns.
Casey et al. Content-based music information retrieval: Current directions and future challenges
Tillmann et al. Implicit learning of musical timbre sequences: statistical regularities confronted with acoustical (dis) similarities.
Bello et al. Techniques for Automatic Music Transcription.
Krumhansl Music psychology: Tonal structures in perception and memory
Benetos et al. Automatic music transcription: challenges and future directions
Eerola et al. MIR In Matlab: The MIDI Toolbox.
Harte et al. Detecting harmonic change in musical audio
Peeters Deriving musical structures from signal analysis for music audio summary generation:“sequence” and “state” approach
Cambouropoulos Musical parallelism and melodic segmentation:: A computational approach
Kostek Perception-based data processing in acoustics: applications to music information retrieval and psychophysiology of hearing
US6225546B1 (en) Method and apparatus for music summarization and creation of audio summaries
Typke Music retrieval based on melodic similarity

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07870143

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase in:

Ref document number: 2707621

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 2707621

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 1009632.9

Country of ref document: GB

ENP Entry into the national phase in:

Ref document number: 1009632

Country of ref document: GB

Kind code of ref document: A

Free format text: PCT FILING DATE = 20071231

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct app. not ent. europ. phase

Ref document number: 07870143

Country of ref document: EP

Kind code of ref document: A1