DE102006056106A1 - Signal classifier manufacturing and utilizing device, has visualization module and two level classifier modules formed to perform manufacturing and using signal classifiers in parameter modified knowledge discovery in databases process - Google Patents

Abstract

The device has a visualization module for producing visualized representation of transformed series data and visualized representation of classification decision and/or annotation facts of signal classifiers. Two level classifier modules produce visualized representation of sub-symbolic signatures and representation of level classifiers, for classification and/or annotation of time series data, respectively. The modules are formed in order to manufacture and use the signal classifiers in a parameter modified knowledge discovery in databases process. Independent claims are also included for the following: (1) a method for manufacturing level classifiers for an event in a time series (2) a method for extraction of trifold level classifier signature (3) a method for extraction of an amount of symbolic signatures for an event class of signals (4) a computer program with a program code for performing a method for manufacturing and using signal classifiers.

Description

1.1 Subject of the invention

The The invention relates to a system and a multi-stage method for Production and Application of Signal Classifiers (hereafter also SK). These are computer-implemented machines for classifying digital time series (e.g., audio data).

1.2 Background of the Invention

In Many technical applications may require specific Data patterns in discrete time series using signal classifiers to locate or classify. The necessary signal classifiers have to meet a number of requirements. Their behavior depends on a large number of parameters. It are usually complicated machines in one complicated procedures with the help of computer scientists, programmers and / or engineers must be created. With the help of the described Invention makes it possible Significantly reduce the effort involved in their manufacture.

1.3 State of the art

In The past has been a multitude of procedures and algorithms for the classification of data patterns in time series. It is an important area of computer science. Signal classifiers can be found, for example, in speech recognition systems or in devices for Monitoring of medical data (e.g., ECG, EEG).

Specific Systems and processes that specifically target the complex process of Optimize the knowledge-based production of signal classifiers including design, implementation and testing, can be found in the scientific literature and in the patent literature Not.

1.4 Object of the invention

According to the invention is a complex multilevel supervised System and method which describes the areas of design, implementation, Testing, modification and application of signal classifiers.

1.5 Formal Foundations and Notation

A time series s → is a time-discrete measurement series of equidistant PCM-coded measured values. A time series with N elements can be understood as a vector of dimension N or as a point in an N-dimensional space:

A time series corpus A is a set of time series in the form of files:

Each file s → _m A can be assigned exactly one label. The set = {0, ..., M - 1} is the set of all file labels of the time series corpus A.

Each file s → _m ε A has an explicit time stamp, which indicates the start time tm / 0 of the start of the measurement series. The elements (measured values) in the s → _m do not have an explicit timestamp. The sampling frequency δ of the time series in A is constant. The time t of the i-th element in each s → _m ε A is implicitly given by:

bezeichnet. Eine nicht näher spezifizierte Subsequenz einer Datei in A wird im Folgenden kurz mit x → bezeichnet. Die Länge l einer Sequenz ist ihre Kardinalität l(x →) = |x →|. Die zeitliche Ausdehnung Δt einer Sequenz ist gegeben durch:

A sequence of adjacent elements in s → from an element with the time t ₁ to an element with the time t ₂ , t ₁ <t ₂ is through

designated. An unspecified subsequence of a file in A is briefly referred to below as x → net. The length l of a sequence is its cardinality l (x →) = | x → |. The temporal extension Δt of a sequence is given by:

One phenomenon is a phenomenon that is directly or indirectly perceptible to the senses is. Directly perceptible are e.g. acoustic phenomena that directly with the sense of hearing can be perceived. Indirectly perceivable are e.g. acoustic phenomena that one scientific visualization (e.g., a spectrogram) may be required to be perceived. An acoustically perceptible phenomenon in the following is called the short acoustic phenomenon. A visually perceptible phenomenon in the following is called visual phenomenon for short.

1.5.1 First-level classifiers

ensteht, so dass V = υ(x →) (1.5) The two-dimensional visualization V of a sequence x → of length l is an x × y matrix over R with the temporal extent .delta.t V = Δt (x →) passing through an illustration

arises, so that V = υ (x →) (1.5)

erhalten, wobei gilt ϕA = EA(x →ϕ) (1.6) The representation of an acoustic phenomenon is a sequence x → φ the length l, in which by playing clearly a certain acoustic phenomenon can be audibly perceived. The acoustic signature φ ^{A of} an acoustic phenomenon is a vector x ~ φ the length l ', which x → φ represents, but easier than x → φ is constructed. The acoustic signature is indicated by a picture

obtained, where applicable φ A = E A (x → φ ) (1.6)

The representation of a visual phenomenon is a visualization V ^φ , in which a specific phenomenon can be clearly perceived visually by graphical representation. The visual signature φ ^{V of} a visual phenomenon is an x '× y' matrix over R, which represents the phenomenon observable in V ^φ , but is simpler than V ^φ .

A visual signature is obtained by mapping E ^V : {V (x × y, R)} → E ^V : {V (x '× y', R)} such that φ V = E V (υ (x → φ )) (1.7)

One phenomenon in A is called elementary, if it exactly characterized by exactly one signature and represents can be. The signature of an elementary phenomenon is hereafter also elementary signature.

in einem s →_m ∊ A, m < M begrenzt. Es existiert für jedes wahrnehmbare Phänomen in A eine akustische oder eine visuelle Signatur. Alle akustischen Phänomene, die in einer Untersuchung von Interesse sind, können durch eine Menge akustischer Signaturen

repräsentiert werden. Alle visuellen Phänomene, die in einer Untersuchung von Interesse sind, können durch eine Menge visueller Signaturen

repräsentiert werden.Corollary: Every perceptible acoustic or visual phenomenon in A is on a subsequence

in s → _m ε A, m <M limited. There is an acoustic or a visual signature for every perceptible phenomenon in A. All acoustic phenomena that are of interest in a study may be due to a lot of acoustic signatures

be represented. All visual phenomena that are of interest in a study may be due to a lot of visual signatures

be represented.

The elementary phenomena domain Φ of an SK manufacturing process is defined by all the acoustic and visual signatures of interest in a given context such that

sei die Menge aller Klassenlabel der Problemdomäne Φ.Each signature φ _n ε Φ can be assigned exactly one class label. The amount

Let be the set of all class labels of the problem domain Φ.

sei die Menge aller KKF der Problemdomäne. Eine KKF c_n ∊ C, erhält als Input eine Sequenz x → der Länge 1. Durch c_n wird der Hypothese, dass in x → das der Signatur ϕ_n entsprechende Phänomen enthalten ist, eine reellwertige Maßzahl zuordnet. Diese Maßzahl wird im Folgenden auch Klassenähnlichkeit s (Similarity) genannt. Eine KKF ist somit eine Funktion c : R^l → R, so dass s = c(x →), c ∊ C (1.9) Each signature φ _n ε Φ can be assigned a signature-specific class correlation function (also referred to below as KKF). The amount

Let be the amount of all KKF of the problem domain. A KKF c _n ε C, receives as input a sequence x → of length 1. By c _n , the hypothesis that in x → that the signature φ _n corresponding phenomenon is included, assigns a real-valued measure. This measure will be referred to below as class similarity s (similarity). A KKF is thus a function c: R ^l → R, so that s = c (x →), c ∈ C (1.9)

von First-Level-Annotationsalgorithmen (im Folgenden auch First-Level-AA). Ein First-Level-AA γ_n ∊ Γ sucht in A mit Hilfe der KKF c_n ∊ C Instanzen des Elements ϕ_n ∊ Φ. Zeigt die KKF in einem bestimmten Intervall einen Wert an, der einem definierten Kriterium entspricht (z.B. einen festgesetzten Ähnlichkeitsschwellwert überschreitet), wird von dem First-Level-AA ein Annotationsfaktum generiert.The quantity C corresponds to a quantity

first-level annotation algorithms (hereafter also first-level AA). A first-level AA γ _n ε Γ searches in A with the help of the KKF c _n ∈ C instances of the element φ _n ε Φ. If the KKF displays a value in a certain interval that corresponds to a defined criterion (eg exceeds a set similarity threshold), the first-level AA generates an annotation factor.

A first-level SK (hereinafter FLC = First-Level Classifier) κ is a 3-tuple of the form κ = (φ, c, γ), φ ∈ Φ, c ∈ C, γ ε T (1.10)

von FLC. Ein FLC κ ∊ K erhält als Input eine Datei ss → ∊ A und generiert daraus eine abzählbare Menge F^κ = {F κ / 0, F κ / 1, ...} von FLC-Annotationen, d.h. κ : A → F^κ.The elementary phenomenon domain Φ corresponds to the set

from FLC. An FLC κ ε K receives as input a file ss → ε A and generates therefrom a countable set F ^κ = {F κ / 0, F κ / 1, ...} of FLC annotations, ie κ: A → F ^κ ,

An FLC annotation F ^K / i ∈ F ^K in time-frequency space is a 5-tuple of the form F κ i = (ψ, ω, r, f, Λ) (1.11)

Here, ψ ε Ψ is a file label, ω ε Ω is the class label of the signature belonging to κ, r is a time in A, f is a frequency value, and Λ = {λ ₁ , λ ₂ , ...} is an optionally populated list of variables, which may be required in a specific context.

Corollary: The goal of the fabrication process of FLC is to define in a given context an elementary phenomena domain Φ and to find a suitable set K of FLC with which the phenomena belonging to Φ in A can be reliably located. With the aid of K, the set F ^{K of} the FLC annotations can then be created from A by machine.

1.5.2 Second-Level Classifiers

Often are patterns in signals and signal sequences too complex to pass through Model FLC alone. But many complex phenomena can be considered as constellations of elementary phenomena be understood in the time-frequency space.

A symbolic element φ ^s characterizes the position of an elementary signature φ ε Φ in one Section of the time-frequency space. It can be as 4-tuple of the form φ s = (ω, r, f, Λ) (1.12) be understood. Here, ω ε Ω is the class label of the elementary signature, r is a time relative to a hypothetical zero point, f is a frequency value, and Λ = {λ ₀ , λ ₁ ,...} Is a countable list of variables to be optionally populated.

aufgefasst werden.The symbolic signature of a complex phenomenon describes a constellation of symbolic elements in a section of the time-frequency space. It can be considered as a set of form

be understood.

One complex phenomenon in A is called symbolically describable, if it by a constellation can be exactly characterized by symbolic elements. each symbolically describable phenomenon in A can be represented by a symbolic signature.

The symbolic problem domain Φ ^{S of} a manufacturing process is defined by all symbolic signatures of interest in a given scientific context, such that

Each symbolic signature φ ^S ε Φ ^S can be assigned exactly one class label. The set Ω ^S = {0, ..., M - 1} is the set of all class labels of the symbolic problem domain Φ ^S.

The search for complex phenomena in A can be understood as a search for instances of symbolic signatures from Φ ^S in a suitable set F ^K of FLC annotations. In F ^K , time-frequency constellations of annotation facts must be found which correspond to the constellations of the symbolic elements of φ ^S ε Φ ^S.

The set Φ ^S corresponds to an amount Γ ^S = {γ S / 0,..., Γ S / M-1} of second-level annotation algorithms (hereinafter also second-level AA). A second-level AA γ S / i ε Γ ^S searches in F ^K with the aid of a suitable merger rule instances of the element φ ^S ε Φ ^S. If a constellation of facts in F ^K corresponds to a criterion defined by the fusion rule, then an annotation factor is generated by the second-level AA.

A second-level SK π (hereinafter also SLC = second-level classifier) can be used as a 2-tuple of the form π = <φ s , γ s >, Φ S Ε Φ S , γ S Ε Γ S (1.15) be understood. The symbolic problem domain Φ ^S corresponds to a set P = {π ₀ , ..., π _M-1 } of SLC. An SLC π ε P receives as input a set F ^K of FLC annotations and generates therefrom a countable set F ^π = {F π / 0, F π / 1, ...} of SLC annotations, ie π: F ^K F ^π .

A second-level annotation is a 6-tuple of the form F P i = <Ψ, ω S , τ, f, Λ, Θ> (1.16)

Here, ψ ε Ψ is a file label, ω ^S ε Ω ^{S is} the class label of the symbolic signature to be searched, τ is a time in A, f is a frequency value and Λ = {λ ₀ , λ ₁ , ...} is a list of optionally populated Variables. Θ is a symbolic signature modified by the instantiation. The modification is carried out by the AA or the fusion rule. Modifications typically affect the positions of symbolic elements in time-frequency space.

Corollary: The goal of the manufacturing process of SLC is to define a symbolic domain domain Φ ^S in a given context and to find a suitable set P of SLC with which the phenomena belonging to Φ ^S in F ^K can be reliably located. With the aid of P, the set F ^{P of} the SLC annotations can then be generated automatically from F ^K.

1.6 General procedure

• 1. First-Level-SK (FLC) basieren auf elementaren Signaturen (Kennlinien) von Phänomenen und arbeiten direkt über den Zeitreihen. Mit Hilfe einer Menge K von FLC wird über einer Menge A von Zeitreihen die Menge F^K von FLC-Annotationen generiert.
• 2. Second-Level-SK (SLC) basieren auf symbolischen Signaturen und arbeiten über der Menge F^K. Auf Grundlage einer Menge P von SLC generiert ein Suchverfahren über F^K die Menge F von SLC-Annotationen.

In the formal model presented above, basically two different types of SK are distinguished for the annotation or classification of time series: first-level and second-level SK (see also 3.1 ).

• 1. First-level SK (FLC) are based on elementary signatures (characteristics) of phenomena and work directly over the time series. Using a set K of FLC, the set F ^K of FLC annotations is generated over a set A of time series.
• 2. Second-level SK (SLC) are based on symbolic signatures and work over the set F ^K. Based on a set P of SLC, a search method over F ^K generates the set F of SLC annotations.

The Quantities K and P correspond to two manufacturing problems. Hereinafter is shown as the manufacture of both classifier types by KDD methods make sense supports can be.

1.6.1 The FLCD process

FLC optimized for a particular domain can be understood as a special form of expert knowledge. The production of FLC is a knowledge-intensive process, except for the simplest cases. FLCs can be generated, evaluated and refined as part of a modified KDD process (see [2]). It results in the 3.2 illustrated FLC discovery process (FLCD process).

The The goal of the FLCD process is to work in a given context to define an elementary phenomena domain Φ and to find an appropriate amount K of FLC with their help the phenomena associated with Φ Quantity A of time series files reliably localized and annotated can be.

• 1. Zeitreihen-Daten-Basis: Die Zeitreihen-Daten müssen unter Berücksichtigung des Vorwissens und des jeweiligen Aufgabestellung in geeigneter Art und Weise aufgezeichnet werden.
• 2. Zeitreihen-Daten-Interface: Um mit Zeitreihen-Daten umgehen zu können, bedarf es eines hierfür geeigneten Zeitreihen-Daten-Interfaces, welches Werkzeuge zur wissenschaftlichen Visualisierung und zur Navigation in den Zeitreihen-Daten zur Verfügung stellt.
• 3. Zeitreihen-Daten-Selektion: In den meisten Fällen ist es notwendig, aus einer Menge von rohen Zeitreihen-Daten eine Teilmenge auszuwählen, die für im gegebenen Herstellungskontext am geeignetsten erscheint. In die Entscheidung darüber, welche Daten in den nachfolgenden Schritten weiterverwendet werden, fließen Betrachtungen über Inhalt, Qualität, Quantität und Struktur der Daten ein. Es ergibt sich eine Menge von vorläufigen Zieldaten.
• 4. Zeitreihen-Daten-Bereinigung: Die Menge der vorläufigen Zeitreihen-Zieldaten muss in der Regel noch bereinigt, d.h. von unbrauchbaren (gestörten, verzerrten, zu leisen) Aufnahmen befreit werden. Die verbleibenden Daten werden bereinigte Zieldaten A' genannt.
• 5. FLC-Date-Mining: Das FLC-Data-Mining zielt darauf ab, neuartige und interessante elementare Signaturen zu finden. Diese können durch geeignete Data-Mining-Verfahren generiert oder verfeinert werden. Das FLC-Data-Mining umfasst folgende Schritte: (a) Wahl des Data-Mining-Algorithmus: Es muss ein konkreter Algorithmus zum Finden von elementaren Signaturen gewählt werden. Dieser Schritt beinhaltet detaillierte Betrachtungen über Parameter und mögliche Modelle sowie das Gesamtumfeld des FLCD-Prozesses. (b) Anwendung des Data-Mining-Algorithmus: Die Anwendung des Data-Mining-Algorithmus zur Suche nach elementaren Signaturen kann unsupervised oder supervised ablaufen. Im Falle einer überwachten Anwendung des Data-Mining-Algorithmus müssen die Zwischenergebnisse evaluiert werden. Vom Data-Mining-Algorithmus entdeckte Signaturen können automatisch mit Klassfikationsfunktionen zu FLC verknüpft werden. (c) Evaluation: Eine erfolgreiche Anwendung des Data-Mining-Algorithmus auf die bereinigten Zieldaten A' resultiert in einer Menge K_TMP von vorläufigen, nicht-evaluierten FLC. Diese müssen bewertet werden. Im Bedarfsfalle kann der Data-Mining-Schritt wiederholt werden. Es entsteht die Menge K der evaluierten FLC.
• 6. Die Menge K: Das Produkt des FLCD-Prozesses ist eine Menge K von FLC. Diese Menge modelliert Wissen mit Referenz auf die Ebene elementarer Phänomene in den Zeitreihen-Daten. Dieses Wissen kann zur Annotation der Zieldaten in A' verwendet werden, es kann aber auch für sich interessante Informationen über die den Zeitreihen-Daten zugrundeliegenden Prozesse enthalten.
• 7. Anwendung: Durch Anwendung der Klassifikatoren in K auf A' entsteht die Menge F^K der FLC-Annotationen. Diese Menge kann entweder direkt verwendet oder in weiterführenden KDD-Schritten auf höherer Ebene ausgewertet werden.

The following list shows the individual steps of the FLCD process. Each step is interactive and can be iterated if necessary.

• 1. Time series data basis: The time series data must be recorded in a suitable manner taking into account the prior knowledge and the respective task.
• 2. Time series data interface: In order to deal with time series data, a suitable time series data interface is required, which provides tools for scientific visualization and navigation in the time series data.
• 3. Time Series Data Selection: In most cases, it is necessary to select from a set of raw time series data a subset that appears most appropriate for the given manufacturing context. When deciding which data to continue to use in subsequent steps, consideration is given to the content, quality, quantity, and structure of the data. There is a lot of preliminary target data.
• 4. Time Series Data Cleansing: The amount of preliminary time series target data usually needs to be adjusted, ie freed from useless (disturbed, distorted, too soft) recordings. The remaining data is called cleaned target data A '.
• 5. FLC Date Mining: FLC data mining aims to find novel and interesting elementary signatures. These can be generated or refined by means of suitable data mining methods. FLC data mining involves the following steps: (a) Choice of the data mining algorithm: a concrete algorithm for finding elementary signatures must be chosen. This step includes detailed considerations about parameters and possible models as well as the overall environment of the FLCD process. (b) Application of the Data Mining Algorithm: The application of the data mining algorithm to search for elementary signatures can be unsupervised or supervised. In the case of a monitored application of the data mining algorithm, the intermediate results must be evaluated. Signatures detected by the data mining algorithm can be automatically linked to FLC classifier functions. (c) Evaluation: Successful application of the data mining algorithm to the cleaned target data A 'results in a set K _TMP of preliminary, non-evaluated FLC. These must be evaluated. If necessary, the data mining step can be repeated. The result is the set K of the evaluated FLC.
• 6. The Quantity K: The product of the FLCD process is a set K of FLC. This set models knowledge with reference to the level of elementary phenomena in the time series data. This knowledge can be used to annotate the target data in A ', but it can also contain interesting information about the processes underlying the time series data.
• 7. Application: Applying the classifiers in K to A 'gives the set F ^{K of} the FLC annotations. This amount can either be used directly or evaluated in higher level KDD steps.

1.6.2 The SLCD process

SLC optimized for a particular domain of phenomena can also be understood as a special form of expert knowledge. Manufacturing SLC is a knowledge-intensive process, ignoring the simplest of cases. SLC can be generated, evaluated and refined as part of a modified KDD process. It results in the 3.3 illustrated SLC discovery process (SLCD process).

The objective of the SLCD process is to define a symbolic problem domain Φ ^S in a given manufacturing context and to find a suitable set P of SLC with which the phenomena in F ^{K associated} with Φ ^{S are} reliably located and annotated can.

• 1. FLC-Annotationen F^K: Die Menge der FLC-Annotationen F^K entsteht durch Anwendung der Klassifikatoren in K auf A'. F^K kann ggf. auch durch manuelle Annotationen von A' erhalten werden.
• 2. Daten-Interface: Um mit den Elementen in F^K operieren zu können, bedarf es eines speziell hierfür gestalteten Daten-Interfaces, welches Werkzeuge zur Visualisierung von Annotationsfakten und zur Navigation in Mengen von Annotationsfakten zur Verfügung stellt.
• 3. Daten-Selektion: Im SLCD-Prozess kann es notwendig sein, aus F^K eine Teilmenge auszuwählen, welche für die Untersuchung am geeignetsten erscheint. In die Entscheidung darüber, welche Daten in den nachfolgenden Schritten verwendet werden, fließen Betrachtungen über Inhalt, Qualität, Quantität und Struktur der Daten ein. Es entsteht die Menge der vorläufigen Zieldaten.
• 4. Daten-Bereinigung: Die Menge der vorläufigen Zieldaten muss ggf. bereinigt, d.h. von unbrauchbaren Elementen befreit werden. Idealerweise wird die Menge fehlerhafter Daten schon während der Annotation von A' minimiert. Die verbleibenden Daten werden bereinigte Zieldaten F'^K genannt.
• 5. SLC-Data-Mining: Das SLC-Data-Mining zielt darauf ab, neuartige und interessante symbolische Signaturen zu finden. Diese können durch geeignete Data-Mining-Verfahren generiert oder verfeinert werden. Das SLC-Data-Mining umfasst folgende Schritte: (a) Wahl des Data-Mining-Algorithmus: Es muss ein konkreter Algorithmus zum Auffinden von neuartigen symbolischen Signaturen komplexer Phänomene gewählt werden. Dieser Schritt beinhaltet detaillierte Betrachtungen über Parameter und mögliche Modelle sowie das Gesamtumfeld des SLCD-Prozesses. (b) Anwendung des Data-Mining-Algorithmus: Die Anwendung des Data-Mining-Algorithmus zur Suche nach symbolischen Signaturen komplexer Phänomene kann unsupervised oder supervised ablaufen. Im Falle einer überwachten Anwendung des Data-Mining-Algorithmus müssen die Zwischenergebnisse evaluiert werden. (c) Evaluation: Eine erfolgreiche Anwendung des Data-Mining-Algorithmus auf die bereinigten Zieldaten F'^K resultiert in einer Menge P_TMP von vorläufigen, nicht-evaluierten Klassifikatoren für komplexe Phänomenklassen. Im Bedarfsfalle kann der Data-Mining-Schritt wiederholt werden, bis die Menge P der evaluierten SLC entsteht.
• 6. Die Menge P: Das Produkt des SLCD-Prozesses ist die Menge P. Diese Menge modelliert Wissen mit Referenz auf Konstellationen elementarer Phänomene im Zeit-Frequenz-Raum. Dieses Wissen kann zur Annotation der Zieldaten in F'^K verwendet werden.
• 7. Anwendung: Durch Anwendung der Klassifikatoren in P auf F'^K' entsteht die Menge F^P der SLC-Annotationen. Diese Menge kann entweder direkt verwendet oder in weiterführenden KDD-Schritten auf höherer Ebene ausgewertet werden.

The following list shows the individual steps of the SLCD process. Each step is interactive and can be iterated if necessary.

• 1. FLC annotations F ^K : The set of FLC annotations F ^K is obtained by applying the classifiers in K to A '. If necessary, F ^K can also be obtained by manual annotations of A '.
• 2. Data Interface: In order to be able to operate with the elements in F ^K , a specially designed data interface is required, which provides tools for visualizing annotation facts and for navigating in sets of annotation facts.
• 3. Data selection: In the SLCD process, it may be necessary to select from F ^K a subset which appears most suitable for the investigation. When deciding which data to use in subsequent steps, consideration is given to the content, quality, quantity, and structure of the data. The amount of preliminary target data is created.
• 4. Data cleanup: The amount of preliminary target data may have to be cleaned up, ie freed from unusable elements. Ideally, the amount of erroneous data is already minimized during the annotation of A '. The remaining data is called adjusted target data F ^'K.
• 5. SLC Data Mining: SLC data mining aims to find novel and interesting symbolic signatures. These can be generated or refined by means of suitable data mining methods. SLC data mining involves the following steps: (a) Choice of the data mining algorithm: a concrete algorithm for finding novel symbolic signatures of complex phenomena must be chosen. This step includes detailed considerations about parameters and possible models as well as the overall environment of the SLCD process. (b) Application of the Data Mining Algorithm: The application of the data mining algorithm to search for symbolic signatures of complex phenomena can be unsupervised or supervised. In the case of a monitored application of the data mining algorithm, the intermediate results must be evaluated. (c) Evaluation: Successful application of the data mining algorithm to the adjusted target data F ' ^K results in a set P _TMP of preliminary, non-evaluated classifiers for complex classes of phenomena. If necessary, the data mining step may be repeated until the set P of the evaluated SLC is established.
• 6. The set P: The product of the SLCD process is the set P. This set models knowledge with reference to constellations of elementary phenomena in the time-frequency space. This knowledge can be used to annotate the target data in F ' ^K.
• 7. Application: Applying the classifiers in P to F '^K' results in the set F ^{P of} the SLC annotations. This amount can either be used directly or evaluated in higher level KDD steps.

1.7 Tasks in the production of signal classifiers

• • Verfahrenstechnische-Aufgaben: (1) Verfahren zur Generation von akustischen und visuellen Phänomenen; (2) Verfahren zur Extraktion von elementaren und symbolischen Signaturen; (3) Implementationen von KKF; (4) Implementationen von First-Level- und Second-Level-AA; (5) Data-Mining-Instrumente zur Suche nach neuartigen elementaren und symbolischen Signaturen; (6) Instrumente zur Visualisierung von Signaturen und Annotationen; (7) Tools zum Management aller KDD-Einzelschritte; (8) Datenbank-Tools zur Verwaltung der Mengen F^K und F^P.
• • Experten-Aufgaben, also kontextspezifische Aufgaben, die von Experten im Rahmen des FLCD- oder SLCD-Prozesses gelöst werden müssen. Hierzu gehören: (1) Die Suche und Spezifikation von akustischen und visuellen Phänomenen; (2) Die Generation von elementaren und symbolischen Signaturen; (3) Die Auswahl von KKF und Annotationsalgorithmen; (4) Die Justierung von Parametern; (5) Die Auswahl und Anwendung von Data-Mining-Instrumenten zur Suche nach neuartigen elementaren und symbolischen Signaturen; (6) Die Ausführung und Überwachung des FLCD- und SLCD-Prozesses; (7) Die Evaluation und Anwendung von Ergebnissen des FLCD- und SLCD-Prozesses.

The tasks to be solved in the production of SK can be divided into two groups:

• • Process engineering tasks: (1) methods for generation of acoustic and visual phenomena; (2) methods for extracting elemental and symbolic signatures; (3) implementations of KKF; (4) implementations of first-level and second-level AA; (5) data mining tools to search for novel elemental and symbolic signatures; (6) instruments for visualizing signatures and annotations; (7) tools for managing all KDD steps; (8) Database tools for managing quantities F ^K and F ^P.
• • Expert tasks, which are context-specific tasks that have to be solved by experts in the context of the FLCD or SLCD process. These include: (1) the search and specification of acoustic and visual phenomena; (2) The generation of elementary and symbolic signatures; (3) the selection of KKF and annotation algorithms; (4) the adjustment of parameters; (5) the selection and application of data mining tools to search for novel elemental and symbolic signatures; (6) the execution and monitoring of the FLCD and SLCD process; (7) The evaluation and application of results of the FLCD and SLCD process.

The The invention relates to a system which covers the field of process engineering Tasks covers, but at the same time is designed so that the expert tasks by experts of the respective field of application of the produced SK interactively solved can be.

1.8 Modeling of Spectrographic FLC

This section describes how to obtain FLC based on spectrographic visualizations. An FLC κ ε K can be understood as a 3-tuple (see Section 1.5): κ = <φ, c, γ> (1.17)

spectrographic Signatures can be programmed explicitly or inductively. Explicit programming here means that φ is based on explicitly selected examples is generated. Inductive programming means that φ is on Based on many examples, it is machine-supervised or unsupervised becomes.

1.8.1 Spectrographic visualization

Spectrograms of time series data based on the Fourier transform or other time-frequency representations (Wigner-Ville, Choi-Williams, Cone-Shaped, Gabor, etc.) to the most important tools in signal processing [4, 7, 6].

The following are visual signatures based on the DFT. It should be noted, however, that the other transformations mentioned can be similarly used for purposes of spectrographic visualization. Let Φ = Φ ^V. The signatures are based on a set V of visual representations (visualizations) of phenomena, ie

V sei ein Ausschnitt eines Spektrogramms SP von einer Sequenz x → der Länge l.The two-dimensional visualization V ∈ V of a phenomenon is an x × y matrix

Let V be a section of a spectrogram SP of a sequence x → of length l.

Die Spalten der Matrix werden erhalten, indem die gefensterte DFT auf x' Subsequenzen von x → angewendet wird. Die Subsequenzen können sich überlappen oder x → nur in Teilen abdecken.The spectrogram SP of a sequence x → is an x '× y' matrix, 0 ≦ y ≦ y ', 0 ≦ x ≦ x'

The columns of the matrix are obtained by applying the windowed DFT to x 'subsequences of x →. The subsequences may overlap or cover x → only in part.

The DFT calculates the Fourier coefficients for a complex signal x ^ of length W by:

Since the elements in SP are real-valued, the amount of each complex element must be calculated:

The Input signal x ^ for the DFT is obtained by first using the elements in the x subsequences of x → a window function are multiplied. The imaginary part of the elements in x ^ is set to 0. The real share corresponds the window function weighted elements of the subsequences.

The window function Blackman (with α = 15, discussion in [5]), which is always used here, is defined by:

The Visualizations used here are color-coded representations of spectrograms. The time in ms is plotted on the x-axis. On the y-axis the frequency is plotted in Hz. The visible in a visualization Range of a spectrogram may possibly be restricted by an upper and lower frequency band is determined. The maximum frequency of the upper frequency band corresponds to the Nyquist frequency = sampling frequency ÷ 2 of Time series file.

A particular problem in the field of signal processing is often the high dynamic range of the time series data. In order to visualize the widest possible range of values in a spectrographic visualization, the elements of the matrix can be logarithmized and / or represented in a special color-coded manner. The color coding can be obtained by assigning to each value υ ε V, a color value color (υ). The function color (υ) is also called color palette of the spectrographic visualization in the following. For the 24 bit bitmaps it is defined by: color (υ) = red (υ) + 2 8th · Green (υ) + 2 16 · Blue (υ)

There are a total of 2 ²⁴ different color values but only 2 ⁸ different shades of gray possible. The color separation red (υ) of the palette is defined by:

The function depends on two parameters: R _pos and R _width . The value R _pos is the value at which the color red reaches its maximum value 255. The value R _pos + is the value at which the color red decreases again in one step. The value R _pos - 255 is the first value from which the color red is displayed. The value R _pos + R _width + 255 is the last value up to which the color red is displayed. Similarly, the colors green and blue are calculated.

By suitable color palettes can the values in V using brightness and color contrasts in one huge Dynamic range are displayed. This has opposite the traditional grayscale appearance decisive advantages: An overall larger value range becomes visible and fine gradations in adjacent values occur through color contrasts clearly visible.

1.8.2 Spectrographic signatures

The Signature, which is a phenomenon characterized, should be compact while keeping all relevant information contain - otherwise The calculation of the KKF is too complex and unnecessary information could be theirs Affect accuracy.

Visual signatures are obtained via a function E ^V for extracting features from time series. In the following, a method for the realization of this function is described on the basis of spectrographic visualizations (see Section 1.8.1). The signatures are a subset of the original spectrographic visualization. The method comprises the following steps:

Method: extraction of spectrographic signatures

Input: A

Output:

• 1. Generate the visualization V of a desired phenomenon in A using the method in Section 1.8.1. It should be noted that all characteristics of the phenomenon in V must be made clear. V is an x × y matrix over
  
• 2. In V, select an area V 'corresponding to the position and extent of the phenomenon in the time-frequency space. V 'is an x' × y 'matrix over
  
  which contains a subset of V.
• 3. Extract from V 'the spectrographic signature φ of the phenomenon. Proceed as follows: (a) In V ', select all elements that are greater than a given threshold value ς. Divide this set into two matrices: (1) the matrix φ ⁺ , which includes all elements that obviously have a special relevance for classification decisions, and (2) the matrix φ ⁰ , which includes all remaining elements, ie all elements, of which it is not yet known if they are relevant for classification decisions. In φ ⁺ only local maxima of columns and rows are contained in V ': φ = {υ x, y Ε V '| υ x, y > ς, f M (υ x, y ) = 1} (1.21) The matrix φ ⁰ is needed when merging signatures. It contains data points that can possibly be transformed into local maxima of columns and rows by merging operations (see Section 1.8.10): φ 0 = {υ x, y Ε V '| υ x, y > ς, f M (υ x, y ) = 0} (1.22) The function f _M :
  
  → {1,0} indicates whether an element υ _{x, y is} a local maximum in a column or row of V '. This approach is based on the assumption that most real-world time series can be represented by their maximum Fourier coefficients [3, 1]. The function f _M is defined by
  
  (b) Then select in V 'all elements that are smaller than ς and at the same time in the immediate vicinity of elements from φ ⁺ or φ ⁰ . The neighborhood is defined by a respective distance variable dx in horizontal and dy vertical direction. These elements form the outline φ ^- the visualization of a phenomenon, as this is reflected in appropriate distance variables, the outline of the phenomenon. φ = {υ x, y Ε V '| υ x, y <ς, f U (υ x, y ) = 1} (1.24) Here, f _U :
  
  → {1, 0} a function that indicates whether a data point υ _{x, y is} in the neighborhood of a point of φ ⁺ or φ ⁰ . The function f _U is defined by
  
  (c) Delete from φ ⁺ , φ ⁰ , and φ ^- Manually all elements that obviously do not belong to the phenomenon that is to be modeled. The result is the spectrographic signature of the phenomenon: φ = <φ ⁺ , φ, φ> (1.26)
• 4. Evaluate the properties of the signature in the given manufacturing context. If necessary, repeat the previous steps.

This If necessary, the procedure can be iterated. A gradual refinement allows to obtain signatures that are in a manufacturing context are appropriate.

1.8.3 KKF for spectrographic signatures

With With help from the KKF, the annotation algorithm compares a signature with a sequence. By the KKF is the hypothesis that in the Sequence the signature corresponding phenomenon is included, a real-valued measure s - the Class similarity - assigns.

Since a spectrographic signature can be understood as a subset of a spectrogram, a measure of the distance between the signature and the spectrogram of the sequence is suitable for calculating the class similarity. The procedure described below for calculating the KKF is schwellwertba Siert. It requires that the temporal extent of sequence and signature correspond to each other. The algorithm is characterized by great stability and easy parameter fitting:

Method: KKF spectrographically

Input: x →,

Output: s

• 1. Compile x → a spectrogram using all the parameters used to generate the signature underlying the signature. The result is a spectrogram
  
  whose elements can be compared directly with those of φ.
• 2. Extract
  
  two matrices whose elements correspond to the defined elements in φ ⁺ and φ ^- :
  
  Now we have N = | φ ⁺ | = | X ⁺ | and M = | φ ^- | = | X ^- |.
• 3. Calculate the similarity between φ and x → by:
  

Here, d ⁺ and d ^{- are} two different functions for calculating the distance:

The parameters dy ₀ , c ₀ , dy ₁ , c ₁ , dy ₂ and c ₂ are normalization parameters. The distance functions Δ ⁺ and Δ ^- are defined by:

The idea with this procedure is to include the difference between an element in X ⁺ and an element in φ ⁺ only in the result if the first element is smaller, ie does not correspond to the specification. On the other hand, if it exceeds the default, the resulting negative distance is not included. The same applies to the difference between an element in X ^- and an element in φ ^- : Only if the specification is not fulfilled is the difference used to calculate the result. With this type of weighting, the high dynamic range is better taken into account in many signals than with simpler measures, such as the Manhattan distance.

1.8.4 Algorithmic Annotation

The third element of an FLC is the annotation algorithm, which is the KKF systematically applied to the files of the time series corpus A. and generates FLC annotations under certain conditions. Hereinafter a procedure is described which is especially good for the above KKF described is suitable. The AA uses the KKF for a time series file s → to calculate one smoothed Correlation vector D →. From this then intervals are isolated, whose Elements all over a certain threshold th lie. At each interval becomes then generate an FLC annotation at the time D → within the Interval has its maximum. The annotation is unsupervised according to the following procedure:

Method: FLC annotation

Input: s →, th, φ

Output: F ^K

• 1. Create from a time series file ss → with help the signature-specific spectrographic KKF a correlation vector DD →, the for any Date a statement about the class similarity between signature and time series. Then smoothen the Vector.
• 2. Find in the smoothed Correlation vector D → the intervals I whose elements are all larger as the predetermined similarity threshold th.
• 3. Generate exactly one FLC annotation for each interval. The time of the annotation corresponds to the time at which the smoothed correlation vector has its maximum within the interval. The set of all thus generated FLC annotations is F ^K.

In 3.6 the operation of the annotation algorithm is shown schematically: D → is the smoothed correlation vector. th is the similarity threshold. I ₀ , I ₁ , I ₂ , I ₃ are the intervals at which the elements of D → are greater than th. t ₀ , t ₁ , t ₂ , t ₃ are the times at which the algorithm generates an annotation.

idea with this method is to generate annotations only then if the distance between signature and time series file above a threshold and at the same time is a local maximum. The smoothing serves to avoid false annotation decisions by local fluctuations in DD →.

1.8.5 Manual programming of FLC

In many cases is it sufficient to create FLC and also SLC manually. A method Explicit programming of FLC consists of examples of Phenomenon classes using a corresponding time series data interface and select according to the method described above spectrographic signatures to generate. Subsequently can these are linked to the KKF and the annotation algorithm. All necessary parameters can be adjusted by hand. The classifiers obtained in this way can be evaluated step by step and be refined. It's also possible to group them like this that a class of physics covered by several classifiers.

1.8.6 Notation and visualization

• • Die Zeit-Frequenz Position der Datenpunkte in ϕ.
• • Die Aufspaltung des Klassifikators in ϕ⁺, ϕ⁰ und ϕ^–.

The performance of an FLC depends on its signature itself, but also on a number of algorithmic parameters. Immediate visual feedback on the following properties of a classifier is helpful in designing and adjusting the parameters:

• • The time-frequency position of the data points in φ.
• • The splitting of the classifier into φ ⁺ , φ ⁰ and φ ^- .

3.4 illustrates this. Left: φ ⁺ and φ ^- in colors of a palette (see section 1.8.1). Right: φ ⁺ (green), φ ⁰ (blue) and φ ^- (red) in contrasting colors.

In predicate logic, an FLC annotation can be noted as an atomic formula of the form ω (ψ, r, f, Λ). This form of presentation is compact, but it is difficult to intuitively connect to the annotated event in ψ. The behavior of the KKF and its constituents may also be of interest in the manufacturing context. A more comprehensible form of representation of FLC annotations and KKF shows 3.5 Annotation decisions are displayed by highlighting in contrast colors and crosshairs. Below: Graph of the correlation function s (blue) and its constituents d ⁺ (green) and d ^- (red). The horizontal blue line corresponds to th.

1.8.7 Thinning out features

Often are signatures despite the restriction described in Section 1.8.2 the maximum Fourier coefficients are too large, i. they contain too many data points. This circumstance influences in particular the Speed of the machine evaluation of time series data.

One way to solve this problem is by ei the amount of data in a signature Another filter to reduce. A simple but efficient way of thinning features is to leave only every nth element in the horizontal or every mth element in the vertical direction after a second filter stage: φ + 1 : = {υ x, y Ε φ + | (x mod n = 0), (y mod m = 0)} (1.34) φ + 1 : = {υ x, y Ε φ + | (x mod n = 0), (y mod m = 0)} (1.35)

This Action does not immediately lead to a reduction in accuracy lead an FLC. To assess the behavior of a signature manipulated in this way can, It is recommended to monitor the Graphs of the KKF.

1.8.8 Cloning of signatures

often It is necessary to group several FLCs through them a large Frequency range is covered. To facilitate this task, Is it possible, generate a bunch of clones from a single signature, covering a given frequency range. The clones differ different from the original one Signature only by the frequency range assigned to them is.

3.7 illustrates the process of cloning FLC: ( 1 ) Oscillogram of a wavelet; ( 2 ) Spectrogram of the wavelet; ( 3 ) Signature of the wavelet; ( 4 ) Clones of the signature

In the example becomes for the signature of a hair wavelet generates a bevy of clones. First, will an FLC signature is formed from the spectrogram of the wavelet. Subsequently, will out of this a flock of other FLC's generated a large frequency range covers.

1.8.9 Automatic extraction of spectrographic signatures

Spectrographic signatures can be automatically extracted from a set F ^K of annotation facts. The prerequisite for this is that the time series files over which the quantity F ^{K was} formed are available. The following is an example of a single annotation factor F κ i Ε F K a method for the automatic extraction of spectrographic signatures is shown:

Method: extraction of spectrographic signatures

Input: F κ / i

Output: φ

• 1. Given the annotation decision F κ i Ε F K of an elementary classifier κ = <φ, c, γ>. F κ i refers to an event in a time series file s → ε A. The extension of φ in the time-frequency space corresponds to a rectangle R = (t ₁ , f ₂ , t ₂ , f ₁ ). Here t ₁ and t _{2 are} times relative to a zero point to be specified. f ₁ and f ₂ are values on the frequency scale in Hertz. The temporal extent of φ is Δt = t ₂ -t ₁ . The extension of φ in the frequency domain is Δf = f ₂ -f ₁ . t ₁ and f ₂ form the upper left corner of the rectangle, t ₂ and f ₁ form the lower right corner of the rectangle ,
• 2. To the annotation decision F κ i should automatically be a signature φ κ i extracted, where: (a) φ κ i should be based on a spectrographic visualization that does not necessarily use the same parameters that underlie the signature φ of κ (see Section 1.8.1). (b) The extraction of the signature φ κ i should be carried out according to the same procedure as the extraction of φ. For extracting the signature φ κ i the same parameters should be used as were used to extract φ (see Section 1.8.2).
• 3. From R, define a search space R _s for φ such that R _s ( t s 1 , f s 1 , t s 2 , f s 1 ). This may, but does not have to t s 1 ≤ t 1 , f s 2 ≥ f 2 , t s 2 ≥ t 2 , f s 1 ≤ f 1 be valid. The temporal extent of R _s is .delta.t s = t s 2 - t s 1 , The extent of R _s in the frequency domain is .delta.f s = f s 2 - f s 1 (see. 3.8 ).
• 4. The annotation F κ i κ corresponds to a time t κ i in s →. Choose from this and the time coordinates of the search space t s 1 , t s 2 from s → a sequence
  
• 5. Generate by the procedure in 1.8.1
  
  a spectrogram SP. If necessary, set other parameters than when generating φ.
• 6. Using the procedure in 1.8.2 from SP, generate a spectrographic Signature φ κ / i. Use the same parameters as for Generation of φ. The selection window, that for the extraction the signature relevant area V 'is defined by the parameters f s / 1 and f s / 2 automatically limited.

1.8.10 Merging of spectrographic signatures

A further elementary technique for the modeling of FLC is the merging of multiple signatures to a single. The goal of the merger is, out of a number from signatures filter out those properties which in the majority of the signatures are present and at the same time those To find out characteristics that only in a few or a few Examples are represented.

For the fusion of a set Φ = {φ ₀ , ..., φ _N-1 } of spectrographic signatures into a single fused signature φ ^M , the method described below is suitable. The condition is that all signatures are based on spectrograms with identical parameters:

Method: Merging spectrographic signatures

Input: Φ

Output: φ ^M

• 1. Check, whether the signatures in Φ on Spectrograms based on identical parameters. If necessary, sort signatures out, which are not mergeable with the rest.
• 2. For each element φ _i ε Φ, 0 ≤ i <N, generate the union V i : = φ + i ∪ φ 0 i ∪ φ - i , 0 ≤ i <N, (1.37) such that a set V = {V ₀ ,..., V _N-1 } arises from x × y matrices. Now form an x × y-matrix M (Merged) with the mean values of the coefficients of the matrices in V:
  
• 3. Extract from M by the method in Section 1.8.2 a spectrographic signature of the form φ ^M = <φ ⁺ , φ ⁰ , φ ^- >.

3.10 illustrates merging V to M: From 67 signatures (not all shown for space) a single one is generated by merging. The method is suitable for extracting noise and other disturbances in a large number of signatures. It can also be used to find out which features are frequently represented in a set of signatures.

By Merging can be a lot of noisy signatures of an event type, a single signature will be generated that matches the Event type is better modeled than the individual elements of the set.

1.8.11 Similarity-based merging of spectrographic signatures

For the similarity-based merging of a set Φ = {φ ₀ , ..., φ _N-1 } of signatures to a set Φ ^M = {φ M / 0, ... φ M / N'-1} of merged signatures the procedure described below. The merging process is dependent on a similarity parameter σ. This determines the minimum distance required for a merge operation from two signatures.

Procedure: Similarity-driven-merge

Input: Φ, σ

Output: Φ ^M

• 1. Check, whether the signatures in Φ on Spectrograms based on identical parameters. If necessary, sort signatures out, which are not mergeable with the rest.
• 2. For each element φ _i ε Φ, 0 ≤ i <N, generate the union V i : = φ + i ∪ φ 0 i ∪ - i , 0 ≤ i <N, (1.39) such that a set V = {V ₀ ,..., V _N-1 } arises from x × y matrices. Then, after the procedure SDMerge V, merge to a set M = {M ₀ , ..., M _N'-1 } of merged x × y matrices. Procedure: SDMerge
  
  
  The distance function corresponds to the distance measure in section 1.8.3. This function can be varied depending on the particular manufacturing context. Simpler measures are the sum of the absolute distances of the coefficients
  
  the positive part of the KKF:
  
  where Δ ⁺ (V _ij , M _ij ) is described in equation 1.32. The Quantity function corresponds to the merging of elementary signatures in Section 1.8.10. Since, however, the number of mergers per signature is not known a priori, the number nom (number of merged) of the mergers carried out so far must be logged for each element in M. This makes it possible to find coefficients in To adequately weight V and M before averaging:
  
• 3. From M, extract a set Φ ^M of spectrographic signatures of the form according to the procedure in Section 1.8.2 Φ M = {φ M 0 , ... φ M N'-1 | φ M i = <Φ + i , φ 0 i , φ - i >}, 0 ≤ i <N '

3.11 illustrates the similarity-based merging of V to M. The examples are the automatically-extracted signatures 3.9 : The set V = {V ₀ , ..., V ₆ } is merged into the set M = {M ₀ , M ₁ }.

1.8.12 Inductive programming

In This section introduces a similarity-based learning procedure described a finite amount of FLC. The procedure is based on methods for automatic extraction (see Section 1.8.9) and similarity-based Fusion (see Section 1.8.11) of spectrographic signatures.

The Idea is, with the help of a supergeralizing Class identifier in A a set of spectrographic phenomena to automatically generate signatures for these phenomena, which then resembles similarity to merge and with a KKF and an annotation algorithm to link to FLC.

• 1. Die zu gruppierenden akustischen Phänomene müssen adle ein eindeutiges gemeinsames Merkmal besitzen, das mit Hilfe einer spektrographischen Signatur ϕ repräsentiert werden kann. Ist dies der Fall, so ist durch ϕ implizit die Klasse Ω^ϕ der ϕ-repräsentierbaren Phänomene in A definiert.
• 2. Für das Klassenmerkmal ϕ muss ein elementarer Klassifikator κ = 〈ϕ, c, γ〉 existieren, mit dessen Hilfe Instanzen der Elemente der Klasse Ω^ϕ in Ω^ϕ zuverlässig lokalisiert werden können. Ist dies der Fall, so ist durch κ implizit die Klasse Ω^κ der ϕ-identifizierbaren Phänomene in A definiert.

The method enables spectrographic phenomena in A (and new data) to be grouped into clusters by a set of FLCs according to a defined similarity criterion. For a successful implementation of this procedure, the following conditions must be met:

• 1. The acoustic phenomena to be grouped must adle have a unique common feature, which can be represented by means of a spectrographic signature φ. If this is the case, the class Ω ^{φ of} the φ-representative phenomena in A is implicitly defined by φ.
• 2. An elementary classifier κ = <φ, c, γ> must exist for the class feature φ, with the help of which instances of the elements of the class Ω ^φ in Ω ^φ can be reliably located. If this is the case, κ implicitly defines the class Ω ^{κ of} the φ-identifiable phenomena in A.

If these prerequisites are met, the following will teach a set K of FLC that will diversify the set Ω ^φ similarity bias to clusters, with each element in K corresponding to exactly one cluster. With the help of the FLC in K it is possible to machine acoustic phenomena corresponding to the respective clusters into A:

Procedure: FLC induction

Input: φ, A

Output: K

• 1. Check, whether the conditions described above for the acoustic to be grouped Phenomena are fulfilled.
• 2. Generate for the common class feature φ an elementary classifier κ = <φ, c, γ> with the help of which instances of the elements of the class Ω ^φ in A can be reliably located.
• 3. Using κ, generate a set F ^κ of elementary annotation facts about A.
• 4. Determine a search space R _s for κ. R _s must be dimensioned so that the phenomena to be grouped from Ω ^φ are completely covered by R _s (see Section 1.8.9).
• 5. Using the method of F ^κ and A described in Section 1.8.9, generate a set Φ ^E of spectrographic signatures of the phenomena in Ω ^φ .
• 6. Make the set from the set Φ ^E using the procedure described in Section 1.8.11 Φ M = {φ M 0 , ..., φ M N-1 } the similarity-based merged signatures.
• 7. Form by linking with the KKF c ε κ and the annotation algorithm γ ε κ from Φ ^M a set K _TMP of FLC of the form K TMP = {κ 0 , ..., κ N-1 | κ i = <Φ M i , c, γ>}, 0 ≤ i <N
• 8. Evaluate the FLC in K _TMP and, if necessary, iterate the previous steps.

All FLC of the set K _TMP may need to be evaluated and refined. The linking of signatures with KKF and annotation algorithms takes place automatically in step 7. If necessary, the parameters of the KKF and the annotation algorithm must be adjusted manually.

Example: 3.12 illustrates the individual steps of this procedure: ( 1 ) κ with search space R _s . ( 2 ) Annotations of κ in a sequence of Pfifflauten. ( 3 ) Merged signatures of K _TMP ; ( 4 ) Refined signatures of K. The second signature is shown for better distinction with a different color palette; ( 5 ) Annotation decisions by K.

In the example, a sequence of Pfflflauten appears. The sounds consist of a tone that rises slowly and then drops relatively quickly. Not in all cases does the second part of the sound exist - so the last Pfifflaut is abruptly aborted after the rise. The goal here is to generate two different FLCs for such sounds: the first one should correspond to the rising part of the pinnacle, the second one the declining one. Since Pfflflauten is based on a sinusoidal oscillation, they can be identified in A with a FLC κ for short-term sinusoidal oscillations (20 ms). However, annotations of such an FLC say nothing about whether the identified sound corresponds to the rising or falling volume portion. For κ, therefore, a search space R _{s is} set, which is dimensioned such that the pinnacle phenomena are approximately covered by it. From the set of annotations of κ is extracted in compliance with R _s is a set of signatures and fused similarity is based by the method described in Section 1.8.11. By linking with c and γ from κ results in a set K _TMP of _un -evaluated FLC. These correspond to ascending or descending sound portions. By gradual refinement, the quantity K of the evaluated FLC is created.

1.9 Modeling SLC

This section shows how SLC can be obtained on the basis of spectrographic FLC. An SLC P ^S can be understood as a 2-tuple: P S = <Φ S γ S > (1.43)

Here, φ ^{S is} a symbolic signature and γ ^{S is} a second-level annotation algorithm (see Section 1.5.2).

The The task of a symbolic signature is a complex one acoustic phenomenon with the help of a constellation of symbolic elements in time-frequency space to describe. This is possible if the complex phenomenon in a lot of elemental phenomena can be decomposed and for each of the constituent single phenomena an FLC exists, the Instances of the phenomenon can locate in time series data.

• 1. Manuelle Modellierung: Die Signatur ϕ^S wird durch manuelles Positionieren von symbolischen Elementen in einem virtuellen Ausschnitt des Zeit-Frequenz-Raumes generiert. Alle notwendigen Parameter werden manuell festgesetzt.
• 2. Modellierung durch Interpretation: Die Signatur ϕ^S wird erhalten, indem eine Signalsequenz x → zunächst mit Hilfe einer Menge K von FLC annotiert wird. Danach werden die entstandenen Annotationsfakten F^K automatisch zu einer symbolischen Signatur ϕ^S assembliert. Alle notwendigen Parameter werden maschinell festgesetzt.
• 3. Induktive Programmierung: Die Signatur ϕ^S wird supervised oder unsupervised durch ein maschinelles symbolisches Induktionsverfahren aus einer Menge von Annotationsfakten generiert. Alle notwendigen Parameter werden maschinell festgesetzt.

Symbolic signatures can be obtained by manual modeling, by automatically interpreting a sequence of time series data, or by induction over a set of annotation facts:

• 1. Manual Modeling: The signature φ ^S is generated by manually positioning symbolic elements in a virtual section of the time-frequency space. All necessary parameters are set manually.
• 2. Modeling by Interpretation: The signature φ ^S is obtained by first annotating a signal sequence x → by means of a set K of FLC. Thereafter, the resulting annotation facts F ^{K are} automatically assembled into a symbolic signature φ ^S. All necessary parameters are set by machine.
• 3. Inductive Programming: The signature φ ^S is generated supervised or unsupervised by a machine symbolic induction method from a set of annotation facts. All necessary parameters are set by machine.

In The following sections introduce several approaches and methods of interpretation and inductive programming represented by SLC. Manual programming techniques are not explicitly described here.

The statements assume that the symbolic elements underlying the signatures (see Section 1.5.2) are the position of elementary annotation facts in the time-freq reflect the quenz space. The way in which the annotation facts were obtained is of subordinate importance. It is quite possible that they were created purely manually. It is also conceivable that frequency parameters are completely ignored (eg in the modeling of rhythmic structures in signal sequences).

1.9.1 Modeling symbolic signatures through interpretation

The modeling of a symbolic signature can be done by interpreting a signal sequence x → using a set K of FLC. The goal is to find a symbolic signature φ ^S in x → for a complex phenomenon. φ ^S should only contain the information relevant for the characterization of the phenomenon.

It have to The following prerequisites can be met: The complex phenomenon can be decomposed into a number of elementary events. Each Elementary event can be assigned to exactly one point in time-frequency space become. For every elementary event exists an FLC with whose help the event can be localized in x →. The amount of this FLC is K.

In order to obtain from the sequence of time series data x → a symbolic signature φ ^S describing the complex phenomenon, the following method is suitable:

Method: Modeling by interpretation

Input: x →, K

Output: Φ ^S

• 1. Generate from x → the visualization V of the to be described phenomenon - e.g. With Using the method in Section 1.8.1. Please note that all the characteristics of the phenomenon are within the range of V must be located section. The quality of the visual Representation plays a subordinate role here, as the coefficients of V can not be used directly to form the signature.
• 2. Select an area V 'according to the position and extent of the phenomenon in V. Set a time r ₁ for the beginning of the complex phenomenon in x →.
• 3. Interpret the complex phenomenon in V 'by annotation using the elements in K. This results in a set F ^K of FLC annotations of the form F K = {F 0 , ..., F N-1 | F i = <Ψ, ω i , τ i , f i , A i >}, 0 ≤ i <N, N = | F K |. (1:44) 4. Transform all elements in F ^K into symbolic elements, so that a symbolic signature φ ' ^{S of} the form φ ' S = {φ S 0 , ..., φ s N-1 | φ s i = <Ω i (Τ i - τ 1 ), f i , Λ i >}, 0 ≤ 2 <N (1.45) ensteht. The signature φ ' ^S is not adjusted, ie it may still contain symbolic elements that are noisy or similar. in x →.
• 5. Delete from φ ' ^S possibly all elements that do not belong to the phenomenon that is to be modeled. The result is the adjusted symbolic signature of the complex phenomenon: φ S = {φ s 0 , φ s M-1 }, M ≤ N (1.46)
• 6. Evaluate the signature in the given manufacturing context. If necessary, repeat the previous steps.

3.13 ) illustrates this method: ( 1 ) The section V '(inverted) in V with a complex phenomenon (here a Pfifflaut); ( 2 ) Visualization of the set F ^K generated by a family of Haar wavelet based FLCs; ( 3 ) Uncleared signature φ ' ^S - different symbolic elements go back to noise in the area of the Pfifflautes; ( 4 ) Purified symbolic signature φ ^{S of} the peacock

1.9.2 Algorithmic annotation

The second element of an SLC is the annotation algorithm γ ^S , which searches for instances of φ ^S in a set F ^K of elementary annotation facts and, under certain conditions, SLC annotations generated.

• 1. Zeitverschiebungen von Elementen: Symbolische Elemente ϕ^s ∊ ϕ^S werden unter Tolerierung einer Abweichung auf der Zeitachse instanziiert. Die Toleranzgrenze ist durch einen Parameter t^tol festgesetzt.
• 2. Frequenzverschiebungen von Elementen: Symbolische Elemente ϕ^s ∊ ϕ^S werden unter Tolerierung einer Abweichung auf der Frequenzachse instanziiert. Die Toleranzgrenze ist durch einen Parameter f^tol festgesetzt.
• 3. Nicht instanziierbare Elemente: ϕ^S wird auch dann instanziiert, wenn eine Anzahl von konstituierenden symbolischen Elementen ϕ^s ∊ ϕ^S trotz der Toleranz im Zeit- und Frequenzbereich nicht instanziiert werden kann. Die Toleranzgrenze für nicht instanziierbare Elemente ist durch einen Parameter i^tol festgelegt.

The annotation method described below works hierarchically. For each request, a hierarchy is set that controls the search. This can be achieved under favorable circumstances, a significant reduction of the computational effort. The procedure generates unsupervised SLC annotations. The instantiation of symbolic signatures is fault-tolerant in three ways:

• 1. Time shifts of elements: Symbolic elements φ ^s ε φ ^S are instantiated while tolerating a deviation on the time axis. The tolerance ^{limit is} fixed by a parameter t ^tol .
• 2. Frequency shifts of elements: Symbolic elements φ ^s ε φ ^S are instantiated while tolerating a deviation on the frequency axis. The tolerance ^{limit is} fixed by a parameter f ^tol .
• 3. Non-Instancable Elements: φ ^S is also instantiated if a number of constituent symbolic elements φ ^s ε φ ^S can not be instantiated despite the tolerance in the time and frequency domain. The tolerance limit for non-instantiable elements is defined by a parameter i ^tol .

For reasons of clarity, it is assumed here that F ^{K was} generated using exactly one time series file with the label ψ. The procedure works as follows:

Method: SLC annotation

Input: φ ^S , F ^K , t ^tol , f ^tol , i ^tol

Here is φ S = {φ s 0 , ..., φ s N-1 | φ s n = <Ω n , τ n , f n , Λ n >}, 0 ≤ n <N (1.47) and F K = {F 0 , ..., F M-1 | F m = <Ψ, ω m , τ m , f m , Λ m >}, 0 ≤ m <M (1.48)

Output: F

Here is F P = {F P 0 , ..., F P I-1 | F P i = <Ψ, ω S i , τ i , f i , Λ i , Θ i >}, 0 ≤ i <I (1.49) where Θ _i ⊆ F ^{K is} a constellation of FLC annotations that corresponds to the constellation of the symbolic elements in φ ^S.

The time τ _i is the time of the temporally first element in Θ _i . For all Θ _i all F P i Ε F P applies Θ i = {F 0 , ..., F N-1 | F n = <Ψ, ω F n , τ F n , f F n , Λ F n >}, 0 ≤ n <N (1.50) where the condition (ω F n = ω n ) ∧ (τ F n Ε [(τ i + τ n - t tol ) ... (τ i + τ n + t tol )] ∧ (f F m Ε [(f n - f tol ) ... (f n + f tol )]) (1.51) is satisfied.

The set F ^P is thus the set of all SLC annotations which describes all permissible constellations of FLC annotations in the set F ^K. Permitted are precisely the constellations, which reflect the constellation of the symbolic elements of φ ^s in time-frequency space, observing the specified tolerance limits. The method comprises the following steps:

• 1. Define a hierarchy for the constituents of the signature φ ^s . Ξ = {1, ..., H}, let Hε N be the set of labels of the hierarchical levels. The highest hierarchical level is 1. It applies ∀ φ s Ε φ s : φs → h, h ε Ξ (1.52)
• 2. Group the elements in φ ^s according to their affiliation to a hierarchy level in H subsigna turen, so that φ ~ S = {φ S 1 , φ S H } (1.53)
• 3. Picture about F ^K from the partial signature φ S 1 in accordance with equations 1.49 and 1.50 with condition 1.51 the set of SLC annotations F P C (C from Current). Generate an SLC annotation even if at a certain time for a maximum of i ^tol elements off φ S 1 no equivalent is found in F ^K. Save for each item in F P C the number of uninstantiated constituents with the aid of a variable y.
• 4. Set h: = 2 and φ S C : = φ S 1
• 5. Combine the partial signature φ S C , With φ S H ie φ S C : = φ S C U φ S H
• 6. Check for each item in F P C whether there is an SLC annotation of the partial signature φ S C can be extended, which satisfies the equations 1.49 and 1.50 with the condition 1.51. Extend an SLC annotation even if for maximum (i ^tol - y) elements off φ S C , no correspondence is found in F ^K , where y is the number of non-instantiated constituents of the annotation. Save for each item in F P C the current total number of uninstantiated elements in y.
• 7. Delete all non-extended elements F P C
• 8. Increase h by 1. If h ≤ H go to step 5
• 9th F P : = F P C ;

The method described is characterized by general advantages and disadvantages of hierarchy-based search methods. On the one hand, the search in F ^{K can be} significantly accelerated, on the other hand, in the worst case, valid constellations of elements in F ^K can be overlooked. This is the case, for example, if all elements of a layer can not be instantiated at a certain time, even though an equivalent signature with only one layer would be instantiable at the same time.

1.9.3 Notation and visualization

In predicate logic notation, a symbolic signature φ ^S of length N can be used as a list of the form φ ^S = [[ω ₀ , τ ₀ , τ ₀ , [ λ 0 0 , λ 1 0 , ...]], ..., [ω _N-1 , T _N-1 , f _N-1 , [ λ 0 N-1 , λ 1 N-1 , ...]]] (1.54). Here, ω _n , τ _n , f _n and λ 0 n , λ 1 n , ... constants that define the spatial positions of the elementary symbolic constituents of the signature. A second-level annotation corresponds to an atomic formula of the form ω ^S (ψ, τ, f, Λ, Θ).

The tolerance ^limits t ^tol and f ^tol and the hierarchical structure of an SLC are of crucial importance for the performance of the described annotation method. In textual (eg in predicate logic) notation, a symbolic signature is difficult to understand intuitively. One of the tasks of a system for the production of SLC is therefore the possibility to graphically represent symbolic signatures and SLC annotations.

3.14 shows a way to visualize SLC: Hierarchical levels are color coded. The parameter t ^tol is here 30% of the temporal extent of the constituents of φ ^S and is indicated by vertical lines. f ^tol is zero here, but can be characterized by horizontal lines analogously to t ^tol .

In the context of the visualization of the results of manufacturing steps, it may be of interest which constituents of a symbolic signature can be instantiated in which situations and which not. This can also be represented by a color coding of the corresponding annotation decisions. 3.15 shows an example of SLC annotations with i ^tol = 25: Left a peal, for which all constituents of the symbolic signature of the SLC 3.14 could be instantiated. Right a similar Pfifflaut for which only a part of the constituents could be instantiated. All non-instancable constituents are shown in blue.

1.9.4 Automatic extraction of symbolic signatures

Symbolic signatures can be automatically extracted from a set F ^K of annotation facts. The time series files over which the set F ^{K was} formed need not be available. The following is a general method for automatically extracting sequential symbolic signatures that exploits neighborhood relationships of elements in F ^K. Sequential be here indicates that the constituents of the signatures are arranged chronologically. For the sake of clarity, it is assumed in the following that F ^{K was} generated using exactly one time series file.

The Procedure is based on the following assumptions and definitions:

• 1. Let τ: F ^K →
  
  a function indicating the timing of an element in F ^K. Furthermore let f: F ^K →
  
  a function indicating the frequency center of an element in F ^K.
• 2. A symbolic signature φ S X can be obtained by a trivial transformation of a constellation X ⊆ F ^K. For this it is only necessary to transform the annotation facts by omitting the file label and normalizing the time information into symbolic elements. Let ζ: P (F ^K ) → Φ ^{S be} the transformation function, so that every sought symbolic signature can be obtained with the help of ζ, ie
  
• 3. A criterion β: P (F ^K ) → {1,0} can be defined with the aid of which F ^K can be used to select the set X of all subsets permissible for the ζ transformation, such that X = {X 0 , ..., X N-1 | X n ⊆ ∧ β (X n ) = 1}, 0 ≤ n <N (1.56) Thus, the amount Φ S β the sought signatures given by Φ S β = {ζ (X 0 ), ..., ζ (X N-1 ) | X n Ε X}, 0 ≤ n <N (1.57)
• 4. If the elements in F ^K are arranged chronologically, ie τ (F _i ) ≤ r (F _{i + 1} ), 0 ≤ i <| F ^K |, then β can be defined as concatenation of neighboring elements:
  
  where ν: F ^K × F ^K → {1, 0}, makes a statement as to whether two FLC annotations F _a , F _b are adjacent in time-frequency space.
• 5. The neighborhood relationship ν can be defined as follows: Δt _max and Δf _max are threshold values for maximum permissible distances of two elements on the time or frequency axis. Furthermore, let Δt _min and Δf _{min be} threshold values for the minimum distance between two elements on the time or frequency axis. It applies
  
  Based on these definitions can Φ S β supervised or unsupervised by the following steps from F ^K.

Method: extraction of symbolic signatures

Input: F ^K , Δt _min , Δf _min , Δt _max , Δf _max

Output: Φ S β

• 1. Order F ^K according to the time of the elements.
• 2. Delete all elements of F ^K that have no neighbors according to Equation 1.59.
• 3. Specify a minimum length l _min and optionally a maximum length l _max for the requested signatures.
• 4. Generate the set X according to the equations 1.56 and 1:58.
• 5. Transform X according to Equation 1.57 in Φ S β ,
• 6. If necessary, clean up X within the given manufacturing context.

3.16 illustrates the automatic extraction of sequential symbolic signatures from F ^K. In real-world issues, the step of cleaning up X can be more non-trivial steps require: ( 1 ) Visualization of F ^K ; ( 2 ) Φ S β = {Φ S1 β , Φ S2 β , Φ S3 β , Φ S4 β } ; ( 3 ) Annotations F ^P of Φ S β (blue crosshairs).

1.9.5 Merging symbolic signatures

A more basic technique for The modeling of SLC is the merging of multiple signatures to a single. In analogy to the merging of elementary Signatures is also the goal here, from a number of signatures to filter out those features that exist in the majority of Signatures are present and at the same time those properties to find out which only in a few or a few examples are represented.

A lot Φ S = {φ S 0 , ..., φ S N-1 } of symbolic signatures can be summarized simply by the union of their constituents, so that a single signature φ ~ ^S through φ ~ S = ⋃ φ S 0 , ..., φ S N-1 (1.60) results. However, this procedure is useless in practice, because in the resulting amount φ ~ ^S usually more elements than necessary are included. This is because the identity of two symbolic elements φ s 1 = <ω 1 , τ 1 , f 1 Λ 1 > and φ s 2 = <ω 2 , τ 2 , f 2 Λ 2 > is given in the rarest cases, since τ ₁ , τ ₂ and f ₁ , f _{2 are} real-valued quantities (Λ will not be considered below for reasons of simplification).

• 1.
  
  sei eine Funktion, die den Zeitpunkt eines symbolischen Elementes angibt.
  
  sei eine Funktion, die den Frequenzmittelpunkt eines symbolischen Elementes angibt. ω: ϕ ~^S → Ω sei eine Funktion, die das Label eines symbolischen Elementes angibt. Es gelte fernerhin M = |ϕ ~^S|.
• 2. v : ϕ ~^S × ϕ ~^S → {1, 0} sei ein Kriterium, das angibt, ob zwei symbolische Elemente miteinander verschmelzbar sind oder nicht. Ausgehend von diesem Kriterium kann die Menge ϕSV der verschmelzbaren symbolischen Elemente in ϕ ~^S definiert werden durch ϕSV := {ϕsi ∊ ϕ ~S|∃ ϕsj : υ(ϕsi ,ϕsj ) = 1}, i <> j, 0 ≤ i < M, 0 ≤ j < M (1.61)
• 3. Δt_max und Δf_max seien Schwellwerte für maximal zulässige Entfernungen zweier symbolischer Elemente auf der Zeit- bzw. Frequenzachse. Das Verschmelzbarkeitskriterium υ kann dann definiert werden durch
  
• 4. Zwei miteinander verschmelzbare symbolische Elemente ϕsa , ϕsb in ΦSV werden zu einem Element ϕsμ verschmolzen, indem für eines der beiden Elemente die Position im Zeit-Frequenz-Raum neu berechnet und das andere gelöscht wird.

Thus, a technique is needed that identifies and merges appropriate elements in φ ~ ^S. By such a merger, the cardinality of φ ~ ^{S can be} significantly reduced. At the same time, it offers a way of distinguishing typical elements of a pattern from untypical ones. The process is based on the following specifications:

• 1.
  
  Let be a function that indicates the time of a symbolic element.
  
  Let u be a function that specifies the frequency center of a symbolic element. ω: φ ~ ^S → Ω is a function that specifies the label of a symbolic element. Furthermore, M = | φ ~ ^S |.
• 2. v: φ ~ ^S × φ ~ ^S → {1, 0} is a criterion that indicates whether two symbolic elements can be merged together or not. Starting from this criterion, the quantity φ S V of the fusible symbolic elements in φ ~ ^{S are} defined by φ S V : = {φ s i Ε φ ~ S | ∃ φ s j : υ (φ s i , φ s j ) = 1}, i <> j, 0 ≤ i <M, 0 ≤ j <M (1.61)
• 3. Δt _max and Δf _max are threshold values for maximum permissible distances of two symbolic elements on the time or frequency axis. The blending criterion υ can then be defined by
  
• 4. Two fused symbolic elements φ s a , φ s b in Φ S V become an element φ s μ merged by recalculating for one of the two elements the position in the time-frequency space and erasing the other one.

und der neue Frequenzmittelpunkt f_u ist

The new time τ _μ of φ s μ is

and the new frequency center f _u is

On Basis of these concepts a general method for merging a set of Specify symbolic signatures as follows: Procedure: Merge symbolic signatures

Input: Φ ^S , Δt _max , Δf _max

Output: φ s μ

• 1. Form the union φ ~ ^S of Φ ^S.
• 2. Make the crowd φ S V of the fusible symbolic elements according to Equation 1.61 and the Merge Criterion in Equation 1.62.
• 3. Make the crowd Φ S R : = Φ ~ S \ Φ S V of non-merging symbolic elements.
• 4. Merge all mergeable pairs of symbolic elements into φ S V to the crowd φ ' S μ , Save for each item in φ ' S μ the number of merger operations in a variable nom (number of merged).
• 5. Form the merged signature φ S μ : = φ S R ⋃ φ ' S μ In particular, the method of merging symbolic signatures described in this section is suitable for merging a set of signatures obtained from a set of instances of a single signature, since the forced merging by the merge operation in Equation 1.60 is meaningful only if all signatures in Φ ^{S are of} the same basic type. 3.17 shows an example: ( 1 ) An automatically extracted signature 3.16 ; ( 2 ) A signature merged from 19 instances of the first signature. Evident is a more uniform arrangement of the constituents.

1.9.6 Similarity-based Merging symbolic signatures

In This section will be an extension of the previous section described method for merging symbolic signatures shown. The procedure allows elements within a set taking into account signatures a similarity criterion automatically merge. This technique can be used around typical patterns or deviations from typical patterns in acoustic To discover data.

• 1.
  
  sei eine Funktion, die angibt, wieviele Elemente zweier symbolischer Signaturen ϕSa , ϕSb ∈ ΦS entsprechend dem Verschmelzbarkeitskriterium in Gleichung 1.62 miteinander verschmelzbar sind, wenn alle symbolischen Elemente in ϕSb um einen Wert Δτ auf der Zeitachse verschoben werden. Δτ kann positiv oder negativ sein.
• 2. Die für eine Verschmelzung optimale Zeitbasis Δτ^opt sei der kleinst mögliche Wert, um den die Elemente in ϕSb auf der Zeitachse verschoben werden müssen, damit σ^Δτ den maximal möglichen Wert erreicht. σ^opt: Φ^S × Φ^S →
  
  sei die maximal mögliche Anzahl miteinander verschmelzbarer symbolischer Elemente zweier symbolischer Signaturen.
• 3. Das Kriterium
  
  gibt die für eine Verschmelzung minimal erforderliche Anzahl symbolischer Elemente an. σ^min kann konstant sein oder dynamisch für jeden Vergleich neu berechnet werden, z.B. in Prozent der Länge der zweiten Signatur durch σmin := c·|ϕSb |/100. Zwei Signaturen sind genau dann verschmelzbar, wenn σ^opt ≥ σ^min

Since different symbolic signatures can have different temporal extents, in the case of similarity-based merging, the symbolic elements of the signatures must be abstracted from the absolute time. Instead of this, an optimal time base Δτ ^{opt must} be calculated dynamically for the similarity comparison of two signatures. The method is based on the definitions in Section 1.9.5 and some additional concepts:

• 1.
  
  Let it be a function that indicates how many elements of two symbolic signatures φ S a , φ S b ∈ Φ S can be merged with one another according to the merge-ability criterion in equation 1.62 if all the symbolic elements in φ S b be shifted by a value Δτ on the time axis. Δτ can be positive or negative.
• 2. The optimum time base Δτ ^opt for a fusion is the smallest possible value by which the elements in φ S b must be shifted on the time axis, so that σ ^{Δτ reaches} the maximum possible value. σ ^opt : Φ ^S × Φ ^S →
  
  Let be the maximum possible number of symbolic elements of two symbolic signatures that can be merged together.
• 3. The criterion
  
  specifies the minimum number of symbolic elements required for a merge. σ ^min can be constant or dynamically recalculated for each comparison, eg as a percentage of the length of the second signature by σ min : = c · | φ S b | / 100 , Two signatures can be merged if and only if σ ^opt ≥ σ ^min

3.18 illustrates the similarity ^comparison of two symbolic signatures: (1) The similarity σ ^Δτ of φ S a and φ S b is 2 at Δτ = 0. (2) The similarity of φ S a and φ S b is 3 (ie maximum) if the time shift is optimal, ie Δτ = Δτ ^opt . The dashed lines symbolize a fusion between two elements. The circles symbolize symbolic elements and their position in time-frequency space. The hatches symbolize the different labels of the symbolic elements.

The Symbolic Similarity Driven Merge (SSDM) method generates a set of ver from Φ ^S melted signatures Φ S μ , The function Merge corresponds to the procedure in section 1.9.5:

Procedure: Symbolic-similarity-driven-merge

The Nearest function finds the element with the greatest similarity Φ S i :

With minor Modifications may include the conceptual clustering method described here Patterns are used in the time-frequency space. The procedure is non-linear, as the signatures are at each merge operation change.

Following simple or similarity-based merging, symbolic signatures of non-common elements can be cleaned up. For this purpose, those symbolic elements are deleted from a merged signature that corresponds to a low nom value (see Section 1.9.5). A color coding of the nom value allows a direct overview of the results of merging operations. 3.19 illustrates the cleanup process: ( 1 ) A signature fused with 41 symbolic signatures; ( 2 ) The same signature with color-coded representation of the parameter nom. The color light green corresponds to high nom values. The color dark red corresponds to very low nom values; ( 3 ) By clearing the elements with nom <2 cleansed signature; ( 4 ) The cleansed signature in normal non-color-coded representation.

1.9.7 Inductive programming

This section describes a method for inducing a finite set of symbolic signatures. The signatures generated by this method summarize sets of elementary annotation facts in F ^K into groups. The method is based on the methods of automatic extraction (see Section 1.9.4) and similarity-based merging (see Section 1.9.6) of symbolic signatures.

The idea is to look for constellations of neighboring annotation facts in F ^K and transform them into a set of symbolic signatures using the automatic extraction method (see Section 1.9.4). Subsequently, the elements of this set can be merged similarity-based. By linking with a suitable annotation algorithm (see Section 1.9.2) a lot of P from SLC. With the help of the SLC in P it is possible to machine the acoustic phenomena corresponding to the signatures in new time series data:

Procedure: SLC induction

Input: F ^K

Output: P

• 1. Extract a set Φ ^S of symbolic signatures from F ^K using the procedure in Section 1.9.4.
• 2. Merge Φ ^S in a similar way based on the procedure in Section 1.9.6 Φ S μ fused symbolic signatures.
• 3. Clean up the signatures Φ S μ of rarely occurring symbolic elements.
• 4. Evaluate the signatures in Φ S μ and if necessary, iterate the previous steps.
• 5. Link the evaluated signatures in with appropriate second-level AA to a lot P from SLC.

1.10 system for the production of

Signal classifiers

The The system and procedure described in this section enables one systematic coordinated and supervised Use of the individual methods described in Sections 1.8 and 1.9. This will make it possible the complicated process of producing signal classifiers to systematize and decisively improve. After its production can the classifiers are integrated into software or chips.

1.10.1 Basic architecture

The system consists of three modules that are interlinked. Its basic architecture is in 3.20 shown.

• 1. Ein Modul (1) zur Herstellung von Visualisierungen und zur Visualisierung von Annotationen: Siehe auch 3.21;
• 2. Ein Modul (2) zur Herstellung und Anwendung von FLC: Siehe auch 3.22;
• 3. Ein Modul (3) zur Herstellung und Anwendung von SLC: Siehe auch 3.23;

The three modules are used for the systematic production of visualizations, FLC, SLC, FLC annotations and SLC annotations. The visualization module also allows visual evaluation of the behavior of sets of FLC and SLC by the annotation visualization techniques described. It is a matter of:

• 1. A module ( 1 ) for visualization and annotation visualization: See also 3.21 ;
• 2. A module ( 2 ) for the production and use of FLC: See also 3.22 ;
• 3. A module ( 3 ) on the manufacture and use of SLC: See also 3.23 ;

• 1. Visualisierungen, die im Editor des Visualisierungs-Moduls sichtbar sind, können zur Initialisierung von FLC-Signaturen (z.B. spektrographischen Signaturen) benutzt werden. Die Visualisierung (z.B. ein Spektrogramm – vgl. Abschnitt 1.8.2) dient als initiale Schablone für die FLC-Signatur (siehe auch Verfahren in Abschnitt 1.8.2).
• 2. Klassifikationsentscheidungen von FLC (d.h. FLC-Annotationen) können in Visualisierungen von Zeitreihen-Daten nach dem Verfahren in Abschnitt 1.8.6 eingeblendet (projiziert) werden. Zusätzlich kann unterhalb der Visualisierung der Graph der KKF und ihrer Konstituenten eingeblendet werden (vgl. 3.5).
• 3. Visualisierungen, die im Visualisierungs-Editor sichtbar sind, können zur Initialisierung von SLC-Objekten benutzt werden. Die Visualisierung (z.B. ein Spektrogramm – vgl. Abschnitt 1.8.2) dient als initiale Schablone für die SLC-Signatur.
• 4. Klassifikationsentscheidungen von SLC (d.h. SLC-Annotationen) können in Visualisierungen von Zeitreihen-Daten nach dem Verfahren in Abschnitt 1.9.3 eingeblendet (projiziert) werden.

The modules enable the systematic and coordinated application of the individual methods described above. All modules and methods are computer-implemented. The interlocking of the modules serves the optimization, quality assurance and acceleration of the manufacturing process:

• 1. Visualizations that are visible in the editor of the visualization module can be used to initialize FLC signatures (eg, spectrographic signatures). The visualization (eg a spectrogram - see Section 1.8.2) serves as an initial template for the FLC signature (see also the procedure in Section 1.8.2).
• 2. Classification decisions of FLC (ie FLC annotations) may be displayed (visualized) in visualizations of time series data according to the procedure in Section 1.8.6. In addition, below the visualization, the graph of the KKF and its constituents can be displayed (cf. 3.5 ).
• 3. Visualizations visible in the visualization editor can be used to initialize SLC objects. The visualization (eg a spectrogram - see Section 1.8.2) serves as an initial template for the SLC signature.
• 4. Classification decisions of SLC (ie SLC annotations) may be displayed in visualizations of time series data according to the procedure in Section 1.9.3.

1.10.2 System for producing visualizations and for the visualization of annotations

• 1. Aus einer Sequenz von Zeitreihen-Daten (1) wird eine Visualisierung hergestellt (2). Hierbei kann das Verfahren in Abschnitt 1.8.1 verwendet werden. Alternativ können auch andere Zeit-Frequenz-Darstellungen oder Zeit-Energie-Darstelllungen verwendet werden.
• 2. Die Visualisierung wird in einem interaktiven grafischen Editor (3) dargestellt. Der Editor ermöglicht den Zugriff auf die Visualisierung (z.B. mit einem Mauszeiger). Hierdurch können Bereiche innerhalb der Visualisierung ausgewählt und markiert werden.
• 3. Mit Hilfe eines Property-Editors (4) können die Parameter der Visualisierung verändert werden. Hierzu gehören insbesondere Parameter der zugrundeliegenden Transformation (z.B. DFT-Parameter), Parameter der Farbpalette, Zeit- und Frequenz-Parameter, welche den sichtbaren Ausschnitt definieren, sowie Energie-Darstellungs-Parameter (z.B. ob spektrale Energiekoeffizienten linear oder logarithmisch dargestellt werden sollen.).
• 4. Die Visualisierung wird in einer Menge von Visualisierungs-Repräsentationen (10) gespeichert.
• 5. Das Visualisierungs-Modul beinhaltet eine integrierte Datenbank (11), welche den Zugriff auf und die Verwaltung von Mengen von Visualisierungs-Repräsentationen in Form von Datensätzen ermöglicht.
• 6. Das Visualisierungs-Modul beinhaltet einen integrierten File-Manager (12), welcher den Zugriff auf und die Verwaltung von Mengen von Visualisierungs-Repräsentationen in Form von Dateien ermöglicht.
• 7. Das Visualisierungs-Modul ist so gestaltet, dass alle Einzelschritte des Herstellungsprozesses separat ausgeführt und ggf. interiert werden können.
• 8. Das Visualisierungs-Modul ist so gestaltet, dass in Visualisierungen FLC-Annotationen (5) und/oder SLC-Annotationen (6) nach den Verfahren in 1.8.6 bzw. 1.9.3 eingeblendet (8) werden können.
• 9. Das Visualisierungs-Modul ist so gestaltet, dass Graphen (9) von KKF (7) und ihren Konstituenten innerhalb der Benutzerschnittstelle des Moduls dargestellt werden können.

In this section, the production of visualizations using the visualization module will be described wrote. System and manufacturing process are in 3.21 shown schematically.

• 1. From a sequence of time series data ( 1 ) a visualization is produced ( 2 ). The procedure in section 1.8.1 can be used here. Alternatively, other time-frequency representations or time-energy representations may be used.
• 2. The visualization is displayed in an interactive graphical editor ( 3 ). The editor allows access to the visualization (eg with a mouse pointer). This allows areas within the visualization to be selected and marked.
• 3. Using a property editor ( 4 ) the parameters of the visualization can be changed. These include in particular parameters of the underlying transformation (eg DFT parameters), parameters of the color palette, time and frequency parameters which define the visible section, and energy representation parameters (eg whether spectral energy coefficients are to be represented linearly or logarithmically). ,
• 4. The visualization comes in a lot of visualization representations ( 10 ) saved.
• 5. The visualization module contains an integrated database ( 11 ), which provides access to and management of sets of visualization representations in the form of records.
• 6. The visualization module includes an integrated file manager ( 12 ), which allows access to and management of sets of visualization representations in the form of files.
• 7. The visualization module is designed so that all individual steps of the manufacturing process can be carried out separately and possibly integrated.
• 8. The visualization module is designed so that in visualizations FLC annotations ( 5 ) and / or SLC annotations ( 6 ) according to the procedures in 1.8.6 or 1.9.3 ( 8th ) can be.
• 9. The visualization engine is designed to allow graphs ( 9 ) of KKF ( 7 ) and their constituents within the user interface of the module.

1.10.3 System for the production of FLC and FLC annotations

• 1. Aus einer Visualisierung (1) wird nach dem Verfahren in Abschnitt 1.8.2 über einen Initialisierungs-Schritt (2) ein FLC generiert.
• 2. Der FLC wird in einem interaktiven grafischen FLC-Editor (3) (siehe auch Abschnitt 1.8.2) dargestellt. Der FLC-Editor (3) ermöglicht den Zugriff auf die Signatur und die Modifikation derselben mit einem Mauszeiger und Bildbearbeitungswerkzeugen. Hierdurch können Bereiche innerhalb der Signatur ausgewählt und markiert werden. Mit Hilfe der Bildbearbeitungswerkzeuge ist es möglich, Koeffizienten der zugrundeliegenden Matrix zu löschen oder andersartig zu verändern. Das Verhalten des sichtbaren FLC kann im Visualisierungs-Editor anhand seiner KKF und Annotationsentscheidungen in Echtzeit verfolgt werden.
• 3. Mit Hilfe eines Property-Editors (4) können die wichtigsten Parameter des FLC verändert werden. Hierzu gehören alle Parameter, welche auch im Visualisierungs–Editor zu Verfügung stehen. Zusätzlich können Parameter, die das Verhalten der KKF beeinflussen, Suchraum–Parameter (vgl. Abschnitt 1.8.9) und Parameter, die das Verhalten des AA beeinflussen, verändert werden. Der zum FLC gehörende AA (5) und die KKF (6) können mit Hilfe des Property-Editors aus einer Anzahl von Varianten ausgewählt werden. Zumindest die KKF aus Abschnitt 1.8.3 und der AA aus Abschnitt 1.8.4 werden hier zur Auswahl gestellt.
• 4. Der FLC wird in einer Menge von FLC-Repräsentationen (7) gespeichert.
• 5. Von dem FLC-Modul aus können verschiedene Verfahren zur Generierung von Mengen von FLC und/oder FLC-Signaturen aus angesteuert werden. Hierbei handelt es sich zumindest um die Verfahren 'Ausdünnen von Merkmalen' (11) (vgl. Abschnitt 1.8.7), 'Klonen von Signaturen' (12) (vgl. Abschnitt 1.8.8), 'Verschmelzen von spektrographischen Signaturen' (8) (vgl. Abschnitt 1.8.10), 'Ähnlichkeitsbasiertes Verschmelzen von spektrographischen Signaturen' (9) (vgl. Abschnitt 1.8.11) und 'Induktive Programmierung' (10) (vgl. Abschnitt 1.8.12). Die Resultate dieser Verfahren werden ggf. automatisch mit geeigneten KKF und AA verknüpft und in einer Menge von Reräsentationen von FLC-Repräsentationen gespeichert. Es besteht auch die Möglichkeit, frei definierbare externe Testroutinen (13) auf Mengen von FLC ablaufen zu lassen.
• 6. Von dem FLC-Modul aus kann eine Menge von FLC zur Annotation einer Menge von Zeitreihen-Daten ausgewählt und der Annotationsprozess (14) mit Hilfe des Verfahrens in Abschnitt 1.8.4 gestartet werden. Die entstehenden Annotationsfakten (15) werden in einer Menge von Repräsentationen von FLC-Annotationen gespeichert.
• 7. Von dem FLC-Modul aus kann mit Hilfe des Verfahrens 'Automatische Extraktion' (16) in Abschnitt 1.8.9 eine Menge von FLC-Annotationen (15) ausgewertet werden, um aus einer Menge von geeigneten Zeitreihen-Daten (17) eine Menge von FLC zu extrahieren.
• 8. Das FLC-Modul ist so gestaltet, dass aus einer Menge von gespeicherten FLC-Annotationen eine Untermenge ausgewählt und in den Hauptspeicher geladen werden kann. Der Visualisierungs-Editor ist so gestaltet, dass die Elemente dieser Untermenge nach dem Verfahren in Abschnitt 1.8.6 in die Visualisierung eingeblendet werden können, falls sie auf die in der Visualisierung sichtbare Sequenz referieren.
• 9. Das FLC-Modul beinhaltet eine integrierte Datenbank (18), welche den Zugriff auf und die Verwaltung von Mengen von FLC in Form von Datensätzen ermöglicht.
• 10. Das FLC-Modul beinhaltet einen integrierten File-Manager (19), welcher den Zugriff auf und die Verwaltung von Mengen von FLC in Form von Dateien ermöglicht.
• 11. Das FLC-Modul beinhaltet eine integrierte Datenbank (20), welche den Zugriff auf und die Verwaltung von Mengen von FLC-Annotationen in Form von Datensätzen ermöglicht.
• 12. Das FLC-Modul beinhaltet einen integrierten File-Manager (21), welcher den Zugriff auf und die Verwaltung von Mengen von FLC-Annotationen in Form von Dateien ermöglicht.
• 13. Das FLC-Modul ist so gestaltet, dass alle Einzelschritte des Herstellungsprozesses separat ausgeführt und ggf. iteriert werden können.
• 14. Das FLC-Modul ist so gestaltet, dass einzelne FLC aus einer Menge von FLC ausgewählt (7) und in den FLC-Editor (3) geladen werden können.

This section describes how to make FLC and FLC annotations using the FLC module. System and manufacturing process are in 3.22 shown schematically.

• 1. From a visualization ( 1 ) is executed following the procedure in Section 1.8.2 via an initialization step ( 2 ) generates an FLC.
• 2. The FLC is displayed in an interactive graphical FLC editor ( 3 ) (see also section 1.8.2). The FLC Editor ( 3 ) provides access to the signature and modification thereof with a mouse pointer and image editing tools. As a result, areas within the signature can be selected and marked. With the help of image processing tools, it is possible to delete or otherwise modify coefficients of the underlying matrix. The behavior of the visible FLC can be tracked in real time in the visualization editor based on its KKF and annotation decisions.
• 3. Using a property editor ( 4 ) the most important parameters of the FLC can be changed. This includes all parameters that are also available in the visualization editor. In addition, parameters that influence the behavior of the CCF, search space parameters (see Section 1.8.9) and parameters that influence the behavior of the AA can be changed. The AA belonging to the FLC ( 5 ) and the KKF ( 6 ) can be selected from a number of variants using the Property Editor. At least the KKFs from section 1.8.3 and the AA from section 1.8.4 are offered here for selection.
• 4. The FLC is represented in a set of FLC representations ( 7 ) saved.
• 5. From the FLC module, various methods of generating sets of FLC and / or FLC signatures can be addressed. These are at least the methods 'thinning out features' ( 11 ) (see Section 1.8.7), 'Cloning signatures' ( 12 ) (see Section 1.8.8), 'Merging Spectrographic Signatures' ( 8th ) (see Section 1.8.10), 'Similarity-based fusion of spectrographic signatures' ( 9 ) (see Section 1.8.11) and 'Inductive programming' (see 10 ) (see section 1.8.12). The results of these methods may be automatically linked to appropriate KKF and AA and stored in a set of representations of FLC representations. There is also the possibility of freely definable external test routines ( 13 ) to run on quantities of FLC.
• 6. From the FLC module, a set of FLCs to annotate a set of time series data may be selected and the annotation process ( 14 ) using the procedure in section 1.8.4. The resulting annotation facts ( 15 ) are stored in a set of representations of FLC annotations.
• 7. From the FLC module, you can use the Automatic Extraction ( 16 ) in Section 1.8.9 a lot of FLC annotations ( 15 ) are evaluated from a set of appropriate time series data ( 17 ) to extract a lot of FLC.
• 8. The FLC module is designed so that from a set of stored FLC annotations a sub can be selected and loaded into main memory. The visualization editor is designed so that the elements of this subset can be displayed in the visualization according to the procedure in Section 1.8.6 if they refer to the sequence visible in the visualization.
• 9. The FLC module includes an integrated database ( 18 ), which allows access to and management of sets of FLC in the form of records.
• 10. The FLC module includes an integrated file manager ( 19 ), which allows access to and management of amounts of FLC in the form of files.
• 11. The FLC module includes an integrated database ( 20 ), which allows access to and management of sets of FLC annotations in the form of records.
• 12. The FLC module includes an integrated file manager ( 21 ) which allows access to and management of sets of FLC annotations in the form of files.
• 13. The FLC module is designed so that all individual steps of the manufacturing process can be executed separately and iterated if necessary.
• 14. The FLC module is designed to select individual FLCs from a set of FLCs ( 7 ) and the FLC Editor ( 3 ) can be loaded.

1.10.4 System for manufacturing SLC and SLC annotations

• 1. Aus einer Visualisierung (1) wird über einen Initialisierungs-Schritt ein SLC-generiert. Dessen symbolische Signatur (2) ist zunächst noch leer. Zeit- und Frequenzparameter der Visualisierung werden hierbei im Hintergrund gespeichert.
• 2. Der SLC wird in einem interaktiven grafischen SLC-Editor (3) (siehe auch Abschnitt 1.9.3) dargestellt. Der SLC-Editor (3) ermöglicht den Zugriff auf die Signatur und ihre Manipulation mit Hilfe von einem Mauszeiger und grafischen Bearbeitungswerkzeugen. Hierdurch können Bereiche innerhalb der Signatur ausgewählt und markiert werden. Mit Hilfe der grafischen Werkzeuge ist es möglich, symbolische Elemente der sichtbaren Signatur zu markieren, zu löschen, zu verschieben, Hierarchie-Eigenschaften zu definieren oder andersartige Eigenschaften zu ändern. Die Änderungen werden soweit möglich in dem Property-Editor des Moduls angezeigt.
• 3. Mit Hilfe eines Property-Editors (4) können die wichtigsten Parameter des SLC verändert werden. Hierzu gehören alle Parameter, welche auch im Visualisierungs-Editor (s.o.) zu Verfügung stehen und solche Parameter, die das Verhalten des AA beeinflussen. Der zum SLC gehörende AA (5) kann mit Hilfe des Property-Editors aus einer Anzahl von Varianten ausgewählt werden. Zumindest der AA aus Abschnitt 1.9.2 wird hier zur Auswahl gestellt.
• 4. Der SLC wird in einer Menge von SLC-Repräsentationen (6) gespeichert.
• 5. Von dem SLC-Modul aus können verschiedene Verfahren zur Generierung von Mengen von SLC oder SLC-Signaturen aus gestartet werden. Hierbei handelt es sich zumindest um die Verfahren 'Modellierung symbolischer Signaturen durch Interpretation' (13) (vgl. Abschnitt 1.9.1), 'Verschmelzen symbolischer Signaturen' (10) (vgl. Abschnitt 1.9.5), 'Ähnlichkeitsbasiertes Verschmelzen symbolischer Signaturen' (11) (vgl. Abschnitt 1.9.6) und 'Induktive Programmierung' (12) (vgl. Abschnitt 1.9.7). Die Resultate dieser Verfahren werden ggf. automatisch mit einem geeigneten AA verknüpft und in einer Menge von SLC-Repräsentationen gespeichert. Es besteht auch die Möglichkeit frei definierbare externe Testroutinen (14) auf Mengen von SLC ablaufen zu lassen.
• 6. Von dem SLC-Modul aus kann eine Menge von SLC zur Annotation einer Menge von Zeitreihen-Daten (15) ausgewählt und der Annotationsprozess (7) mit Hilfe des Verfahrens aus Abschnitt 1.9.2 gestartet werden. Die entstehenden SLC-Annotationen (8) werden in einer Menge von Reräsentationen von SLC-Annotationen gespeichert.
• 7. Von dem SLC-Modul aus kann mit Hilfe des Verfahrens 'Automatische Extraktion von symbolischen Signaturen' (9) in Abschnitt 1.9.4 eine Menge von FLC-Annotationen ausgewertet werden, um aus einer Menge von geeigneten Zeitreihen-Daten (15) eine Menge von SLC zu extrahieren.
• 8. Das SLC-Modul ist so gestaltet, dass aus einer Menge von SLC-Annotationen (8) eine Untermenge ausgewählt und in den Hauptspeicher geladen werden kann. Der Visualisierungs-Editor ist so gestaltet, dass die Elemente dieser Untermenge nach dem Verfahren in Abschnitt 1.9.3 in die im Visualisierungs-Editor sichtbare Visualisierung eingeblendet werden können.
• 9. Das SLC-Modul beinhaltet eine integrierte Datenbank (16), welche den Zugriff auf und die Verwaltung von Mengen von SLC in Form von Datensätzen ermöglicht.
• 10. Das SLC-Modul beinhaltet einen integrierten File-Manager (17), welcher den Zugriff auf und die Verwaltung von Mengen von SLC in Form von Dateien ermöglicht.
• 11. Das SLC-Modul beinhaltet eine integrierte Datenbank (18), welche den Zugriff auf und die Verwaltung von Mengen von SLC-Annotationen in Form von Datensätzen ermöglicht.
• 12. Das SLC-Modul beinhaltet einen integrierten File-Manager (19), welcher den Zugriff auf und die Verwaltung von Mengen von SLC-Annotationen in Form von Dateien ermöglicht.
• 13. Das SLC-Modul ist so gestaltet, dass alle Einzelschritte des Herstellungsprozesses separat ausgeführt und ggf. iteriert werden können.
• 14. Das SLC-Modul ist so gestaltet, dass einzelne SLC aus einer Menge von SLC ausgewählt (6) und in den SLC-Editor (3) geladen werden können.

This section describes how to fabricate SLC using the SLC module. System and manufacturing process are in 3.23 shown schematically.

• 1. From a visualization ( 1 ) an SLC is generated via an initialization step. Its symbolic signature ( 2 ) is initially empty. Time and frequency parameters of the visualization are stored in the background.
• 2. The SLC is displayed in an interactive graphical SLC editor ( 3 ) (see also section 1.9.3). The SLC editor ( 3 ) provides access to the signature and its manipulation with the help of a mouse pointer and graphical editing tools. As a result, areas within the signature can be selected and marked. With the help of the graphical tools, it is possible to mark, delete or move symbolic elements of the visible signature, to define hierarchy properties or to change other properties. The changes will be displayed as far as possible in the module's Property Editor.
• 3. Using a property editor ( 4 ) the most important parameters of the SLC can be changed. This includes all parameters that are also available in the visualization editor (see above) and parameters that influence the behavior of the AA. The AA belonging to SLC ( 5 ) can be selected from a number of variants using the Property Editor. At least the AA from section 1.9.2 is here to choose from.
• 4. The SLC is represented in a set of SLC representations ( 6 ) saved.
• 5. Different methods of generating sets of SLC or SLC signatures can be started from the SLC module. These are at least the methods 'modeling of symbolic signatures by interpretation' ( 13 ) (see Section 1.9.1), 'Merging symbolic signatures' ( 10 ) (see Section 1.9.5), 'Similarity-based fusion of symbolic signatures' ( 11 ) (see section 1.9.6) and 'inductive programming' (see 12 ) (see section 1.9.7). The results of these procedures may be automatically linked to a suitable AA and stored in a set of SLC representations. There is also the possibility of freely definable external test routines ( 14 ) to run on amounts of SLC.
• 6. From the SLC module, a set of SLCs for annotating a set of time series data ( 15 ) and the annotation process ( 7 ) are started using the procedure in section 1.9.2. The resulting SLC annotations ( 8th ) are stored in a set of re-presentations of SLC annotations.
• 7. From the SLC module, the method 'Automatic extraction of symbolic signatures' ( 9 ) in Section 1.9.4, a set of FLC annotations are evaluated to obtain from a set of appropriate time series data ( 15 ) extract a lot of SLC.
• 8. The SLC module is designed to handle a set of SLC annotations ( 8th ) a subset can be selected and loaded into main memory. The visualization editor is designed so that the elements of this subset can be displayed in the visualization visible in the visualization editor using the procedure in Section 1.9.3.
• 9. The SLC module includes an integrated database ( 16 ), which allows access to and management of sets of SLC in the form of records.
• 10. The SLC module includes an integrated file manager ( 17 ), which allows access to and management of sets of SLC in the form of files.
• 11. The SLC module includes an integrated database ( 18 ), which gives access to and administration of sets of SLC annotations in the form of records.
• 12. The SLC module includes an integrated file manager ( 19 ), which allows access to and management of sets of SLC annotations in the form of files.
• 13. The SLC module is designed so that all individual steps of the manufacturing process can be executed separately and iterated if necessary.
• 14. The SLC module is designed to select individual SLCs from a set of SLCs ( 6 ) and in the SLC editor ( 3 ) can be loaded.

1.10.5 User Interfaces

In the 3.24 - 3.31 Examples of the user interfaces of the three modules are shown at different stages of the manufacturing process. On the left side of the picture either the file manager or the property editor of the respective module can be seen. In the center you can see the interactive graphical editor for the respective objects. A task bar in the upper area controls the various modeling tools.

3.24 shows the interface of the visualization module. The editor shows the spectrographic visualization of a stanza of the Alpenbraunelle. One syllable of the stanza is selected and displayed inverted.

3.25 shows the interface of the visualization module. The editor shows the oscillographic visualization of a stanza of the alpine brown cell. One syllable of the stanza is selected and displayed inverted.

3.26 shows the interface of the FLC module. The editor shows a spectrographic signature after the initialization step. As a template here was the visualization in 3.25 ,

3.27 shows the interface of the FLC module. The editor shows a finished spectrographic signature.

3.28 shows the interface of the visualization module. In the visualization editor two FLC annotations are shown in the spectrographic visualization of a stanza of the Alpenbraunelle. The FLC annotations and the KKF graph below the editor were used by the FLC in

3.27 generated via the audio sequence visible in the visualization editor.

3.29 shows the interface of the SLC module. The editor shows a symbolic signature after the initialization step. As a template here was the visualization in 3.25 ,

3.30 shows the interface of the SLC module. The editor shows a finished symbolic signature. It was generated by the method 'Modeling symbolic signatures through interpretation' (see section 1.9.1). She models the stanza of Alpenbraunelle in 3.25 ,

3.31 shows the interface of the visualization module. In the visualization editor, two SLC annotations are shown in the spectrographic visualization of several successive stanzas of the Alpine brown cell. The SLC annotations were performed by the SLC in 3.30 generated via the audio sequence visible in the visualization editor.

1.11 Further Usages

Lots The individual methods described above are based on spectrographic Signatures, which in turn are based on the DFT or the FFT. The procedures can also use signatures based on other transformations, so on two-dimensional or three-dimensional time-frequency or time-energy representations. these can with the aid of the Wigner-Ville transformation, the Choi-Williams transformation, the cone-shaped transformation, the Gabor transformation, the discrete one Cosine transformation or the class of wavelet transformations based. The described methods, in particular the SLC modeling methods, as well as their constituents must be adjusted only slightly.

The described method and its constituents do not necessarily have to PCM-ko based on time series. Other forms of encoding the time series data are also usable.

applications result in virtually all areas of time series analysis. In particular, the system and method is always particularly useful if classifiers for Patterns modeled in time series with the help of expert knowledge should be.

Possible applications lie in the following areas: language processing, bioacoustics, medical technology (ECG, EEG, brain waves), Earthquake research, stock trading, error diagnosis and quality assurance in mechanical engineering, classification of radar and echolocation signals (Sonar signals), classification of acoustic and electromagnetic Signals, music data processing, various applied sciences.

1.12 economy

The Invention can greatly contribute to the cost of development Lowering Signal Classifiers: This task can, under Use of the invention, also be made by persons the above no or only limited knowledge in computer science and engineering feature.

The Invention can significantly contribute to the quality of signal classifiers too Improve: The development of signal classifiers can, under Use of the invention, also made by experts in the field be, for the classifiers are to be produced. Thus, e.g. Signal classifiers for medical purposes produced, modified and modified by medical professionals be tested. In the field of bioacoustics can signal classifiers by biologists themselves, modified and tested. In the field of speech processing can signal classifiers made by linguists or phonetics themselves, modified and be tested. For the remaining this applies equally.

Abbreviations

• KDD

  Knowledge Discovery in databases

  KDSE

  Knowledge Discovery Support Environment

  FLC

  First-level classifier (s)

  SLC

  Second-level classifier (s)

  AA

  Annotations algorithm

  KKF

  Class correlation function

  SSDM

  Symbolic-Similarity-Driven Merge

  FLCD

  First-level classifier Discovery

  SLCD

  Second-level classifier Discovery

Claims (54)

Contraption ( 3.20 ) for the production and use of signal classifiers which are suitable for machine classification or annotation ( 3.1 ) of time series signals (eg audio signals), characterized by a device ( 3.20 -1, 3.21 , in the following 'visualization module') for producing visualizable representations of transformed time-series data (visualization representations) and visualizable representations of classification decisions or annotation facts of signal classifiers, a device ( 3.20 -2, 3.22 , in the following FLC module) for the production of visualizable representations of subsymbolic signatures (FLC signatures) and representations of first-level classifiers (FLC) as well as for the classification or annotation of time-series data with the aid of FLC and a device ( 3.20 -3, 3.23 in the following SLC module) for the production of visualizable representations of symbolic signatures (SLC signatures) and representations of second-level classifiers (SLC) as well as for the classification or annotation of time-series data with the aid of SLC, wherein the three devices are formed the production and application of signal classifiers in modified KDD processes ( 3.2 . 3.3 ). Device according to claim 1, wherein the visualization module ( 3.20 -1, 3.21 ) is designed to provide time series data ( 3.21 -1) in time-frequency or time-energy representations ( 3.21 -2), from these using an interactive graphical user interface ( 3.21 -3, 4, 8, 9) sets of visualization representations ( 3.21 -10) used to initialize classifier signatures ( 3.22 -1, 2; 3.23 -1, 2) are used and representations of annotation facts ( 3.21 -5, 6) in visualization representations ( 3.21 -8) as well as visualizable graphs ( 3.21 -9) of class correlation functions (KKF) ( 3.21 -7). Apparatus according to claim 1, wherein the FLC module ( 3.20 -2, 3.22 ) is designed to work from visualization representations ( 3.2 1-10, 3.22 1) initial representations of FLC signatures ( 3.22 -2) to extract representations of FLC ( 3.22 -3, 7, 8, 9, 10, 12, 13), to change ( 3.22 -3, 4, 11), to manage ( 3.22 -7, 18, 19), to undergo test routines ( 3.22 -13) to replace constituents of representations of FLC ( 3.22 -4, 5, 6) to classify time series data using representations of FLC ( 3.22 -14), representations of FLC annotations ( 3.22 -15), manage ( 3.22 -15, 20, 21), and representations of FLC annotations ( 3.22 -15) and time series data ( 3.22 -17) Representations of FLC ( 3.22 -7) to extract automatically. Apparatus according to claim 1, wherein the SLC module ( 3.20 -3, 3.23 ) is designed to work from visualization representations ( 3.21 -10, 3.23 -1) initial representations of SLC signatures ( 3.23 -2) to extract representations of SLC ( 3.23 -3, 4, 5, 6, 10, 11, 12, 13), to change ( 3.23 -3, 4), to manage ( 3.23 -6, 16, 17), to undergo test routines ( 3.23 -14) to replace constituents of representations of SLC ( 3.23 -4,5) to classify time series data using representations of SLC ( 3.23 -7), representations of SLC annotations ( 3.23 -8) to manage, manage ( 3.23 -8, 18, 19), and representations of SLC annotations ( 3.23 -8) and time series data ( 3.23 -15) Representations of SLC ( 3.23 -6) to extract automatically. Apparatus according to claim 1, wherein the data exchange between the three modules over access to a shared file system, database system or network. Apparatus according to claim 2, wherein the production of visualization representations ( 3.21 -10) is performed by a user using an interactive graphical user interface (hereinafter visualization editor, 3.21 -3) selects and sets time and frequency parses, valid transformation methods, parameters and color codings with controls, and then triggers the computation of the representations. Apparatus according to claim 2, which is designed DFT-based, wavelet-based or other time-frequency representations as well as linear or logarithmic time-energy representations of Time series data to calculate, store and with the help of a display device making them visible by transforming the representations of the transformed ones Converts time series data into graphic objects (e.g., bitmap objects), adapts to the size of the display area and load into the image memory of the display device. Apparatus according to claim 2, which is adapted to facilitate the production of visualization representations ( 3.21 -10) can be controlled by buttons or buttons by clicking on buttons or pressing buttons to change the time and / or frequency parameters of the representations and then recalculate them with the parameters changed in this way. Apparatus according to claim 2, wherein areas of a visualization editor ( 3.21 3) visible representation can be marked by moving variable-size rectangle representations, polygonal representations or elliptic representations to desired locations in the graph using keys or dragging with a mouse and calculating marker parameters related to those of the graphic Refer to the representation of underlying data. Apparatus according to claim 2, wherein visualization representations ( 3.21 -10) as files or data records and by clicking on corresponding icons or table fields in the visualization editor ( 3.21 -3) are made visible. Apparatus according to claim 2, wherein parameter sets which are sufficient for producing visualization representations are stored as files or data records and the representations on which they are based are automatically regenerated and stored in the visualization editor ( 3.21 -3) are made visible. Apparatus according to claim 2, wherein visualization parameters, for the production of a visualization, valid transformations and color palettes of a currently in the editor ( 3.21 -3) visible representation can be changed by the corresponding variables using a property editor or a Selection control to be modified. Apparatus according to claim 2, wherein representations of calculated KKF ( 3.22 -6) and their constituents ( 3.5 ) within the user interface of the visualization module ( 3.20 -1) and the time axis of the KKF display is synchronized with the time axis of the currently visible visualization. Apparatus according to claim 2, wherein FLC and SLC annotation facts are converted into graphical representations and mapped to visualizations of time series data when applied to the visualization editor ( 3.21 -3) refer to the visible section of a time series. Apparatus according to claim 2, wherein in the visualization editor visible visualization representations to initialize FLC signatures by using the their underlying dataset as an initial template for a new one FLC signature is used. Apparatus according to claim 2, wherein in the visualization editor marked areas of visualization representations for initialization of FLC signatures are used by only those data points the underlying dataset that is within a marked Range to generate an initial template for a new one FLC signature used become. Apparatus according to claim 2, wherein in the visualization editor visible visualization representations used to initialize SLC by using them lying time and / or frequency parameters for producing a initial template for a new SLC representation be used. Apparatus according to claim 3, wherein the production of FLC representations ( 3.22 -7) is performed by a user using an interactive graphical user interface (hereinafter FLC Editor, 3.22 -3) Selects or sets time and frequency fields, parameters, data points, thresholds, and color codes with controls, thereby defining relevant characteristics of an FLC. Apparatus according to claim 3, wherein representations FLC signatures are made visible in the FLC editor by converted into graphic objects (e.g., bitmap objects) adapted to the size of the display area and loaded into the image memory of a display device. Apparatus according to claim 2, wherein areas of a FLC editor ( 3.22 3) can be marked by moving variable-size rectangle representations, polygonal representations or elliptic representations to desired locations in the graph using keys or by dragging with a mouse and calculating marker parameters related to that of the graph from their size and position refer to underlying data. Apparatus according to claim 3, wherein in the FLC editor with the aid of mouse-controlled events, individual data points of one currently visible signature, deleted, transformed, copied and blurred. Apparatus according to claim 3, wherein in the FLC editor Marked areas of a currently visible signature as a mask for data filters serve by passing through the filter only those areas of the underlying signature changed which lie within the marked areas. Apparatus according to claim 3, which is designed so that in the visualization editor ( 3.21 -3) visible transformed time series data and FLC visible in the FLC editor the KKF and annotation decisions are calculated in real time. Apparatus according to claim 3, wherein visualization parameters, KKF-specific parameters, search space parameters and parameters of the annotation algorithm of a current FLC editor ( 3.22 -3) visible FLC can be modified by modifying the corresponding variables using a property editor. Device according to Claim 3, characterized by a device for selecting and linking implementations of annotation algorithms ( 3.22 -5), implementations of KKF ( 3.22 -6) and representations of FLC signatures for the purpose of producing complete FLC and for their permanent storage on a data medium. Apparatus according to claim 3, characterized by means for selecting a subset of FLC representations from a pool of FLC representations ( 3.22 7) and by a device for selecting and starting a method for modifying or extending this subset by means of an elementary method ( 3.22 -8, 9, 10, 11, 12 or others), which requires a set of FLC representations as input and generates as output a set of FLC representations or representations of FLC signatures. Apparatus according to claim 3, characterized by means for selecting a subset of FLC representations from a pool of FLC representations ( 3.22 7) and by a device for selecting and starting a test procedure ( 3.22 -13) for FLC, which requires a set of FLC representations as input and generates as output a set of test-specific data. Apparatus according to claim 3, characterized by means for selecting a subset of FLC representations from a pool of FLC representations ( 3.22 7) and by a device for selecting a set of time series data ( 3.22 -17) as well as the start of a method for machine annotation ( 3.22 -14) of the time-series data using the subset of FLC representations so that a set of FLC annotations is automatically generated and stored on a data carrier ( 3.22 -15). Apparatus according to claim 3, characterized by means for selecting a subset of FLC representations from a pool of FLC representations ( 3.22 7) and by a device for selecting a set of FLC annotations ( 3.22 -15) and to start an automatic extraction procedure ( 3.22 -16) of FLC signatures or FLC representations from annotations and time series data. Apparatus according to claim 3, wherein accessing sets of FLC representations ( 3.22 -7) through a database ( 3.22 -18) or through a file manager ( 3.22 -19). Apparatus according to claim 3, wherein accessing sets of FLC annotations ( 3.22 -15) through a database ( 3.22 -20) or through a file manager ( 3.22 -21). Apparatus according to claim 4, wherein the production of SLC representations ( 3.23 -6) is performed by a user using an interactive graphical user interface (hereinafter SLC editor, 3.23 -3) selects and sets time and frequency excerpts, parameters, symbolic elements, threshold values and color codes with controls, thereby defining relevant properties of an SLC. Apparatus according to claim 4, wherein representations SLC signatures are made visible in the SLC editor by converted into graphic objects (e.g., bitmap objects) adapted to the size of the display area and loaded into the image memory of a display device. Apparatus according to claim 4, wherein areas of a SLC editor ( 3.23 -3) visible representation by moving variable-size rectangle representations, polygonal representations or elliptic representations to desired locations on the graph using keys or by dragging with a mouse and from their size and position marking parameters are calculated which are those of the graph underlying data. Apparatus according to claim 4, wherein in the SLC editor with the aid of mouse-controlled events individual symbolic elements a currently visible signature is marked, deleted and copied. Apparatus according to claim 4, wherein in the SLC editor Marked areas of a currently visible signature as a mask for data filters Serve by filtering only those areas of the underlying changed signature which correspond to the marked areas. Apparatus according to claim 4, which is designed so that in the visualization editor ( 3.21 -3) visible transformed time series data and SLC annotation decisions visible in the SLC editor are calculated in real time. Apparatus according to claim 4, wherein visualization parameters, search space parameters, hierarchy parameters, logical operators and parameters of the annotation algorithm of a current SLC editor ( 3.23 -3) visible SLC can be modified by modifying their corresponding variables using a property editor. Device according to claim 4, characterized by an apparatus for the selection and linking of implementations of annotation algorithms ( 3.23 -5) and representations of SLC signatures for the purpose of establishing complete SLC and for permanently storing SLC representations thus produced on a data medium. Apparatus according to claim 4, characterized by means for selecting a subset of SLC representations from a pool of SLC representations ( 3.23 6) and by a device for selecting and starting a method for modifying or extending this subset by means of an elementary method ( 3.23 -10, 11, 12, 13, or others) that requires a set of SLC representations as input and generates as output a set of SLC representations or representations of SLC signatures. Apparatus according to claim 4, characterized by means for selecting a subset of SLC representations from a pool of SLC representations ( 3.23 6) and by a device for selecting and starting a test procedure ( 3.23 -14) for SLC, which requires a set of SLC representations as input and generates as output a set of test-specific data. Apparatus according to claim 4, characterized by means for selecting a subset of SLC representations from a pool of SLC representations ( 3.23 6) and by a device for selecting a set of time series data ( 3.23 -15) and to start a machine annotation procedure ( 3.23 7) of the time series data using the subset of SLC representations so that a set of SLC annotations is automatically generated and stored on a data carrier ( 3.23 -8th). Apparatus according to claim 4, characterized by means for selecting a subset of SLC representations from a pool of SLC representations ( 3.23 6) and by a device for selecting a set of SLC annotations ( 3.23 -8) and to start an automatic extraction process ( 3.23 -9) of SLC signatures or SLC representations from annotations and possibly time series data. Apparatus according to claim 4, wherein accessing sets of SLC representations ( 3.23 -6) through a database ( 3.23 -16) or through a file manager ( 3.23 -17). Apparatus according to claim 4, wherein access to sets of SLC annotations ( 3.23 -8) through a database ( 3.23 -18) or through a file manager ( 3.23 -19). Method of making a first level classifier for a Event in a time series s →, characterized by the following steps: Generate a visualization of the event from a snippet of s →, so that all the characteristic features of the event in the visualization are well recognizable, extracting an initial FLC signature from the representation the visualization of the event, linking the extracted signature with an implementation of a KKF and an implementation of a Annotation algorithm to a complete classifier, saving a physical representation of the classifier thus formed on a data carrier or in a memory chip. Method for extracting a triple FLC signature from an n × m matrix M of data points, characterized by the following steps: extracting three subsets φ ⁺ , φ ^- and φ ⁰ from M such that φ ^{+ contains} only the elements from M, which are above a threshold value σ and local maxima, φ ^{0 contains} only the elements of M which are above a threshold value σ but are not local maxima, φ ^{- contains} only the elements of M which are below a threshold value σ and in time Frequency space to the elements of φ ⁺ and 0 ° are in a neighborhood relationship, combining the three subsets in a representation of the form (φ ⁺ , φ ^- , φ ⁰ ), storing a physical representation of the signature on a disk or in a memory chip. A method of manufacturing a triple FLC signature from a set of three times the FLC signatures, characterized by the steps of: forming a separate matrix from each individual signature by combining the three subsets φ ^+, φ ^-, and φ ^0, summing up all thus resulting matrices and averaging, generating a triple FLC signature from the resulting matrix according to the method of claim 47, storing a physical representation of the signature on a data carrier or in a memory device. Procedure for applying an FLC classifier to a time series s →, characterized by fol the following steps: calculating the graph G ^{+ of} a correlation or distance function of φ ⁺ for the entire signal, calculating the graph G ^- a correlation or distance function of φ ^- for the entire signal, extracting times at which G ⁺ and G ^- at the same time have a high correlation or a small distance to the signal, generating an annotation factor for each time thus extracted, storing a physical representation of each annotation factor thus generated on a data carrier or in a memory device. A method for producing a second-level classifier for an event in a time series s →, characterized by the following steps: generating a set K of FLC, which can detect a set of features characterizing the event in s →, generating a visualization in the Event is visible, initializing the SLC by taking over the time and possibly frequency expansion of the visualization, generating a set of annotation facts F ^K by annotating the visible in the visualization section of s → using the FLC in K, converting the set of annotation facts F ^K into a symbolic signature, normalizing the time parameters of the symbolic elements in the signature, linking the signature with an implementation of an annotation algorithm that evaluates sets of FLC annotations, storing a physical representation of the classifier on a data carrier or in a memory device. A method for extracting a set of symbolic signatures for an event class E from a signal s →, characterized by the steps of: generating a set K of FLC that can detect a set of features characterizing E in a time series, generating a set F ^K of FLC Annotations by annotating s → using the FLC in K, extracting a set SQ of sequences on the time and possibly frequency axis of adjacent annotation facts in F ^K , converting each sequence in SQ into a symbolic signature, so that a set SQ ' of signatures arises, normalizing the time parameters of the symbolic elements in SQ ', storing a physical representation of each element in SQ on a data carrier or in a memory device. Method for merging two symbolic signatures, characterized by the following steps: Determining a parameter Δτ ^opt , which specifies a time shift relevant to all symbolic elements of one of the two signatures, so that a maximum number of symbolic elements can be merged with each other by applying them, two symbolic ones Elements are then merge with each other when they are adjacent in time-frequency space (see Eqs. 1.61 and 1.62); Shifting all elements in one of the two signatures by the factor Δt ^opt , generating an amount φ S / V of fusible symbolic elements from both signatures, generating a set φ S / R of non-fusible symbolic elements from both signatures, merging all fusible pairs symbolic Elements in φ S / V to the set φ 'S / μ, forming the new signature by φ S / μ: = φ S / R ∪ φ' S / μ, storing a physical representation of the signature φ S / μ on a data carrier or in a memory chip. A method for applying an SLC classifier to a time series s →, characterized by the steps of: generating a set F ^K of FLC annotations using the FLC underlying the symbolic elements in the signature of the SLC, calculating a time for each subsequence of elements in F ^K that is in a threshold-exceeding similarity relationship to the signature of the SLC, generating an SLC annotation factor for each time value thus extracted, storing a physical representation of the annotation facts thus generated on a data carrier or in a memory device. Computer program with a program code to carry out the Method of making and using signal classifiers, according to the claims 46 to including 53, when the computer program runs on a data processing system.