EP4097695B1 - Method and device for identifying acoustic anomalies - Google Patents

Description

Embodiments of the present invention relate to a method and a device for detecting acoustic anomalies. Further exemplary embodiments relate to a corresponding computer program. According to exemplary embodiments, the detection of a normal situation and the detection of anomalies in comparison to this normal situation take place.

In real acoustic scenes there is usually a complex overlay of several sound sources. These can be positioned in the foreground or background and in any spatial position. A variety of possible sounds are also conceivable, which can range from very short transient signals (e.g. clapping, gunshot) to longer, stationary sounds (siren, passing train). A recording typically covers a specific period of time, which is divided into one or more time windows when viewed subsequently. Based on this subdivision and depending on the length of the noise (cf. transient or longer, stationary sound), a sound can extend over one or more audio segments/time windows.

In many application scenarios, an anomaly, i.e. a sound deviation from the "acoustic normal state", i.e. the amount of noise that is considered "normal", must be detected. Examples of such anomalies are breaking glass (burglary detection), a pistol shot (monitoring public events) or a chainsaw (monitoring nature reserves).

The problem is that the sound of the anomaly (out-of-order class) is often not known or cannot be precisely defined or described (e.g. what can a broken machine sound like?).

The second problem is that novel algorithms for sound classification using deep neural networks are very sensitive to changing (and often unknown) acoustic conditions in the operational scenario. Classification models that are trained with audio data, for example with a high-quality microphone, achieve this recorded, only poor recognition rates when classifying audio data that was recorded using a poorer microphone. Possible solutions lie in the area of “domain adaptation”, i.e. adapting the models or the audio data to be classified in order to achieve greater robustness in recognition. In practice, however, it is often logistically difficult and too expensive to record representative audio recordings at the later location of use of an audio analysis system and then annotate them with regard to the sound events they contain.

The third problem with audio analysis of environmental noise lies in data protection concerns, since classification methods can theoretically also be used to recognize and transcribe speech signals (e.g. when recording a conversation near the audio sensor).

The classification models of existing state-of-the-art solutions are as follows:
If the sound anomaly to be detected can be precisely specified, a classification model based on machine learning algorithms can be trained to recognize specific noise classes using supervised learning. Current studies show that neural networks in particular are very sensitive to changing acoustic conditions and that additional adaptation of classification models to the respective acoustic situation of the application must be carried out.

Furthermore, the disclosure of the prior art becomes EP 2 988 105 A2 , which describes a device of the method for the automatic detection and classification of audible acoustic signals in a surveillance area.

Based on the disadvantages explained above, there is a need for an improved approach. The object of the present invention is to create a concept for detecting anomalies that is optimized with regard to the learning behavior and that enables reliable and accurate detection of anomalies.

The task is solved by independent patent claims.

Embodiments of the present invention provide a method for detecting acoustic anomalies. The method includes the steps of obtaining a long-term recording, which has a duration of at least greater than 1 minute or at least 10 minutes or at least 1 hour or at least 24 hours, with a plurality of first audio segments assigned to respective first time windows and analyzing the plurality of first audio segments to get to each of the A plurality of first audio segments have a first feature vector describing the respective first audio segment, such as. B. to obtain a spectrum for the audio segment (time-frequency spectrum) or an audio fingerprint with certain characteristics for the audio segment. For example, the result of the analysis of a long-term recording divided into a large number of time windows is a large number of first (single- or multi-dimensional) feature vectors for the large number of first audio segments (assigned to the corresponding times/windows of the long-term recording), which represent the "normal state". . The method includes further steps of obtaining another recording with one or more second audio segments associated with respective second audio windows and analyzing the one or more second audio segments to obtain one or more feature vectors describing the one or more second audio segments. In this respect, the result of the second part of the method is, for example, a large number of second feature vectors (e.g. with corresponding times of further recording). In a subsequent step, the one or more second feature vectors are compared with the plurality of first feature vectors (e.g. by comparing the identities or similarities or by recognizing a sequence) in order to detect at least one anomaly. According to exemplary embodiments, it would be conceivable to recognize different forms of anomalies, namely a sound anomaly (i.e. a recognition of the first appearance of a previously unheard sound), a temporal anomaly (e.g. changed repetition pattern of a sound that has already been heard) or a spatial anomaly ( Occurrence of a sound that has already been heard in a previously unknown spatial position).

Embodiments of the present invention are based on the knowledge that an “acoustic normal state” and “normal noises” can be learned independently simply through a long-term sound analysis (phase 1 of the method comprising the steps of obtaining a long-term recording and analyzing it). This means that this long-term sound analysis results in an independent or autonomous adaptation of an analysis system to a specific acoustic scene. No annotated training data (recording + semantic class annotation) is required, which represents a great saving in time, effort and cost. When this acoustic "normal state" or the "normal" noises are recorded, the current noise environment can be carried out in a subsequent analysis phase (phase 2 with the steps of obtaining another recording and analyzing it). This involves comparing the current audio segment / current background noise with the “normal” noises recognized or learned in advance / in phase 1. In general, this means that phase 1 involves learning a model using the normal background noise based on a statistical procedure or machine learning, whereby this model then allows (in phase 2) to compare currently recorded background noise with regard to its degree of novelty (probability of an anomaly).

Another advantage of this approach is that the privacy of people who may be in the immediate vicinity of the acoustic sensors is protected. This is called privacy by design. Due to the system, speech recognition is not possible because the interface (audio in, anomaly probability function out) is clearly defined. This can dispel possible data protection concerns when using acoustic sensors.

Since the long-term recording represents the normal acoustic situation, the multitude of first audio segments describes this normal situation in themselves and/or in their order. In this respect, the large number of first audio segments represent a kind of reference on their own and/or in their combination. The aim of the method is to detect anomalies in comparison to this normal situation. This means that, according to exemplary embodiments, the result of the clustering described above is a description of the reference based on first audio segments. In the step in which the anomaly is detected, the second audio segments alone or in their combination (i.e. order) are then compared with the reference to represent the anomaly. The anomaly is a deviation of the current acoustic situation described by the second feature vectors from the reference described by the first feature vectors. In other words, this means that, according to exemplary embodiments, the first feature vectors alone or in their combination represent a reference image of the normal state, while the second feature vectors alone or in their combination describe the current acoustic situation, so that in step 126 the anomaly in Form of a deviation of the description of the current acoustic situation (see second feature vectors) from the reference (see first feature vectors) can be recognized. The anomaly is therefore defined by the fact that at least one of the second acoustic feature vectors deviates from the sequence of the first acoustic feature vectors. Possible deviations can be: sound anomalies, temporal anomalies and spatial anomalies.

According to one exemplary embodiment, phase 1 captures a large number of first audio segments, which are also referred to below as “normal” or “normal” considered noises/audio segments. According to exemplary embodiments knowing these "normal" audio segments makes it possible to detect a so-called sound anomaly. The sub-step of identifying a second feature vector that differs from the analyzed first feature vectors is then carried out.

According to further embodiments, when analyzing, the method includes the substep of identifying a repeat pattern in the plurality of first time windows. This involves identifying repeating audio segments and determining the resulting pattern. According to exemplary embodiments, identification is carried out using repeating, identical or similar first feature vectors belonging to different first audio segments. According to exemplary embodiments, identical and similar first feature vectors or first audio segments can also be grouped into one or more groups during identification.

According to exemplary embodiments, the method includes recognizing a sequence of first feature vectors associated with the first audio segments or recognizing a sequence of groups of identical or similar first feature vectors or first audio segments. The basic steps make it advantageously possible to recognize normal noises or to recognize normal audio objects. The combination of these normal audio objects in a certain order or a certain repetition pattern in terms of time then represents a normal acoustic state.

According to further exemplary embodiments, it would also be conceivable for a repetition pattern to be recognized in the one or more second time windows and/or a sequence of second feature vectors associated with different second audio objects or groups of identical or similar second feature vectors. According to further exemplary embodiments, this method then enables the sub-step of comparing the repetition pattern of the first audio segments and/or order in the first audio segments with the repetition pattern of the second audio segments and/or order in the second audio segments to be carried out during matching. This comparison enables the detection of a temporal anomaly.

According to a further exemplary embodiment, the method may include the step of determining a respective position for the respective first audio segments. According to one exemplary embodiment, the respective position can also be determined for the respective second audio segments are made. According to one exemplary embodiment, this then enables the detection of a spatial anomaly to be carried out by the substep of comparing the position assigned to the respective first audio segments with the position assigned to the corresponding respective second audio segment.

It should be noted that for spatial localization, for example, at least 2 microphones are used, while one microphone is sufficient for the other two types of anomalies.

As already indicated above, each feature vector (first and second feature vector) can each have one dimension or several dimensions for the different audio segments. A possible realization of a feature vector would be, for example, a time-frequency spectrum. According to one embodiment, the dimensional space can also be reduced. In this respect, according to exemplary embodiments, the method includes the step of reducing the dimensions of the feature vector.

According to a further exemplary embodiment, the method can have the step of determining a probability of occurrence of the respective first audio segment and of entering the probability of occurrence together with the respective first feature vector. Alternatively, the method can have the step of determining a probability of occurrence of the respective first audio segment and outputting the probability of occurrence with the respective first feature vector and an associated first time window. In this respect, the probability of occurrence for the respective audio segment or a more precise probability of the occurrence of the audio segment at this point in time is output. The output takes place with the corresponding data record or feature vector.

According to one exemplary embodiment, the method can also be implemented using a computer. In this respect, the method has a computer program with a program code for carrying out the method.

Further exemplary embodiments relate to a device with an interface and a processor. The interface serves to obtain a long-term recording with a plurality of first audio segments assigned to respective first time windows and to obtain a further recording with one or more second audio segments assigned to respective second time windows. The processor is designed to handle the multitude to analyze the first audio segments in order to obtain a first feature vector describing the respective first audio segment for each of the plurality of first audio segments. Furthermore, the processor is designed to analyze the one or more second audio segments in order to obtain one or more feature vectors describing the one or more second audio segments. Furthermore, the processor is designed to match the one or more second feature vectors with the plurality of first feature vectors in order to detect at least one anomaly.

According to exemplary embodiments, the device comprises a recording unit connected to the interface, such as. B. a microphone or a microphone array. The microphone array advantageously enables position determination, as already explained above. According to further exemplary embodiments, the device comprises an output interface for outputting the probability of occurrence explained above.

Fig. 1

ein schematisches Flussdiagramm zur Illustration des Verfahrens gemäß einem Basisausführungsbeispiel;

Fig. 2

eine schematische Tabelle zur Illustration von unterschiedlichen Anomalietypen; und

Fig. 3

ein schematisches Blockschaltbild zur Illustration einer Vorrichtung gemäß einem weiteren Ausführungsbeispiel.

Embodiments of the present invention are explained with reference to the accompanying drawings. Show it:

Fig. 1

a schematic flowchart illustrating the method according to a basic embodiment;

Fig. 2

a schematic table illustrating different anomaly types; and

Fig. 3

a schematic block diagram to illustrate a device according to a further exemplary embodiment.

Before the following exemplary embodiments of the present invention are explained with reference to the accompanying drawings, it should be noted that elements and structures with the same effect are provided with the same reference numerals so that the description of them can be applied to one another or interchangeable.

Fig. 1 shows a method 100, which is divided into two phases 110 and 120.

In the first phase 110, referred to as the adjustment phase, there are two basic steps. This is marked with reference numbers 112 and 114. Step 112 includes a long-term recording the acoustic normal state in the application scenario. Here, for example, the analysis device 10 (cf. Fig. 3 ) placed in the target environment so that a long-term recording 113 of the normal state is captured. According to the invention, this long-term recording can last for at least 1 minute or at least 10 minutes or at least 1 hour or at least 24 hours.

This long-term recording 113 is then broken down, for example. The breakdown can be divided into time periods of equal length, such as: B. 1 second or 0.1 seconds or dynamic time ranges. Each time range includes an audio segment. In step 114, commonly referred to as analyzing, these audio segments are examined separately or in combination. For this purpose, a so-called feature vector 115 (first feature vectors) is determined for each audio segment during analysis. Generally speaking, it is said that in the conversion of a digital recording 113 into one or more feature vectors 115 - e.g. B. using deep neural networks - with each feature vector 115 "encoding" the sound at a specific time. Feature vectors 115 can be determined, for example, by an energy spectrum for a specific frequency range or generally a time-frequency spectrum.

It should be noted at this point that it is optionally possible for the dimensionality of the feature space of the feature vectors 115 to be reduced using statistical methods (e.g. principal component analysis). In step 114, typical or dominant noises can then optionally be identified using unsupervised learning methods (e.g. clustering). Here, time periods or audio segments are grouped that have similar feature vectors 115 and that accordingly have a similar sound. No semantic classification of a sound (e.g. “car” or “plane”) is necessary. In this respect, so-called unsupervised learning takes place based on the frequencies of repeating or similar audio segments. According to a further exemplary embodiment, it would also be conceivable that in step 114 an unsupervised learning of the temporal order and/or typical repetition patterns of certain noises takes place.

The result of clustering is a compilation of audio segments or noises that are normal or typical for this area. For example, each audio segment can also be assigned a probability of occurrence. Furthermore, one can also Repeat patterns or a sequence, i.e. a combination of several audio segments, are identified that are typical or normal for the current environment. For this purpose, different audio segments can also be assigned a probability to each grouping, each repeat pattern or each sequence.

At the end of the adjustment phase, audio segments or grouped audio segments are known and described as feature vectors 115 that are typical for this environment. In a next step or in a next phase 120, this learned knowledge is then applied accordingly. Phase 120 has the three basic steps 122 and 124 and 126.

In step 122, an audio recording 123 is again recorded. This is typically significantly shorter compared to the audio recording 113. For example, this audio recording is shorter compared to audio recording 113. However, it can also be a continuous audio recording. This audio recording 123 is then analyzed in a subsequent step 124. This step is comparable in content to step 114. This in turn involves converting the digital audio recording 123 into feature vectors. If these second feature vectors 125 are now available, they can be compared with the feature vectors 115.

• klangliche Anomalie (neuer, bisher ungehörter Klang)
• zeitliche Anomalie (bereits gehörter Klang tritt zeitlich "unpassend" auf, wiederholt sich zu schnell oder tritt in falscher Reihenfolge mit anderen Klängen auf)
• räumliche Anomalie (bereits gehörter Klang tritt an "ungewohnter" räumlicher Position auf oder die entsprechende Quelle folgt einem ungewohnten räumlichen Bewegungsmuster)

The comparison takes place in step 126 with the aim of detecting anomalies. Very similar feature vectors and very similar ordering of feature vectors indicate that there is no anomaly. Deviations from previously determined patterns (repetition patterns, typical sequences, etc.) or deviations from the previously determined audio segments characterized by different/new feature vectors indicate an anomaly. These are recognized in step 126. At step 126, different types of anomalies may be detected. These are, for example:

• sound anomaly (new, previously unheard sound)
• temporal anomaly (a sound that has already been heard occurs "inappropriately" in time, repeats itself too quickly or occurs in the wrong order with other sounds)
• spatial anomaly (already heard sound occurs in an "unusual" spatial position or the corresponding source follows an unfamiliar spatial movement pattern)

These anomalies are referred to Fig. 2 explained in more detail.

Optionally, a probability for each of the three anomaly types can be output at time X. This is with the arrows 126z, 126k and 126r (one arrow for each type of anatomy). Fig. 3 illustrated.

At this point it should be noted that when comparing the feature vectors there is often no identity, but only similarity. In this respect, according to exemplary embodiments, threshold values can be defined as to when feature vectors are similar or when groups of feature vectors are similar, so that the result then also presents a threshold value for an anomaly. This threshold application can also be linked to the output of the probability distribution or appear in this in combination, e.g. B. to enable more accurate temporal detection of anomalies.

According to further exemplary embodiments, it is also possible to detect spatial anomalies. For this purpose, step 114 in adjustment phase 110 can also include unsupervised learning of typical spatial positions and/or movements of certain noises. Typically in such a case, instead of the in Fig. 3 Microphone 18 shown has two microphones or a microphone array with at least two microphones. In such a situation, spatial localization of the current dominant sound sources/audio segments is then possible in the second phase 120 through multi-channel recording. The underlying technology here can be, for example, beamforming.

Referring to Figs. 2a-2c Three different anomalies will now be explained. Fig. 2a illustrates the temporal anomaly. Here, audio segments ABC for both phase 1 and phase 2 are plotted along the time axis t. In phase 1 it was recognized that a normal situation or normal order exists such that the audio segments ABC appear in the order ABC. For one, a repeat pattern was recognized that can be followed by another group ABC after the first group ABC.

If exactly this pattern ABCABC is recognized in phase 2, it can be assumed that there is no anomaly or at least no temporal anomaly. However, if the pattern ABCAABC shown here is recognized, there is a temporal anomaly because another audio segment A is arranged between the two groups ABC.

This audio segment A or anomalous audio segment A is provided with a double frame.

Further in Fig. 2b a sonic anomaly is illustrated. In phase 1, the audio segments ABCABC were again recorded along the time axis t (cf. Fig. 2a ). The sound anomaly during recognition is shown by the fact that another audio segment, here audio segment D, appears in phase 2. This audio segment D has an increased length, e.g. B. over two time ranges and is therefore illustrated as DD. The sound anomaly is double-framed in the audio segment species order. This sound anomaly could, for example, be a sound that was never heard during the learning phase. For example, there may be thunder that differs from the previous elements ABC in terms of loudness/intensity and length.

In relation to Fig. 2c a local anomaly is illustrated. In the initial learning phase, two audio segments A and B were detected at two different positions, position 1 and position 2. During phase 2, both elements A and B were recognized, with localization determining that both audio segment A and audio segment B were at positions 1. The presence of audio segment B at position 1 represents a spatial anomaly.

Referring to Fig. 3 A device 10 for sound analysis will now be explained. The device 10 essentially includes the input interface 12, such as. B. a microphone interface and a processor 14. The processor 14 receives the one or more (simultaneously present) audio signals from the microphone 18 or the microphone array 18 'and analyzes them. To this end, he essentially leads in connection with Fig. 1 Steps 114, 124 and 126 explained. In each phase, the result to be output (cf. output interface 16) is a set of feature vectors that represent the normal state or, in phase 2, an output of the detected anomalies, e.g. B. assigned to a specific type and/or assigned to a specific time.

In addition, the interface 16 can refer to a probability of anomalies or a probability of anomalies at certain times or, in general, a probability of feature vectors at certain times.

• Sicherheitsüberwachung von Gebäuden und Anlagen
  • o Detektion von Einbrüchen (z. B. Glasbruch)/Beschädigungen (Vandalismus)
• Predictive Maintenance
  • o Erkennung von beginnendem Fehlverhalten von Maschinen aufgrund ungewöhnlicher Klänge
• Überwachung öffentlicher Plätze/Ereignisse (Sportereignisse, Musikereignisse, Demonstrationen, Kundgebungen usw.)
  • o Erkennung von Gefahrengeräuschen (Explosion, Schuss, Hilfeschreie)
• Verkehrsmonitoring
  • o Erkennen bestimmter Fahzeuggeräusche (z. B. durchdrehende Reifen - Raser)
• Logistikmonitoring
  • ∘ Überwachung von Baustellen - Erkennung von Unfällen (Einsturz, Hilfeschreie)
• Health
  • o akustische Überwachung des normalen Alltags älterer/kranker Menschen
  • o Erkennung von Stürzen/Hilfeschreien

According to exemplary embodiments, the device 10 or the audio system is designed to (simultaneously) detect various types of anomalies, e.g. B. at least two anomalies to be recognized. The following areas of application would be conceivable:

• Security monitoring of buildings and facilities
  • o Detection of break-ins (e.g. broken glass)/damage (vandalism)
• Predictive maintenance
  • o Detection of early machine malfunctions due to unusual sounds
• Monitoring public places/events (sporting events, music events, demonstrations, rallies, etc.)
  • o Detection of danger sounds (explosion, gunshot, cries for help)
• Traffic monitoring
  • o Detecting certain vehicle noises (e.g. spinning tires - speeding)
• Logistics monitoring
  • ∘ Monitoring of construction sites - detection of accidents (collapse, cries for help)
• Health
  • o acoustic monitoring of the normal everyday life of elderly/sick people
  • o Detection of falls/cries for help

Although some aspects have been described in connection with a device, it is understood that these aspects also represent a description of the corresponding method, so that a block or a component of a device is also to be understood as a corresponding method step or as a feature of a method step. Similarly, aspects described in connection with or as a method step also represent a description of a corresponding block or detail or feature of a corresponding device. Some or all of the method steps may be performed by a hardware apparatus (or using a hardware device). Apparatus), such as a microprocessor, a programmable computer or an electronic circuit can be carried out. In some embodiments, some or more of the key process steps may be performed by such apparatus.

Depending on particular implementation requirements, embodiments of the invention may be implemented in hardware or in software. The implementation may be using a digital storage medium such as a floppy disk, a DVD, a Blu-ray Disc, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, a hard drive or other magnetic or optical memory are carried out on which electronically readable control signals are stored, which can interact or interact with a programmable computer system in such a way that the respective method is carried out. Therefore, the digital storage medium can be computer readable.

Some embodiments according to the invention thus include a data carrier that has electronically readable control signals that are capable of interacting with a programmable computer system such that one of the methods described herein is carried out.

In general, embodiments of the present invention may be implemented as a computer program product with a program code, the program code being effective to perform one of the methods when the computer program product runs on a computer.

The program code can, for example, also be stored on a machine-readable medium.

Other embodiments include the computer program for performing one of the methods described herein, the computer program being stored on a machine-readable medium.

In other words, an exemplary embodiment of the method according to the invention is therefore a computer program that has a program code for carrying out one of the methods described herein when the computer program runs on a computer.

A further exemplary embodiment of the method according to the invention is therefore a data carrier (or a digital storage medium or a computer-readable medium) on which the computer program for carrying out one of the methods described herein is recorded. The data carrier, digital storage medium or computer-readable medium is typically tangible and/or non-transitory.

A further exemplary embodiment of the method according to the invention is therefore a data stream or a sequence of signals which represents the computer program for carrying out one of the methods described herein. The data stream or the sequence of signals can, for example, be configured to be transferred via a data communication connection, for example via the Internet.

Another embodiment includes a processing device, such as a computer or a programmable logic device, configured or adapted to perform one of the methods described herein.

Another embodiment includes a computer on which the computer program for performing one of the methods described herein is installed.

A further embodiment according to the invention includes a device or system designed to transmit a computer program to a receiver for carrying out at least one of the methods described herein. The transmission can take place electronically or optically, for example. The recipient may be, for example, a computer, a mobile device, a storage device or a similar device. The device or system can, for example, comprise a file server for transmitting the computer program to the recipient.

In some embodiments, a programmable logic device (e.g., a field programmable gate array, an FPGA) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform any of the methods described herein. In general, the methods in some embodiments are carried out by one any hardware device. This can be universally applicable hardware such as a computer processor (CPU) or hardware specific to the method, such as an ASIC.

The devices described herein may be implemented, for example, using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

The devices described herein, or any components of the devices described herein, may be at least partially implemented in hardware and/or in software (computer program).

The methods described herein may be implemented, for example, using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

The methods described herein, or any components of the methods described herein, may be performed at least in part by hardware and/or by software.

The embodiments described above are merely illustrative of the principles of the present invention. It will be understood that modifications and variations of the arrangements and details described herein will occur to others skilled in the art. Therefore, it is intended that the invention be limited only by the scope of the following claims and not by the specific details presented from the description and explanation of the exemplary embodiments herein.

Scientific literature

• [Borges_2008] N. Borges, GGL Meyer: Unsupervised Distributional Anomaly Detection for a Self-Diagnostic Speech Activity Detector, CISS, 2008, pp. 950-955 .
• [Ntalampiras_2009] S. Ntalampiras, I. Potamitis, N. Fakotakis: On Acoustic Surveillance of Hazardous Situations, ICASSP, 2009, pp. 165-168 .
• [Borges_2009] N. Borges, GGL Meyer: Trimmed KL Divergence between Gaussian Mixtures for Robust Unsupervised Acoustic Anomaly Detection, INTERSPEECH, 2009 .
• [Marchi_2015] E. Marchi, F. Vesperini, F. Eyben, S. Squartini, B. Schuller: A Novel Approach for Automatic Acoustic Novelty Detection using a Denoising Autoencoder with Bidirectional LSTM Neural Networks, ICASSP 2015, pp. 1996-2000 .
• [Valenzise_2017] G. Valenzise, L. Gerosa, M. Tagliasacchi, F. Antopnacci, A. Sarti: Scream and Gunshot Detection and Localization for Audio-Surveillance Systems, IEEE ICAVSBS, 2017, pp. 21-26 .
• [Komatsu_2017] T. Komatsu, R. Kondo: Detection of Anomaly Acoustic Scenes based on a Temporal Dissimilarity Model, ICASSP 2017, pp. 376-380 .
• [Tuor_2017] A. Tuor, S. Kaplan, B. Hutchinson, N. Nichols, S. Robinson: Deep Learning for
• Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streams, AAAI 2017, pp. 224231 .

Claims (15)

1. Method (100) for recognizing acoustic anomalies, comprising:

   obtaining (113) a long-term recording having a plurality of first audio segments (ABCD) associated to respective first time windows; wherein the long-term recording comprises at least a duration of more than 1 minute, of at least 10 minutes or at least 1 hour or at least 24 hours;

   analyzing (114) the plurality of the first audio segments (ABCD) to obtain, for each of the plurality of the first audio segments (ABCD), a first characteristic vector describing the respective first audio segment (ABCD);

   obtaining (123) a further recording having one or more second audio segments (ABCD) associated to respective second time windows;

   analyzing (124) the one or more second audio segments (ABCD) to obtain one or more characteristic vectors describing the one or more second audio segments (ABCD);

   matching (126) the one or more second characteristic vectors with the plurality of the first characteristic vectors to recognize at least one anomaly when compared to an acoustic normal situation for this environment.
2. Method (100) in accordance with claim 1, wherein the anomaly comprises a sound, temporal and/or spatial anomaly; and/or
   wherein the anomaly comprises a sound anomaly in combination with a temporal anomaly or a sound anomaly in combination with a spatial anomaly or a temporal anomaly in combination with a spatial anomaly.
3. Method (100) in accordance with claim 1 or 2, the method (100), when analyzing, comprising the sub-step of identifying a repetition pattern in the plurality of the first time windows; or
   the method (100), when analyzing, comprising the sub-step of identifying a repetition pattern in the plurality of the first time windows; wherein identifying is performed using repeating, identical or similar first characteristic vectors belonging to different first audio segments (ABCD).
4. Method (100) in accordance with claim 3, wherein, when identifying, grouping of identical or similar first characteristic vectors to form one or more groups is performed; and/or
   the method (100) comprising recognizing an order of first characteristic vectors belonging to different first audio segments (ABCD) or recognizing an order of groups of identical or similar first characteristic vectors.
5. Method (100) in accordance with any of claims 3 to 4, the method (100) comprising identifying a repetition pattern in the one or more second time windows; and/or
   the method (100) comprising recognizing an order of second characteristic vectors belonging to different second audio segments (ABCD) or recognizing an order of groups of identical or similar second characteristic vectors.
6. Method (100) in accordance with claim 5, the method (100) comprising the sub-step of matching the repetition pattern of the first audio segments (ABCD) and/or order in the first audio segments (ABCD) with the repetition pattern of the second audio segments (ABCD) and/or order in the second audio segments (ABCD) in order to recognize a temporal anomaly.
7. Method (100) in accordance with any of the preceding claims, wherein matching comprises the sub-step of identifying a second characteristic vector, which differs from the first characteristic vectors analyzed, in order to recognize a sound anomaly.
8. Method (100) in accordance with any of the preceding claims, wherein the characteristic vector comprises one dimension, more dimensions or a reduced dimension space; and/or
   wherein the method (100) comprises the step of reducing the dimensions of the characteristic vector.
9. Method (100) in accordance with any of the preceding claims, the method (100) comprising the step of determining a respective position for the respective first audio segments (ABCD), or
   the method (100) comprising the step of determining a respective position for the respective first audio segments (ABCD); the method (100) comprising the step of determining a respective position for the respective second audio segments (ABCD), and the method (100) comprising the sub-step of matching the position associated to the respective first audio segment (ABCD) with the position associated to the corresponding respective second audio segment (ABCD) in order to recognize a spatial anomaly.
10. Method (100) in accordance with any of the preceding claims, the method (100) comprising the step of determining a probability of occurrence of the respective first audio segment (ABCD) and outputting the probability of occurrence with the respective first characteristic vector, or the method (100) comprising the step of determining a probability of occurrence of the respective first audio segment (ABCD) and outputting the probability of occurrence with the respective first characteristic vector and a first time window.
11. Method in accordance with any of the preceding claims, wherein the plurality of the first audio segments and/or the plurality of the first audio segments in their order describe an acoustic normal state in the application scenario and/or represent a reference; and/or
    wherein the one anomaly is recognized when one or more second characteristic vectors deviate from the plurality of the first characteristic vectors.
12. Method in accordance with any of the preceding claims, wherein the further recoding comprises a time window or, in particular, a time window of less than 5 minutes, less than 1 minute, or less than 10 seconds.
13. Computer program comprising program code which, when running on a computer, executes one or more steps of the method (100) in accordance with the preceding claims.
14. Apparatus (10) for recognizing acoustic anomalies, comprising:

    an interface (12) for obtaining a long-term recording (113) having a plurality of first audio segments (ABCD) associated to respective first time windows, and for obtaining a further recording (123) having one or more second audio segments (ABCD) associated to respective second time windows; wherein the long-term recording comprises at least a duration of more than 1 minute, or of at least 10 minutes or at least 1 hour or at least 24 hours;

    a processor (14) configured for analyzing the plurality of the first audio segments (ABCD) to obtain, for each of the plurality of the first audio segments (ABCD), a first characteristic vector describing the respective first audio segment (ABCD), and configured for analyzing the one or more second audio segments (ABCD) to obtain one or more characteristic vectors describing the one or more second audio segments (ABCD), and configured for matching the one or more second characteristic vectors with the plurality of the first characteristic vectors to recognize at least one anomaly when compared to an acoustic normal situation for this environment.
15. Apparatus (10) in accordance with claim 14, the apparatus (10) comprising a microphone (18) or a microphone array connected to the interface (12); and/or
    the apparatus (10) comprising an output interface for outputting a probability of occurrence of the respective first audio segment (ABCD) having the respective first characteristic vector or for outputting a probability of occurrence of the respective first audio segment (ABCD) having the respective first characteristic vector and a first time window.

Images