DE102016226042A1 - Device, means of transport and method for designing a transition between two audio files - Google Patents

Abstract

A device, a means of locomotion and a method for designing a transition (11) during an audio reproduction (9) from a first audio file (6a) to a second audio file (6b) are proposed. The method comprises the steps of: automatically examining the audio files (6a, 6b) with respect to an audio material (8) contained in each case and / or automatically examining a vehicle condition and designing the transition (11) as a function of a result of the examination.

Description

The present invention relates to a device, a means of locomotion and a method for designing a transition in an audio reproduction of a first audio file to a second audio file. In particular, the present invention relates to a better adaptation of audio transitions in music enjoyment in means of transportation.

Software is known from the prior art, which allows two songs temposynchronisiert with a volume crossover or a so-called "cut" merge into each other. However, the transition always takes place in a predefined period of time. This is z. B. 8 beats. Only the time 8 beats before the time considered as the end of the song is set as out-point. The same applies to the in-point, which is set to 8 beats after the time considered as the beginning of the song.

A software which has implemented the described method is, for. Serato Pyro. In this software, an automatic arrangement of a lot of songs is realized, which relies on meta information such as speed, pitch, and other parameters for similarity determination of two digital audio signals. Based on these parameters, the songs are ranked consecutively based on similarity.

The software "Pacemaker" is also known from the prior art, which also automatically creates synchronized transitions between two songs. The shape of the transitions is limited to various forms of volume crossover, transitions with little influence on the different frequency bands by an equalizer, as well as a cut and leaves out further creative transitional forms. The software determines several potential transitional periods depending on the songs being transposed. The selection of the transition times takes place exclusively taking into account the maximization of the so-called "transition quality".

Based on the aforementioned prior art, it is an object of the present invention to better exploit the technical possibilities for acoustically appealing automatic design of transitions between two audio files.

The above object is achieved by a method for designing a transition in an audio playback of a first audio file to a second audio file. As a "transition" is understood in the context of the present invention, a time sequence between the audio playback of two or more files without significant break. The transition can be hard ("cut") or with a temporary simultaneous playback of the two audio files. In particular, the transition should allow a continuous enjoyment of the music of the user. Audio files can be understood as any digital files in which an audio signal is encoded and provided for playback via an audio system. The audio file may include metadata and audio material. Examples of audio files are .wav, mp3, .m4a, .aac, etc. In a first step, the first and the second audio file, optionally also other audio files, are automatically examined. In particular, a signal processing examination of the audio material contained in the audio files is made. This can be done automatically as soon as an appropriate command has been issued for an upcoming playback of multiple audio files. For example. For example, the user may initiate audio playback of stored files and / or saved playlists. Examining the audio material may include, for example, advanced song analysis, as described in detail below. In particular, in the investigation of the audio material is not understood an evaluation of metadata, which are usually contained in audio files and include specific information on the year of publication, artist, title, genre, etc. As an alternative or in addition to the aforementioned investigations relating to the audio material, a vehicle state (also vehicle operating state, vehicle residence, traffic situation, vehicle operating state or the like) can be examined. In other words, a situational adaptation is made taking into account a means of locomotion of the user. This z. As a speed, a longitudinal / lateral acceleration, a location, a traveled road type, a linguistic / scenic region are determined as the location of the means of transport. The aforesaid information can be used to individually or in any combination with each other in each case to make the transition in the audio playback in response to a result of the investigation. Both the examination and the design can be triggered and / or performed automatically. As a result, the present invention offers the possibility of experiencing the music enjoyment of a variety of music files seamlessly and with a maximum of transient quality.

The dependent claims show preferred developments of the invention.

Designing the transition between the first and second audio files may define a temporal association between the playback of the first audio file and the second audio file. For example. For example, a transitional period not equal to 0 seconds of simultaneous playback and / or synchronization of the music files can take place. In addition, volume characteristics during the transitional period can be shaped in accordance with the invention to design the transition. Alternatively or additionally, a ranking of the available audio files can be understood as a design of the transition. In other words, the order of reproduction can be modified in response to the above examination results. In this case, a second audio file can be reset in favor of a third audio file or completely disregarded with respect to a current playback order. The signal can also be changed at the frequency level by effects such as EQ, Flanger, Phaser, etc., which are not an exclusively playback volume related action. The transitions can be designed in this way in a situation-related and / or audiosignal-related manner and fit better into the overall situation, especially when used in a means of transportation. As a result, the enjoyment of the music of the user is increased and acoustically unappealing transitions are avoided. As a result, the traffic safety of the means of transport and the user experience can be improved.

Examination of the audio files may involve segmentation of the audio material. In particular, the respective audio files before the definition of the transition into respective units of meaning such. B. "Intro", "stanza", "chorus", "Outro" etc. are divided. In the design of the transition can be provided that the transition refers only to one of the aforementioned segments of the first audio file and / or on one of the segments of the subsequently reproduced audio file. In particular, the segments can be chosen such that acoustically usually matching segments are selected for the design of the transition. For example, it has proved advantageous to reproduce (mix) only one segment during the transition in which vocals are contained. The segmentation can therefore in particular define categories of vocal-containing and purely instrumental units of meaning. It is important for the segmentation in the sense of the present invention that certain temporary properties of temporally contiguous regions of the audio material are retained until, with the end of a considered segment, another segment begins with different properties. Such a segmentation enables a high degree of transient quality and the best possible audio experience when listening to music of consecutively reproduced audio files.

The examination of the audio files may include a categorization of the segments. Predefined categories can be found for the segments of the potentially reproduced audio files. Alternatively or additionally, a respective audio file can be broken down into as many segments (non-predefined category) as the variation of the acoustic characteristics of the segments requires. The segments or their categories may have a particular meaning or character of a musical nature, which may (or may not) be suitable for determining suitability to design a transition to another audio file segment. The categorization of the segments according to the invention can therefore provide a further improvement of the audio experience as well as the traffic safety when listening to music in the means of transportation. In this case, the headers or metadata are not read out and compared with one another in order to determine the suitability of two audio files to form a common transition. However, this does not exclude the possibility that such headers / metadata may be used in addition to the design of the transition in order to improve the information situation for the design of the transition.

The investigation of the vehicle state or vehicle operating state can include the analysis of a driving situation, a traffic situation, a vehicle situation and in particular an operating situation of the means of locomotion. The latter describes the condition resulting from the interaction with the user. In this case, an interaction with the vehicle guidance (not the operation of a music reproduction system, for example) should be understood in particular. The inventors have recognized that in particular the evaluation of characteristics of the audio material for the design of a transition can be dependent on an operating situation of the means of transportation. In other words, an operating situation can decide which characteristics are to be used in the evaluation of audio material or with a particularly high priority when the design of a transition from an audio file to the second audio file is required. For example. In a sporty driving style, the dynamics of a transition or the abundance of events in the transition area can play a more important role than during the "traveling". Accordingly, a user activity level ("how active is the user currently?") May then be determined in response to a result of examination of an operation state of a music reproduction control function (a vehicle or a stand-alone device). In relaxed driving situations, for example, intro and outro two music files suitable for a Transition region, while in dynamic driving situations, using respective refractions to design a transition is more appropriate.

In particular, it is preferred to evaluate the music playback behavior of different users and to use for the design of the transition of a watched playback system. For example. For example, the music consumption behavior, the reproduction behavior, the reproduction control can be determined in a first music reproduction system, transmitted in data form to a server and evaluated there or processed with further information from other music reproduction systems. This results in a self-learning system, which can use findings from the determined music reproduction behaviors to design the transition in another means of transportation. For example. For example, the acceptance of a parameter of a transition between two songs can be determined such that similar transitions are accepted by different users. The processes of the self-learning system generate a personal set of rules, which automatically carries out an evaluation of potential transitions and, in addition, can, for example, be carried out by offering an element for the evaluation query during a transition. In other words, during playback of a transition according to the invention, an output (eg on a screen) can be output, by means of which the user can carry out an evaluation of the quality of the transition. If a comparable transition by different users is evaluated as positive, the transition can be used preferably for the design of transitions in a means of transport according to the invention. If a transition or parameters of a transition are predominantly evaluated as negative, it is possible to avoid using corresponding transitions / parameters in the future.

For example. For example, the method according to the invention can be used to replace missing meta information in an audio file or to supplement existing meta information. For example. For example, characteristics in the audio material may be recognized to be a genre, an epoch, an artist, or the like. to investigate. Optionally, a corresponding meta-information (eg, in response to user confirmation or comparison with a datum-accessible reference) may supplement the meta-information missing in the file. In other words, the audio file is accessed in writing and the music material is used to write automatically determined information into the header of the audio file. In particular, the method according to the invention can be triggered in response to the detection of a missing meta information in order to generate transitions corresponding to the meta information.

Examination of the audio files may include determining and evaluating a semantic relationship of the audio files. The semantic relationship can use semantic prioritization (see below). The semantic relationship can be based on a semantic macro-segmentation and / or a semantic micro-segmentation (see below). Depending on the result of the evaluation, the design of the transition can take place and, in particular, a sequence of the audio files provided for the purpose of rendering can be used.

According to a second aspect of the present invention, there is provided an apparatus for designing a transition in audio reproduction from a first audio file to a second audio file. The device comprises an evaluation unit, which, for example, as an electronic control unit, microcontroller, programmable processor o. Ä. can be designed. An output unit is provided to output the information obtained by the evaluation unit about the design of the transition. In particular, the output unit may comprise a power amplifier and / or a loudspeaker. The evaluation unit is set up to automatically examine the audio files (two, three or more audio files) for each audio material contained. In other words, a signal analysis of the audio material is done to prepare a transition design. Alternatively or additionally, a vehicle state (vehicle operating state, driving situation, traffic situation, etc. (see above)) can be automatically investigated and used for the design of the transition. In each case, the transition is then made depending on one of the above results or both of the above results. In other words, the device according to the invention is set up to achieve the features, combinations of features and the resulting advantages of the method according to the invention in a manner that can be seen in a corresponding manner so that reference is made to the above statements to avoid repetition.

According to a third aspect of the present invention, a computer program product is proposed which has instructions which, when executed on a programmable processor (eg, an evaluation unit of a device according to the invention), cause the processor to carry out the steps of the above-mentioned method. The computer program product can, for example, as a CD, DVD, Blu-ray Disc, flash memory, ROM / RAM o. Ä. be designed.

According to a fourth aspect of the present invention, a signal sequence is proposed which represents instructions which, when executed on a programmable processor (eg, an evaluation unit of a device according to the invention), cause the processor to perform the steps of a method mentioned above. The signal sequence also protects, in case of the case, the logic according to the invention in data form, in that the memory means required for the reproduction of the instructions are positioned outside the scope of the appended claims.

According to a fifth aspect of the present invention, there is proposed a traveling means (e.g., a car, vans, trucks, motorcycles, air and / or water vehicles) having a device according to the second aspect of the invention. Also with regard to the means of locomotion, therefore, the features, feature combinations and advantages according to the above statements.

• 1a eine schematische Darstellung eines Ausführungsbeispiels eines erfindungsgemäßen Fortbewegungsmittels mit einem Ausführungsbeispiel einer erfindungsgemäßen Vorrichtung;
• 1b eine Detailansicht auf Komponenten eines Ausführungsbeispiels einer erfindungsgemäßen Vorrichtung;
• 2 ein Blockdiagramm zur Veranschaulichung der Funktionsweise eines erfindungsgemäßen Systems für die Verwendung aggregierter Flottendaten;
• 3 ein qualitatives Diagramm veranschaulichend eine mögliche Kategorisierung unterschiedlicher Musikstile nach Geschwindigkeit und Melodie;
• 4 eine schematische Übersicht über Elemente eines Ausführungsbeispiels einer erfindungsgemäßen Vorrichtung;
• 5 eine schematische Darstellung von Zusammenhängen bei der erfindungsgemäßen Signalverarbeitung;
• 6 eine schematische Darstellung zur Missionsplanung im Rahmen eines erfindungsgemäßen Verfahrens;
• 7 eine beispielhafte Übersicht über die Vorgänge und Elemente einer erweiterten Songanalyse;
• 8 eine Übersicht veranschaulichend Zusammenhänge bei der erfindungsgemäßen Gestaltung von Übergängen; und
• 9 eine schematische Darstellung einer manuellen Einflussnahme eines Anwenders auf die erfindungsgemäße Signalverarbeitung.

Further advantages, features and embodiments of the present invention will become apparent from the following statements in conjunction with the accompanying drawing figures. Show it:

• 1a a schematic representation of an embodiment of a means of transport according to the invention with an embodiment of a device according to the invention;
• 1b a detailed view of components of an embodiment of a device according to the invention;
• 2 a block diagram illustrating the operation of a system according to the invention for the use of aggregated fleet data;
• 3 a qualitative diagram illustrating a possible categorization of different music styles by speed and melody;
• 4 a schematic overview of elements of an embodiment of a device according to the invention;
• 5 a schematic representation of relationships in the signal processing according to the invention;
• 6 a schematic representation of the mission planning in the context of a method according to the invention;
• 7 an exemplary overview of the processes and elements of an extended song analysis;
• 8th an overview illustrative of relationships in the design of transitions according to the invention; and
• 9 a schematic representation of a manual influence of a user on the signal processing according to the invention.

• 1a zeigt eine schematische Darstellung eines Ausführungsbeispiels eines erfindungsgemäßen Pkws 10 als Fortbewegungsmittel, auf dessen Fahrersitz ein Anwender Platz genommen hat. Der Anwender führt ein Smartphone 20 mit sich, welches Audio-Dateien in einem (nicht dargestellten) Datenspeicher vorhält, welche über Lautsprecher 1 und eine Auswerteeinheit 2 in erfindungsgemäßer Weise gemixt wiedergegeben werden sollen. Ein Touchscreen 3 dient der Entgegennahme von Anwendereingaben. Bspw. kann der Anwender mittels des Touchscreens 3 Bewertungen für wiedergegebene Übergänge abgeben. Zudem kann der Anwender eine Wiedergabe starten, im Ansprechen auf welche das erfindungsgemäße Verfahren ausgeführt wird. Insbesondere kann der Anwender Einfluss auf Parameter, in Betracht zu ziehende Audio-Dateien und Gewichtungen von Parametern für die Gestaltung eines erfindungsgemäßen Übergangs vornehmen.
• 1b zeigt Komponenten eines Ausführungsbeispiels einer erfindungsgemäßen Vorrichtung. In einem Datenspeicher 5 sind Audio-Dateien 6a, 6b, 6c enthalten, welche von einem Anwender zur Wiedergabe vorgesehen sind. Die Audio-Dateien 6a, 6b, 6c können die gesamte Mediathek des Anwenders oder einen Teil (z. B. eine Wiedergabeliste) darstellen. Die Audio-Dateien 6a, 6b, 6c weisen jeweils einen Header 7 auf, in welchem Metadaten (z. B. Genre, Titel, Interpret, Album, Erscheinungsjahr etc.) enthalten sind. Zusätzlich weisen die Audio-Dateien 6a, 6b, 6c Audio-Material 8 auf, welches die eigentliche Musikwiedergabeinformation enthält. Eine erfindungsgemäße Segmentierung des Audio-Materials 8 hat Segmente vordefinierter Kategorien (Intro 12, Strophe 13, Refrain/Chorus 14 und Outro 15) identifiziert. Dies kann im Ansprechen auf das Starten einer Wiedergabe und/oder beim Anlegen der Dateien im Datenspeicher 5 erfolgen. Erfindungsgemäß werden die Segmente bei der Gestaltung von Übergängen durch ein elektronisches Steuergerät 2 als Auswerteeinheit zur Wiedergabe mittels des Lautsprechers 1 verwendet. Im Ergebnis weist ein Audiosignal 9 einen Schalldruckpegel L_p über der Zeit t auf, welcher in einem Zeitbereich durch die Audio-Datei 6a, in einem Übergangsbereich 11 durch die Audio-Dateien 6a und 6b und in einem dritten Zeitbereich durch die zweite Audio-Datei 6b definiert wird. Der Übergangsbereich 11 ist insbesondere nicht durch eine vordefinierte Zeitdauer, sondern durch eine Dauer zweier solcher Segmente der Musikdateien 6a, 6b definiert, welche sich als für die Gestaltung des Übergangs geeignet herausgestellt haben. Insbesondere kann der Übergangsbereich 11 eine gesamte Dauer eines ausgewählten Segmentes der ersten Audio-Datei 6a, eines ausgewählten Segmentes der zweiten Audio-Datei 6b und/oder eines jeweiligen ausgewählten Segmentes der Audio-Dateien 6a, 6b aufweisen.
• 2 zeigt eine schematische Darstellung eines Verfahrens zur Aggregation von Flottendaten, mittels welcher anwenderübergreifende Erkenntnisse und Vorlieben zur Gestaltung eines erfindungsgemäßen Übergangs herangezogen werden können. ClientSysteme 18 sammeln das Anwenderverhalten 19 bezüglich einer Vielzahl von Musikplayern, das Fahrverhalten 21 einer Vielzahl von Fahrzeugen umfassend die Musikplayer 19 und zugeordnete umweltbedingte Kontexte 22 für eine Vielzahl von Anwender 27. Die persönlichen Präferenzen 23 der Anwender 27 werden an ein Backend-System 17 in Form eines Servers gesendet. Der Server findet Gemeinsamkeiten in den persönlichen Präferenzen 23 der Anwender 27 und ermittelt allgemeine Präferenzen 25 aus diesen. Schließlich erstellt das Backend-System 17 ein allgemeines Regelwerk 26 zur Gestaltung erfindungsgemäßer ansprechender Übergänge. Diese können einem vorliegend betrachteten Clienten-System 16 eines betrachteten Anwenders zur Verfügung gestellt werden, welcher ebenfalls über ein Benutzungsverhalten 19 des Musikplayers, ein Fahrverhalten 21 und einen umweltbedingten Kontext 22 die Bestimmung eines Übergangs in erfindungsgemäßer Weise optimieren lassen möchte. Die persönlichen Präferenzen 23 münden in einem persönlichen Regelwerk 24. Die persönlichen Präferenzen 23 werden ebenfalls mit den allgemeinen Präferenzen 25 im Backend-System 17 zur Erzeugung des allgemeinen Regelwerkes 26 verwendet. Das allgemeine Regelwerk 26 wird schließlich bei der Erstellung des persönlichen Regelwerkes 24 verwendet.
• 3 zeigt eine Clusterung bzw. Kategorisierung unterschiedlicher Musikstilrichtungen in einem Diagramm, in welchem auf der Abszisse die Tendenz der Kategorien hinsichtlich eines melodischen Charakters (Melodien enthaltend) und auf dessen Ordinate die Geschwindigkeit (Beats per minute, BPM) aufgetragen sind.
• 4 zeigt eine alternative Darstellung der Zusammenhänge zwischen einer Vielzahl Clienten-Systeme 18, welche eine Vielzahl Smartphones 20 umfassen, einem Backend-System 17, welches einen Datenspeicher 5 aufweist und zur Erzeugung einer Wiedergabereihung 28 eingerichtet ist, sowie einem betrachteten Clienten-System 16, welches aus einem Pkw 10 und einem informationstechnisch mit dem Pkw 10 gekoppelten Smartphone 20 besteht. Das Smartphone 20 des Clienten-Systems 16 dient als Datenspeicher für Audio-Dateien und weist eine Auswerteeinheit (nicht dargestellt) auf. Sensorisch durch den Pkw 10 ermittelte Fahrzeugsituationen und -zustände werden drahtlos an das Smartphone 20 gesendet, welches die Schritte eines erfindungsgemäßen Verfahrens zur Gestaltung eines Übergangs ausführt.
• 5 veranschaulicht ein Ausführungsbeispiel von Schritten bei der Übergangsgestaltung in einem Clienten-System mit automatischer DJ-Software. Aus einer Online-Datenbank als Datenspeicher 5, einem persönlichen Regelwerk 24 und einem Benutzungsverhalten 19 wird auf Basis einer Reihung 28 umfassend eine Vielzahl Musiktitel mit Metainformationen eine Missionsplanung 29 erstellt. Die Missionsplanung 29 basiert auf einer erweiterten Songanalyse 37, in welcher ein Audiogramm 30 jeweiliger Audio-Dateien in Segmente aufgeteilt und segmentweise analysiert wurden. Anhand des persönlichen Regelwerkes 24 sowie anhand des Nutzungsverhaltens 19 werden in Abhängigkeit der Audiogramme 32, 33 zweier aufeinanderfolgend wiederzugebender Audio-Dateien Übergangszeitpunkte 31 gefunden. Anschließend werden erneut in Abhängigkeit des persönlichen Regelwerkes 24 und des Nutzungsverhaltens 19 Übergänge 34 konstruiert, wobei - wie im Ausschnitt 35 gezeigt-Höhen, Mitten, Bässe und Verstärkung über der Zeit im Übergangsbereich für die beteiligten Audio-Dateien definiert werden. Anschließend wird bei der Musikwiedergabe der Übergang 36 durchgeführt. Aus den Übergangszeiträumen der Lieder A und B wird ein Audio-Segment „Audio C“ ermittelt. Die Ausgabe des Main-Audio-Buffers lautet: „Wiedergabelied A bis Übergangszeitraum“ - „Audio C“- „Restlied B“.
• 6 zeigt eine schematische Darstellung eines Ausführungsbeispiels einer Missionsplanung 29 zur Wiedergabe einer Vielzahl Audio-Dateien gemäß einem Ausführungsbeispiel eines erfindungsgemäßen Verfahrens. Mittels einer Online-Datenbank in Form eines Datenspeichers 5, einer grundsätzlichen Songanalyse 38 und einer erweiterten Songanalyse 39 werden Metainformationen 40 generiert, welche in den Musikdateien Song A bis Song H bislang nicht enthalten waren. Durch eine Güteberechnung 41 wird eine Übergangsgüte und eine semantische Güte jeweiliger Übergänge ermittelt und anschließend aufgrund eines Anwenderverhaltens 19 und eines persönlichen Regelwerkes 24 eine Clusterung 42 ausgeführt. Hierbei werden die Audio-Dateien in Lied-Cluster A, B und C kategorisiert, welche sich durch Gemeinsamkeiten hinsichtlich der Metainformationen auszeichnen. Es entsteht ein erstes Cluster 43 und eine Clusterreihenfolge 44. Innerhalb der Lied-Cluster A, B, C werden interne Reihenfolgen 45 der einzelnen Audio-Dateien festgelegt. Hieraus ergibt sich eine Reihung (Englisch: „Queue“) 46, welche durch eine manuelle Modifikation 47 und eine Änderung 48 aufgrund eines Anwenderverhaltens zu einer modifizierten Reihung 28 verarbeitet wird.
• 7 zeigt eine schematische Darstellung einer erweiterten Songanalyse, in welcher eine Audio-Datei 6a einer Makrosegmentierung 49 unterzogen wird. Die Makrosegmente werden anschließend einer Mikrosegmentierung 50 und einer Genre-Klassifikation 51 unterzogen. Die Chorus-Segmente 52 sind hierbei fett dargestellt. Anschließend erfolgen eine semantische Makrosegmentierung 53, wobei die Genre-Klassifikation 51 Einfluss auf die semantische Mikrosegmentierung 50 nimmt. Durch eine inhaltliche Betrachtung kann eine Segmentinterpretation 52 für die Genre-Klassifikation 51 verwendet werden. Zwei Phasen (Phase 1, Phase 2) werden definiert und Übergangsfaktoren 54 bestimmen anschließend die Audio-Wiedergabe 9.
• 8 zeigt eine schematische Darstellung des Prozessblocks zur Bestimmung von Übergangszeitpunkten, in welcher die Segmente 58 der Audio-Datei 6a, sowie die Segmente 59 der Audio-Datei 6b vorliegen. In mehreren Prozessschritten wird anhand der Segmentgrenzen die Auswahl an Übergangszeitpunkten für jede Audio-Datei auf die Anzahl 1 eingeschränkt. Übergangszeitpunkte 60 unter Einfluss der Anwenderaktivität (eines Anwenderaktivitätslevels oder einer Anwenderaktivitätskategorie oder einer Anwenderaktivitätsstufe) zeigen für Audio-Datei 6a die potenziellen Übergangszeitpunkte, nach ausschließlichem Einfluss der Anwenderaktivität. Übergangszeitpunkte 61 unter Einfluss des persönlichen Regelwerks zeigt für Audio-Datei 6a die daraus gewählte Menge an Übergangszeitpunkte unter Einfluss des persönlichen Regelwerks, sowie inhaltlichem Parameter. Der final gewählte einzelne Übergangszeitpunkt für Audio-Datei 6a, welcher anhand eines technischen Parameters festgelegt wird, ist im final gewählten Übergangszeitpunkt unter Einfluss technischer Parameter dargestellt. Dieser stellt für einen Übergang aus Audio-Datei 6a in Audio-Datei 6b den Klimax des Übergangszeitraumes dar.
• 9 zeigt eine schematische Skizze zur Anwenderaktivitätsberechnung, wie sie zur Anwendung in einem erfindungsgemäßen Verfahren verwendet werden kann. Das Benutzungsverhalten 19 eines Musikplayers, das Fahrverhalten 21 eines Fortbewegungsmittels sowie ein umweltbedingter Kontext 22 führen zur Definition einer Position eines Reglers 55 zwischen den Eigenschaften „aktiv“ und „wenig aktiv“. Über eine Anwenderschnittstelle 57 kann der Anwender eine manuelle Einstellung 56 vornehmen und somit Einfluss auf die anwenderaktivitätsbezogene Gestaltung von Übergängen nehmen.

Embodiments of the invention:

• 1a shows a schematic representation of an embodiment of a car according to the invention 10 as a means of transportation on the driver's seat a user has taken place. The user is holding a smartphone 20 with him, which maintains audio files in a (not shown) data storage, which via speakers 1 and an evaluation unit 2 to be reproduced mixed in accordance with the invention. A touch screen 3 serves the receipt of user input. For example. the user can use the touchscreen 3 Give ratings for rendered transitions. In addition, the user can start a playback in response to which the inventive method is performed. In particular, the user can influence parameters, audio files to be considered and weighting parameters for the design of a transition according to the invention.
• 1b shows components of an embodiment of a device according to the invention. In a data store 5 are audio files 6a . 6b . 6c contained, which are provided by a user for playback. The audio files 6a . 6b . 6c can represent the user's entire library or a part (such as a playlist). The audio files 6a . 6b . 6c each have a header 7 which contains metadata (eg genre, title, artist, album, year of publication, etc.). In addition, the audio files point 6a , 6b, 6c audio material 8th on which contains the actual music playback information. A segmentation of the audio material according to the invention 8th has segments of predefined categories (Intro 12 , Stanza 13 , Chorus / refrain 14 and outro 15 ) identified. This may be in response to starting a replay and / or creating the files in the data repository 5 respectively. According to the invention, the segments are used in the design of transitions by an electronic control unit 2 as an evaluation unit for playback by means of the loudspeaker 1 used. The result is an audio signal 9 a sound pressure level L _p over the time t, which in a time domain through the audio file 6a in a transition area 11 through the audio files 6a and 6b and in a third time range through the second audio file 6b is defined. The transition area 11 is in particular not by a predefined period of time, but by a duration of two such segments of the music files 6a . 6b defined as having been found suitable for the design of the transition. In particular, the transition area 11 a total duration of a selected segment of the first audio file 6a , a selected segment of the second audio file 6b and / or a respective selected segment of the audio files 6a, 6b.
• 2 shows a schematic representation of a method for aggregation of fleet data, by means of which cross-user insights and preferences for the design of a transition according to the invention can be used. Client systems 18 collect the user behavior 19 regarding a variety of music players, the driving behavior 21 a variety of vehicles including the music players 19 and associated environmental contexts 22 for a variety of users 27 , The personal preferences 23 the user 27 be sent to a backend system 17 sent in the form of a server. The server finds similarities in personal preferences 23 the user 27 and determines general preferences 25 from these. Finally, the backend system creates 17 a general set of rules 26 for the design of appealing transitions according to the invention. These can be a client system considered here 16 of a considered user, which also has a usage behavior 19 of the music player, a driving behavior 21 and an environmental context 22 want to optimize the determination of a transition in accordance with the invention. The personal preferences 23 lead to a personal set of rules 24 , The personal preferences 23 are also using the general preferences 25 in the backend system 17 for the generation of the general rules 26 used. The general rules 26 Finally, when creating the personal rules 24 used.
• 3 shows a clustering or categorization of different music style directions in a diagram in which the tendency of the categories with respect to a melodic character (containing melodies) is plotted on the abscissa and the velocity (beats per minute, BPM) are plotted on the ordinate.
• 4 shows an alternative representation of the relationships between a variety of client systems 18 a variety of smartphones 20 include, a backend system 17 which is a data store 5 and for generating a replay 28 is set up, as well as a considered client system 16 , which from a passenger car 10 and an information technology with the car 10 coupled smartphone 20 consists. The smartphone 20 of the client system 16 serves as a data store for audio files and has an evaluation unit (not shown). Sensory by the car 10 Detected vehicle situations and conditions are wirelessly to the smartphone 20 which carries out the steps of a method according to the invention for designing a transition.
• 5 illustrates an embodiment of steps in transition design in a client system with automatic DJ software. From an online database as a data store 5 , a personal set of rules 24 and a usage behavior 19 is based on a ranking 28 includes a variety of music tracks with meta information mission planning 29 created. The mission planning 29 based on an extended song analysis 37 in which an audiogram 30 audio files were segmented and analyzed segment by segment. Based on the personal rules 24 as well as the usage behavior 19 become dependent on the audiograms 32 . 33 two consecutive audio files to be transited 31 found. Subsequently, again depending on the personal rules 24 and the usage behavior 19 transitions 34 constructed, with - as in the neckline 35 Shown are treble, mids, bass, and gain over time in the transition area for the audio files involved. Then, when the music is playing, the transition 36 carried out. From the transitional periods of the songs A and B, an audio segment "Audio C" is determined. The output of the main audio buffer is: "Play Song A to Transition Period" - "Audio C" - "Remaining Song B".
• 6 shows a schematic representation of an embodiment of a mission planning 29 for reproducing a plurality of audio files according to an embodiment of a method according to the invention. By means of an online database in the form of a data store 5 , a fundamental song analysis 38 and an extended song analysis 39 become meta-information 40 generated, which were not included in the music files Song A to Song H so far. By a quality calculation 41 a transition quality and a semantic quality of respective transitions are determined and subsequently based on a user behavior 19 and a personal set of rules 24 a clustering 42 executed. Here, the audio files are categorized into song clusters A, B and C, which are characterized by similarities in meta-information. The result is a first cluster 43 and a cluster order 44 , Within the song clusters A, B, C become internal orders 45 of the individual audio files. This results in a ranking (English: "Queue") 46, which by a manual modification 47 and a change 48 due to user behavior to a modified ranking 28 is processed.
• 7 shows a schematic representation of an extended song analysis in which an audio file 6a a macrosegmentation 49 is subjected. The macro segments are then micro-segmented 50 and a genre classification 51 subjected. The chorus segments 52 are shown in bold here. Subsequently, a semantic macro-segmentation takes place 53 , where the genre classification 51 Influence on semantic microsegmentation 50 takes. By a contentwise consideration, a segment interpretation 52 for the genre classification 51 be used. Two phases (phase 1 , Phase 2 ) are defined and transition factors 54 then determine the audio playback 9 ,
• 8th shows a schematic representation of the process block for determining transition times, in which the segments 58 the audio file 6a , as well as the segments 59 the audio file 6b available. In several process steps, the selection of transition times for each audio file is based on the segment boundaries on the number 1 limited. Transition times 60 under the influence of the user activity (a user activity level or a user activity category or a user activity level) point for audio file 6a the potential transitional periods, after the sole influence of user activity. Transition times 61 under the influence of personal rules shows for audio file 6a the selected amount of transitional periods under the influence of personal rules and content parameters. The final selected single transition time for audio file 6a , which is determined on the basis of a technical parameter, is shown under the influence of technical parameters in the finally selected transition time. This provides for a transition from audio file 6a in audio file 6b the climax of the transitional period
• 9 shows a schematic sketch for user activity calculation, as it can be used for application in a method according to the invention. The usage behavior 19 a music player, the driving behavior 21 a means of transport and an environmental context 22 lead to the definition of a position of a controller 55 between the properties "active" and "not very active". Via a user interface 57 the user can do a manual adjustment 56 and thus influence the user-activity-related design of transitions.

In addition to the above description, further possible embodiments are discussed below by way of example and without any restrictive intention with regard to the inventive concept, the features of which can be combined in particular with the above objects.

The present invention relates to novel software for automatically adjusting the order of a given set of songs, selecting a transition time, as well as a transitional shape between two digital songs, and blending the songs using a flowing, harmonic, synchronized, and semantically appropriate transition.

These functions are adapted to the real Auflegeverhalten of contemporary, human radio or club DJs, as well as the situation-dependent needs of the user to give him an optimal music experience. The adaptation is done in addition to technical and semantic criteria based on the stored preferences of the user, an analyzed user activity, as well as analyzed social trends from aggregated fleet data.

The invention described herein relates to an implementation within a music player with access to digital audio files. This music player can be implemented within any technical device.

The invention described herein relates to an implementation in the automobile and the advantages resulting therefrom. Nevertheless, the software works outside of the car, with only the functionality is limited in some places.

The demands on the music experience have risen ever more today. This can be recognized by the fact that nowadays there is a multitude of services which are automatically suitable for the user Select songs. DJing in particular has become increasingly popular at the moment, as evidenced by the fact that festivals predominantly play DJs, contemporary chart productions are predominantly made by DJs and more and more platforms for the consumption of so-called DJ mixes exist. The consumption of a music experience, which is constructed by a human DJ and adapted to the needs of the consumer, many music consumers prefer to the conventional consumption of radio streams or the seamless playback of individual songs. The music consumer thus prefers to consume a music experience made by a personal DJ, in which the needs of the user are taken into account.

The present invention relates to software that simulates a personal DJ. Here, the arrangement of a given set of songs, the selection of a transition time, the choice of a transitional form and the implementation of this adapted transitional form as a transition between two songs are defined as activities of a personal DJs. These are carried out autonomously by the software.

Especially the early fading of a song into a follow-up song is very popular nowadays and can lead to an increase of the musical experience.

The functionality of selecting a set of suitable songs adapted to the needs of the user already exists and enhances the quality of the music experience created with this invention.

The song structure of modern, popular entertainment music is more and more in line with general trends and therefore becomes more and more standardized within a music genre. Therefore, the technical possibility is given, taking into account the trends of the song structure of the genre, automatically subdivide a plurality of songs into correct segments and to examine them for the meaning of the song structure.

The structure of today's songs is also due to the clear song structure, clear rhythm elements and a mostly constant speed very well suited for the transition by means of flowing transitions of different, creative transitional forms.

As the processors built into mobile devices become faster and faster and audio signal processing procedures become more efficient, this allows the software to function in a timely manner.

Access to a variety of music tracks is becoming easier with digital audio formats and a variety of music streaming services. Due to the easy access to a high amount of music, more and more suitable songs for the user can be selected, between which also the transition quality is almost maximum.

The technology of adapting the order of a set of songs to achieve a variety of high transient quality transitions already exists in the prior art, but does not take into account semantic criteria, situation-dependent personal preferences of the user, as well as societal trends.

The technology of correctly segmenting songs and creating smooth, harmonious and synchronized transitions already exists in the prior art. However, segmentation has so far largely been done without a semantically correct designation of the segments. The choice of a transitional time, as well as a transitional form takes place so far without consideration of semantic criteria or the situation-dependent preferences of the user.

Software is known from the prior art, which allows two songs temposynchronisiert with a volume crossover or cut merge into each other. However, the transition always takes place in a predefined period of time. This is for example eight beats. Only the time eight beats before the end of the song considered time is set as an out point, as it is with the in-point, which is set eight beats after the time considered the beginning of the song. One software that has implemented the described method is Serato Pyro. The content of this song is not analyzed for semantic segments, which means that no further potential In Points and Out Points are determined where a transition between two digital songs can take place. The choice of transitional periods and transitional forms is therefore unchanged.

In the software Serato Pyro an automatic arrangement of a lot of songs is realized, which is based on meta information such as speed, pitch and other parameters for the similarity determination of two digital audio signals. Using these parameters, the high-similarity songs are more likely to be played directly one after the other. However, the alignment of the songs is not adapted to the situation-dependent, personal needs of the user, which is a significant difference to the so-called mission planning in this invention.

Also known from the prior art is the Pacemaker software which also automatically synchronizes transitions between two songs. The shape of the transitions is limited to various forms of volume crossover, transitions with little influence on the different frequency bands by an equalizer, as well as a cut and leaves further creative transitional forms. The software determines several potential transitional periods depending on the songs being transposed. The selection of transitional periods, as well as transitional forms, however, does not take into account semantic criteria or situation-dependent, personal needs of the user, but only taking into account the maximization of the transitional quality.

By the term "synchronized" is meant that two digital songs are matched in their speed, and the individual beats, which are characterized by the pulse-like beats of the bass drum in the low-frequency range, are time exactly superimposed. This time-exact overlay of the songs is maintained in the synchronous playback of both songs, and can even be corrected with minimal change in the speed within the song. The technical term for this is the automatic "Beat Matching". The technology described is state of the art and is required for the invention.

By the term "transitional quality" is meant the quality of the transition, which increases with more positive perception by a human listener. The transition should therefore be considered as "flowing".

• - situationsabhängiges Verhalten des Users (Useraktivität) unter Berücksichtigung:
  • - des Benutzungsverhaltens des Musikplayers,
  • - des Fahrverhaltens sowie
  • - des umweltbedingten Kontextes, und
• - persönliche Präferenzen des Users (persönliches Regelwerk).

The adaptation to situation-dependent needs means an adjustment taking into account the following criteria:

• - Situation-dependent behavior of the user (user activity) taking into account:
  • - the usage behavior of the music player,
  • - the driving behavior as well
  • - the environmental context, and
• - personal preferences of the user (personal rules).

• - Allgemeine Präferenzen aller User der Software (allgemeines Regelwerk).

By adapting to social trends from aggregated fleet data, an adjustment is made taking into account the following criteria:

• - General preferences of all users of the software (general rules).

A song is a digital audio signal including meta information.

A queue represents a list of songs in which the playing order of the songs can be determined.

The software can be implemented within a music player. The software needs a lot of songs to be played back. These are stored in a queue while using the software. It is irrelevant whether the songs are added to the queue manually by the user or by an external software automatically.

• - Das Hinzufügen und Löschen von Liedern in einer Queue
• - Die Abspielreihenfolge der Queue verändern
• - Das Vor- und Zurückspulens innerhalb von Liedern
• - Den Wechseln zum in der Queue nächsten Lied oder einem beliebigen Lied
• - Das Pausieren und erneute Abspielen eines Liedes

This software assumes an implementation within a music player. This has the following functionality:

• - Adding and deleting songs in a queue
• - Change the playing order of the queue
• - Fast-forwarding and rewinding within songs
• - Changing to the next song in the queue or any song
• - Pause and replay a song

Although it is not necessary for the software described here, it describes the implementation of the software within a music player which provides an automatic DJ function. Without this functionality, only the functionality of the invention described herein would be limited. Such a music player with automatic DJ function is hereinafter referred to as automatic DJ software.

• - Unterbrechen des automatisch eingeleiteten Überganges durch die Software
• - Manuelles Einleiten eines automatischen Überganges zwischen dem aktuellen Lied und dem Folgelied in der gegebenen Abspielreihenfolge
• - Manuelles Einleiten eines Blitzübergangs zwischen dem aktuellen Lied und dem Folgelied in der gegebenen Abspielreihenfolge
• - Die Möglichkeit den Mix Modus zu verändern

The automatic DJ software also offers the user the following manual intervention:

• - Interrupt the automatically initiated transition through the software
• - Manually initiating an automatic transition between the current song and the following song in the given play order
• - Manually initiating a flash transition between the current song and the following song in the given play order
• - The possibility to change the mix mode

• - Die Anordnung der Lieder in einer gegebenen Queue
• - Den abgespielten Audiostream

The software can influence the following functions within the music player:

• - The arrangement of songs in a given queue
• - The played audio stream

The overall system of the automatic DJ software consists of a client software that can be implemented within a music player, as well as a backend system with an online database to which the entirety of all client systems has access.

The software contains a general set of rules on the backend system, which can be modified automatically under the influence of aggregated fleet data as well as manually under the influence of an adaptation by the provider of the software. The modification is done with the purpose of adapting the functionality of the software to societal trends in music consumption behavior. The general rules can be loaded on the client software to influence the personal rules that exist there.

The software can be used by different users, whereby each user must create his own user profile and must first select this before each use unit. Each user profile is assigned personal preferences, a personal set of rules, as well as stored user activity information.

A usage unit refers to the period of use of the software from rebooting the software to terminating the software.

The software saves the frequency during which each song is played by the user of the current user profile. This value is summed with past play rates. The songs can be sorted in terms of the frequency of the general plays. The average value of the frequencies of the plays of all songs that have been played more than twice is calculated. All songs that have a frequency of plays above this value are marked as favorite songs.

• - Die dynamische Anordnung der Lieder in einer vorgegebenen Queue (Missionsplanung)
• - Analyse des Liedes zur Erhebung von Metainformationen, außerhalb der grundsätzlichen Parameter. (Erweiterte Songanalyse)
• - Bestimmung von potenziellen In-Points und Out-Points innerhalb eines Liedes / Auswahl eines Out-Points im aktuell spielenden Lied, sowie einem In Point in dem nächsten Lied in der Queue. (Übergangszeitpunkt auswählen)
• - Wahl einer abstrakten Übergangsform / Anpassen der abstrakten Übergangsform an die aktuelle Übergangssituation. (Übergang konstruieren)
• - Automatische Durchführung des Überganges zwischen zwei Liedern (Übergang Durchführen)

The functionalities of the software are structured as follows:

• - The dynamic arrangement of songs in a given queue (mission planning)
• - Analysis of the song for the collection of meta-information, outside the fundamental parameters. (Advanced song analysis)
• - Determining potential In points and Out points within a song / Selecting an Out point in the currently playing song, as well as an In Point in the next song in the queue. (Select transition date)
• - Choice of an abstract transitional form / Adjusting the abstract transitional shape to the current transitional situation. (Construct transition)
• - Automatic execution of the transition between two songs (Transition Perform)

Mission Planning:

Mission planning is a decision engine that orders a given set of songs in the queue in a new order that matches the situation-dependent needs of the user.

• - Metainformationen
• - Güteberechnung
• - Clustering
• - Erstes Cluster
• - Clusterreihenfolge
• - Interne Reihenfolge

It consists of six phases:

• - Meta information
• - Quality calculation
• - Clustering
• - First cluster
• - Cluster order
• - Internal order

In addition, mission planning is a function for adjusting the queue during manual modification of the queue, as well as a function for automatic, dynamic adjustment of the queue when user activity changes.

For the functionality of the software basic meta-information of a song is needed.

• - Geschwindigkeit
• - Tonart (nur Grundton und Dur/Moll-Tonalität)
• - hörbarer Anfangs/Endzeitpunkt
• - Dauer
• - Titel
• - Interpret
• - Genre
• - Erscheinungsjahr
• - Album
• - Remixartist
• - Produzent (oder Komponist)
• - Energie des Liedes
• - Stimmung des Liedes

These sat down for each song as follows:

• - speed
• - key (only root and major / minor tonality)
• - audible start / end time
• - Duration
• - Title
• - Artist
• - genre
• - Publishing year
• - Album
• - Remix artist
• - producer (or composer)
• - Energy of the song
• - mood of the song

These parameters are taken from the metadata of the audio file, from the online database or a basic song analysis. The completeness of the last six pieces of information is not optional.

If the genre does not have an entry within the metadata of the audio file, or the online database, it can be filled using the advanced song analysis information. If the genre already exists, the advanced song analysis information will be in a new variable genre 2 additionally filed.

• - Geschwindigkeit
• - Tonart (nur Grundton und Dur/Moll-Tonalität)
• - hörbarer Anfangs/Endzeitpunkt

With a fundamental song analysis analyzes are meant to obtain the following information:

• - speed
• - key (only root and major / minor tonality)
• - audible start / end time

These are standard technologies that are considered prerequisite for this invention.

Methods are known in the art for operationalizing the mood and energy of a song. These parameters may already be stored in the meta-information of each song or determined by a separate method within the basic song analysis.

meta:

At the beginning of using the software, all basic meta-information of the songs must be loaded into the queue. For this, the entirety of all songs in the queue must first be subjected to a fundamental song analysis, if the information of the basic song analysis has not already been loaded locally or is stored in the database for the corresponding title. The title to be analyzed is identified with regard to the title name, artist name and duration of the song. If the information in the online database exists within a record with the same key, the information is loaded from the online database.

If there is no match in the database, the basic song analysis is performed on the title. The information from the basic song analysis is stored in the online database for a record with title name, artist name, and duration of the song as a key.

Grade calculation:

• - Übergangsgüte
• - Semantische Güte

The mission planning calculates the general quality for the given queue. This consists of the following parameters:

• - Transient quality
• - Semantic goodness

Transition goodness:

The transient quality is the measure by which the quality of a potential transition between two digital audio signals can be measured. The quality is composed of the properties seamless, harmonious and tempo-like. A high transient quality represents a potential, optimal transition and would provide the listener with an exciting musical experience when making a transition between the songs.

A gapless transition means a transition that does not contain a period of time between two independent digital audio signals without digital audio material.

By harmonic transition is meant a transition that represents a transition between two independent digital audio signals, with the harmonies of the songs being appropriate. By this it is meant that the key of the song is appropriate with regard to the circle of fifths. The transition only has the highest quality in terms of harmony if the following song in relation to the currently playing song in the same key, the parallel minor / major key, the next higher, shifted by a fifth key in the same tone, or the next lower, by a fifth shifted key in the same Tongeschlecht, is.

With a tempo-like transition, it is meant that the two songs within the transition can be matched in their speed without the song being perceived as too much modified. With increasing similarity of the speed between the songs, the quality of transition increases linearly. Speed differences above 20 BPM tend to be inappropriate for a transition, as the speed of two songs must be changed by at least 10 BPM, which makes a song noticeably altered and inedible. Therefore, speed differences exceeding the value of 20 BPM result in a nonlinear reduction in transient performance. The difference is always from the original one Speeds of the songs in the meta-information calculated and not from the current speed of the already modified audio signal. However, there are abstract transitional forms that can handle a higher speed difference. If these abstract transitional forms are highly prioritized in the user's personal rules, exceptions lead to a changed calculation of the transition quality between two songs of specific genres.

Semantic quality:

For the calculation of the semantic quality, the basic meta-information genre, year of publication and producer are considered. The parameters release year and producer are not necessary, but improve the music experience.

The semantic quality is linearly dependent on the distance between the genres of the songs. The removal of two genres comes from the personal set of rules. From a graph one would determine the Euclidean distance of a vector between two genres as the distance of two genres.

The semantic quality is also linearly dependent on the distance between the release years of the songs. The removal of two years of publication also comes from the personal rules. The years of publication are classified so that a song with the release year 2007 and a song with the release year 2009 would have a distance of 0. A smaller distance in the list means a higher semantic quality.

The semantic quality increases by a defined factor for a potential transition between two songs, which have in their meta-information a match of the registered producers, but no agreement with regard to the interpreters.

In the calculation of the semantic quality also the similarities of two songs in terms of their energy and mood flow.

The semantic quality can be influenced by other parameters that describe a similarity between two songs.

Mission planning now calculates a general goodness of transient quality and semantic quality for each potential song tuple within the queue.

clustering:

Songs from the same genre class are collected in genre clusters.

The classification of genres in genre classes is stored in the personal rules.

New cluster templates are created, into which later songs of a genre cluster are added. The size of the cluster templates represents the number of songs that will be added to the cluster later.

The size of the cluster templates is based on the parameter of the user activity. If the user activity is high, this means that the user is very active and accordingly demands an exciting, especially dynamic music experience. For this reason, a faster change takes place between different clusters. A lower user activity leads to a slower change between clusters, since the user wants a relaxed music experience.

The size of the cluster templates is not set absolutely based on user activity. Only a tolerance space is defined in which the size of the cluster templates varies randomly. The expansion of the tolerance space increases linearly with an increase in user activity. The tolerance space, however, widens unevenly in the direction of smaller cluster sizes.

• - Ähnlichkeit der Genredistanz zwischen zwei Liedern
• - Ähnlichkeit der allgemeinen Güte zwischen zwei Liedern
• - Ähnlichkeit der Distanz der Erscheinungsjahre zwischen zwei Liedern
• - Ähnlichkeit der Stimmung zwischen zwei Liedern
• - Ähnlichkeit der Energie zwischen zwei Liedern

The filling of the cluster template with songs from a genre cluster is done according to the following criteria in mentioned prioritization:

• - Similarity of the genredistanz between two songs
• - Similarity of the general goodness between two songs
• - Similarity of the distance of the year of publication between two songs
• - Similarity of mood between two songs
• - Similarity of energy between two songs

For example, two songs of high general goodness within a genre class are thus more likely to be put together in a common cluster than songs of lesser general goodness. The filled cluster templates are referred to in the following only as clusters.

First cluster:

• - Anzahl der Lieblingslieder innerhalb eines Clusters
• - Durchschnittsgeschwindigkeit der Lieder eines Clusters
• - Durchschnittliche Stimmung der Lieder eines Clusters
• - Durchschnittliche Energie der Lieder eines Clusters

In the following, it is clarified which cluster is selected as the first played cluster. An appropriate selection of the cluster to be played primarily is particularly important since the order of the remaining clusters is adapted to this cluster. The cluster is selected, which is considered suitable depending on the user activity. A correlation with the following four parameters is established:

• - Number of favorite songs within a cluster
• - Average speed of the songs of a cluster
• - Average mood of the songs of a cluster
• - Average energy of the songs of a cluster

The cluster with the highest value for a parameter receives the function value for the respective parameter 1 , All songs in between contain the function value for the parameter $\frac{1}{POsitiOninSOrtierunG},$

User activity is also mapped to a scale of values between 0 for minimum and 1 for maximum. The difference between the function values and the value of the user activity is formed for each parameter. The cluster is selected, which has the largest difference over all parameters with the lowest value compared to other songs.

Cluster order:

• - Priorisierung der Cluster ausschließlich nach Anzahl der Lieblingslieder
• - Semantische Priorisierung

Regarding the order of the clusters, there are two different ways that can be implemented.

• - prioritization of the clusters only by number of favorite songs
• - Semantic Prioritization

Prioritization of the clusters exclusively by number of favorite songs

The prioritization of the clusters exclusively according to the number of favorite songs is as follows. With high user activity, clusters are placed at a higher position in the queue, which contains a high number of favorite songs. The distribution of the clusters with favorite songs represents a Gaussian normal distribution. The sigma value becomes lower with higher user activity, but is limited over a lower and upper limit. As a result, less and less favorite songs are played toward the end than at the beginning of a use unit, since the songs are removed from the queue after playing. Even if the user activity were very high at a late stage of use, fewer favorite songs would be available for playback. However, this approach is followed because it is assumed that the number of songs in the queue often exceeds the number of songs actually played in a session.

Semantic prioritization

The semantic prioritization works as follows.

• - Entfernung der Genreklassen des Clusters
• - Anzahl der Lieblingslieder innerhalb des Clusters
• - Durchschnitt über alle Erscheinungsjahre der Lieder des Clusters
• - Durchschnittsgeschwindigkeit des Clusters
• - Durchschnittliche Stimmung des Clusters
• - Durchschnittliche Energie des Clusters

The determination of the subsequent cluster is determined based on the similarity of the currently considered cluster and the subsequent cluster with regard to the following parameters:

• - Removal of the genre classes of the cluster
• - Number of favorite songs within the cluster
• - Average over all years of publication of the songs of the cluster
• - Average speed of the cluster
• - Average mood of the cluster
• - Average energy of the cluster

The selection of the follow-up cluster is a crucial function of mission planning, as it sets the intensity of mood changes, as opposed to the frequency of mood changes over a period of time. A high dissimilarity between clusters leads to a strong change of mood. A strong change of mood between clusters is often undesirable, but can also be preferred in certain situations. In this invention, a low change of mood between clusters is preferred. Thus, clusters with high similarity would be more likely to follow one another directly in terms of the parameters described above.

The prioritization of the parameters as well as the correlation between them is freely selectable or can be set depending on the user activity. For example, with high user activity, the parameter of the similarity of the average mood of the cluster songs between two clusters could be weighted higher than all other parameters.

Internal order:

• - Allgemeine Güte
• - Harmonieflow
• - Geschwindigkeitsflow

After the order of the clusters has been determined, the song order within the clusters is determined. This is done on the basis of three criteria:

• - General quality
• - Harmonylow
• - Speed flow

First of all, the start song and final song of each cluster are determined. The beginning song of a cluster, as well as the final song must meet special requirements. The first song of a cluster must have a high general goodness with the final song of a previous cluster and preferably fulfill the missions of harmony and velocity flow. The last song of a cluster must have a high general goodness with the first song of a subsequent cluster and preferably fulfill the missions of harmony and velocity flow.

According to the set songs, the rest of the cluster will be adjusted. The overall goodness between the songs in the cluster must preferably be high, and the mission of the Harmonieflow, and Speedflow, must be tracked. Since this is not always possible with the given songs of the queue, a prioritization must be defined, which criterion may be violated first. The prioritization is as follows: General Q, Harmony Low, Velocity Flow. Jumps in harmony are thus allowed. Jumps in speed are also allowed, but not necessarily audible, as the velocities of the songs are adjusted as the transition allows.

The harmony low describes the mission in terms of the sequence of key changes between the songs over the entire queue. Within the clusters, a harmony flow is built up. The harmony flow is preferably continued over as many clusters as possible. The change to the next higher key shifted by a fifth leads to a more energetic, happier change of mood. The change to the next lower key shifted by a fifth leads to a melancholic change of mood. The nature of the harmony flow is dependent on the user activity. With a high level of user activity, the circle of fifths expires clockwise, resulting in more energetic, happier changes between songs. The order of the songs within each cluster is now preferably arranged so that the keys of the songs differ by a maximum of one fifth at a time corresponding to the circle of fifths in the clockwise direction. For a low user activity would be done in accordance with the contrary. The keys of the songs may also have the same key of the potential previous song, or the parallel minor / major key. All in all, however, the goal is to achieve regularly increasing fifths of the fifths over the entire queue in accordance with the circle of fifths.

The velocity flow is the mission to change the speed of the songs between the songs across the queue. Within the clusters, a velocity flow is built up. This can be continued or changed across any number of clusters. By the speed flow, it is meant that the speed of the songs in the play order is constantly changed. The user activity represents the parameter which determines how the speed for this use unit is changed. With increased user activity, the mission is to increase the speed across the songs. This is done by arranging the songs within the cluster in ascending order of speed. For a low user activity would apply accordingly contrary. Speed, unlike harmonic mission, does not have to be clustered. This is not always possible because the speed of "R & B" songs is mostly lower than that of "House" songs.

The velocity flow can be achieved even if the velocities of the songs within a cluster are prioritized lower due to the prioritization of the general goodness and harmony sorting. For this, the speeds of the songs are artificially increased using digital audio processing techniques, while maintaining the pitch. This procedure is considered state of the art and is therefore considered to be prerequisite. The velocities of songs within a cluster are still aligned during a transition. However, the speed of each song is artificially heard. Thus, a song A, which in the meta-information has the value 123 BPM as the velocity parameter, would be steadily raised to 125 BPM after the transition. Follow song B, with an original speed of 124 BPM, would be temporarily increased to 125 BPM during Song A transition to undermine a synchronized transition. As a result, the song would be raised from 125 BPM to 126 BPM. Thus, a steady increase in speed from 123 BPM (before song A) to 126 BPM (after song B) would be achieved, even though the songs have different original speeds. The maximum artificial increase in the speed of a song is, with exceptions, limited to + 10 BPM at the original speed. An analogous speed reduction for influencing the velocity flow as described here does not take place.

This artificial speed increase takes place only with very high user activity.

The parameters of the general quality, the harmony flow and the velocity flow are optimally correlated taking into account the prioritization. This results in the internal order of the individual clusters.

The order within a cluster, as well as the cluster order is now known. The queue is resorted to contain the songs in that order.

The following is an example of the operation of mission planning with low user activity:

The cluster templates would be larger on average, so the clusters would contain a larger number of songs. The size of the cluster templates would also vary slightly. Initially, song clusters with a lower average speed would be selected. The harmony low would cause the songs to be arranged over the entire queue with later play order so that, ideally, the key of the follower song is not equal or parallel, one fifth lower in the same tone gender. The velocity flow would optimally result in lower speed songs being placed at least within a cluster farther back in the queue.

Parameters such as the general good between two consecutive songs within a cluster, as well as the two consecutive songs across the cluster boundaries, would always be preferred, regardless of the user activity.

Modify the queue

Dynamically modifying the queue by changing the order or by adding or deleting songs does not limit the mission planning functionality. The manually modified song position within the queue, does not affect the position of the remaining songs in the order.

However, it is possible for the user to add individual songs or even a lot of songs, for example in the form of a playlist, to the queue, taking into account the mission planning.

• - Modifizieren der Queue ohne Berücksichtigung der Missionsplanung
• - Modifizieren der Queue unter Berücksichtigung der Missionsplanung
• - Anhängen der Lieder an das Ende der Queue unter Berücksichtigung der Missionsplanung

The user is thus presented with three choices:

• - Modify the queue without consideration of mission planning
• - Modify the queue taking into account the mission planning
• - Attach the songs to the end of the queue, taking into account the mission planning

Modifying the queue, taking into account mission planning, causes the current queue to undergo mission planning again whenever the play order is modified.

The current queue refers to the current queue after manual modification, including the totality of all newly added songs.

Attaching the songs to the end of the queue, taking into account mission planning, means that the order of previous songs will be preserved, and the newly added songs will undergo their own mission planning and be added to the end of the current queue in that order. Only the last cluster of the current queue is automatically modified to match the songs' internal order to the first cluster of newly added songs, so that the general quality, harmony flow and velocity flow specifications are adhered to as far as possible.

Songs that have been manually modified once in the queue in terms of their playing order will continue to be statically bound to this playhead, disabled for mission planning, and thus never influenced by mission planning. The rest of the queue builds up so seen around these play positions. The song can be activated by re-adding in consideration of the mission planning.

Dynamic adaptation of mission planning

Mission planning is initially re-executed with each new usage unit, initially using the average value for the user activity from past usage units.

Since the last dynamic adaptation of the mission planning, the user activity is loaded in a defined sampling rate. The average x of past user activity values is calculated. In addition, the average y of the past user activity values is calculated within a defined time window. If the difference between x and y exceeds a defined threshold, this is considered a massive change in user activity and the dynamic adjustment of mission planning is re-initiated with the current user activity value.

The time window for the calculation of the average past user activity y, becomes shorter with higher user activity.

The dynamic adaptation of the mission planning as a function of the user activity represents a core innovation of the invention. In the following, the causality between a user's mood change reflected in the user activity and mission planning is illustrated by way of example. If the user activity significantly increases, e.g. due to sporty driving after a long period of quiet driving, and maintained over a period of time, mission planning would be re-applied to the queue, without affecting the currently playing song.

The mission planning could also be controversial to the user activity used to deliberately counteract the activity of the user. If the software is implemented inside a car using speed detection, one could identify dangerous driving behavior from very high speeding violations or other factors. Here one could head the mission planning to an operation with a low user activity. The driver would thus subconsciously change his driving behavior to a quieter driving behavior. But still the same songs are played out of the cue and the transitional construction still has the personal preferences, as well would take into account generalized preferences within his personal set of rules, he would not necessarily notice the change in function.

Extended song analysis

• - Genrebestimmung
• - Songsegmentierung

The extended song analysis is divided into two functionalities:

• - Genre determination
• - Song segmentation

Song segmentation describes an audio signal processing method that extracts information about the structure of a song from the song's PCM audio data. The method used here segments the song taking into account the information about the genre of a song.

After the song segmentation, each of the segments contains information about the semantic significance of the individual segment. This information is used in this invention to determine potential transition times in songs, to select and to derive and adapt an abstract transitional shape to a transient situation at one of the times.

The song segmentation method presented in this invention differs from conventional song segmentation methods in terms of the objective pursued and therefore in the approach. While traditional song segmentation methods rely heavily on technical possibilities, special emphasis is placed on segmentation in terms of musical knowledge. The approach presented here combines low level technologies with high level knowledge about building modern, popular songs. The correct determination of the time limits of segments is crucial for the further use of the results. In this song segmentation method, the consideration of the bass drum rhythm as well as the perceived volume (RMS) plays a particularly important role.

The correct semantic designation of a segment in terms of its meaning for the song structure is extracted. A high degree of correctness of the designation of the segments compared to a ground truth is, although not required, for the increase in the quality of the functioning of the software. If segments are mislabeled and the transition does not occur on the appropriate segment, it is highly likely to have a high transient quality because the time limits of the segments are set correctly and the mission planning songs are chosen such that the transient quality is independent of the transition time is maximized. A transition between two songs, which are adjusted in terms of speed and beyond even have a matching key, will rarely be perceived as "very unpleasant" for the user. Achieving the transition to an unscheduled segment, however, alters the appearance of the transition and therefore promotes a varied musical experience.

The content song analysis method described here is designed for song segmentation and interpretation of modern, popular songs that have a clearly identifiable bass drum structure and a clearly identifiable rhythm.

The song segmentation is thus particularly suitable for songs that have a high dance factor.

Finally, the results of the analysis serve to construct a transition between two digital songs as a simulation of the hang-up behavior of a human club or radio DJ. Since the music played by club or radio DJs is nowadays mostly modern and popular, and the product of the DJs is often supposed to create a danceable mood, the song analysis is also geared towards this goal.

• - Berechnung der klanglichen Ähnlichkeit zweier Audio Signale unterschiedlicher Länge
• - Berechnung der melodischen Ähnlichkeit zweier Audio Signale unterschiedlicher Länge

The method described below is based on already known technologies of music information retrieval and audio signal processing. The methods are considered to be presupposed for this invention and will therefore not be discussed further here. The implementation of these methods is independent of the functionalities described here and can be freely modified. The patent does not claim the following technologies:

• - Calculation of the phonetic similarity of two audio signals of different lengths
• - Calculation of the melodic similarity of two audio signals of different lengths

• - Berechnung der Ähnlichkeit der Rythmusstruktur zweier Audio Signale unterschiedlicher Länge
• - Erstellen einer Rythm Map

The methods described below represent further music information retrieval methods, which are described in more detail in this invention:

• Calculation of the similarity of the rhythm structure of two audio signals of different lengths
• - Creating a rhythm map

The similarity in sound of two audio signals of different lengths describes the similarity based on the MFCC vectors (Mel Frequency Cepstral Coefficients) of two audio signals. For signals of different lengths, the vectors are adjusted and compared using the Dynamic Time Warping algorithm. The similarity of the MFCC vectors of given audio signals is hereafter referred to as aural similarity of two audio signals and represented as a scalar.

The melodic similarity of two audio signals of different lengths describes the similarity of the chromium vectors of individual audio signals. The similarity of the chromium vectors of given audio signals is referred to below as the melodic similarity of two audio signals and represented as a scalar.

The creation of a rhythm map can be realized with different methods. A qualitatively satisfactory method is described below. A given audio signal is initially normalized. A highest peak in the low frequency spectrum below 120 Hz in a given time window is defined as the bass drum signal. If this value exceeds a certain threshold for a defined time unit in a time window of defined length, this is interpreted as identification of a bass drum pulse. In the following it is assumed that there is a steady 4/4 beat over the entire song, whereby the speed remains constant. This requires the speed of the song from the meta-information. The song is now evenly divided in terms of speed from the appearance of the very first bass drum impulse into 16 beats per bar. At 120 BPM, that would be 480 beats in one minute at a distance of 0.125 milliseconds. Starting with the very first bass drum pulse, each identified bass drum pulse is given a time stamp t, with the time of the highest peak in a frequency below 120 Hz within a time window of defined length. The bass drum pulses are now entered in a rhythm map in the counting time, which has the smallest difference with the specific time stamp t. The rhythm map contains at this count time an entry with the original timestamp t of the bass drum impulse in the song.

• - Macrosegmentierung
• - Microsegmentierung
• - Genreklassifikation
• - Chorussegmente
• - Semantische Macrosegmentierung
• - Semantische Microsegmentierung
• - Segmentinterpretation
• - Übergangsfaktoren

The advanced song analysis consists of the following phases:

• - Macrosegmentation
• - Microsegmentation
• - Genre classification
• - Chorus segments
• - Semantic Macrosegmentation
• - Semantic microsegmentation
• - Segment Interpretation
• - transition factors

Macro segmentation

The following describes a method for calculating differences in the rhythmic structure of two audio segments, which is defined as rhythmic similarity. The entries of two segments of selected length at selected times are loaded from the rhythm map. One of the segments is set as Groundtruth. The entries of the other area are moved to the next entry in Groundtruth. The differences are summed up. A consideration of the differences in the number of entries is not necessary here.

The very first bass drum impulse within the song is determined. Based on this, the rhythm map is created.

From the first entry of the rhythm map, the entries of the following four bars including the entry at the current count time are considered. The rhythm structure of the current full measure must be with the three subsequent bars have a rhythmic similarity above a defined threshold. The threshold depends on the genre, for example, in songs of the genre "House" less rhythmic differences occur within a segment than for example in a song of the genre "R &B".

If this is fulfilled and the rhythm map contains at least one further entry at any count time within the current full measure or the next three measures, a microsegment is created.

The left boundary of the Macrose segment is defined with the original time stamp t of the first bass drum impulse in the song, the right limit is defined as the sum of the time stamp t and the length of four bars.

Following the last count, the next four bars are evaluated. Here it is again checked whether at least two entries occur within the rhythm map. If so, the four bars are added to the previous Macrose segment. This is done by extending the end time stamp of the Macrose segment by a duration of four bars. This process repeats along the time course of the song until within a section of four bars no two entries are entered in the rhythm map. This represents the end of this Macrosegment.

The next entry for a beat in the rhythm map is searched. From this entry, the process continues. This procedure divides the entire song into macrose segments or free sections. Although, with this method, songs in which a large change in speed occurs can not be segmented correctly, as will become apparent in later phases, this is not relevant to the quality of the invention. This is due to the fact that only rate-constant songs have to be correctly segmented, since only these subvert a segmentation based transient construction.

The choice of four bars as a size for the review of an area of the song is related to the fact that in modern light music the lower limit for the duration of a song section is mainly set to four bars.

The approach described here assumes that song parts begin with a bass drum impulse on the first beat, as this is almost exclusively the case in modern music. A missing bass drum impulse at the earliest beat of a segment, with the following bars containing a bass drum impulse on the first beat, does not change the result, since this bar would not be included in the Macrose segment due to rhythmic dissimilarity and thus not the wrong count time as the beginning of the beat Clock would set. However, due to segment segmentation later explained, this clock would still be captured.

Micro segmentation

For the segmentation into micro segments of defined length, it is assumed that the number of cycles of a Macrose segment is a multiple of 4. If this property is not met, the segment of macro segmentation must be expanded or contracted.

The method of segment length unification is described below. Similarities of timbre between bars are considered. First of all, it is examined for both segment ends, whether an extension or reduction of the segment boundary should take place.

First, an internal similarity z between two internal clocks is calculated. The inner clocks are represented by the clock t _x , 3 clocks to the left of the rightmost clock t ( _{n-1) of} the Macrose segment, and the clock t _y , 3 clocks to the right of the leftmost clock t ₀ . Z is calculated from the tonal difference of t _x and t _y .

Next, the similarity of the outermost clock of a segment to one of the inner clocks is calculated. This similarity must exceed a certain threshold. The threshold depends on z. With a high similarity, an extension takes place at this boundary, otherwise a reduction.

Afterwards, the corresponding ends of the segment are extended or reduced. For this purpose, a maximum of three more bars per end can be given or annexed. The following process is performed per page. The currently outermost clock t _{a of} the Macrose segment at one of the segment ends, is with the next inner clock compared. For this the sound similarity x is calculated. Subsequently, the clock t _a , is compared with the next clock. For this the sound similarity y is calculated.

The difference Δd = x - y is formed. If this value exceeds a threshold dependent on z, the clock is appended to the end of the segment and thus annexed to the Macrose segment. If the threshold is not exceeded, the clock is delivered to the next outermost clock, which does not belong to any Macrosegment.

If the segment still does not have a length which is a multiple of four, first the threshold value is gradually reduced. If the threshold has arrived at a defined lower limit, the segment length still not being a multiple of four, clocks are equally removed or reduced at both ends. With an odd segment length, the clock is first removed or reduced at one of the segment boundaries, which has the higher acoustic similarity with the corresponding clock t _a .

The Macrose segments have a number of clocks after segment length unification that is a multiple of four. The Macrose segment is now evenly divided into four-bar micro segments from the beginning of the first beat.

Genre classification

• - House
• - RnB
• - Other

The genre classification is executed, regardless of whether or not a genre can be extracted from the meta-information. The genre classifier, classifies a given song roughly into the three main classes:

• - House
• - RnB
• - Other

A song is classified as a house song if the speed of the song is between 105 and 170 BPM and e.g. 40% of the bars within the rhythm map, which contain at least two entries within a bar, have a 90% match in terms of rhythmic similarity to the house pattern (bass drum on each quarter note in time). If the song does not fall under this genre class, the micro segments containing at least two entries within the rhythm map will be clustered with respect to their rhythmic similarity using an exploratory clustering method. If the largest cluster contains e.g. At least 40% of all micro segments, the quality of the bass drum will be checked. The bass drum peaks in the song at the original timestamps taken from the Beat Map entries are checked. If, after normalization of the song, these are on average above a fixed threshold, the song is classified as R & B. A song that has not been classified as a House or R & B song is classified as Other. A second, optional step of the genre classification, follows the segment interpretation. In the presented method, songs that are not part of the genre R & B often fall into this genre class. However, this is intentional because these genre classes are predominantly relevant for transition selection. A rock song classified as a Class R & B song can be transposed with the same transition as an actual R & B song. Likewise, R & B songs that have rhythmic resemblance to the house pattern can also be blended into house affine transitions.

Chorus segments

For each existing micro segment, the melodic and sonic similarity is calculated and stored as a self similarity matrix. With the method of the Self Similarity Analysis, all microsegments are clustered with respect to the two similarities. For this purpose, a k-methods method with different parameters is used. Finally, the clusters are selected from the calculations with the parameter where the Euclidean distance from the center of the cluster to the elements is the least. The microsegments that belong to the cluster after the clustering process, in which the elements have the highest similarity to each other, are marked as chorus segments. This procedure is considered State Of The Art and will not be discussed further. The patent does not claim this procedure.

This approach suggests that the chorus of a song is always accompanied by a bass drum rhythm. Since a high level of correctness of the segment names is not required, this approach is acceptable.

Semantic Macrosegmentation

The fact that the song structure of each song can be broken down to alternating chorus and verse sections is used in the formation of semantic Macrose segments. Micro segments marked as chorus segments are joined to all macros segments directly adjacent to each segment. The semantic Macrose segments formed in this way are called chorus segments and receive an Id. However, it must be considered that the limits of the microsegments remain stored for later use.

All remaining microsegments are also connected to Macrose segments with the adjacent micro-segments, which are not marked as chorus segments, or with adjacent free areas. The semantic Macrose segments thus formed are called Versesegment and receive an Id.

At this time, each section of the audio signal is within a semantic Macrosegment.

Semantic microsegmentation

The semantic Macrose segments, which are called chorus or versesegment, are segmented into smaller semantic micro segments.

• - Einteilung der Versesegmente in Kick und Non Kick Parts
• - Einteilung des ersten Macrosegments mit Sprachanteil in Intro und Verse bzw. Chorus Parts
• - Segmentierung der Kick enthaltenden Macrosegmente in verschiedene Parts

The following is the segmentation of the semantic Macrose segments:

• - Classification of the verse segments in Kick and Non Kick Parts
• - Classification of the first Macrosegment with voice part in intro and verses or chorus parts
• - Segmentation of the kick containing Macrose segments into different parts

• - Kick Flag
• - Voice Flag

For this, the following flags are introduced, which contain extended information about the segments:

• - Kick Flag
• - Voice flag

Classification of the verse segments in Kick and Non Kick Parts

The verse segments are now divided into individual semantic micro segments. All more than four bars long directly consecutive bars, which have an entry in the rhythm map, are each joined together to form a semantic microsegment. The kick flag is set to 1 for this microsegment. The remaining contiguous regions within the verse segment, which do not contain an entry in the rhythm map, are each joined together to form their own semantic micro segments, the kick flag being set to 0.

Classification of the first segment with speech content in intro and verse parts

The very first Macrose segment in the song, if it contains a portion of speech, is segmented into either Intro and Verse micro segments or into Intro and Chorus micro segments. For this purpose, various methods for recognizing speech components in audio signals can be used. In this invention, the information is extracted from a metadata-based method.

The timestamp of the first voice portion of the song is noted within a rhythm map analogous to the voice map. However, the time stamp is entered at a count time which is removed from the first entry of a bass drum pulse in the rhythm map in a number of cycles which is a multiple of 4. The clock is chosen which has the smallest time interval from the original time stamp of the speech component.

If there is a period of at least four measures without an entry in the voice map in the first Macrose segment containing an entry in the voicemap, this period is segmented as a semantic microsegment and marked with the label Intro. The other part of the Macrose segment is separated as a microsegment and retains its name.

For all micro segments, which contain an entry in the voicemap, the voiceflag is set to 1.

In the invention presented here, the information about speech portions in songs is extracted by using meta information about time-synchronized speech portions in songs derived from a database. In this regard, there are services that provide such information.

The data contains information about the beginning of the speech portion, as well as the spoken / sung textual content.

If the song contains an entry concerning the language parts, these data are first processed. All entries are ignored, which are only a concatenation of contents like "Yeah", "Oh", "Mhh" or the name of one of the song interpreters or song producers.

The text blocks to be ignored are taken from a previously defined list, as well as the meta information of the song regarding the artist name and the producer name. If a match of song content and any concatenation of the text modules occurs, this entry is ignored. This processing serves as filtering of asynchronous language portions to the rhythm of the song. The occurrence of such a speech module therefore does not characterize a new song segment.

Segmentation of the kick containing segments into different parts

All parts within the verse segments or chorus segments that are more than four bars long and have a kick duration of four bars are now further segmented. A Kickpart with a duration of four bars, describes a four-bar period within a segment, which has at least two entries in the Rythm Map within this period.

• - Differenz des durchschnittlichen RMS Pegels der Segmente, nach Filterung durch einen Hochpassfilter
• - Rhythmische Ähnlichkeit der Segmente
• - Klangliche Ähnlichkeit der Segmente

A technical similarity between two audio segments results from a formula regarding the similarity factors of two 4-stroke segments. The similarity factors are calculated from the following components:

• - Difference of the average RMS level of the segments, after filtering by a high-pass filter
• - Rhythmic similarity of the segments
• - Sound similarity of the segments

All the verse segments are now temporarily subdivided into equal 4-beat segments from the beginning of the first bass drum pulse within the segment. Each segment is now checked for rhythm structure anomalies at the beginning or end of the segment. Upon identification of such an anomaly, the technical similarity of the current 4-stroke segment and the subsequent 4-stroke segment is calculated. If the technical similarity falls below a high, genreabhängigen threshold, the two 4-stroke segments are divided as separate segments. Otherwise, the segments remain contiguous.

In addition, the technical similarity of all 4-bar segments within the Macrosegments is compared independently of anomalies of the rhythm structure. For this purpose, the genreabhängige threshold is set much lower, so that the dissimilarity between the two segments must be significantly higher to initiate a segmentation.

If the similarity of this genreabhängige threshold value is a segmentation of the considered 4-stroke segments instead. Otherwise, the segments remain contiguous.

Direct consecutive 4-stroke segments, between which a sufficiently high dissimilarity value has not been determined, are combined again, if both are in a segmented, segmented form.

Check for anomalies of the rhythm structure using the following procedure. Two consecutive 4-stroke segments are compared for their entries in the beat map. If the entries in the beat maps of two 4-stroke segments within at least two bars have a high rhythmic similarity above a threshold s1, within the first or last one to two bars but a similarity is achieved, which does not exceed a low threshold s2 or the difference in the number of entries exceeds a threshold s3, an anomaly is detected. The two 4-stroke segments are thus segmented as such. The mentioned thresholds describe a universal approach and can be varied depending on the genre. For music belonging to the genre "House", the identification of a missing single bass drum pulse at the beginning or end of a 4-bar segment already leads to a segmentation.

Interpretation of the segments

The individual semantic microsegments within a macrosegment are interpreted in two phases:

phase 1

The microsegments within a Chorus Macrose segment are finally clustered on the basis of their sonic similarity to all other micro segments within a Macro Chorus segment. From this clustering process result new clusters cl. The elements of clusters cl with the smallest differences between each other are given the label, c_ [n] ', since they represent the real chorus, where n stands for the temporal position in the song. The remaining microsections of the remaining clusters cl are given the label, cx_ [n] ', where n stands for the time position in the song.

The very first micro segment is marked, if not already marked as, Intro '. The very last micro segment is marked as 'Outro'. The micro segments within the verses Macrose segments are marked chronologically with the label, v_ [n] ', where n stands for the temporal position in the song. The last verse Microsegment, which occurs before a Chorus Macrosegment and has the Kick Flag at 0, is marked as 'Bridge'.

phase 2

By segments are meant in the following directly consecutive 4-stroke units, regardless of the micro segments. The first occurring maximum of eight consecutive segments representing a kick part and marked as 'Intro' or 'v_ [n]' are represented by the last occurring maximum of eight consecutive segments representing one kick part and either as, cx_ [n ] ',, v', or 'Outro' are compared in terms of their technical similarity. In addition, the dissimilarity to the 'c_ [n]' segments must be above a certain threshold. If this similarity is above a certain threshold, the segments are additionally marked as 'Extended_Intro' or 'Extended_Outro' Part, depending on which end of the song they are closer to. The Extended_Outro / Extended_Intro designation is written to the microsegment containing the marked segments.

The boundaries of each semantic microsegment represent the potential transition times.

A more detailed genre classification can be based on the information from Phase 2 be won. The algorithm of genre classification can be changed by the provider of the software at any time to adapt to current social trends.

An example of a more detailed genre classification is described below. A song with Extended Part can be classified in any case, taking into account the Extended-Intro and Extended_Outro Microsegment. Songs containing these micro segments are initially classified as Extended Songs. Now a comparison of the average RMS levels of the 'Extended_Intro' microsegment with the first occurring Chorus microsegments, c_0 'is made. Before the average RMS level is calculated, the song is normalized and the frequencies are filtered by a high pass filter. Now, the average RMS levels of the 'Extended_Intro' marked micro segments and the 'c_0' micro segments are compared.

If RMS (c_0) + tolerance> RMS (Extended_Intro) → Genre:, Electro House '
Otherwise Genre: 'commercial Deep House', 'Deep House', 'Techno'

The number of potential bass drum pulses in the song is compared to the actual number of bass drum pulses within the rhythm map. $$\frac{\left(\left(\text{\# 4}telZäHlzeiten\right)-\left(\text{\#}Bassdrumimpulse\right)\right)}{\left(\text{\# 4}telZäHlzeiten\right)}>0.4$$

→ Genre: 'commercial Deep House'
Otherwise: 'Deep House', Techno '

Now the ratio of the average RMS level of the frequencies in the mid and high frequency spectrum is set in relation to the average RMS level of the frequencies in the low frequency spectrum over the entire song.
Relationship> Techno Threshold → Genre: 'Techno'
Otherwise: Genre: 'Deep House'

Calculate transition factors

• - Durchschnittlicher RMS Pegel nach RMS-Normalisierung
• - Durchschnittlicher RMS Pegel pro Frequenzband (Low: 0-250Hz, Mid: 251-5000Hz, High: 5000-20000Hz) nach RMS-Normalisierung
• - MFCC Vektor
• - Kick Flag
• - Voice Flag

The transition factors per micro segment are limited to the following components but are expandable:

• - Average RMS level after RMS normalization
• - Average RMS level per frequency band (Low: 0-250Hz, Mid: 251-5000Hz, High: 5000-20000Hz) after RMS normalization
• - MFCC vector
• - Kick Flag
• - Voice flag

• - Menge an Microsegmenten
  • ◯ Semantische Bezeichnung pro Microsegment
    • ■ Semantische Bezeichnung des übergeordneten Macrosegments {Verse, Chorus}
    • ■ Semantische Bezeichnung des Microsegments {Intro, v_[n], c_[n], c_x[n], Bridge, Outro}
    • ■ Semantische Bezeichnung des interpretierten Microsegments (Intro, Extended Intro, v_[n], c_[n], c_x[n], Extended Outro, Outro)
  • ◯ Grenzen pro Microsegment
    • ■ Linke Grenze Zeitstempel
    • ■ Rechte Grenze Zeitstempel
  • o Übergangsfaktoren pro Microsegment:
    • ■ Durchschnittlicher RMS Pegel nach RMS-Normalisierung
    • ■ Durchschnittlicher RMS Pegel pro Frequenzband (Low: 0-250Hz, Mid: 251-5000Hz, High: 5000-20000Hz) nach RMS-Normalisierung
    • ■ MFCC Vektor
    • ■ Kick Flag
    • ■ Voice Flag
• - Genreklasse des Liedes: {House, RnB, Other}

The result of the extended song analysis are thus the following components:

• - amount of micro segments
  • ◯ Semantic designation per microsegment
    • ■ Semantic label of the parent Macrosegment {verses, chorus}
    • ■ Semantic Microsegment Description {Intro, v_ [n], c_ [n], c_x [n], Bridge, Outro}
    • ■ Semantic label of the interpreted microsegment (Intro, Extended Intro, v_ [n], c_ [n], c_x [n], Extended Outro, Outro)
  • ◯ Limits per microsegment
    • ■ Left border timestamp
    • ■ Right border timestamp
  • o Transition factors per microsegment:
    • ■ Average RMS level after RMS normalization
    • ■ Average RMS level per frequency band (Low: 0-250Hz, Mid: 251-5000Hz, High: 5000-20000Hz) after RMS normalization
    • ■ MFCC vector
    • ■ Kick Flag
    • ■ Voice flag
• - Genre class of the song: {House, RnB, Other}

For all other phases, the result of any song analysis can be used, which fulfills the format given above.

Select transition time

The function for selecting the transition time represents a core innovation of this invention. In addition to the technical parameters, this also depends on semantic parameters, on the personal, situation-dependent needs of the user.

• - allgemeine persönliche Präferenzen des Users (gespeicherte Userdaten),
• - situationsabhängige Präferenzen des Users (Useraktivität) unter Berücksichtigung:
  • - des Benutzungsverhaltens des Musikplayers,
  • - des Fahrverhaltens sowie
  • - des umweltbedingten Kontextes, und
• - allgemeine gesellschaftliche Trends (aggregierte Flottendaten).

These were composed as follows:

• - general personal preferences of the user (stored user data),
• - situation-dependent preferences of the user (user activity) under consideration:
  • - the usage behavior of the music player,
  • - the driving behavior as well
  • - the environmental context, and
• - general social trends (aggregated fleet data).

• - Useraktivitätsfaktor
• - Semantischer Parameter
• - Inhaltlicher Parameter
• - Technischer Parameter

The segments obtained from the extended song analysis represent the transitional points in time for the transitional construction. An attempt is made to select a segment x in song A and a segment y in song B in such a way that the quality of a transition described in the corresponding rules is maximized , In the following, the segments x and y are referred to as the selected segment combination (x, y). In the following, the factors for the selection of the segment combination are explained in more detail. These were composed of the following parameters:

• - User activity factor
• - Semantic parameter
• - Content parameter
• - Technical parameter

User activity factor

The user activity factor is the influence of user activity on the selection of the segment combination. Depending on the size of the user activity factor, earlier segments are selected within song A and later segments in song B. This shortens the duration of the song, as well as the duration until the climax of the new song, resulting in a more exciting musical experience. The user activity factor thus determines a time window in which segments can be selected.

Semantic parameter

The semantic parameter refers to the influence of the segmental properties on the song structure. These influence the semantic transition quality of the transition. An attempt is made to maximize the semantic quality of transition between two segments, which results in the transition being perceived as exciting by the user.

The prioritization of the segment combinations is done taking into account the personal rules. The rules are ordered by prioritization. In the personal set of rules of a user certain segment combinations can be general, exclusively for one genre or only for one song. The personal rules are fused from personal preferences, as well as a general set of rules. In this framework, affinities to segments within the general public of songs, as well as specific genres and songs are already formulated as rules with priority value.

• - Ein Chorussegment x endet in einem Versesegment y
• - Ein Extended_Outro Segment x endet in einem Versesegment y
• - Ein Extended_Outro Segment x endet in einem Extended Intro Segment y
• - Ein Chorussegment x vor eine Bridge y endet in einem Versesegment y
• - Ein Chorussegment x endet in einem Chorussegment y
• - Ein Versesegment x endet in einem Versesegment y

The following is an excerpt of an exemplary set of rules for the genre House of a user U1 presented. The rules are ordered by priority rule:

• - A chorus segment x ends in a verse segment y
• - An Extended_Outro segment x ends in a verse segment y
• - An Extended_Outro segment x ends in an Extended Intro segment y
• - A chorus segment x in front of a bridge y ends in a verse segment y
• - A chorus segment x ends in a chorus segment y
• - A verse segment x ends in a verse segment y

In addition to the rules applied by the policy, we try to adhere to the following guidelines:

• - Es wird versucht, den Übergangszeitpunkt so zu wählen, dass die Anzahl von präferierten Segmenten in Lied A vor dem Übergangszeitpunkt zu Lied B maximal ist. Es wird außerdem versucht den Übergangszeitpunkt so zu wählen, dass ein präferiertes Segment das letzte Segment x vor dem Übergangszeitpunkt darstellt.
• - Es wird versucht, den Übergangszeitpunkt so zu wählen, dass die Anzahl von präferierten Segmenten in Lied B nach dem Übergang maximal ist.
• - Es wird versucht, den Übergangzeitpunkt so zu wählen, dass der Übergang nicht in einem nicht präferierten Segment y in Lied B endet.

The preferred segments mentioned here are segments to which the user or the general public has a high affinity according to personal rules.

• It is attempted to choose the transition time such that the number of preferred segments in song A is maximum before the transition to song B. It is also attempted to choose the transition time such that a preferred segment represents the last segment x before the transition time.
• It is attempted to choose the transition time such that the number of preferred segments in song B after the transition is maximum.
• An attempt is made to choose the transition time such that the transition does not end in a non-preferred segment y in song B.

Content parameter

The content parameter refers to the influence of technical song parameters, which influence the transitional quality of the transition. An attempt is made to maximize the crossover quality between two segments, which results in the user perceiving the transition as fluid. The technical song parameters are the segment properties.

• - Falls die Tonarten der beiden Lieder hinsichtlich der in beschriebenen passenden Tonarten nicht übereinstimmen, wird, falls vorhanden, eines der Extended-Intro oder Extended_Outro Segmente eines der Lieder oder bevorzugt beider Lieder als Übergangszeitpunkt gesetzt.
• - Die Segmentkombination wird so gewählt, dass nicht in den Segmenten beider Lieder ein Sprachanteil vermerkt ist.
• - Es werden, falls vorhanden, bevorzugt Extended-Intro oder Extended_Outro Segmente als Übergangszeitpunkte verwendet.
• - Falls ein Extended_Intro oder Extended_Outro Segmente verwendet wird, sollte das Segment des anderen Liedes ein Segment mit Kick Flag auf 1 sein.
• - Es werden alle Segmentkombinationen priorisiert, welche nicht beide die Kick Flag auf 0 haben.
• - Bei der Wahl eines Chorussegments als Segment x, wird bevorzugt das letzte Chorussegment, vor Auftreten eines Verses gewählt.
• - Bei der Wahl eines Versesegments als Segment y, wird bevorzugt das erste Versesegment nach Auftreten des Intros oder nach Auftreten eines Chorussegments gewählt.
• - Bei der Wahl eines Versesegments als Segment y, wird bevorzugt das erste Versesegment gewählt, welches die Voice Flag auf 1 hat.

The prioritization of the segment combinations takes place taking into account the following defined rules, which can only be changed by the provider of the software:

• If the keys of the two songs do not match with respect to the corresponding keys described in FIG. 1, one of the Extended Intro or Extended_Outro segments of one of the songs, or preferably both songs, if any, is set as the transition time.
• - The segment combination is chosen so that not in the segments of both songs a voice component is noted.
• If present, Extended-Intro or Extended_Outro segments are preferably used as transition times.
• - If an Extended_Intro or Extended_Outro segments are used, the segment of the other song should be a Segment with Kick Flag at 1.
• All segment combinations that do not both have the kick flag at 0 are prioritized.
• - When choosing a chorus segment as segment x, the last chorus segment before the occurrence of a verse is chosen.
• - When selecting a versesegments as segment y, the first versesegment is preferably chosen after the occurrence of the intro or after the occurrence of a chorus segment.
• When selecting a verse segment as segment y, the first verse segment is preferably chosen which has the voice flag at 1.

Technical parameter

• - Es wird eine Segmentkombination bevorzugt, welche hinsichtlich des durchschnittlichen RMS-Pegels am ähnlichsten von allen anderen Kombinationen ist.
• - Es wird eine Segmentkombination bevorzugt, welche sich hinsichtlich des durchschnittlichen RMS-Pegels im Frequenzband Low am ähnlichsten von allen anderen Kombinationen ist.
• - Es wird eine Segmentkombination bevorzugt, welche eine hohe klangliche Ähnlichkeit aufweist.

The technical parameter means the following:

• A segment combination which is most similar in terms of average RMS level to all other combinations is preferred.
• A segment combination is preferred which, in terms of the average RMS level in the frequency band Low, is most similar to all other combinations.
• - A segment combination is preferred which has a high similarity in sound.

The four factors are correlated. The user activity factor determines the approximate time of the transition (within the first 25-50% of song A, 0-25% of song B). The semantic parameter influences the prioritization of possible segment combinations during this period. The content parameter determines a selection from a set of segment combinations with the highest priority. The size of the set depends on the values of the semantic parameter. This selects all segment combinations with ratings above a user activity threshold. The final selection of a segment combination is determined by the technical parameter. This selected segment combination is defined as a transitional period and used in the transitional design.

The user can manually request a next clean transition at any time. For this purpose, the four factors user activity factor, semantic parameters, content parameters and technical parameters are correlated as described. However, in this case the currently played song is set as the earliest transition time. The latest transition time results from the sum of the current time and a maximum waiting offsets depending on the user activity. This maximum waitoffset becomes shorter with higher user activity. Within the period thus determined, the possible segment combinations are prioritized using the semantic and content-related parameters, and the best segment combination, taking into account the technical parameter, is selected as the transitional time.

In order to increase the music experience, a random machine is used for the selection of the transition period, which can vary the time window in both songs, for possible segment selection, which is determined by the user activity factor, within a tolerance space. The tolerance space is not extended linearly in both directions with increasing user activity factor.

Construct transition

• - die Auswahl einer abstrakten Übergangsform und
• - die situationsabhängige Anpassung der abstrakten Übergangsform.

The phase of the transitional construction is divided into:

• - the selection of an abstract transitional form and
• - the situation-dependent adaptation of the abstract transitional form.

The selection of an abstract transitional form

• - Auswahl in Abhängigkeit des Genres,
• - Auswahl in Abhängigkeit der Semantik,
• - Auswahl in Abhängigkeit Useraktivität,
• - Auswahl in Abhängigkeit persönlicher Präferenzen, Auswahl in Abhängigkeit gesellschaftlicher Trends und
• - Auswahl in Abhängigkeit der Verwendungen.

The selection of an abstract transitional form from a set of predefined, abstract transitional forms occurs chronologically in the following phases. Each phase filters the current selection of abstract transitional forms:

• - selection depending on the genre,
• - selection depending on the semantics,
• - selection depending on user activity,
• Selection according to personal preferences, selection depending on social trends and
• - Selection depending on the uses.

An abstract transitional form describes a transitional procedure between two digital audio signals. An abstract transitional form is stored within the personal set of rules, but can be modified. This is done by the provider of the software.

• - nichtleeres Tupel von jeweils einer nichtleeren Menge von Genres,
• - nichtleeres Quadrupel von jeweils einer nichtleeren Menge an Segmenten,
• - nichtleeres Quadrupel von jeweils einer nichtleeren Menge von erlaubten Kick Flags für die Segmente,
• - nichtleeres Quadrupel von jeweils einer nichtleeren Menge von erlaubten Voice Flags für die Segmente,
• - Aktivitätsfaktor,
• - allgemeiner Popularitätsfaktor,
• - persönlicher Popularitätsfaktor und
• - Information über die Anzahl der Verwendungen innerhalb dieser Benutzungseinheit.

An abstract transitional form consists of meta information as well as content data. The meta-information contains the following information:

• - non-empty tuples of one non-empty set of genres,
• - empty quadruple of each one non-empty set of segments,
• nonempty quadruple of each of a nonempty set of allowed kick flags for the segments,
• non-empty quadruple of each one non-empty set of allowed voice flags for the segments,
• - Activity factor,
• - general popularity factor,
• - personal popularity factor and
• - Information about the number of uses within this unit of use.

• - allgemeinen Popularitätsfaktor und
• - persönlichen Popularitätsfaktor

für eine abstrakte Übergangsform sind in dem persönlichen Regelwerk definiert und ergeben sich ursprünglich aus den persönlichen Präferenzen, sowie den allgemeinen Präfenzen.The meta-information:

• - general popularity factor and
• - personal popularity factor

for an abstract transitional form are defined in the personal set of rules and arise originally from the personal preferences, as well as the general prefaces.

• - Taktlänge der Übergangsdauer,
• - relative Position der Klimaxoffsets k₁, k₂,
• - abstrakte Funktion des Lautstärkeeffekts,
• - abstrakte Funktion der EQ-Aktivität pro Band (Low, Mid, High),
• - abstrakte Funktion der Hoch/Tiefpass-Filteraktivität,
• - abstrakte Funktion der Aktivität verschiedener Parameter weiterer DJ- Effekte,
• - (Informationen zur dynamischen Modifikation der Einspielung von Liedern (Loop, Scratch, Cue Effekte)) und
• - (Information zur Einspielung von Audiosamples: (Klimaxoffset k3, Audiosample Id). Alle Einträge bis auf die letzten beiden müssen ausgefüllt sein.

The content consists of abstract information for the implementation of a transitional procedure, which relatively describes the modification of the fading audio signals:

• - cycle length of the transition period,
• relative position of the climatic offsets k ₁ , k ₂ ,
• - abstract function of the volume effect,
• - abstract function of EQ activity per band (Low, Mid, High),
• - abstract function of high / low pass filter activity,
• - abstract function of the activity of various parameters of other DJ effects,
• - (Information on the dynamic modification of the recording of songs (loop, scratch, cue effects)) and
• - (Information about the recording of audio samples: (Climate offset k3, Audiosample Id) All entries except the last two must be completed.

The available effects of the audio signal processing methods are chosen taking into account the effects available to human club and radio DJs. For this reason, a 3 band EQ and a high / low pass filter similar to the 3 band EQ and the high / low pass filter on a DJ mixer were chosen.

• - Phaser
• - Flanger
• - Beatcrusher
• - Gater

The above-mentioned other DJ effects are digital simulations of typical audio effects that DJs apply to songs using analog or digital processors or software. A selection of these is presented here, but is expandable:

• - Phaser
• - Flanger
• - beat crusher
• - Gater

• - Ineinandermischen zweier Extended Parts in House Liedern
• - „Wechsel der Bassdrum“ in House Liedern: Anwendung eines Hochpassfilters auf Lied A für einige Takte zum „Muten“ der Bassdrumimpulse in dem Kick Flag 1 Segment von Liedern A, während eines Kick Flag 0 Segments in Lied B, kurz vor Beginn des Kick Flag 1 Segments in Lied B.
• - beliebig langes Ineinandermischen beliebiger Segmente in Techno Liedern
• - Anwendung eines halbierenden Loops auf das Segment x in Lied A kurz vor dem Beginn des Segment y in Lied B, welches ein Chorussegment darstellt.
• - Scratch Effekte auf den ersten Beat eines R&B Liedes
• - langsame Steigerung der Frequenz eines Hochpassfilters für einen Übergang zwischen House Liedern
• - ein Takt nach dem letzten Bassdrumimpuls in Liedern des Genres Other, das nächste Lied schnell hereinmischen
• - Anwendung eines der blitzartigen Übergangsformen für Lieder des Genres Other, welche fast keine Kick Flags auf 1 haben

Some examples of abstract transitional forms, which can be realized by using different DJ effects, are listed below:

• - Mixing up two Extended Parts in House Songs
• - "Changing the Bass Drum" in House Songs: Applying a high-pass filter on song A for a few bars to "mute" the bass drum pulses in the kick flag 1 Segment of songs A, during a kick flag 0 segment in song B, just before the kick flag starts 1 Segments in song B.
• - Any length of mixing of any segments in techno songs
• Applying a halving loop to segment x in song A just before the beginning of segment y in song B, which is a chorus segment.
• - Scratch effects on the first beat of an R & B song
• Slowly increase the frequency of a high pass filter for a transition between house songs
• - one bar after the last bass drum impulse in songs of the genre Other, quickly mix in the next song
• - Applying one of the flashy transitional styles for songs of the genre Other, which have almost no kick flags on 1

The set of abstract transitional forms is expandable. The effort to define an abstract transitional form is very small. Conceivable here are API methods to extend the abstract transitional forms, not only by the provider of the software, but also by music enthusiastic users or human club or radio DJs.

By a relative curve is meant a function which represents the relative time course of the effectiveness of the respective audio signal processing method. The effectiveness is shown as a function. The function arguments represent the time axis and represent relative values from 0 to 1. The function values represent the value of a parameter for the respective algorithm within the audio signal processing method, relative to values from 0 to 1.

Selection depending on the genre

Based on the genre combination of the songs to be blended, a selection from the set of predefined, abstract transitional forms is chosen. For this purpose, the more detailed genre used in advanced song analysis is used. The meta-information of the transitional forms is checked for an entry of this gene recombination and added upon identification of such an entry.

For each combination of the extended song analysis resulting genre of the two songs at least one transitional form is defined.

Selection depending on the semantics

With the term transition time, the time stamp is defined exactly between k ₁ and k ₂ . With this term, the nearest segments within the songs can be indirectly described around the transition time.

• - Lied A stellt das auszublendende Lied dar.
• - Lied B stellt das einzublendende Lied dar.
• - x0 stellt die Segmentbezeichnung (ohne ID) des Segmentes vor dem Übergangszeitpunkt des Liedes A dar.
• - x1 stellt die Segmentbezeichnung (ohne ID) des Segmentes nach dem Übergangszeitpunkt des Liedes A dar.
• - y0 stellt die Segmentbezeichnung (ohne ID) des Segmentes vor dem Übergangszeitpunkt des Liedes B dar.
• - y1 stellt die Segmentbezeichnung (ohne ID) des Segmentes nach dem Übergangszeitpunkt des Liedes B dar.

The transition forms are selected, which in the meta information have an entry with respect to the segment combination (x0, x1, y0, y1).

• - Song A represents the song to be hidden.
• - Song B represents the song to be played.
• - x0 represents the segment name (without ID) of the segment before the transition time of the song A.
• x1 represents the segment name (without ID) of the segment after the transition time of the song A.
• - y0 represents the segment name (without ID) of the segment before the transition time of the song B.
• y1 represents the segment name (without ID) of the segment after the transition time of the song B.

In addition, in the meta information of the transition, it is checked whether the transitions in the meta information have a suitability with regard to the set kick flag and voice flag for the segment combination (x0, x1, y0, y1).

Selection depending on user activity

The filtered amount is sorted according to the activity factor. The selection of transitional shapes is selected that matches the user activity factor within a specified tolerance.

Selection depending on personal preferences

Selection depending on social trends

From the personal popularity factor, the general popularity factor, a popularity grade 'g' is calculated. The filtered amount is sorted in terms of popularity. A set of transitional forms from the top entry to a particular bottom entry is selected. The size of the crowd increases with higher user activity.

Selection depending on the uses

Finally, the filtered amount of transitions are sorted in terms of the number of uses during the current usage unit, and one of the less-played transitional forms is randomly used.

The situation-dependent adaptation of the abstract transitional form

The selected transitional shape is adapted to the real transient situation, so that an audio signal processing method can individually modify the two to be crossed songs according to the parameters and mix them together.

The transition period describes the length between the beginning of the recording of the song B, as well as the end of the playback of the song A in bars. In addition, the position of the Klimaxoffsets k ₁ for the song A, and the Klimaxoffsets k ₂ for the song B is given within this cycle. The recording of the song B begins from the clock t ₂ within the song B, which is the clock number k ₂ from the left boundary of segment y1 away. The recording of the song B begins from the clock t ₁ within the song A, which is the clock number k ₁ from the left boundary of segment x1 away. The modification of song B begins with the import and ends after the length of the transition period. Then song B is played further. The modification of Song A starts from the clock t ₁ and is terminated after the clock number of the transition period. At this point song A is stopped.

The function for the parameters of the abstract transition must now be adjusted. The value set is between 0 and 1 and represents the effectiveness of the respective algorithm in the audio signal processing method. The functions of the parameters are now adapted to the technical parameters of the songs. For example, the volume curve function is adjusted for the average RMS levels of the segments. The abstract volume curve is referenced to a particular RMS level for each segment (x0, x1, y0, y1). Depending on the calculated RMS level of the segments, the volume function is adjusted. The same applies to the function for the effectiveness of the respective bands of the EQ. Each EQ band is referenced to an RMS level. Depending on the RMS levels of the individual frequency bands of the segments, the function is adjusted to compensate for the RMS level differences. Ex .: If the difference of the real RMS level differences between segment x0 and y1 and the referenced RMS level differences is very high and the real RMS level difference between the two segments x0 and y1 is negative, the function would be influenced as follows: The volume increase of song B in segment y1 would grow faster to compensate for the audible drop in volume. The causality between changing the functions of the parameters for different effects and the technical parameters of the abstract transitions is defined within the software.

Make transition

First, the advanced song analysis data is loaded for the current playing song and the song immediately after it, if they have not already been loaded on the client. If these are not available, the two songs undergo an extended song analysis. The resulting data is loaded into the online database. Following this, the information from the extended song analysis is loaded from the online database for all other songs of the queue, if they are not already available on the client. Then the mission planning is carried out. If these extended song analysis data are not available for each song in the queue, the advanced song analysis is performed asynchronously to the basic song analysis on all songs in the queue. Because advanced song analysis may change the entries in the genre of songs, it may re-run mission planning after a certain number of songs have undergone an extended song analysis.

After performing mission planning, the transitional construction process is performed on all consecutive songs for which information from the extended song analysis is already loaded.

It is assumed that song A is currently playing and is currently in the audio buffer. The stream of the audio signal of song B is first normalized. Subsequently, the audio signal of song B is modified in speed so that song B is raised from the original speed of B to the beginning of the transitional period to a speed requested by mission planning v1. Song A and song B are made equal in speed during the transitional period so that song A and song B are synchronized in parallel during the transitional period. Song A and Song B have the velocity v1 at the beginning of the transitional period and a velocity v2 required by the mission planning after the transitional period. After the end of the transitional period, song B continues at speed v2. The used effects, which are noted in the meta information of the abstract transitional form, are applied to the songs A and B by means of audio signal processing. The function values of the functions of a corresponding effect are transferred with a defined sampling rate to the algorithms of the audio signal processing, which individually modify the audio signals of the songs during the transitional periods. A new audio signal C is generated from the mixture of the modified songs A and B, to which the rest of the modified song B is added. In a defined period of time before the start of the transitional period in song A, the audio signal C is loaded into the audio buffer at the corresponding position. At this position, the audio signal C thus replaces the audio signal of song A. The audio signal of song A is nevertheless temporarily stored until the transition is over, if the user interrupts the transition.

The user has the possibilities to interrupt a transition. In this case, the original audio signal from song A is loaded into the audio buffer. Nevertheless, the song A is blended with the latest possible transition to song B.

• - Blitzübergang oder
• - nächstmöglicher, sauberer Übergang.

The user has the possibility to request a transition at any time. He can choose between two types:

• - Lightning transition or
• - next possible, clean transition.

Lightning transition

With a lightning transition is meant a transition, which is universally applicable to any song and makes a temporary quick change between two songs, with a low transition quality is accepted in the transition. Examples include the following effects: tape stop, delay, filter, non-complex volume crossover. This transition will be executed immediately upon request at the next clock start.

Next possible, clean transition

By the next possible, clean transition is meant the next possible transitional time, which is considered suitable only according to content and semantic criteria.

The present invention also relates to software for automatically analyzing user activity when using a music player, preferably within an automobile. The user activity can be used, for example, as a parameter in different phases of a music player with automatic DJ function. Influencing the functioning of the automatic DJ function based on the user activity results in an increased musical experience, as it is adapted to the situation-dependent needs of the user.

The invention described herein relates particularly to an implementation in the automobile and the advantages resulting therefrom.

State of the art

From the state of the art, no technologies are known which analyze the user activity when using a music player and driving in an automobile to influence the operation of a music player with automatic DJ function based on the information obtained or otherwise improve the music experience.

A queue represents a list of songs in which the playing order of the songs can be determined.

• - das Hinzufügen und Löschen von Liedern in einer Queue,
• - die Abspielreihenfolge der Queue verändern,
• - das Vor- und Zurückspulens innerhalb von Liedern,
• - den Wechseln zum in der Queue nächsten Lied oder einem beliebigen Lied und
• - das Pausieren und erneute Abspielen eines Liedes.

This software is based on an implementation within a music player that has the following functionality:

• - adding and deleting songs in a queue,
• - change the playing order of the queue,
• - fast forwarding and rewinding within songs,
• - the change to the next song in the queue or any song and
• - pausing and playing a song again.

Although it is not necessary for the software described here, it describes the implementation of the software within a music player which provides an automatic DJ function. Without this functionality, only the functionality of the invention described herein would be limited.

Such a music player with automatic DJ function is hereinafter referred to as automatic DJ software.

• - Unterbrechen des automatisch eingeleiteten Überganges durch die Software,
• - manuelles Einleiten eines automatischen Überganges zwischen dem aktuellen Lied und dem Folgelied in der gegebenen Abspielreihenfolge,
• - manuelles Einleiten eines Blitzübergangs zwischen dem aktuellen Lied und dem Folgelied in der gegebenen Abspielreihenfolge und
• - die Möglichkeit den Mix Modus zu verändern.

The automatic DJ software also offers the user the following manual intervention:

• - interrupting the automatically initiated transition by the software,
• manually initiating an automatic transition between the current song and the following song in the given play order,
• - Manual initiation of a flash transition between the current song and the following song in the given play order and
• - the possibility to change the mix mode.

• - Benutzungsverhalten des Musikplayers,
• - Fahrverhalten des Users und
• - umweltbedingter Kontext

automatisch einen Useraktivitätsfaktor.The present invention calculates using the following parameters:

• - usage behavior of the music player,
• - Driving behavior of the user and
• - environmental context

automatically a user activity factor.

• - die Auswahl einer Menge an Liedern (nicht in der automatischen DJ Software beschrieben),
• - die Reihenfolge einer gegebenen Menge an Liedern,
• - die Wahl eines Übergangszeitpunktes und
• - die Wahl einer abstrakten Übergangsform.

For example, the calculated user activity may affect the following features of the automatic DJ software:

• - the selection of a lot of songs (not described in the automatic DJ software),
• the order of a given amount of songs,
• - the choice of a transitional period and
• - the choice of an abstract transitional form.

The software uses information about the segment structure of the songs from a song segmentation as an integrated technology of the automatic DJ software for some of the analysis steps described below. Song segmentation is therefore considered as presupposed for this invention.

Embodiment of the invention:

The analysis methods described below can be used to affect various functionalities of an automatic DJ software. The aim is to dynamically adapt the functionalities of the automatic DJ software to the needs of the user.

• - häufigerer Wechsel zwischen Liedern verschiedener Genreklassen innerhalb eines Zeitraums,
• - ausgehende Übergänge zu früheren Zeitpunkten im Lied,
• - eingehende Übergänge zu späteren Zeitpunkten im Lied und
• - Häufigeres Auftreten von als aufregend empfundenen Übergängen.

A high level of user activity, for example, leads to an exciting functioning of the automatic DJ software. This may affect the operation of the automatic DJ software, among others, as follows:

• - more frequent changes between songs of different genre classes within one period,
• outgoing transitions to earlier times in the song,
• - detailed transitions to later times in the song and
• - More frequent occurrences of transitions that are perceived as exciting.

By "user activity" is meant the following:

The calculated mood of a user who moves between the two poles, active and less active. The mood of a user could be divided into even more precise moods. Exemplary moods are shown on the Circumplex of Model Affect. Since this more accurate classification of the tuning is not necessary for most phases of the automatic DJ software, here only the relationship between the poles is determined active and less active. Thus, only the vertical axis of the activity is used on the Circumplex of Model Affect.

The user activity represents a real number between 0 and 1.

The current user activity is calculated during a use unit in a defined sampling rate and reset at the next use unit and initially recalculated.

The calculation of user activity includes changes in the influencing factors. The operation of the automatic DJ software thus changes dynamically during a use unit to be optimally adapted to the needs of the user.

For each user of the software, a separate profile is created on the client. For the profile used, for each usage unit, all information that influences the value of the user activity is stored in the sampling rate. In addition, all information which has influenced the user activity determination in past usage units is loaded from the user profile.

All parameters are related to the average of previously stored information of the respective user profile. Each parameter is compared with the mean value from all previous units of use and the increasing or reducing factor is calculated from the difference between the mean value and the current values of the influencing variables.

For example, if a user has a high activity regarding his behavior of manually creating transitions, as he has, for example, manually created 25 of 50 transitions between songs in that usage unit, this can be interpreted as high user activity in the current usage unit, if the user is on average per user unit in the past has only caused 2 out of 50 transitions between songs manually. Likewise, an aggressive acceleration behavior of the vehicle by the user would only increase the user activity if the mean value of the acceleration behavior of past usage units is lower.

The new information of the influencing variables are stored, as already described, and thus influence the mean value in the next use unit.

With a high current user activity, it is thus indirectly described below that the current user activity is relatively high with the mean value of the measured user activity in past usage units.

In the following, user activity and user activity factor are used synonymously.

The manually configurable Mix mode, provides a graphical interface in which the desired style of the automatic DJ software can be set manually. Here a Ratioskala is used. The calculated user activity factor is shown in this Ratioscale. One pole stands for a minimal user activity, the other pole stands for a maximum user activity. Thus, the one pole stands for a setting of the functioning of the DJ software, which is optimal for a low activity of the user. The other Pol represents a setting of how the DJ software works, which is optimal for high user activity. The value taken from the ratio scale thus represents the parameter for influencing the automatic DJ software.

The calculated user activity factor can be changed manually by the user within this graphical interface. This user activity factor is maintained throughout the usage unit. The next time the software is used, the manually set value is reset and the user activity factor is automatically re-determined and dynamically adjusted by the software. Due to the way the automatic DJ software works, the poles could be called "party" and "lounge" modes. The "Party" mode leads to an exciting, very dynamic music experience, the "Lounge" mode to a relaxed, even music experience. Accordingly, mixtures between the two poles are possible.

Usage behavior of the music player

• - die Aktivität der Modifikation der Queue,
• - die Art der Auswahl der Lieder,
• - das manuelle Spulverhalten innerhalb eines Liedes,
• - die Quantität der manuellen Wechsel des aktuell spielenden Liedes zu einem beliebigen Lied,
• - die Quantität des manuellen Anforderns eines Übergangs zu dem Folgelied,
• - der Zeitpunkt des manuellen Anforderns eines Übergangs zum Folgelied und
• - die Quantität des manuellen Unterbrechens eines Übergangs zu dem Folgelied.

The usage behavior of the music player influences the user activity factor and is calculated from the following components:

• the activity of the modification of the queue,
• - the way of choosing songs,
• - the manual winding behavior within a song,
• the quantity of manual changes of the currently playing song to any song,
• the quantity of manually requesting a transition to the following song,
• the time of manually requesting a transition to the follower song and
• the quantity of manually interrupting a transition to the following song.

The activity of the modification of the queue.

If the queue of titles to be played is often modified, this suggests that the user has a high interest in an optimal music experience. This particular activity of the user is used as a positive factor to calculate the activity of the user.

Modifying the queue means manually adding or deleting a title in the queue or manually changing the play order.

In the following, songs whose play order has been changed in the queue or which have been manually added to the queue by the user are described with the following term: "Songs which have been changed with regard to the queue."

Deleted songs are not considered here. Songs added automatically to the queue and not manually moved in order or added to the queue by the user are also disregarded.

The type of selection of songs.

The basic parameters: speed and genre of songs in the queue are considered. It is important that only the variance in speed and genre between the songs, which were changed with regard to the queue, is considered. In addition, the number of favorite songs, within the songs, which have been changed with respect to the queue, is taken into account.

When there is a high variance between the genre classes and velocities of songs that are changed in queue, this is incorporated into the user activity as an increasing factor. Thus, if in the queue often songs with distant genre classes, and speed classes changed with respect to the queue, this flows as an increasing factor in the user activity.

The distance used to measure variance is calculated from the distances shown between genre classes and velocity classes.

If there is a high variance between the genres and velocities of two directly consecutive songs that have been changed with respect to the queue, this also indicates high user activity. If the user thus manually changes the queue, so that often a fast song is followed by a slow song and even a fast song afterwards, this is included as an increasing factor in the user activity.

The manual winding behavior within a song

Spooling behavior within a song means that the user, while a song is being played, rewinds or rewinds in the song. This is justified by the fact that the user particularly likes the part of the song or does not like it, which is currently being played, and that the user has a high interest in his music experience and therefore actively intervenes. Therefore, such behavior also flows into the user activity as an increasing factor.

The quantity of manual changes of the currently playing song to any song A high quantity of manual changes of the currently playing song to any song flows into the user activity as an increasing factor.

The quantity of manually requesting a transition to the follower song

A high quantity of manually requested transitions between the currently playing song and the subsequent song flows into the user activity as an increasing factor.

The time of manually requesting a transition to the follower song.

• Das Anfordern eines Übergangs zu früheren Zeitpunkten im Lied als von der automatischen DJ Software vorgeschlagen, erhöht die Useraktivität.
• Das Anfordern eines Übergangs zu späteren Zeitpunkten im Lied als von der automatischen DJ Software vorgeschlagen, reduziert die Useraktivität.

The timing of manually requesting an automatic transition to a follow-up song plays the following role:

• Requesting a transition to earlier times in the song as suggested by the automatic DJ software increases user activity.
• Requesting a transition to later times in the song as suggested by the automatic DJ software reduces user activity.

The Quantity of Manually Pausing a Transition to the Folder Song Analogously to manually requesting a transition, the quantity of manually pausing a transition executed by the automatic DJ software may also be considered. The frequent interruption of transients indicates that the user considers the high activity of the automatic DJ software to be too active. This flows into the user activity as a reducing factor.

Driving behavior of the user

The following section deals with the causality between driving behavior and user activity, if the software is implemented within a vehicle. For this purpose, access to the braking behavior, acceleration behavior, steering angle and the speed of the corresponding vehicle is essential. The existence of a system for determining the maximum permitted speed in the current driving situation can improve the analysis functionality, but is not essential.

• - Dynamik im Bremsverhalten,
• - Dynamik im Beschleunigungsverhalten,
• - Dynamik in der Querführung des Fahrzeuges und
• - Geschwindigkeit des Fahrzeuges in Vergleich mit fahrsituationsabhängiger, erlaubter Höchstgeschwindigkeit.

The analysis of the driving behavior initially amounts to the following factors, but is expandable:

• - dynamics in braking behavior,
• - dynamics in the acceleration behavior,
• - Dynamics in the transverse guidance of the vehicle and
• - Speed of the vehicle in comparison with driving situation-dependent, permitted maximum speed.

It is only distinguished between the characteristics of sporty and quiet driving, with any gradation between the two poles is possible. Sporty behavior results from an aggressive braking behavior, an aggressive acceleration behavior, as well as a strong steering angle at high vehicle speeds. A speed over allowed Top speed also does not increase linear the tendency to a sporty driving behavior. Sporty driving behavior influences the user activity as an increasing factor. Quiet driving affects user activity as a reducing factor.

Environmental Context

The context-free user activity in the following refers to the calculated user activity before the influence of environmental factors.

The context-related user activity in the following refers to the calculated user activity after the influence of environmental factors.

The environmental context refers to external parameters related to weather, driving situation and time of day.

The weather situation can be determined via an external service using the GPS position of the terminal. Here, a distinction is made between simple weather situations which strongly influence the mood of the user, e.g. Rain, Mist, Hail, Sun, Cloudy.

The driving situation can be determined via the car's map system or via the terminal's GPS position via an external map system. The road type is relevant here. There is a distinction between: city streets, highways and highway. In an implementation outside the car, further classifications are conceivable.

The time of day is determined by the time of day. Here it is easy to distinguish between morning, morning, afternoon, evening and night. It is also possible to adjust the time limits of the day to the personal habits manually by the user or to capture them automatically using external software.

In the user profile, the user activity is stored for each context.

If the user is again in a situation with a corresponding context, the average user activity for this context is loaded from past usage units and is related to the currently determined context-free user activity. The contextual user activity contains the product of the context-free user activity and a factor f. If the average user activity for this context from past usage units exceeds the current context-free user activity, the factor f> 1. If the average user activity for this context from past usage units falls short of the current context-free user activity, the factor 0 <f <1.

In addition, the general user activity information for this context is referred to by the backend and additionally adds or reduces the contextual user activity if the general user activity for that context is higher or lower than the current contextless user activity.

Context-free user activity is stored for the current context for this usage unit. Although the average user activity for this context from past usage units is used to influence the value of the current contextual user activity, it does not store the contextual user activity to avoid feedback.

For example, if the user is in a rainy evening, driving in the evening and driving on a country road, the average user activity of past usage units for that context is loaded. If this is higher than the current context-free user activity, the value of the context-related user activity is artificially increased.

However, only the current context-free user activity for this usage unit is saved.

The user activity that ultimately affects the functionality of the client systems of the automatic DJ software is the contextual user activity.

LIST OF REFERENCE NUMBERS

1

speaker

2

electronic control unit

3

touchscreen

5

data storage

6a, 6b, 6c

Audio file

7

header

8th

Audio material

9

audio signal

10

car

11

crossing

12

Intro

13

verse

14

Chorus / Chorus

15

Outro

16

Client system

17

Backend system

18

Client Systems

19

using behavior

20

Smartphone

21

driving behavior

22

environmental context

23

personal preferences

24

personal rules

25

general preferences

26

general rules

27

user

28

ranking

29

mission planning

30

audiogram

31

Transition times

32, 33

audiograms

34

The transition structure

35

circle

36

Transitional implementing

37

advanced song analysis

38

basic song analysis

39

advanced song analysis

40

meta

41

grade calculation

42

clustering

43

first cluster

44

Clustering order

45

internal order

46

queue

47

manual modification of the queue

48

Change of user activity

49

macro segmentation

50

microsegmentation

51

Genre classification

52

Chorus segments

53

semantic macrosegmentation

54

Transfer factors

55

regulator

56

manual adjustment

57

Screen display user interface

58, 59

Segmented audiograms with segment names

60

Transition times under the influence of user activity

61

Transition times under the influence of personal rules

62

Final transitional time under the influence of technical parameters

Claims (14)

A method of designing a transition (11) in an audio reproduction (9) from a first audio file (6a) to a second audio file (6b), comprising the steps of: - Automatic examination (100) of the audio files (6a, 6b) with respect to an audio material contained in each case (8) and / or - Automatic examination (300) of a vehicle condition and - shapes (400) the transition (11) as a function of a result of the examination. Method according to Claim 1 wherein the shaping of the transition (11) - a temporal association between a reproduction of the first audio file (6a) and the second audio file (6b) and / or - a reproduction volume related action and / or - a resetting of a reproduction of the second audio file (6b) and / or - another signal-changing measure, in particular an audio effect defined. Method according to Claim 1 or 2 wherein examining the audio files (6a, 6b) comprises segmentation. Method according to Claim 3 wherein examining the audio files (6a, 6b) comprises categorizing the segments (12, 13, 14, 15). The method of any preceding claim, wherein designing the transition (11) comprises selecting a first segment (14) within the audio material (8) of the first audio file (6a) and a corresponding second segment (14) within the audio material (8) of the second audio file (6b). Method according to one of the preceding claims, wherein as a vehicle condition or in addition to the vehicle condition a driving situation and / or a vehicle situation and / or - An operating situation of a means of transport (10) in the design of the transition (11) is taken into account. Method according to one of the preceding claims further comprising: - Determining (500) a music reproduction behavior of a means of locomotion (10), and depending on the music reproduction behavior - Shaping (600) of the transition (11) in a second means of transport. Method according to one of the preceding claims further comprising: - determining (200) that meta-information (7) of an audio file (6a, 6b, 6c) is not available. Method according to one of the preceding claims further comprising: Determining and evaluating a semantic relatedness and / or a semantic prioritization of the audio files (6a, 6b) and depending on a result of the evaluation - design of the transition (11). Device for designing a transition (11) during audio reproduction from a first audio file (6a) to a second audio file (6b) comprising: an evaluation unit (2) and - An output unit (1), wherein - The evaluation unit (2) is arranged to automatically examine the audio files (6a, 6b) with respect to an audio material contained in each case (8) and / or - to automatically inspect a vehicle condition and / or - to design the transition depending on a result of the investigation. Device after Claim 10 which is set up, a method according to one of Claims 1 to 9 perform. Computer program product comprising instructions which, when executed on a programmable processor, cause the processor to perform the steps of a method according to any one of the above Claims 1 to 9 perform. Signal sequence representing instructions which, when executed on a programmable processor, cause the processor to perform the steps of a method according to any one of the above Claims 1 to 9 perform. Means comprising a device according to one of Claims 10 or 11 ,

Images