WO2015154159A1 - Systèmes et procédés pour analyse musicale et détermination de compatibilité dans une production audio - Google Patents

Systèmes et procédés pour analyse musicale et détermination de compatibilité dans une production audio Download PDF

Info

Publication number
WO2015154159A1
WO2015154159A1 PCT/CA2014/050368 CA2014050368W WO2015154159A1 WO 2015154159 A1 WO2015154159 A1 WO 2015154159A1 CA 2014050368 W CA2014050368 W CA 2014050368W WO 2015154159 A1 WO2015154159 A1 WO 2015154159A1
Authority
WO
WIPO (PCT)
Prior art keywords
stem
stems
data
song
frequency
Prior art date
Application number
PCT/CA2014/050368
Other languages
English (en)
Inventor
Mark VESPRINI
Original Assignee
Vesprini Mark
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vesprini Mark filed Critical Vesprini Mark
Priority to PCT/CA2014/050368 priority Critical patent/WO2015154159A1/fr
Publication of WO2015154159A1 publication Critical patent/WO2015154159A1/fr

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/0008—Associated control or indicating means
    • G10H1/0025—Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
    • G10H2210/061—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for extraction of musical phrases, isolation of musically relevant segments, e.g. musical thumbnail generation, or for temporal structure analysis of a musical piece, e.g. determination of the movement sequence of a musical work
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/101—Music Composition or musical creation; Tools or processes therefor
    • G10H2210/125—Medley, i.e. linking parts of different musical pieces in one single piece, e.g. sound collage, DJ mix
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/101—Music Composition or musical creation; Tools or processes therefor
    • G10H2210/151—Music Composition or musical creation; Tools or processes therefor using templates, i.e. incomplete musical sections, as a basis for composing
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2220/00—Input/output interfacing specifically adapted for electrophonic musical tools or instruments
    • G10H2220/091—Graphical user interface [GUI] specifically adapted for electrophonic musical instruments, e.g. interactive musical displays, musical instrument icons or menus; Details of user interactions therewith
    • G10H2220/101—Graphical user interface [GUI] specifically adapted for electrophonic musical instruments, e.g. interactive musical displays, musical instrument icons or menus; Details of user interactions therewith for graphical creation, edition or control of musical data or parameters
    • G10H2220/106—Graphical user interface [GUI] specifically adapted for electrophonic musical instruments, e.g. interactive musical displays, musical instrument icons or menus; Details of user interactions therewith for graphical creation, edition or control of musical data or parameters using icons, e.g. selecting, moving or linking icons, on-screen symbols, screen regions or segments representing musical elements or parameters
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
    • G10H2240/075—Musical metadata derived from musical analysis or for use in electrophonic musical instruments
    • G10H2240/081—Genre classification, i.e. descriptive metadata for classification or selection of musical pieces according to style
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
    • G10H2240/075—Musical metadata derived from musical analysis or for use in electrophonic musical instruments
    • G10H2240/085—Mood, i.e. generation, detection or selection of a particular emotional content or atmosphere in a musical piece
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
    • G10H2240/121—Musical libraries, i.e. musical databases indexed by musical parameters, wavetables, indexing schemes using musical parameters, musical rule bases or knowledge bases, e.g. for automatic composing methods
    • G10H2240/131—Library retrieval, i.e. searching a database or selecting a specific musical piece, segment, pattern, rule or parameter set
    • G10H2240/141—Library retrieval matching, i.e. any of the steps of matching an inputted segment or phrase with musical database contents, e.g. query by humming, singing or playing; the steps may include, e.g. musical analysis of the input, musical feature extraction, query formulation, or details of the retrieval process
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
    • G10H2240/121—Musical libraries, i.e. musical databases indexed by musical parameters, wavetables, indexing schemes using musical parameters, musical rule bases or knowledge bases, e.g. for automatic composing methods
    • G10H2240/145—Sound library, i.e. involving the specific use of a musical database as a sound bank or wavetable; indexing, interfacing, protocols or processing therefor

Definitions

  • stems database Due to the limited number of available stems present in the stem database, it is highly likely that whatever stem the user chooses has already been used before in other productions, leading to a production that is not unique. The potential for lack of originality is also observed with the use of pre-produced catalogs, as the same production is licensed to several users. [0004] As the number of stems in a stems database continues to grow, it becomes increasingly difficult to navigate such a database and select a particular stem. Thus, it may be difficult and inconvenient for a user to select the desired stems, or to produce a complete musical work, especially, for example, when the user has little or no knowledge of musical theory, audio mixing, or audio mastering skills.
  • FIG. 1 is a system diagram showing an example embodiment of one or more servers connected to one or more user computing devices for the creation of music.
  • FIG. 2 is a block diagram of an example user computing device.
  • FIG. 3 is a block diagram of an example server configuration including a Music Composition Module (MCM).
  • MCM Music Composition Module
  • FIG. 4 is a schematic diagram of example stems in a stems database.
  • FIG. 5 is a block diagram of example audio effects in an effects database.
  • FIG. 6 is a flow diagram illustrating example processor executable instructions for the initial characterization of a stem file.
  • FIG. 7 is a flow diagram illustrating example processor executable instructions for the composition of an audio track.
  • FIG. 8 is a flow diagram illustrating another example of processor executable instructions for the composition of an audio track.
  • FIG. 9 is a schematic diagram of example Graphical User Interface (GUI) components for the selection of genre and subgenre.
  • GUI Graphical User Interface
  • FIG. 10 is a schematic diagram of example GUI components for the selection of tempo.
  • FIG. 1 1 is an alternative schematic diagram to FIG. 10 showing example GUI components for the selection of tempo.
  • FIG. 12 is a schematic diagram of example GUI components for the selection of emotion.
  • FIG. 13 is a schematic diagram of example GUI components for the selection of a song-starter.
  • FIG. 14 is a schematic diagram of example GUI components for the selection of layers.
  • FIG. 15 is a schematic diagram of example GUI components for the selection of percussion.
  • FIG. 16 is a schematic diagram of example GUI components for the configuration of song structure.
  • FIG. 17 is a schematic diagram of example GUI components for modifying the length of one or more song sections.
  • FIG. 18 is a schematic diagram of example GUI components for selecting an alternative song section.
  • FIG. 19 is a schematic diagram of example GUI components for adding effects.
  • FIG. 20 is a schematic diagram of example GUI components for accepting user feedback on stems.
  • FIG. 21 is a schematic diagram of example GUI components for accepting user payment for a finished production.
  • FIG. 22 is a flow diagram illustrating example processor executable instructions for characterizing a stem according to frequency fingerprint.
  • FIG. 23 is a flow diagram illustrating example processor executable instructions for obtaining a frequency fingerprint from a stem file.
  • FIG. 24 is a plot of an example spectral analysis as obtained from a stem file.
  • FIG. 25 is a flow diagram illustrating example processor executable instructions for the characterization of a stem according to melodic compatibility.
  • FIG. 26 is a flow diagram illustrating example processor executable instructions for obtaining melodic compatibility data from a stem file.
  • FIG. 27 is a flow diagram illustrating example processor executable instructions for recommending stem files based primarily on stem melodic compatibility data.
  • FIG. 28 is a flow diagram illustrating example processor executable instructions for the characterization of a stem according to temporal compatibility.
  • FIG. 29 is a flow diagram illustrating example processor executable instructions for obtaining temporal compatibility data from a stem file.
  • FIG. 30 is a flow diagram illustrating example processor executable instructions for using the frequency fingerprint to make a ranked suggestion of stems.
  • FIG. 31 is a flow diagram illustrating example processor executable instructions for using the frequency fingerprint to identify and rectify frequency and amplitude-related problems during the mixing and mastering stage of the production.
  • FIG. 32 is a flow diagram illustrating example processor executable instructions for using melodic compatibility data to identify and suggest compatible stems.
  • FIG. 33 is a flow diagram illustrating example processor executable instructions for using temporal compatibility data to identify and suggest compatible stems.
  • the term "user computing device”, as used herein, includes, but is not limited to devices or media players used to at least one of record, produce, and play any one or more of music, video, still images, or other media.
  • Other examples of user computing devices include personal computers, laptops, MP3 players, digital audio workstations, gaming consoles, cameras, cellular phones, tablets, personal digital assistants (PDAs), net books, other communication devices, and other types of computing devices.
  • PDAs personal digital assistants
  • Other electronic devices and equipment, or combination of those, are included in the term user computing device, insofar as the systems, methods and principles described herein are applicable.
  • the user computing device may include mobile devices and non-mobile devices.
  • stem as used herein, will be understood, but is not limited solely, to refer to an audio layer that is recorded or derived from digital instruments, live instruments, or vocal performances.
  • the stem is a single layer of audio data (e.g. a single instrument, a single vocal performance, etc.).
  • the stem is a single layer of audio data (e.g. a single instrument, a single vocal performance, etc.).
  • a stem includes multiple layers of audio data (e.g. multiple instruments, vocal performances, etc.).
  • ong-starter as used herein, will be understood, but is not limited solely, to mean a stem that is the fundamental stem in the whole of the production, from which other parts of the music production will be based on and compatible with.
  • textural compatibility will be understood, but is not limited solely, to mean a metric to determine compatibility of a given stem with one or more other stems based on texture or frequency characteristics.
  • An example of frequency characteristics is a frequency fingerprint.
  • meltodic compatibility as used herein, will be understood, but is not limited solely, to mean a metric to determine compatibility of a given stem with one or more other stems based on the musical key, scale, note or chord progression.
  • temporal compatibility will be understood, but is not limited solely, to mean a metric to determine compatibility of a given stem with one or more other stems based on the tempo (e.g. beats per minute), time signature, or whether the stem exhibits a swing to the beat.
  • frequency fingerprint will be understood, but is not limited solely, to include the frequency make-up, or spectrogram, and the amplitude averages of pre-defined frequency bins of a given audio signal, for example as obtained from the Fast Fourier Transform of an audio file.
  • beat sometimes used interchangeably with the term “percussion” herein, will refer to percussion-based instruments, including but not limited to drum kits, electronic drum kits, and beat sequencers.
  • scale refers to a grouping of a series of notes, each differing in pitch according to a specific scheme (e.g. the "heptatonic scale”, or the “pentatonic scale”).
  • key refers to the principal tonality of a musical work, consisting of a plurality of tones in a fixed relationship to a central tonic, and having a characteristic key signature (e.g. the "A major” key, or the "G minor” key).
  • Some of these reasons include the lack of modularity in the available stems and pre- produced content in current databases or libraries. Other reasons include the inability of the music production software or catalogs of pre-produced production to allow a user to customize specific properties of the instrumental stems or pre-produced tracks. For example, such properties may include genre, tempo, key, melody, emotion, song structure, song arrangement, or instrumentation.
  • music production software that does offer flexibility and some degree of customization, is typically too complex for a user with little musical training.
  • Existing music production software does not provide such a user with full control over aspects such as tempo, genre, mood, specific instrumentation, and exact song structure in a manner that is easy to use and intuitive.
  • Existing music production software typically do not include an automated mechanism to mix and master the final music product, which can be produced to be consistent with industry standards.
  • a database is provided for storing audio stems that are characterized for textural, melodic, and temporal compatibility in addition with providing other tagged metadata (e.g. genre, emotion, musical note data, and instrumentation information).
  • This data may be used to determine the compatibility of a given stem with another stem, or a combination of multiple other stems.
  • the software can subsequently filter the database for additional compatible stems and present ranked lists of stem recommendations to the user.
  • Such software can be implemented on, for example, a web application, a server, or personal computing devices (e.g. computers, laptops, tablet computers, smartphones).
  • the compatibility data may be derived from the musical note data (e.g. a MIDI file) stored in association with the stems.
  • the software is able to derive compatibility data from direct analysis of the audio file.
  • the specific properties of the musical production and of the stems are able to be customized by the user.
  • Example custom izations may include genre, tempo, key, melody, emotion, song structure, song arrangement, or instrumentation. Audio effects may also be applied.
  • the software analyzes the audio production to ensure that there are no undesirable audio artifacts (melodically, temporally, or in terms of unwanted or excessive frequency peaks) and that the production is mixed and mastered consistent with industry standards.
  • the system proposed herein allows users to control each layer of their production, in a manner that is intuitive.
  • the software for example, provides multi-criteria musical compatibility prediction and dynamic filtering of the stem database, which guides the user to produce a final product tailored to their specific desired qualities in terms of genre, tempo, key, melody, mood, exact structure/arrangement, and specific instrumentation. Due to the large size of the stems database and the amount of potential combinations and alternative configurations of stems, in addition to the potential to apply audio effects to specific layers for specific sections of the song, it is highly likely that the exact manner in which the stems are combined, arranged and effected will result in a final product that is unique to each user.
  • the software automatically excludes stems in the database based on incompatibility with the parameters and options selected by the user, the manner in which the filtering occurs is within the user's control.
  • the user makes decisions first broadly (e.g. genre, tempo, emotion) then specifically (e.g. which instrument to add, which chord progression to begin with, which melody to add) before moving onto the next stage.
  • decisions first broadly e.g. genre, tempo, emotion
  • specifically e.g. which instrument to add, which chord progression to begin with, which melody to add
  • the manner in which the user is guided through the production of their song significantly reduces time spent listening through numerous samples of irrelevant stems to find one that matches their parameters and specifications, thereby drastically reducing the time spent producing their song while at the same time allowing the user the control and flexibility they desire.
  • the user's specifications are used to present them with recommendations that fit the cumulative effect of the choices they have made up to that point.
  • artists receive emotional benefits from using the system proposed herein in the sense that they are able to inject their artistic identity into a unique creation that is a better reflection of the personal nature of the artistic process than the software currently available.
  • the benefit of being able to control the emotion of the music they are producing and control over tempo, structure/arrangement, specific instrumentation, key, choice of melody, and chord progression makes this proposed system a valuable tool.
  • a user is able to control the production of music having little or no musical training, and does not need to work with a producer, which can be both time consuming and vastly more expensive than the use of the proposed musical compatibility system.
  • the system categorizes incoming source material based on musical compatibility criteria and stores the characterized source material (or stems) in a database to facilitate the upload of new material by any user. It is herein recognized that there are existing online production catalogs that allow producers to submit fully produced instrumentals for the purpose of generating sales, but none of the existing websites or platforms permit the upload of single- layer source material for automatic characterization of musical compatibility.
  • the proposed system's ability to categorize and automatically predict compatibility of single-layer source material (e.g. stems) makes large-scale engagement of the producer population possible, by eliminating the need to spend time to listen to each stem individually.
  • the proposed system serves as a platform for the co-creation of value by various types of users.
  • the proposed system also reduces the laborious and time consuming process of producers seeking out artists to work with and vice versa.
  • any module or component exemplified herein that executes instructions may include or otherwise have access to computer readable media such as storage media, computer storage media, or data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape.
  • Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data.
  • Examples of computer storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by an application, module, or both. Any such computer storage media may be part of the user computing device, server, or database, or otherwise accessible or connectable thereto. Any application or module herein described may be implemented using computer readable/executable instructions that may be stored or otherwise held by such computer readable media.
  • FIG. 1 is a system diagram illustrating an example embodiment of one or more servers 101 connected to one or more various user computing devices through a communication network 102.
  • the user computing devices are generally herein referenced by numeral 103, but specific implementation of a user computing device are identified by an alphabetic suffix (e.g. "a”, "b", "c”, etc.).
  • Non-limiting examples of user computing devices 103 include computers 103a, tablets 103c, or smartphones 103b.
  • the server(s) 101 may be implemented locally, remotely, on a content distribution network, in the cloud, or in any other way accessible through the network.
  • the user computing devices are connected to the server 101 through a communication network 102.
  • Examples of this network may include, but are not limited to, directly attached, local area, peer-to-peer, ad-hoc, the Internet, and may be implemented wired or wirelessly.
  • server or system of servers 101 are herein simply referred to as a "server”, although a system of servers can be used.
  • a "user computing device” will herein be used, although multiple user computing devices can be used.
  • a user computing device may preferably include output devices 202 for outputting visual or audio feedback to the user, as well as the final audio production. Examples of such devices include displays, speakers, broadcasting equipment, audio/visual systems, recordable media, etc.
  • input devices 205 are preferably included to accept user commands. Such devices may include, but are not limited to, a mouse, keyboard, touchscreen, stylus, or other methods of issuing a user command or selection.
  • memory devices 203 are preferably included to store, either temporarily or permanently, the audio production as well as any associated and necessary data and instructions, including but not limited to stem audio, stem metadata, filtered database results, arrangement information, effects and parameters, visual and audio feedback data, and processor executable instructions for the software and system.
  • production peripherals 204 are preferably included to record, gather, collect, or otherwise obtain sounds, audio, note data, or other representation(s) of musical audio (such as, e.g., beat sequencer instructions for an electronic drum kit, trigger(s) for audio samples, etc.).
  • one or more communication devices 206 are provided to connect, or otherwise link, the user computing device with the server 101 through a communication network 102.
  • Software 207 is included to provide an interface to the user and to accept user commands, and may be implemented as a digital audio workstation (DAW) 208, music playback application 209, movie making application 210, music composition application 211 , Music Composition Module 212 (MCM, to be discussed later), or the like.
  • DAW digital audio workstation
  • MCM Music Composition Module
  • processors or central processing units 201 CPU is included to execute commands from the software, and to communicate with the various devices as mentioned.
  • FIG. 3 shows an exemplary block diagram of a server configuration 101 .
  • the server preferably includes communication devices 31 1 , storage devices 312, Input/Output devices 313, one or more processors 314, and memory 315. Additionally, the server may include a Music Composition Module (MCM) 212.
  • MCM Music Composition Module
  • the MCM includes a Categorization Module 301 , a Temporal Compatibility Module 304, a Melodic Compatibility Module 302, an Emotional Compatibility Module 303, a Textural Compatibility Module 306, a Graphical User Interface (GUI) module 305, an Effects Database 307, a Stems Database 308, and an Effects Module 316.
  • GUI Graphical User Interface
  • the Stems Database 308 stores the stem audio files (such as, for example, instrumental stems 309 or vocal stems 310) and the associated metadata and characterization data. Note that this is an example implementation of the MCM, and other implementations may include different combinations of a plurality of such modules, or include additional modules, as will be appreciated by those skilled in the relevant art.
  • the MCM may be implemented on the server 101 (as exemplified in FIG. 3), or on a user computing device 103.
  • a Categorization Module 301 is configured to categorize stems being uploaded or stored on the server 101 .
  • a Melodic Compatibility Module 302 is configured to determine the melodic compatibility (e.g. as based on key, chord progression, or scale) of a stem with another stem, or with a combination of other stems.
  • An Emotional Compatibility Module 303 is configured to determine the emotional compatibility (e.g. as determined by the similarity of the stem's emotional traits compared to the user's specifications) of a stem with another stem, or with a combination of other stems.
  • a Temporal Compatibility Module 304 is configured to determine the temporal compatibility (e.g.
  • a Textural Compatibility Module 306 is configured to determine the textural compatibility (e.g. as based on the frequency distribution or amplitude peaks throughout time) of a stem with another stem, or with a combination of other stems.
  • a GUI Module 305 is configured to provide the user with the necessary GUI components in the functioning of the system and methods as described later. It will be appreciated that GUI Module 305 is not limited to displaying a visual interface to the user. For example, the GUI Module 305 may use audio or tactile feedback to augment, or totally replace, the visual interface in certain example embodiments of the present invention. Additionally, an Effects Module 316 is configured to apply one or multiple audio effects to incoming audio signals.
  • FIG. 4 depicts an example schematic diagram of stems in the MCM stems database 308.
  • the stems are stored in association with stem metadata and stem
  • stem metadata may include, but are not limited to, genre 402, artist, album, instrument 409, song-starter information 401 (to be discussed later), etc.
  • stem compatibility data may include, but are not limited to, beats per minute (BPM) 403, time signature 404, textural compatibility data or frequency fingerprint 405, melodic compatibility data 406, emotion 407, key 408, etc.
  • musical note data 410 e.g. MIDI
  • additional metadata, compatibility data, or other relevant identifying information may be included in the stems database.
  • a given stem may indicate a stem ID 400, such as a number or other unique identifier.
  • the song-starter value 401 may be a Boolean indicator which identifies whether or not the given stem is a song-starter.
  • the genre 402 for a given stem may include, for example, any of Pop, Rock, Hip-Hop, jazz, Classical, electronic dance music (EDM), or any other genres of music, or combinations thereof.
  • the BPM 403 may be a numeric value (e.g. 120 bpm).
  • the time signature value 404 may indicate the time signature of the underlying music in a stem, such as, for example, 4/4, 8/8, 6/8, 9/8, etc.
  • the textural compatibility data or frequency fingerprint 405 may contain processor-readable information describing the frequency distribution of a stem throughout the duration of the stem.
  • the melodic compatibility data 406 may contain a series of notes, chords, scales, or progressions thereof that may be used, either individually or in combination, to calculate for the melodic compatibility of a stem with other stems.
  • the emotional compatibility data 407 may contain emotional tags that describe the general subjective feeling expressed by the stem.
  • the key data 408 may include the predominant musical key or keys that a stem is recorded in.
  • the instrument data 409 may include information about the instrument(s) that are featured in the stem.
  • the instrument data 409 may include information about the categorization of particular instruments, such as, for example, a "Bass Synth” instrument being classified under the "Synth” instrument category.
  • the note data 410 may be stored in association with the stem, and may include information about the arrangement and timing of the notes, chords, or other musical information in a particular stem.
  • the note data 410 may be stored, for example, in a MIDI file format.
  • the stems are single-layer audio files as obtained from a user, artist, content producer, etc., or derived from other files (e.g. a video clip, broadcast or other recordings).
  • the stems may also be obtained through, for example, indexing or crawling the Internet or other databases.
  • the stem may be stored in any computer-readable file format and encoding scheme (e.g. MP3, WAVE, AIFF, FLAC, etc.) and may contain one audio layer.
  • a stem includes multiple audio layers. Different stem content may have different forms of audio (e.g. vocal track, percussion, musical piece, etc.) and may have different lengths. For example, some stems may have lengths that are in multiples of 4 bars for sake of conforming to conventional musical structure.
  • FIG. 5 depicts an example block diagram of audio effects 501 in the MCM's effects database 307.
  • the effects are stored in the database in association with their relevant parameters. Examples of such effects include reverb 501 a, delay 501 b, side chain compression 501 c, saturation 501 d, distortion 501 e, phaser 501 f, highpass filter 501 g, lowpass filter 501 h, flanger 501 i, and chorus 501 j.
  • the effects and their associated parameters may be applied to any stem or song segment in the production, or to the entire production as a whole. A plurality of effects may be combined and applied to the same stem, song segment, or to the entire production to obtain different desired outcomes.
  • example computer or processor executable instructions are provided for obtaining stem compatibility data through the characterization of a stem.
  • the instructions are applicable, for example, in the Categorization Module 301 , as shown in FIG. 3.
  • the audio data file and any applicable stem data such as metadata, MIDI note information, etc., are obtained.
  • data used to characterize the stem is obtained, such as the name of stem, genre, emotion, instrument type, song-starter information, name of the user uploading the stem, etc.
  • the stem is processed by the Categorization Module 301 to characterize the associated compatibility data.
  • the stem is characterized for its textural fingerprint at block 604.
  • the stem is also characterized for its melodic compatibility data at block 605.
  • Example melodic compatibility data include the musical notes 406, musical key 408, and chord progression 606.
  • the stem may also be characterized for its temporal compatibility data at block 607.
  • Example temporal compatibility data include the BPM 403, time signature 404, and whether or not the beat exhibits a swing 608. Example implementations of the characterization algorithms are discussed later.
  • the characterization data, stem data, and compatibility data are stored in association with the stem in the stem database 308.
  • compatibility metrics e.g. 403, 404, 406, 408, 606, and 608
  • other relevant metrics may be included in the various characterization steps.
  • particular order in which the characterization steps occur are not mandatory; steps may be switched, added, or deleted.
  • FIG. 7 is an example overview of processor executable instructions for the composition flow of an audio production.
  • the instructions may be implemented, for example, in a MCM 212 available on a server 101 , or on a user computing device 103.
  • a user interface (Ul) is provided to display stem criteria for user selection.
  • the software at block 703 filters the stem database 308 for stems having compatible tempo and that match all of the optionally selected criteria.
  • a stem with compatible tempo to the user's selection may mean that the BPM of the stem is exactly matched with the user's choice (e.g. 40 bpm), or is a multiple thereof (e.g. 80 bpm or 120 bpm).
  • the software provides a Ul displaying the stems from the filtered stems that are song-starters (e.g. as determined by the song-starter information 401 in FIG. 4).
  • the software receives a song-starter selection at block 705, and then proceeds at block 706 to further filter stems that are compatible with the selected song-starter based on at least one of, and preferably all of, textural, melodic, and temporal compatibility.
  • the software provides a Ul to display further filtered stems for user selection.
  • the software receives a stem selection for a next layer at block 709, and, depending on the user's next choice, proceeds to compile the audio track in 710, or repeats blocks 707 and 709 based on at least one of textural, melodic and temporal compatibility at block 708.
  • the software compiles the audio track at block 710.
  • the software provides a Ul to add further modifications to the production (e.g. percussion, song structure, effects, etc.).
  • the software receives a selection for at least one of these modifications at block 712, and, depending on the user's next choice, proceeds to compile the audio track in 713, or repeats steps 711 and 712 based on at least one of textural, melodic and temporal compatibility 714.
  • the modifications listed herein are only given by way of example, and other modifications to the stems, segments of the production, or the entire production as a whole are possible and will be apparent to one skilled in the relevant art.
  • FIG. 8 is a detailed overview of example processor executable instructions for the composition flow of an audio production. Each block of example processor executable instructions will be described in further detail subsequently.
  • the software provides a Ul to receive genre (and optionally subgenre) selection from the user.
  • the software receives the genre (and optionally subgenre) selection from the user, and at block 803, the software filters the stems database 308 to exclude stems that do not match the selected genre.
  • the software provides a Ul to receive a tempo selection from the user.
  • the software receives the tempo selection from the user, and at block 806, the software filters the already filtered stems from block 803 to exclude stems that do not match the selected tempo.
  • the software provides a Ul to receive an emotional selection from the user.
  • the emotional selection may be a single emotional tag, or a combination of emotional tags.
  • the software receives the emotional selection from the user, and at block 810, the software filters the already filtered stems from block 806 to exclude stems that do not match or are not compatible with the selected emotion. It is hereby recognized that the emotional selection process, as outlined by block 807, is not a necessary step, and may be skipped in the overall composition flow of an audio production.
  • the software identifies the stems from the filtered results of block 810 that are specifically song-starters (for example, as determined by the song-starter information 401 ).
  • the software provides a Ul to receive an instrument and a song- starter selection from the user.
  • the software receives the instrument and song- starter selection from the user and at block 814, the software filters the already filtered stems from block 81 1 to exclude stems that do not match the selected instrument and song-starter criteria, based on at least one of textural, melodic, and temporal compatibility.
  • the software provides a Ul to add an additional layer to the musical production.
  • the available stems for display at block 815 may consist of the stems as filtered at block 814.
  • the software receives the layer selection from the user.
  • the software provides a Ul to receive alternative instrumentation selections for parts of the musical production.
  • the parts may include certain song segments, sections, or stems present in the production.
  • the software receives the instrumentation change selection(s) from the user. It is hereby recognized that the alternative instrumentation selection process, as outlined by block 817, is not a necessary step, and may be skipped in the overall composition flow of an audio production.
  • the software provides a Ul to add a percussion layer to the musical production.
  • the available stems for display at block 820 may consist of the stems as filtered at block 814.
  • the software receives the layer selection from the user.
  • the software provides a Ul to receive alternative percussion selections for parts of the musical production.
  • the parts may include certain song segments, sections, or stems present in the production.
  • the software receives the alternative percussion selection(s) from the user. It is hereby recognized that the alternative percussion selection process, as outlined by block 822, is not a necessary step, and may be skipped in the overall composition flow of an audio production.
  • the software provides a Ul to modify the song structure in the musical production. Modifications may comprise of, but are not limited to, the choice to include a song intro or outro, the number of verses or choruses, etc.
  • the software receives the song structure selection from the user.
  • the software provides a Ul to modify the length of song sections in the musical production. Modifications may include the lengthening or shortening of song sections, the reordering of song sections, etc.
  • the software receives the song section length modifications from the user. It is hereby recognized that the song section length modification process, as outlined by block 827, is not a necessary step, and may be skipped in the overall composition flow of an audio production.
  • the software provides a Ul to receive an alternative stem selection in the musical production.
  • Alternative stems may include different chord progressions, instrumentation, a change in the musical key, etc.
  • the stems database 308 may be filtered by the software to display only the alternative stems that are compatible with the current production based on at least one of textural, melodic, and temporal compatibilities.
  • the software receives the alternative stem selection(s) from the user. It is hereby recognized that the alternative stem selection process, as outlined by block 830, is not a necessary step, and may be skipped in the overall composition flow of an audio production.
  • the software provides a Ul to add audio effects to the musical production.
  • the software receives the audio effects selection from the user.
  • the software provides a Ul to configure the said audio effects.
  • the software receives the audio effects configurations from the user. It is hereby recognized that the process of adding and configuring audio effects, as outlined by block 833, is not a necessary step, and may be skipped in the overall composition flow of an audio production.
  • the software provides a Ul to receive stem ratings from the user.
  • the software receives the stem ratings from the user.
  • the software may be configured, in one example embodiment, to receive additional feedback information about the stems (such as, for example, user comments).
  • the software provides a Ul to checkout and export the audio production.
  • the software receives the checkout and export confirmation from the user (including, for example, payment information, audio track delivery method, etc.).
  • the software provides the final mixed audio track to the user.
  • Example graphical user interface (GUI) components as described in the steps of FIGs. 7 and 8 are shown in FIGs. 9 through 21 , and will now be described in an example embodiment of the proposed musical compatibility system. It will be appreciated that the terms GUI and Ul are herein used interchangeably. The GUI components and executable instructions are shown together.
  • GUI graphical user interface
  • FIG. 9 is a schematic of example GUI components for the selection of genre and subgenre.
  • the genre selection flow is divided into a genre selection Ul 901 and a subgenre selection Ul 905.
  • the software first presents the genre selection GUI 901 to the user.
  • a plurality of genres 402 is presented to the user, and selecting a specific genre 906 triggers the display of a popup window 903 that presents popular artists categorized under the selected genre.
  • the desired genre is selected by selecting the radio button 902a beside the corresponding genre, and then selecting the "ACCEPT" button 904.
  • the software then presents a Ul 905 displaying a list of subgenres 912 that are categorized under the selected genre.
  • Selecting 907 on a specific subgenre 912a will trigger the playback 908 of an exemplary stem from the selected subgenre, and the opening of a popup window 903 that presents popular artists categorized under the selected subgenre 912a.
  • the playback indicator 913a will change visually 913b to indicate that an exemplary stem from the selected subgenre is currently being previewed.
  • the subgenre is selected by selecting the radio button beside the corresponding genre, and then selecting the "ACCEPT" button 904.
  • the genres may be, but are not limited to, Hip-Hop 402a, Rap 402b, Pop 402c, and R&B 402d.
  • the software receives a selection for the "Pop" genre 402c from the first genre selection Ul 901 , and proceeds to display a subgenre selection Ul 905 listing subgenres 912 under the genre Pop 402c.
  • Example subgenres for "Pop” may include, but are not limited to, EDM 912a, Power Pop 912b, and Synth Pop 912c.
  • the selections of the GENRE "Pop” 402c and SUBGENRE "Electronic Dance Music” 912a (EDM) are received by the software at block 909.
  • the software searches through the database and eliminates stems that are not tagged with the selected GENRE and SUBGENRE.
  • the stems not tagged, for example, with “Pop” and “EDM” are filtered out by the software, and only stems tagged with "Pop” and "EDM” remain as options 91 1.
  • the playback indicators 913a and 913b may be utilized anywhere in the Ul where appropriate.
  • the playback indicator may be used for previewing stems of a specific tempo(s), emotion(s), or instrument(s).
  • subsequent appearances of the playback indicator, whether in its inactive 913a or active state 913b, in the example Ul components will be assumed to function the same as described above, and that for sake of avoiding obscurity, there will be no more referring to the playback indicators 913a and 913b by description, or their respective figure numerals in the subsequent text.
  • the tempo selection Ul includes a first component 1001 for the selection of a rough tempo (e.g. 50 bpm, 70 bpm, 90 bpm, etc.), and a second component 1002a for the selection of an exact tempo (e.g. 60 bpm).
  • a rough tempo e.g. 50 bpm, 70 bpm, 90 bpm, etc.
  • second component 1002a for the selection of an exact tempo (e.g. 60 bpm).
  • An option is also provided to the user to immediately proceed to the exact tempo selection Ul component 1002a without the need of first selecting a rough tempo.
  • Such an option may be implemented in the form of a "SKIP" button 1006.
  • the user is first presented with a list of stems 403, each at a specific tempo.
  • the stems each exhibit a tempo (e.g. beats per minute) that varies from one another.
  • the software is able to help a user determine a desired rough tempo range.
  • the stems presented in 1001 e.g. stems 403a - 403e
  • the software When the Ul 1001 receives a selection on a playback icon, the software is triggered to play an audio preview of the selected stem of the selected tempo.
  • a rough tempo e.g. 50 bpm (403a)
  • the software presents the exact tempo selection Ul component 1002a.
  • the tempo choices 403f-403i presented in this component correspond to a range of tempos that bracket the rough tempo selected in 1001 .
  • the software in an example embodiment, may present tempo choices for every single bpm ranging from 40 bpm (403f) to 60 bpm (403i).
  • the tempo choices presented in this step are corresponding to the range between the selected rough tempo minus 10 bpm and the selected rough tempo plus 10 bpm.
  • the software filters the previously filtered genre stem database for stems having a compatible tempo (block 1004).
  • a stem with compatible tempo to the user's selection may mean that the BPM of the stem is exactly matched with the user's choice (e.g. 60 bpm), or is a multiple thereof (e.g. 120 bpm), as shown at block 1005.
  • FIG. 1 1 is another schematic diagram of example GUI components for the selection of tempo.
  • the tempo selection Ul includes a component 1 101 for the selection of a rough tempo range (e.g. 80-1 10 bpm), and a second component 1002b for the selection of an exact tempo (e.g. 100 bpm).
  • a rough tempo range e.g. 80-1 10 bpm
  • a second component 1002b for the selection of an exact tempo (e.g. 100 bpm).
  • the plurality of stems each with a different tempo, is presented on a horizontal axis of increasing BPM from left to right 1 101 .
  • This provides an alternative visualization for the different tempo choices, and may be a beneficial visualization for some users.
  • An option is also provided to the user to immediately proceed to the exact tempo selection Ul component 1002b without the need of first selecting a rough tempo.
  • Such an option may be implemented in the form of a "SKIP" button 1 103.
  • the user is presented with the exact tempo selection Ul 1002b in the same fashion as the Ul 1002a shown in FIG. 10.
  • the software undergoes the steps 1004 and 1005 as shown in FIG. 10 to filter the database for stems having a compatible tempo.
  • the tempo selection process as described may be expanded to encompass other rhythmic metrics that are used to describe a musical work, including, for example, time signature, changing tempos, or the presence of a swing beat or straight beat, etc. These metrics, and other relevant rhythmic metrics, may be used in any combination as part of the tempo selection process, or as criteria to filter the stems database 308 (such as, for example, at block 814).
  • FIG. 12 the emotion selection process and a schematic diagram of example Ul components involved are shown.
  • the selection of an emotion may serve to further refine the database in accordance of the desired emotional characteristics of the production. It is recognized that there are multiple different emotions that may be used to describe a stem, and tagging each stem with specific emotion(s) will dramatically reduce the amount of available stems within a specific emotion to choose from.
  • the emotional spectrum is simplified and divided into major emotional tags 407.
  • the emotional tags may be "Love" 407a,
  • “Happiness” 407b, “Courage” 407c, “Excitement” 407d, “Hate” 407e, “Sadness” 407f, “Fear” 407g, and “Anger” 407h although additional or substitute emotional tags may also be identified, as is apparent to one skilled in the relevant art. It is also important to note that a significant amount of emotional content regarding vocal recordings is determined by the lyrical content and the vocal performance itself, and therefore the categorization of stems is done according to musical characteristics that make each stem compatible with one or several similar emotions. Such musical characteristics may include, but are not limited by, melody, note accents, tone, key, etc.
  • the emotional tags are consistent with the lyrics and tone of the vocals to facilitate the emotion selection process based on the user's requirements.
  • An option 1202 is also provided to the user to skip the selection of emotion(s) 1203 and immediately proceed to the song-starter selection Ul component as shown in FIG. 13. If the emotion selection step is skipped, subsequent database queries for compatible stems will exclude the emotion data, thus leading to a larger number of potential stems 1204.
  • an initial Ul component 1201 a displaying all of the emotional tags 407 is presented.
  • a first selection for the emotion tag "Courage" 407c is received.
  • the software records the selected emotion for subsequent queries (block 1208) and proceeds to refresh the Ul component 1201 b to display an updated view of the emotional tags.
  • the visual representation of the selected emotional tags 407c may change (for example, by colour, style of the font, blinking, etc.) to notify the user of the selection.
  • Additional emotions that are strongly tired to the selected emotion e.g. "Bold” 1205a, "Powerful” 1205b, "Brave” 1205c, and "Confident” 1205d) appear on the Ul as user- selectable buttons. Note that adjacent emotional tags to the currently selected emotional tags may shift on the Ul display to accommodate the addition of the new emotion tags.
  • the visual representations of conflicting or completely contradictory emotional tag(s) to the currently selected emotional tag(s) are modified to notify the user of their compatibility status.
  • conflicting emotional tag e.g. "Hate” 1206a, "Sadness” 1206b, or "Anger” 1206c
  • conflicting emotional tag e.g. "Hate” 1206a, "Sadness” 1206b, or "Anger” 1206c
  • conflicting emotional tag e.g. "Hate” 1206a, "Sadness” 1206b, or “Anger” 1206c
  • conflicting emotional tag e.g. "Hate” 1206a, "Sadness” 1206b, or “Anger” 1206c
  • completely contradictory emotional tags e.g. "Fear” 1207a
  • the emotional tag "Fear” 1207a which contradicts the selected emotion "Courage" is removed from the Ul, or its representation is crossed out.
  • the selection of a second emotional tag results in further shifting of the emotion buttons, and the appearance of additional emotions (e.g. "Inspired” 1205e) that are consistent with the two selected emotional tags.
  • Emotional tags that are conflicting with the selected emotional tags are diminished 1206 (such as, for example, "Anger” 1206d), and emotional tags that are conflicting with the selected emotional tags (such as, for example, "Hate” 1207b and “Sadness” 1207c) are even further under- emphasized or removed from the Ul 1207.
  • desired emotion(s) may be selected.
  • the selected emotions are used as criteria to further exclude stems that are not tagged with the selected emotion(s) 1208. For example, if the emotions "Inspired" 1205e and "Confident” 1205d are selected (block 1209), the database will further filter the previously filtered list of stems to exclude stems that are not tagged with both "Inspired” and "Confident” 1210.
  • the emotional tags attached to stems may be a weighted representation of each emotional tag 407.
  • the similar emotions "Worried” and “Anxious” are both determined to consist of the emotional tags “Fear” and “Excitement", although not with equal weight. "Worried” exhibits a greater portion of the emotional tag “Fear” than it does “Excitement”, and may be tagged as such (for example, as represented by a greater multiplier attached to the "Fear” tag than to the "Excitement” tag).
  • "Anxious” is determined to consist of more “Excitement” than Fear, and may be appropriately tagged.
  • stems may be recommended with priority over other stems depending on, for example, which specific emotion was selected, or depending on the weighted multiplier attached to each selected emotion.
  • At least one of seemingly contradictory emotional tags may be combined, for example, when one of the tags selected is "Love".
  • "Love” is considered to be a positive emotional tag, when it is combined with "Sadness", for example, emotions such as "Alone” and "Hurt” appear as possible choices.
  • the emotion "Love” in a certain context can contribute to a negative emotion.
  • resultant compatible stems will be tagged with the combination of the two categories as well as the dominant category by itself.
  • the software presents the Ul components for the selection of the song-starter.
  • a list of instrument categories 409 e.g. Piano 409a, Acoustic guitar 409b, Synthesizer (various) 409c is presented 1301 .
  • instrument categories 409 e.g. Piano 409a, Acoustic guitar 409b, Synthesizer (various) 409c
  • subcategories may be necessary to further differentiate more specific instruments.
  • selecting on the instrument category 1305 e.g. selecting on Synth 409c
  • will present an additional Ul component 1303 that presents further specific instruments such as, for example, Pluck, Pad, Bass, and Brass).
  • Selecting on the playback indicator 1302 or 1304 allows previewing of an example stem from the selected instrument category or subcategory, with the previewed stem being compatible with the genre, tempo, and emotion criteria as previously selected.
  • the user need not know the underlying musical qualities associated with each song-starter instrument, or the required musical theory to select a musically pleasing or otherwise desired song-starter, since the relevant criteria (e.g. tempo, genre, and emotion) have already been defined in the previous steps, and are used to filter the stems database 308 for compatible stems.
  • a choice of a song-starter instrument is selected when the software receives a selection on the ACCEPT button in 1306.
  • a list of a plurality of compatible song-starter options 401 recorded with the selected instrument is displayed on the Ul component 1307.
  • the song-starter options e.g. 401 a - 401 c
  • Each song-starter option has an accompanying playback indicator which triggers playback of the associated stem when selected 1308.
  • Selecting the "Similar" button 1318 next to each song-starter option displays an additional Ul component 1314 presenting a plurality of alternative song- starters 1316a - 1316c that are similar to the intended song-starter 1309.
  • alternative options may include, but are not limited to, different arrangement of the same chords, the same chord progression in different time signatures, or slight variations of chords (such as changing the configuration of a single chord within the progression).
  • a "Change Key” button 1315 may be implemented that allows transposing the song-starter into a different key. This will be a useful feature for vocal artists who like the feeling and characteristics of a particular song-starter that is outside of their preferred vocal range.
  • the software After the desired song-starter is chosen by selecting the ACCEPT button 1310, the software presents a prompt dialog Ul 131 1 for either proceeding to add an instrumental layer (the "Add Layer” option 1312), or to add a percussion layer (the “Add Beat” option 1313). Selecting the "Add Layer” option 1312 instructs the software to display the relevant Ul components for adding a new instrumental layer, as shown in FIG. 14. Selecting the "Add Beat” option 1313 instructs the software to display the relevant Ul components for adding a new percussion layer, as shown in FIG. 15. After the song-starter is selected, the software filters the previously filtered stems database 308 to exclude stems that are not compatible with at least one of (and preferably all) textural, melodic, and temporal compatibility data of the selected song-starter 1317.
  • the software presents example Ul components to select and add an additional instrumental layer, in a fashion similar to the selection of the song-starter in FIG. 13.
  • a list of a plurality of instrument categories e.g. Piano, Acoustic guitar, Synthesizers, etc.
  • instrument categories e.g. Synth
  • subcategories may be necessary to further differentiate more specific instruments.
  • selecting on the instrument category e.g. Synth
  • will present a callout Ul dialog that presents further specific instruments (not shown here for sake of preventing obscurity).
  • the software will provide the user with, for example, a "Vocal" category and appropriate subcategories (e.g. Male, Female, and other voice descriptions).
  • voice descriptions may include, but are not limited to, vocal timbre or characteristics (e.g. raspy, soulful, etc.).
  • the software may present a "POPULAR" button 1417 that facilitates the selection of a stem. Selecting the POPULAR button 1417 will bring up a Ul component displaying a plurality of stems that have the highest combination of compatibility (e.g. textural, melodic, and temporal compatibilities as determined by the software) and stem popularity (determined by the frequency the stem is chosen by users who have either selected similar production parameters, or similar song-starter to be discussed later) 1403.
  • the recommended stems suggested by the POPULAR Ul dialog may consist of different instrumental or vocal stems and are all highly compatible with the previously-selected song- starter.
  • Selecting on the playback indicator allows previewing of an example stem from the selected instrument category or subcategory 1402, with the previewed stem being compatible with the genre, tempo, emotion criteria, and song-starter as previously selected. Additionally, the instrumental stem currently being previewed may be superimposed in playback over the selected song-starter to allow previewing of how the desired instrumental stem will sound in conjunction with the selected song-starter.
  • a specific instrument e.g. "Piano” may be selected as a desired instrument for the stem 1408.
  • the software displays a Ul component 1409 with a list of stems 1410a - 1410c recorded with the selected instrument, where each stem is furthermore compatible with the genre, tempo, emotion criteria, and song-starter as previously selected.
  • the list of stems may be ranked by one or more criteria, such as, for example, emotional similarity, textural compatibility, melodic compatibility, temporal compatibility, lyrical content, popularity, and previous user rating.
  • the software may offer additional functions to aid in the selection of a desired stem, such as, for example, a "Similar” stem button 141 1 , a "Change instrument” button 1412, and a "Solo” button 1407.
  • Selecting the "Similar" button 141 1 displays a Ul component 1405 with a list of a plurality of stems (e.g. 1406a, 1406b) recorded with the previously-selected instrument, that are variations of the original stem recommendation (for example, in the arrangement of notes).
  • the stem variation may be based on a variation of timing. For example, chords may hit twice per quarter note in the stem variation, as opposed to once per quarter note for the original stem recommendation.
  • the stem variation may be based on variations in melody (e.g. musical scale), while still conforming to the same general musical qualities of the original stem recommendation.
  • the stem variations may contain different lyrics, with the lyrics recorded by the same currently selected vocalist.
  • stem variations present to the user a large amount of options to customize the production.
  • the user is not required to listen through all of the stems and stem variations in the database to select those that are compatible, or otherwise sound musically pleasing together with the rest of the production, since all stems and stems variations suggested (for example, in the previous step 1309) are compatible with the genre, tempo, emotion criteria, and song-starter as previously selected.
  • Selecting the "Change Instrument” button 1412 displays a Ul component 1414 with a list of a plurality of stem recommendations 1415 that have the same note or chord pattern as the original stem recommendation, but are recorded with different instruments 1413.
  • the user first selects an instrument to add next, and then decides once they hear a melody or chord progression, that they like the pattern of notes, but feel a different instrument would better incorporate the note pattern into their production.
  • the stem variations may contain the same lyrics, but the lyrics are recorded by a different vocalist.
  • Each stem recommendation in 1414 will be clearly labeled with the instrument featured in the stem (e.g. "Piano” 1415a or “Guitar” 1415b).
  • Selecting the playback indicator allows previewing of the desired instrumental stem. Additionally, the instrumental stem currently being previewed may be superimposed in playback over the selected song-starter to allow previewing of how the desired instrumental stem will sound in conjunction with the selected song-starter.
  • a "Solo" button 1407 may be implemented into the software to aid in the previewing of the stem. Selecting the "Solo" button 1407 when a stem is playing results in the stem being previewed in isolation from the rest of the production, that is, the song- starter, all other layers, and instruments are muted or disabled. In other words, when the stem is being previewed in its "solo" state, only the selected stem and any effect(s) applied to the selected stem are played.
  • the software After the desired instrumental layer is chosen by selecting the ACCEPT button, the software presents a prompt dialog Ul 131 1 for either proceeding to add another instrumental layer (the "Add Layer” option 1312), or to add a percussion layer (the “Add Beat” option 1313). Selecting the "Add Layer” option 1312 returns to the starting Ul component for adding a new layer 1401 . Selecting the "Add Beat” option 1313 instructs the software to display the relevant Ul components for adding a new percussion layer, which will be discussed in further detail with respect to FIG. 15. [00129] Continuing with FIG. 14, after adding additional layers or beats, the system filters stems to exclude stems that are not compatible with the user selections made thus far (block 1416). For example, stems that are available to the user have a certain genre, tempo and emotion, and are compatible with the selected song starter and the other selected layers.
  • the software presents the option to select and add an additional percussion layer, in a fashion similar to the addition of an instrumental layer shown in FIG. 14.
  • the Ul components of FIG. 15 are displayed.
  • a Ul component 1501 presents a list of percussion categories (e.g. 1514, 1515) from which the user may select. Selecting the playback indicator allows previewing of an example stem from the selected percussion category 1502. Additionally, the percussion stem currently being previewed may be superimposed in playback over the selected song-starter, or the rest of the musical production to allow previewing of how the desired percussion stem will sound in conjunction with the selected song-starter, or the rest of the musical production.
  • the percussion stems may be categorized into two major categories, “Acoustic” (recorded acoustic drum kit(s) 1514), or “Electronic” (percussion synthesized from electronic samples and kits 1515). Other categorization methods for percussion stems are also available and will be appreciated by one skilled in the art.
  • a desired percussion category e.g. "Acoustic” 1514
  • the software displays a Ul component 1504 with a list of percussion stems 1505a - 1505c belonging to the selected percussion category, where each stem is furthermore compatible with the genre, tempo, emotion criteria, song- starter, and previous layer(s) as previously selected.
  • the percussion stems e.g. 1505a - 1505c
  • the percussion stems may be recorded with different drum kits, but still fall under the selected percussion category (e.g. 1514).
  • the list of stems may be ranked by at least one of a plurality of criteria, including but not limited to, emotion similarity, textural compatibility, melodic compatibility, temporal compatibility, popularity, and previous user rating.
  • the software may offer additional functions, such as, for example, a "Similar” stem button 141 1 , a "Change Drum Kit” button 1509, and a "Solo” button 1407.
  • Selecting the "Similar" button 141 1 displays a Ul component with a list 1507 of stem recommendations 1508a, 1508b recorded with the previously-selected drum kit that are variations in the percussion beats of the original stem recommendation 1506.
  • the variations include, for example, the same drum kit performing a different percussion beat.
  • the variations include of the same electronic drum beat sounds performing a different percussion beat.
  • Other percussion variations include, but are not limited to, slight changes to the beat, addition or omission of certain layers or drums in the drum kits, etc.
  • Selecting the "Change Drum Kit” button 1509 displays a Ul component 151 1 with a list of a plurality of stem recommendations 1512a, 1512b that have the same percussion pattern as the original stem recommendation, but are recorded with a different drum kit 1510.
  • a "Solo" button 1407 may be implemented into the software to aid in the previewing of the stem. Selecting the "Solo" button 1407 when a stem is playing results in the stem being previewed in isolation from the rest of the production, that is, the song- starter, all other layers, and instruments are muted or disabled 1516.
  • the software presents a prompt dialog Ul 1513 for either proceeding to add another instrumental layer (the "Add Layer” option 1312), or to configure the overall musical structure (the "Select Structure” option 1410). Selecting the "Add Layer” option 1312 returns to the starting Ul component 1401 for adding another instrumental layer in FIG. 14. Selecting the "Select Structure” option 1410 instructs the software to display the relevant Ul components for the configuration of the overall musical structure of the production, as shown in FIG. 16.
  • the software presents example Ul components for the selection and configuration of the desired song structure.
  • the user is prompted for the following choices: whether or not an "Intro” is desired 1602, how many "Verses” are desired 1603, how many "Choruses” are desired 1604, whether or not a "Bridge” is desired 1605, and whether or not an "Outro” is desired 1606.
  • the software may also present options 1619, 1620 to select a higher number of "Verses” and "Choruses” than the choices currently present on the Ul. Selecting on the options 1619, 1620 may bring up an additional Ul dialog for further length choices.
  • the further length choices may be represented on the Ul, for example, in a drop-down list, a list box, a text input box, a slider control, etc.
  • parts of a song that are arranged in specific patterns to produce a conventional work of music. These parts are the "Intro”, “Verse”, “Chorus”, “Bridge”, and “Outro”. Specific rules in the arrangements of these parts may be used to produce a musical work that is logical and sounds musically pleasing.
  • a song arrangement that is often used consists of 2 verses, 3 choruses, and 1 bridge segment.
  • each verse is 16 bars; the first and second choruses are 8 bars; the bridge is 8 bars; and the final chorus is 16 bars.
  • a variety of song structures may be programmed into the software such that it can recognize patterns in the user's song structure selections (e.g. number of verses, choruses, and the presence of a bridge), and use the arrangement information to make an informed decision on an ideal song structure.
  • Certain song structure choices made by the user may conform to conventionally used song structures, and the pattern will be recognized and used by the software.
  • the software is able to make recommendations for example song arrangements. At any point, the full control over song arrangement is available to the user.
  • the software will verify that the desired selection of song parts conform to the arrangement rules as outlined earlier. In the case that the user's selection of song parts do not conform to the said arrangement rules, the software may notify the user (for example, by a visual alert message, visual indicator, or audible alert, etc.) and prompt that the error be rectified before continuing.
  • selecting the "PREVI EW” button 1607 displays a preview of the selected song structure 1608.
  • an example arrangement may be chosen, such that the user desires an "Intro", 2 "Verses”, 3 "Choruses", a "Bridge”, and an "Outro” in their song arrangement.
  • the software may display the selected song arrangement of 1609 in an example Ul component as shown in 1610.
  • the horizontal axis of the example Ul component represents time, wherein the leftmost side of the Ul component is at time zero (for example, at the start of the song), and the rightmost side of the Ul component is at the end time (usually, at the end of the song).
  • the separate sections of the example Ul component represent different sections of the song, wherein 161 1 is the “Intro”, 1612 is the “Verse”, 1613 is the “Chorus”, 1614 is the “Bridge”, and 1615 is the "Outro".
  • the software may display the selected song arrangement of 1609 in an example Ul component as shown in the Ul form 1618.
  • the horizontal axis of the example Ul component represents time, wherein the leftmost side of the Ul component is at time zero (for example, at the start of the song), and the rightmost side of the Ul component is at the end time (usually, at the end of the song).
  • the separate sections of the example Ul component represent different sections of the song, wherein 161 1 is the "Intro", 1612 is the "Verse”, 1613 is the "Chorus”, 1614 is the "Bridge”, and 1615 is the "Outro".
  • the letters "A”, "B", “C”, and "D" represent different audio layers of the production.
  • different layers may contain, but are not limited to, different instrumental tracks, percussion tracks, or vocal tracks.
  • the sections of the song may be represented in other ways on the Ul, such as, for example, with the Roman numerals "i" to represent the Intro, "ii” to represent the first Verse, “iii” to represent the first Chorus, "iv” to represent the Bridge, and "v” to represent the Outro, etc.
  • the software arranges at block 1617 the stems of every layer of the user's production such that it is consistent with the selected arrangement in 1609.
  • stems are looped to occupy each song section.
  • the software makes informed segment arrangement decisions in 1617 based on conventional song structure and arrangement rules as previously described.
  • the intra and outro will be the same length, set at eight bars each.
  • the software displays an example Ul implementation of the "Beat Builder" 1701 for the display arrangement of the song sections.
  • a plurality of available audio effects 1702 At the top of the Beat Builder window is a plurality of available audio effects 1702, to be discussed later.
  • Each layer of the audio production is separated vertically, with horizontal bars 1708 representing the presence of each layer throughout the duration of the song.
  • the horizontal axis of the example Ul component represents time, wherein the leftmost side of the Ul component is at time zero (for example, at the start of the song), and the rightmost side of the Ul component is at the end time (usually, at the end of the song).
  • the production is further divided into layers, wherein each layer a different instrument may be represented by, for example, different instrumental tracks, percussion tracks, or vocal tracks.
  • each layer may be represented with an icon representing the instrument that is featured within the layer (e.g. an icon of a drum kit 409a, or of a piano 409c), with a text description of the layer (e.g. the word "Guitar” 409b), with a unique identifier (e.g. "P1 " for percussion track number 1 ), or a user-editable name (e.g. "wah-wah vocals").
  • the horizontal bar representing the layer is visually segmented horizontally according to the song segments, with lines clearly indicating the transitions between different sections.
  • buttons 1709, 1710, 171 1 , 1712, 1713 corresponding to each song section (for the present example, there are eight song sections).
  • the buttons may correspond to the Intro 1709, first Verse 1710, first Chorus 171 1 , Bridge 1712, and Outro 1713 sections.
  • the stems corresponding to each specific song part are repeated throughout the same parts in the production (for example, the same stem used in the first chorus, will be used in the second chorus, third chorus, etc.).
  • the song parts are represented (for example, visually through the use of a colour indicator, a unique symbol, etc.) in the same fashion on the Beat Builder Ul.
  • the length selection Ul component shows the currently selected song section 1705, the current section length 1706 (for example, 8 bars), and a plurality of alternative choices for the section length 1707a - 1707d.
  • section lengths in multiples of four bars may be selected in the case that a stem exhibits 4/4 timing. Additionally, section lengths in multiples of four bars remain consistent with the standard length of stems (eight bars) such that stems may be either halved or doubled to fill each song section. It will be appreciated that standard stem lengths will be set depending on the time signature present in the stem, and that the stem length of four bars as described is only by way of example. Changing the length of a particular section affects all layers within that section. For example, if the section length of a verse is changed, the software will produce a prompt asking the user if they wish to make all other verses present in the production consistent with the new length (a "Yes" or "No" command will instruct the software accordingly).
  • An example Ul component 1701 is provided wherein at the edge of each song segment 1804 there is an arrow 1805 which will present a drop-down list 1806 presenting at least one of alternative stems 1807a - 1807c upon user selection 1803.
  • the stems may be listed in order of relevancy (e.g. as evaluated by textural, melodic, or temporal compatibility, mood similarity, user rating, etc.), and every option is musically compatible (e.g. as evaluated by at least one of textural, melodic, or temporal compatibility) with the rest of the production.
  • alternative stem options contain a different pattern of notes, different chord progression, or arrangement etc. so as to give the song section a different feel or consistency from the rest of the production, while still retaining a musically relevant progression from one section to the next.
  • the color, or other unique identifiers, of that section will change to highlight the difference.
  • the stem alternative is selected by selecting the "CHANGE" button 1816.
  • a prompt dialog 1809 on an example Ul component 1808 will arise asking if the newly selected stem should be applied to all similar sections in the production (e.g. all verses or all choruses, respectively).
  • the newly selected stem will be applied to all similar sections in the production.
  • the newly selected stem will not be applied to all other similar sections in the production; instead, the newly selected stem will only be applied to the current section (as specified, for example, in 1803).
  • the software verifies (block 1812) that the selected stem is musically compatible, based on at least one of textural, melodic, and temporal compatibility, with the rest of the production.
  • a prompt (block 1814) will arise instructing the user as to which layer(s) or stem(s) need to be changed to be consistent with the alternative stem selected.
  • a visual indicator of incompatibility (such as a "0" symbol) may be displayed on top of the incompatible stem. The user is then presented the option to select the conflicting segment 1815 and view a list of stem recommendations that are compatible with the variation chosen. If the change in a layer allows the entire collection of selected layers to be compatible, then the change is made without further action (block 1813).
  • layers of the audio production that are determined to be incompatible will be excluded from the final music product.
  • the incompatible layers will not be played when playing back the produced music product.
  • the software provides a Ul component to remove a song segment whereby the user is able to select the song segment for deletion, and then either drag it to the garbage bin icon 1802, or select the "delete" key on their keyboard or other input device.
  • the software at this point will display a prompt dialog to confirm the intended delete action.
  • a prompt dialog 1809 on an example Ul component 1808 will arise asking if all similar sections in the production (for example, all verses or all choruses, respectively) should also be removed.
  • the software also provides a Ul component to add additional audio layers to specific song sections. For example, this may be implemented in the form of a plus "+" button 1801 above each audio section.
  • the software presents the relevant Ul components for the additional of a new layer 1401 , as shown in FIG. 14.
  • a prompt dialog 1809 on an example Ul component 1808 will arise asking if the newly added stem should be applied to all similar sections in the production (e.g. all verses or all choruses, respectively).
  • the software may also provide the function to rearrange specific section(s) of the production. For example, this may in be implemented in the form of a drag and drop operation, wherein the user may drag a section they wish to reorder, and drop the section into its intended destination.
  • the drag and drop operation may also be implemented to accommodate multiple song sections, or groupings thereof.
  • the software will inform the user and provide a Ul displaying alternative stems that are compatible with the other layers in that song section.
  • the drag and drop operation may be implemented such that when a specific song section is dropped into a new location, a prompt will arise asking the user whether they wish to move the selected section to the new location, or copy it there (in which case the section would be duplicated).
  • FIG. 19 is a schematic diagram illustrating an example implementation of the Beat Builder Ul 1901 featuring the audio effects toolbar 1902.
  • Each audio effect is represented by a clearly labeled icon, where each icon may be a different color, or otherwise uniquely identified (e.g. by a unique symbol, by caption text, etc.).
  • a legend 1912 of available audio effects may also be displayed, however, in other implementations the legend may be displayed elsewhere, or is not needed.
  • effects may be added to song segments, which are specific sections of specific layers inside the production.
  • the desired effect icon 1902g may be selected, dragged across the Ul, and dropped on top of the desired song segment 1905.
  • the cursor changes shape (for example, into a paintbrush 1904) to indicate that the effect is ready to be applied.
  • the Ul component 1906 shows that the brush cursor 1904 is dragged into the song segment 1905.
  • the software detects that the effect has been released or put into the song segment 1905, the effect is applied to said segment 191 1 .
  • the target song segment 1905 changes color 1908 (or changes using another visual indicator) so as to be consistent with the color or visual indicator of the added effect icon 1902g. Additionally, left and right boundaries to the visual representation of the added audio effect may be implemented, which may be dragged to apply the effect to more or less of the segment.
  • a control element (e.g. in the form of a dial 1909) appears on the edge of the song segment block.
  • the control element is used to control the parameters inherent to the applied audio effect.
  • the control element will have at least one parameter that may, for example, range from 0 (1910a) to 100 (1910b) (e.g. as a percentage value) to control the amount of the audio effect being applied.
  • a setting of "0” means that there is no audio effect and a setting of "100" means that the audio effect is applied to the maximum extent.
  • the control elements may control different effects parameters, depending on the nature of the audio effect being applied.
  • Additional control elements may be implemented in the case that an applied audio effect requires such configuration (such as, for example, for the high-pass filter, where a cut-off frequency and amount of roll-off are configurable parameters).
  • an audio effect is added to a verse or chorus section, a prompt will arise asking if the newly applied effect should be applied to all similar sections in the production (e.g. all verses or all choruses, respectively).
  • a configurable parameter may be the amount of the "mix" of the effect module with a setting of "0" corresponding to a completely dry mix (for example, no proportion of the segment is affected), and a setting of "100" corresponding to a completely wet mix (for example, no proportion of the audio signal is unaffected). Setting the amount of mix towards "100” will result in a denser, richer, more audible reverb, and a more prominent delay.
  • the effect time parameters e.g.
  • the amount of delay for the delay filter, or the decay time for the reverb filter will be automatically synchronized with the tempo of the production (as previously selected by the user). Instead of having to manually synchronize and configure the timers, the user will only be required to make choices such as whether to implement a short, medium or long reverb or delay time.
  • a configurable parameter may be the amount of the source material that is directed or processed through the Effects Module 316.
  • the parameter is similar to the "mix" parameter of time-based effects, with a setting of "100" corresponding to the entire audio signal being processed by the effect and "0" corresponding to none of the audio signal being processed by the effect.
  • a configurable parameter may be the range of frequencies that the low-pass EQ filter is "swept" through, between a low preset frequency and a high preset frequency.
  • a setting of "100" would correspond to the low-pass EQ filter being swept from 20Hz to 20kHz (example preset extreme frequencies).
  • the rate at which the frequency sweep occurs deviates from linear. For example, for more extreme filter sweeps, it is conventional for the rate of increase of filter frequency to become faster toward the end of the effect duration.
  • Such internal parameters are parameters that the user does not need to alter, as the software provides one simplified adjustment that the user can configure according to their desired effect outcome. This simplified implementation of effect parameters is beneficial and advantageous than other, more complex DAWs, especially, for example, for users with limited or no musical production knowledge.
  • a configurable parameter may be the amount of the "ducking" effect. Increasing the value of this setting results in a greater reduction of amplitude (or “ducking") of the chosen segment every time a loud amplitude peak during the playback of the audio production (such as, for example, when a kick drum hits in the beat). This is an audio effect commonly utilized in popular music to "duck" low frequency sound sources, such as when the kick drum is triggered, the use of which results in a pleasurable audible effect, and more audio clarity in the low frequency range of the audio spectrum. In a particular implementation, changing the value of the setting from "0" to "100" affects the threshold of compression.
  • Attack and release times are automatically determined based on the tempo of the production (as previously selected by the user), and the frequency makeup of the song segment to which the effect is applied.
  • the software presents an example Ul component for accepting user feedback on stems. Feedback received from users is beneficial to the system, as it may be used to contribute to the ranking and recommendation of stems as previously mentioned.
  • a star icon 2002 that allows the user to rate the stem in its given context. Selecting on the star icon results in the appearance of a pop-up window or Ul element 2003 containing 4 blank stars (2004).
  • one star may correspond to a bad rating (for example, if a stem is either miscategorised or incompatible in its context), and four stars may correspond to a good rating (for example, if a stem is well-categorized and perfectly compatible in its context).
  • the software receives the user's rating 2005, it checks if the rating received is either one of either a 1 -star rating, or a 2-star rating (2006). If the user rating is one or two stars, an additional Ul component 2008 will appear asking the user to provide additional information 2009 as to why they believe the stem to be poorly categorized or incompatible.
  • the various potential reasons may be displayed on a list in the Ul component 2008, and example potential reasons may include, but are not limited to, "Wrong Instrument” 2009a, “Wrong Key” 2009b, “Wrong Tempo”, “Wrong Mood/Emotion", or "Generally Incompatible in this Context”.
  • the rating justification is submitted by selecting on the "Rate” button 2010. After the software receives the stem rating (and, in the case of a 1 -star or 2-star rating, the additional justifications for such a rating), the software stores the feedback information in association with the stem 2007.
  • the software presents an example Ul component for accepting payment and for the exportation of a finished production.
  • the software receives a command that the user is finished with the production.
  • the command may be implemented in the form of an on-screen Ul button, within a commands menu, or the like.
  • the software receives the command, it presents the Ul components relevant for accepting payment and for the exportation of the finished production.
  • the software displays a prompt 2101 asking the user to confirm whether or not he is finished with the production. If a selection on the "No" 2106 choice is received, the software returns the user to the Beat Builder view in FIG. 17. If a selection on the "Yes" 2102 choice is received, the software proceeds to the checkout process 2103. The user is presented with a Ul displaying the terms and conditions of the software and musical production (including, for example, information on the details of licensing the material, consequences of stealing material etc.)- After the terms and conditions have been accepted, the software prompts the user to select a desired music lease type (such as, for example, non-exclusive, exclusive, various types of sync licenses, etc.).
  • a desired music lease type such as, for example, non-exclusive, exclusive, various types of sync licenses, etc.
  • the software prompts the user to select the desired audio download quality and codec (such as, for example, .wav or .mp3). Then, the software prompts the user for the option to download the music production in "Tracked-Out" form, wherein each individual layer in the final production is exported individually as separate audio files, instead of a single audio file. Finally the user is prompted for payment of exporting the production.
  • processor executable instruments may be implemented, for example, at block 2104. After payment is received and verified, the software masters the production for optimal quality (to be described later), and the final product is available immediately for the user 2105 (e.g. via download, playback, broadcast, etc.).
  • FIGs. 22 through 33 primarily include processor executable instructions for the determination of stem compatibility data, as well as the utilization of said data to determine a given stem's compatibility with other stems within the same musical production.
  • FIG. 22 is a flow diagram illustrating example processor executable instructions for the textural characterization of a stem according to frequency fingerprint.
  • the software receives a stem to analyze.
  • the stem may be submitted by content creators, artists, producers, or obtained otherwise automatically by the software, for example, from an online stems repository, or by crawling the Internet for audio stems, etc.
  • the software determines whether the stem received is either a musical or a vocal stem. If the stem received is a musical or a vocal stem, the stem is sent at block 2203 to the
  • the Categorization Module 301 for the characterization of the frequency fingerprint 2204 of the stem.
  • the stem is neither a vocal nor musical stem (such as, for example, in the case of a percussion stem)
  • the note data 410 accompanying the given stem is analyzed at block 2205 for whether or not the individual percussion beats are triggered in isolation (e.g. not simultaneously with any other percussion beat within the stem).
  • the stem is sent at block 2206 to the Categorization Module 301 for the characterization of the frequency fingerprint 2204 of the percussion stem.
  • the software performs an additional step at block 2207 to analyze the note data for percussion beats that are triggered simultaneously with other percussion beats within the stem. Then, the stem is sent to the Categorization Module 301 for the characterization of the frequency fingerprint 2204 of the percussion beats. After receiving the frequency fingerprint characterization data, the software at block 2208 will subtract the the frequency fingerprint of an individually triggered percussion from the total frequency fingerprint of the percussion stem to obtain the isolated frequency fingerprint of a specific percussion instrument.
  • an example percussion stem consists of three percussion instruments playing in a musical measure.
  • the instruments and the timing are presented in TABLE 2 below, along with whether or not the instrument was triggered at a specific beat (denoted by an "X").
  • This information may be derived by the software from analysis of the stem note data (e.g. MIDI data), or, in the absence of stem note data, from direct analysis of the stem audio data.
  • the kick drum is triggered simultaneously as the hi-hat in beat 1
  • the snare is triggered simultaneously as the hi- hat in beat 3. Therefore, the kick drum and the snare are never triggered in isolation in the above example percussion stem.
  • the software deduces this information at block 2207, records it temporarily, and sends the percussion stem to the Categorization Module 301 for the characterization of the frequency fingerprints 2204 associated with each beat in the stem.
  • the software recognizes that the hi-hat is triggered in isolation and first characterizes the frequency fingerprint associated with the hi-hat beat (e.g. in beat 2 and beat 4).
  • the software characterizes the frequency fingerprint associated with the kick drum beat (e.g. in beat 1 ). This produces the frequency fingerprint of the summation of two percussion sounds, consisting of the frequency fingerprints of the hi-hat and the kick drum. Then, at block 2208, the software subtracts from this frequency fingerprint the isolated frequency fingerprint of the hi-hat (e.g. as obtained from beat 2), to produce the isolated frequency fingerprint of the kick drum. The same process is carried out to isolate for the frequency fingerprint of the snare drum. In this way, the frequency fingerprint for each instrument can be obtained.
  • FIG. 23 is a flow diagram illustrating example processor executable instructions for the textural characterization algorithm, as continued from numeral 2204 in FIG. 22.
  • the software splits the incoming stem into equal time segments (such as, for example, 100 ms). Then, for each time segment 2302 of the stem, the software repeats the following steps: calculate the spectral distribution of audio frequencies within each time segment (preferably using a Fast Fourier Transform, or "FFT") at block 2303; construct frequency bins comprising a pre-determined range of frequencies at block 2304; and obtain the average values of amplitudes within each frequency bin at block 2305.
  • FFT Fast Fourier Transform
  • the software stores the resulting amplitude averages for each time segment in association the stem as a frequency fingerprint.
  • the choice of 100 ms for the time segment is only by way of example, and that other durations of time segments may be selected as is appropriate for the stem music type, or as determined by one skilled in the art.
  • the time segments may be variable in duration, with the duration determined by analysis of the note data, and varying in such a fashion that the duration of a time segment is long enough to obtain an accurate frequency fingerprint in the textural characterization 2204, but short enough to minimize the likelihood of another percussion beat being struck within the same time segment.
  • FIG. 24 is a plot of an example spectral analysis as obtained from a stem file.
  • Such spectral analysis data may be used, for example, to obtain the frequency fingerprint 405 or other textural compatibility data of a stem file.
  • the curve 2303a represents the spectral distribution of the audio frequencies in the stem during the time interval from 0.0 to 0.1 second (in this particular example, the interval corresponds to the first 100 ms of the audio stem).
  • the horizontal axis represents the frequency (in units Hz), and the vertical axis represents the amplitude (in units dB).
  • the frequency bins 2304a, 2304b, etc. consist of different ranges of frequencies, and are separated with dotted lines on the plot in FIG.
  • frequency bins in the present plot include 20 Hz-50 Hz (2304a), 50 Hz-70 Hz (2304b), etc.
  • an average 2305 of the amplitude values within the frequency bin is obtained.
  • the amplitude averages 2305 within each bin are plotted on FIG. 24 for sake of example.
  • the frequency bins are not required to be the same size (e.g. spanning the same number of frequencies), and that the boundaries of the bin domains (for example, at 20 Hz, 50 Hz, 70 Hz, etc.) are also not required to be constant and may be adjusted as appropriate.
  • additional algorithms may be identified and used at block 2303 to obtain the spectral distribution of audio frequencies instead of the FFT, and the use of the FFT in this particular
  • FIG. 25 is a flow diagram illustrating example processor executable instructions for the melodic characterization of a stem.
  • the software receives a stem to analyze.
  • the software determines whether the stem has music note data 410 (e.g. MIDI) stored in association with the stem. If the associated music note data for the stem is available, the music note data is sent at block 2503 to the Categorization Module 301 for the characterization 2504 of melodic compatibility data of the stem. In the case that the stem does not have associated music note data 410 available, the stem audio is sent at block 2505 to the Categorization Module 301 for the characterization 2504 of melodic compatibility data of the stem. The characterization will be carried out by analyzing the stem audio file instead of the music note data 410.
  • music note data 410 e.g. MIDI
  • chords There are 136 commonly used piano chords, which can be separated into the following categories: Major chords, Minor chords, Diminished chords, Augmented chords, Major 7 th chords, Minor 7 th chords, Dominant 7 th chords and Diminished 7 th chords.
  • main scales used in contemporary music may include, but are not limited to, Major and Minor Pentatonic, Blues, Major, Natural Minor, Major 7 th and Natural Minor 7 th .
  • the software first analyzes the notes struck simultaneously in the chords existing in the stem audio, to determine if the chord or chord progression belongs exclusively to one of the recognized keys (to be discussed later with respect to TABLE 4). If so, the accompanying scale is used to determine the compatible notes for melodies. In this case, the analysis of the chord progression will result in seven different notes used to make up the chords, and compatible melodies would contain some combination of those seven notes. Melodies containing notes other than those seven will be considered incompatible.
  • the software uses processes to determine which set of notes may or may not be compatible with the source material.
  • the initial filtering of the library of stems based on key is performed when the user selects their desired song-starter.
  • the song-starter has inherent melodic characteristics that make some stems potentially more compatible melodically, some stems less compatible, and some stems incompatible.
  • the selected song-starter is a chord progression
  • the presence of more notes in the progression will likely result in more extensive filtering of potentially melodically compatible material.
  • the song- starter is monophonic (such as, for example, a bass line)
  • the same principle that is, the more different the notes are played within the song-starter, the more likely it is that only a few matching scales will be identified.
  • FIG. 26 shows a flow diagram illustrating example processor executable instructions for the characterization and saving of melodic compatibility data in association with a stem ("tagging"). Such instructions are applicable, for example, in the Categorization Module 301 .
  • the notes present in a stem are analyzed and stored temporarily (e.g. in the user computing device memory 203 or the server memory 315).
  • the notes analyzed may be simultaneously played in the case of polyphonic stems, or individually played in the case of monophonic stems.
  • the stem may be a song-starter, a vocal stem, an instrumental stem, a percussion stem, etc.
  • the software may perform the analysis using the musical note data 410 (e.g.
  • the software determines if the stem is polyphonic or monophonic. In the case that the stem is polyphonic, at block 2603 the notes contained in the stem are cross-referenced against a database of commonly-used scales, keys, and chords to determine the potential for compatibility. The scales, keys, and chords that contain all notes present in the stem are considered potentially compatible.
  • the stem is tagged by the software with all compatible scales, keys, and chords.
  • the notes contained in the stem are cross- referenced against a database of commonly-used scales and keys to determine the potential for compatibility.
  • the scales and keys that contain all notes present in the stem are considered potentially compatible.
  • the stem is tagged by the software with all compatible scales and keys.
  • the software may tag the stem with data related to the notes present in the stem.
  • FIG. 27 a flow diagram illustrating example processor executable instructions for the recommendation of a further stem selection is shown.
  • Such processor executable instructions are applicable, for example, in the processes described by blocks 814 and 815 of FIG. 8.
  • the software may compare the existing melodic compatibility data associated with a stem in the database the melodic compatibility data of other stems present in the production.
  • the first stem present in the production is the selected song-starter; however, the process for
  • stem selections as described herein is also applicable when the other stem(s) present are instrumental stems, percussion stems, vocal stems, etc.
  • the software analyzes the melodic compatibility data associated with the song-starter. For example, the software analyzes the potentially compatible scales and keys associated with the song-starter.
  • the software compares the melodic compatibility data associated with each remaining stem in the database.
  • the software uses the results of block 2702 to filter the database of remaining stems to include only such stems that are melodically compatible with the song-starter.
  • the software may check at block 2703 whether the notes contained in a given stem are consistent with at least one of the matching scales stored in association with the song-starter and, if so, the given stem is considered compatible. Otherwise, the given stem is considered incompatible with the song-starter.
  • any stems containing notes other than those determined to be compatible with the song-starter will be rejected and excluded from the list of compatible stems.
  • the melodically compatible stem recommendations are ranked according to the degree of melodic compatibility with the selected song-starter.
  • stems containing notes that are compatible with all or most of the keys and scales tagged to the song-starter will be considered melodically compatible to a higher degree than those containing a set of notes unique to only one of the tagged scales or keys.
  • the compatibility between first stem and the song-starter is greater than the compatibility between the second stem and the song-starter.
  • the melodic compatibility ranking may be augmented by usage data of the recommended stems (e.g. the number of times a given stem has been used in conjunction with a selected song-starter).
  • the melodic compatibility ranking may be augmented by previous user ratings or feedback (e.g. as discussed in FIG. 20).
  • a stem is imported to the database that has some combination of musical elements which has not yet been exhibited by stems already in the database, it will first be analyzed to determine if it is consistent with one or more keys or scales. In the case that the imported stem is not consistent with one or more keys or scales, a person skilled in the relevant art will be required to preview the stem and manually determine and assign its musical relevance.
  • Stem recommendations made to the user may be based on the melodic compatibility as described, and on user rankings of said stem or of similar stems. If a certain melody present in a certain stem is ranked highly by a user when combined with a certain chord progression present in another stem, the software records the combination and the fact that it was favorable. The more positive rankings a stem combination gathers, the higher the likelihood that one stem will be recommended when the other is chosen. More generally, the software uses an extensive foundation of music theory (e.g. melodic compatibility or other compatibility metrics as described) to first exclude stems that don't have the potential to be compatible with already existing stems in the user production, then the software uses user feedback to determine which of the remaining stems will yield the most musically pleasing results.
  • melodic compatibility or other compatibility metrics as described
  • a first musical note in a melody is a C note.
  • the melody is transposed up by one semitone, and the first note becomes a C# note.
  • Transposing the melody up by another one semitone results in the first note becoming a D note, etc. After twelve such transpositions, the first note in the melody is back to a C note (now a full octave higher than the original C note).
  • D# natural minor D#m, E#dim, F#, G#m, A#m, B, C# 2, 4, 6, 7, 9, 11, 12
  • chord notations may represent the same chords.
  • the D#min-maj7 chord is the same chord as the Ebmin-maj7 chord.
  • a flow diagram illustrates example processor executable instructions for the characterization of a stem according to temporal compatibility.
  • the software receives a stem to analyze.
  • the software determines whether the stem has music note data 410 (e.g. MIDI) stored in association with the stem. If the associated music note data for the stem is available, the music note data is sent at block 2803 to the Categorization Module 301 for the characterization 2804 of melodic compatibility data of the stem. In the case that the stem does not have associated music note data 410 available, the stem audio is sent at block 2805 to the Categorization Module 301 for the characterization 2804 of melodic compatibility data of the stem. The characterization will be carried out by analyzing the stem audio file instead of the music note data 410.
  • music note data 410 e.g. MIDI
  • the software provides the example processor executable instructions for the characterization of temporal compatibility data from a stem file 2804.
  • the software analyzes the notes present in the stem (either by analyzing the note data accompanying the stem, or by direct analysis of the audio data contained by the stem file in the case that no note data is available) to find the basic tempo of the stem, represented by beats per minute 403.
  • the software determines the time signature 404 of the stem.
  • the software determines whether the rhythm exhibits a swing beat or a regular beat (for example, in genres such as jazz, the swing beat is sometimes utilized), and other temporal compatibility data. Then, at block 2904, the software stores the resulting temporal compatibility in association the stem.
  • FIG. 30 a flow diagram illustrates example processor executable instructions for using the frequency fingerprint to make a ranked suggestion of stems. This may be applicable, for example, when a new stem is being added to the musical production, such as when a percussion track is being added in 1504 of FIG. 15, when the add layer button 1801 is selected in FIG. 18, or when stem alternatives are suggested by the software in 1806.
  • the software receives a first selected stem from the user.
  • the stem may be already present in the production (such as, for example, an existing song- starter), or may be added immediately prior to block 3001 (such as, for example, when a percussion track is added). Alternative, multiple stems may be already present in the production.
  • the software presents the Ul to receive a further stem layer selection. An example of an implementation of said Ul is shown in FIG. 14.
  • the software receives an intention to place another stem (for example, of a specified instrument type) into the production at the same time point as the previous stem(s) in the production.
  • the software obtains the frequency fingerprints 405 of the stem(s) currently in use at the said time point of the production, and stores this information temporarily for later use.
  • the software queries the stems database 308 to retrieve a list of all available stems matching the specified instrument type, with the stems further compatible temporally, melodically, and emotionally with the rest of the production.
  • a second stem will have a higher compatibility score with a first stem, if the second stem exhibits low amplitude values in its frequency fingerprint simultaneously when the first stem has high amplitude values in the frequency fingerprint, or vice versa.
  • a first stem chosen is a stem composed of a bass line, with high amplitude values present in the low-frequency bins of the frequency fingerprint.
  • Subsequent suggestions for additional stems will therefore rank the stems exhibiting high amplitude values in the low-frequency range as low compatibility in terms of textural compatibility.
  • subsequent stems with low amplitude values in the lower frequency ranges may receive a higher textural compatibility score. This compatibility evaluation helps to avoid buildup in certain frequency ranges.
  • Another example of computing textural compatibility includes aligning the time segments of a first frequency fingerprint of the first stem with a second frequency fingerprint of the second stem.
  • the example computation then includes calculating a sum of amplitude values of the first and the second frequency fingerprints during a given time segment.
  • the software identifies a number of amplitude peaks greater than an amplitude threshold.
  • the textural compatibility score between the first and the second stems is computed such that the textural compatibility score is inversely proportional to the number the number of amplitude peaks greater than the amplitude threshold.
  • the amplitude threshold can be computed a number of ways based on the frequency fingerprint of the stems, or can be a predetermined value. In this way, a greater number of such amplitude peaks leads to a lower compatibility score between stems.
  • the potentially compatible stems are ranked according to the textural compatibility score.
  • the software presents the ranked list of suggested stems to the user on a provided Ul.
  • the potentially compatible stems as presented in 3008 may be ranked based on a plurality of criteria, or a combination of a plurality of criteria, including but not limited to, emotion similarity, textural compatibility, melodic compatibility, temporal compatibility, user rating, etc. If the user selects an additional stem at block 3008, the software returns 3009 to block 3003 to receive further stem additions.
  • FIG. 31 is a flow diagram illustrating example processor executable instructions for using the frequency fingerprint to identify and rectify frequency and amplitude-related problems. This may be applicable, for example, during the mixing and mastering stage of the production, such as when the software presents the final produced audio track to the user at block 2105 of FIG. 21 .
  • the software receives an intention to prepare for the exportation of the production (for example, when the Ul element 2102 is selected).
  • the software presents the relevant Ul for song exportation.
  • the software at block 3103, divides the song into segments (e.g. "Intro”, “Verse”, “Chorus”, etc.) according to song structure.
  • the software selects the first song segment for processing.
  • the frequency fingerprint 2303a of the audio including, for example, the stems and applied effects
  • the frequency fingerprint is compared with a desired frequency fingerprint, with the desired frequency fingerprint being specified by the user, by the software through the user's guidance (for example, through the selection of a particular genre), by the software automatically, etc.
  • the software identifies amplitude peaks present in the frequency bins resulting from the analysis performed by block 3106.
  • Amplitude peaks are unwanted or excessive spikes in the amplitude at certain frequencies, and may, for example, be musically unpleasing, or sonically overwhelm other, quieter parts of the production.
  • an excessive amplitude peak may be identified by analyzing the amplitude peaks within a specific frequency range or ranges relative to the average amplitude of the entire frequency spectrum.
  • an amplitude peak of a specific frequency range is greater than the average amplitude of the entire frequency spectrum by some set value, or by some percentage value, then the amplitude peak is considered excessive.
  • an excessive amplitude peak may be identified if the amplitude within a specific frequency range or ranges exceeds that of a threshold cut-off value.
  • the software determines whether any amplitude peaks exist in the given song segment. In the case that there are no amplitude peaks in the current song segment, the software at block 31 13 proceeds to analyze the next song segment 31 14, until all song segments have been processed. In the case that amplitude peaks exist in the current song segment, the software proceeds to block 3109 to analyze the frequency fingerprint of each stem present in the song segment to determine the combination of stem(s) that contribute to the amplitude peak.
  • the software identifies the specific stem(s) that contribute to the amplitude peaks, and at block 31 1 1 applies an audio equalizer to the said stem(s) to reduce the amplitude values in the peak frequencies.
  • the software characterizes the resulting frequency fingerprint of the corrected song segment, and determines that the frequency fingerprint is now compliant with the target frequency response.
  • the software proceeds to analyze the next song segment, until all song segments have been processed. The process is finished 31 15 after all song segments in the production have been analyzed and adjusted.
  • example processor executable instructions for using melodic compatibility data 406 to identify and suggest compatible stems are shown. The instructions may be implemented, for example, in the Melodic Compatibility Module 302 in an embodiment of the MCM 212.
  • the software receives a first stem selection from the user.
  • the software obtains the melodic compatibility data of said stem (including, for example, compatible chord progressions or compatible notes).
  • the software queries the stems database 308 to retrieve a list of all available stems that are melodically compatible with the current stem, with the stems further compatible temporally, texturally, and emotionally with the rest of the production.
  • the software presents the filtered stems on a provided Ul (e.g. on the Ul component 1409) for user selection.
  • example processor executable instructions for using temporal compatibility data to identify and suggest compatible stems may be implemented, for example, in the Rhythmic Compatibility Module 304 in an embodiment of the MCM 212.
  • Examples of temporal compatibility data may include, but are not limited to, BPM 403, time signature 404, the presence of a swing beat or straight beat 608, etc.
  • the software receives a first stem selection from the user.
  • the software obtains the temporal compatibility data of said stem (including, for example, the bpm, time signature, or presence of straight or swing beat, etc.).
  • the software queries the stems database 308 to retrieve a list of all available stems that are temporally compatible with the current stem, with the stems further compatible texturally, melodically, and emotionally with the rest of the production.
  • the software presents the filtered stems on a provided Ul (e.g. on the Ul component 1409) for user selection.
  • a method performed by a computing device for combining stems in audio production includes: presenting a graphical user interface (GUI) to display stem criteria for user selection; receiving one or more selected stem criteria via the GUI; obtaining filtered stems from a stem database that match the one or more selected stem criteria; displaying one or more song-starters for user selection, the one or more song-starters being a subset of the filtered stems; receiving a selected song-starter from the one or more song-starters via the GUI; obtaining at least one of temporal data, melodic data and textural data from the selected song-starter, the textural data comprising frequency data of the selected song-starter; determining further filtered stems, which are another subset of the filtered stems, that are compatible with the selected song-starter based on at least one of the temporal data, the melodic data and the textural data; displaying the further filtered stems for user selection; receiving one or more selected further filtered
  • the textural data comprises the frequency data in relation to time and amplitude.
  • the textural data comprises a frequency fingerprint, the frequency fingerprint comprising multiple time segments of data of the selected song- starter, each one of the time segments of data comprising the frequency data mapped in relation to amplitude.
  • the given further filtered stem when a given further filtered stem exhibits low amplitude values in its frequency fingerprint at a same time segment when the selected song-starter has high amplitude values in its frequency fingerprint, the given further filtered stem is determined to be texturally compatible with the selected song-starter.
  • the method further includes: calculating a sum of all frequency fingerprints of the one or more selected further filtered stems and the selected song-starter during a given time segment; identifying an amplitude peak greater than an amplitude threshold in a given frequency range; and applying an audio equalizer to the one or more selected further filtered stems to reduce an amplitude value in the given frequency range.
  • the stem criteria further include genre and emotion.
  • the method further includes displaying one or more audio effects for user selection to add to the audio track.
  • the GUI when presenting the GUI for the tempo selection, includes playback controls for playing different stems, each having a different tempo, and the GUI includes a control to select one of the different stems to allow a user to make the tempo selection.
  • the one or more song-starters when displaying the one or more song-starters for user selection, are organized by instrument categories.
  • the melodic data includes at least one of one or more scales and one or more keys that are compatible with the song-starter, and the further filtered stems are determined to be characterized by at least one of the one or more scales and the one or more keys.
  • a method performed by a computing device for determining textural compatibility between at least two stems in audio production includes: obtaining a first stem and a second stem; obtaining a first frequency fingerprint and a second frequency fingerprint for each of the first and the second stem, each one of the frequency fingerprints comprising multiple time segments of data, each one of the time segments of data comprising frequency data and amplitude data mapped in relation to each other; aligning the time segments of the first and the second frequency fingerprints; calculating a sum of amplitude values of the first and the second frequency fingerprints during a given time segment; identifying a number of amplitude peaks greater than an amplitude threshold; and computing a textural compatibility score between the first and the second stems, the textural compatibility score inversely proportional to the number the number of amplitude peaks.
  • the method further includes: determining another number of instances when one of the first and the second frequency fingerprints comprises low amplitude values at a same time segment when the other one of the first and the second frequency fingerprints comprises high amplitude values; and further computing the textural compatibility score between the first and the second stems to be proportional to the other number of instances.
  • the method further includes: obtaining a third stem and a third frequency fingerprint; aligning the time segments of the first and the third frequency fingerprints; computing another textural compatibility score between the first and the third stem by comparing the first and the third frequency fingerprints; and ranking the second stem as being more compatible with the first stem, compared to the third stem and the first stem, when the textural compatibility score is greater than the other textural compatibility score.
  • the frequency data includes a set of frequency bands and the amplitude data comprises a set of average amplitude values, each one of the average amplitude values corresponding to an average amplitude value over each one of the set of frequency bands.
  • a method performed by a computing device for determining melodic compatibility between at least two stems includes: obtaining a first stem and a second stem; determining at least one of one or more scales and one or more keys that are compatible with notes in the first stem; and if notes of the second stem are characterized to match at least one of the one or more scale and the one or more keys, determining the second stem is compatible with the first stem.
  • the method further includes increasing or decreasing a compatibility rating between the first and the second stem based on a number of times that the first and the second stem were previously combined in audio production.
  • the method further includes increasing or decreasing a compatibility rating between the first and the second stem based on a user feedback rating of the second stem.
  • a method performed by a computing device for characterizing a stem includes: obtaining textural data, the textural data comprising a frequency fingerprint of the stem, the frequency fingerprint comprising multiple time segments of data, each one of the time segments of data comprising frequency data and amplitude data mapped in relation to each other; obtaining melodic data comprising musical notes and a musical key; obtaining temporal data comprising beats per minute and time signature; and storing the textural data, the melodic data and the temporal data in association with the stem in a database.
  • the method further includes obtaining at least one of a genre, an emotion, and an instrument of the stem.
  • the method further includes using the characterized information to determine compatibility when combining the stem with one or more other stems in audio production.

Abstract

L'invention concerne des systèmes et des procédés permettant de catégoriser des pistes ("stems") instrumentales (par exemple des données audio enregistrées à partir d'instruments numériques ou en direct ou de performances vocales, etc.) et de présenter ce matériau source à des utilisateurs possédant peu ou pas d'expérience musicale de manière à leur donner le contrôle total d'aspects de la chanson tels que le genre, le tempo, l'instrumentation spécifique, la clé, la mélodie, les effets audio et la structure de la chanson dans un environnement de production musicale. La base de données de stems disponibles va être filtrée de manière dynamique sur la base de chaque sélection ou personnalisation effectuée par l'utilisateur de manière à présenter aux utilisateurs exclusivement des recommandations de stems qui sont compatibles sur le plan temporel, mélodique, émotionnel et textural avec la somme des choix qu'ils ont déjà sélectionnés. Le système peut comprendre un site internet en ligne accessible par le biais d'un navigateur Web ou sous la forme d'une application pouvant être téléchargée sur des appareils informatiques de l'utilisateur.
PCT/CA2014/050368 2014-04-10 2014-04-10 Systèmes et procédés pour analyse musicale et détermination de compatibilité dans une production audio WO2015154159A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CA2014/050368 WO2015154159A1 (fr) 2014-04-10 2014-04-10 Systèmes et procédés pour analyse musicale et détermination de compatibilité dans une production audio

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CA2014/050368 WO2015154159A1 (fr) 2014-04-10 2014-04-10 Systèmes et procédés pour analyse musicale et détermination de compatibilité dans une production audio

Publications (1)

Publication Number Publication Date
WO2015154159A1 true WO2015154159A1 (fr) 2015-10-15

Family

ID=54287037

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA2014/050368 WO2015154159A1 (fr) 2014-04-10 2014-04-10 Systèmes et procédés pour analyse musicale et détermination de compatibilité dans une production audio

Country Status (1)

Country Link
WO (1) WO2015154159A1 (fr)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10349196B2 (en) 2016-10-03 2019-07-09 Nokia Technologies Oy Method of editing audio signals using separated objects and associated apparatus
US10514882B2 (en) 2018-02-21 2019-12-24 Microsoft Technology Licensing, Llc Digital audio processing system for adjoining digital audio stems based on computed audio intensity/characteristics
WO2021023296A1 (fr) 2019-08-08 2021-02-11 Harman International Industries, Incorporated Procédé et système permettant de réaliser une mise en correspondance et un ajout intelligents de pistes rythmiques
US10997986B2 (en) 2019-09-19 2021-05-04 Spotify Ab Audio stem identification systems and methods
US11195502B2 (en) 2016-06-30 2021-12-07 Lifescore Limited Apparatus and methods for cellular compositions
US11238839B2 (en) 2019-09-19 2022-02-01 Spotify Ab Audio stem identification systems and methods
EP3864647A4 (fr) * 2018-10-10 2022-06-22 Accusonus, Inc. Procédé et système de traitement de stems audio
EP4068273A3 (fr) * 2021-03-31 2022-11-09 DAACI Limited Système et procédés pour générer automatiquement une composition musicale ayant une forme audiblement correcte
US11514877B2 (en) 2021-03-31 2022-11-29 DAACI Limited System and methods for automatically generating a musical composition having audibly correct form
WO2023112010A3 (fr) * 2021-12-15 2023-07-27 Distributed Creation Inc. Génération à base de similarité évolutive de mélanges musicaux compatibles

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050120868A1 (en) * 1999-10-18 2005-06-09 Microsoft Corporation Classification and use of classifications in searching and retrieval of information
US20090260507A1 (en) * 2008-04-22 2009-10-22 Peter Gannon Systems and methods for composing music
US20120222540A1 (en) * 2011-03-02 2012-09-06 Yamaha Corporation Generating tones by combining sound materials

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050120868A1 (en) * 1999-10-18 2005-06-09 Microsoft Corporation Classification and use of classifications in searching and retrieval of information
US20090260507A1 (en) * 2008-04-22 2009-10-22 Peter Gannon Systems and methods for composing music
US20120222540A1 (en) * 2011-03-02 2012-09-06 Yamaha Corporation Generating tones by combining sound materials

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4020459A1 (fr) * 2016-06-30 2022-06-29 Lifescore Limited Appareils et procédés pour les compositions cellulaires
US11881195B2 (en) * 2016-06-30 2024-01-23 Lifescore Limited Apparatus and methods for cellular compositions
US11195502B2 (en) 2016-06-30 2021-12-07 Lifescore Limited Apparatus and methods for cellular compositions
EP3479372B1 (fr) * 2016-06-30 2022-02-16 Lifescore Limited Appareil et procédés pour compositions musicales cellulaires
US20220157283A1 (en) * 2016-06-30 2022-05-19 Lifescore Limited Apparatus and Methods for Cellular Compositions
US10349196B2 (en) 2016-10-03 2019-07-09 Nokia Technologies Oy Method of editing audio signals using separated objects and associated apparatus
US10623879B2 (en) 2016-10-03 2020-04-14 Nokia Technologies Oy Method of editing audio signals using separated objects and associated apparatus
US10514882B2 (en) 2018-02-21 2019-12-24 Microsoft Technology Licensing, Llc Digital audio processing system for adjoining digital audio stems based on computed audio intensity/characteristics
EP3864647A4 (fr) * 2018-10-10 2022-06-22 Accusonus, Inc. Procédé et système de traitement de stems audio
EP4010901A4 (fr) * 2019-08-08 2023-09-06 Harman International Industries, Incorporated Procédé et système permettant de réaliser une mise en correspondance et un ajout intelligents de pistes rythmiques
WO2021023296A1 (fr) 2019-08-08 2021-02-11 Harman International Industries, Incorporated Procédé et système permettant de réaliser une mise en correspondance et un ajout intelligents de pistes rythmiques
US10997986B2 (en) 2019-09-19 2021-05-04 Spotify Ab Audio stem identification systems and methods
US11238839B2 (en) 2019-09-19 2022-02-01 Spotify Ab Audio stem identification systems and methods
US11568886B2 (en) 2019-09-19 2023-01-31 Spotify Ab Audio stem identification systems and methods
US11514877B2 (en) 2021-03-31 2022-11-29 DAACI Limited System and methods for automatically generating a musical composition having audibly correct form
US11646007B1 (en) 2021-03-31 2023-05-09 DAACI Limited Generative composition with texture groups
EP4068273A3 (fr) * 2021-03-31 2022-11-09 DAACI Limited Système et procédés pour générer automatiquement une composition musicale ayant une forme audiblement correcte
US11842710B2 (en) 2021-03-31 2023-12-12 DAACI Limited Generative composition using form atom heuristics
WO2022207765A3 (fr) * 2021-03-31 2022-11-10 DAACI Limited Système et procédés de génération automatique d'une composition musicale ayant une forme audible correcte
US11887568B2 (en) 2021-03-31 2024-01-30 DAACI Limited Generative composition with defined form atom heuristics
WO2023112010A3 (fr) * 2021-12-15 2023-07-27 Distributed Creation Inc. Génération à base de similarité évolutive de mélanges musicaux compatibles

Similar Documents

Publication Publication Date Title
WO2015154159A1 (fr) Systèmes et procédés pour analyse musicale et détermination de compatibilité dans une production audio
US9563701B2 (en) Sound data processing device and method
US8710343B2 (en) Music composition automation including song structure
JP5007563B2 (ja) 音楽編集装置および方法、並びに、プログラム
US9053696B2 (en) Searching for a tone data set based on a degree of similarity to a rhythm pattern
JP5982980B2 (ja) 楽音発生パターンを示すクエリーを用いて演奏データの検索を行う装置、方法および記憶媒体
EP3792909A1 (fr) Procédé et système de construction de chanson hybride basée sur l'ia
US20210326102A1 (en) Method and device for determining mixing parameters based on decomposed audio data
JP5970934B2 (ja) 楽音発生パターンを示すクエリーを用いて演奏データの検索を行う装置、方法および記録媒体
US11837207B2 (en) Method and system for generating an audio or MIDI output file using a harmonic chord map
AU2009295348A1 (en) Video and audio content system
US20110271187A1 (en) Musical Composition System
Hargreaves et al. Structural segmentation of multitrack audio
JP2010066366A (ja) 音楽情報処理装置、音楽配信システムおよび音楽情報処理方法
JP2008276169A (ja) オンライン作曲のためのシステム、方法及びコンピュータ読取り可能な記録媒体
JP5879996B2 (ja) 音信号生成装置及びプログラム
RU2808611C2 (ru) Способ и система для генерирования выходного звукового файла или файла midi посредством гармонической карты аккордов
EP4024392A1 (fr) Procédé et système de construction de chanson basés sur l'énergie
EP4020458A1 (fr) Procédé et système de génération de variante basée sur modèle de chanson hybride générée par ai
Collins In the Box Music Production: Advanced Tools and Techniques for Pro Tools
Fonseca Musical Cross Synthesis using Matrix Factorisation
EP2793222B1 (fr) Procédé de mise en oeuvre d'une session automatique de musique jam.
Brené et al. How Musical Instrumentation Affects Perceptual Identification of Musical Genres
GB2507284A (en) Mixing multimedia tracks including tempo adjustment to achieve correlation of tempo between tracks

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14888837

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14888837

Country of ref document: EP

Kind code of ref document: A1