WO2015114216A3 - Audio segments analysis to determine danceability of a music and for video and pictures synchronisaton to the music. - Google Patents
Audio segments analysis to determine danceability of a music and for video and pictures synchronisaton to the music. Download PDFInfo
- Publication number
- WO2015114216A3 WO2015114216A3 PCT/FI2015/050059 FI2015050059W WO2015114216A3 WO 2015114216 A3 WO2015114216 A3 WO 2015114216A3 FI 2015050059 W FI2015050059 W FI 2015050059W WO 2015114216 A3 WO2015114216 A3 WO 2015114216A3
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- music
- segment
- audio signal
- audio
- video
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/36—Accompaniment arrangements
- G10H1/40—Rhythm
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/43—Querying
- G06F16/438—Presentation of query results
- G06F16/4387—Presentation of query results by the use of playlists
- G06F16/4393—Multimedia presentations, e.g. slide shows, multimedia albums
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/0008—Associated control or indicating means
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H1/00—Details of electrophonic musical instruments
- G10H1/36—Accompaniment arrangements
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/041—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal based on mfcc [mel -frequency spectral coefficients]
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/051—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for extraction or detection of onsets of musical sounds or notes, i.e. note attack timings
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/061—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for extraction of musical phrases, isolation of musically relevant segments, e.g. musical thumbnail generation, or for temporal structure analysis of a musical piece, e.g. determination of the movement sequence of a musical work
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/071—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for rhythm pattern analysis or rhythm style recognition
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
- G10H2210/031—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
- G10H2210/076—Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal for extraction of timing, tempo; Beat detection
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2240/00—Data organisation or data communication aspects, specifically adapted for electrophonic musical tools or instruments
- G10H2240/075—Musical metadata derived from musical analysis or for use in electrophonic musical instruments
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
- G10H2250/005—Algorithms for electrophonic musical instruments or musical processing, e.g. for automatic composition or resource allocation
- G10H2250/015—Markov chains, e.g. hidden Markov models [HMM], for musical processing, e.g. musical analysis or musical composition
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
- G10H2250/025—Envelope processing of music signals in, e.g. time domain, transform domain or cepstrum domain
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
- G10H2250/131—Mathematical functions for musical analysis, processing, synthesis or composition
- G10H2250/135—Autocorrelation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
- G10H2250/131—Mathematical functions for musical analysis, processing, synthesis or composition
- G10H2250/215—Transforms, i.e. mathematical transforms into domains appropriate for musical signal processing, coding or compression
- G10H2250/235—Fourier transform; Discrete Fourier Transform [DFT]; Fast Fourier Transform [FFT]
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G11B27/036—Insert-editing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
- H04N1/00132—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture in a digital photofinishing system, i.e. a system where digital photographic images undergo typical photofinishing processing, e.g. printing ordering
- H04N1/00185—Image output
- H04N1/00196—Creation of a photo-montage, e.g. photoalbum
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/233—Processing of audio elementary streams
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Acoustics & Sound (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Auxiliary Devices For Music (AREA)
Abstract
A technique for audio processing that comprises obtaining four types of features descriptive of characteristics of a segment of audio signal representing a piece of music and, based on these features, deriving a "club score" that is indicative of at least beat strength associated with said segment of audio signal, thus describing a "danceablility" of music. An application comprises obtaining one or more audio attributes characterizing a segment of audio signal representing the piece of music, calculating a club score, and selecting a switching pattern from a plurality of predetermined switching patterns based on the club score, wherein a switching pattern is arranged to indicate discontinuities, or temporal positions and/or frequency of changes of video sources or image, in a visual content associated with said segment of audio signal, in relation to temporal locations of beats or downbeats identified for the segment of audio signal, for example for generating a visual presentation to accompany remixed music, synchronised to the music.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB1401626.5A GB2522644A (en) | 2014-01-31 | 2014-01-31 | Audio signal analysis |
GB1401626.5 | 2014-01-31 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2015114216A2 WO2015114216A2 (en) | 2015-08-06 |
WO2015114216A3 true WO2015114216A3 (en) | 2015-11-19 |
Family
ID=50344136
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/FI2015/050059 WO2015114216A2 (en) | 2014-01-31 | 2015-01-30 | Audio signal analysis |
Country Status (2)
Country | Link |
---|---|
GB (1) | GB2522644A (en) |
WO (1) | WO2015114216A2 (en) |
Families Citing this family (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10014841B2 (en) | 2016-09-19 | 2018-07-03 | Nokia Technologies Oy | Method and apparatus for controlling audio playback based upon the instrument |
WO2019053766A1 (en) * | 2017-09-12 | 2019-03-21 | Pioneer DJ株式会社 | Song analysis device and song analysis program |
CN111243618B (en) * | 2018-11-28 | 2024-03-19 | 阿里巴巴集团控股有限公司 | Method, device and electronic equipment for determining specific voice fragments in audio |
GB2583441A (en) * | 2019-01-21 | 2020-11-04 | Musicjelly Ltd | Data synchronisation |
CN113223487B (en) * | 2020-02-05 | 2023-10-17 | 字节跳动有限公司 | Information identification method and device, electronic equipment and storage medium |
CN112435641B (en) * | 2020-11-09 | 2024-01-02 | 腾讯科技(深圳)有限公司 | Audio processing method, device, computer equipment and storage medium |
CN115250360A (en) * | 2021-04-27 | 2022-10-28 | 北京字节跳动网络技术有限公司 | Rhythm interaction method and equipment |
CN113590076B (en) * | 2021-07-12 | 2024-03-29 | 杭州网易云音乐科技有限公司 | Audio processing method and device |
CN113674723B (en) * | 2021-08-16 | 2024-05-14 | 腾讯音乐娱乐科技(深圳)有限公司 | Audio processing method, computer equipment and readable storage medium |
CN113793580B (en) * | 2021-08-31 | 2024-05-24 | 云境商务智能研究院南京有限公司 | Music genre classification method based on deep learning |
CN114268814A (en) * | 2021-11-29 | 2022-04-01 | 广州繁星互娱信息科技有限公司 | Music video acquisition method and device, storage medium and electronic equipment |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040027369A1 (en) * | 2000-12-22 | 2004-02-12 | Peter Rowan Kellock | System and method for media production |
US20050217462A1 (en) * | 2004-04-01 | 2005-10-06 | Thomson J Keith | Method and apparatus for automatically creating a movie |
WO2011051279A1 (en) * | 2009-10-30 | 2011-05-05 | Dolby International Ab | Complexity scalable perceptual tempo estimation |
SG178778A1 (en) * | 2007-03-02 | 2012-03-29 | Animoto Llc | Automatically generating audiovisual works |
WO2013164661A1 (en) * | 2012-04-30 | 2013-11-07 | Nokia Corporation | Evaluation of beats, chords and downbeats from a musical audio signal |
WO2014001849A1 (en) * | 2012-06-29 | 2014-01-03 | Nokia Corporation | Audio signal analysis |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080300702A1 (en) * | 2007-05-29 | 2008-12-04 | Universitat Pompeu Fabra | Music similarity systems and methods using descriptors |
US20130275421A1 (en) * | 2010-12-30 | 2013-10-17 | Barbara Resch | Repetition Detection in Media Data |
-
2014
- 2014-01-31 GB GB1401626.5A patent/GB2522644A/en not_active Withdrawn
-
2015
- 2015-01-30 WO PCT/FI2015/050059 patent/WO2015114216A2/en active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040027369A1 (en) * | 2000-12-22 | 2004-02-12 | Peter Rowan Kellock | System and method for media production |
US20050217462A1 (en) * | 2004-04-01 | 2005-10-06 | Thomson J Keith | Method and apparatus for automatically creating a movie |
SG178778A1 (en) * | 2007-03-02 | 2012-03-29 | Animoto Llc | Automatically generating audiovisual works |
WO2011051279A1 (en) * | 2009-10-30 | 2011-05-05 | Dolby International Ab | Complexity scalable perceptual tempo estimation |
WO2013164661A1 (en) * | 2012-04-30 | 2013-11-07 | Nokia Corporation | Evaluation of beats, chords and downbeats from a musical audio signal |
WO2014001849A1 (en) * | 2012-06-29 | 2014-01-03 | Nokia Corporation | Audio signal analysis |
Non-Patent Citations (3)
Title |
---|
DANIEL P. W. ELLIS: "Beat Tracking by Dynamic Programming", JOURNAL OF NEW MUSIC RESEARCH, vol. 36, no. 1, 16 July 2007 (2007-07-16), pages 51 - 60, XP055177341, ISSN: 0929-8215, DOI: 10.1080/09298210701653344 * |
ELIAS PAMPALK: "Computational Models of Music Similarity and their Application in Music Information Retrieval", DOCTOR THESIS, 1 March 2006 (2006-03-01), Wien, XP055177322, Retrieved from the Internet <URL:http://www.ofai.at/~elias.pampalk/publications/pampalk06thesis.pdf> [retrieved on 20150317] * |
HERRERA PERFECTO ET AL: "Detrended Fluctuation Analysis of Music Signals: Danceability Estimation and Further Semantic Characterization", AES CONVENTION 118; MAY 2005, AES, 60 EAST 42ND STREET, ROOM 2520 NEW YORK 10165-2520, USA, 1 May 2005 (2005-05-01), XP040507217 * |
Also Published As
Publication number | Publication date |
---|---|
GB201401626D0 (en) | 2014-03-19 |
GB2522644A (en) | 2015-08-05 |
WO2015114216A2 (en) | 2015-08-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2015114216A3 (en) | Audio segments analysis to determine danceability of a music and for video and pictures synchronisaton to the music. | |
EP2824663A3 (en) | Audio processing apparatus | |
EP4254145A3 (en) | Head pose mixing of audio files | |
WO2018013192A3 (en) | Extraction of features from physiological signals | |
EP3032537A3 (en) | Proximity based temporary audio sharing | |
WO2016191737A3 (en) | Systems and methods for proactively identifying and surfacing relevant content on a touch-sensitive device | |
EP3531714A3 (en) | Facilitating calibration of an audio playback device | |
IN2013CH03069A (en) | ||
MY184715A (en) | Apparatus and method for screen related audio object remapping | |
JP2007248895A5 (en) | ||
WO2018085613A3 (en) | Intuitive occluded object indicator | |
EP2980758A3 (en) | Method and device for providing image | |
WO2015008469A3 (en) | Information processing apparatus, information processing method, and information processing system | |
MX2015004848A (en) | Method relating to presence granularity with augmented reality. | |
WO2014150780A3 (en) | Determination of joint condition based on vibration analysis | |
WO2011041424A4 (en) | Providing visual responses to musically synchronized touch input | |
AU366258S (en) | A display screen or portion thereof with an image from a sequence of images forming an animated graphical user interface | |
MX2016000843A (en) | Playback control method and apparatus, and electronic device. | |
EP3383036A3 (en) | Information processing device, information processing method, and program | |
EP2818215A3 (en) | Method and system for expressing emotion during game play | |
WO2013072554A3 (en) | Spatial visual effect creation and display such as for a screensaver | |
EP4239498A3 (en) | Image selection suggestions | |
EP2932889A3 (en) | Apparatus for performing multidimensional velocity measurements using amplitude and phase in optical interferometry | |
MY195593A (en) | Animated Character Head Systems And Methods | |
PH12016000288A1 (en) | Game information analysis system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15704581 Country of ref document: EP Kind code of ref document: A2 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 15704581 Country of ref document: EP Kind code of ref document: A2 |