WO2020017732A1

WO2020017732A1 - Method and apparatus for frequency based sound equalizer configuration prediction

Info

Publication number: WO2020017732A1
Application number: PCT/KR2019/003396
Authority: WO
Inventors: Ashish Chopra; Kaushal Prakash SHARMA; Kanishk NAGPAL; Rajan Dahiya
Original assignee: Samsung Electronics Co., Ltd.
Priority date: 2018-07-17
Filing date: 2019-03-22
Publication date: 2020-01-23

Abstract

Embodiments herein provide a method for frequency-based sound equalizer configuration prediction. The method includes receiving at least one audio content and decoding the at least one audio content to extract byte streams. The method also includes performing a frequency analysis of each of the byte streams of the at least one audio content and predicting at least one sound equalizer configuration for the at least one audio content based on the frequency analysis. Further, the method also includes storing the at least one sound equalizer configuration.

Description

METHOD AND APPARATUS FOR FREQUENCY BASED SOUND EQUALIZER CONFIGURATION PREDICTION

The disclosure relates to wireless communication, more particularly relates to method and electronic device for frequency-based sound equalizer configuration prediction.

In general, electronic devices include various applications for playing various kinds of multimedia content such as audio tracks, video files, etc. The electronic devices which can play the multimedia content are equipped with sound equalizers to accommodate different listening preferences of users for an audio track. Traditionally, a genre is used to categorize the audio tracks and provide a specific sound equalizer configuration for each genre. The user is also allowed to customize the sound equalizer configuration while listening to the audio track on the electronic device.

For example, consider the user of the electronic device has a collection of various audio tracks i.e., audio track 1, audio track 2 and audio track 3, etc. When the user of the electronic device plays the audio track 1, default sound equalizer configuration is applied to the audio track 1. Further, all the audio tracks are played with the default sound equalizer configuration irrespective of the change in audio tracks, as shown in FIG. 1. Further, if the user prefers to play the audio track 1 with a 'pop' genre, then even if the track is changed, the same genre is continued.

In another example, the user of the electronic device customizes the sound equalizer configuration for the audio track 1 based on user's listening preference. When the user changes to audio track 2, the sound equalizer configuration which was customized by the user for audio track 1 is continued for audio track 2, as shown in the FIG. 1. If the user wishes to use a different equalizer setting for the audio track 2, then the user will have to manually change the equalizer setting which reduces ease of operation and makes the process cumbersome if the user continuously changes the audio tracks.

The above information is presented as background information only to help the reader to understand the present invention. Applicants have made no determination and make no assertion as to whether any of the above might be applicable as prior art with regard to the present application.

According to an embodiment of the disclosure, an electronic device and a method are provided for frequency-based sound equalizer configuration prediction.

In one embodiment, a method is provided. The method includes receiving at least one audio content and decoding the at least one audio content to extract byte streams. The method also includes performing a frequency analysis of each of the byte streams of the at least one audio content and predicting at least one sound equalizer configuration for the at least one audio content based on the frequency analysis. Further, the method also includes storing the at least one sound equalizer configuration.

In another embodiment, an electronic device is provided. The electronic device includes a memory, a processor and a sound equalizer configuration engine. The sound equalizer configuration engine is configured to receive at least one audio content and decode the at least one audio content to extract byte streams. Further, the sound equalizer configuration engine is also configured to perform a frequency analysis of each of the byte streams of the at least one audio content and predict at least one sound equalizer configuration for the at least one audio content based on the frequency analysis. Furthermore, the sound equalizer configuration engine is also configured to store the at least one sound equalizer configuration.

In another embodiment, a method is provided. The method includes performing a frequency analysis for audio contents. The method also includes determining a sound equalizer configuration for the audio contents based on a result of the frequency analysis. Further, the method also includes storing the sound equalizer configuration.

In another embodiment, an electronic device is provided. The electronic device includes a memory, a processor coupled to the memory, a communicator coupled to the memory and the processor, and a sound equalizer configuration engine coupled to the memory and the processor. The sound equalizer configuration engine is configured to perform a frequency analysis for audio contents. The sound equalizer configuration engine is also configured to determine a sound equalizer configuration for the audio contents based on a result of the frequency analysis. Further, the sound equalizer configuration engine is also configured to store the sound equalizer configuration.

Embodiments of the disclosure provide a method includes predicting the sound equalizer configuration for the at least one audio track based on the analysis of the content of the audio track.

FIG. 1 is an example illustrating various cases of applying a sound equalizer configuration for at least one audio content, according to a prior art;

FIG. 2A is an example illustrating different sound equalizer configurations predicted for different audio tracks based on a frequency analysis, according to an embodiment as disclosed herein;

FIG. 2B is an example illustrating the at least one sound equalizer configuration predicted for a cluster based on the frequency analysis, according to an embodiment as disclosed herein;

FIG. 3A is a block diagram illustrating various hardware elements of an electronic device for frequency-based sound equalizer configuration prediction, according to an embodiment as disclosed herein;

FIG. 3B is a block diagram illustrating various hardware elements of a sound equalizer configuration engine, according to an embodiment as disclosed herein;

FIG. 4A is a flow chart illustrating a method for the frequency-based sound equalizer configuration prediction, according to an embodiment as disclosed herein;

FIG. 4B is a flow chart illustrating a method for performing the frequency analysis, according to an embodiment as disclosed herein;

FIG. 4C is a flow chart illustrating a method for predicting the at least one sound equalizer configuration for the at least one audio content based on the frequency analysis, according to an embodiment as disclosed herein;

FIG. 5 illustrates decoding the at least one audio content to extract byte streams, according to an embodiment as disclosed herein;

FIG. 6A illustrates a range of frequencies for the audio track, according to an embodiment as disclosed herein;

FIG. 6B illustrates generation of a soothing curve for the audio track, according to an embodiment as disclosed herein;

FIG. 6C illustrates spectrogram generation for the at least one audio track, according to an embodiment as disclosed herein;

FIG. 7 illustrates the generation of a frequency data set from frequency values, according to an embodiment as disclosed herein;

FIG. 8 illustrates formation of clusters and prediction of the sound equalizer configuration for each cluster, according to an embodiment as disclosed herein;

FIGS. 9A-9B are examples illustrating prediction of the sound equalizer configuration for the cluster based on a weighted mean average of the frequency analysis and a user preferred sound equalizer configuration for the at least one audio tracks of the cluster, according to an embodiment as disclosed herein;

FIG. 10 is an example illustrating application of the at least one sound equalizer configuration and adaption of the sound equalizer configuration modified by the user for the audio tracks in the cluster, according to an embodiment as disclosed herein;

FIGS. 11A-11B is an example illustrating various functions that can be performed using the cluster of audio tracks, according to an embodiment as disclosed herein; and

FIG. 12 is an example illustrating prediction of the sound equalizer configuration for the cluster based on user preferred playback parameters for the at least one audio track, according to an embodiment as disclosed herein.

The principal object of the embodiments herein is to provide an electronic device and method for frequency-based sound equalizer configuration prediction.

Another object of the embodiments herein is to decode an at least one audio content to extract byte streams.

Another object of the embodiments herein is to perform a frequency analysis of each of the byte streams of the at least one audio content.

Another object of the embodiments herein is to predict the sound equalizer configuration for the at least one audio content based on the frequency analysis.

Another object of the embodiments herein is to categorize the at least one audio content into a cluster using a machine learning technique based on the frequency analysis.

Another object of the embodiments herein is to predict the at least one sound equalizer configuration common for all audio content in the cluster based on the frequency analysis.

Another object of the embodiments herein is to customize the sound equalizer configuration of the at least one audio content of the cluster based on a weighted mean average of the data set of the at least one audio content of the cluster as per the frequency analysis.

Another object of the embodiments herein is to customize the sound equalizer configuration of the at least one audio content of the cluster based on a weighted mean average of at least one playback parameter of the at least one audio content of the cluster.

Various embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. In the following description, specific details such as detailed configuration and components are merely provided to assist the overall understanding of these embodiments of the present disclosure. Therefore, it should be apparent to those skilled in the art that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

Also, the various embodiments described herein are not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments.

Herein, the term "or" as used herein, refers to a non-exclusive or, unless otherwise indicated. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein can be practiced and to further enable those skilled in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.

As is traditional in the field, embodiments may be described and illustrated in terms of blocks which carry out a described function or functions.　These blocks, which may be referred to herein as units, engines, manager, modules or the like, are physically implemented by analog and/or digital circuits such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits and the like, and may optionally be driven by firmware and/or software. The circuits may, for example, be embodied in one or more semiconductor chips, or on substrate supports such as printed circuit boards and the like.　The circuits constituting a block may be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware to perform some functions of the block and a processor to perform other functions of the block.　Each block of the embodiments may be physically separated into two or more interacting and discrete blocks without departing from the scope of the disclosure.　Likewise, the blocks of the embodiments may be physically combined into more complex blocks without departing from the scope of the disclosure.

The term audio track and audio content have been used interchangeably throughout the specification; and both the terms means one and the same.

Accordingly, the embodiments herein provide a method for frequency-based sound equalizer configuration prediction. The method includes receiving at least one audio content and decoding the at least one audio content to extract byte streams. The method also includes performing a frequency analysis of each of the byte streams of the at least one audio content and predicting at least one sound equalizer configuration for the at least one audio content based on the frequency analysis. Further, the method also includes storing the at least one sound equalizer configuration.

In an embodiment, predicting by an electronic device the at least one sound equalizer configuration for the at least one audio content based on the frequency analysis includes obtaining frequency values over entire range of the at least one content audio content based on the frequency analysis. Further, the method also includes generating frequency data set from the obtained frequency values and clustering the at least one audio content into a cluster using a machine learning technique based on the frequency data set. Furthermore, the method also includes predicting the at least one sound equalizer configuration common for all audio content in the cluster based on the frequency data set.

In an embodiment, performing the frequency analysis includes performing a fast Fourier transform (FFT) on frequency values in entire stream of the at least one audio content and determining high magnitude frequency values in entire stream of the at least one audio content based on the FFT. Further, the method also includes generating a soothing curve by superimposing the high magnitude frequency values in the entire stream of the at least one audio content, where the soothing curve includes fixed number of the high magnitude frequency values.

In an embodiment, the method further includes detecting user inputs on the data set of at least one audio content of a cluster and performing a frequency analysis of byte streams of the at least one audio content of the cluster. Further, the method also includes customizing the sound equalizer configuration of the at least one audio content of the cluster based on a weighted mean average of the data set of the at least one audio content of the cluster as per the frequency analysis.

In an embodiment, the method further includes representing on a screen of the electronic device a plurality of clusters each of which is represented by a unique indicator, wherein audio contents belonging to each of the clusters has a common sound equalizer configuration. Further, the method also includes detecting a cluster selected from the plurality of cluster and displaying the audio contents belonging to the selected cluster or automatically sharing the audio contents belonging to the selected cluster.

In an embodiment, the method further includes playing back a first audio content from a plurality of audio contents belonging to a cluster, where each of the audio contents in the cluster has a common sound equalizer configuration. Further, the method includes detecting a change in at least one playback parameter of the first audio content and automatically customizing the sound equalizer configuration of remaining audio contents in the cluster based on the at least one changed playback parameter of the first audio content. Further, the method also includes detecting a second audio content selected from said cluster and playing back the second audio content with the customized sound equalizer configuration.

In conventional methods and systems, genre based sound equalizer configurations are provided where each genre has a predefined sound equalizer configuration. Unlike to the conventional methods and systems, the proposed method includes predicting the sound equalizer configuration for the at least one audio track based on the analysis of the content (i.e., frequency) of the audio track.

In conventional methods and systems, a default of 0db is provided for the audio tracks when the user does not apply any specific equalizer configuration. Unlike to the conventional methods and systems, the proposed method, automatically performs the frequency analysis on the audio track and provides a soothing curve setting.

In conventional methods and systems, the user has to manually apply the desirable sound equalizer configuration for each of the audio content. Further, the process of applying sound equalizer configuration for each of the audio content becomes cumbersome when the audio content being played is switched or changed since the sound equalizer configuration needs to be changed again as per the current audio content. Unlike to the conventional methods and systems, in the proposed method the electronic device saves the user preference of the sound equalizer configuration for each of the audio content and the sound equalizer configuration is automatically applied when the audio content is played.

In conventional methods and systems, the "auto" (automatic) sound equalizer configuration mode is provided in which the suggested sound equalizer configuration for the audio content is specified in a meta-data of the audio content. However, non-availability of suggested sound equalizer configuration in the meta-data of the audio content renders the "auto" mode useless.

In conventional methods and systems, the "auto" (automatic) sound equalizer configuration specified in the meta-data may not be suitable for the audio content every time the auto configuration is applied. Further, the suggested sound equalizer configuration may not be supported by the electronic device. Unlike to the conventional methods and systems, in the proposed method the electronic device performs the content based analysis of the audio content which will be compatible with the audio content and the electronic device.

Unlike to the conventional methods and systems, in the proposed method clustering is used to create similar groups of audio content which have similar frequency distribution, so that if the user applied some sound equalizer configuration to some audio content then the learning pattern will be applied to the remaining un-played audio contents.

Unlike to the conventional methods and systems, in the proposed method when the user modifies the sound equalizer configuration automatically applied by the electronic device, an unsupervised machine learning technique is applied to learn and adapt to the user's music listening pattern. Further, a modified sound equalizer configuration is applied to other audio content in the cluster to reduce the manual effort.

Unlike to the conventional methods and systems, in the proposed method the user preferences will be saved according to audio accessory being used, such as earphones, Bluetooth headset or default (Device's speaker). Therefore, due to saving of the sound equalizer configuration with respect to the audio accessory the difference in audio playback quality is addressed.

These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications.

Referring now to the drawings and more particularly to FIGS. 2A through 11, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments.

FIG. 2A is an example illustrating different sound equalizer configurations predicted for different audio tracks based on the frequency analysis, according to an embodiment as disclosed herein.

Referring to the FIG. 2A, consider that the electronic device 100 receives a plurality of audio tracks such as audio track 1, audio track 2, audio track 3 and audio track 4.

In the conventional methods and systems, the user will have to manually set the sound equalizer configuration for the specific audio track every time the specific audio track is played on the electronic device 100. Unlike to the conventional methods and systems, the proposed method provides the content based mechanism to determine the sound equalizer configuration according to which the electronic device 100 decodes each of the audio tracks and performs the frequency analysis on the decoded audio tracks. Further, the electronic device 100 automatically predicts the sound equalizer configuration (i.e., best suited sound equalizer configuration) for each of the audio tracks based on the frequency analysis of the audio track.

Further, if the user changes the sound equalizer configuration based on the listening preference, then the proposed method allows the electronic device 100 to save the user preferred sound equalizer configuration. Therefore, when the user plays the specific audio track, the user preferred sound equalizer configuration is automatically applied to the audio track.

FIG. 2B is an example illustrating the at least one sound equalizer configuration predicted for the cluster based on the frequency analysis, according to an embodiment as disclosed herein.

Referring to the FIG. 2B, in conjunction with FIG. 2A, the plurality of audio tracks are further divided into the clusters based on the frequency analysis. The audio tracks with similar frequency distribution are categorized into a same cluster. The audio track 1, the audio track 2 belong to a cluster 1 and the audio track 3, the audio track 4 belong to a cluster 2. Unlike to the conventional methods and systems, which categorize the audio tracks based on the genre, the proposed method allows the electronic device 100 to categorize the audio tracks into clusters based on the frequency analysis of the audio tracks. Further, the electronic device 100 predicts the sound equalizer configuration for each cluster.

In another embodiment, audio contents are categorized into clusters based on characteristic of the audio contents. The characteristic of the audio contents may be a which device is used to output the audio contents. That is, the at least one the audio contents outputted using a same device is categorized into a same cluster. For example, the audio track 1 and the audio track 2 outputted through a headphone belong to a cluster 1 and the audio track 3 and the audio track 4 outputted through a speaker belong to a cluster 2. For this categorizing, the electronic device detects devices used to output each of the audio contents, and groups the audio contents into each cluster according to the detected devices.

In another embodiment, audio contents are categorized into clusters by user selection. For example, the electronic device detects each category of each of the audio contents selected by the user and groups the audio contents into each cluster for each selected category.

For example, when the audio track 1 belonging to the cluster 1 is played, the electronic device 100 automatically applies the sound equalizer configuration predicted for the cluster 1. Furthermore, when the user changes the audio track 1 and plays the audio track 4 belonging to the cluster 2, the sound equalizer configuration predicted for the cluster 2 is automatically applied to the audio track 4.

FIG. 3A is a block diagram illustrating various hardware elements of the electronic device 100 for frequency-based sound equalizer configuration prediction, according to an embodiment as disclosed herein.

Referring to the FIG. 3A, in an embodiment, the electronic device 100 can be a mobile phone, a smart phone, Personal Digital Assistants (PDAs), a tablet, a wearable device, a display device, an Internet of things (IoT) device, electronic circuit, chipset, and electrical circuit (i.e., System on Chip (SoC)), etc.

The electronic device 100 includes a communicator 110, a sound equalizer configuration engine 120, a processor 130, a memory 140 and a display 150.

In an embodiment, the communicator 110 is configured to receive and send the at least one audio content. For example, the user of the electronic device 100 can share all the audio tracks belonging to the cluster by selecting the cluster.

In an embodiment, the sound equalizer configuration engine 120 is configured to decode the at least one audio content and extract byte streams of the at least one audio content. Further, the sound equalizer configuration engine 120 is also configured to perform the frequency analysis of each of the byte streams of the at least one audio content and predict the at least one sound equalizer configuration for the at least one audio content based on the frequency analysis.

Further, the sound equalizer configuration engine 120 is also configured to detect that the user has changed the sound equalizer configuration of the at least one audio content of the cluster. Further, the sound equalizer configuration engine 120 is also configured to perform the frequency analysis of the byte streams of the at least one audio content of the cluster. The sound equalizer configuration engine 120 is also configured to determine the weighted mean average of the frequency analysis of the byte streams of the at least one audio content of the cluster and the user changed sound equalizer configuration of the at least one audio content of the cluster. Further, the sound equalizer configuration of all the audio content in the cluster is modified based on the weighted mean average.

The sound equalizer configuration engine 120 is also configured to determine and save the at least one playback parameter of the at least one audio content changed by the user. Further, the sound equalizer configuration engine 120 is also configured to customize the sound equalizer configuration for the remaining audio contents in the cluster based on the at least one playback parameter of the at least one audio content changed by the user. The playback parameters can be for example playback volume, playback speed, etc. For example, the sound equalizer configuration engine 120 is configured to modify the sound equalizer configuration for the remaining audio contents.

In an embodiment, the processor 130 is configured to interact with the hardware elements such as the communicator 110, the sound equalizer configuration engine 120, the memory 140 and the display 150 for frequency-based sound equalizer configuration prediction.

In an embodiment, the memory 140 is configured to store the at least one sound equalizer configuration predicted for the at least one audio content based on the frequency analysis. The memory 140 can include non-volatile storage elements. Examples of such non-volatile storage elements may include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories. In addition, the memory 140 may, in some examples, be considered a non-transitory storage medium. The term "non-transitory" may indicate that the storage medium is not embodied in a carrier wave or a propagated signal. However, the term "non-transitory" should not be interpreted that the memory 140 is non-movable. In some examples, the memory 140 is configured to store larger amounts of information than the memory. In certain examples, a non-transitory storage medium may store data that can, over time, change (e.g., in Random Access Memory (RAM) or cache).

In an embodiment, the display 150 is configured to display the predicted sound equalizer configuration for the at least one audio content. Further, the display 150 is also configured to represent all the audio contents of the cluster by a common indicator on the electronic device 100.

Although the FIG. 3A shows the hardware elements of the electronic device 100 but it is to be understood that other embodiments are not limited thereon. In other embodiments, the electronic device 100 may include less or more number of elements. Further, the labels or names of the elements are used only for illustrative purpose and does not limit the scope of the invention. One or more components can be combined together to perform same or substantially similar function for frequency-based sound equalizer configuration prediction.

FIG. 3B is a block diagram illustrating various hardware elements of the sound equalizer configuration engine 120, according to an embodiment as disclosed herein.

Referring to the FIG. 3B, the sound equalizer configuration engine 120 can include an audio content decoder 121, a frequency analyzer 122, a sound equalizer predictor 123, a machine learning engine 124 and a cluster determination engine 125.

In an embodiment, the audio content decoder 121 is configured to decode the at least one audio content to determine the frequency values of the at least one audio content across the entire stream of the audio content. The audio content decoder 121 extracts the encoded audio content and generates a sequence of byte array (i.e., chunks of byte stream). The extracted byte streams includes the frequency values of the at least one audio content across the whole duration of the audio content.

In an embodiment, the frequency analyzer 122 is configured to receive the extracted byte streams from the audio content decoder 121 and perform the FFT on the extracted byte streams. Further, the frequency analyzer 122 is also configured to determine the high magnitude frequency values of transformed byte streams. Further, the frequency analyzer 122 is also configured to superimpose the high magnitude frequency values of the transformed byte streams to generate the soothing curve.

In an embodiment, the sound equalizer predictor 123 is configured to predict the at least one sound equalizer configuration for the at least one audio content based on the frequency analysis performed by the frequency analyzer 124. The sound equalizer predictor 123 is also configured to obtain the frequency values over the entire range of the at least one audio content based on the frequency analysis. Further, the sound equalizer predictor 123 is also configured to generate the frequency data set from the obtained frequency values by plotting the frequency values in a K-dimensional plane (as described in FIG.7). Further, the sound equalizer predictor 123 is configured to predict the at least one sound equalizer configuration for the at least one audio content based on the frequency analysis.

In an embodiment, the machine learning engine 124 is configured to learn the user preference of the sound equalizer configurations for the at least one audio content. Further, the user preference of the sound equalizer configurations is taken as a feedback for learning which is utilized to customize the sound equalizer configuration. Further, the customized sound equalizer configuration is applied to all the audio content in the cluster. The machine learning engine 124 is also configured to learn the user preference with respect to various playback parameters such as playback speed and playback volume. Furthermore, the machine learning engine 124 is also configured to learn the user preferences of the sound equalizer configuration according to audio accessory such as earphones, Bluetooth headset or default (Device's speaker).

In an embodiment, the cluster determination engine 125 is configured to generate clusters for the at least one audio content based on the frequency analysis. Further, the cluster determination engine 125 is configured to categorize the at least one audio content into the clusters generated. Furthermore, the cluster determination engine 125 is also configured to dynamically generate clusters based on the various sound equalizer configurations predicted by the sound equalizer predictor 123. For example, when a similar pattern of sound equalizer configuration is predicted by the sound equalizer predictor 123, then only one cluster will be formed else a plurality of clusters will be formed.

FIG. 4A is a flow chart 400a illustrating the method for frequency-based sound equalizer configuration prediction, according to an embodiment as disclosed herein.

Referring to the FIG. 4A, at step 410, the electronic device 100 receives the at least one audio content. For example, in the electronic device 100 as illustrated in the FIG. 3A, the communicator 110 can be configured to receive the at least one audio content.

At step 420, the electronic device 100 decodes the at least one audio content to extract byte streams. For example, in the electronic device 100 as illustrated in the FIG. 3A, the sound equalizer configuration engine 120 can be configured to decode the at least one audio content to extract the byte streams.

At step 430, the electronic device 100 performs the frequency analysis of each of byte streams of the at least one audio content. For example, in the electronic device 100 as illustrated in the FIG. 3A, the sound equalizer configuration engine 120 can be configured to perform the frequency analysis of each of the byte streams of the at least one audio content.

At step 440, the electronic device 100 predicts the at least one sound equalizer configuration for at least one audio content based on frequency analysis. For example, in the electronic device 100 as illustrated in the FIG. 3A, the sound equalizer configuration engine 120 can be configured to predict the at least one sound equalizer configuration for the at least one audio content based on frequency analysis. For example, the electronic device 100 generates the at least one sound equalizer configuration for at least one audio content based on frequency analysis.

At step 450, the electronic device 100 stores the at least one sound equalizer configuration. For example, in the electronic device 100 as illustrated in the FIG. 3A, the memory 140 can be configured to store the at least one sound equalizer configuration.

The various actions, acts, blocks, steps, or the like in the method may be performed in the order presented, in a different order or simultaneously. Further, in some embodiments, some of the actions, acts, blocks, steps, or the like may be omitted, added, modified, skipped, or the like without departing from the scope of the invention.

FIG. 4B is a flow chart 400b illustrating the method for performing the frequency analysis, according to an embodiment as disclosed herein.

Referring to the FIG. 4B, at step 432, the electronic device 100 performs the FFT on the frequency values in the entire stream of the at least one audio content. For example, in the electronic device 100 as illustrated in the FIG. 3A, the sound equalizer configuration engine 120 can be configured to perform the FFT on the frequency values in the entire stream of the at least one audio content.

At step 434, the electronic device 100 determines the high magnitude frequency values in the entire stream of the at least one audio content based on the FFT. For example, in the electronic device 100 as illustrated in the FIG. 3A, the sound equalizer configuration engine 120 can be configured to determine the high magnitude frequency values in the entire stream of the at least one audio content based on the FFT.

At step 436, the electronic device 100 generates the soothing curve by superimposing high magnitude frequency values in the entire stream of the at least one audio content. For example, in the electronic device 100 as illustrated in the FIG. 3A, the sound equalizer configuration engine 120 can be configured to generate the soothing curve by superimposing high magnitude frequency values in the entire stream of the at least one audio content.

FIG. 4C is a flow chart 400c illustrating the method for predicting the at least one sound equalizer configuration for the at least one audio content based on the frequency analysis, according to an embodiment as disclosed herein.

Referring to the FIG. 4C, at step 442, the electronic device 100 obtains the frequency values over the entire range of the at least one audio content based on the frequency analysis. For example, in the electronic device 100 as illustrated in the FIG. 3A, the sound equalizer configuration engine 120 can be configured to obtain the frequency values over the entire range of the at least one audio content based on the frequency analysis.

At step 444, the electronic device 100 generates the frequency data set from the obtained frequency values. For example, in the electronic device 100 as illustrated in the FIG. 3A, the sound equalizer configuration engine 120 can be configured to generate the frequency data set from the obtained frequency values.

At step 446, the electronic device 100 clusters the at least one audio content into cluster using a machine learning technique based on the frequency data set. For example, in the electronic device 100 as illustrated in the FIG. 3A, the sound equalizer configuration engine 120 can be configured cluster the at least one audio content into cluster using the machine learning technique based on the frequency data set. For example, the electronic device 100 groups the at least one audio content into cluster using a machine learning technique based on the frequency data set.

At step 448, the electronic device 100 predicts the at least one sound equalizer configuration common for all the audio content in the cluster based on the frequency data set. For example, in the electronic device 100 as illustrated in the FIG. 3A, the sound equalizer configuration engine 120 can be configured to 100 predict the at least one sound equalizer configuration common for all the audio content in the cluster based on the frequency data set. For example, the electronic device 100 generates the at least one sound equalizer configuration common for all the audio content in the cluster based on the frequency data set.

FIG. 5 illustrates the decoding the at least one audio content to extract the byte streams, according to an embodiment as disclosed herein.

Referring to the FIG. 5, consider the at least one audio content received by the electronic device 100 is an mp3 file. The mp3 file is decoded through a media extractor which extracts the encoded mp3 file using the decoder and puts all the extracted data in a byte buffer. The output of the byte buffer is sequence of byte array. Further, for the sequence of byte array the FFT is applied in order to calculate the frequency magnitude values of the mp3 file.

FIG. 6A illustrates the range of frequencies for the audio track, according to an embodiment as disclosed herein.

Consider the audio track which has a plurality of frequency components in the audible range. A graph is plotted with sound pressure level on y-axis and frequency on x-axis (as shown in FIG. 6A). A curve is plotted defining an upper level and a lower level of the audible range. The area of curve represents the audible capacity of human ear. The frequency values of audio tracks such as mp3 files lie between 'limit of damage risk' range which is the upper level of the audible range and the lower level of the audible range. The area music/speech is the area that is audible.

FIG. 6B illustrates generation of the soothing curve for the audio track, according to an embodiment as disclosed herein.

Referring to the FIG. 6B, the plurality of frequency components in the audible range are indicated using upward and downward arrows. For example, the location of the frequency components of the 60Hz frequency in the audio track is indicated with respect to the sound pressure level using the first upward arrow (as shown in FIG. 6B). Similarly, various other frequency components are indicated with the upward and downward arrows. A soothing range of frequencies are selected which have a soothing effect to the human ear. The frequency components indicated with the upward and downward arrows are either increased or decreased to fall on the soothing curve. Furthermore, the soothing curve is generated by superimposing the plurality of frequency components which are present in the soothing range of frequencies. The soothing effect provides an enhanced musical experience to the user while listening to the audio track.

FIG. 6C illustrates spectrogram generation for the at least one audio track, according to an embodiment as disclosed herein.

Referring to the FIG. 6C the various frequency bands within a human audible frequency spectrum are analyzed are as follows:

Bass: The frequency range which determines how "fat" or " thin" the sound is. The bass sound ranges from 60Hz to 250Hz.

Low midrange: The frequency range comprises of the lower order harmonics of most instruments (250 to 500 Hz). It is generally viewed as bass presence range.

Midrange: The midrange frequency band determines how prominent an instrument is in the audio. The midrange frequency ranges from 500Hz to 2 kHz.

Upper Midrange: The human hearing is extremely sensitive at the　high midrange　frequencies. Small change around the range results in a huge change in the sound timbre. The upper midrange frequency band ranges from 2 kHz to 4 kHz. Vocals are most prominent in the range, as in the midrange.

Presence: The　presence　range is responsible for clarity and definition of the sound. The presence band ranges from 4 kHz to 6 kHz.

Brilliance: The brilliance range is composed entirely of harmonics and is responsible for sparkle and air of the sound, i.e., a higher contribution of brilliance band in the audio track will make the quality similar to Hi-Fi audio. The brilliance band ranges from 6 kHz to 20 kHz.

Further based on the analysis of the various frequency bands within the human audible frequency spectrum, a set of seven high magnitude frequency values are taken into consideration for the proposed method. The high magnitude frequency values include 60 Hz, 150 Hz, 400Hz, 1 kHz, 3 kHz, 8 kHz and 16 kHz.

Furthermore, based on the determined high magnitude frequency values and the conventional sound equalizer configuration in the electronic device 100, the proposed method provides a set of four frequency bands categories to provide enhanced listening experience. The four frequency bands categories are as follows:

BASS: Comprises of frequency bands Bass and Lower Midrange, and ranges from 60Hz to 500Hz.

MUSIC: Comprises of Midrange Frequency spectrum band and ranges from 500Hz to 2 kHz.

VOCAL: Comprises of frequency spectrum band Upper Midrange and ranges from 2 kHz to 4 kHz

HI-FI: Comprises of frequency spectrum bands Presence and Brilliance and ranges from 4 kHz to 16 kHz.

Further, based on the frequency bands the spectrogram is generated and displayed to the user on the screen of the electronic device 100.

FIG. 7 illustrates the generation of the frequency data set from the frequency values, according to an embodiment as disclosed herein.

The frequency data set of all the audio tracks in the electronic device 100 is generated by the sound equalizer predictor 124. The frequency data set is formed by the superimposing the frequency values as explained in FIG. 6B.

The frequency values generated for each audio track based on the frequency analysis are plotted on a 7-dimensional plane (i.e., K dimensional plane in general) based on the seven values obtained in the soothing curve. Further, the frequency values of all the audio tracks in the electronic device 100 are plotted on the 7-dimensional planes. Further, the data set is divided into four clusters and by aggregating the audio track band values within the clusters, centroids have been produced which act as centre for the K-dimensional plane. The centroids are further used to form clusters for the multiple audio tracks in the electronic device 100.

Further, the generated dataset is used for learning by the machine learning engine 125 and for predicating the clusters to which the audio tracks belong.

FIG. 8 illustrates formation of the clusters and prediction of the sound equalizer configuration for each cluster, according to an embodiment as disclosed herein.

Clustering is performed to group similar audio tracks based on the frequency analysis of the individual audio tracks. The proposed method provides four clusters based on different types of frequency distribution of the data of the audio tracks. The entire data set obtained from the individual audio track is divided into four clusters.

The audio tracks in the electronic device 100 are categorized and the clusters are formed on the basis of shortest distance among the centroids of the four clusters. The clusters will have the audio tracks which are closer to the specific cluster in similarity on the basis of frequency distribution, as shown in the FIG. 8. Further, the sound equalizer configuration common for all audio content in the cluster is predicted based on the frequency data set of the audio tracks within the clusters, as shown in the FIG. 8.

FIGS. 9A-9B are examples illustrating prediction of the sound equalizer configuration for the cluster based on the weighted mean average of the frequency analysis and the user preferred sound equalizer configuration for the at least one audio tracks of the cluster, according to an embodiment as disclosed herein.

Referring to the FIG. 9A, at step 902, consider the electronic device 100 receives the audio track 1, the audio track 2, the audio track 3 and the audio track 4. The electronic device 100 decodes and extracts the byte streams for each of the audio track 1, the audio track 2, the audio track 3 and the audio track 4. Further, the electronic device 100 performs frequency analysis on the extracted byte streams of all the audio tracks and predicts the sound equalizer configuration based on the frequency analysis of all the audio tracks. Furthermore, the electronic device 100 obtains the frequency values over the entire range of individual audio tracks and generates the frequency data set from the obtained frequency values, for each of the audio tracks. Further, the electronic device 100 categorized the audio track 1, the audio track 2, the audio track 3 and the audio track 4 into clusters based on the similar frequency data sets and predicts the common sound equalizer settings for the clusters, as shown in step 904.

The user of the electronic device 100 plays the audio track 1 and at step 906, the user changes the sound equalizer configuration for the audio track 1. The electronic device 100 saves the user preference of the sound equalizer configuration for the audio track 1. Further, anytime the user plays the audio track 1, the saved sound equalizer configuration for the audio track 1 will be automatically applied to the audio track 1 by the electronic device 100.

At step 908, a root mean square value of the user preference of the sound equalizer configuration for the audio track 1 and the sound equalizer configuration predicted for cluster 1 based on the frequency analysis will be obtained. Further, the root mean square value is used to generate the modified sound equalizer configuration for cluster 1 (as shown at step 908), which will be applied to all the audio tracks in the cluster 1 except the audio track 1.

Furthermore, referring to the FIG. 9B, consider the sound equalizer configuration for another audio track within the cluster 1 is changed by the user i.e., the sound equalizer configuration for the audio track 4 (as shown in step 910). Therefore, the electronic device 100 has the user preferred sound equalizer configuration for two audio tracks i.e., for the audio track 1 and the audio track 4, which will be saved. Further, anytime the audio track 1 and the audio track 4 are played by the user, the user preferred sound equalizer configuration will be automatically applied to the two tracks. Furthermore, the root mean square value of the user preference of the sound equalizer configuration for the audio track 1, the sound equalizer configuration for the audio track 4 and the sound equalizer configuration predicted for cluster 1 based on the frequency analysis will be obtained. Further, the root mean square value is used to generate the modified sound equalizer configuration for cluster 1 (as shown at step 912), which will be applied to all the audio tracks in the cluster 1 except the audio track 1 and the audio track 4.

Furthermore, the process keeps repeating until all the audio tracks have a different sound equalizer configuration. Hence, the proposed method allows the user to have a different sound equalizer configuration for every audio track based on user preference. Further, the user preferences are automatically saved for every individual track and are automatically applied every time the user plays the audio track thereby reducing user's manual effort.

FIG. 10 is an example illustrating application of the at least one sound equalizer configuration and adaption of the sound equalizer configuration modified by the user for the audio tracks in the cluster, according to an embodiment as disclosed herein.

The electronic device 100 performs frequency analysis on each of the audio tracks received by the electronic device 100 and predicts the sound equalizer configuration for each of the audio tracks based on the frequency analysis. Further, the audio tracks are categorized into clusters using machine learning technique based on the frequency analysis and the common sound equalizer configuration is predicted for all the audio tracks within the cluster. The automatic application of the predicted sound equalizer configuration to all the audio tracks within the cluster is symbolically represented as 'auto_config_1', as shown in the FIG. 10 at step 1002. Further, 'auto_config_1' indicates the sound equalizer configuration applied to all the audio tracks belonging to cluster 1 (as shown in step 1004) and also that the sound equalizer configuration for any of the audio tracks belonging to cluster 1 has not been changed by the user.

However, the auto_config_1 need not be in line with the user preference for audio track 1. Hence, at step 1006, the user changes the sound equalizer configuration for audio track 1. The user changing the sound equalizer configuration for audio track 1 is symbolically represented as 'custom' in front of audio track 1, as shown in step 1008. Further, the user preference of the sound equalizer configuration for audio track 1 is used by the machine learning system of the electronic device 100 for the learning phase. Further, the sound equalizer configuration of the cluster 1 is modified based on the weighted mean average of the user customized sound equalizer configuration and the auto_config_1. Further, the sound equalizer configuration for all the audio tracks belonging to cluster 1 will be changed using the modified sound equalizer configuration. Further, the modified sound equalizer configuration applied to all the audio tracks belonging to cluster 1 is symbolically represented as 'adapt' symbol provided in front of the audio tracks belonging to cluster 1, as shown at step 1010.

FIGS. 11A-11B is an example illustrating various functions that can be performed using the cluster of the audio tracks, according to an embodiment as disclosed herein.

Referring to the FIG. 11A, at step 1102, the clusters to which each of the audio tracks belong are provided in the UI of the electronic device 100 along with the audio tracks.

The electronic device 100 allows the user to create multiple playlists for categorizing the audio tracks. Further, the user can add all the audio tracks which are categorized within the cluster into specific playlists just by selecting the cluster. At step 1104, the user selects the "add to playlist 1" option to add the audio tracks to the playlist 1. Further, the user selects the cluster 1 option to automatically add all the songs belonging to the cluster 1 to the playlist 1.

Furthermore, all the audio tracks belonging to the cluster 1 are shared with another user just by selecting the cluster 1 option, as shown in the FIG. 11B.

Referring to the FIG. 12, at step 1202, the user plays the audio track 1 belonging to the cluster 1. The playback volume is at 41 and the playback speed is at 1.0x (default values). At step 1204, the user changes the playback volume to 54 and the playback speed is at 1.4x for the audio track 1 which is currently being played by the electronic device 100. The electronic device 100 saves the user's preference of playback speed and the playback volume for the audio track 1. According to the proposed method, the electronic device 100 automatically customizes the sound equalizer configuration of the remaining audio tracks in the cluster 1 based on the changed playback volume and the playback speed of the audio track 1.

At step 1206, the user plays the audio track 2 also belonging to the cluster 1. The electronic device 100 plays the audio track 2 at the customized sound equalizer configuration determined based on the changed playback volume and the playback speed of the audio track 1. Further, the customized sound equalizer configuration provides the playback volume of 39 and the playback speed of 1.2x for all the audio tracks in the cluster 1. At step 1208, the user changes the playback volume and the playback speed of the audio track 2. The audio track 2 playback volume is increased to 46 and the playback speed is decreased to 0.9x. The electronic device 100 saves the user's preference of the playback speed and the playback volume for the audio track 2. Further, the electronic device 100 again automatically customizes the sound equalizer configuration of the remaining audio tracks in the cluster 1 (except audio track 1 and audio track 2 which will be played at user preferred playback speed and playback volume) based on the changed playback volume and the playback speed of the audio track 2. Therefore, the sound equalizer configuration is customized based on a weighted mean average of the user preferred playback volume and playback speed of both the audio track 1 and the audio track 2.

Further, the playback volume and the playback speed preferences for the audio track 1 and the audio track 2 are saved by the electronic device 100. If the user plays the audio track 1, then the audio track 1 will be automatically played at the user preferred playback volume and playback speed i.e., at the playback volume of 54 and the playback speed of 1.4x.

Therefore, the user does not have to manually adjust the playback volume and the playback speed every time the audio file is being played on the electronic device 100. Thus, along with sound equalizer configuration, the playback parameters such as the playback volume and the playback speed, etc can also be automatically customized for the audio tracks and video tracks which increase the ease of operation and provides enhanced user experience while listening to audio track on the electronic device 100.

In another embodiment, the sound equalizer configurations, the playback parameters such as playback volume and playback speed, etc can be saved for the audio tracks with respect to specific devices. For example, when the user plays the audio track over a headphone set, the user might want a specific sound equalizer configuration, the playback volume and the playback speed. However, when the user plays the same audio track over a speaker, the user preferences of the sound equalizer configuration, the playback volume and the playback speed may be different. Hence, the user need not manually set the sound equalizer configuration each time the audio tracks are played.

The embodiments disclosed herein can be implemented using at least one software program running on at least one hardware device and performing network management functions to control the elements.

The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the spirit and scope of the embodiments as described herein.

Claims

A method for operating an electronic device, the method comprising:

performing a frequency analysis for audio contents;

determining a sound equalizer configuration for the audio contents based on a result of the frequency analysis; and

storing the sound equalizer configuration.
The method of claim 1, wherein determining the sound equalizer configuration for the audio contents based on the result of the frequency analysis comprises:

obtaining frequency values of each of the audio contents based on the result of the frequency analysis;

generating frequency data set from the obtained frequency values;

grouping the audio contents into a cluster based on the frequency data set; and

determining the sound equalizer configuration common for audio contents in the cluster based on the frequency data set.
The method of claim 1, wherein performing the frequency analysis comprises:

performing a fast Fourier transform (FFT) on frequency values of the audio contents;

determining high magnitude frequency values of the audio contents based on the FFT; and

generating a soothing curve by superimposing the high magnitude frequency values of the audio contents, wherein the soothing curve includes fixed number of the high magnitude frequency values.
The method of claim 1, further comprising:

detecting user inputs on data set of the audio contents of a cluster; and

modifying the sound equalizer configuration of the audio contents of the cluster based on a weighted mean average of the data set of the audio contents of the cluster based on the result of the frequency analysis.
The method of claim 1, further comprising:

displaying a plurality of clusters each of which is represented by an indicator, wherein the audio contents belonging to each of the plurality of the clusters has a common sound equalizer configuration;

detecting a cluster selected from the plurality of clusters; and

displaying the audio contents belonging to the selected cluster or automatically sharing the audio contents belonging to the selected cluster.
The method of claim 1, further comprising:

playing back a first audio content from the audio contents belonging to a cluster, wherein each of the audio contents in the cluster has a common sound equalizer configuration;

detecting a change in at least one playback parameter of the first audio content; and

modifying the sound equalizer configuration of remaining audio contents of the audio contents excluding the first audio content in the cluster based on the at least one changed playback parameter of the first audio content.
The method of claim 6, further comprising:

detecting a second audio content selected from the cluster; and

playing back the second audio content with the modified sound equalizer configuration.
An electronic device comprising:

a memory;

a processor coupled to the memory;

a communicator coupled to the memory and the processor; and

a sound equalizer configuration engine coupled to the memory and the processor, wherein the sound equalizer configuration engine is configured to:

perform a frequency analysis for audio contents;

determine a sound equalizer configuration for the audio contents based on a result of the frequency analysis; and

store the sound equalizer configuration.
The electronic device of claim 8,

wherein the sound equalizer configuration engine is further configured to:

obtain frequency values of each of the audio contents based on the result of the frequency analysis;

generate frequency data set from the obtained frequency values;

group the audio contents into a cluster based on the frequency data set; and

determining the sound equalizer configuration common for audio contents in the cluster based on the frequency data set.
The electronic device of claim 8,

wherein the sound equalizer configuration engine is further configured to:

performing a fast Fourier transform (FFT) on frequency values of the audio contents;

determining high magnitude frequency values of the audio contents based on the FFT; and

generating a soothing curve by superimposing the high magnitude frequency values of the audio contents, wherein the soothing curve includes fixed number of the high magnitude frequency values.
The electronic device of claim 8,

wherein the sound equalizer configuration engine is further configured to:

detect user inputs on data set of the audio contents of a cluster; and

modify the sound equalizer configuration of the audio contents of the cluster based on a weighted mean average of the data set of the audio contents of the cluster based on the result of the frequency analysis.
The electronic device of claim 8,

wherein the sound equalizer configuration engine is further configured to:

display a plurality of clusters each of which is represented by a unique indicator, wherein the audio contents belonging to each of the plurality of the clusters has a common sound equalizer configuration;

detect a cluster selected from the plurality of clusters; and

display the audio contents belonging to the selected cluster or automatically sharing the audio contents belonging to the selected cluster.
The electronic device of claim 8,

wherein the sound equalizer configuration engine is further configured to:

play back a first audio content from the audio contents belonging to a cluster, wherein each of the audio contents in the cluster has a common sound equalizer configuration;

detect a change in at least one playback parameter of the first audio content; and

modify the sound equalizer configuration of remaining audio contents of the audio contents excluding the first audio content in the cluster based on the at least one changed playback parameter of the first audio content.
The electronic device of claim 8,

wherein the sound equalizer configuration engine is further configured to:

detect a second audio content selected from the cluster; and

playing back the second audio content with the modified sound equalizer configuration.