HK40112923A - Soundscape augmentation system and method of forming the same - Google Patents

Description

Soundscape enhancement system and its formation method

Cross-references to related applications

This application claims the benefit of priority to Singapore application No. 10202204451S, filed on 27 April 2022, the contents of which are incorporated herein by reference in their entirety for all purposes.

Technical Field

Embodiments of this application relate to soundscape enhancement systems. Embodiments of this application also relate to methods for forming soundscape enhancement systems.

Background Technology

The World Health Organization (WHO) has listed noise exposure as the second leading cause of environmental pollution after air pollution. Noise exposure has also been compared to secondhand smoke. The urgent call to take action to mitigate noise exposure stems from mounting evidence of the health effects of noise, such as an increased risk of ischemic heart disease, irritability, and sleep disturbances. The WHO also emphasizes the impact of environmental noise exposure on mental health and well-being, with new evidence highlighting its adverse effects on indicators of depression and anxiety.

A paradigm shift from noise management to soundscape management is emerging. ISO 12913-1 defines "soundscape" as "the perceived or experienced acoustic environment and/or the acoustic environment as understood by one or more people in a context." In contrast to noise management, the soundscape management framework treats sound as a resource, not waste; it focuses on preferred sounds rather than unpleasant ones; and it manages unwanted sounds by masking unwanted ones, and by reducing unwanted sounds rather than simply lowering sound levels. Therefore, soundscape intervention techniques exist that enhance or introduce "masking" sounds into the acoustic environment to improve the overall perceived acoustic comfort. Typically, this intervention involves enhancing natural sounds (e.g., through loudspeakers) and has been tested in outdoor recreational spaces and indoor spaces such as nursing homes to improve acoustic comfort. Soundscape enhancement utilizing desired sounds is similar to sound masking systems, which are commonly used in office environments to reduce disturbances. The key difference between sound masking systems and soundscape-based enhancement systems is that masking is based on objective metrics, while soundscape enhancement is primarily based on context-based subjective metrics. Context "includes the spatial and temporal interrelationships between people and activities and places," and is further described in detail by example in ISO 12913-1. It is also worth noting that soundscapes are not limited to real-world environments; they can include virtual environments and even memories.

Summary of the Invention

Embodiments of this application may provide a soundscape enhancement system. The soundscape enhancement system may include a data acquisition system configured to provide ambient soundscape data. The soundscape enhancement system may also include a database comprising multiple masking settings. The soundscape enhancement system may further include a perception attribute predictor coupled to the data acquisition system and the database, configured to generate, based on the ambient soundscape data, a prediction representing perception on one or more predefined perception attribute metrics for each of the multiple masking settings. The soundscape enhancement system may additionally include a masking setting ranking system configured to determine one or more optimal masking settings based on the predictions generated by the perception attribute predictor. The soundscape enhancement system may also include a playback system configured to play back or reproduce one or more optimal masking settings.

Embodiments of this application may relate to a method for forming a soundscape enhancement system. The method may include providing a data acquisition system configured to provide ambient soundscape data. The method may also include providing a database comprising a plurality of masking settings. The method may further include coupling a perceptual attribute predictor to the data acquisition system and the database, the perceptual attribute predictor being configured to generate, based on the ambient soundscape data, a prediction representing perception on one or more predefined perceptual attribute metrics for each of the plurality of masking settings. The method may additionally include providing a masking setting ranking system configured to determine one or more optimal masking settings based on the predictions generated by the perceptual attribute predictor. The method may further include providing a playback system configured to play back or reproduce one or more optimal masking settings.

Attached Figure Description

In the accompanying drawings, similar reference numerals generally refer to the same parts in different views. The drawings are not necessarily drawn to scale, but rather focus on illustrating the principles of various embodiments. In the following description, various embodiments of the invention will be described with reference to the following drawings.

Figure 1 shows a schematic diagram of a soundscape enhancement system according to various embodiments.

Figure 2 shows a schematic diagram of a method for forming a soundscape enhancement system according to various embodiments.

Figure 3 shows a schematic diagram of a soundscape enhancement system according to various embodiments.

Figure 4 illustrates a two-dimensional circular octet model of the emotional quality attributes proposed in ISO 12913-3. The predictions of the perceptual attribute predictors according to various embodiments can be based on this two-dimensional circular octet model.

Figure 5 is a schematic diagram of a perceptual attribute predictor according to various embodiments.

Figure 6 is a schematic diagram of a perceptual attribute predictor according to various embodiments.

Figure 7 is a schematic diagram of a perceptual attribute predictor according to various embodiments.

Figure 8 illustrates the training and inference schemes for the automatic mask selection system (AMSS) according to various embodiments.

Figure 9 illustrates (a) the basic convolutional recurrent neural network (CRNN) architecture used according to various embodiments; (b) a possible implementation of the feature map block in (a) according to various embodiments; and (c) other possible implementations of the feature map block in (a) according to various other embodiments.

Figure 10 is a table showing the mean fold mean squared errors (MSEs) (± standard deviation) of the probabilistic perceptual attribute predictor (PPAP) in 10 runs (top: cross-validation set, bottom: test set) for each setting according to various embodiments.

Figure 11 illustrates mask selection using 50 models (dot product variables) on the test set, with a naive maximum _μk value selection scheme (left) according to various embodiments and a random sampling scheme (right) according to various embodiments to incentivize masker exploration.

Figure 12A shows a schematic diagram of a probabilistic perceptual attribute predictor (PPAP) according to various embodiments.

Figure 12B illustrates three feature enhancement methods based on calculation formulas (8), (9), and (10) in various embodiments.

Figure 13 illustrates algorithms for reducing total runtime according to various embodiments.

Figure 14 shows (top) a diagram illustrating mean squared error (MSE) as a function of attention block type, illustrating a violin diagram of the validation MSE for pleasure prediction according to various embodiments for each attention block type and feature enhancement method; and (bottom) a diagram illustrating mean absolute error (MAE) as a function of attention block type and feature enhancement method, illustrating a violin diagram of the validation MAE for pleasure prediction according to various embodiments.

Figure 15A is a diagram illustrating ISO pleasure as a function of log gain according to various embodiments. The diagram shows gain interpolation for the masker category "water" using additive attention (AA) and CONV enhancements (seed 5, verification fold 2).

Figure 15B is a diagram illustrating ISO pleasure as a function of logarithmic gain according to various embodiments, showing gain interpolation for the masker category "traffic" using additive attention (AA) and CONV enhancement (seed 5, verification fold 2).

Figure 15C is a diagram illustrating ISO pleasure as a function of logarithmic gain according to various embodiments, showing gain interpolation for the masker category "bird" using additive attention (AA) and CONV enhancements (seed 5, verification fold 2).

Figure 15D is a diagram of ISO pleasure as a function of logarithmic gain according to various embodiments, illustrating gain interpolation for the masker category "Construction" using additive attention (AA) and CONV enhancements (seed 5, verification fold 2).

Figure 15E is a diagram of ISO pleasure as a function of logarithmic gain according to various embodiments, illustrating gain interpolation for the masker category "Mute" using additive attention (AA) and CONV enhancement (seed 5, verification fold 2).

Detailed Implementation

The following detailed description refers to the accompanying drawings, which illustrate specific details and embodiments in which the invention can be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. Other embodiments may be used, and structural, logical, and electrical changes may be made without departing from the scope of the invention. The various embodiments are not necessarily mutually exclusive, as some embodiments may be combined with one or more other embodiments to form new embodiments.

Features described in the context of one embodiment may be adapted accordingly to the same or similar features in other embodiments. Features described in the context of one embodiment may be adapted accordingly to other embodiments, even if not explicitly described in those other embodiments. Furthermore, additions and/or combinations and/or substitutions described for a feature in the context of one embodiment may be adapted accordingly to the same or similar features in other embodiments.

In the context of the various embodiments, the articles “a,” “an,” and “the” used with respect to features or elements include references to one or more features or elements.

In the context of the various embodiments, the term “about” or “approximately” applied to numerical values covers both exact values and reasonable differences, where a reasonable difference is, for example, within 10% of a specified value.

As used herein, the term “and/or” includes any and all combinations of one or more of the listed related items.

"Including" means, but is not limited to, anything that follows the word "including". Therefore, the use of the term "including" indicates that the listed elements are necessary or mandatory, but other elements are optional and may or may not be present.

"Composed of" means anything included in and limited to the phrase "composed of". Therefore, the phrase "composed of" indicates that the listed elements are necessary or mandatory, and that no other elements exist.

The embodiments described in the context of one soundscape enhancement system are equally effective for other soundscape enhancement systems. Similarly, the embodiments described in the context of a method are equally effective for soundscape enhancement systems, and vice versa.

The most advanced enhancement or masking interventions currently involve static sound playback through loudspeakers, where sound levels and tracks must be manually selected. Various embodiments can utilize a pioneering artificial intelligence model to automatically select a track that produces the highest (or lowest) value on any subjective metric (e.g., perceived loudness, the ISO 12913-3 Circumplex Octant Scale – pleasantness, annoyance, tranquility, calmness, vibrancy, eventfulness, etc.) at the most suitable sound level. The artificial intelligence (AI) model was trained by a research team at Nanyang Technological University (NTU) on a large dataset of human subjective responses from the local population in Singapore, collected based on the ISO 12913-2 standard. The ISO 12913 series of standards represents a paradigm shift in sound environment management and details a perception-based approach for the comprehensive assessment and analysis of sound environments. In addition, artificial intelligence models can also serve as predictive tools for automatically assessing the perceived quality of the sound environment without having to conduct tedious and laborious surveys or questionnaires as required by ISO 12913-2.

Figure 1 is a schematic diagram of a soundscape enhancement system according to various embodiments. The soundscape enhancement system may include a data acquisition system 102 configured to provide ambient soundscape data. The soundscape enhancement system may also include a database 104 including a plurality of masking settings. The soundscape enhancement system may further include a perception attribute predictor 106 coupled to the data acquisition system 102 and the database 104, configured to generate a prediction representing perception on one or more predefined perception attribute metrics for each of the plurality of masking settings based on the ambient soundscape data. The soundscape enhancement system may additionally include a masking setting ranking system 108 configured to determine one or more optimal masking settings based on the predictions generated by the perception attribute predictor 106. The soundscape enhancement system may also include a playback system 110 configured to play or reproduce one or more optimal masking settings.

In other words, the soundscape enhancement system may include a perceptual attribute predictor 106, which may be connected to the data acquisition system 102 and the database 104. The soundscape enhancement system may also include a masking setting sorting system 108 connected to the perceptual attribute predictor 106 and a playback system 110 connected to the masking setting sorting system 108.

To avoid any doubt, Figure 1 is intended to illustrate some features of the soundscape enhancement system according to various embodiments, and is not intended to limit, for example, the size, shape, orientation, or arrangement of the various components.

In various embodiments, the data acquisition system 102 may also be configured to provide population statistics or contextual data. The perceived attribute predictor 106 may also be configured to generate predictions based on the population statistics or contextual data. The population statistics or contextual data may be received, measured, or sensed directly by the data acquisition system 102, or received from an external source, such as a portable health monitoring device or a storage device, as described in more detail below.

Group statistics or contextual data may include data related to environmental parameters, group statistics of one or more listeners, psychological data of one or more listeners, physiological data of one or more listeners, or any combination thereof.

In this regard, one or more listeners may include real listeners and/or imaginary listeners.

In various embodiments, environmental data related to environmental parameters may or may include location-related data, visual environment-related data, the number of people nearby, air temperature, humidity, wind speed, or any combination thereof. Environmental parameters may be non-acoustic environmental parameters. For example, data acquisition system 102 may include or be coupled to a Global Positioning System (GPS) sensor to determine location-related data. Data acquisition system 102 may include or be coupled to a camera to determine data related to the number of people nearby and/or to obtain visual information from the environment (e.g., determining the presence of specific types of furniture, pets, etc., determining human movement, determining the percentage of greenery or water present, etc.). Data acquisition system 102 may include or be coupled to a thermometer to determine air temperature-related data. Data acquisition system 102 may include or be coupled to a hygrometer to determine humidity-related data. Data acquisition system 102 may include or be coupled to an anemometer to determine wind speed-related data. In this context, the term "coupled" may refer to a direct or indirect connection. The connection may be established, for example, via wired or wireless means.

In various embodiments, the group statistics of one or more listeners may be or may include data related to age, gender, occupation, or any combination thereof. In various embodiments, the group statistics of one or more listeners provided by the data acquisition system 102 may be obtained by the data acquisition system 102 from an external source. In various embodiments, the group statistics of one or more listeners may be provided in digital form attached to the data acquisition system 102, or may be provided as pre-loaded data coupled to (e.g., directly coupled or coupled via a cloud-based server) a storage device coupled to the data acquisition system 102. In various embodiments, the group statistics of one or more listeners may be manually provided to the data acquisition system 102, for example, by manually typing into the data acquisition system 102, or by sending to the data acquisition system 102 from a remote system according to a user-provided manual instruction. In various embodiments, the data acquisition system 102 may be configured to automatically obtain the group statistics of one or more listeners, for example, by wired or wireless means from a remote system.

In various embodiments, the psychological data of one or more listeners may be, or may include, data related to noise sensitivity (using, for example, the Weinstein Noise Sensitivity Scale), perceived stress (using, for example, the Cohen’s Perceived Stress Scale), well-being index scores (using, for example, the WHO-5 Well-Being Index)), or any combination thereof. In various embodiments, the psychological data of one or more listeners provided by the data acquisition system 102 may be obtained by the data acquisition system 102 from an external source. In various embodiments, the psychological data of one or more listeners may be provided in digital form attached to the data acquisition system 102, or may be provided as pre-loaded data coupled to (e.g., directly coupled or coupled via a cloud-based server) a storage device coupled to the data acquisition system 102. In various embodiments, the psychological data of one or more listeners can be manually provided to the data acquisition system 102, for example, by manually typing it into the data acquisition system 102, or by sending it from a remote system according to a manual instruction provided by the user. In various embodiments, the data acquisition system 102 can be configured to automatically acquire the psychological data of one or more listeners, for example, by acquiring it from a remote system via wired or wireless means.

In various embodiments, the physiological data of one or more listeners may be or may include data related to heart rate, blood pressure, body temperature, or any combination thereof. For example, data acquisition system 102 may include or be coupled to (e.g., directly coupled or coupled via a cloud-based server) one or more portable health monitoring devices configured to determine or measure the physiological data of one or more listeners. Data acquisition system 102 may be configured to acquire the physiological data of one or more listeners from one or more portable health monitoring devices, either wired or wirelessly.

In various embodiments, the perceptual attribute predictor 106 may be configured to generate predictions based on one or more masking gain inputs. For example, the one or more masking gain inputs may be one or more digital gain levels or one or more masking gain waveforms.

In various embodiments, one or more predefined perceptual attribute indicators may include pleasantness, activity, calmness, perceived loudness, sound quality, sharpness, roughness, or any combination thereof. In various embodiments, one or more predefined perceptual attribute indicators may be selected from the emotional quality attributes defined in ISO 12913-3.

In various embodiments, the generated prediction may or may not be a single numerical value. In various embodiments, the generated prediction may be nondeterministic. In various other embodiments, the generated prediction may be deterministic. In various embodiments, the prediction may be a probability distribution, random values extracted from a probability distribution, a vector or multivariate representation of one or more predefined perceptual attributes. In various embodiments, the generated prediction may be a predicted attribute distribution.

In various embodiments, the perceptual attribute predictor 106 may be or may include a probabilistic perceptual attribute predictor. In various embodiments, the perceptual attribute predictor 106 may be configured to generate a prediction by mixing, combining, or adding ambient soundscape data and each masking setting. In various embodiments, the ambient soundscape data and each masking setting may be added, mixed, or combined in, for example, an inference engine to form enhanced soundscape data before the enhanced soundscape data is input to the perceptual attribute predictor 106. The masking waveform of each masking setting may be weighted (i.e., multiplied) with, for example, a masking gain waveform, from a masking gain input to generate a weighted masking waveform. The weighted masking waveform may be combined with the ambient soundscape waveform of the ambient soundscape data (i.e., added to the ambient soundscape waveform of the ambient soundscape data) to generate an enhanced soundscape waveform. The prediction block of the perceptual attribute predictor 106 may receive the enhanced soundscape waveform to generate a prediction.

In various embodiments, the perceptual attribute predictor 106 may include a sound scene feature extractor configured to extract features from the surrounding sound scene data. The perceptual attribute predictor 106 may include a masking feature extractor configured to extract features from each masking setting. The perceptual attribute predictor 106 may include a feature-level enhancer configured to generate one or more enhanced sound scene features based on the extracted features from the surrounding sound scene data, the extracted features from each masking setting, and a masking gain input, such as a digital gain level. The perceptual attribute predictor 106 may include a prediction block configured to generate a prediction based on one or more enhanced sound scene features, the extracted features from the surrounding sound scene data, and the extracted features from each masking setting. The perceptual attribute predictor 106 may be configured to generate a prediction based on the extracted features from the surrounding sound scene data and the extracted features from each masking setting.

In various embodiments, the perceptual attribute predictor 106 may include an objective feature block configured to extract objective features from surrounding acoustic scene data and each masking setting. The perceptual attribute predictor 106 may be further configured to extract objective features based on masking gain input. The perceptual attribute predictor 106 may include a subjective feature block configured to extract subjective features from crowd statistics or contextual data. The subjective feature block may also be configured to process the extracted subjective features and the extracted objective features to generate a prediction. The perceptual attribute predictor 106 may be configured to generate a prediction based on the extracted objective features and the extracted subjective features.

In various embodiments, the perceptual attribute predictor 106 may include a linear regression model configured to generate predictions based on acoustic or psychoacoustic parameters calculated based on ambient soundscape data and each masking setting.

In various embodiments, the perceptual attribute predictor 106 may include one or more deep neural networks configured to generate predictions based on raw audio, spectrogram rendering, or a combination of raw audio and spectrogram rendering, the spectrogram rendering being calculated based on surrounding soundscape data and each masking setting.

In various embodiments, the perceptual attribute predictor 106 may include a composite network as described herein or any combination of networks and/or models as described herein.

In various embodiments, ambient sound data may be directly received, recorded, measured, or sensed by the data acquisition system 102, or it may be received from an external source. In various embodiments, the data acquisition system 102 may be configured to provide or generate ambient sound data based on input from recording devices, receiving devices, and/or storage devices. The data acquisition system 102 may be configured to provide or generate ambient sound data based on input from one or more microphones, one or more antennas, storage media, or any combination thereof. In various embodiments, the data acquisition system 102 may be configured to provide or generate ambient sound data by directly recording or sensing the ambient sound of the environment.

In various embodiments, the multiple masking settings may include one or more recorded or synthesized sound tracks, one or more silent sound tracks, or one or more sound tracks derived from the one or more recorded or synthesized sound tracks and the one or more silent sound tracks.

In various embodiments, the playback system 110 may include one or more speakers, one or more virtual reality or augmented reality headsets, one or more headphones, one or more earbuds, or any combination thereof.

In various embodiments, one or more computing, processing, or electronic devices or systems may be used to implement the various components of the soundscape enhancement system. In one example, the data acquisition system 102, database 104, perceptual attribute predictor 106, masking setting sorting system 108, and playback system 110 may be implemented in a single computing or processing device or system including a microphone and speakers, such as a physical server or cloud-based server, personal computer system, or mobile device. In another example, the data acquisition system 102 may be implemented via a mobile device (with a microphone), while the database 104, perceptual attribute predictor 106, masking setting sorting system 108, and playback system 110 may be implemented via a personal computer system having speakers that directly or indirectly communicate wirelessly with the mobile device (e.g., via WiFi or Bluetooth). In yet another example, the data acquisition system 102 may be implemented using a laptop computer remotely connected to a microphone and environmental sensors. The database 104, perceptual attribute predictor 106, and masking setting sorting system 108 may be implemented using another computing device communicating with a laptop computer via a server. The playback system 110 may be headphones that communicate wirelessly with computer equipment.

Figure 2 illustrates a schematic diagram of a method for forming a soundscape enhancement system according to various embodiments. The method may include providing a data acquisition system in step 202, the data acquisition system being configured to provide ambient soundscape data. The method may further include providing a database comprising a plurality of masking settings in step 204. The method may further include coupling a perception attribute predictor to the data acquisition system and the database in step 206, the perception attribute predictor being configured to generate a prediction representing perception on one or more predefined perception attribute metrics for each of the plurality of masking settings based on the ambient soundscape data. The method may additionally include providing a masking setting ranking system in step 208, the masking setting ranking system being configured to determine one or more optimal masking settings based on the predictions generated by the perception attribute predictor. The method may further include providing a playback system in step 210, the playback system being configured to play back or reproduce one or more optimal masking settings.

In other words, the method may include coupling a perceptual attribute predictor to a data acquisition system and a database containing multiple masking settings. The method may also include coupling a masking setting sorting system to the perceptual attribute predictor and a playback system to the masking setting sorting system.

To avoid any doubt, Figure 2 is not intended to restrict the order of the steps. For example, step 202 may occur before, after, or simultaneously with step 204.

In various embodiments, the data acquisition system may also be configured to provide population statistics or contextual data. The perceived attribute predictor may also be configured to generate predictions based on population statistics or contextual data.

In various embodiments, group statistics or contextual data may include data related to environmental parameters, group statistics of one or more listeners, psychological data of one or more listeners, physiological data of one or more listeners, or any combination thereof.

In various embodiments, data related to environmental parameters may include location-related data, visual environment-related data, the number of people nearby, air temperature, humidity, wind speed, or any combination thereof.

In various embodiments, group statistics for one or more listeners may include data related to age, gender, occupation, or any combination thereof.

In various embodiments, the psychological data of one or more listeners may include data related to noise sensitivity, perceived stress, well-being index scores, or any combination thereof.

In various embodiments, the physiological data of one or more listeners may include data related to heart rate, blood pressure, body temperature, or any combination thereof.

In various embodiments, the perceptual attribute predictor can be configured to generate predictions based on one or more masking gain inputs.

In various embodiments, one or more predefined perceptual attribute metrics may include pleasantness, activity, eventfulness, calmness, perceived loudness, sound quality, sharpness, roughness, or any combination thereof.

In various embodiments, the perception attribute predictor can be configured to generate predictions by combining surrounding soundscape data and each masking setting.

In various embodiments, the perceptual attribute predictor may include a soundscape feature extractor configured to extract features from the surrounding soundscape data. The perceptual attribute predictor may also include a masking feature extractor configured to extract features for each masking setting. The perceptual attribute predictor may be configured to generate a prediction based on the extracted features from the surrounding soundscape data and the extracted features for each masking setting.

In various embodiments, the perceptual attribute predictor may include an objective feature block configured to extract objective features from surrounding acoustic data and each masking setting. The perceptual attribute predictor may also include a subjective feature block configured to extract subjective features from crowd statistics or contextual data. The perceptual attribute predictor may be configured to generate predictions based on the extracted objective and subjective features.

In various embodiments, the perceptual attribute predictor may include a linear regression model configured to generate predictions based on acoustic or psychoacoustic parameters calculated based on surrounding acoustic scene data and each masking setting.

In various embodiments, the perceptual attribute predictor may include one or more deep neural networks configured to generate predictions based on raw audio, spectrogram rendering, or a combination of raw audio and spectrogram rendering, wherein the spectrogram rendering is calculated based on surrounding soundscape data and each masking setting.

In various embodiments, the data acquisition system may be configured to provide or generate ambient sound scene data based on inputs from one or more microphones, one or more antennas, storage media, or any combination thereof.

In various embodiments, the multiple masking settings may include one or more recorded or synthesized sound tracks, one or more muted sound tracks, or one or more sound tracks derived from the one or more recorded or synthesized sound tracks and the one or more muted sound tracks.

In various embodiments, the playback system may include one or more speakers, one or more virtual reality or augmented reality headsets, one or more headphones, one or more earbuds, or any combination thereof.

In various embodiments, the generated predictions may be non-deterministic.

Figure 3 illustrates a schematic diagram of a soundscape enhancement system according to various embodiments. The soundscape enhancement system may also be referred to as an automatic masker selection system (AMSS). The soundscape enhancement system may include a data acquisition system 302, which is responsible for providing ambient soundscape data 312, i.e., audio of the acoustic environment and/or parameters calculated directly from the audio of the acoustic environment. To obtain the ambient soundscape data 312 from the acoustic environment, the data acquisition system 302 may use, for example, real-time recordings from one or more microphones, digital audio signals received from a wireless antenna, data stored on an attached storage medium, or any combination thereof.

Furthermore, the soundscape enhancement system may include a database comprising multiple masking settings 304 (alternatively referred to as candidate masking setting groups). The term "masking setting" can refer to any audio track that can be added to the acoustic environment in 312 through any form of playback, in combination with other parameters or effects required for the intended playback of the audio track. Masking settings can also be referred to as "masks." These parameters or effects may include, for example, the application of digital gain levels, spatialization, dynamic envelope, and/or filtering. Examples of masking settings may include, but are not limited to, audio tracks consisting of recorded or synthesized sounds played at different volumes, audio tracks consisting only of silence, and/or audio tracks derived from functions of one or more other audio tracks in 304, the playback of which constitutes no sound addition to the acoustic environment in 312. Masking settings can alternatively or additionally refer to the process of selecting a mask, such as the process of selecting a mask from the database of audio tracks 304. Masking strategies can be autonomous and can adapt to the dynamics of the current soundscape (e.g., amplitude, frequency content, context) by means of monitoring the microphone or by digital input.

The data acquisition system 302 may also optionally be responsible for collecting group statistics or contextual data 314 as auxiliary parameters of the acoustic environment itself, and for providing group statistics or contextual data 314. The group statistics or contextual data 314 may include, but are not limited to, (1) non-acoustic environmental parameters, such as location (collected by a GPS sensor), number of people nearby (collected by a camera), air temperature (collected by a thermometer), humidity (collected by a hygrometer) and/or wind speed (collected by an anemometer); (2) group statistics of real or hypothetical listeners in the acoustic environment (e.g., collected by a digital form attached to the system or as pre-loaded data on a storage device), such as age, gender and/or occupation; (3) psychological data of real or hypothetical listeners in the acoustic environment (collected by, for example, methods similar to group statistics), such as noise sensitivity (obtained using, for example, the Weinstein Noise Sensitivity Scale), perceived stress (obtained using, for example, the Cohen Perceived Stress Scale), and/or well-being index scores (obtained using, for example, the WHO-5 Well-being Index); and/or physiological data of real or hypothetical listeners in the acoustic environment (obtained by, for example, a portable health monitoring device), such as heart rate, blood pressure and/or body temperature.

The data acquisition system 302 can provide ambient soundscape data 312 and optional crowd statistics or contextual data 314 to the perceptual attribute predictor 306. Given the ambient soundscape data 312, the masking settings of the database 304, and the optional crowd statistics or contextual data 314 as input, the perceptual attribute predictor 306 can output a prediction on one or more predefined perceptual attribute metrics for each masking setting in the database 304 (a prediction of the audience corresponding to the data in 314), as if the playback of each masking setting were to occur in the acoustic environment. For example, one or more predefined perceptual attribute metrics can be derived from the ISO 12913-3 octet metrics used for each masking setting in the database 304. For example, one or more predefined perceptual attribute metrics can be based on pleasantness, eventfulness, sound quality, perceived loudness, sharpness, roughness, etc., used for each masking setting in the database 304. Figure 4 illustrates a two-dimensional octet model of emotional quality attributes presented in ISO 12913-3, and the predictions of the perceptual attribute predictor 306 according to various embodiments can be based on this two-dimensional octet model.

In Figure 3, the set of predictions is represented by 316. Prediction 316 may or may not be a single numerical value. Similarly, in various embodiments, the generated prediction 316 may be nondeterministic, while in various other embodiments, the generated prediction 316 may be deterministic. The term "deterministic" can mean that the same output is produced whenever the same inputs of ambient soundscape data 312, multiple masking settings of database 304, and optional crowd statistics or contextual data 314 are provided to the perceptual attribute predictor 306. Probability distributions (or random values drawn from probability distributions), vectors, or any other multivariate representation of perceptual attributes can be possible outputs of prediction 316.

The perceptual attribute predictor 306 may be or may include any model that takes the same input from ambient soundscape data 312, multiple masking settings of database 304, and optional crowd statistics or contextual data 314 to output prediction 316. The model may internally transform the ambient soundscape data 312, the multiple masking settings of database 304, and the crowd statistics or contextual data 314 before delivering prediction 316 as output. Examples of the model may include, but are not limited to: (1) a linear regression model that takes acoustic or psychoacoustic parameters calculated from ambient soundscape data 312, multiple masking settings of database 304, and aggregated averages calculated from crowd statistics or contextual data 314 to output prediction 316; (2) a deep neural network that takes presentations calculated from ambient soundscape data 312, multiple masking settings of database 304, wherein presentations of crowd statistics or contextual data 314 are concatenated, to output prediction 316; (3) a composite network as shown in Figures 5 through 7; and/or (4) any combination as described herein.

Figure 5 is a schematic diagram of a perceptual attribute predictor 506 according to various embodiments. In various embodiments, the perceptual attribute predictor 506 may be configured to generate predictions by combining ambient soundscape data and each masking setting, for example, by combining them in an inference engine with pre-calibrated soundscape-to-masker ratios. The masking waveform 504a of each masking setting may be weighted (i.e. multiplied) with a masking gain input 504b, such as a masking gain waveform, to generate a weighted masking waveform. The weighted masking waveform may be combined with the ambient soundscape waveform 512 of the ambient soundscape data (i.e., added to the ambient soundscape waveform 512 of the ambient soundscape data) to generate an enhanced soundscape waveform 518. The prediction block 520 of the perceptual attribute predictor 506 may receive the enhanced soundscape waveform 518 to generate a prediction 510.

Various other embodiments may not require mixing, combining, or adding ambient sound scene data and each weighted masking waveform to form the enhanced sound scene waveform before inputting the enhanced sound scene waveform into the perceptual attribute predictor. By eliminating the need to mix, combine, or add ambient sound scene waveforms and each weighted masking waveform to form the enhanced sound scene data before inputting the enhanced sound scene waveform into the perceptual attribute predictor, these embodiments may allow for more efficient computational and bandwidth systems in practical deployments.

Figure 6 is a schematic diagram of a perceptual attribute predictor 606 according to various embodiments. In various embodiments, the perceptual attribute predictor 606 may include a soundscape feature extractor 622, which is configured to extract features 624 of the surrounding soundscape data, specifically features 624 of the surrounding soundscape waveform 612. The perceptual attribute predictor 606 may include a masking feature extractor 626, which is configured to extract features 628 for each masking setting, specifically features 628 of the masking waveform 604a for each masking setting. The perceptual attribute predictor 606 may include a feature level enhancer 630, which is configured to generate one or more enhanced soundscape features 632 based on the extracted features 624 of the surrounding soundscape data, the extracted features 628 of each masking setting, and a masking gain input 604b, such as a digital gain level. The perceptual attribute predictor 606 may include a prediction block 620, which is configured to generate a prediction 610 based on one or more enhanced soundscape features 632, extracted features 624 of the surrounding soundscape data, and extracted features 628 of each masking setting. In summary, the perceptual attribute predictor 606 may be configured to generate a prediction 610 based on extracted features 624 of the surrounding soundscape data and extracted features 628 of each masking setting.

Figure 7 is a schematic diagram of a perceptual attribute predictor 706 according to various embodiments. In various embodiments, the perceptual attribute predictor 706 may include an objective feature block 734, which is configured to extract objective features 736 from the surrounding soundscape data and each masking setting, specifically from the surrounding soundscape waveform 712 of the surrounding soundscape data and the masking waveform 704a of each masking setting. The perceptual attribute predictor 706 may be further configured to extract the objective features 736 based on the masking gain input 704b. The perceptual attribute predictor 706 may include a subjective feature block 738, which is configured to extract subjective features from crowd statistics or contextual data 714. The subjective feature block 738 may also be configured to process the extracted subjective features and the extracted objective features 736 to generate a prediction 710. The perceptual attribute predictor 706 may be configured to generate a prediction 710 based on the extracted objective features 736 and the extracted subjective features.

Referring again to Figure 3, the prediction 316 provided by the perceptual attribute predictor 306 can be input into the masking setting ranking system 308 to generate a single optimal masking setting 340. The masking setting ranking system 308 can determine the optimality using any suitable metric, which can be, but is not limited to, the maximum or minimum value of perceptual loudness, sound quality, sharpness, roughness, one or more emotional quality attributes shown in Figure 4, and/or any combination thereof. Alternatively, it is conceivable that the prediction 316 provided by the perceptual attribute predictor 306 can be input into the masking setting ranking system 308 to generate multiple optimal masking settings. For a given metric, there can be multiple optimal masking settings (e.g., if multiple masking settings are bound to the metric determined by the user).

The optimal masking setting 340 can be used as input to the playback system 310. The playback system 310 can realize the actual playback or reproduction of the optimal masking setting 340 in an acoustic environment. The playback system 310 may include, but is not limited to, one or more speakers, one or more virtual reality or augmented reality headsets, one or more headphones, one or more earbuds, and/or any combination thereof.

Various embodiments can be used in outdoor urban areas with poor sound quality (e.g., parks next to highways) to mitigate the harmful effects of perceived noise in the surrounding acoustic environment, or in indoor areas (e.g., inside nursing homes) to improve acoustic comfort and overall quality of the surrounding acoustic environment. Because the system is effectively designed to alter the perception of the surrounding acoustic environment, any commercial entity with indoor or outdoor areas whose acoustic environment needs to be maintained under fixed or desired conditions may be interested in the system.

Various embodiments are capable of selecting and adding masks to both real and imaginary ambient soundscapes. If an arbitrary audio input data acquisition system is used in place of a microphone for real-time recording, the embodiments can also create abstract “experience zones” by adding masks. Any commercial entity interested in creating or modifying soundscapes in virtual reality, mixed reality, or augmented reality based on audience perception will be of interest in this.

Various embodiments may involve systems that, based on optimized predicted perceptual responses (e.g., perceived loudness, ISO 12913-3 octet indexes—pleasantness, annoyance, tranquility, calmness, activity, eventfulness, etc.), select and set audio tracks (i.e., maskers) to be introduced into an existing soundscape (either played through, for example, speakers or digitally added through, for example, headphones) in a mixed/enhanced soundscape.

Various embodiments can select masking settings or maskers to be added to the surrounding acoustic environment based on predictions of human perception by a perceptual attribute predictor. This perceptual attribute predictor outputs values or distributions of perceptual attribute indicators based on surrounding soundscape data (real or hypothetical), data on candidate masking settings, and group statistics or contextual data of the audience for the possible surrounding soundscape. This embodies the ISO 12913 definition of a "soundscape," which considers the acoustic environment as perceived by one or more people within a context. In contrast, existing methods may select masking settings or maskers based on objective acoustic parameters of the surrounding acoustic environment (e.g., spectrum, sound pressure level, sound source type, and direction), and may not consider the subjective influence of human perception on masking selection or masking setting selection.

In various embodiments, the perceptual attribute predictor can output perceptual predictions in near real-time (with a delay of less than 10 seconds between input and prediction), and therefore can automatically and in real-time suggest optimal masking settings for a time-varying acoustic environment. In other words, as the acoustic environment changes over time, the embodiments can autonomously change the masking (i.e., audio track) and/or settings (e.g., gain level, spatialization) being played. In contrast, existing systems only allow static masking settings (potentially applying only dynamic volume/filter effects) or user-controlled masking settings (i.e., one must manually change the settings based on their own personal/expert opinion on optimality).

In various embodiments, the user can configure the system to automatically select the best mask from multiple options. The selection options can be arbitrary, but various embodiments may allow the following selection options not available in conventional solutions: (1) probabilistic top-k: sorting the predicted perceptual response values of the combined mask and the existing soundscape in different masks, and randomly selecting a mask setting from the k mask settings given the top (or bottom) k sorting values; (2) probabilistic distribution: extracting the predicted perceptual response values of the combined mask and the existing soundscape as random samples from the probability distribution generated by the system.

Existing systems and masking selection strategies may not allow for the selection of the best mask through dynamic strategies and may have one of the following characteristics: (1) manual selection: system users or administrators select the best mask based on their own opinions; (2) determinism: the selection of the best mask from different masks as a deterministic, predefined function of the combined mask and the perceived response value of the existing soundscape.

Various embodiments may include one or more audio transducers (e.g., speakers, headphones) and one or more sensors (e.g., microphones, environmental sensors), wherein the audio transducers and sensors are coupled to one or more computing devices. Various embodiments may offer advantages not available in existing systems/solutions.

For example, various embodiments may employ non-energy masking. Energy masking utilizes the physical limitations of peripheral human auditory processes. The amplitude and frequency bandwidth of the masking sound are adjusted so that the target noise is completely or partially inaudible at the inner ear. Therefore, objective metrics such as sound pressure level or psychoacoustic parameters are typically used to optimize or select the energy masker. Various embodiments may refer to (but are not limited to) the soundscape concept as defined in ISO 12913-1. In the literature, soundscape enhancement using additional sound is also commonly referred to as masking. Masking techniques may include energy masking and information masking (or perceptual masking), where information masking consists of effects related to the perceptibility of sound, which is associated with audio in the higher brain centers. Therefore, the mask can be determined by perceived factors (such as perceived emotional attributes and perceived annoyance in ISO 12913-2) and is not based solely on objective indicators in existing systems/solutions.

The advantages of non-energy masking are evidenced in literature on traffic noise masking using natural sounds, where the reduction in perceived loudness from energy masking gradually disappears as the target noise level increases. This contrasts with the statistically significant reduction in perceived loudness and improvement in soundscape quality observed when the masking device providing information masking is presented at a level up to 6 dB lower at higher levels of traffic noise. Furthermore, similar significant reductions in perceived loudness and improvements in soundscape quality were observed when using birdsong as a masking device, where birdsong as a masking device fails to energy mask traffic noise, further demonstrating the effectiveness of non-energy masking.

The ability to contextualize audio input via auxiliary input allows various embodiments to select a mask for any target soundscape (instead of being limited to office environments described in existing literature). Masks according to various embodiments may also be not limited to audio tracks generated from random noise, nor to natural sounds as described in existing literature. The number of masks that can be added is unlimited, as long as the desired perceptual properties are optimized (optimization can mean "maximizing" or "minimizing").

In various embodiments, the audio input data can be of any format and can be real-time data from a physical microphone or an audio track accessed from a storage medium. Similarly, the masking according to various embodiments can be streamed to speakers (or headphones) or digitally mixed with a target soundscape on a storage medium and played as a single combined audio track in a virtual environment. Therefore, the various embodiments are capable of masking soundscapes in both real-world and virtual environments (e.g., in metaverse applications).

Dynamic masking described in existing literature is based on objective metrics, such as sound level, audio spectrum, or proximity to the identified target noise source. Various embodiments can dynamically or adaptively adjust the masking settings based on predicted perceptual properties. The masking settings are not limited to sound level or spectrum adjustments.

Example Study 1

introduce

It is known that indications based solely on the sound pressure level (SPL) of a given acoustic environment are often insufficient to reflect the level of annoyance caused by excessive noise and its impact on quality of life. This has led to the rise of the soundscape approach to noise control, which focuses on interventions that improve perceived noise properties rather than simply reducing SPL. The ISO 12913 series of international standards on soundscapes aims to standardize this approach by providing a circular model of pleasantness and eventability, which allows for the comparison of interventions based on subjective assessments of the surrounding acoustic environment.

Therefore, many studies have utilized soundscape enhancement techniques to modify the perception of soundscapes by adding maskers to urban or indoor acoustic environments, thereby optimizing metrics such as subjective evaluations of pleasantness or calmness. However, the selection of maskers is often arbitrary, expert-guided, or based on post-hoc analysis. Arbitrary masker selection may be unreliable in achieving the desired perceptual changes, expert-guided selection is laborious and time-consuming, and post-hoc analysis may not generalize to unseen maskers and soundscapes in unobserved contexts.

One approach to overcome these limitations is to train a predictive model to predict the value of one or more perceptual attributes, as subjectively assessed by a person given the original auditory stimulus, based on a diverse selection of acoustically varied soundscapes and maskers. After training, simulated additions of unseen soundscapes to the masker can be fed into the model as input to obtain predictions of the perceptual attributes. The masker that produces the most enhancement or reduction to the attribute can then be selected as the optimal mask for real-time augmentation.

Various embodiments may relate to a neural approach for an automatic masker selection system (AMSS) that enhances urban soundscapes by optimizing the pleasantness of the acoustic environment. By using a probabilistic output scheme, the model can predict the pleasantness distribution for each combination of soundscape and masker, instead of a single deterministic value. This allows for explicit acquisition of the confidence level of the prediction as well as the predicted pleasantness.

Related work

In soundscape research, studies developing predictive models for perceptual attributes have primarily focused on simpler machine learning models, such as linear regression, support vector machines (SVMs), and shallow multilayer perceptrons. These machine learning models utilize input features based on acoustic measurements, psychoacoustic parameters, environmental features, or one or more linear combinations of machine learning models elucidated by principal component analysis. Scaling metrics for Likert scales have also been proposed to account for nonlinearities in the scales used to develop these systems.

On the other hand, recent systematic reviews show that while deep neural networks are prevalent in more “objective” tasks such as sound event localization, detection, and classification, research on using deep neural networks to predict soundscape perception attribute values is scarce. In this regard, the most important research appears to be a comparison of the performance of SVM, convolutional neural networks (CNN), long short-term memory networks (LSTM), and fine-tuned VGGish networks on the Emo-Soundscapes dataset. Based on subjective pairwise comparisons using a self-assessment manikin model, the Emo-Soundscapes dataset contains 1213 audio segments ranked by valence (pleasure) and arousal (eventality). The fine-tuned VGGish model and the CNN trained from scratch exhibited the smallest mean squared errors (MSEs) in predicting the valence and arousal of the audio segments, respectively. A more recent study used a similar model, employing a circular model similar to ISO 12913, to classify music into one of four quadrants using a dataset of approximately one thousand audio clips, and further categorized music into neutral, calm, happy, sad, angry, and fearful categories. However, soundscapes typically encompass more than just music, so it remains to be investigated whether the results can generalize to a wider range of acoustic stimulus categories.

Furthermore, these existing models in the literature are deterministic models, meaning that the same acoustic environment always maps to the same predicted value, without considering the inherent uncertainty in subjective ratings used as baseline true values. However, these ratings given by individuals are inherently random, given factors that cannot be reasonably controlled, such as a person's current stress level or noise sensitivity. Therefore, various embodiments may involve probabilistic methods in which a neural network is trained to predict a distribution of possible ratings for the output, and then extracted from that distribution, allowing the model to account for different degrees of uncertainty in each unique soundscape.

The proposed method

Figure 8 illustrates the training and inference diagrams of the automatic mask selection system (AMSS) according to various embodiments.

Consider a soundscape and a set of K candidate masks { _Mk } that can be added to the soundscape. Let represent the enhanced soundscape with mask _Mk . In other words, is the enhanced soundscape, where the responses of the playback system and the real-world environment are considered. Finding the optimal that maximizes the evaluation of a given perceptual (or emotional) attribute f of the enhanced soundscape is equivalent to finding , where f can be, for example, a rating on a numerical metric for pleasantness, eventiness, or other perceptual attributes. Then, we can use the enhancement to obtain .

For example, a naive implementation of AMSS could use one or more approximators to predict the perceptual attributes of each enhanced soundscape output and select the mask with the highest value. However, as mentioned earlier, this deterministic output scheme ignores the inherent uncertainty associated with subjective ratings on perceptual attribute metrics. For instance, one soundscape might be very pleasant to some but very disturbing to others, while another might be slightly pleasant to almost universally listeners, depending on the context.

To collect this uncertainty, it can be viewed as a random variable with a joint distribution with one or more “random” soundscapes, where the observed augmented soundscape is considered its realization. The uncertainty is then expressed as the variance of . In this case, the predictive model will attempt to model the conditional distribution as one or more functions of the augmented soundscape. For simplicity, is abbreviated as .

A neural approximator 806 can be used for the output. The neural approximator 806 can be called a probabilistic perceptual attribute predictor (PPAP). In the proposed AMSS shown in Figure 8, each mask can be passed to the aforementioned PPAP 806, which outputs one or more parameters of one or more predefined distribution families. The PPAP 806 can be trained to output the mean and log-standard deviation of a normal distribution, which are used to model the output. Once sufficient (or all) predictive distributions have been computed, the “best” mask can be selected based on one or more predetermined criteria. For example, the mask can be selected based solely on the highest value, a set of similar or analogous values can be sampled to incentivize mask exploration, or more complex criteria can be considered.

Although only a few observations are available and the true distribution is unknown, the model can still be optimized given the output distribution by maximizing the log probability of the ground truth in a Bayesian optimization-inspired manner. Given the soundscape, the set of masks, and the ground truth, the contribution of the loss function to the soundscape can be given by the following equation:

(1)

(2)

Here, is the logarithmic density function of the output distribution, where the additive constant is omitted. During training, the model can be optimized using batches of soundscape-mask pairs with available benchmark true values.

As shown in Equation (2), the loss function using a normal distribution can be considered as a loss of weighted MSE regularized by log-standard deviation. This loss function is inherently stable relative to the first incentive term, while the second incentive term is smaller. Of course, some other choices of the output distribution can be similarly reduced to other deviation measures, such as a regularized Laplace distribution of the mean absolute error. It should be noted that deterministic learning can also be forced by setting _σk to some predetermined constant and training the model using a loss of pure MSE between the ground and the ground. For the purpose of ablation studies, the following equation can be used.

(3)

As the deterministic counterpart of formula (2), the deterministic loss in formula (3) can be considered as formula (2) with static = 1.

Verification Experiment

To validate the proposed system, we use the normalized pleasure metric defined by the ISO 12913-3 standard, also known as the ISO pleasure metric. Specifically,

(4)

These represent the degree to which participants perceived the enhanced soundscape as pleasant, disturbing, calm, chaotic, active, and monotonous, expressed using a 5-point Likert scale. For each, a predictive distribution can be generated, where the “benchmark true value” label is the participant’s ISO pleasantness rating. As an ablation study, the model trained using the proposed method will be compared with a deterministic model based on direct prediction. Validation experiments were conducted on a dataset of subjective responses to each enhanced soundscape.

Dataset

This dataset contains 12,648 samples in total, including 12,600 unique sets of subjective responses to the enhanced soundscape from the 5-fold cross-validation set, and 48 additional sets of responses from the independent test set. Given a set of responses to the following six subsets of the ISO 12913-2 Affective Response Questionnaire: “To what extent do you agree or disagree that your current surrounding sound environment is {pleasant, chaotic, active, calm, disturbing, monotonous}?”, each sample maps the enhanced soundscape (as a raw audio recording) to an ISO pleasantness value.

Participants responded using a 5-point scale with multiple labels: “Strongly Disagree,” “Disagree,” “Neither Agree nor Disagree,” “Agree,” and “Strongly Agree,” coded as 1, 2, 3, 4, and 5, respectively. The coded responses were then used in Equation (4) to calculate the baseline true value label for ISO pleasure level for each enhanced soundscape.

Enhance soundscape

The enhanced soundscapes in the 5-fold cross-validation set were created by adding 30-second extracts of stereo recordings of soundscapes from the Urban Soundscapes of the World (USotW) database as “masks” to the original recordings. The USotW database is a comprehensive dataset of urban soundscapes. Unenhanced soundscapes were also included as a control. All recordings used were sampled at 44.1 kHz.

Each fold contains a group of 56 maskers, which are categorized as follows: birds (16), construction (8), traffic (8), water (16), and wind (8). These categories were chosen to cover a range of sound types that were generally assessed as pleasant and disturbing. The maskers and soundscapes in each fold are disjoint.

The enhanced soundscapes in the test set were created in a similar manner, with seven masks (eight masks in total, including the unenhanced control mask) independent of the masks in the cross-validation set, and six stereo recordings of the soundscapes independent of the USotW dataset, recorded using the same Soundscape Indices Protocol. The seven masks were extracted from xeno-canto track IDs 640568 (bird "B1") and 568124 (bird "B2"), and Freesound track IDs 586168 (construction "Co"), 587219 (traffic "Tr"), 587000 (water "W1"), 587759 (water "W2"), and 587205 (wind "Wi"). The masks were exhaustively added to all soundscapes in the test set, resulting in 48 samples. Before adding a mask to the test set with a constant, 0 dB soundscape to mask ratio and to the cross-validation set with a mask in dB values randomly selected from {-6,-3,0,3,6}, all soundscapes were calibrated to an A-weighted equivalent SPL measured in real time according to the method described by K. Ooi et al. (“Automation of Stereo Headphone Audio Calibration on an Artificial Head”, MethodsX, Vol. 8, No. 2, p. 101288, 2021).

Subjective reaction

To obtain subjective responses to the enhanced soundscapes, 300 participants were recruited, each rating 42 unique enhanced soundscapes randomly selected from a compromise of the validation set. An additional 5 participants were recruited, each rating 48 enhanced soundscapes in the test set. Thus, each enhanced soundscape in the validation and test sets was rated by one participant and five participants, respectively. All participants listened to the calibrated enhanced soundscapes using a pair of over-ear headphones (Beyerdynamic Custom One Pro) driven by an external sound card (Creative Sound Blaster E5). After listening to each enhanced soundscape, they answered the 6 questionnaires described above (under the heading “Datasets”), and ISO pleasantness was calculated from their responses according to Equation 4.

Model and Training

Log-mel spectrograms for enhanced soundscapes were used as input. Log-mel spectrograms were extracted using a 4096-sample Hann window with 50% overlap, and then compressed to 64 mel bins. Figure 9 illustrates (a) the basic convolutional recurrent neural network (CRNN) architecture used according to various embodiments; (b) a possible implementation of the feature mapping block in (a) according to various embodiments; and (c) other possible implementations of the feature mapping block in (a) according to various other embodiments. Four different feature mapping blocks were investigated: a vanilla mapping block using a bidirectional gated recurrent unit (BiGRU), an additive attention block, a dot-product attention block, and a multi-head attention block with four heads. The vanilla mapping block is shown in Figure 9(b), while all attention-based blocks sharing the same general workflow are shown in Figure 9(c). The last few layers of the model in Figure 9(a) are dense fully connected layers, which ultimately output _μk and . In the deterministic ablation model, these are ignored.

All models were trained using a 5-fold cross-validation scheme, with 10 models per fold, for a total of 50 models. Each fold used the same 10 seeds for multiple models. All models were trained using the Adam optimizer with a learning rate of 5 × ^10⁻⁵ for a maximum of 100 epochs. For each model, the model weights with the optimal validation loss were evaluated on both the cross-validation and test sets.

Results and Discussion

Figure 10 illustrates the mean fold mean squared errors (MSEs) (± standard deviation) of the Probabilistic Aware Attribute Predictor (PPAP) across 10 runs for each test setting according to various embodiments (top: cross-validation set, bottom: test set). Each run may include five models, each on a different fold of the cross-validation set, but with the same initial conditions. For all models, the contribution to the MSE is calculated. An asterisk (*) indicates a statistically significant improvement (p<0.05). Figure 10 summarizes the results of the validation experiments described above on both the 5-fold cross-validation set and the independent test set. For reference, results for a simple (trivial) deterministic “label mean model” are also provided. In this “label mean model”, the label mean from the training set is used as the prediction for all stimuli in the validation and test sets. All other research models outperform the label mean model, thus demonstrating the benefit of feature extraction by other research models. This may be because the test set is smaller and less diverse (48 samples, 7 masks) compared to the validation set (12,600 samples, 280 masks).

Furthermore, for the four architectures tested, the model with probabilistic output outperformed the equivalent model with deterministic output on both the validation and test sets. Figure 10 also shows the percentage reduction in MSE for each model. It can be observed that the CRNN with multi-head attention blocks and the basic CRNN showed the largest improvements in MSE on the validation and test sets, at 1.1% and 7.8%, respectively.

To quantify the significance of these reductions, two-sided Wilcoxon signed-rank tests were performed on both the validation and test set MSEs, comparing deterministic and probabilistic models. In Figure 10, reductions significant at the 5% significance level are marked with an asterisk. For both the base CRNN and PPAP with additive attention, the reduction in test set MSE was statistically significant, providing evidence to support the generalizability of the proposed method on unseen data. This can be attributed to the inherent randomness of the contributions from test set participants, which differs from that of participants in the training set, and PPAP's good generalization to this randomness. However, further ablation studies are needed to validate this hypothesis.

However, the lowest improvement in test set MSE was actually observed in PPAP with multi-head attention blocks. This can be attributed to the fact that PPAP with multi-head attention blocks has about 50% more parameters than the other three models (about 120K vs. about 80K), which may cause the trained models (both deterministic and probabilistic) to overfit the relatively small dataset of 12,600 training and validation samples.

Figure 11 illustrates mask selection using 50 models (dot product variables) on the test set, employing a naive maximum selection scheme (left) and a random sampling scheme (right) to incentivize masker exploration, according to various embodiments. The selection criteria used for each basic soundscape are shown at the top of each subplot. Cell color represents the average ISO pleasure rating by test set participants. Circle size indicates the number of models selected for that mask. Masker descriptions have been provided previously (under the heading "Enhanced Soundscapes").

In the naive choice scenario, most models selected the first birdcall masker (B1) across all basic soundscapes. This aligns with the consensus in previous soundscape studies, which generally observed higher pleasantness responses to birdcall maskers. However, the second birdcall masker (B2) likely received lower overall participant ratings, and consequently, fewer model votes, because it was perceived as less pleasant in the context of the test set soundscape, as observed in previous studies.

In the random selection scenario, the system still tends to choose B1, which provides a good pleasure score for most soundscapes. However, the system now explores other masks much more extensively than in the naive scenario. This can be beneficial in real-time systems where human feedback can be obtained in real time to adaptively adjust mask selection or improve future models. It can also be seen that in the case of soundscape 5, the unenhanced soundscape already has a relatively high pleasure score and a higher exploration rate than other basic soundscapes, allowing for a more diverse acoustic experience with only minor compromises in pleasure level.

in conclusion

Various embodiments may relate to an Automatic Masking Selection System (AMSS) for human-centered urban soundscape enhancement, which uses a Probabilistic Perceived Attribute Predictor (PPAP). The proposed PPAP is implemented using a convolutional recurrent neural network and trained to output predictions probabilistically. This allows the system to predict the perceptual attributes of soundscapes while taking into account the inherent randomness in the human subjective perception of acoustic stimuli. Through a large-scale listening test with over 300 participants and over 12,000 unique soundscapes, we validated the effectiveness of our PPAP in predicting the pleasantness of enhanced soundscapes, which included soundscapes generated from unseen and masked soundscapes. Future work on AMSS may include real-time implementation to assess its ecological validity, and studies of the proposed method on other perceptual attributes (f), such as eventiness or calmness, since the proposed method is not specific to any particular attribute. In fact, since the main assumption upon which PPAP is based is the main assumption of randomized baseline true value labeling, it can also be envisioned for application to any situation requiring the prediction of subjective evaluations.

Soundscape enhancement, which involves adding sounds called “masks” to a given soundscape, is a human-centered urban noise mitigation measure aimed at improving the overall soundscape quality. However, mask selection is often based on cumbersome predictive processes and is inflexible in the face of the time-varying nature of real-world soundscapes. Due to the perceptual uniqueness of each soundscape and the inherent subjectivity of human perception, this application proposes a Probabilistic Perceptual Attribute Predictor (PPAP), which predicts randomly distributed parameters as outputs instead of a single deterministic value. Using PPAP, this application develops an Automatic Mask Selection System (AMSS) that selects the best mask candidate based on a predicted distribution of ISO 12913-3 pleasantness ratings for a given soundscape. A PPAP model was trained using a 5-fold cross-validation scheme by collecting 12,600 subjective responses, each for a unique enhanced soundscape, through a large-scale listening test with 300 participants. In tests using a convolutional recurrent neural network backbone and several variables with an attention mechanism for PPAP, this application evaluated the proposed system using a blind test set of 48 unseen enhanced soundscapes to assess the effectiveness of the probabilistic output scheme relative to conventional deterministic systems.

Example Study 2

introduce

Reducing urban noise pollution is a complex and multifaceted problem, where eliminating or reducing noise sources is often not a practical option. Unlike conventional noise control methods that aim to reduce sound pressure levels or sound energy, the soundscape approach proposes a more comprehensive, human-centered strategy based on perceived acoustic comfort. This technique, often called soundscape enhancement, involves adding "desired" sounds to the acoustic environment to "mask" noise and improve overall perceived acoustic quality. This technique has achieved satisfactory results for various types of noise sources in both virtual and real environments.

However, the selection of “masks” and playback levels is typically arbitrary, expert-guided, or based on post-hoc analysis. These masking selection processes are not only often time-consuming and labor-intensive, but also inflexible regarding the dynamic characteristics of real-world soundscapes. Therefore, these static selections of masks and playback levels often cannot consistently achieve optimal acoustic comfort. To address these limitations, Example Study 1 demonstrates a first attempt at adaptive soundscape enhancement using a masking selection system driven by a probabilistic deep learning model that predicts a probability distribution of “pleasure” for a specific enhanced soundscape.

While the Probabilistic Attribute Predictor (PPAP) model described above provides a compelling proof of concept for deep learning-driven automatic soundscape enhancement, it may not be designed for real-time deployment. Implementing this PPAP model requires a premixed track of the enhanced soundscape as input. The mixing process is performed at a pre-calibrated soundscape-to-mask ratio relative to a known real-time SPL level on the recorded, unenhanced soundscape track. This could lead to a model that ignores the masking gain level—a critical playback parameter for real-time deployment. Furthermore, requiring the mixing of the base soundscape recording with the mask as input adds unnecessary complexity to the entire system. First, all candidate masks require a digital gain lookup table that plays the desired soundscape-to-mask ratio (SMR) for a given soundscape. Second, during the selection process, each candidate mask-SMR pair requires computation of the enhanced soundscape track, increasing the system's computational overhead.

Various embodiments can present a PPAP model with a modified computational formula, which will allow for more efficient computational and bandwidth systems in real-world deployments. The proposed model can decouple the base soundscape, mask, and gain level from each other at the input stage, and instead introduce separate feature extractor branches for the base soundscape and mask. The "enhancement" process occurs only in the feature space instead of the waveform space, using the mask's digital gain level, independent of the real-time sound pressure level (SPL), as a modulating input. Then, based on the base soundscape features, masking features, and gain modulating features, an attention mechanism is used to "query" the distribution of the target's perceptual attributes. The proposed method leads to a reduction in the complexity of several system components. First, it is not necessary to send the raw waveform of the soundscape to the inference engine for mixing with the mask, as less bandwidth-intensive spectral data can be used as model input. For cloud inference, this means a significant reduction in the data egress rate at the edge. Second, the masking features can be pre-computed independently of the gain level, which reduces inference time and thus overall latency. Third, attribute prediction for multiple gain levels of the same mask can now be performed faster without recalculating the enhanced soundscape or even the soundscape and masking features. Finally, new masks can now be added to the system without creating lookup tables.

The proposed method

Preparation

Consider a digital soundscape recording, where C≥1 is the number of channels and is the sample index. The real-time A-weighted equivalent sound pressure level (SPL) of s_{_i} is expressed in dBA. For simplicity, all SPL references refer to 30 seconds unless otherwise stated. It is assumed that all soundscapes are recorded using the same real-time recording setup, and that, under the assumption of approximate linearity, the sound pressure-to-digital-level ratio (SPDR), i.e., the ratio of the relative SPL on a linear scale to the set digital full-scale (DFS), can be considered approximately constant, and is denoted by , with units of , where is the reference SPL. Since all soundscape inputs originate from the same recording device, this also means that real-time SPL information is implicitly embedded in the input spectrogram data.

Consider single-channel masks. For masks, assume they are not all recorded by the same recording device. In practice, this is because masks are often derived from recording systems different from the real-time recording device and are typically obtained through open content providers such as Freesound and Xeno-canto. Therefore, the SPDR of each mask is usually unknown and cannot be assumed to be the same as the SPDR of any other mask. Let represent the SPDR of the mask. Under the assumption of approximate linearity (verified by thorough gain calibration of all masks used in this paper), the mask can be normalized to the same SPDR as the soundscape recording by scaling the mask. If both the soundscape and the real-time SPL are known, soundscape enhancement in dBA units with a specific soundscape-to-mask ratio (SMR) can be performed relatively easily without further calibration. However, in practice, this is often unclear, so the conventional solution is to calibrate using a playback system with a digital level-to-sound pressure ratio (DSPR) to obtain the digital gain required for sound pressure level reproduction from among multiple values of . The calibration required to obtain an accurate value of often requires specialized equipment, such as a recording device with an artificial head, and can be time-consuming and/or labor-intensive, depending on the calibration method used. In this paper, is explicitly used as an adjustable input, thus eliminating the need for SMR or masking calibration related to SPL.

Model

Figure 12A illustrates a schematic diagram of the Probabilistic Aware Attribute Predictor (PPAP) according to various embodiments. Let be the log-Mel spectrogram of the soundscape, and be the log-Mel spectrogram of the mask, where is the number of time frames of the spectrogram, and is the number of Mel bins. In this paper, , , and correspond to a 30-second signal sampled at 44.1 kHz with a short-time Fourier transform (STFT) window size of 4096 samples and a 50% frame overlap rate. The soundscape and mask embeddings are computed by feature extractors 1222 and 1226, respectively, such that

(5)

(6)

Where is the embedding dimension and the number of feature time frames. Each feature extractor 1222, 1226 can include 5 convolutional blocks, each containing a convolutional layer with a 3×3 kernel and a stride of 1; batch normalization; dropout with a probability of 0.2; Swish activation; and a 2×2 average pooling layer. The convolutional layers contain 16, 32, 48, 64, and 64 output channels in sequence.

In the case of an attention-based query-key-value (QKV) model, the perceptual attribute prediction process can be viewed as using a masked query map, where the unenhanced soundscape provides the key, and the enhanced soundscape provides the value. We consider performing "enhancement" in the embedding domain (in feature enhancement block 1230) instead of performing soundscape enhancement in the audio or spectral domain. This can be accomplished using a simple map adjusted to mask digital gain, so that...

(7)

This refers to the query gain level, which is the gain adjustment enhancement layer. There are three different implementations, namely...

(8)

(9)

(10)

Where tensor | is the tensor concatenation on the last axis, || is the tensor stacking operator that creates the additional axis, is a fully connected layer with D output units, and is a convolutional layer with D filters that uses a one-dimensional kernel to compress the stacked dimensions into a single axis. Figure 12B illustrates three feature enhancement methods according to the computation of equations (8), (9), and (10) in the various embodiments. During training, if the mask is a silent track (i.e., no enhancement), we will randomize, where and are the mean and standard deviation of the logarithmic gain of the training samples with non-silent masks. This is to teach the model to ignore the gain value of the silent mask. With the query, key, and value embeddings in place, any QKV-compatible attention block 1220a can be used to compute the output embeddings such that

(11)

Then, output block 1220b computes the predicted distribution, and output block 1220b contains the parameters of the predicted distribution. Three types of attention can be considered: additive attention (AA), dot-product attention (DPA), and multi-head attention (MHA). In this example, 4-head MHA is used.

loss function

As in Example Study 1, subjective human responses can be considered inherently random. On the one hand, the participants are a sample of the population; on the other hand, each participant's response during the listening test is also a random sample of their inherently nondeterministic perceptions. Therefore, the label used for the target attribute (i.e., ISO pleasantness) can be viewed as an observation of a random variable representing the distribution of the target attribute, the true distribution of which is unknown. Using the maximum likelihood formula, the optimization objective of the PPAP model can be given by the following formula:

()(12)

Here, represents the model parameters, and represents the logarithmic density of X estimated at x. Subjective responses to each acoustic scene are modeled using independent normal distributions, and the optimization objective in equation (12) is transformed into a loss function.

(13)

This includes training rounds.

Optimize reasoning

Although the model sees multiple batches of different soundscape-mask-gain samples during training, during real-time inference, the model sees only one basic soundscape at a time, while simultaneously experiencing multiple maskers and gain levels to find the most suitable masker-gain pair for a specific soundscape. Therefore, a significant performance improvement can be achieved by eliminating redundant computations instead of performing a new forward pass for each masker-gain combination. Let , , and represent the execution time required for , , and , respectively. Let be the number of masks to be queried for each soundscape, and be the number of gain values to be queried for each mask. The naive batching scheme would require a total runtime of . In contrast, by optimizing the query process, as shown in Algorithm 1 of Figure 13, the total runtime can be reduced to . Figure 13 illustrates algorithms for reducing the total runtime according to various embodiments. Additionally, for a fixed set of maskers, masking features can be pre-computed, which allows for a further reduction in the total runtime to .

Dataset

The dataset used in this paper is an extension of the dataset used in Example Study 1, with additional participants, totaling 18,564 data points (42 stimuli per participant) for 442 participants. Each data point represents a participant's subjective response to the augmented soundscape.

A. Stimulation

The enhanced soundscape was created by adding a 30-second “mask” to a 30-second stereo soundscape recording sourced from the City Soundscapes of the World (USotW) database, resulting in five mutually exclusive trade-offs. An unenhanced soundscape (i.e., one “enhanced” using a silence mask) was also included as a control. All recordings used were sampled at 44.1 kHz. More details can be found in Example Study 1.

Masks were added to the soundscape at a specific soundscape-to-mask ratio (SMR) of {-6,-3,0,3,6} dBA, where the SMR is defined by the ratio of the soundscape's SPL to the mask's SPL. While the real-time SPL of the base soundscape track is known, the mask data is sourced from online repositories, specifically Freesound and Xeno-canto, and therefore requires calibration. A lookup table was created to amplify the j-th mask to a specific SPL by thoroughly calibrating the track to SPL values in a closed interval of 46 dBA to 83 dBA on a dummy head (GRAS 45BB KEMAR Head & Torso) in 1 dBA steps. The calibration process was semi-automatic, using the same audio playback devices as the participants. Since the real-time SPL of the base soundscape is typically non-integer, the non-integer digital gain was interpolated from the lookup table, where...

(14)

This refers to the calibration gain used for the mask under the SPL of dBA.

B. Subjective reaction

All participants listened to calibrated enhanced soundscapes using a pair of Beyerdynamic Custom One Pro surround sound headphones driven by an external sound card (Creative SoundBlaster E5). After listening to each enhanced soundscape, participants answered a six-item questionnaire based on the ISO 12913-2 standard: “To what extent do you agree or disagree with your current surrounding sound environment [...]?”, where [...] is one of pleasant, chaotic, active, calm, disturbing, or monotonous. Each questionnaire item had multiple options: “Strongly Disagree,” “Disagree,” “Neither Agree nor Disagree,” “Agree,” and “Strongly Agree,” which were numerically coded from 1 to 5. The baseline true value label of ISO pleasantness for each enhanced soundscape was then calculated using the following formula.

(15)

The numbers 1, 2, 3, 4, 5, 6, and 7 represent the numerical ratings of the questionnaire items, respectively, indicating pleasant, disturbing, calm, chaotic, active, and monotonous feelings.

test

For all model types in this paper, each model was trained using 5-fold cross-validation with the same 10 seeds for each validation set, for a total of 50 models per model type. Each model was trained for a maximum of 100 epochs using an Adam optimizer with a learning rate of 5 × ^10⁻⁵ . The learning rate was halved if the validation mean squared error (MSE) did not decrease within at least 5 epochs, and training was stopped early if the validation MSE did not decrease within at least 10 epochs. The MSE and mean absolute error (MAE) disclosed in this paper are calculated using the difference between the mean of the predicted distribution and the baseline true label.

In addition to the three attention types mentioned above (under the heading "Model"), the test also included a "pass-through" attention block as the baseline model. The baseline model is represented by an "X" in Figure 14. A pass-through block can be defined as...

(16)

Figure 14 shows (top) a graph of mean squared error (MSE) as a function of attention block type, illustrating a violin plot of the validation MSE for pleasure predictions for each attention block type and feature enhancement method according to various embodiments; and (bottom) a graph of mean absolute error (MAE) as a function of attention block type, illustrating a violin plot of the validation MAE for pleasure predictions for each attention block type and feature enhancement method according to various embodiments. Figure 14 shows the prediction error distribution of MSE and MAE for each attention and enhancement type. Among all attention types, the CONV enhancement method performs best in both MSE and MAE, followed by CAT and ADD. This can be attributed to the CONV method and kernel filtering. The CONV method naturally encourages feature alignment between attention and feature blocks, while kernel filtering allows for more flexible enhancement operations in the feature space compared to the other two methods.

It can also be seen that using QKV attention generally allows for better performance compared to models without attention. However, the type of attention mechanism does not seem to have a significant impact on model performance as long as some form of attention is used to couple the query, key, and value tensors. Although AA and DPA only have one attention head, the four-head MHA does not seem to achieve a noticeable improvement in prediction error. It should also be noted that the choice of feature augmentation blocks seems to have a significant impact on model accuracy, as evidenced by the fact that the non-attentional CONV augmentation model generally performs better than the attentional ADD model.

Although the model never saw the audio data of the enhanced soundscape presented to the participants, the best-mean ...

Figure 15A is a diagram illustrating ISO pleasure as a function of logarithmic gain, showing gain interpolation for the masker category "Water" using additive attention (AA) with CONV enhancement (seed 5, verification fold 2) according to various embodiments. Figure 15B is a diagram illustrating ISO pleasure as a function of logarithmic gain, showing gain interpolation for the masker category "Traffic" using additive attention (AA) with CONV enhancement (seed 5, verification fold 2) according to various embodiments. Figure 15C is a diagram illustrating ISO pleasure as a function of logarithmic gain, showing gain interpolation for the masker category "Bird" using additive attention (AA) with CONV enhancement (seed 5, verification fold 2) according to various embodiments. Figure 15D is a diagram illustrating ISO pleasure as a function of logarithmic gain, showing gain interpolation for the masker category "Construction" using additive attention (AA) with CONV enhancement (seed 5, verification fold 2) according to various embodiments. Figure 15E is a graph illustrating ISO pleasantness as a function of logarithmic gain, showing gain interpolation for the masker category "Mute" using additive attention (AA) with CONV enhancements (seed 5, validation fold 2) according to various embodiments. For this visualization, all masks are tuned to 65.0 dBA at zero logarithmic gain. The base soundscape has a real-time SPL level of 65.32 dBA. During training, the model does not see the base soundscape or all masks. The solid lines in Figures 15A through 15E represent the mean of the prediction distribution. The shaded areas represent the pleasantness level from the highest to above the prediction mean and the lowest to below the prediction mean, representing the standard deviation of the prediction.

Figures 15A through 15E demonstrate the model's gain-aware nature by querying the ISO pleasantness distribution across 256 logarithmic gain values in the [-2,2] interval for different types of maskers. A silent "masker" track is also included to test the model's ability to consider both the masker itself and its gain level. While the pleasantness predictions are not perfectly constant, it can be seen that the model can discern a silent masker without considering the query gain value and outputs a roughly constant pleasantness prediction. For other non-silent maskers, it can be seen that the model is aware of the characteristics of each masker. Consistent with previous findings, the model predicts pleasantness as the bird song masker gain increases until it approaches the ambient level (γ≈-0.5), after which the pleasantness drops below the pleasantness of the unenhanced soundscape (i.e., the silent masker prediction). Underwater acoustic maskers exhibit a similar effect, with the highest pleasantness level (up to γ≈−0.5) exceeding that of the unenhanced soundscape. However, due to the more continuous nature of underwater sounds, the pleasantness level begins to decrease at lower gains than with birdsong maskers. For maskers known to be unpleasant, such as traffic and construction, the model correctly outputs increasingly unpleasant predictions as the gain level increases.

in conclusion

Various embodiments may involve an improved probabilistically aware attribute predictor model that allows for gain-aware prediction of subjective responses to enhanced soundscapes. The proposed model decouples the masker from soundscape feature extraction and models the soundscape enhancement process in the feature space, thereby eliminating the need for computationally costly mixing processes in the waveform domain. Furthermore, the model is refactored to consider the digital gain level of the masker instead of the soundscape-to-mask ratio, allowing for the use of masks from multiple sources without time-consuming calibration. The modular design of the model allows for the reuse and pre-computation of a large number of features during inference time, reducing the overall latency and computational resources required for deployment. The model has been demonstrated to accurately predict pleasure ratings associated with soundscape, masker, and gain using a large dataset of 18,000 subjective responses from 442 participants.

In soundscape enhancement systems, the selection of maskers and playback gain levels is crucial to the effectiveness of the system in improving the overall acoustic comfort of a given environment. Conventionally, the selection of appropriate maskers and gain levels is determined by expert opinions that may not be representative of the target audience, or by potentially time-consuming and laborious listening tests. Furthermore, the resulting static selections of maskers and gains are often inflexible in relation to the dynamic characteristics of real-world soundscapes. In this paper, a deep learning model is used to jointly select the optimal masker and the optimal gain level for a given soundscape. The proposed model employs a highly modular building block design, allowing for an optimized inference process that can rapidly search a large number of masker and gain combinations. Moreover, the introduction of feature-domain soundscape enhancement with tuning at the digital gain level eliminates the computationally costly waveform-domain mixing process during inference, as well as the cumbersome pre-calibration process required for maskers. The proposed system was validated on a large dataset of subjective responses to enhanced soundscapes from more than 440 participants, which ensures the model’s ability to predict the combined effect of the mask and the mask’s gain level on the perceived pleasure level.

Claims (20)

1. Soundscape enhancement system, including: A data acquisition system is configured to provide ambient sound data. The database includes multiple masking settings; A perception attribute predictor, coupled to the data acquisition system and the database, is configured to generate a prediction representing perception on one or more predefined perception attribute metrics for each of the plurality of masking settings, based on the surrounding soundscape data. A masking setting ranking system, configured to determine one or more optimal masking settings based on the predictions generated by the perceptual attribute predictor; and A playback system configured to play or reproduce the one or more optimal masking settings. 2. The soundscape enhancement system according to claim 1, wherein the data acquisition system is further configured to provide crowd statistics or contextual data. 3. The soundscape enhancement system of claim 2, wherein the perceptual attribute predictor is configured to generate the prediction based on the group statistics or contextual data. 4. The soundscape enhancement system according to any one of claims 2 to 3, wherein the group statistics or contextual data includes data related to environmental parameters, group statistics of one or more listeners, psychological data of one or more listeners, physiological data of one or more listeners, or any combination thereof. 5. The soundscape enhancement system according to claim 4, wherein the environmental parameter-related data includes location-related data, visual environment-related data, the number of people nearby, air temperature, humidity, wind speed, or any combination thereof. 6. The soundscape enhancement system according to claim 4 or 5, wherein the group statistics of the one or more listeners include data related to age, gender, occupation, or any combination thereof. 7. The soundscape enhancement system according to any one of claims 4 to 6, wherein the psychological data of the one or more listeners includes data related to noise sensitivity, perceived stress, well-being index scores, or any combination thereof. 8. The soundscape enhancement system according to any one of claims 4 to 7, wherein the physiological data of the one or more listeners includes data related to heart rate, blood pressure, body temperature or any combination thereof. 9. The soundscape enhancement system according to any one of claims 1 to 8, wherein the perceptual attribute predictor is configured to generate predictions based on one or more masking gain inputs. 10. The soundscape enhancement system according to any one of claims 1 to 9, wherein the one or more predefined perceptual attribute indicators include pleasantness, activity, eventfulness, calmness, perceived loudness, sound quality, sharpness, roughness, or any combination thereof. 11. The soundscape enhancement system according to any one of claims 1 to 10, wherein the perceptual attribute predictor is configured to generate the prediction by combining the surrounding soundscape data and each masking setting. 12. The soundscape enhancement system according to any one of claims 1 to 11, The perceptual attribute predictor includes a sound scene feature extractor configured to extract features from the surrounding sound scene data. The perceptual attribute predictor includes a masking feature extractor configured to extract features for each masking setting; and The perceptual attribute predictor is configured to generate the prediction based on the extracted features of the surrounding soundscape data and the extracted features of each masking setting. 13. The soundscape enhancement system according to any one of claims 1 to 12, The perceptual attribute predictor includes objective feature blocks configured to extract objective features from the surrounding acoustic scene data and each masking setting; The perceptual attribute predictor includes a subjective feature block configured to extract subjective features from the group statistics or contextual data; and The perceptual attribute predictor is configured to generate the prediction based on the extracted objective features and the extracted subjective features. 14. The soundscape enhancement system according to any one of claims 1 to 13, wherein the perceptual attribute predictor comprises a linear regression model, the linear regression model being configured to generate the prediction based on acoustic or psychoacoustic parameters calculated based on the surrounding soundscape data and each masking setting. 15. The soundscape enhancement system according to any one of claims 1 to 14, wherein the perceptual attribute predictor comprises one or more deep neural networks configured to generate the prediction based on raw audio, spectrogram rendering calculated based on the surrounding soundscape data and each masking setting, or a combination of the raw audio and the spectrogram rendering. 16. The soundscape enhancement system according to any one of claims 1 to 15, wherein the data acquisition system is configured to generate the surrounding soundscape data based on inputs from one or more microphones, one or more antennas, a storage medium or any combination thereof. 17. The soundscape enhancement system according to any one of claims 1 to 16, wherein the plurality of masking settings comprises one or more recorded or synthesized sound tracks, one or more silent sound tracks, or one or more sound tracks derived from the one or more recorded or synthesized sound tracks and the one or more silent sound tracks. 18. The soundscape enhancement system according to any one of claims 1 to 17, wherein the playback system comprises one or more speakers, one or more virtual reality or augmented reality headsets, one or more headphones, one or more earbuds, or any combination thereof. 19. The soundscape enhancement system according to any one of claims 1 to 18, wherein the generated prediction is uncertain. 20. A method for forming a soundscape enhancement system, the method comprising: A data acquisition system is provided, wherein the data acquisition system is configured to provide ambient sound scene data; Provide a database that includes multiple masking settings; A perception attribute predictor is coupled to the data acquisition system and the database. The perception attribute predictor is configured to generate a prediction representing perception on one or more predefined perception attribute metrics for each of the plurality of masking settings based on the surrounding soundscape data. Provide a masking setup ranking system, the masking setup ranking system being configured to determine one or more optimal masking setups based on the predictions generated by the perceptual attribute predictor; and A playback system is provided, which is configured to play or reproduce the one or more optimal masking settings.