CN116825060B - AI generation music optimization method based on BCI emotion feedback and related device - Google Patents

AI generation music optimization method based on BCI emotion feedback and related device Download PDF

Info

Publication number
CN116825060B
CN116825060B CN202311108995.9A CN202311108995A CN116825060B CN 116825060 B CN116825060 B CN 116825060B CN 202311108995 A CN202311108995 A CN 202311108995A CN 116825060 B CN116825060 B CN 116825060B
Authority
CN
China
Prior art keywords
data
music
emotion
current
preset
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311108995.9A
Other languages
Chinese (zh)
Other versions
CN116825060A (en
Inventor
胡方扬
魏彦兆
唐海波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xiaozhou Technology Co ltd
Original Assignee
Xiaozhou Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xiaozhou Technology Co ltd filed Critical Xiaozhou Technology Co ltd
Priority to CN202311108995.9A priority Critical patent/CN116825060B/en
Publication of CN116825060A publication Critical patent/CN116825060A/en
Application granted granted Critical
Publication of CN116825060B publication Critical patent/CN116825060B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/0008—Associated control or indicating means
    • G10H1/0025—Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
    • G06F18/256—Fusion techniques of classification results, e.g. of results related to same input data of results relating to different input data, e.g. multimodal recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2123/00—Data types
    • G06F2123/02—Data types in the time domain, e.g. time-series data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12—Classification; Matching
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2220/00—Input/output interfacing specifically adapted for electrophonic musical tools or instruments
    • G10H2220/155—User input interfaces for electrophonic musical instruments
    • G10H2220/371—Vital parameter control, i.e. musical instrument control based on body signals, e.g. brainwaves, pulsation, temperature, perspiration; biometric information
    • G10H2220/376—Vital parameter control, i.e. musical instrument control based on body signals, e.g. brainwaves, pulsation, temperature, perspiration; biometric information using brain waves, e.g. EEG
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2250/00—Aspects of algorithms or signal processing methods without intrinsic musical character, yet specifically adapted for or used in electrophonic musical processing
    • G10H2250/311—Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Dermatology (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Neurosurgery (AREA)
  • Neurology (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The application discloses an AI generation music optimization method based on BCI emotion feedback and a related device, wherein through acquiring the electroencephalogram characteristic data of a listener user in the current music environment, the current music environment is the environment in which first AI music data is being played, the current emotion data of the listener user is identified according to the electroencephalogram characteristic data by utilizing a preset emotion classification model, so that the emotion feedback of the listener user when hearing the first AI music data is automatically perceived in real time; generating a music model by using a preset AI, and performing parameter adjustment on the first AI music data based on the current emotion data to obtain second AI music data; and performing music optimization by taking the second AI music data as new first AI music data until the current emotion data accords with the preset emotion data, so as to obtain target AI music data. To automatically adjust the music parameters of the first AI music data, thereby improving the intelligentization and parameter tuning accuracy of the music composing device.

Description

AI generation music optimization method based on BCI emotion feedback and related device
Technical Field
The application relates to the technical field of artificial intelligence, in particular to an AI generated music optimization method based on BCI emotion feedback and a related device.
Background
Music is a smart medicine for smoothing heart wounds and adjusting emotion, so that people have more positive emotion value. In order to enable people to resonate and emotion with music and enable musical compositions to be more infectious and interactive, in the existing music creation process, music artists need to consider emotion experience of listeners.
Currently, traditional musical composition relies on subjective emotional feedback of listeners and subjective artistic understanding of music artists. However, the audience is not necessarily capable of timely and accurately expressing subjective emotion feelings, and cannot accurately perceive emotion of a user when listening to music in real time; the music artist needs to understand the emotion of the audience and then adjust relevant music parameters of the music composing device according to the understanding, but subjective art understanding of the music artist has many differences. It can be seen that the conventional music composition cannot accurately sense the emotion of the user in real time and cannot automatically adjust the parameters of the music composition equipment.
Disclosure of Invention
The application provides an AI generation music optimization method based on BCI emotion feedback, which aims to solve the technical problems that current music creation equipment cannot sense emotion of a user in real time and cannot automatically adjust music parameters.
In order to solve the technical problems, in a first aspect, the present application provides an AI generated music optimization method based on BCI emotion feedback, including:
acquiring electroencephalogram characteristic data of a listener user in a current music environment, wherein the current music environment is an environment in which first AI music data are being played;
identifying current emotion data of the audience user according to the electroencephalogram characteristic data by using a preset emotion classification model;
generating a music model by using a preset AI, and performing parameter adjustment on the first AI music data based on the current emotion data to obtain second AI music data;
and performing music optimization by taking the second AI music data as new first AI music data until the current emotion data accords with preset emotion data, so as to obtain target AI music data.
In some implementations of the first aspect, the acquiring the electroencephalogram feature data of the listener user in the current music environment includes:
playing the first AI music data, and collecting brain electrical signals of a plurality of brain areas of the audience user based on brain electrical detection equipment;
and extracting the electroencephalogram characteristic data of the electroencephalogram signals by utilizing a preset signal characteristic extraction algorithm corresponding to the brain regions for the electroencephalogram signals of each brain region.
In some implementations of the first aspect, the preset emotion classification model includes a plurality of emotion classification networks corresponding to brain regions and a fusion decision network, and the identifying current emotion data of the listener user according to the electroencephalogram feature data by using the preset emotion classification model includes:
for the brain electrical characteristic data of each brain region, identifying regional emotion data represented by the brain region according to the brain electrical characteristic data of the brain region by utilizing an emotion classification network corresponding to the brain region;
and fusing the regional emotion data represented by the brain regions by using the fusion decision network to obtain the current emotion data of the audience user.
In some implementations of the first aspect, the expression of the preset emotion classification model is:；
wherein ,for the current emotion data>For fusing the activation functions of the decision network, +.>For the number of brain areas>Is->Weight matrix of individual brain regions, +.>Indicate->Emotion classification network of individual brain areas, +.>For activating function of emotion classification network, +.>Is->Weight matrix of individual emotion classification networks, +.>Electroencephalogram feature data for brain region, +.>Is->Bias vector of individual emotion classification network, +.>To fuse the bias vectors of the decision network.
In some implementations of the first aspect, the generating a music model using a preset AI, and performing a parameter tuning on the first AI music data based on the current emotion data to obtain second AI music data, includes:
inputting the current emotion data and the first AI music data into the preset AI to generate a music model, and calculating an error value between the current emotion data and the preset emotion data;
and adjusting the music parameters of the first AI music data based on the error value to obtain second AI music data.
In some implementations of the first aspect, the calculating the error value between the current emotion data and preset emotion data includes:
calculating an error value between the current emotion data and preset emotion data by using a preset error function, wherein the expression of the preset error function is as follows:；
wherein ,representing an error value between the current emotion data and the preset emotion data,/a>Represents a maximum function>Representing current emotion data, < >>And representing preset emotion data.
In some implementations of the first aspect, the performing music optimization with the second AI music data as new first AI music data until the current emotion data accords with the preset emotion data, to obtain target AI music data includes:
the second AI music data is used as new first AI music data, and emotion data of the audience user in the current AI music environment is identified;
if the emotion data of the audience user in the current music environment accords with preset emotion data, determining that the second AI music data is target music data;
if the emotion data of the audience user in the current music environment does not accord with the preset emotion data, continuing to optimize the new first AI music data in a gradient descent mode until the emotion data of the audience user in the current music environment accords with the preset emotion data.
In a second aspect, the present application further provides an AI generated music optimizing apparatus based on BCI emotion feedback, including:
the acquisition module is used for acquiring the electroencephalogram characteristic data of the audience user in the current music environment, wherein the current music environment is an environment in which the first AI music data is being played;
the identification module is used for identifying the current emotion data of the audience user according to the electroencephalogram characteristic data by utilizing a preset emotion classification model;
the parameter adjusting module is used for generating a music model by utilizing a preset AI, and adjusting parameters of the first AI music data based on the current emotion data to obtain second AI music data;
and the optimizing module is used for performing music optimization by taking the second AI music data as new first AI music data until the current emotion data accords with preset emotion data, so as to obtain target AI music data.
In a third aspect, the present application also provides a computer device, including a processor and a memory, where the memory is configured to store a computer program, where the computer program, when executed by the processor, implements the AI-generated music optimization method based on BCI emotion feedback according to the first aspect.
In a fourth aspect, the present application also provides a computer readable storage medium storing a computer program, which when executed by a processor implements the AI-generated music optimization method based on BCI emotion feedback as described in the first aspect.
Compared with the prior art, the application has at least the following beneficial effects:
acquiring the current emotion data of a listener user in a current music environment, wherein the current music environment is an environment in which first AI music data is being played, and identifying the current emotion data of the listener user according to the electroencephalogram feature data by utilizing a preset emotion classification model so as to automatically sense emotion feedback of the listener user when hearing the first AI music data in real time; generating a music model by using a preset AI, and performing parameter adjustment on the first AI music data based on the current emotion data to obtain second AI music data so as to automatically adjust music parameters of the first AI music data, so that an artist does not need to manually adjust the music parameters of the music creation equipment about the first AI music data based on experience understanding, and the intelligentization and parameter adjustment accuracy of the music creation equipment are improved; and performing music optimization by taking the second AI music data as new first AI music data until the current emotion data accords with preset emotion data to obtain target AI music data, thereby realizing automatic parameter tuning.
Drawings
FIG. 1 is a schematic flow chart of an AI music generation optimizing method based on BCI emotion feedback according to an embodiment of the application;
fig. 2 is a schematic structural diagram of an AI generated music optimizing apparatus based on BCI emotion feedback according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Referring to fig. 1, fig. 1 is a flow chart of an AI music generation optimizing method based on BCI emotion feedback according to an embodiment of the present application. The AI generated music optimizing method based on BCI emotion feedback can be applied to computer equipment with music creation function, wherein the computer equipment comprises but is not limited to intelligent mobile phones, notebook computers, tablet computers, desktop computers, physical servers, cloud servers and the like. As shown in fig. 1, the AI generated music optimizing method based on BCI emotion feedback of the present embodiment includes steps S101 to S104, which are described in detail as follows:
step S101, acquiring electroencephalogram characteristic data of a listener user in a current music environment, wherein the current music environment is an environment in which first AI music data is being played.
In this step, first AI music data, which may be AI music generated randomly, is played in an environment where a listener user is located, and an electroencephalogram detection device is used to detect the listener user, record an electroencephalogram signal thereof, and then perform feature extraction on the electroencephalogram signal to obtain electroencephalogram feature data.
In some embodiments, the step S101 includes:
playing the first AI music data, and collecting brain electrical signals of a plurality of brain areas of the audience user based on brain electrical detection equipment;
and extracting the electroencephalogram characteristic data of the electroencephalogram signals by utilizing a preset signal characteristic extraction algorithm corresponding to the brain regions for the electroencephalogram signals of each brain region.
In this embodiment, a head-mounted detectable emotion-related electroencephalogram signal brain-computer interface (Brain Computer Interface, BCI) headband is provided for a listener, which headband covers the frontal lobe, temporal lobe, etc., and is attached with an electrode network for receiving and recording electroencephalogram signals corresponding to brain regions such as amygdala and frontal lobe regions, inner frontal lobe cortex, and brain island regions. For example, the severity of depression is associated with reduced connection of the amygdala to frontal lobe areas (including dorsolateral prefrontal cortex and anterior cingulate cortex); the medial prefrontal cortex, amygdala, and brain islands of mood disorder patients all exhibit abnormal nerve feedback, and the dorsally lateral prefrontal cortex exhibits weakness in nerve feedback.
Optionally, preprocessing is performed on the acquired electroencephalogram signals, including but not limited to preprocessing operations such as noise removal, filtering, downsampling, and the like, to help improve accuracy of subsequent processing. Meaningful electroencephalogram feature data is extracted from the preprocessed electroencephalogram signals, and feature extraction includes, but is not limited to, time domain features (e.g., mean, variance), frequency domain features (e.g., power spectral density), and time-frequency domain features (e.g., wavelet transform coefficients).
Alternatively, feature selection or dimension reduction methods, such as Principal Component Analysis (PCA) or Linear Discriminant Analysis (LDA), are used to reduce feature dimensions and improve classification performance.
Step S102, identifying the current emotion data of the audience user according to the electroencephalogram characteristic data by utilizing a preset emotion classification model.
In this step, the preset emotion classification model may be a multi-classification model based on a deep learning algorithm. Deep learning can learn features through a deep nonlinear network structure, and form a more abstract deep representation (attribute category or feature) by combining low-level features to realize complex function approximation, so that the essential features of a data set can be learned. Illustratively, a plurality of nodes are used in the input layer, and the brain electrical signal or the like of the brain region is used as an input vector X1 … … … Xn (feature). Each node realizes nonlinear transformation through an activation function and then inputs the nonlinear transformation, and can output the nonlinear transformation through a plurality of hidden layers; the neural network output is compared to the actual or desired output. According to the magnitude of the error, a back propagation algorithm (back-propagation algorithm, BP algorithm) is adopted to change the w value and the b value of the connection weight, and the loss function is continuously iterated and minimized through the model, so that each predicted result value and the actual result value are more and more approximate, and the loss function value is not changed any more, for example, until the error reaches approximately 0.001. For multi-classification problems, a Softmax loss function may be employed. Finally, after the training phase is over, the weight value is fixed to the final value. And analyzing and processing the brain electrical characteristic data through the model, and judging the current emotion feedback of the audience.
In some embodiments, the preset emotion classification model includes a plurality of emotion classification networks corresponding to brain regions and a fusion decision network, and the step S102 includes:
for the brain electrical characteristic data of each brain region, identifying regional emotion data represented by the brain region according to the brain electrical characteristic data of the brain region by utilizing an emotion classification network corresponding to the brain region;
and fusing the regional emotion data represented by the brain regions by using the fusion decision network to obtain the current emotion data of the audience user.
In this embodiment, for example, the classification processes of the brain region features are unified into a complete CNN fusion decision network, which fuses the CNN emotion classification network of the single brain region features, and the emotion classification network is a CNN model composed of a series of convolution, pooling and ReLU activation layers. The network fusion can adopt a network fusion method based on a Point-switch Boltzmann machine (Point-wise Gated Boltzmann Machine, PGBM): splicing the last layer of feature vectors of the CNN emotion classification networks, training the spliced feature vectors serving as input of a visible layer of the PGBM part, and training the PGBM part by adopting a contrast divergence method. And obtaining the characteristic representation of the task related part in the spliced characteristic vector through the network connection weight obtained after training, wherein the part of characteristic representation is used as the input of the newly added full-connection layer, and training the newly added full-connection layer.
In some embodiments, the expression of the preset emotion classification model is:；
wherein ,for the current emotion data>For fusing the activation functions of the decision network, +.>For the number of brain areas>Is->Weight matrix of individual brain regions, +.>Indicate->Emotion classification network of individual brain areas, +.>For activating function of emotion classification network, +.>Is->Weight matrix of individual emotion classification networks, +.>Electroencephalogram feature data for brain region, +.>Is->Bias vector of individual emotion classification network, +.>To fuse the bias vectors of the decision network.
In this embodiment, the electroencephalogram feature data of a plurality of brain regions are respectively used as input of the corresponding emotion classification network, so as to obtain corresponding classification results, and then the fusion decision network is used for carrying out fusion decision on the plurality of classification results, so as to obtain a final emotion classification result, and the influence of each brain region on the emotion classification result can be fully considered, so that the accuracy of emotion classification is improved.
Step S103, a music model is generated by utilizing a preset AI, and the first AI music data is subjected to parameter adjustment based on the current emotion data to obtain second AI music data.
In this step, the music parameters are adjusted according to the output of the emotion model, and parameters such as pitch, timbre, rhythm, sound intensity and the like can also be adjusted according to the preference and emotion feedback of the user, so that the music meets the emotion requirement of the user. For example, the emotion recognition result is fed back to a preset AI to generate a music model, and if the emotion of the listener is negative emotion, the model automatically adjusts the music melody generated by the AI music data and converts the music melody into music with a more positive or lyric style so as to achieve the emotion optimization effect.
Alternatively, a music model is generated for a preset AI, a marked emotion data sample is collected, and music characteristics associated therewith, such as tempo, chord progression, timbre, etc., are recorded. These data are trained using machine learning algorithms to construct a model that correlates emotion classifications with musical features. Based on the emotion classification model and the music generation algorithm, the model can automatically generate a music work conforming to the emotion according to the current emotion state of the user. Music style, tune, tempo, etc. may be selected and adjusted based on the association of emotion classifications with music features. According to the emotion change of the user, the model can generate and adjust the musical compositions in real time so as to adapt to different emotion states. When the emotion of the user changes, the model can be automatically switched to the corresponding music style and characteristics so as to provide a music experience which is more in line with the current emotion of the user. The user may provide feedback, such as likes or dislikes, to the generated musical composition. The model can be adjusted and improved according to feedback of the user so as to improve the accuracy of music generation and the degree of conforming to the emotion of the user.
Through AI music generation technology based on user emotion, users can enjoy music experience matched with the emotion states of users, which helps to promote emotion adjustment, relaxation and entertainment effects. Meanwhile, a way for creating personalized and emotional musical compositions is provided for music creators, and different users can hear the music versions which are more in line with the preference of the users in real time.
In some embodiments, the step S103 includes:
inputting the current emotion data and the first AI music data into the preset AI to generate a music model, and calculating an error value between the current emotion data and the preset emotion data;
and adjusting the music parameters of the first AI music data based on the error value to obtain second AI music data.
In this embodiment, the music parameters of the first AI music data are adjusted according to the error value between the current emotion data and the preset emotion data, so that the preset emotion data can be referred to, and the adjusted first AI music data can be closer to the user authoring requirement. Optionally, the music parameter of the first AI music data is adjusted according to the error value in a gradient descent manner.
In some embodiments, the error value between the current emotion data and the preset emotion data is calculated using a preset error function, where an expression of the preset error function is:；
wherein ,representing an error value between the current emotion data and the preset emotion data,/a>Represents a maximum function>Representing current emotion data, < >>And representing preset emotion data.
In this embodiment, the maximum difference between the current emotion data and the preset emotion data is determined by using the maximum function as an error value, so that the interval between the current emotion data and the preset emotion data can be maximized, and parameter adjustment accuracy is improved.
And step S104, performing music optimization by taking the second AI music data as new first AI music data until the current emotion data accords with preset emotion data, and obtaining target AI music data.
In the step, the second AI music data is taken as new first AI music data, and emotion data of the audience user in the current AI music environment is identified; if the emotion data of the audience user in the current music environment accords with preset emotion data, determining that the second AI music data is target music data; if the emotion data of the audience user in the current music environment does not accord with the preset emotion data, continuing to optimize the new first AI music data in a gradient descent mode until the emotion data of the audience user in the current music environment accords with the preset emotion data.
Optionally, if the emotion data in the current music environment does not accord with the preset emotion data, updating the first generation of music data, entering a new round of circulation, and defining an iteration round number function:where N represents the maximum number of iteration rounds allowed; e represents the error between the current emotion value and the target emotion value, i.e. the error function +.>Is a value of (2); k represents the current iteration round number passing through; f represents the error as a function of the number of iteration rounds, < >>Wherein a, b, c, d are empirical coefficients. I.e. N will dynamically adjust as the current error E and iteration number k change. The larger the error E, the more the maximum number of rounds N allowed, and the more the number of iterative rounds k, the fewer the maximum number of rounds N allowed.
Further, after a plurality of iterative optimization loops, when the emotion of the listener still does not reach the expected target, triggering a dual-confirmation exit mechanism, specifically:
the Euclidean distance d between the current wheel parameter vector P and the upper wheel parameter vector P' is calculated: the parameter vectors P and P 'represent the parameter values of two iterations, the dimensions are the same, and if n parameters exist, P and P' can be expressed as:，the method comprises the steps of carrying out a first treatment on the surface of the Euclidean distance between P and P, wherein ,/>Representing the calculated square root. It will be appreciated that a smaller d means a smaller change in the two-pass parameter, a threshold distance dt=0.05 is set, if d<dt, which means that the optimization tends to be smooth and convergent, the parameters have approached the optimal region, and continuing the iteration means that the computing resources are wasted and ready to exit.
Measuring the distance between the current emotion value v and the target emotion value vDv provides a quantitative indicator that determines whether the current music is capable of achieving the intended affective effect. If dv is less than threshold Et, et may take a value of 0.1, indicating that the current music is emotionally substantially satisfactory. Although it may be possible to continue the optimization slightly, the improvement is very small, and it may be considered that the optimum has been approached, the exit is confirmed, and the parameter Pn of the last round is taken as the parameter of the output target music, so that unnecessary calculation is avoided, and the music generation efficiency is improved.
In this embodiment, the generation of AI music data is continuously improved by using the forward motivation of the emotion of the listener in a closed-loop feedback manner, so as to achieve a more accurate and more personalized response to the emotion of different listeners.
In order to execute the AI generation music optimization method based on BCI emotion feedback corresponding to the method embodiment, corresponding functions and technical effects are realized. Referring to fig. 2, fig. 2 shows a block diagram of an AI generated music optimizing apparatus based on BCI emotion feedback according to an embodiment of the present application. For convenience of explanation, only the parts related to this embodiment are shown, and the AI generated music optimizing apparatus based on BCI emotion feedback provided in the embodiment of the present application includes:
an obtaining module 201, configured to obtain electroencephalogram feature data of a listener user in a current music environment, where the current music environment is an environment in which first AI music data is being played;
the identifying module 202 is configured to identify current emotion data of the audience user according to the electroencephalogram feature data by using a preset emotion classification model;
the parameter tuning module 203 is configured to generate a music model by using a preset AI, and tune the first AI music data based on the current emotion data to obtain second AI music data;
and an optimizing module 204, configured to perform music optimization with the second AI music data as new first AI music data until the current emotion data accords with the preset emotion data, so as to obtain target AI music data.
In some embodiments, the obtaining module 201 is specifically configured to:
playing the first AI music data, and collecting brain electrical signals of a plurality of brain areas of the audience user based on brain electrical detection equipment;
and extracting the electroencephalogram characteristic data of the electroencephalogram signals by utilizing a preset signal characteristic extraction algorithm corresponding to the brain regions for the electroencephalogram signals of each brain region.
In some embodiments, the preset emotion classification model includes a plurality of emotion classification networks corresponding to brain regions and a fusion decision network, and the identification module 202 is specifically configured to:
for the brain electrical characteristic data of each brain region, identifying regional emotion data represented by the brain region according to the brain electrical characteristic data of the brain region by utilizing an emotion classification network corresponding to the brain region;
and fusing the regional emotion data represented by the brain regions by using the fusion decision network to obtain the current emotion data of the audience user.
In some embodiments, the expression of the preset emotion classification model is:；
wherein ,for the current emotion data>For fusing the activation functions of the decision network, +.>For the number of brain areas>Is->Weight matrix of individual brain regions, +.>Indicate->Emotion classification network of individual brain areas, +.>For activating function of emotion classification network, +.>Is->Weight matrix of individual emotion classification networks, +.>Electroencephalogram feature data for brain region, +.>Is->Bias vector of individual emotion classification network, +.>To fuse the bias vectors of the decision network.
In some embodiments, the parameter tuning module 203 includes:
the computing unit is used for inputting the current emotion data and the first AI music data into the preset AI generation music model and computing an error value between the current emotion data and the preset emotion data;
and the adjusting unit is used for adjusting the music parameters of the first AI music data based on the error value to obtain second AI music data.
In some embodiments, the computing unit is specifically configured to:
calculating an error value between the current emotion data and preset emotion data by using a preset error function, wherein the expression of the preset error function is as follows:；
wherein ,representing an error value between the current emotion data and the preset emotion data,/a>Represents a maximum function>Representing current emotion data, < >>And representing preset emotion data.
In some embodiments, the optimizing module 204 is specifically configured to:
the second AI music data is used as new first AI music data, and emotion data of the audience user in the current AI music environment is identified;
if the emotion data of the audience user in the current music environment accords with preset emotion data, determining that the second AI music data is target music data;
if the emotion data of the audience user in the current music environment does not accord with the preset emotion data, continuing to optimize the new first AI music data in a gradient descent mode until the emotion data of the audience user in the current music environment accords with the preset emotion data.
The above-mentioned AI generation music optimization device based on BCI emotion feedback may implement the AI generation music optimization method based on BCI emotion feedback of the above-mentioned method embodiment. The options in the method embodiments described above are also applicable to this embodiment and will not be described in detail here. The rest of the embodiments of the present application may refer to the content of the above method embodiments, and in this embodiment, no further description is given.
Fig. 3 is a schematic structural diagram of a computer device according to an embodiment of the present application. As shown in fig. 3, the computer device 3 of this embodiment includes: at least one processor 30 (only one shown in fig. 3), a memory 31 and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, the processor 30 implementing the steps in any of the method embodiments described above when executing the computer program 32.
The computer device 3 may be a smart phone, a tablet computer, a desktop computer, a cloud server, or other computing devices. The computer device may include, but is not limited to, a processor 30, a memory 31. It will be appreciated by those skilled in the art that fig. 3 is merely an example of the computer device 3 and is not meant to be limiting as the computer device 3, and may include more or fewer components than shown, or may combine certain components, or different components, such as may also include input-output devices, network access devices, etc.
The processor 30 may be a central processing unit (Central Processing Unit, CPU), the processor 30 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 31 may in some embodiments be an internal storage unit of the computer device 3, such as a hard disk or a memory of the computer device 3. The memory 31 may in other embodiments also be an external storage device of the computer device 3, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the computer device 3. The memory 31 is used for storing an operation model, an application program, a BootLoader (BootLoader), data, and other programs, etc., such as program codes of the computer program. The memory 31 may also be used for temporarily storing data that has been output or is to be output.
In addition, the embodiment of the present application further provides a computer readable storage medium, where a computer program is stored, where the computer program is executed by a processor to implement the steps in any of the above-mentioned method embodiments.
Embodiments of the present application provide a computer program product which, when run on a computer device, causes the computer device to perform the steps of the method embodiments described above.
In several embodiments provided by the present application, it will be understood that each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing embodiments have been provided for the purpose of illustrating the general principles of the present application, and are not to be construed as limiting the scope of the application. It should be noted that any modifications, equivalent substitutions, improvements, etc. made by those skilled in the art without departing from the spirit and principles of the present application are intended to be included in the scope of the present application.

Claims (8)

1. An AI generated music optimizing method based on BCI emotion feedback is characterized by comprising the following steps:
acquiring electroencephalogram characteristic data of a listener user in a current music environment, wherein the current music environment is an environment in which first AI music data are being played;
identifying current emotion data of the audience user according to the electroencephalogram characteristic data by using a preset emotion classification model;
generating a music model by using a preset AI, and performing parameter adjustment on the first AI music data based on the current emotion data to obtain second AI music data;
performing music optimization by taking the second AI music data as new first AI music data until the current emotion data accords with preset emotion data to obtain target AI music data;
the preset emotion classification model comprises a plurality of emotion classification networks corresponding to brain areas and a fusion decision network, and the current emotion data of the audience user is identified according to the electroencephalogram characteristic data by utilizing the preset emotion classification model, and the method comprises the following steps:
for the brain electrical characteristic data of each brain region, identifying regional emotion data represented by the brain region according to the brain electrical characteristic data of the brain region by utilizing an emotion classification network corresponding to the brain region;
utilizing the fusion decision network to fuse regional emotion data represented by a plurality of brain regions to obtain current emotion data of the audience user;
the expression of the preset emotion classification model is as follows:，
wherein ,for the current emotion data>For fusing the activation functions of the decision network, +.>For the number of brain regions,is->Weight matrix of individual brain regions, +.>Indicate->Emotion classification of individual brain regionsNetwork (S)>For activating function of emotion classification network, +.>Is->Weight matrix of individual emotion classification networks, +.>Electroencephalogram feature data for brain region, +.>Is->Bias vector of individual emotion classification network, +.>To fuse the bias vectors of the decision network.
2. The AI-generated music optimization method based on BCI emotion feedback of claim 1, wherein said obtaining the electroencephalogram feature data of the listener user in the current music environment comprises:
playing the first AI music data, and collecting brain electrical signals of a plurality of brain areas of the audience user based on brain electrical detection equipment;
and extracting the electroencephalogram characteristic data of the electroencephalogram signals by utilizing a preset signal characteristic extraction algorithm corresponding to the brain regions for the electroencephalogram signals of each brain region.
3. The method for optimizing AI generation music based on BCI emotion feedback of claim 1, wherein generating a music model using a preset AI, and performing a parameter tuning on the first AI music data based on the current emotion data to obtain second AI music data, comprises:
inputting the current emotion data and the first AI music data into the preset AI to generate a music model, and calculating an error value between the current emotion data and the preset emotion data;
and adjusting the music parameters of the first AI music data based on the error value to obtain second AI music data.
4. The AI-generated music optimization method based on BCI emotion feedback of claim 3, wherein said calculating an error value between said current emotion data and preset emotion data comprises:
calculating an error value between the current emotion data and preset emotion data by using a preset error function, wherein the expression of the preset error function is as follows:；
wherein ,representing an error value between the current emotion data and the preset emotion data,/a>Represents a maximum function>Representing current emotion data, < >>And representing preset emotion data.
5. The method for optimizing AI generated music based on BCI emotion feedback of claim 1, wherein said performing music optimization with said second AI music data as new first AI music data until the current emotion data matches the preset emotion data, comprises:
the second AI music data is used as new first AI music data, and emotion data of the audience user in the current AI music environment is identified;
if the emotion data of the audience user in the current music environment accords with preset emotion data, determining that the second AI music data is target music data;
if the emotion data of the audience user in the current music environment does not accord with the preset emotion data, continuing to optimize the new first AI music data in a gradient descent mode until the emotion data of the audience user in the current music environment accords with the preset emotion data.
6. An AI-generated music optimizing apparatus based on BCI emotion feedback, comprising:
the acquisition module is used for acquiring the electroencephalogram characteristic data of the audience user in the current music environment, wherein the current music environment is an environment in which the first AI music data is being played;
the identification module is used for identifying the current emotion data of the audience user according to the electroencephalogram characteristic data by utilizing a preset emotion classification model;
the parameter adjusting module is used for generating a music model by utilizing a preset AI, and adjusting parameters of the first AI music data based on the current emotion data to obtain second AI music data;
the optimizing module is used for performing music optimization by taking the second AI music data as new first AI music data until the current emotion data accords with preset emotion data to obtain target AI music data;
the preset emotion classification model comprises a plurality of emotion classification networks corresponding to brain areas and a fusion decision network, and the identification module is specifically used for:
for the brain electrical characteristic data of each brain region, identifying regional emotion data represented by the brain region according to the brain electrical characteristic data of the brain region by utilizing an emotion classification network corresponding to the brain region;
utilizing the fusion decision network to fuse regional emotion data represented by a plurality of brain regions to obtain current emotion data of the audience user;
the expression of the preset emotion classification model is as follows:，
wherein ,for the current emotion data>For fusing the activation functions of the decision network, +.>For the number of brain regions,is->Weight matrix of individual brain regions, +.>Indicate->Emotion classification network of individual brain areas, +.>For activating function of emotion classification network, +.>Is->Weight matrix of individual emotion classification networks, +.>Electroencephalogram feature data for brain region, +.>Is->Bias vector of individual emotion classification network, +.>To fuse the bias vectors of the decision network.
7. A computer device comprising a processor and a memory for storing a computer program which when executed by the processor implements the BCI emotion feedback based AI generation music optimization method of any of claims 1-5.
8. A computer readable storage medium, characterized in that it stores a computer program which, when executed by a processor, implements the BCI emotion feedback based AI music generation optimization method of any one of claims 1 to 5.
CN202311108995.9A 2023-08-31 2023-08-31 AI generation music optimization method based on BCI emotion feedback and related device Active CN116825060B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311108995.9A CN116825060B (en) 2023-08-31 2023-08-31 AI generation music optimization method based on BCI emotion feedback and related device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311108995.9A CN116825060B (en) 2023-08-31 2023-08-31 AI generation music optimization method based on BCI emotion feedback and related device

Publications (2)

Publication Number Publication Date
CN116825060A CN116825060A (en) 2023-09-29
CN116825060B true CN116825060B (en) 2023-10-27

Family

ID=88127861

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311108995.9A Active CN116825060B (en) 2023-08-31 2023-08-31 AI generation music optimization method based on BCI emotion feedback and related device

Country Status (1)

Country Link
CN (1) CN116825060B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112754502A (en) * 2021-01-12 2021-05-07 曲阜师范大学 Automatic music switching method based on electroencephalogram signals
KR102275436B1 (en) * 2020-01-20 2021-07-09 인하대학교 산학협력단 Fusion technology of image and eeg signal for real-time emotion recognition
CN114492513A (en) * 2021-07-15 2022-05-13 电子科技大学 Electroencephalogram emotion recognition method for adaptation to immunity domain based on attention mechanism in cross-user scene
CN115969377A (en) * 2023-01-09 2023-04-18 东南大学 Emotional guidance system for brain wave music generation based on LongFormer
CN116013228A (en) * 2022-12-28 2023-04-25 深圳大学 Music generation method and device, electronic equipment and storage medium thereof

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102275436B1 (en) * 2020-01-20 2021-07-09 인하대학교 산학협력단 Fusion technology of image and eeg signal for real-time emotion recognition
CN112754502A (en) * 2021-01-12 2021-05-07 曲阜师范大学 Automatic music switching method based on electroencephalogram signals
CN114492513A (en) * 2021-07-15 2022-05-13 电子科技大学 Electroencephalogram emotion recognition method for adaptation to immunity domain based on attention mechanism in cross-user scene
CN116013228A (en) * 2022-12-28 2023-04-25 深圳大学 Music generation method and device, electronic equipment and storage medium thereof
CN115969377A (en) * 2023-01-09 2023-04-18 东南大学 Emotional guidance system for brain wave music generation based on LongFormer

Also Published As

Publication number Publication date
CN116825060A (en) 2023-09-29

Similar Documents

Publication Publication Date Title
JP7137639B2 (en) SOUND QUALITY CHARACTERISTICS PROCESSING METHOD AND SYSTEM BASED ON DEEP LEARNING
Costa et al. An evaluation of convolutional neural networks for music classification using spectrograms
Sarkar et al. Recognition of emotion in music based on deep convolutional neural network
Cartwright et al. Social-EQ: Crowdsourcing an Equalization Descriptor Map.
US20230113072A1 (en) Method, system, and medium for affective music recommendation and composition
Deng et al. Dynamic time warping for music retrieval using time series modeling of musical emotions
KR101891778B1 (en) Method and Apparatus for determining highlight section of a sound source, and computer program for executing the method
Kabani et al. Emotion based music player
CN102004785A (en) Method and device for recommending music
US11314970B1 (en) Reinforcement learning techniques for automated video summarization
CN113813609A (en) Game music style classification method and device, readable medium and electronic equipment
CN109147816A (en) The method and apparatus of volume adjustment is carried out to music
Patel et al. A comparative study of music recommendation systems
CN110853606A (en) Sound effect configuration method and device and computer readable storage medium
Wang et al. Finetuning convolutional neural networks for visual aesthetics
Stasis et al. A model for adaptive reduced-dimensionality equalisation
CN116825060B (en) AI generation music optimization method based on BCI emotion feedback and related device
Cunha Sergio et al. Emotional Video to Audio Transformation Using Deep Recurrent Neural Networks and a Neuro‐Fuzzy System
CN113051425B (en) Method for acquiring audio characterization extraction model and method for recommending audio
Geroulanos et al. Emotion Recognition in Music Using Deep Neural Networks
KR20230148166A (en) Apparatus and method for modifying a user&#39;s emotional state
Blume et al. Huge music archives on mobile devices
Han et al. Audio feature extraction for effective emotion classification
Miklanek et al. Neural grey-box guitar amplifier modelling with limited data
Namgyal et al. What you hear is what you see: Audio quality metrics from image quality metrics

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant