CN114283769A - Method, device and equipment for adjusting accompaniment and storage medium - Google Patents

Method, device and equipment for adjusting accompaniment and storage medium Download PDF

Info

Publication number
CN114283769A
CN114283769A CN202111595166.9A CN202111595166A CN114283769A CN 114283769 A CN114283769 A CN 114283769A CN 202111595166 A CN202111595166 A CN 202111595166A CN 114283769 A CN114283769 A CN 114283769A
Authority
CN
China
Prior art keywords
accompaniment
information
singing
characteristic
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111595166.9A
Other languages
Chinese (zh)
Inventor
蒋亚军
张超
唐玏
张卓鹏
张聪
陈聪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Mobile Communications Group Co Ltd
MIGU Music Co Ltd
MIGU Culture Technology Co Ltd
Original Assignee
China Mobile Communications Group Co Ltd
MIGU Music Co Ltd
MIGU Culture Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Mobile Communications Group Co Ltd, MIGU Music Co Ltd, MIGU Culture Technology Co Ltd filed Critical China Mobile Communications Group Co Ltd
Priority to CN202111595166.9A priority Critical patent/CN114283769A/en
Publication of CN114283769A publication Critical patent/CN114283769A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Reverberation, Karaoke And Other Acoustics (AREA)

Abstract

The invention belongs to the technical field of audio processing, and discloses an accompaniment adjusting method, an accompaniment adjusting device, accompaniment adjusting equipment and a storage medium. The method comprises the following steps: when a user sings, determining the segment number of the current accompaniment; acquiring accompaniment characteristic information of the current accompaniment, singing characteristic information of the user and historical singing characteristic information; obtaining accompaniment adjusting information according to the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user and the historical singing characteristic information; and adjusting the clip accompaniment information of the next segment according to the accompaniment adjustment information and the clip number. Through the above mode, the accompaniment adjustment information can be obtained according to the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user and the historical singing characteristic information, so that the segment accompaniment information of the next segment singed by the user is adjusted in real time, the real-time adjustment of the accompaniment is realized, the adjusted accompaniment is more suitable for the user, and the singing effect of the user is improved.

Description

Method, device and equipment for adjusting accompaniment and storage medium
Technical Field
The present invention relates to the field of audio processing technologies, and in particular, to an accompaniment adjustment method, device, apparatus, and storage medium.
Background
The current accompaniment adjusting method generally adopts a mode of modifying the sound after the singing of the user is finished, can basically ensure the accuracy of the intonation and the volume, but has the condition of tone distortion, most importantly, the timeliness is poor, and the user experience is poor due to the fact that the audio processing needs to be waited.
The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.
Disclosure of Invention
The invention mainly aims to provide an accompaniment adjusting method, an accompaniment adjusting device and a storage medium, and aims to solve the technical problem that in the prior art, the accompaniment cannot be adjusted in real time, so that the user experience is poor.
To achieve the above object, the present invention provides an accompaniment adjusting method, comprising the steps of:
when a user sings, determining the segment number of the current accompaniment;
acquiring accompaniment characteristic information of the current accompaniment, singing characteristic information of the user and historical singing characteristic information;
obtaining accompaniment adjusting information according to the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user and the historical singing characteristic information;
and adjusting the clip accompaniment information of the next segment according to the accompaniment adjustment information and the clip number.
Optionally, before determining the segment number of the current accompaniment, the method further includes:
acquiring accompaniment beat information of the current accompaniment;
segmenting the current accompaniment according to the accompaniment tempo information to obtain a plurality of segmented accompaniments;
and numbering the accompaniment of the plurality of segments to obtain segment numbers.
Optionally, the obtaining accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the singing feature information of the user and the historical singing feature information includes:
obtaining accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the segment accompaniment information of the next segment and a pre-trained accompaniment adjustment model.
Optionally, the training process of the accompaniment adjustment model includes:
obtaining an input feature set according to the accompaniment feature information, the singing feature information and the historical singing feature information;
constructing a multivariate function fitting model, and taking the multivariate function fitting model as a model to be trained;
and training the model to be trained according to the input feature set to obtain an accompaniment adjustment model.
Optionally, the obtaining an input feature set according to the accompaniment feature information, the singing feature information and the historical singing feature information includes:
obtaining a segmented singing score, a segmented loudness characteristic, a segmented tone characteristic and a segmented tone characteristic of each segmented accompaniment according to the accompaniment characteristic information;
obtaining singing loudness characteristics, singing tone characteristics and singing tone characteristics of the segmented accompaniments singed by the user according to the singing characteristic information;
obtaining the historical singing loudness characteristic, the historical singing tone characteristic and the historical singing tone characteristic of the current accompaniment sung performed by the user in the historical manner according to the historical singing characteristic information;
and taking the segmented singing score, the segmented loudness characteristic, the segmented tone characteristic, the singing loudness characteristic, the singing tone characteristic, the historical singing loudness characteristic, the historical singing tone characteristic and the historical singing tone characteristic as an input feature set.
Optionally, the obtaining accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the segment accompaniment information of the next segment, and a pre-trained accompaniment adjustment model includes:
inputting the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user and the historical singing characteristic information into the accompaniment adjusting model to obtain a loudness sequencing result, a tone sequencing result and a tone sequencing result;
determining the optimal loudness matching degree according to the loudness sequencing result;
determining the optimal tone matching degree according to the tone sorting result;
determining the optimal tone matching degree according to the tone sorting result;
and obtaining accompaniment adjusting information according to the optimal loudness matching degree, the optimal tone matching degree and the next segmented clip accompaniment information.
Optionally, before determining the segment number of the current accompaniment, the method further includes:
when the user starts playing the initial accompaniment, determining the beginning accompaniment according to the segment number of the initial accompaniment;
obtaining starting accompaniment feature information according to the starting segment accompaniment;
inputting the historical singing characteristic information into an accompaniment regulation model to obtain beginning segment regulation information;
and adjusting the characteristic information of the beginning accompaniment according to the adjusting information of the beginning fragment to obtain the current accompaniment.
In order to achieve the above object, the present invention further provides an accompaniment adjusting device, comprising:
the determining module is used for determining the segment number of the current accompaniment when the user sings;
the obtaining module is used for obtaining accompaniment characteristic information of the current accompaniment, singing characteristic information of the user and historical singing characteristic information;
the processing module is used for obtaining accompaniment adjusting information according to the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user and the historical singing characteristic information;
and the adjusting module is used for adjusting the clip accompaniment information of the next segment according to the accompaniment adjusting information and the clip number.
Further, to achieve the above object, the present invention also proposes an accompaniment adjusting apparatus including: the accompanying program is configured to implement the steps of the accompanying adjustment method as described above.
In addition, to achieve the above object, the present invention further provides a storage medium having an accompaniment adjustment program stored thereon, wherein the accompaniment adjustment program, when executed by a processor, implements the steps of the accompaniment adjustment method as described above.
When a user sings, determining the segment number of the current accompaniment; acquiring accompaniment characteristic information of the current accompaniment, singing characteristic information of the user and historical singing characteristic information; obtaining accompaniment adjusting information according to the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user and the historical singing characteristic information; and adjusting the clip accompaniment information of the next segment according to the accompaniment adjustment information and the clip number. Through this kind of mode, can obtain the accompaniment adjustment information according to the accompaniment characteristic information of current accompaniment, the singing characteristic information and the historical singing characteristic information of user to the next segmented segment accompaniment information of user singing is adjusted in real time, has realized adjusting the accompaniment in real time, and then makes the accompaniment after the adjustment more suitable user, improves user's singing effect.
Drawings
Fig. 1 is a schematic structural diagram of an accompaniment adjustment apparatus for a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating an accompaniment adjustment method according to a first embodiment of the present invention;
FIG. 3 is a flowchart illustrating an accompaniment adjustment method according to a second embodiment of the present invention;
FIG. 4 is a flowchart illustrating a method for adjusting accompaniment in accordance with a third embodiment of the present invention;
fig. 5 is a block diagram of an accompaniment adjustment apparatus according to a first embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of an accompaniment adjusting apparatus in a hardware operating environment according to an embodiment of the present invention.
As shown in fig. 1, the accompaniment adjusting apparatus may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a Wireless interface (e.g., a Wireless-Fidelity (Wi-Fi) interface). The Memory 1005 may be a Random Access Memory (RAM) Memory, or may be a Non-Volatile Memory (NVM), such as a disk Memory. The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the configuration shown in fig. 1 does not constitute a limitation of the accompaniment adjustment device, and may include more or less components than those shown, or combine certain components, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a storage medium, may include therein an operating system, a network communication module, a user interface module, and an accompaniment adjustment program.
In the accompaniment adjustment apparatus shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 of the accompaniment adjusting device according to the present invention may be provided in the accompaniment adjusting device, which calls the accompaniment adjusting program stored in the memory 1005 through the processor 1001 and executes the accompaniment adjusting method according to the embodiment of the present invention.
An embodiment of the present invention provides an accompaniment adjustment method, and referring to fig. 2, fig. 2 is a flowchart illustrating a first embodiment of an accompaniment adjustment method according to the present invention.
In this embodiment, the accompaniment adjustment method includes the following steps:
step S10: when the user sings, the segment number of the current accompaniment is determined.
It should be noted that the execution subject of this embodiment is a server, and the server may be an entity server, a cloud server, or another server that can implement this function, and this embodiment does not limit this.
It should be understood that, in the existing singing assisting software, basically, after a user finishes singing a song accompaniment, the accompaniment is modified according to the collected singing voice of the user, and finally, a work with the modified accompaniment is obtained, but the accompaniment cannot be adjusted in real time in the singing process of the user, so that the singing effect and the singing score of the user are improved. According to the scheme of the embodiment, the accompaniment information of the segment of the next segment of the current accompaniment sung by the user is adjusted in real time according to the accompaniment characteristic information, the singing characteristic information of the user and the historical singing information, so that the accompaniment of the next segment can better accord with the current user, and the singing effect and the singing score of the user are improved.
In a specific implementation, when the user sings, determining the segment number of the current accompaniment refers to: when singing voice information input by a user through a microphone during playing of the accompaniment is detected, the specific segment of the current accompaniment which the user starts to sing is judged according to the singing voice information, and the detection is finished in real time, namely, the segment number and the segment accompaniment information where the current accompaniment is located are detected and determined while the user sings the accompaniment song. The singing sound information refers to the sound information of the singing accompaniment of the user, and does not include other environmental noises and other sounds of the user, which are irrelevant to the singing song.
Note that the clip number refers to a number of each clip accompaniment obtained by segmenting the current accompaniment in accordance with the accompaniment tempo.
Further, before step S10, in order to accurately segment the accompaniment, the method further includes: acquiring accompaniment beat information of the current accompaniment; segmenting the current accompaniment according to the accompaniment tempo information to obtain a plurality of segmented accompaniments; and numbering the accompaniment of the plurality of segments to obtain segment numbers.
In an implementation, the accompaniment tempo information includes, but is not limited to, accompaniment music tempo related information of the current accompaniment, and other accompaniment related information.
It should be noted that, segmenting the current accompaniment according to the accompaniment tempo information to obtain a plurality of pieces of accompaniment refers to: segmenting the current accompaniment according to the time sequence and the accompaniment tempo according to the accompaniment tempo information, which can be specifically divided into: the prelude, the verse, the joint section of the master and the refrain, the bridge section or the interlude, and the tail may also include other parts, which is not limited in this embodiment. The resulting segments respectively representing the beats of the different accompaniments are the segment accompaniments.
It should be understood that numbering the accompaniment clips refers to numbering all accompaniment clips in the sequence of the current accompaniment after the accompaniment clips are obtained, for example: the clip accompaniment representing the prelude is No. 1, the clip accompaniment representing the master song is No. 2, and the remaining clip accompaniments are numbered sequentially.
Through this kind of mode, realized carrying out segment division and numbering to the current accompaniment of a whole section, obtained a plurality of segment accompaniments that have the segment number, the change of accompaniment tone often takes place in the junction of each part, promptly in same part, the tone difference of accompaniment is not big, has the tone difference between different parts, and then carries out the adjustment of accompaniment in each segment accompaniment, makes the real-time adjustment accompaniment of cooperation user's singing more nimble.
Step S20: acquiring accompaniment characteristic information of the current accompaniment, singing characteristic information of the user and historical singing characteristic information.
In a specific implementation, the accompaniment feature information includes information related to singing scores of all users selecting the current accompaniment for the backward singing, and information related to the tone, tone and loudness of the accompaniment.
Note that the singing feature information refers to score information of each segment when the user sings the current accompaniment, and related information such as the tone, timbre, loudness, and the like of the singing.
It should be understood that the historical singing feature information includes, but is not limited to, historical scoring information of all accompaniment performed by the current user, and information related to loudness, tone and pitch of the historical singing.
Step S30: obtaining accompaniment adjusting information according to the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user and the historical singing characteristic information.
In a specific implementation, obtaining accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the singing feature information of the user and the historical singing feature information refers to: training a model to be trained according to the accompaniment feature information of the current accompaniment, the singing feature information of the user and the historical singing feature information to obtain a tone adjustment model, and inputting the singing feature information into the accompaniment adjustment model to obtain accompaniment adjustment information.
It should be noted that the accompaniment adjustment information refers to the related information for adjusting the key, timbre and loudness of the accompaniment of the next clip, that is, the adjustment basis for modifying the accompaniment information of the clip of the next clip.
Step S40: and adjusting the clip accompaniment information of the next segment according to the accompaniment adjustment information and the clip number.
It should be understood that adjusting the segment accompaniment information of the next segment according to the accompaniment adjustment information and the segment number means: and then, the timbre, the loudness and the tone in the accompaniment information of the next segment are specifically adjusted according to the accompaniment adjustment information, so that the accompaniment of the next segment is adjusted in real time according to the singing sound of each segment sung by the user.
In the embodiment, when a user sings, the segment number of the current accompaniment is determined; acquiring accompaniment characteristic information of the current accompaniment, singing characteristic information of the user and historical singing characteristic information; obtaining accompaniment adjusting information according to the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user and the historical singing characteristic information; and adjusting the clip accompaniment information of the next segment according to the accompaniment adjustment information and the clip number. Through this kind of mode, can obtain the accompaniment adjustment information according to the accompaniment characteristic information of current accompaniment, the singing characteristic information and the historical singing characteristic information of user to the next segmented segment accompaniment information of user singing is adjusted in real time, has realized adjusting the accompaniment in real time, and then makes the accompaniment after the adjustment more suitable user, improves user's singing effect.
Referring to fig. 3, fig. 3 is a flowchart illustrating an accompaniment adjustment method according to a second embodiment of the present invention.
Based on the first embodiment, the accompaniment adjustment method of the present embodiment includes, in the step S30:
step S301: obtaining accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the segment accompaniment information of the next segment and a pre-trained accompaniment adjustment model.
It should be noted that the model to be trained refers to a pre-established untrained model, and the model to be trained is trained to obtain the accompaniment adjustment model.
It should be understood that the accompaniment adjustment model refers to a model that can directly output and adjust the accompaniment adjustment information of the next segment after inputting the singing feature information sung by the user, especially the singing feature information of the current segment of the current accompaniment.
It should be noted that, obtaining the accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the segment accompaniment information of the next segment and the accompaniment adjustment model means inputting the accompaniment feature information, the singing feature information and the historical singing feature information into the trained accompaniment adjustment model, and then obtaining the accompaniment adjustment information according to the output of the accompaniment adjustment model and the segment accompaniment information of the next segment.
Further, in order to obtain an accompaniment adjustment model, step S301 includes: obtaining an input feature set according to the accompaniment feature information, the singing feature information and the historical singing feature information; constructing a multivariate function fitting model, and taking the multivariate function fitting model as a model to be trained; and training the model to be trained according to the input feature set to obtain an accompaniment adjustment model.
In a specific implementation, obtaining an input feature set according to the accompaniment feature information, the singing feature information and the historical singing feature information means: extracting and acquiring parameters related to tone, loudness, tone and score in the accompaniment characteristic information, the singing characteristic information and the historical singing characteristic information, and then using the extracted parameters as an input feature set for training a model to be trained.
It should be noted that the multivariate function fitting model refers to a multivariate function fitting model combining first-order linear multiplication and second-order non-linear combination, and then serves as a model to be trained, wherein the formula of the model to be trained is as follows:
Figure BDA0003429690970000081
wherein eta is the model parameter, and alpha, beta, S, A, T and P are the input feature set of the model input.
Further, in order to obtain an accurate input feature set, the step of obtaining the input feature set according to the accompaniment feature information, the singing feature information and the historical singing feature information includes: obtaining a segmented singing score, a segmented loudness characteristic, a segmented tone characteristic and a segmented tone characteristic of each segmented accompaniment according to the accompaniment characteristic information; obtaining singing loudness characteristics, singing tone characteristics and singing tone characteristics of the segmented accompaniments singed by the user according to the singing characteristic information; obtaining the historical singing loudness characteristic, the historical singing tone characteristic and the historical singing tone characteristic of the current accompaniment sung performed by the user in the historical manner according to the historical singing characteristic information; and taking the segmented singing score, the segmented loudness characteristic, the segmented tone characteristic, the singing loudness characteristic, the singing tone characteristic, the historical singing loudness characteristic, the historical singing tone characteristic and the historical singing tone characteristic as an input feature set.
It should be understood that, obtaining the segmented singing score, the segmented loudness characteristic, the segmented timbre characteristic and the segmented tone characteristic of each segmented accompaniment according to the accompaniment characteristic information means extracting the relevant segmented singing score, the segmented loudness characteristic, the segmented timbre characteristic and the segmented tone characteristic according to the accompaniment characteristic information, and specifically, the obtaining step of the average singing score is as follows: obtaining singing score highest score S after accompaniment is selected by usermaxMinimum score SminMean score of SavgAnd score a decile value S1，S2，S3，S4，S5，S6，S7，S8，S9，S10. And (3) segmented loudness characteristics: amplitude maximum beta of sound wave vibrationmax XMinimum amplitude of vibration of sound wave
Figure BDA0003429690970000091
Mean value of amplitude of sound wave vibration
Figure BDA0003429690970000092
Sound wave vibration amplitude four-quantile value
Figure BDA0003429690970000093
Segmented tone color characteristics: deciles of vibration waveform, spectrum structure including sound wave
Figure BDA0003429690970000094
Segmented tone features: frequency maximum of acoustic wave
Figure BDA0003429690970000095
Minimum value of frequency
Figure BDA0003429690970000096
Mean value of frequency
Figure BDA0003429690970000097
Frequency quartile value
Figure BDA0003429690970000098
In the specific implementation, according to the singing characteristic information, obtaining a singing loudness characteristic, a singing tone characteristic and a singing tone characteristic of each segmented accompaniment sung of a user, carrying out characteristic acquisition by an accompaniment segmented unit, wherein the sampling frequency is 44.1KHz, the sampling bit depth is 24 bits, the song ID with accompaniment singing is adopted, the score of integral singing is score, the singing segment k is adopted, and 24 characteristics of each segment include the singing loudness characteristic, the singing tone characteristic and the singing tone characteristic.
Note that, the singing loudness feature refers to: amplitude maximum value alpha of sound wave vibrationmax XMinimum amplitude of vibration of sound wave
Figure BDA0003429690970000099
Mean value of amplitude of sound wave vibration
Figure BDA00034296909700000910
Sound wave vibration amplitude four-quantile value
Figure BDA00034296909700000911
It should be understood that the vocal timbre characteristics refer to: decile value of vibration waveform and frequency spectrum structure of each section of sound wave
Figure BDA00034296909700000912
In a specific implementation, the singing tone feature refers to: frequency maximum of acoustic wave
Figure BDA0003429690970000101
Minimum value of frequency
Figure BDA0003429690970000102
Mean value of frequency
Figure BDA0003429690970000103
Frequency quartile value
Figure BDA0003429690970000104
In specific implementation, obtaining the historical singing loudness characteristic, the historical singing tone characteristic and the historical singing tone characteristic of the current accompaniment sung historically by the user according to the historical singing characteristic information means that: if the score of the historical K songs of the user is K, and the standard accompaniment segment number K is set when the K songs are played, recording the setting of each segment according to the time sequence loudness: a1, a2, A3 … AN; recording each segment time-sequential timbre setting: t1, T2, T3 … TN; recording each segment tone time-sequentially set: p1, D2, D3 … PN. If N is 1000, setting the loudness, timbre and tone adjusting point in the segment of the standard accompaniment to 1000 points, wherein the value range of the loudness A is 0-100dB, the value range of the timbre T is 20-20000 Hz, the timbre P is adjusted in an enumeration value mode, and presetting 10 grades of timbre adjusting modes.
It should be noted that the input feature set is input to the trainingDuring the model training process, the tone characteristics are used
Figure BDA0003429690970000105
For example, the maximum value is collected at the time of collection
Figure BDA0003429690970000106
Minimum value of frequency
Figure BDA0003429690970000107
Mean value of frequency
Figure BDA0003429690970000108
Frequency quartile value
Figure BDA0003429690970000109
A total of 7 values, in this formula
Figure BDA00034296909700001010
i takes on a value from 1 to 7, i.e., includes
Figure BDA00034296909700001011
Other parameters and
Figure BDA00034296909700001012
similarly, no further description is given. In addition, β in this stepiIs a vector comprising
Figure BDA00034296909700001013
Other parameters and betaiSimilarly, no further description is given.
Secondly, determining that the discrimination target of the model is Karaoke score, namely singing score, and mapping the singing score with historical accompaniment into a [0,1] interval, wherein the decision function is as follows:
Figure BDA00034296909700001014
the score is 1, the selection threshold is 0.8, namely the singing score with the accompaniment is more than 80 points and is a positive example, the score of the historical Karaoke which is lower than 80 points is a negative example, the singing and the accompaniment are limited to belong to the same song ID, so that a sample set is constructed, and the accompaniment adjusting model which has fitting capability to nonlinear characteristics and has deep learning capability is obtained through training.
Through the method, various singing scores, loudness, tone and tone parameters are obtained specifically according to the accompaniment characteristic information, the singing characteristic information and the historical singing characteristic information, so that the input characteristic set is more accurate and comprehensive, and further, the calculation of the trained accompaniment adjustment model is more accurate.
Further, in order to obtain the accompaniment adjustment information more accurately and effectively, and further improve the user experience, adjust the accompaniment in real time, step S301 includes: inputting the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user and the historical singing characteristic information into the accompaniment adjusting model to obtain a loudness sequencing result, a tone sequencing result and a tone sequencing result; determining the optimal loudness matching degree according to the loudness sequencing result; determining the optimal tone matching degree according to the tone sorting result; determining the optimal tone matching degree according to the tone sorting result; and obtaining accompaniment adjusting information according to the optimal loudness matching degree, the optimal tone matching degree and the next segmented clip accompaniment information.
It should be understood that the loudness matching degree, the pitch matching degree and the timbre matching degree refer to the matching degree of each loudness, pitch and timbre corresponding to each segmented accompaniment obtained after the accompaniment feature information, the singing feature information and the historical singing information are input into the accompaniment adjustment model, that is, all loudness values correspond to the matching degree of each segmented accompaniment, and the pitch and the timbre are analogized in the same way. Because the singing characteristic information is updated in real time along with the singing of the user, the loudness matching degree, the tone matching degree and the tone matching degree output by the accompaniment adjustment model are also continuously changed.
In specific implementation, the accompaniment feature information of the current accompaniment, the singing feature information of the user and the historical singing feature information are input into the accompaniment adjustment model, and the obtained loudness ordering result, the tone ordering result and the tone ordering result mean that all the loudness matching degree, the tone matching degree and the tone matching degree are output in the accompaniment adjustment model and ordered according to the matching degree, and the loudness ordering result corresponding to the loudness matching value, the tone ordering result corresponding to the tone matching value and the tone ordering result corresponding to the tone matching value are respectively obtained.
It is to be understood that the optimal loudness matching is determined from the loudness ordering result; determining the optimal tone matching degree according to the tone sorting result; determining the best tone matching degree according to the tone sorting result refers to: and taking the loudness matching degree, the tone matching degree and the timbre matching degree which are ranked first in the loudness ordering result, the tone ordering result and the tone ordering result, namely the corresponding loudness matching degree, the tone matching degree and the timbre matching degree which are the highest in matching degree as the optimal loudness matching degree, the optimal tone matching degree and the optimal timbre matching degree.
It should be noted that, obtaining accompaniment adjustment information according to the optimal loudness matching degree, the optimal pitch matching degree, the optimal timbre matching degree, and the accompaniment information of the next segment means: and determining the optimal values of the segment loudness characteristic, the segment timbre characteristic and the segment key characteristic of the next segment accompaniment as accompaniment adjustment information according to the optimal loudness matching degree, the optimal tone matching degree and the optimal timbre matching degree and by combining the segment accompaniment information of the next segment.
Through this kind of mode, realized according to the accurate next segmentation that obtains of information that accompaniments regulation model output, also be exactly the best loudness matching degree, the best tone matching degree and the best tone matching degree of next segment accompaniment to obtain the accompaniment regulation information, and then make and to adjust the accompaniment of next segmentation to the most suitable current user singing, improve user's singing effect and use experience.
In this embodiment, the accompaniment adjustment information is obtained according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the segment accompaniment information of the next segment, and the pre-trained accompaniment adjustment model. Through this kind of mode, realized obtaining the accompaniment adjustment model according to accompaniment characteristic information, singing characteristic information and historical singing characteristic information, and then obtain the accompaniment adjustment information according to singing characteristic information and accompaniment adjustment model for adjust the user more accurate and comprehensive of the accompaniment that is singing in real time, can be faster adjust the accompaniment to the state that is most suitable for the user, thereby improve user's singing effect and use experience.
Referring to fig. 4, fig. 4 is a flowchart illustrating an accompaniment adjustment method according to a third embodiment of the present invention.
Based on the first embodiment, before the step S10, the accompaniment adjustment method of this embodiment further includes:
step S101: and when the user starts playing the initial accompaniment, determining the beginning accompaniment according to the clip number of the initial accompaniment.
It should be noted that the initial accompaniment refers to the accompaniment that is not adjusted and is played after the user selects a song.
It should be understood that, when the user starts playing the initial accompaniment, determining the beginning accompaniment track according to the track number of the initial accompaniment means: when the user is detected to start playing the initial accompaniment, determining the clip accompaniment of the beginning part according to the clip number of the segment of the initial accompaniment to be used as the beginning clip accompaniment.
Step S102: and obtaining the characteristic information of the beginning accompaniment according to the beginning clip accompaniment.
In a specific implementation, obtaining the accompaniment feature information from the accompaniment of the beginning clip refers to querying the accompaniment feature information for the feature information of the corresponding tone, timbre and loudness of the accompaniment of the beginning clip as the accompaniment feature information of the beginning clip.
Step S103: and inputting the historical singing characteristic information into the accompaniment regulation model to obtain the regulation information of the beginning segment.
The step of inputting the historical singing feature information into the accompaniment adjustment model to obtain the beginning segment adjustment information includes: at this time, the user has not started singing the accompaniment song, but needs to adjust the first segment accompaniment, so the historical singing feature information is input into the accompaniment adjustment model, and the adjustment information for the first segment accompaniment, that is, the first segment adjustment information is obtained.
Step S104: and adjusting the characteristic information of the beginning accompaniment according to the adjusting information of the beginning fragment to obtain the current accompaniment.
It should be understood that adjusting the beginning accompaniment feature information according to the beginning clip adjustment information to obtain the current accompaniment means modifying each feature information of loudness, timbre and tone in the beginning accompaniment feature information of the beginning clip accompaniment according to the beginning clip adjustment information, and the modified accompaniment is the current accompaniment.
In the embodiment, when the user starts playing the initial accompaniment, determining the beginning accompaniment according to the segment number of the initial accompaniment; obtaining beginning accompaniment feature information of the beginning segment accompaniment according to the accompaniment feature information; inputting the historical singing feature information into the accompaniment adjustment model to obtain beginning segment adjustment information; and adjusting the starting accompaniment feature information of the starting clip accompaniment according to the starting clip adjustment information to obtain the current accompaniment. By the method, the first segment of the accompaniment can be adjusted according to the characteristic information of the historical singing of the user when the user does not start singing the accompaniment, so that the first segment of the accompaniment can be most suitable for the singing of the user, the use experience of the user is improved, and the singing effect and the performance of the user are improved.
Furthermore, an embodiment of the present invention further provides a storage medium, wherein the storage medium stores an accompaniment adjustment program, and the accompaniment adjustment program is executed by a processor to realize the steps of the accompaniment adjustment method.
Since the storage medium adopts all technical solutions of all the embodiments described above, at least all the beneficial effects brought by the technical solutions of the embodiments described above are achieved, and are not described in detail herein.
Referring to fig. 5, fig. 5 is a block diagram of an accompaniment adjusting device according to a first embodiment of the present invention.
As shown in fig. 5, the accompaniment adjustment device according to the embodiment of the present invention includes:
the determining module 10 is configured to determine a segment number of the current accompaniment when the user sings.
An obtaining module 20, configured to obtain accompaniment feature information of the current accompaniment, singing feature information of the user, and historical singing feature information.
And the processing module 30 is configured to obtain accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, and the historical singing feature information.
And the adjusting module 40 is configured to adjust the clip accompaniment information of the next clip according to the accompaniment adjustment information and the clip number.
In the embodiment, when a user sings, the segment number of the current accompaniment is determined; acquiring accompaniment characteristic information of the current accompaniment, singing characteristic information of the user and historical singing characteristic information; obtaining accompaniment adjusting information according to the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user and the historical singing characteristic information; and adjusting the clip accompaniment information of the next segment according to the accompaniment adjustment information and the clip number. Through this kind of mode, can obtain the accompaniment adjustment information according to the accompaniment characteristic information of current accompaniment, the singing characteristic information and the historical singing characteristic information of user to the next segmented segment accompaniment information of user singing is adjusted in real time, has realized adjusting the accompaniment in real time, and then makes the accompaniment after the adjustment more suitable user, improves user's singing effect.
In an embodiment, the determining module 10 is further configured to obtain accompaniment beat information of the current accompaniment; segmenting the current accompaniment according to the accompaniment tempo information to obtain a plurality of segmented accompaniments; and numbering the accompaniment of the plurality of segments to obtain segment numbers.
In an embodiment, the processing module 30 is further configured to obtain accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the segment accompaniment information of the next segment, and a pre-trained accompaniment adjustment model.
In an embodiment, the processing module 30 is further configured to obtain an input feature set according to the accompaniment feature information, the singing feature information and the historical singing feature information; constructing a multivariate function fitting model, and taking the multivariate function fitting model as a model to be trained; and training the model to be trained according to the input feature set to obtain an accompaniment adjustment model.
In an embodiment, the processing module 30 is further configured to obtain a segment singing score, a segment loudness characteristic, a segment timbre characteristic, and a segment pitch characteristic of each segment accompaniment according to the accompaniment feature information; obtaining singing loudness characteristics, singing tone characteristics and singing tone characteristics of the segmented accompaniments singed by the user according to the singing characteristic information; obtaining the historical singing loudness characteristic, the historical singing tone characteristic and the historical singing tone characteristic of the current accompaniment sung performed by the user in the historical manner according to the historical singing characteristic information; and taking the segmented singing score, the segmented loudness characteristic, the segmented tone characteristic, the singing loudness characteristic, the singing tone characteristic, the historical singing loudness characteristic, the historical singing tone characteristic and the historical singing tone characteristic as an input feature set.
In an embodiment, the processing module 30 is further configured to input the accompaniment feature information of the current accompaniment, the singing feature information of the user, and the historical singing feature information into the accompaniment adjustment model, so as to obtain a loudness ordering result, a tone ordering result, and a tone ordering result; determining the optimal loudness matching degree according to the loudness sequencing result; determining the optimal tone matching degree according to the tone sorting result; determining the optimal tone matching degree according to the tone sorting result; and obtaining accompaniment adjusting information according to the optimal loudness matching degree, the optimal tone matching degree and the next segmented clip accompaniment information.
In an embodiment, the determining module 10 further includes a preset module, where the preset module is configured to determine an initial accompaniment according to a segment number of an initial accompaniment when the user starts playing the initial accompaniment; obtaining starting accompaniment feature information according to the starting segment accompaniment; inputting the historical singing characteristic information into an accompaniment regulation model to obtain beginning segment regulation information; and adjusting the characteristic information of the beginning accompaniment according to the adjusting information of the beginning fragment to obtain the current accompaniment.
It should be understood that the above is only an example, and the technical solution of the present invention is not limited in any way, and in a specific application, a person skilled in the art may set the technical solution as needed, and the present invention is not limited thereto.
It should be noted that the above-described work flows are only exemplary, and do not limit the scope of the present invention, and in practical applications, a person skilled in the art may select some or all of them to achieve the purpose of the solution of the embodiment according to actual needs, and the present invention is not limited herein.
In addition, the technical details that are not described in detail in the present embodiment can be referred to the accompaniment adjustment method provided by any embodiment of the present invention, and are not described herein again.
Further, it is to be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention or portions thereof that contribute to the prior art may be embodied in the form of a software product, where the computer software product is stored in a storage medium (e.g. Read Only Memory (ROM)/RAM, magnetic disk, optical disk), and includes several instructions for enabling a terminal device (e.g. a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims (10)

1. An accompaniment adjustment method, comprising:
when a user sings, determining the segment number of the current accompaniment;
acquiring accompaniment characteristic information of the current accompaniment, singing characteristic information of the user and historical singing characteristic information;
obtaining accompaniment adjusting information according to the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user and the historical singing characteristic information;
and adjusting the clip accompaniment information of the next segment according to the accompaniment adjustment information and the clip number.
2. The method of claim 1, wherein prior to determining the segment number at which the current accompaniment is located, further comprising:
acquiring accompaniment beat information of the current accompaniment;
segmenting the current accompaniment according to the accompaniment tempo information to obtain a plurality of segmented accompaniments;
and numbering the accompaniment of the plurality of segments to obtain segment numbers.
3. The method of claim 1, wherein obtaining accompaniment adjustment information based on the accompaniment feature information of the current accompaniment, the singing feature information of the user and the historical singing feature information comprises:
obtaining accompaniment adjustment information according to the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the segment accompaniment information of the next segment and a pre-trained accompaniment adjustment model.
4. The method of claim 3, wherein the training process of the accompaniment adjustment model comprises:
obtaining an input feature set according to the accompaniment feature information, the singing feature information and the historical singing feature information;
constructing a multivariate function fitting model, and taking the multivariate function fitting model as a model to be trained;
and training the model to be trained according to the input feature set to obtain an accompaniment adjustment model.
5. The method of claim 4, wherein the deriving an input feature set from the accompaniment feature information, singing feature information and historical singing feature information comprises:
obtaining a segmented singing score, a segmented loudness characteristic, a segmented tone characteristic and a segmented tone characteristic of each segmented accompaniment according to the accompaniment characteristic information;
obtaining singing loudness characteristics, singing tone characteristics and singing tone characteristics of the segmented accompaniments singed by the user according to the singing characteristic information;
obtaining the historical singing loudness characteristic, the historical singing tone characteristic and the historical singing tone characteristic of the current accompaniment sung performed by the user in the historical manner according to the historical singing characteristic information;
and taking the segmented singing score, the segmented loudness characteristic, the segmented tone characteristic, the singing loudness characteristic, the singing tone characteristic, the historical singing loudness characteristic, the historical singing tone characteristic and the historical singing tone characteristic as an input feature set.
6. The method of claim 4, wherein obtaining accompaniment adjustment information based on the accompaniment feature information of the current accompaniment, the singing feature information of the user, the historical singing feature information, the segment accompaniment information of the next segment, and a pre-trained accompaniment pattern comprises:
inputting the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user and the historical singing characteristic information into the accompaniment adjusting model to obtain a loudness sequencing result, a tone sequencing result and a tone sequencing result;
determining the optimal loudness matching degree according to the loudness sequencing result;
determining the optimal tone matching degree according to the tone sorting result;
determining the optimal tone matching degree according to the tone sorting result;
and obtaining accompaniment adjusting information according to the optimal loudness matching degree, the optimal tone matching degree and the next segmented clip accompaniment information.
7. The method of any one of claims 1 to 6, wherein prior to determining the segment number at which the current accompaniment is located, further comprising:
when the initial accompaniment is played, determining the beginning segment accompaniment according to the segment number of the initial accompaniment;
obtaining starting accompaniment feature information according to the starting segment accompaniment;
inputting the historical singing characteristic information into an accompaniment regulation model to obtain beginning segment regulation information;
and adjusting the characteristic information of the beginning accompaniment according to the adjusting information of the beginning fragment to obtain the current accompaniment.
8. An accompaniment adjusting device, comprising:
the determining module is used for determining the segment number of the current accompaniment when the user sings;
the obtaining module is used for obtaining accompaniment characteristic information of the current accompaniment, singing characteristic information of the user and historical singing characteristic information;
the processing module is used for obtaining accompaniment adjusting information according to the accompaniment characteristic information of the current accompaniment, the singing characteristic information of the user and the historical singing characteristic information;
and the adjusting module is used for adjusting the clip accompaniment information of the next segment according to the accompaniment adjusting information and the clip number.
9. An accompaniment adjusting apparatus, characterized in that the apparatus comprises: a memory, a processor, and an accompaniment adjustment program stored on the memory and operable on the processor, the accompaniment adjustment program being configured to implement the accompaniment adjustment method according to any one of claims 1 to 7.
10. A storage medium having an accompaniment adjustment program stored thereon, wherein the accompaniment adjustment program is executed by a processor to implement the accompaniment adjustment method according to any one of claims 1 to 7.
CN202111595166.9A 2021-12-23 2021-12-23 Method, device and equipment for adjusting accompaniment and storage medium Pending CN114283769A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111595166.9A CN114283769A (en) 2021-12-23 2021-12-23 Method, device and equipment for adjusting accompaniment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111595166.9A CN114283769A (en) 2021-12-23 2021-12-23 Method, device and equipment for adjusting accompaniment and storage medium

Publications (1)

Publication Number Publication Date
CN114283769A true CN114283769A (en) 2022-04-05

Family

ID=80875618

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111595166.9A Pending CN114283769A (en) 2021-12-23 2021-12-23 Method, device and equipment for adjusting accompaniment and storage medium

Country Status (1)

Country Link
CN (1) CN114283769A (en)

Similar Documents

Publication Publication Date Title
JP2008516289A (en) Method and apparatus for extracting a melody that is the basis of an audio signal
EP2515296B1 (en) Performance data search using a query indicative of a tone generation pattern
EP2816550A1 (en) Audio signal analysis
CN109979488B (en) System for converting human voice into music score based on stress analysis
CN112382257B (en) Audio processing method, device, equipment and medium
CN106991163A (en) A kind of song recommendations method based on singer's sound speciality
CN104395953A (en) Evaluation of beats, chords and downbeats from a musical audio signal
CN105740394A (en) Music generation method, terminal, and server
EP2515249B1 (en) Performance data search using a query indicative of a tone generation pattern
EP2528054A2 (en) Management of a sound material to be stored into a database
JP2004110422A (en) Music classifying device, music classifying method, and program
EP3929921A1 (en) Melody detection method for audio signal, device, and electronic apparatus
CN106898339B (en) Song chorusing method and terminal
JP2008516288A (en) Extraction of melody that is the basis of audio signal
KR101813704B1 (en) Analyzing Device and Method for User's Voice Tone
CN105895079A (en) Voice data processing method and device
KR100705176B1 (en) Method for generating music file in mobile terminal
Marolt Gaussian Mixture Models For Extraction Of Melodic Lines From Audio Recordings.
Gulati A tonic identification approach for Indian art music
CN114283769A (en) Method, device and equipment for adjusting accompaniment and storage medium
JP4491743B2 (en) Karaoke equipment
WO2019180830A1 (en) Singing evaluating method, singing evaluating device, and program
Kumar et al. Melody extraction from music: A comprehensive study
JP2007241015A (en) Voice evaluation system
CN114038481A (en) Lyric timestamp generation method, apparatus, device and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination