JP6790732B2 - Signal processing method and signal processing device - Google Patents

Description

The present invention relates to a signal processing technique for singing voice.

In recent years, it has been common practice to record a video of a person who is not a professional singer singing and upload it to a video posting site or the like. Such videos are called "singing videos" and are one of the popular genres on video posting sites.

Japanese Unexamined Patent Publication No. 2000-003200

In many cases, the poster of the video that I tried to sing is posting the video in the same way as singing a karaoke song. However, unlike the singing of karaoke songs, the videos posted on the video posting site can be viewed by an unspecified number of users. For this reason, if the singing technique is not sufficient and the singing is unbearable to listen to, the user who watched the video will feel uncomfortable, and will be flooded with harsh comments without hesitation, resulting in a state called "flaming". Sometimes. In such a situation, it will be difficult to post the video after that, so some of the posters of the video that I tried to sing want to correct it to a singing voice that gives the listener the impression that they are singing well. There is. However, conventionally, there has been no technology that meets such needs.

For example, as an example of a technique for changing the impression of a singing voice, a technique disclosed in Patent Document 1 can be mentioned. Patent Document 1 discloses a technique of performing pitch conversion on a male voice and further adding breathing noise according to the formant of the converted voice to convert it into a natural female voice. However, the technique disclosed in Patent Document 1 cannot change the impression of singing skill.

The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a technique capable of changing the impression of singing voice skill while retaining the individuality of the singer.

In order to solve the above problems, the present invention provides a signal processing method having the following specific steps and modification steps. The specific step is a step of specifying a voiced sound section in the singing voice as a correction target section from the singing voice data representing the singing voice. In the correction step, for the correction target section specified in the specific step, the correction to raise or lower the amplitude of the frequency component around the third formant within the range that does not change the shape of the spectral envelope around the third formant is applied to the above singing voice data. Give.

One of the causes that gives the listener the impression that the song is poorly sung is the lack of frequency components around the third formant in the voiced sound section (that is, the amplitude of the frequency components is small). If the frequency component around the 3rd formant is sufficient, the singing voice with a sense of volume as if an opera singer is singing (a singing voice with elasticity, a singing voice that resonates cheerfully, a singing voice with richness and depth, etc.) It may be expressed), that is, it feels like a good singing, but if the frequency component around the third formant is insufficient, it feels like a poor singing with no tension or depth, that is, a poor singing.

According to the present invention, by increasing the amplitude of the frequency component around the third formant in the voiced sound section, it is possible to give the listener the impression that the singing is better than before the correction, and vice versa. By lowering the amplitude of the frequency component around the third formant in the voiced sound section, the listener is given the impression that the singing voice is worse than before the correction (in other words, an amateurish singing voice). Will be possible. Further, according to the present invention, only the voiced sound section is modified, and even if it is the voiced section, the unvoiced sound section is not modified, and the individuality of the singer remains. In addition, since the amplitude of the frequency component around the third formant is raised (or lowered) within the range that does not change the shape of the spectral envelope before modification for the voiced sound section, the singer due to the shape of the spectral envelope is also used for the voiced sound section. Does not completely eliminate the individuality of. In this way, according to the present invention, it is possible to change the impression regarding the skill of the singing voice while retaining the individuality of the singer.

Further, in order to solve the above problems, the present invention provides a signal processing device having the following specific means and correction means. The specific means specifies a voiced sound section in the singing voice as a correction target section from the singing voice data representing the singing voice. The correction means applies a correction to the singing voice data to increase or decrease the amplitude of the frequency component around the third formant within a range that does not change the shape of the spectral envelope around the third formant for the correction target section specified by the specific means. .. Even with such a signal processing device, it is possible to change the impression of the skill of the singing voice while retaining the individuality of the singer.

Further, as another aspect of the present invention, a program for causing a general computer such as a CPU (Central Processing Unit) to execute the signal processing method (in other words, a program for causing the computer to function as the specific means and the modification means). ) Is conceivable. According to such an aspect, a general computer can function as the signal processing device of the present invention, and even such an aspect changes the impression of the skill of the singing voice while retaining the individuality of the singer. Will be possible. Specific provision (distribution) modes of the above program include writing and distributing the above program on a computer-readable recording medium such as a CD-ROM (Compact Disk-Read Only Memory) or a flash ROM, or the Internet. The mode of distribution by downloading via a telecommunication line such as.

It is a figure which shows the structural example of the signal processing apparatus 10A by one Embodiment of this invention. It is a flowchart which shows the flow of the singing voice correction processing which the control unit 100 of the signal processing apparatus 10A executes according to a singing voice correction program 1340A. It is a figure which shows an example of the correction amount data. It is a figure for demonstrating the effect of this embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(A: Embodiment)
FIG. 1 is a diagram showing a configuration example of a signal processing device 10A according to an embodiment of the present invention. The signal processing device 10A is, for example, a personal computer, and as shown in FIG. 1, the control unit 100, the external device interface (hereinafter abbreviated as “I / F”) unit 110, the communication I / F unit 120, and the storage unit 130. , And a bus 140 that mediates the transfer of data between each of these components.

The signal processing device 10A is used by the poster when posting a sung video to a video posting site. The video that I tried to sing is a video that was obtained by recording how the posters of the video sang. Posting a video to a video posting site means uploading video data to the server device of the video posting site. The moving image data of the moving image of the singing includes singing sound data representing the singing sound of the entire singing song to be sung (for example, the singing sound of one song). As a specific example of such singing voice data, there is a sample sequence obtained by sampling the sound wave shape of the singing voice at a predetermined sampling cycle.

The signal processing device 10A is a device that executes a singing voice correction process, which is a signal processing that remarkably shows the characteristics of the present embodiment, with the singing voice data as a processing target. The singing voice correction process is a process of correcting the singing voice data so as to give the listener the impression that the singing is good while retaining the individuality of the singing voice represented by the singing voice data. The poster of the video that I tried to sing gives the impression that the singing voice is good by applying the above singing voice correction processing to the singing voice data included in the video data prior to uploading the video data. It can be modified and posted to the singing voice given to. Hereinafter, the roles of the respective parts constituting the signal processing device 10A will be described.

The control unit 100 is, for example, a CPU. The control unit 100 functions as a control center of the signal processing device 10A by operating according to a program stored in advance in the storage unit 130 (more precisely, the non-volatile storage unit 134). The details of the processing executed by the control unit 100 according to various programs stored in advance in the non-volatile storage unit 134 will be clarified later.

The external device I / F unit 110 is a USB (Universal Serial).
Bus) A collection of interfaces for connecting other electronic devices such as interfaces, serial interfaces, and parallel interfaces. The external device I / F unit 110 delivers the data received from another electronic device connected to itself to the control unit 100, and outputs the data given by the control unit 100 to the other electronic device. In the present embodiment, a recording medium that stores singing voice data representing the singing voice in the moving image that has been sung is connected to the external device I / F unit 110, and the control unit 100 uses the singing voice data stored in the recording medium. Read as a processing target and execute singing voice correction processing.

The communication I / F unit 120 is, for example, a NIC (Network Interface Card). The communication I / F unit 120 is connected to a telecommunication line such as the Internet via a communication line such as a LAN (Local Area Network) cable and a relay device such as a router. The communication I / F unit 120 receives the data transmitted from the telecommunication line to which it is connected and delivers it to the control unit 100, while transmitting the data delivered from the control unit 100 to the telecommunication line. .. For example, the control unit 100 transmits video data including singing voice data that has undergone singing voice correction processing to the server device of the video posting site via the communication I / F unit 120 in response to a user's instruction. This makes it possible to post a video that I tried to sing.

As shown in FIG. 1, the storage unit 130 includes a volatile storage unit 132 and a non-volatile storage unit 134. The volatile storage unit 132 is, for example, a RAM (Random Access Memory). The volatile storage unit 132 is used by the control unit 100 as a work area when executing a program. The non-volatile storage unit 134 is, for example, a flash ROM (Read Only Memory) or a hard disk. The non-volatile storage unit 134 stores in advance a singing voice correction program 1340A for causing the control unit 100 to execute the singing voice correction processing. Further, although detailed illustration is omitted in FIG. 1, a kernel program and a communication control program are stored in advance in the non-volatile storage unit 134. The kernel program is a program for realizing the OS (Operating System) in the control unit 100. The communication control program is, for example, FTP (File Transfer).
This is a program that causes the control unit 100 to execute a process of uploading video data to a server device of a video posting site according to a predetermined communication protocol such as Protocol).

When the power of the signal processing device 10A (not shown in FIG. 1) is turned on, the control unit 100 first reads the kernel program from the non-volatile storage unit 134 to the volatile storage unit 132, and starts executing the kernel program. To do. The control unit 100, which operates according to the kernel program and realizes the OS, operates by operating an operation input device (for example, a mouse or keyboard, not shown in FIG. 1) connected to the external device I / F unit 110. The program instructed to be executed is read from the non-volatile storage unit 134 to the volatile storage unit 132, and the execution of the program is started.

When the execution of the singing voice modification program 1340A is instructed by the operation on the operation input device, the control unit 100 reads the singing voice modification program 1340A from the non-volatile storage unit 134 to the volatile storage unit 132 and executes the program. To start. The control unit 100 operating according to the singing voice correction program 1340A executes the singing voice correction processing. FIG. 2 is a flowchart showing the flow of the singing voice correction process. As shown in FIG. 2, the singing voice correction process includes two steps, a specific step SA100 and a correction step SA110.

The specific step SA100 is a step of specifying a correction target section, which is a section for making corrections to give the listener an impression of good singing from the singing voice represented by the singing voice data to be processed in the singing voice correction processing. Is. In the present embodiment, the control unit 100 specifies the voiced sound section in the singing voice as the correction target section. The voiced sound section is a section in which a voiced sound is pronounced, and the voiced sound in the present embodiment is a vowel. In this embodiment, only vowels are treated as voiced sounds, but in addition to vowels, specific consonants ([b] [d] [g] among plosives, [v] [z] among fricatives, and nasal sounds [m] [n], and liquid consonants [l] [r]) may be included.

In order to specify the voiced sound section in the singing voice, the control unit 100 divides the singing voice data to be processed into frames having a predetermined time length, performs time-frequency conversion, converts the data into frequency domain data, and pitches each frame. (Fundamental frequency) Try to extract. This is because if it is a voiced sound, there is a pitch, but if it is unvoiced or silent, there is no pitch. Next, the control unit 100 indicates the start time and end time of the correction target section when the voiced sound section specified in the above procedure is set as the correction target section and the beginning of the singing voice data to be processed is the start point of the time. Data is written to the volatile storage unit 132 for each correction target section.

The correction step SA110 determines the amplitude of the frequency component around the third formant for each of the correction target sections specified in the specific step SA100, and the shape of the spectral envelope in the correction target section within a range that does not change the shape of the envelope. It is a step to pull up. The formant refers to a plurality of temporally moving peaks appearing in the spectrum of the voice of the person speaking the word, and the third formant refers to the peak having the third lowest frequency. In general, if the amplitude of the frequency component of the third formant and its surroundings (collectively referred to as the "third formant periphery") is sufficient, the singing voice has a voluminous feel as if an opera singer is singing. It may be expressed as a singing voice with a certain sound, a singing voice that resonates cheerfully, a rich and deep singing voice, etc.), that is, it seems to be a good singing, but if the frequency component around the third formant is insufficient, it will become firm. It feels like a poor singing with no depth, that is, a poor singing. Therefore, in the present embodiment, the amplitude of the frequency component around the third formant in the correction target section is increased. Further, the amount of correction of the amplitude of each frequency component around the third formant is limited to the range in which the shape of the spectral envelope is not changed in order to prevent the individuality of the singer from being impaired due to the shape of the spectral envelope. ..

In the singing voice correction program of the present embodiment, correction amount data that defines a correction amount (ratio to the original amplitude) when raising the amplitude of each frequency component within a range that does not change the shape of the spectral envelope around the third formant. (See FIG. 3) is pre-embedded. The frequency range Q and the correction amount G of each frequency component in FIG. 3 may be appropriately set by conducting experiments and the like. The control unit 100 converts the waveform data of the correction target section into frequency domain data for each correction target section, associates the third formant in the frequency section with the center frequency F of the frequency range Q, and surrounds the third formant. The frequency component of is subjected to EQ processing (processing for correcting the amplitudes of both the harmonic component and the non-harmonious component of the voice) according to the correction amount data, and the amplitude of each frequency component is raised.

FIG. 4 is a diagram for explaining the effect of the present embodiment. In FIG. 4, the spectrum envelope before the correction for a certain correction target section is drawn with a dotted line, and the spectrum envelope after the correction is drawn with a solid line. Further, in FIG. 4, the notes constituting the score of the singing voice to be corrected are drawn in a rectangular shape, and the frequency section of the frequencies f3s to f3e in FIG. 4 is the frequency section around the third formant, and the frequency section of the frequency section. The center frequency is the third formant. According to the signal processing device 10A of the present embodiment, the amplitude of the frequency component belonging to the frequency section is corrected according to the correction amount data, and is corrected to the spectral envelope shown by the solid line in FIG. As a result, it is corrected to a good singing voice with a sense of volume as if an opera singer is singing. In FIG. 4, since the amplitude of the frequency component around the third formant is not corrected except for the section to be corrected, the solid line spectrum envelope and the dotted line spectrum envelope overlap.

According to the signal processing device 10A of the present embodiment, it is possible to modify the singing audio data of the "sung video" to be posted to the video posting site to give a better impression to the listener and post the video. become. In addition, in the present embodiment, only the voiced sound section is modified, and the individuality of the singer remains in the unmodified section. In addition, the individuality of the singer does not completely disappear even in the modified section. This is because the shape of the spectral envelope around the third formant is maintained before and after the modification. In this way, according to the present embodiment, it is possible to change the impression of the skill of the singing voice while retaining the individuality of the singer. In this embodiment, the amplitudes of both the harmonious component and the anharmonic component of the voice are corrected, but by separating the harmonious component and the anharmonic component and correcting only the former amplitude, a higher effect (better) is achieved. (To give the listener an impression) may be realized.

(B: Other embodiments)
Although the embodiment of the present invention has been described above, it is of course possible to add the following modifications to this embodiment.
(1) In the above embodiment, a mode in which the singing voice correction processing is executed after singing, that is, a case where the singing voice correction processing is executed as a non-real-time processing for the singing has been described, but the singing and singing voice correction processing are performed. In parallel, that is, the singing voice correction process may be executed as a real-time process for the singing. Specifically, a microphone may be connected to the external device I / F unit 110 of the signal processing device 10A, and the singing voice data to be processed may be input to the signal processing device 10A via the microphone. Further, in this case, a headphone speaker for feeding back the singing voice represented by the singing voice data (that is, the uncorrected singing voice) or the corrected singing voice to the singer is connected to the external device I / F unit 110. You may.

(2) In the above embodiment, the case where the correction target section specified in the specific step is always corrected by the correction step has been described. However, the user is allowed to select a correction target section (or a correction target section that is not corrected) for correcting the amplitude of the frequency component around the third formant from the correction target sections specified in the specific step by operating the operation input means or the like. Alternatively, the user may be allowed to specify the degree of correction for each correction target section.

(3) In the above embodiment, the case where the singing voice data is modified so as to give the listener the impression that the singing is good while retaining the individuality of the singer has been described, but the impression that the singing is poor. The singing voice data may be modified so as to give. For example, the amplitude of the frequency component around the third formant in the section to be corrected may be reduced within a range that does not change the shape of the spectral envelope. This is because the range of production is widened, such as emphasizing the amateurishness by deliberately modifying the singing voice to be poor.

(4) In the above embodiment, the personal computer used by the poster of the singing video is made to function as the signal processing device of the present invention, but the singing voice modification program is installed in the server device at the video posting site. The server device may function as the signal processing device of the present invention. Further, in the above embodiment, the singing voice correction program for causing the control unit 100 to execute the singing voice correction processing that remarkably shows the feature of the present invention is pre-installed in the non-volatile storage unit 134, but the singing voice correction program is used alone. May be provided at. Further, each of the specific means for executing the processing in the specific step and the correction means for executing the processing in the correction step are realized by hardware such as an electronic circuit, and these hardwares are combined to form the signal processing device of the present invention. Is also good.

(5) In each of the above embodiments, an example of application of the present invention to the modification of the singing audio data included in the video data of the "sung video" posted on the video posting site has been described. However, the modification target by the present invention is not limited to the singing audio data included in the moving image data. For example, the present invention may be applied to a modification of singer voice data posted on a posting site containing only singing voice. In short, the signal processing device of the present invention has a specific means for specifying a voiced sound section in the singing voice as a correction target section from the singing voice data representing the singing voice, and a third correction target section specified by the specific means. Any one having a correction means for increasing or decreasing the amplitude of the frequency component around the third formant within the range that does not change the shape of the spectral envelope around the formant is applied to the singing voice data.

10A ... signal processing device, 100 ... control unit, 110 ... external device I / F unit, 120 ... communication I / F unit, 130 ... storage unit, 132 ... volatile storage unit, 134 ... non-volatile storage unit, 1340A ... singing Voice modifier, 140 ... bus.

Claims (2)

A specific step for specifying a voiced sound section in the singing voice as a correction target section from the singing voice data representing the singing voice, and
For the correction target section specified in the specific step, the singing voice data is corrected to increase or decrease the amplitude of the frequency component around the third formant within a range that does not change the convex shape of the spectral envelope around the third formant. Correction steps and
A signal processing method characterized by having. A specific means for specifying a voiced sound section in the singing voice as a correction target section from the singing voice data representing the singing voice, and
For the correction target section specified by the specific means, a correction is made to the singing voice data to raise or lower the amplitude of the frequency component around the third formant within a range that does not change the convex shape of the spectral envelope around the third formant. Means and
A signal processing device characterized by having.

Images