JP2009244705A - Pitch shift system and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent the occurrence of change to an undesired tone by a pitch shift of a musical piece. <P>SOLUTION: The pitch shift system includes: a harmonic extracting means for extracting a harmonic component constituting a harmonic structure out of the components based on spectral distribution of components in a target musical piece (s120); a non-harmonic extracting means for extracting a non-harmonic component not constituting a harmonic structure out of the components of the target musical piece by removing the harmonic component extracted by the harmonic extracting means from the components of the target musical piece (s130); a pitch shift means for shifting a pitch of the harmonic component extracted by the harmonic extracting means according to a pitch shift ratio (s140); and a component synthesizing means for synthesizing the non-harmonic component extracted by the non-harmonic extracting means and the harmonic component shifted in pitch by the pitch shift means so that respective time axes of the components are matched (s160). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a pitch shift system for shifting a target musical piece with a predetermined pitch shift magnification.

Conventionally, when the music (input sound) is pitch-shifted, the pitch is shifted regardless of the component (see, for example, Patent Document 1).
In general, first, frequency analysis of the waveform along the time axis indicating the composition of the music is performed for each unit section by Fourier transform, etc., and the resulting spectrum distribution component is shifted to the frequency band corresponding to the pitch shift magnification. In addition, the pitch shift is realized by a processing procedure of returning this to each waveform of the unit section along the time axis.
JP-A-8-171387

  As described above, when the pitch shift is performed, the component of the music is analyzed for each unit section. Therefore, in order to appropriately reflect the steep change in the component, it is necessary to shorten the unit section. Since the processing load increases as the unit interval is shortened, a certain unit interval must be secured by trade-off with the processing load.

  Then, if the component of the music contains a non-harmonic component that does not have a harmonic structure such as a percussion instrument sound that changes abruptly within a unit interval, after performing the pitch shift, If the sound of the component indicating the change disappears or the sound of the component indicating a gradual change, the sound may change to an unintended sound.

  The present invention has been made to solve such a problem, and an object of the present invention is to provide a technique for preventing an unintended change in sound due to a pitch shift of music.

  In order to solve the above-mentioned problem, it is conceivable that a pitch shift system for shifting the target music piece with a predetermined pitch shift magnification is configured as shown in a first configuration (claim 1) described below.

  In this configuration, based on the spectral distribution of the components in the target music, harmonic extraction means for extracting the harmonic components having the harmonic structure of the components, and the harmonics from the components of the target music By removing the harmonic component extracted by the extracting means, the non-harmonic extracting means for extracting the non-harmonic component that does not form the harmonic structure among the components of the target music, and the harmonic extracting means Pitch shift means for shifting the extracted harmonic components in accordance with the pitch shift magnification, the non-harmonic components extracted by the non-harmonic extraction means, and the harmonics pitch-shifted by the pitch shift means. Component synthesis means for synthesizing the wave components so that the respective time axes coincide with each other.

  In the pitch shift system having such a configuration, the harmonic component having the harmonic structure and the non-harmonic component not having the harmonic structure are extracted from the component of the target music, and the non-harmonic component and the pitch shift are extracted. The applied harmonic components are synthesized. That is, the pitch shift is performed only for the harmonic component among the components of the target music, and the pitch shift is not performed for the non-harmonic component.

  The non-harmonic component is a component that changes sharply, such as percussion instrument sound, and determines the rhythm and tempo of the target song. There is no big problem without shifting.

  Rather, shifting the non-harmonic component is a component that shows a steep change when a steep change in the sub-harmonic component occurs within the unit interval when the target music is frequency-analyzed. If the sound disappears or the sound of a component that shows a gradual change, the sound will change to an unintended sound.

  However, in the above configuration, only the harmonic component separated from the component of the target musical piece is pitch-shifted, and the non-harmonic component is not pitch-shifted. Can be.

  In this configuration, the method for extracting the harmonic component from the component in the target music is not particularly limited, but, for example, a spectrum obtained by frequency analysis of the waveform along the time axis indicating the component of the target music by Fourier transform or the like In the distribution, it is conceivable to extract, as a harmonic component, a component in which a spectral intensity peak appears periodically (having a harmonic structure).

Further, as a specific configuration for pitch shifting the harmonic component, for example, a second configuration (claim 2) shown below can be considered.
In this configuration, for the harmonic component pitch-shifted by the pitch shift means, the resolution indicating the interval in which the component is distributed along the frequency axis of the harmonic component is the same as the reciprocal of the pitch shift magnification. Resolution conversion means for changing the interval to be a magnification interval, and the component synthesis means includes the non-harmonic component extracted by the non-harmonic extraction means and the harmonic converted by the resolution conversion means. Synthesize wave components.

  With this configuration, the resolution indicating the interval is converted so that the interval in which the components are distributed along the frequency axis of the pitch-shifted harmonic component is the same as the reciprocal of the pitch shift magnification. Thus, the harmonic component and the non-harmonic component can be synthesized.

  When the harmonic component is pitch-shifted, if the pitch-shift magnification A is greater than 1, the frequency distribution in that component spreads to A (<1) times, and the interval in which the components are distributed along the frequency axis is widened. If the magnification A is less than 1, the frequency distribution of the component is narrowed to A (<1) times, and the interval at which the component is distributed along the frequency axis is narrowed.

  In the above configuration, by changing the resolution so that the interval of the component distribution along the frequency axis in the spectral distribution of the harmonic component becomes the same magnification interval as the reciprocal of the pitch shift magnification, Changes can be offset.

  For example, when the pitch shift magnification A is larger than 1, the interval at which the components are distributed along the frequency axis “spreads” due to the pitch shift, this interval becomes a “narrow” interval having the same magnification as the reciprocal of the pitch shift magnification. By changing the resolution, it is possible to cancel the change in the interval.

  On the other hand, when the pitch shift magnification A is less than 1, the interval at which the components are distributed along the frequency axis is “narrowed” by the pitch shift, but this interval becomes a “wide” interval having the same magnification as the reciprocal of the pitch shift magnification. By changing the resolution, it is possible to cancel the change in the interval.

  In this way, even if the interval in which the components are distributed along the frequency axis in the spectral distribution of the harmonic components changes before and after the pitch shift, the change can be offset.

  By the way, when the pitch shift magnification is larger than 1 in the above configuration, the frequency distribution is expanded to ½ or more of the sampling frequency Fs due to the pitch shift, and this may become a so-called aliasing noise. Therefore, in order to prevent such noise from occurring, it is desirable that the above-described configuration be a third configuration (claim 3) shown below.

  In this configuration, when the pitch shift magnification A is larger than 1, filtering means for removing a frequency component equal to or higher than the frequency Fu defined by the following expression 1 from the component of the target musical piece is provided. Then, when the pitch shift magnification A is greater than 1, the pitch shift means performs pitch shift based on the target musical piece from which the frequency component equal to or higher than the frequency Fu is removed by the filtering means.

Fu = Fs / (2 · A) (Formula 1)
With this configuration, when the pitch shift magnification is larger than 1, the pitch shift is performed based on the target musical piece from which the frequency component equal to or higher than the frequency Fu (= Fs / (2 · A)) is removed.

  This frequency band equal to or higher than the frequency Fu is a band where the frequency distribution can be expanded to 1/2 or more of the sampling frequency Fs by performing a pitch shift with the pitch shift magnification A, so that the frequency equal to or higher than the frequency Fu as in the above configuration. By removing the components in advance, it is possible to prevent the occurrence of aliasing noise due to the frequency distribution spreading to 1/2 or more of the sampling frequency Fs.

  In addition, when the pitch shift magnification is less than 1 in each of the above configurations, the frequency distribution is narrowed by the pitch shift, so that there is a frequency component equal to or higher than the maximum frequency in which the component exists in the frequency distribution. It will not be in a state.

In order to prevent such a situation, it is desirable to make each of the above configurations as a fourth configuration (claim 4) shown below.
In this configuration, when the pitch shift magnification A is less than 1, the lowest level component is selected from the harmonic component converted by the resolution conversion means as the frequency component equal to or higher than the frequency Fo defined by the following equation (2). Component interpolation means for interpolating. When the pitch shift magnification A is less than 1, the component synthesis unit is configured to extract the non-harmonic component extracted by the non-harmonic extraction unit and the harmonic component obtained by interpolating the component by the component interpolation unit. Is synthesized.

Fo = (A · Fs) / 2 (Formula 2)
With this configuration, when the pitch shift magnification is less than 1, a distribution having a component equal to or higher than the maximum frequency in which a component exists in the frequency distribution after the pitch shift is represented by a lowest level component (for example, a component having a value of 0). ), It is possible to distribute components that exceed the frequency.

  Moreover, in order to solve the said subject, it is good also as a program (Claim 5) for functioning a computer system as all the means with which the pitch shift system in any one of the said 1st-4th structure is equipped.

If it is a computer system controlled by this program, the same operation | movement and effect as the pitch shift system in any one of the said 1st-4th structure can be acquired.
The above-described program is composed of an ordered sequence of instructions suitable for processing by a computer system, and is provided to a pitch shift system or a user who uses this through various recording media and communication lines. Is.

Embodiments of the present invention will be described below with reference to the drawings.
(1) Hardware Configuration As shown in FIG. 1, the pitch shift system 1 includes a control unit 11 that controls the entire system, a storage unit 13 that stores various information, a communication unit 15 that controls communication via the network 2, This is a known computer system including a user interface (U / I) unit 17 including a keyboard and a display, a media drive 19 for inputting and outputting information via a recording medium, and the like.

  When the pitch shift system 1 receives a predetermined command from the outside via the user interface unit 17 or the communication unit 15, the control unit 11 executes various processes according to the programs stored in the storage unit 13, thereby The function as the pitch shift system of the invention is exhibited.

In the present embodiment, the pitch shift system 1 is configured by a single computer system, but it is needless to say that the pitch shift system 1 can be configured by a plurality of computer systems that operate in cooperation with each other.
(2) Processing by Control Unit 11 Hereinafter, various processing procedures executed by the control unit 11 according to a program stored in the storage unit 13 will be described.
(2-1) Pitch Shift Process First, a pitch shift process that is started when a prescribed operation is performed via the user interface unit 17 or a command is received from the network 2 via the communication unit 15. The processing procedure will be described with reference to FIG.

  When this pitch shift process is started, first, music data (input music data) for reproducing a predetermined music (hereinafter referred to as “target music”) is acquired (s110). Here, the input music data designated through the operation on the user interface unit 17 is acquired from the storage unit 13 or the media drive 19 (that is, the recording medium) or received together with the command through the communication unit 15. Input music data is acquired.

  Next, for the target music indicated by the music data acquired in s110, the harmonic component of the components in the target music is extracted (s120). Here, in the spectrum distribution obtained by frequency analysis of the waveform along the time axis indicating the component of the target music by Fourier transform or the like, the peak of the spectrum intensity appears periodically (having a harmonic structure). ) Component is extracted, and a component along the frequency axis indicated by the spectrum distribution is extracted as a harmonic component.

  Next, for the target music indicated by the music data acquired in s110, the non-harmonic component of the components in the target music is extracted (s130). Here, by subtracting the harmonic component extracted in s120 as a waveform along the time axis from the component of the target song, the component resulting from the subtraction removes the harmonic component from the component of the target song. Are extracted as non-harmonic components.

Next, the pitch shift of the harmonic component extracted in s120 is performed by a processing procedure described later (s140).
Next, the harmonic component pitch-shifted in s140 and the non-harmonic component extracted in s130 are combined as a waveform along the time axis (s160). In this way, the target music that is pitch-shifted by the pitch-shift magnification A is generated.

Then, music data (output music data) indicating the target music synthesized in s160 is returned as a response to the command that triggered the start of this pitch shift process (s170). Here, the output music data is stored in the storage area (the storage area in the storage unit 13 or the recording medium) designated through the operation on the user interface unit 17, or the input music data is transmitted through the communication unit 15. The output music data is returned to the transmission source that has transmitted.
(2-2) Pitch Shift Implementation Procedure for Harmonic Components Next, the pitch shift implementation procedure for the harmonic components in s140 will be described with reference to FIG.

  Here, first, “pitch shift magnification (the ratio of the pitch after the pitch shift to the pitch before the pitch shift” specified through the operation to the user interface unit 17 or received together with the command via the communication unit 15. It is checked whether or not “)” is greater than 1 (s210).

  When it is determined in s210 that the pitch shift magnification is greater than 1 (s210: YES), among the frequency components included in the harmonic component extracted in s120, the pitch shift magnification A and the harmonic component (or target) The frequency component equal to or higher than the cutoff frequency Fu (= Fs / (2 · A)) defined by the sampling frequency Fs of the music) is removed (s220).

As described above, when the pitch shift magnification A is larger than 1, a state in which some frequency components are removed by s220 is shown in FIGS. 4 (a) and 4 (b).
Next, a pitch shift is applied to the harmonic component from which the partial frequency component has been removed in s220 (s230). Here, the pitch is corrected so that the frequency band is expanded to the high frequency side, specifically, so that the frequency of each frequency component of the harmonic component becomes a value obtained by integrating the same magnification as the pitch shift magnification A. FIG. 4C shows how the pitch shift is performed in this way.

  Next, with regard to the harmonic component to which the pitch shift has been performed in the above s230, the distribution interval of the components along the frequency axis in the spectral distribution of the harmonic component is an interval having the same magnification as the reciprocal of the pitch shift magnification A. Thus, the resolution indicating the interval is converted (s240).

  When the harmonic component is pitch-shifted by the above s230, the pitch shift magnification A is larger than 1, so that the frequency distribution in that component spreads A (> 1) times, and the interval in which the components are distributed along the frequency axis is increased. It becomes wide. Therefore, in this s240, the interval in which the components are distributed along the frequency axis is converted into a narrow interval having the same magnification as the reciprocal of the pitch shift magnification A, so that the interval returns to the interval before the pitch shift. ing. A state in which the resolution is converted in this way is shown in FIG.

After the resolution conversion by s240 is performed, the process returns to the pitch shift process (s160).
On the other hand, when it is determined in s210 that the pitch shift magnification is less than 1 (s210: NO), the harmonic component frequency extracted in s120 is the frequency of this harmonic component, similar to s240. The resolution indicating the distribution interval of the components along the axis is converted (s250).

  As will be described later, a pitch shift is applied to the harmonic component extracted in s120. When the harmonic component is pitch-shifted in this way, the pitch shift magnification A is less than 1. For this reason, the frequency distribution of the component is narrowed by A (<1) times, and the interval at which the component is distributed along the frequency axis is narrowed. For this reason, in s260, the interval in which the components are distributed along the frequency axis is converted in advance so as to be a wide interval having the same magnification as the reciprocal of the pitch shift magnification, so that the interval is the same before and after the pitch shift. It is trying to become. A state in which the resolution is converted in this way is shown in FIGS.

  Next, among the frequency components included in the harmonic component to which the pitch shift has been performed in s260, the cutoff frequency Fo (defined by the pitch shift magnification A and the sampling frequency Fs of the harmonic component (or the target music)). = (A · Fs) / 2) The lowest level component (“0” in this embodiment) is interpolated as a frequency component equal to or higher than (s270).

  When the pitch shift magnification is less than 1, as described above, the frequency distribution is narrowed by the pitch shift. As a result, there is no frequency component equal to or higher than the maximum frequency in which the component exists in the frequency distribution. It becomes a state. Therefore, in s270, the lowest level component that does not affect the characteristics is interpolated as the frequency component.

As described above, when the pitch shift magnification A is less than 1, the state in which the partial frequency components are interpolated by this s270 is shown in FIGS.
Thus, after the frequency component is interpolated, the process returns to the pitch shift process (s160).
(3) Operation and Effect With the pitch shift system 1 configured as described above, the harmonic component having the harmonic structure and the non-harmonic component not having the harmonic structure are extracted from the components of the target musical piece (see FIG. 2 s120, s130), the non-harmonic component and the harmonic component subjected to pitch shift are synthesized (s160). That is, the pitch shift is performed only for the harmonic component among the components of the target music, and the pitch shift is not performed for the non-harmonic component.

  The non-harmonic component is a component that changes sharply, such as percussion instrument sound, and determines the rhythm and tempo of the target song. There is no big problem without shifting.

  Rather, shifting the non-harmonic component is a component that shows a steep change when a steep change in the sub-harmonic component occurs within the unit interval when the target music is frequency-analyzed. If the sound disappears or the sound of a component that shows a gradual change, the sound will change to an unintended sound.

  However, in the above embodiment, only the harmonic component separated from the component of the target music is subjected to pitch shift (s140 in the figure), and the non-harmonic component is not subjected to pitch shift. It is possible to prevent a change to sound.

  Further, in the above embodiment, the resolution of the harmonic component is set so that the interval in which the components are distributed along the frequency axis in the harmonic component that has been pitch-shifted is the same interval as the reciprocal of the pitch shift magnification. After conversion (s240 and s250 in FIG. 3), the harmonic component and the non-harmonic component can be synthesized.

  When the harmonic component is pitch-shifted (s230 and s260 in the figure), if the pitch shift magnification A is greater than 1, the frequency distribution in that component spreads to A (<1) times, and the interval at which the components are distributed along the frequency axis. (See FIGS. 4 (a) and 4 (b)), if the pitch shift magnification A is less than 1, the frequency distribution in the component is narrowed to A (<1) times, and the component of the component along the frequency axis is reduced. The distribution interval is narrowed (see FIGS. 5B and 5C).

  In the above configuration, by changing the interval of the component distribution along the frequency axis in the spectral distribution of the harmonic component so as to be an interval having the same magnification as the reciprocal of the pitch shift magnification, the change in the interval as described above can be achieved. Can be offset.

  For example, when the pitch shift magnification A is larger than 1, the interval at which the components are distributed along the frequency axis is expanded due to the pitch shift (see FIG. 4C), this interval is the reciprocal of the pitch shift magnification. By converting to a narrow interval of the same magnification 1 / A (<1) (s240 in FIG. 3), the change in the interval can be canceled (see FIG. 4D).

  On the other hand, when the pitch shift magnification A is less than 1, the interval at which components are distributed along the frequency axis is narrowed by the pitch shift (see FIG. 5C), and this interval is the same magnification as the reciprocal of the pitch shift magnification. Is converted into a wide interval (s250 in FIG. 3), the change in the interval can be canceled (see FIGS. 5B to 5C).

  In this way, even if the interval in which the components are distributed along the frequency axis in the spectral distribution of the harmonic components changes before and after the pitch shift, the change can be offset.

  Further, in the above embodiment, when the pitch shift magnification is larger than 1 (s210 “YES” in FIG. 3), the frequency component higher than the frequency Fu (= Fs / (2 · A)) is removed. Pitch shift is performed based on the music (s220, s230 in the figure).

  This frequency band equal to or higher than the frequency Fu is a band in which the frequency distribution can be expanded to ½ or more of the sampling frequency Fs by performing a pitch shift with the pitch shift magnification A (see FIGS. 4B and 4C). As described above, by removing the frequency components higher than the frequency Fu in advance, it is possible to prevent the occurrence of aliasing noise due to the frequency distribution spreading to 1/2 or more of the sampling frequency Fs.

In the above embodiment, when the pitch shift magnification is less than 1 (s210 “NO” in FIG. 3), the frequency component equal to or higher than the maximum frequency in which the component exists in the frequency distribution after the pitch shift. Is interpolated with the lowest level component (component of “0”), so that components having a frequency higher than that can be distributed. As a result, it is possible to prevent the frequency distribution from being narrowed by the pitch shift, so that no frequency component equal to or higher than the maximum frequency in which the component exists in the frequency distribution does not exist.
(4) Modifications The embodiment of the present invention has been described above. However, the present invention is not limited to the above embodiment, and can take various forms as long as they belong to the technical scope of the present invention. Needless to say.

For example, in the above embodiment, when the pitch shift magnification is less than 1, the pitch is shifted after the resolution conversion (s250 and s260 in FIG. 3). However, the resolution conversion in this case may be performed after the pitch shift.
(5) Correspondence with the Present Invention In the embodiment described above, s120 in FIG. 2 is the harmonic extraction means in the present invention, s130 is the non-harmonic extraction means in the present invention, and s160 in FIG. 3 is a component synthesizing means in the present invention, s230 and s260 in FIG. 3 are pitch shift means in the present invention, s240 and s250 in FIG. 3 are resolution converting means in the present invention, and s220 in FIG. 3 is a filtering means in the present invention. FIG. S270 shows the component interpolation means in the present invention.

The figure which shows the hardware constitutions of the pitch shift system Flow chart showing pitch shift processing Flow chart showing the pitch shift implementation procedure for harmonic components The figure which shows the process in which a harmonic component is pitch-shifted by the pitch shift magnification larger than one. The figure which shows the process in which a harmonic component is pitch-shifted by the pitch shift magnification of less than 1

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Pitch shift system, 2 ... Network, 11 ... Control part, 13 ... Memory | storage part, 15 ... Communication part, 17 ... User interface part, 19 ... Media drive.

Claims (5)

A pitch shift system for pitch-shifting a target song with a predetermined pitch shift magnification,
Harmonic extraction means for extracting a harmonic component having a harmonic structure among the components based on the spectral distribution of the component in the target music;
Non-harmonic extraction for extracting non-harmonic components that do not have a harmonic structure among the components of the target music by removing the harmonic components extracted by the harmonic extraction means from the components of the target music Means,
Pitch shift means for shifting the harmonic component extracted by the harmonic extraction means in accordance with the pitch shift magnification;
Component synthesizing means for synthesizing the non-harmonic component extracted by the non-harmonic extracting means and the harmonic component pitch-shifted by the pitch shift means so that their respective time axes coincide with each other. A pitch shift system characterized by For the harmonic component that has been pitch-shifted by the pitch-shifting means, the resolution indicating the interval in which the component is distributed along the frequency axis of the harmonic component is set to an interval with the same magnification as the reciprocal of the pitch-shift magnification. Resolution conversion means for changing
2. The pitch shift according to claim 1, wherein the component synthesizing unit synthesizes the non-harmonic component extracted by the non-harmonic extracting unit and the harmonic component converted by the resolution converting unit. system. Filtering means for removing a frequency component equal to or higher than the frequency Fu defined by the following equation 1 from the component of the target musical piece when the pitch shift magnification A is greater than 1;
2. The pitch shift unit according to claim 1, wherein, when the pitch shift magnification A is greater than 1, the pitch shift unit performs pitch shift based on a target musical piece from which a frequency component equal to or higher than the frequency Fu is removed by the filtering unit. The pitch shift system according to claim 2.
Fu = Fs / (2 · A) (Formula 1) Component interpolation means for interpolating a lowest level component as a frequency component equal to or higher than the frequency Fo defined by the following equation 2 from the harmonic component converted by the resolution conversion means when the pitch shift magnification A is less than 1 , And
The component synthesizing unit synthesizes the harmonic component extracted by the non-harmonic extracting unit and the harmonic component interpolated by the component interpolating unit when the pitch shift magnification A is less than 1. The pitch shift system according to any one of claims 1 to 3, wherein:
Fo = (A · Fs) / 2 (Formula 2)   A program for causing a computer system to execute various processing procedures for causing all of the means according to any one of claims 1 to 4 to function.

Images