JP4924513B2 - Time stretch system and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a time stretch system and program capable of achieving satisfactory time stretch of a musical piece without causing sense of incongruity auditorily in each of nonharmonic component and harmonic component. <P>SOLUTION: In the time stretch system, the harmonic component constituting a harmonic structure and the nonharmonic component constituting no harmonic structure are first extracted from components of the musical piece being an object (s120, s130), the nonharmonic component is constituted by rearranging each of the divided components when divided per unit section along time base at front and rear positions on the time base in accordance with time stretch magnification (s150), and the harmonic component is time-stretched in accordance with time stretch magnification to extend or contract its waveform along the time base (s140). The nonharmonic component and the harmonic component time-stretched by separate means in this way are synthesized to apply time stretch to the musical piece being the object (s160). <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a time stretch system for time-stretching a target music piece with a predetermined time stretch magnification.

Conventionally, when time-stretching a target song, when the component (waveform) of the target song is divided into unit sections along the time axis, the position on the time axis of each divided component is set as the time stretch magnification. The technique of rearranging in the back-and-forth position accordingly is used (see Patent Document 1).
Japanese Patent No. 3212293

  The above-mentioned technique is used for non-harmonic components that do not have a harmonic structure, specifically, components such as percussion instrument sounds whose levels change sharply by rearranging the components of the target music on the time axis. It is possible to achieve time stretching while maintaining high reproducibility.

  However, in a harmonic component having a harmonic structure, specifically, a voice-like component whose level changes gradually, for example, a silent part formed between each divided component in a unit interval, The result of the cross-fade process performed on the overlapping portions of the divided components may cause unintended auditory discomfort.

  The present invention has been made to solve such a problem, and its purpose is to realize time stretching of music so that there is no sense of incongruity in hearing in each of the non-harmonic component and the harmonic component. Is to provide technology.

In order to solve the above problem, a time stretch system for time-stretching the object music pieces in a predetermined time stretching magnification may be like the first configuration shown below.

  In this configuration, based on the spectral distribution of the components in the target music, harmonic extraction means for extracting the harmonic components having the harmonic structure of the components, and the harmonics from the components of the target music By removing the harmonic components extracted by the extracting means, the non-harmonic extracting means for extracting non-harmonic components that do not have a harmonic structure among the components of the target music, and the non-harmonic extracting means Based on the extracted non-harmonic component, the non-harmonic component is divided for each unit section along the time axis, and each divided component (hereinafter referred to as “divided sub-harmonic component”) on each time axis The component rearrangement means for rearranging the position to a position moved back and forth according to the time stretch magnification, and the harmonic component extracted by the harmonic extraction means so that the waveform expands and contracts along the time axis. Thailand Waveform expansion / contraction means for time-stretching according to the stretch magnification, and harmonic components before or after time-stretching by the waveform expansion / contraction means, the pitch of the harmonic component is lowered according to the time stretch magnification. Alternatively, the time shift by the waveform expansion / contraction means and the pitch shift by the pitch shift means are added to the pitch shift means to be increased and the non-harmonic component in which the divided subharmonic components are rearranged by the component rearrangement means. Component synthesizing means for synthesizing such harmonic components so that their time axes coincide with each other.

  In the time stretch system having such a configuration, first, a harmonic component having a harmonic structure and a non-harmonic component not having a harmonic structure are extracted from the components of the target music. After that, the non-harmonic component is rearranged at the position on the time axis that has been divided according to the time stretch magnification when each divided component is divided for each unit section along the time axis. Is time stretched according to the time stretch magnification so that the waveform expands and contracts along the time axis. Then, the non-harmonic component and the harmonic component that have been time-stretched by different methods are combined to perform time-stretching on the target music.

  As described above, in the above configuration, time stretching is performed to rearrange the position on the time axis of each component divided for each unit section with respect to the non-harmonic component of the components of the target music. A time stretch is performed on the component to expand and contract the waveform along the time axis.

  Specifically, the non-harmonic component is a component such as a percussion instrument sound whose level changes abruptly, and the change of the steep level is likely to appear in the unit interval. Even if the position is rearranged, time stretch with high reproducibility while maintaining the steep level change can be performed.

  On the other hand, the harmonic component is, specifically speaking, a sound-like component whose level changes slowly, and if such a component is divided into unit sections, such a change tendency tends to occur. As a result of not being maintained, there is a risk that unintentional sense of incongruity may occur.

  As used herein, “auditory discomfort” means that if the time stretch ratio is greater than 1, a silent portion is formed between each of the divided components in the unit section, and there is no sound in that section. It is generated by Further, if the time stretch ratio is less than 1, it occurs because the moderate level change is not maintained by the cross-fade process applied to the overlapping portions of the divided components in the unit section.

  To deal with such problems, the above configuration is not a method of rearranging the position on the time axis of each component divided for each unit section with respect to the harmonic component of the components of the target music, but expanding and contracting the waveform. The time stretch is carried out by the method to make it.

  With this time stretch, the waveform shape only expands and contracts along the time axis, so the harmonic component whose level changes slowly maintains the trend of change along the time axis while maintaining the time stretch. As a result, it is possible to prevent an unintended sense of incongruity from occurring.

  As described above, in the above configuration, the time stretch of the target music is converted to the non-harmonic component and the harmonic component by performing time stretch by a method suitable for each of the non-harmonic component and the harmonic component in the component of the target music. Each can be realized without any sense of incongruity in hearing.

  In this configuration, the method for extracting the harmonic component from the component in the target music is not particularly limited, but, for example, a spectrum obtained by frequency analysis of the waveform along the time axis indicating the component of the target music by Fourier transform or the like In the distribution, it is conceivable to extract, as a harmonic component, a component in which a spectral intensity peak appears periodically (having a harmonic structure).

In addition, as a specific configuration for time-stretching the harmonic component by expanding and contracting the waveform, for example, when the target music is represented by music data obtained by sampling a predetermined music at a predetermined sampling frequency Fs, better to a second like configuration shown.

  In this configuration, the frequency conversion means for frequency-converting the harmonic component extracted by the harmonic extraction means to the sampling frequency Fs · A obtained by integrating the sampling frequency Fs by the same magnification as the time stretch magnification A. , Is provided. The waveform expansion / contraction means time-stretches the harmonic component frequency-converted by the frequency conversion means so that the waveform expands and contracts along the time axis.

  With this configuration, the harmonic component sampling frequency is converted to the sampling frequency Fs · A obtained by integrating the same magnification as the time stretch magnification A, and then the harmonic component is time stretched. Can do.

  When the harmonic component is time stretched, if the time stretch magnification A is greater than 1, the frequency distribution in that component is narrowed to 1 / A (less than 1) times and the apparent sampling frequency is lowered, while the time stretch magnification is If A is less than 1, the frequency distribution in that component is expanded by 1 / A (greater than = 1) and the apparent sampling frequency is increased.

  In the above configuration, by changing the sampling frequency Fs to Fs · A prior to the time stretch of the harmonic component, the change in the apparent sampling frequency due to the time stretch can be offset. .

  For example, when the time stretch ratio A is greater than 1, the apparent sampling frequency is lowered to 1 / A (less than 1) times due to the time stretch, and this sampling frequency is set to the sampling frequency Fs · A in advance. By converting (up-sampling), the sampling frequency after time stretching can be returned to the original frequency Fs as a result (Fs · A · 1 / A = Fs).

  On the other hand, when the time stretch magnification A is less than 1, the apparent sampling frequency becomes as high as 1 / A (greater than 1) times due to the time stretch, and this sampling frequency is changed to the sampling frequency Fs · A in advance. By performing frequency conversion (down sampling), the sampling frequency after time stretching can be returned to the original frequency Fs as a result (Fs · A · 1 / A = Fs).

  By the way, when the time stretch magnification is less than 1 in the above configuration, the frequency distribution is expanded to 1/2 or more of the sampling frequency Fs by the time stretch (sampling frequency conversion and expansion / contraction along the time axis of the waveform). This may cause so-called aliasing noise.

In order for such noise is prevented from occurring, for example, it is conceivable to have as a third configuration shown below the structure.
In this configuration, when the time stretch magnification is less than 1, filtering means for removing a frequency component having a frequency Fc (= (A · Fs) / 2) or more defined from the component of the target musical piece, I have. Then, the frequency conversion means performs frequency conversion based on the target musical piece obtained by removing frequency components equal to or higher than the frequency Fc by the filtering means.

With this configuration, when the time stretch magnification is less than 1, a frequency component having a frequency Fc (= (A · Fs) / 2) or more is previously removed from the harmonic components.
Since the frequency band above the frequency Fc is a band in which the frequency distribution can be expanded to 1/2 or more of the sampling frequency Fs in the process of time stretching at the time stretch magnification A, the frequency component above the frequency Fc as in the above configuration. Is removed in advance so that frequency components do not interfere with each other in the frequency distribution expanded in the process of time stretching, thereby preventing the occurrence of aliasing noise due to the expansion of the frequency distribution.

Further, in order to aliasing noise described above does not occur, it is conceivable to have as a fourth configuration shown below the structure.
In this configuration, when the time stretch magnification is less than 1, the harmonic component extracted by the harmonic extraction means is multiplied by the same magnification as the reciprocal of the time stretch magnification A to the sampling frequency Fs. Pre-conversion means for converting the frequency to the sampling frequency Fs / A. When the time stretch magnification is less than 1, the waveform expansion / contraction means has the time stretch magnification such that the harmonic component frequency-converted by the prior conversion means is expanded and contracted along the time axis. Depending on the time stretch.

  Further, when the time stretch magnification is less than 1, the harmonic component formed by the time stretch by the waveform expansion / contraction means and the pitch shift by the pitch shift means has the same magnification as the time stretch magnification A at the sampling frequency. And post-conversion means for converting the frequency to a sampling frequency obtained by integrating. Then, the component synthesizing unit converts the harmonic component frequency-converted by the posterior converting unit into the non-harmonic component obtained by rearranging the divided non-harmonic component by the component rearranging unit, respectively. Synthesize so that the axes match.

  With this configuration, when the time stretch magnification is less than 1, the harmonic component sampling frequency is converted to 1 / A times in advance prior to the time stretch, and after the time stretch, the harmonic component sampling frequency is converted. Is frequency converted to A times.

  If the sampling frequency is frequency converted to 1 / A (greater than 1) times prior to time stretching, the frequency distribution will be expanded according to the magnification, but since the sampling frequency is only converted, the sampling is performed. The frequency components are not distributed in a partial band of 1/2 or less of the frequency Fs / A. As a result, even if the frequency distribution is expanded in the process of time stretching, the expanded frequency components do not interfere with each other, so that the occurrence of aliasing noise due to the expanded frequency distribution can be prevented.

  Then, the sampling frequency before and after the time stretch can be matched by frequency-converting the sampling frequency of the harmonic component again by A (less than 1) times after the time stretch.

In this configuration, the sampling frequency of the harmonic component is frequency-converted to 1 / A times in advance, and then frequency-converted to A times in the process of time stretching. Can be said to be offset. Therefore, for the series of frequency conversion it may be possible to omit the implementation. For this purpose, it is conceivable to make the fifth configuration in which as shown below the first configuration.

  In this configuration, when the target music is indicated by music data obtained by sampling a predetermined music at a predetermined sampling frequency Fs, the waveform expansion / contraction means is used when the time stretch magnification is less than 1. The harmonic component formed by the time stretch by the pitch shift and the pitch shift by the pitch shift means is the sampling frequency Fs ′ ″ (= Fs ″ · A = Fs) having the same sampling frequency Fs as the time stretch magnification A. And post-conversion means for frequency conversion to Then, the component synthesizing unit converts the harmonic component frequency-converted by the posterior converting unit into the non-harmonic component obtained by rearranging the divided non-harmonic component by the component rearranging unit, respectively. Synthesize so that the axes match.

With this configuration, it is possible to obtain the same effects as the fourth configuration while omitting a series of frequency conversions and reducing the processing load.
Incidentally, as described above, when time stretch ratio is less than 1, when time stretching of the non-harmonic component, may sixth configured as Configuring.

  In this configuration, when the time stretch magnification is less than 1, the component rearrangement means arranges the position on the time axis of each of the divided non-harmonic components at a position moved back and forth according to the time stretch magnification. At the same time, the divided subharmonic components adjacent on the time axis are cross-faded.

  With this configuration, even when the time stretch magnification is less than 1 and each of the subharmonic components (divided subharmonic components) divided for each unit section overlaps on the time axis, the divided non-adjustment The overlapping portions of the wave components can be connected by crossfading.

Further, in order to solve the above problem may be a program for causing a computer system as all means comprising pitch shift system in the first to sixth any one of the. If it is a computer system controlled by this program, the same operation and effect as the time stretch system in any one of the first to sixth configurations can be obtained.

  The above-described program is composed of an ordered sequence of instructions suitable for processing by a computer system, and is provided to a time stretch system or a user who uses this through various recording media and communication lines. Is.

Embodiments of the present invention will be described below with reference to the drawings.
(1) Hardware Configuration As shown in FIG. 1, the time stretch system 1 includes a control unit 11 that controls the entire system, a storage unit 13 that stores various information, a communication unit 15 that controls communication via the network 2, This is a known computer system including a user interface (U / I) unit 17 including a keyboard and a display, a media drive 19 for inputting and outputting information via a recording medium, and the like.

  When the time stretch system 1 receives a predetermined command from the outside via the user interface unit 17 or the communication unit 15, the control unit 11 executes various processes according to a program stored in the storage unit 13, thereby It functions as a time stretch system of the invention.

In the present embodiment, the time stretch system 1 is configured by a single computer system, but it is needless to say that it can be configured by a plurality of computer systems that operate in cooperation with each other.
(2) Processing by Control Unit 11 Hereinafter, various processing procedures executed by the control unit 11 according to a program stored in the storage unit 13 will be described.
(2-1) Time Stretching Process First, a time stretching process started when a prescribed operation is performed via the user interface unit 17 or when a command is received from the network 2 via the communication unit 15. The processing procedure will be described with reference to FIG.

  When this time stretching process is activated, first, music data (input music data) for reproducing a predetermined music (hereinafter referred to as “target music”) is acquired (s110). Here, the input music data designated through the operation on the user interface unit 17 is acquired from the storage unit 13 or the media drive 19 (that is, the recording medium) or received together with the command through the communication unit 15. Input music data is acquired.

  Next, for the target music indicated by the music data acquired in s110, the harmonic component of the components in the target music is extracted (s120). Here, in the spectrum distribution obtained by frequency analysis of the waveform along the time axis indicating the component of the target music by Fourier transform or the like, the peak of the spectrum intensity appears periodically (having a harmonic structure). ) Component is extracted and returned to a waveform along the time axis, whereby this waveform is extracted as a harmonic component.

  Next, for the target music indicated by the music data acquired in s110, the non-harmonic component of the components in the target music is extracted (s130). Here, by subtracting the harmonic component extracted in s120 from the component of the target music on the time axis, the component resulting from the subtraction removes the harmonic component from the component of the target music. Extracted as a wave component.

  Next, time stretching of the harmonic component extracted in s120 is performed by a processing procedure described later (s140). Here, the time stretch according to the time stretch magnification A is performed so that the waveform of the harmonic component expands and contracts along the time axis.

  Next, time stretching of the non-harmonic component extracted in s130 is performed (s150). Here, based on the non-harmonic component extracted in s130, each non-harmonic component is divided into unit sections along the time axis (hereinafter referred to as “divided non-harmonic component”). Time stretching is performed by a processing procedure such as rearranging each position on the time axis to a position moved back and forth according to the time stretch magnification A.

  In the case of the time stretch here, when the time stretch magnification is less than 1, the positions on the time axis of each of the divided non-harmonic components are rearranged, and the divided non-harmonic components adjacent on the time axis are Are connected by cross-fading.

  Next, the harmonic component time-stretched in s140 and the non-harmonic component time-stretched in s150 are combined as a waveform along the time axis (s160). In this way, the target music that has been time stretched by the time stretch magnification A is generated.

Then, music data (output music data) indicating the target music synthesized in s160 is returned as a response to the command that triggered the start of this pitch shift process (s170). Here, the output music data is stored in the storage area (the storage area in the storage unit 13 or the recording medium) designated through the operation on the user interface unit 17, or the input music data is transmitted through the communication unit 15. The output music data is returned to the transmission source that has transmitted.
(2-2) Time stretch implementation procedure “1” for harmonic components
Next, the procedure “No. 1” for performing time stretching on the harmonic component in s140 will be described with reference to FIG.

  Here, first, the “time stretch magnification (reproduction after time stretch with respect to the original reproduction time in the target music) designated through the operation to the user interface unit 17 or received together with the command via the communication unit 15 is performed. It is checked whether or not “time ratio” ”is less than 1 (s210).

  If it is determined in s210 that the time stretch magnification is less than 1 (s210: YES), among the frequency components included in the harmonic component extracted in s120, the time stretch magnification A and the harmonic component (or After the frequency component equal to or higher than the cutoff frequency Fc defined by the following equation 1 is removed from the sampling frequency Fs of the target music) (s220), the process proceeds to the next process (s230).

  In s220, a low-pass filter that removes a frequency component equal to or higher than the cutoff frequency Fc (= (A · Fs) / 2) defined by the time stretch magnification A and the sampling frequency Fs is generated, and the frequency is based on this. Component removal is performed.

As described above, when the time stretch magnification A is less than 1, a state in which some frequency components are removed by this s220 is shown in FIGS.
If it is determined in s210 that the time stretch ratio is not less than 1 (s210: NO), the process proceeds to the next process (s230) without performing s220.

  Next, the sampling frequency Fs of the harmonic component extracted in s120 or the harmonic component from which the frequency component is partially removed through s220 is integrated by multiplying the same magnification as the time stretch magnification A. The frequency is converted to the frequency Fs · A (s230).

  In this way, the state where the sampling frequency is frequency-converted is shown in FIG. 4C when the time stretch magnification A is less than 1, and when the time stretch magnification A is not less than 1, the state shown in FIG. , (B). As can be seen from these figures, this frequency conversion narrows or widens the frequency distribution to a frequency band of (Fs / 2) · A or less. Specifically, if the time stretch magnification A is less than 1, the frequency band is narrowed according to the value, and if it is not less than 1, the frequency band is expanded according to the value.

  Next, a time stretch corresponding to the time stretch magnification A is performed on the harmonic component subjected to frequency conversion in s230 so that the waveform expands and contracts along the time axis (s240).

  As described above, the time stretched state is shown in FIG. 4D when the time stretch magnification A is less than 1, and is shown in FIG. 5C when the time stretch magnification A is not less than 1. As can be seen from this figure, the frequency distribution expands or narrows as the time stretches along the time axis. Specifically, if the time stretch magnification A is less than 1, the frequency band is expanded according to the value, and if it is not less than 1, the frequency band is decreased according to the value.

  When the frequency band of the harmonic component is changed in this way, the sampling frequency is also changed to the frequency Fs · A / A (= Fs) corresponding to the time stretch magnification A accordingly.

  Then, the harmonic shift component subjected to the time stretch in s240 is subjected to a pitch shift so that the pitch is corrected to be low or high according to the time stretch magnification A (s250).

  Here, the pitch is corrected so that each frequency component in the harmonic component has a value obtained by integrating the same magnification as the time stretch magnification A. That is, if the time stretch magnification A is less than 1, the pitch is corrected so that the frequency band narrows to the low frequency side, whereas if the time stretch magnification A is not less than 1, the frequency band moves to the high frequency side. The pitch is corrected to expand.

Thus, the state in which the pitch is corrected is shown in FIG. 4E when the time stretch magnification A is less than 1, and is shown in FIG. 5D when the time stretch magnification A is not less than 1. .
(2-3) Time stretch execution procedure “2” for harmonic components
Subsequently, the implementation procedure “part 2” of time stretching for the harmonic component in s140 will be described. Note that the same processing steps as those of “No. 1” are denoted by the same reference numerals, and a detailed description thereof will be omitted.

  In this execution procedure, the points at which s230 to s250 are performed are the same as those of “No. 1”, but s230 is performed when s220 is not performed and the time stretch magnification A is less than 1. The difference is that the sampling frequency is frequency-converted before and after s250.

  Specifically, first, when it is determined in s210 that the time stretch magnification is less than 1 (s210: YES), the sampling frequency Fs of the harmonic component extracted in s120 is the time stretch. Frequency conversion is performed to a sampling frequency Fs / A obtained by integrating the same magnification as the reciprocal of the magnification A (s212).

FIG. 7B shows a state where the sampling frequency is frequency-converted when the time stretch magnification A is less than 1 as described above.
Thereafter, the frequency component (s230), the time stretch (s240), and the pitch shift (s250) are respectively applied to the harmonic components subjected to the frequency conversion in advance as in the case of “No. 1”. FIGS. 7C to 7E show a state in which frequency conversion is performed, a state in which time stretching is performed, and a state in which pitch shift is performed in this case.

  The sampling frequency Fs of the harmonic component subjected to the pitch shift by s250 is frequency-converted to a sampling frequency Fs obtained by integrating the same magnification as the time stretch magnification A (s252).

  In this case, the sampling frequency of the harmonic component becomes Fs / A by the frequency conversion of s212, becomes (Fs / A) · A (= Fs) by the frequency conversion of s230, and becomes Fs / A by the time stretch of s240. , S252 frequency conversion results in (Fs / A) · A (= Fs).

Thus, when the time stretch magnification A is less than 1, the state where the sampling frequency is frequency-converted is shown in FIG. As can be seen from this figure, the frequency conversion returns the frequency distribution to the same frequency band as the original harmonic component (the harmonic component extracted at s120).
(3) Action and Effect In the time stretch system 1 having such a configuration, first, a harmonic component having a harmonic structure and a non-harmonic component not having a harmonic structure are extracted from the components of the target music. (S120, s130 in FIG. 2). Thereafter, the non-harmonic component is rearranged at the position on the time axis that is divided according to the time stretch magnification when each divided component is divided for each unit section along the time axis (s150 in the figure). ), The harmonic component is time stretched according to the time stretch magnification so that the waveform expands and contracts along the time axis (s140 in the figure). Then, by combining the non-harmonic component and the harmonic component that have been time-stretched by different methods in this manner, time-stretching to the target music is performed (s160 in the figure).

  As described above, in the above embodiment, time stretching for rearranging the position on the time axis of each component divided for each unit section is performed on the non-harmonic component of the components of the target music piece, A time stretch is performed on the harmonic component to expand and contract the waveform along the time axis.

  Specifically, the non-harmonic component is a component such as a percussion instrument sound whose level changes abruptly, and the change of the steep level is likely to appear in the unit interval. Even if the position is rearranged, time stretch with high reproducibility while maintaining the steep level change can be performed.

  On the other hand, the harmonic component is, specifically speaking, a sound-like component whose level changes slowly, and if such a component is divided into unit sections, such a change tendency tends to occur. As a result of not being maintained, there is a risk that unintentional sense of incongruity may occur.

  In order to deal with such a problem, in the present embodiment, with respect to the harmonic component of the components of the target music, the waveform is not a method of rearranging the position on the time axis of each component divided for each unit section. Time stretch is performed by a method of expanding and contracting.

  With this time stretch, the waveform shape only expands and contracts along the time axis, so the harmonic component whose level changes slowly maintains the trend of change along the time axis while maintaining the time stretch. As a result, it is possible to prevent an unintended sense of incongruity from occurring.

  As described above, in the above-described embodiment, the time stretch of the target musical piece is performed by performing the time stretch by a method suitable for each of the non-harmonic component and the harmonic component in the component of the target musical piece. Each component can be realized so that there is no sense of incongruity in hearing.

  In the above embodiment, the harmonic component sampling frequency is converted to the sampling frequency Fs · A obtained by integrating the same magnification as the time stretch magnification A, and then the harmonic component is time stretched. (S230 to s240 in FIG. 3).

  When the harmonic component is time stretched, if the time stretch magnification A is greater than 1, the frequency distribution in that component is narrowed to 1 / A (less than 1) times and the apparent sampling frequency is lowered, while the time stretch magnification is If A is less than 1, the frequency distribution in that component is expanded by 1 / A (greater than = 1) and the apparent sampling frequency is increased.

  In the above embodiment, the sampling frequency Fs is frequency-converted to Fs · A prior to the time stretching of the harmonic component, thereby canceling the apparent sampling frequency change caused by the time stretching as described above. can do.

  For example, when the time stretch ratio A is greater than 1, the apparent sampling frequency is lowered to 1 / A (less than 1) times due to the time stretch, and this sampling frequency is set to the sampling frequency Fs · A in advance. If conversion (upsampling) is performed, the sampling frequency after time stretching will eventually return to the original frequency Fs (Fs · A · 1 / A = Fs) (FIGS. 4C to 4E). reference).

  On the other hand, when the time stretch magnification A is less than 1, the apparent sampling frequency becomes as high as 1 / A (greater than 1) times due to the time stretch, and this sampling frequency is changed to the sampling frequency Fs · A in advance. If frequency conversion (downsampling) is performed, the sampling frequency after time stretching will eventually return to the original frequency Fs (Fs · A · 1 / A = Fs) (FIGS. 5B to 5D). )reference).

In this way, the apparent change in sampling frequency due to time stretching can be canceled out.
Further, in the above-described embodiment, when the time stretch magnification is less than 1 in the time stretch execution procedure “part 1” for the harmonic component, the cutoff frequency Fc (= (A · Fs) in the harmonic component is previously set. / 2) The above frequency components are removed (see s220 in FIG. 3, FIG. 4B).

  Since the frequency band above the frequency Fc is a band in which the frequency distribution can be expanded to 1/2 or more of the sampling frequency Fs in the process of time stretching at the time stretch magnification A, the frequency component above the frequency Fc as in the above configuration. Is removed in advance so that frequency components do not interfere in the frequency distribution that has been expanded in the process of time stretching (see (c) in the figure), thereby preventing the occurrence of aliasing noise due to the expansion of the frequency distribution. it can.

  Further, in the above embodiment, in the time stretch execution procedure “part 2” for the harmonic component, when the time stretch magnification is less than 1, the sampling frequency of the harmonic component is 1 / A in advance prior to the time stretch. The frequency is doubled (s212 in FIG. 6), and after the time stretching, the sampling frequency of the harmonic component is frequency-converted to A times (s252 in FIG. 6).

  If the sampling frequency is frequency converted to 1 / A (greater than 1) times prior to time stretching, the frequency distribution will be expanded according to the magnification, but since the sampling frequency is only converted, the sampling is performed. The frequency components are not distributed in a partial band of 1/2 or less of the frequency Fs / A (see FIG. 7B). As a result, even if the frequency distribution is expanded in the process of time stretching, the expanded frequency components do not interfere with each other, so that the occurrence of aliasing noise due to the expanded frequency distribution can be prevented.

  Then, after the time stretch, the sampling frequency of the harmonic component is frequency-converted again by A (less than = 1) times (s252 in FIG. 6), so that the sampling frequencies before and after the time stretch can be matched (FIG. 7 ( f)).

Moreover, in the said embodiment, even when each of the subharmonic components divided | segmented for every unit area overlaps on a time axis, the overlapping part of these subharmonic components can be connected by cross fade. .
(4) Modifications The embodiment of the present invention has been described above. However, the present invention is not limited to the above embodiment, and can take various forms as long as they belong to the technical scope of the present invention. Needless to say.

  For example, in the above-described embodiment, in the time stretch execution procedure “part 2” for the harmonic component, after the harmonic component sampling frequency is converted to 1 / A times in advance (s212 in FIG. 6), the time stretch is performed. In this process, the frequency conversion is A times (s230 in the figure), and it can be said that the effect of both frequency conversions is canceled at that time. Therefore, the implementation of these series of frequency conversions may be omitted. For this purpose, as shown in FIG. 8, when it is determined “YES” in s210, the process may be shifted to s240 without performing s212 and s230.

In this case, it is possible to obtain the same operation and effect as the time stretching execution procedure “part 2” for harmonic components while reducing the processing load by omitting a series of frequency conversions.
In this case, FIG. 9 shows a time waveform (waveform along the time axis) and spectrum distribution in the process of time stretching. Here, FIG. 9B shows the state after the time stretch (s240 in FIG. 8), and FIG. 9C shows the state after the pitch shift (s250 in FIG. 8). FIG. 9D shows a state after the subsequent frequency conversion (s252 in FIG. 9) is performed.
(5) Correspondence with the Present Invention In the embodiment described above, s120 in FIG. 2 is the harmonic extraction means in the present invention, s130 is the non-harmonic extraction means in the present invention, and s150 in FIG. The component rearrangement means in the present invention, s240 in FIGS. 3, 6 and 8 is the waveform expansion / contraction means in the present invention, s250 in FIG. 2 is the pitch shift means in the present invention, and s160 in FIG. 2 is the component in the present invention. 3, s230 of FIG. 3, 6 and 8 is the frequency converting means in the present invention, s220 in FIG. 6 is the filtering means in the present invention, s212 in FIG. 6 is the pre-converting means in the present invention, 6 and s252 in FIG. 8 are post-conversion means in the present invention.

Diagram showing the hardware configuration of the time stretch system Flow chart showing time stretch processing Flowchart showing time stretch execution procedure “part 1” for harmonic components The figure which shows the process in which a harmonic component is time-stretched by the time stretch magnification of less than 1 in the implementation procedure "the 1" of time-stretching to a harmonic component. The figure which shows the process in which a harmonic component is time-stretched by the time stretch magnification which is not less than 1 in the implementation procedure "the 1" of time stretching with respect to a harmonic component. Flowchart showing time stretch execution procedure “part 2” for harmonic components The figure which shows the process in which the harmonic component is time-stretched by the time stretch magnification of less than 1 in the implementation procedure “part 2” of the time stretch with respect to the harmonic component. The flowchart which shows the implementation procedure "the 2" of the time stretch with respect to a harmonic component in another embodiment The figure which shows the process in which harmonic component is time-stretched by the time stretch magnification of less than 1 in the implementation procedure "the 2" of harmonic stretch in another embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Time stretch system, 2 ... Network, 11 ... Control part, 13 ... Memory | storage part, 15 ... Communication part, 17 ... User interface part, 19 ... Media drive.

Claims (5)

A time stretch system for time-stretching a target music with a predetermined time stretch magnification,
Harmonic extraction means for extracting a harmonic component having a harmonic structure among the components based on the spectral distribution of the components in the target music;
Non-harmonic extraction for extracting non-harmonic components that do not form a harmonic structure among the components of the target music by removing the harmonic components extracted by the harmonic extraction means from the components of the target music Means,
Based on the non-harmonic component extracted by the non-harmonic extraction means, each divided component when the non-harmonic component is divided into unit sections along the time axis (hereinafter referred to as “divided non-harmonic component”) Component rearrangement means for rearranging the position on each time axis to a position moved back and forth according to the time stretch magnification;
Waveform expansion / contraction means for time-stretching the harmonic component extracted by the harmonic extraction means according to the time stretch magnification so that the waveform expands and contracts along the time axis;
For harmonic component after being by Rita im stretch waveform expansion means, and the pitch shifting means for lower or higher, depending on the pitch of said harmonic components in the time stretch ratio,
The subharmonic components obtained by rearranging the divided subharmonic components by the component rearrangement unit are replaced with harmonic components obtained by time stretching by the waveform expansion / contraction unit and pitch shift by the pitch shift unit, respectively. Component synthesizing means for synthesizing so that the time axes match ,
In the case where the target music is indicated by music data obtained by sampling a predetermined music at a predetermined sampling frequency Fs,
The waveform expansion / contraction means converts the harmonic component extracted by the harmonic extraction means to a sampling frequency Fs · A obtained by integrating the sampling frequency Fs by the same magnification as the time stretch magnification A. The time-stretched harmonic component having undergone frequency conversion is stretched so that the waveform expands and contracts along the time axis.
Further, when the time stretch ratio is less than 1, the waveform expansion and contraction is performed after filtering out frequency components of a frequency Fc (= (A · Fs) / 2) or more defined from the components of the target music. A time stretch system comprising frequency component conversion means for performing frequency conversion and time stretch by the means . A time stretch system for time-stretching a target music with a predetermined time stretch magnification,
Harmonic extraction means for extracting a harmonic component having a harmonic structure among the components based on the spectral distribution of the components in the target music;
Non-harmonic extraction for extracting non-harmonic components that do not form a harmonic structure among the components of the target music by removing the harmonic components extracted by the harmonic extraction means from the components of the target music Means,
Based on the non-harmonic component extracted by the non-harmonic extraction means, each divided component when the non-harmonic component is divided into unit sections along the time axis (hereinafter referred to as “divided non-harmonic component”) Component rearrangement means for rearranging the position on each time axis to a position moved back and forth according to the time stretch magnification;
Waveform expansion / contraction means for time-stretching the harmonic component extracted by the harmonic extraction means according to the time stretch magnification so that the waveform expands and contracts along the time axis;
Pitch shift means for lowering or raising the pitch of the harmonic component according to the time stretch magnification for the harmonic component after being time stretched by the waveform expansion / contraction means;
The subharmonic components obtained by rearranging the divided subharmonic components by the component rearrangement unit are replaced with harmonic components obtained by time stretching by the waveform expansion / contraction unit and pitch shift by the pitch shift unit, respectively. Component synthesizing means for synthesizing so that the time axes match,
further,
When the time stretch magnification is less than 1, the harmonic component extracted by the harmonic extraction means is integrated with the sampling frequency Fs /, which is obtained by adding the same magnification as the reciprocal of the time stretch magnification A to the sampling frequency Fs. After frequency conversion to A, the harmonic component is subjected to time stretching by the waveform expansion / contraction means, and the harmonic component subjected to time stretching by the waveform expansion / contraction means and pitch shift by the pitch shifting means is After frequency conversion to a sampling frequency obtained by adding the same magnification as the time stretch magnification A to the sampling frequency, the harmonic component subjected to frequency conversion and the divided non-harmonic component are arranged by the component rearranging means. A frequency component change that causes the component synthesis means to synthesize the corrected non-harmonic component. Is equipped with means, the
A time stretch system characterized by that. A time stretch system for time-stretching a target music with a predetermined time stretch magnification,
Harmonic extraction means for extracting a harmonic component having a harmonic structure among the components based on the spectral distribution of the components in the target music;
Non-harmonic extraction for extracting non-harmonic components that do not form a harmonic structure among the components of the target music by removing the harmonic components extracted by the harmonic extraction means from the components of the target music Means,
Based on the non-harmonic component extracted by the non-harmonic extraction means, each divided component when the non-harmonic component is divided into unit sections along the time axis (hereinafter referred to as “divided non-harmonic component”) Component rearrangement means for rearranging the position on each time axis to a position moved back and forth according to the time stretch magnification;
Waveform expansion / contraction means for time-stretching the harmonic component extracted by the harmonic extraction means according to the time stretch magnification so that the waveform expands and contracts along the time axis;
Pitch shift means for lowering or raising the pitch of the harmonic component according to the time stretch magnification for the harmonic component after being time stretched by the waveform expansion / contraction means;
The subharmonic components obtained by rearranging the divided subharmonic components by the component rearrangement unit are replaced with harmonic components obtained by time stretching by the waveform expansion / contraction unit and pitch shift by the pitch shift unit, respectively. Component synthesizing means for synthesizing so that the time axes match,
In the case where the target music is indicated by music data obtained by sampling a predetermined music at a predetermined sampling frequency Fs,
When the time stretch magnification is less than 1, the harmonic component formed by the time stretch by the waveform expansion / contraction means and the pitch shift by the pitch shift means has the sampling frequency Fs of the same magnification as the time stretch magnification A. After frequency conversion to the sampling frequency Fs ′ ″ (= Fs ″ · A = Fs), the frequency-converted harmonic component and the divided non-harmonic component are rearranged by the component rearrangement means. Frequency component converting means for performing synthesis by the component synthesizing means with the non-harmonic component formed by
A time stretch system characterized by that. The component rearrangement unit rearranges the position on the time axis of each of the divided non-harmonic components to a position moved back and forth according to the time stretch magnification when the time stretch magnification is less than 1. The time stretch system according to any one of claims 1 to 3 , wherein the divided subharmonic components adjacent on the time axis are crossfaded. A program for causing a computer system to execute various processing procedures for causing all of the means according to any one of claims 1 to 4 to function.

Images