JPH07287576A - Musical sound reproducing method - Google Patents

Abstract

PURPOSE:To eliminate the discontinuity of an audio signal and smoothly continue a sound with an audio signal which resembles the original audio signal by correcting data which are reproduced in a specific transition period. CONSTITUTION:A transition frame consists of two frames succeeding to a last frame F8 which is not shifted, i.e., F8 which is inserted overlapping and following F9, and a period wherein a vocal signal belongong to those frames is the transition period. And, the original audio signal F(m) which corresponds to the transition period and is not shifted is faded out during the transition period, and an audio signal G(m) which is in the shifted frame and not smoothed yet is faded in the original frame string corresponding to the transition period, and an audio signal H(m) to be reproduced in the transition period is generated from the composite signal of the both.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of transposing and extending or compressing a reproduction time in reproducing recorded music, and more specifically, to a method of converting digitally recorded music information. The present invention relates to a method of reproducing by changing sex or changing reproduction speed.

[0002]

2. Description of the Related Art A conventionally known digital music reproducing device is provided with a device for reproducing recorded original music information by changing keys or tempo. Music information used in synthesizers and keyboards of electronic musical instruments is MID
It is used in the format specified in the I standard. This is, so to speak, a specification for generating musical tones, and since this kind of music information can be freely processed during computer processing, for example, the tempo can be freely changed without changing the tonality, or the tempo can be changed. It can be reproduced by changing the tonality without changing.

The music information recorded by the MIDI standard does not lose the recorded music information or the part thereof is not reproduced twice even if such a change is made. Can be completely reproduced. In other words, even if such a change is made, the reproduced music completely contains music information that is exactly the same as the original music information, so smooth, noise-free and sound-free music is obtained. Is played.

However, a problem arises when reproducing music information containing vocals. The music information of the vocal portion cannot be recorded in the MIDI standard, and is recorded in a digital form by sampling an analog sound wave waveform, in other words, a sound pressure waveform curve.

When reproducing this, the recorded digital signal sequence is decoded and converted into a sound pressure signal sequence, which is then demodulated by a digital-analog converter into an analog audio signal and amplified. Then, input it to the speaker and play the music.

Therefore, when changing the tempo of a piece of music by a conventionally known time stretching / compressing method, the music information to be reproduced is divided into frame trains at a constant minute time Δt, and the frame trains are divided every N frames. A method of dividing into one block and duplicating or thinning out music information of an arbitrary one frame in each block is adopted.

Since one frame of music information is inserted or thinned out, the frames constituting one block are (N ±
1) It becomes a frame, but if the music information of this block is played during the time (N ± 1) ・ Δt, the tempo can be changed without changing the tonality, and it is played during the time N ・ Δt. If possible, transposition can be performed without changing the tempo.

However, according to this method, the continuity of the sound pressure curve is impaired before and after the inserted or thinned frame, so that sound dropouts and noises are perceived, the sound becomes rough, and the sound quality deteriorates. Occurs.

However, for example, in a karaoke apparatus, it is necessary to change the tonality for reproduction without changing the tempo, and in the case of dance music, it is required to change the tempo without changing the tonality. .

[0010]

DISCLOSURE OF THE INVENTION In reproducing a digitally recorded vocal signal, it is possible to reproduce the vocal signal recorded with a MIDI signal with one touch at a time without damaging the sound quality. It would be very convenient if transposition and tempo adjustment were possible.

[0011]

The above object of the present invention is to:
An audio signal composed of a large number of sound pressure signal sequences to be reproduced is converted into a minute time Δt composed of M / 2 continuous sound pressure signals.
It is divided into every frame sequence, and the frame sequence is consecutive N
An audio signal belonging to an arbitrary one frame in each block is repeatedly inserted or thinned out, and the number of frames of the block is set to (N ± 1), 1) A new audio signal consisting of frames is time N · Δt, or (N ± 1) · Δt
In the music playback method, in which the tonality of the original audio signal is changed or the playback speed is changed by playing back during playback, even if repeated insertion or decimation is performed, the original frame sequence is shifted. An audio signal for two frames following the last frame of the unsuccessful frame sequence or for a plurality of frames consisting of the two frames and at least one frame consecutive thereto (hereinafter, these two frames or a plurality of frames are referred to as transition frames), This is achieved by replacing the audio signal calculated by the following formula 1 and reproducing. It is recommended that the number of transition frames be 2 frames or 4 frames, each consisting of 1 frame continuous before and after the transition frame.

[Equation 4] Here, m = 0, 1, 2, 3, ..., M-1, MH (m) is an audio signal of a transition frame processed by the method of the present invention. F (m) is an unshifted original frame sequence audio signal corresponding to a period in which H (m) is reproduced (hereinafter referred to as a transition period). K
_out (m) is a fade-out function in which K _out (0) = 1 K _out (m)> K _out (m + 1) K _out (M) = 0. G (m) is the audio signal of the frame corresponding to the transition period of the new frame sequence shifted with respect to the original frame sequence. K _in (m) _{is, K in (0) = 0} K in (m) <K in (m + 1) fade-in function, which is a K _in (M) = 1.

Fade-out function K _out used in the present invention
In (m) and the fade-in function K _in (m) [hereinafter, both are simply referred to as a fade function K (m)], m is a discontinuous digital variable, but this function K ( It is desirable that m) be a function that can be differentiated with respect to m when m is any positive number that is not larger than M, and that its first derivative value does not become excessively large and that changes monotonously and gradually.

Thus, the most desirable fade function K
(M) is

[Equation 5] Is.

Since the time Δt per frame is extremely short, generally, F (m) ≈G (m). Therefore, if Equation 2 is adopted, K _out (m) + K _in (m) = 1, so that H (m) = F (m) * K _out (m) + G (m) * K.
_in (m) ≈ F (m) * [K _out (m) + K _in (m)] ≈ F (m) ≈ G (m).

In this way, during the transition period which is strongly affected by repeated insertion or decimation of frames, F
(M) is smoothly faded out, and G
Since (m) is smoothly faded in, and the sound pressure curve during the fade period is naturally and smoothly continuous, defects due to repeated insertion or thinning of frames are eliminated and the sound quality is improved. .

Also, a linear fade function, ie,

[Equation 6] Therefore, it is also recommended to perform fade-out and fade-in.

This method has the advantage that the fade operation is simple, and the effect of repeated insertion or decimation of frames can be discerned from the difference from the case where the function of the above-mentioned expression 2 is adopted. Are extremely few.

Furthermore, as these fade functions, it is possible to adopt a monotonically and gradually decreasing or increasing function that is approximate to the above equations 1 and 2, respectively.

As a modification, for example, it is possible not to perform fade-out and fade-in at the beginning and end of the transition engine, but to perform it only during the intermediate period, or as a fade function. It is composed of a combination of known functions such as a quadratic function, an exponential function, a logarithmic function and a trigonometric function, and K _out (0) = 1 K _out (m)> K _out (m + 1) K _out (M) = 0 and K _in (0) = is _{0 K in (m) <K} in (m + 1) be capable of adopting a gradual monotonic decreasing function and monotonically decreasing function which satisfies K _in (M) = 1.

[0020]

BEST MODE FOR CARRYING OUT THE INVENTION The method of the present invention will be described below with reference to the drawings. In the following description, analogy when using other fade functions is considered to be extremely easy, so only the example in which the above expression 2 is adopted as the fade function will be described.

FIG. 1 is a block diagram showing the configuration of a music reproducing apparatus for carrying out the method of the present invention, FIG. 2 is a time chart showing an audio signal sequence recorded in a memory device, and FIG. 3 is an audio signal shown in FIG. A time chart showing a new audio signal sequence formed by overlappingly inserting one frame of audio signal sequence into the sequence, FIG. 4 is a graph showing a fade function, FIG. 5 is a smoothing according to the present invention to the audio signal sequence shown in FIG. A time chart showing how to perform processing,
6 is a time chart similar to that shown in FIG. 2, and FIG. 7 is a time chart showing a new audio signal sequence formed by thinning out one frame of the audio signal sequence from the audio signal sequence shown in FIG. 8 is a time chart showing a method for performing the smoothing processing according to the present invention on the audio signal sequence shown in FIG. 7, and FIG. 9 is a block diagram showing the configuration of the vocal signal processing device.

The device shown in FIG. 1 can be used as a reproducing device for karaoke and dance music.
Stores MIDI data, for example, CD-RO
M, memory device such as hard disk, 2 reproduction condition setting device, 3 MIDI information processing device, 4 vocal signal processing device, 5 and 6 D / A converter, 7 microphone,
8 is a mixer, 9 is an amplifier, 10 is a speaker, 11 is a video disc player, 12 is a telop controller, 13 is a priority circuit, and 14 is a video display.

In the present embodiment, when the device is used as karaoke, a memory device storing music information to be reproduced and karaoke telop information to be displayed in synchronization with the music information, and a display as a background of the telop. VD that records video information to be used is used,
The video information is displayed on the video display 14. When this device is used as a dance music reproducing device, a memory device in which only music information is recorded is used, and video-related equipment such as the video display 14 is not used.

Now, assuming that the device is used as karaoke, a case where the tonality is transferred will be mainly described. Music information including MIDI information and vocal signals and karaoke telop information are recorded in the memory device 1 to be used, and the information is first read by the memory device 1.

In the music information, the part related to the MIDI information is sent to the MIDI information processing device 3, the vocal signal is sent to the vocal signal processing device 4, and the karaoke telop information is sent to the telop controller 12.

Conditions for reproducing music, that is, tonality and reproduction speed, are set by the reproduction condition setting unit 2. The playback condition setting device 2 is provided with a tempo setting dial 21 and tonality setting keys 22 to 26. The tempo of music played by the tempo setting dial 21 can be designated, and the tonality setting keys 22 to By pressing any of the keys 26, the keys can be moved up and down in semitone steps.

When the tempo setting dial 21 is set to the central 0 position, the performance is performed at the standard tempo, but when the dial is turned to the minus side, the playing tempo becomes slower, and when it is turned to the plus side, the playing tempo is made faster. .

More specifically, when the original key setting key 24 is pressed and placed, the performance is performed in the original tone, but when 23 and 22 are pressed, the performance is performed in semitones with high tonality.
If you press 5, 26, the performance will be performed in semitones with low tonality. Thus, these set value signals are stored in the memory device 1, M
It is sent to the IDI information processing device 3, the vocal signal processing device 4 and the telop controller 12, whereby the operation of each device is controlled.

In the MIDI information processing device 3, the MIDI information is decoded at a desired reproduction speed by a known method, and its output is converted into a direct current signal by the D / A converter 5 and sent to the mixer 8.

The vocal signal in the music information is processed by the vocal signal processing device 4 by the method of the present invention described later and transmitted to the D / A converter 6 in synchronization with the output from the MIDI information processing device 3. And is converted into a direct current signal by this, and sent to the mixer 8. The input signals of these mixers 8 are mixed with the input from the microphone 7, sent to the speaker 10 via the amplifier 9, and reproduced as karaoke music.

In synchronization with this karaoke music, a telop signal is sent from the telop controller 12 to the priority circuit 13, where the telop signal is superimposed and combined with the back scene information sent from the video disc player 11, and sent to the video display 14 for reproduction. To be done.

The vocal signal processing device 4 and the music information processing method using the same will be described below. The vocal signal to be processed is recorded as digital information in the memory device 1, and this signal is read by the memory device 1 and sent to the vocal signal processing device 4.
In FIG. 2 and subsequent figures, this signal is shown as an analog sound pressure signal for easy understanding.

These signals are divided into frames of time ΔT, are divided into blocks of N frames, and are processed for each block. That is, one block of these signals contains N frames, that is, frames F ₁ , F ₂ , F ₃ , ... F ₇ , F ₈ , F ₉ , F.
₁₀ , F ₁₁ , F ₁₂ , ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ F _N-1 and F _N
And each frame contains M / 2 digital vocal signals. These digital vocal signals indicate the sound pressure of the reproduced sound wave, and the voice coil of the speaker 10 is driven by a current proportional to the sound pressure signal.

FIG. 2 shows that among these frames, F ₇ , F
_{8, F 9, F 10,} F 11 and F ₁₂ are shown. Thus, the first embodiment described below has 1 in 1 block.
This is an example of duplicating, duplicating, and inserting data for one frame, and the second embodiment is an example of deleting data for one frame.

The overlapped insertion or decimation processing of such a frame is performed by the vocal signal processing device 4. The vocal signal processing device 4, as shown in FIG.
A CPU 41, a buffer RAM 42 capable of recording at least (N + 1) frames of music information, a working RAM 43, an output buffer RAM 44, a fade ROM 44 recording a fade function, and an arithmetic circuit 45.
It consists of and.

The digital music information output from the memory device 1 is first input to the buffer RAM 42. When the data for one block is stored in the buffer RAM 42, the data is transferred to the working RAM 43. At this time, the address control controls the overlapping insertion of the data of the frame F ₈ and the shifting of the frames after the frame F _9. .

Thus, ΔT is a short time of about several tens of microseconds, and the sound pressure waveforms included in the adjacent frames are actually close to each other, but here the effect of the present invention is clarified. For the sake of illustration, a sharp waveform change is shown.

First, the first embodiment will be described.
For example, when the frame F ₈ is duplicated, a vocal signal string as shown in FIG. 3 is obtained, and these vocal signals are loaded into the working RAM 43.

In this case, the frames from the frame F ₁ to the original frame F ₈ are not shifted, but the frames from the inserted second frame F _{8 to} the last frame F _N are the frame F ₈ is shifted backwards on the time axis. Thus the transition frames in this case, the shift have not been the last two frames following the frame F _8, i.e., it continues F ₈ was duplicated inserted F
_The period is ₉ and the period during which the vocal signals belonging to these frames are reproduced is the transition period.

Therefore, the gist of the present invention is to eliminate the discontinuity of the audio signal sequence as shown in FIG. 3 and to use the audio signal having a waveform that approximates the sound to the original audio signal sequence. In order to make the sound smoothly continuous, the data reproduced during this transition period is modified.

This modification means that the original unshifted audio signal F (m) corresponding to the transition period is phased out during the transition period, and the original frame sequence corresponding to the transition period is changed to the original frame sequence. On the other hand, the audio signal G (m) that has not been smoothed in the shifted frame is phased in, and the composite signal of the two is used as the audio signal H (m) to be reproduced within the transition period.

That is, the signal train shown in FIG. 2 is a vocal signal output from the memory device 1, and the frames F ₉ and F ₁₀ therein are the original frames which are not yet shifted corresponding to the transition period. Column, F (m) is the audio signal of the original frame sequence F ₉ and F ₁₀ .

The signal sequence shown in FIG. 3 is obtained by duplicating and inserting F _{8 of} the signal sequence shown in FIG. Since the second frame F ₈ is inserted, the frame for transition to F ₉ is 1
The frame is shifted, so this block is (N +
1) It becomes a frame. The data for this (N + 1) frame is temporarily stored in the working RAM 43.

Of the signal sequence shown in FIG. 3, the second frame F ₈ and the subsequent frame F ₉ are new shifted frame sequences corresponding to the transition period, and
(M) is an audio signal sequence of the frame sequences F ₈ and F ₉ .

Thus, F (m) is multiplied by the fade function K _out (m) shown in FIG. 4, and G (m) is multiplied by the fade function K _in (m) shown in FIG. The sum of these products is set to H (m), and part of the data in the working RAM 43, that is, the data for the transition frame, is rewritten to this H (m). The above products and their sums are shown in FIG.

Thus, the fade function here is

[Equation 7] Is.

After this conversion is done, the working RAM
The data of 43 is timely, that is, the output buffer RAM 4
When 4 becomes empty, it is sent to the buffer RAM 44,
It is then serially output and sent to the D / A converter 6.

Therefore, when the above conversion is performed for the purpose of transposition, not for tempo conversion, the time required to read one block of data from the memory device 1 at the standard read speed, that is, Buffer RA within NΔT time
One new block loaded in M44, that is, (N
The data for +1) frames are output. In that case, the frequency of the reproduced sound is (N + 1) / N times.

Contrary to the above, when the purpose is not the transposition but the tempo conversion, the data read speed from the memory device 1 is lowered to the standard read speed N / (N + 1). Is. Then, the data of one new block loaded in the buffer RAM 44, that is, the data of (N + 1) frames is multiplied by the time (N + 1) ΔT and output.

In such a case, the audio signal train for one block which should be output at the time of NΔT is originally (N + T).
1) Since it is stretched to ΔT and outputted, the tempo of the reproduced music falls to N / (N + 1). When the vocal signal sequence shown in FIG. 5 and the vocal signal sequence shown in FIG. 2 are compared, they are extremely close to each other,
Moreover, it is apparent that the contour lines of the audio signal sequence during the transition period are smoothly connected to the contour lines of the audio signal sequences before and after that, regardless of the waveforms of F ₈ and F ₉ .

Next, a second embodiment shown in FIGS. 6 to 8 will be described. This is an example of thinning out frames to speed up the tempo of a song or lower the tonality. The vocal signal train shown in FIG. 6 is similar to that shown in FIG. However, in this example, since the frame F ₈ is cut, the frames corresponding to the transition period are F ₈ and F ₉ , and F (m) is the audio signal sequence of these frames F ₈ and F ₉ .

FIG. 7 shows a state in which the frame F ₈ is cut and the frames after the frame F ₉ are shifted. The frames corresponding to the transition period are F ₉ and F ₁₀ , and G (m) is the audio signal sequence of these frames F ₉ and F ₁₀ . In this case, the number of frames of one new block is (N-1).

The fade-out and fade-in methods are the same as in the above case. That is, also in this case, F (m)
Is multiplied by the fade-out function K _out (m) shown in FIG. 4, and G (m) is multiplied by the fade-in function K _in shown in FIG.
(M) is multiplied and the sum of their products is H (m),
A part of the data in the working RAM 43, that is, the data for the transition frame is rewritten to this H (m). The above products and their sums are shown in FIG.

Therefore, also in this case, the most desirable fade function is

[Equation 8] Is.

The data in the working RAM 43 is timely, that is, when the output buffer RAM 44 becomes empty.
The data is sent to the buffer RAM 44, serially outputted, and sent to the D / A converter 6.

Thus, when the above conversion is intended not for tempo conversion but for transposition, the time required to read one block of data from the memory device 1 at the standard read speed, that is, within NΔT time In addition, the data for one new block loaded in the buffer RAM 44, that is, the data for (N-1) frames is output.

In that case, the frequency of the reproduced sound is (N
+1) / N times. On the contrary to the above, when the purpose is not the transposition but the tempo conversion, the memory device 1
Data read speed from N is the standard read speed N / (N + 1)
It is to be lowered to. And the buffer RAM
One new block loaded in 44, that is, (N +
1) Data for one frame is multiplied by time (N + 1) ΔT and then output.

In such a case, the audio signal sequence for one block, which should originally be output at the time of NΔT, is time (N +
1) Since it is stretched to ΔT and outputted, the tempo of the reproduced music falls to N / (N + 1).

When the vocal signal train shown in FIG. 5 and the vocal signal train shown in FIG. 2 are compared with each other, the two are very similar and the waveforms of F ₈ and F ₉ are similar. It is clear that even a person with a smooth line can smoothly connect the contour line of the audio signal sequence during the transition period to the contour lines of the audio signal sequence before and after the transition period.

The configuration of the present invention is not limited to the above-described embodiment, and frame thinning-out or duplicate insertion may be performed in two or more places for one block, or in several blocks. It may be set at one place. It is also possible to perform transposition and tempo change at the same time.

The fade function curve also has the above-mentioned K (m).
Is mathematically equivalent to the above, and also includes a similar function that has the same effect from a technical point of view and can achieve the object of the present invention, and the number of transition frames is 2 or more, For example, it may be 4 or 6, and the present invention includes all of those modifications.

Further, the hardware used for implementing the present invention is not limited to the one described above, and a suitable C
With the PU, it is possible to carry out the method of the present invention with almost only software, and the present invention includes all of them.

[0063]

Since the present invention is constructed as described above, according to the present invention, even if the tempo of the music to be reproduced is changed and / or transposed, the vocal sound recorded in the music is recorded. But without compromising its sound quality, MIDI
It is reproduced in synchronization with the music information recorded by the signal.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a music reproducing device for implementing the method of the present invention.

FIG. 2 is a time chart showing an audio signal sequence recorded in a memory device.

FIG. 3 is a time chart showing a new audio signal sequence formed by overlappingly inserting one frame of audio signal sequence into the audio signal sequence shown in FIG.

FIG. 4 is a graph showing a fade function.

5 is a time chart showing a method for performing a smoothing process according to the present invention on the audio signal sequence shown in FIG.

FIG. 6 is a time chart similar to that shown in FIG.

7 is a time chart showing a new audio signal sequence formed by thinning out an audio signal sequence for one frame from the audio signal sequence shown in FIG.

8 is a time chart showing a method of performing a smoothing process according to the present invention on the audio signal sequence shown in FIG.

FIG. 9 is a block diagram showing a configuration example of a vocal signal processing device.

[Explanation of symbols]

 1 ... Memory device 2 Playback condition setting device 3 MIDI information processing device 4 Vocal signal processing device 5, 6 ... D / A converter 7 ... Microphone 8 ... Mixer 9 ... Amplifier 10 ・ ・ ・ ・ ・ ・ Speaker 11 ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ Video disc player 12 ・ ・ ・ ・ ・ ・ ・ Telop controller 13 ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ ・ Priority circuit 14 ..... Video display

Claims (3)

[Claims] 1. An audio signal consisting of a large number of sound pressure signal sequences to be reproduced is divided into frame sequences of M / 2 continuous sound pressure signals at every minute time Δt, and the frame sequences are consecutive N. Organized into a block row consisting of individual frames,
The audio signals belonging to any one frame in each block are repeatedly inserted or thinned out, the number of frames of the block is (N ± 1), and a new audio signal composed of the (N ± 1) frames is time N・ Δt or (N ±
1) · In the music playback method in which the tonality of the original audio signal is changed or the playback speed is changed by playing back during Δt, the original frame after repeated insertion or thinning is performed. Frames that have not been shifted with respect to the sequence of two frames following the last frame of the sequence or a plurality of frames consisting of the two frames and at least one frame that is continuous thereto (hereinafter, these two frames or multiple frames are referred to as transition frames) The above-mentioned music reproducing method, characterized in that the audio signal is replaced with an audio signal calculated by the following expression 1 to perform reproduction. [Equation 1] Here, m = 0, 1, 2, 3, ..., M-1, MH (m) is an audio signal of a transition frame processed by the method of the present invention. F (m) is an unshifted original frame sequence audio signal corresponding to a period in which H (m) is reproduced (hereinafter referred to as a transition period). K
_out (m) is a fade-out function in which K _out (0) = 1 K _out (m)> K _out (m + 1) K _out (M) = 0. G (m) is the audio signal of the frame corresponding to the transition period of the new frame sequence shifted with respect to the original frame sequence. K _in (m) _{is, K in (0) = 0} K in (m) <K in (m + 1) fade-in function, which is a K _in (M) = 1. 2. A fade-out function K _out (m) and a fade-out function K _in (m) are respectively expressed as follows: The method for reproducing music according to claim 1, wherein 3. A fade-out function K _out (m) and a fade-out function K _in (m) are respectively represented by the following equations. The method for reproducing music according to claim 1, wherein