JP2000194384A - System and method for recording and synthesizing sound, and infrastracture for distributing recorded sound to be reproduced on remote place - Google Patents

Abstract

PROBLEM TO BE SOLVED: To record sound with high quality and independently of a sampling speed by discriminating selected structure out of common spectral structure and common formant structure in a frame and generating a record including its basic frequency and selected frequency. SOLUTION: After executing sampling allowed to be generated at each 1 ms interval by a sampler 120, basic frequency is extracted by a frame generator 130. The frame generator 130 generates a frame and extracts the basic frequency and the envelope of a spectrum from a sound source 110 through the sampler 120. Then a frame analyzer 140 discriminates selected structure out of common spectral structure and common formant structure in the frame and generates a record including the selected structure and suitable basic frequency. Then the generated record is stored in a 1st storage unit 150.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

FIELD OF THE INVENTION The present invention relates generally to sound recording and playback, and more particularly, to a resolution independent system and method for creating a sound record and subsequently using the record to synthesize the sound.

[0002]

BACKGROUND OF THE INVENTION Current recordings of musical instrument performance are based either on sampling the analog signal of the instrument or recording gestures that are inputs to a controller. This can only be edited in a fixed time domain if the performance is sampled in a playback situation, or the instrument's digital interface (Musical Instrument Digital Inst.
erface: MIDI), only the keyboard and percussion will actually be recorded. Additional, spectral synthesis techniques divide a musical performance into discrete notes as opposed to a continuous performance. In general, the ability to change tempo or key and synchronize performance with external events during playback is achieved at a reasonable cost and often difficult to achieve without unacceptable distortion.

[0003] The generated sound waveform can be characterized by a number of parameters, such as frequency and amplitude. A sound waveform can be represented in the frequency domain as a single spectral frame composed of spectral components using Fourier analysis. The spectral frame contains the lowest or fundamental frequency of the waveform together with its harmonics (spectral components occurring at multiples of the fundamental frequency). Spectral components from stringed instruments or from vowels in speech usually occur near integer multiples of the fundamental frequency, while spectral components from percussion instruments occur at non-integer multiples of the fundamental frequency. I do.

It has been found that the recording of current sounds is at a typical sample rate that is dependent on or prone to unacceptable cost and other recording and playback characteristics that make changes to the distortion level recording. ing. In addition, these limitations make current recordings very restrictive of the format selected for playback. Radio stations offer a choice of recordings that can be programmed for many days into the future, but this is usually only allowed by the ad-hoc, specific and demanding capabilities of the telephone. Even in the case of the selected request, the recording has a completely fixed tempo and key format, as well as its basic arrangement.

[0005] Therefore, what is needed in the art is a high quality, sample rate independent, selectable in any convenient manner that has the ability to modify or adapt to a particular listener. A way to provide recordings.

[0006]

SUMMARY OF THE INVENTION To address the above-mentioned shortcomings of the prior art, the present invention provides for sound-independent resolution-independence.
It provides a system and method for recording and synthesizing in an ndent manner, and an infrastructure for distributing resolution-independent recordings for remote playback. In one embodiment,
One system includes (1) a frame generator that extracts a fundamental frequency and a spectral envelope from a sound and generates a frame therefrom; and (2) a common spectral structure and a common formant structure in the frame. A frame analyzer that identifies the selection and generates a record containing the fundamental frequency and the selected frequency.

Therefore, the present invention records the fundamental frequencies and selected structures in a sound, and generates a record containing these fundamental frequencies and the selected structures, and subsequently synthesizes (plays) them. Introduces the broad concept of providing the basis for). Preferably, the present invention analyzes the sound as a continuous play (independent of individual tones or notes).

In one embodiment of the invention, the frames are discrete, and they can correspond to discrete periods of time. In one embodiment shown and described, a common spectral or common formant structure may be included in the dictionary to compress the total size of the record.

In one embodiment of the invention, a musical instrument produces the sound. Those skilled in the art are familiar with the contents of certain musical instruments, for example, formants such as stringed and wind instruments. The human voice also contains formants that can be captured and adopted in later synthesis. However, the present invention can be applied to any sound.

In one embodiment of the invention, after the frame generator samples the sound, it extracts its fundamental frequency from them. In this embodiment illustrated and described, sampling can occur at 1 ms intervals. However, those skilled in the art will appreciate that the present invention is not limited to a particular sampling frequency.

In one embodiment of the invention, the system further includes a mapping circuit that applies a temporal quantization map to the record. Once that record has been generated, the present invention applies a wide range of sound manipulation techniques that have been or will be developed in the future.

[0012] In one embodiment of the invention, the system further includes an editor that changes selected contents and order of the records. This editor allows you to further manipulate the sound once recorded.

In one embodiment of the invention, the frame analyzer identifies selected ones of the common spectral structure and the common formant structure by Fourier analyzing the frame. Those skilled in the art are particularly familiar with fast Fourier transform techniques that can analyze frequencies. The present invention is compatible with other conventional or later developed spectral analysis techniques, such as wavelets.

The present invention further provides an infrastructure for distributing recordings for remote playback. One infrastructure is included in the recording database, which includes (1) a radio station with an associated recording database, and (2) selected corresponding common spectral and common formant structures. A plurality of recordings; (3) a request receiver coupled to the recording database for receiving a remote request for one of the plurality of recordings; and (4) one of the plurality of recordings in response to the request. And a transmitter coupled to the recording database.

Accordingly, the present invention provides an audio-on-
It provides what is substantially equivalent to "demand", in which a formatted audio file is provided to a remote "radio", which can synthesize audio on the fly. Thus, in one embodiment of the invention, the infrastructure further includes a plurality of remote radios that can receive any of the plurality of recordings and operate digitally. The remote radio may include software that is downloaded and executed on data processing and storage hardware to enable one of the plurality of recordings to be played. This infrastructure is in stark contrast to conventional analog AM or FM radio infrastructure, which simply demodulates and amplifies radio waves received by a remote radio.

In one embodiment of the invention, the transmitter broadcasts one of the plurality of recordings to the receiver. Alternatively, any of the multiple recordings can be addressed to individual remote "radios".

In one embodiment of the present invention, any of the plurality of recordings is implemented in a plurality of bitstream files. The bitstream file contains data about the fundamental frequency and the one selected above.

In one embodiment of the invention, the recording database includes a record of the request. It can track song popularity and advertising dissemination and automatically calculate the exact royalty payment.

The foregoing has outlined rather broad, preferred, and alternative features so that those skilled in the art can better understand the detailed description of the invention that follows. Additional features of the invention will be described hereinafter which form the subject of the claims of the invention. Those skilled in the art can readily use the concepts and specific embodiments disclosed herein as a basis for designing or modifying other structures to accomplish the same purpose as the present invention. I want to be understood. Those skilled in the art will also realize that such equivalent constructions in their broad form do not depart from the spirit and scope of the invention.

[0020]

DETAILED DESCRIPTION OF THE INVENTION Referring first to FIG. 1, a method for recording an instrument constructed in accordance with the principles of the present invention is described.
A block diagram 100 of a resolution-independent system is shown. The block diagram 100 of the resolution-independent system includes a sound source 110, a sampler (s
ampler) 120, a frame generator 130, a frame analyzer 140, a first storage unit 150, a mapping circuit 160, an editor 170, and a second storage unit 180.

The present invention provides a system and method for recording and synthesizing sound in a resolution-independent manner, and an infrastructure for distributing resolution-independent recordings for remote playback. In this embodiment, an instrument that generates the sound can be used. Those skilled in the art are familiar with the formant content of certain instruments, such as stringed and wind instruments. The human voice also contains certain formants, which can be captured and employed in later synthesis used in recording playback. However, the present invention can be applied to any sound.

After the sampling, which can occur at 1 ms intervals, has been performed by sampler 120, the fundamental frequency is extracted by frame generator 130. However, those skilled in the art will appreciate that the invention is not limited to the use of any particular sampling frequency or separate samplers as shown in the figures. Sampler 120 may be included as part of frame generator 130. This embodiment allows the data containing the sound source to be independent of the sampling rate or the desired tempo. Frame generator 130 generates a frame and extracts the fundamental frequency and spectral envelope from sound source 110 through sampler 120. Next, the frame analyzer 130 identifies a selected one of the common spectral and common formant structures in the frame and generates one record containing the selected and the appropriate fundamental frequency. I do. This is then stored in the first storage unit 150.

Accordingly, the present invention stores the fundamental frequencies and selected structures in a sound and generates a record containing those fundamental frequencies and selected structures for subsequent synthesis used in playback. Introduce the broad concept of providing a basis for. The present invention analyzes the sound as a continuous performance independent of individual tones or notes. The frames can be made discrete so that they correspond to discrete time periods. The frame analyzer 140 identifies the selected one of the common spectral structure and the common formant structure by Fourier analysis of the frame. Those skilled in the art are familiar with fast Fourier transform (FFT) techniques that can analyze frequencies. The invention can be used with other conventional techniques or later developed spectral analysis techniques, such as wavelets.

In this embodiment, a common spectral or formant structure can be included in the dictionary to compress the total size of the record. The identification and grouping of common spectral and formant structures organizes them into a record structure that can be accessed in a manner similar to words in a particular dictionary used to reconstruct a particular sound composition. The dictionary can be made to be a custom collection of common structures associated with the sounds sampled, framed and analyzed. However, the dictionary also
It may also include the common structure of a broader collection of sound sources that are recognized and tagged to correspond to the particular sound source to be directed. Further, it may be a collection, such as a dictionary, containing appropriate spectral and formant structures that are recognized and tagged.

Block diagram of system independent of resolution
00 further includes a mapping circuit 160 that applies time quantization to the record. Once the record has been generated, the present invention applies a wide range of conventional and later developed sound manipulation techniques. The quantization map of time allows the records to be synthesized by the ability to change the tempo or key, thereby producing different "feel
Provides "feel factors". It also provides the ability to synchronize playback performance with an external clock when other factors require.

[0026] The resolution independent system of block diagram 100 further includes an editor 170 that can further manipulate the sound once recorded. The editor 170 can change the recorded contents or order. If necessary, to provide effects not included in the original sound source,
The spectral structure or formant structure can be changed. The editor 170 also displays the frequency contour (fr
It can also be used to reconstruct a sequence recorded relative to the time of the frequency contour. The resolution-independent system then stores these mapped and edited alternative records in the second storage unit 180.

Referring now to FIG. 2, there is shown a flowchart 200 of a resolution-independent method of recording an instrument that can be performed in the system of FIG. Flowchart 200 illustrates an example of a method for recording a sound, which includes extracting a fundamental frequency and spectral envelope from the sound, and generating a frame from the fundamental frequency and spectral envelope. Next, one of the common spectral and common formant structures in the frame is identified and selected to generate a record that includes both the fundamental frequency and the selected frequency.

The method begins at step 205, where a decision to create a record is made, and a sound source is selected at step 210, the sound source including the instrument that produces the sound. After the sound is sampled in step 215, a fundamental frequency is extracted from the sampled sound signal. A frame (which may be discrete) is then generated at step 220 from which the fundamental and spectral envelopes are extracted. Next, these frames are analyzed at step 225 using Fourier analysis, and then the common spectral and formant structures are identified at step 230. These common structures are then stored in step 235, which generates a record of the common structure. The quantization map of time is then applied to the record as shown in step 240, or the record is edited in step 24 to edit its contents as needed.
In step 5, one of the contents and order of the record can be selected and changed. The method ends at step 250, where the sound has been selected, sampled, framed, analyzed, recorded, mapped, or modified in this embodiment.

Referring now to FIG. 3, there is illustrated a block diagram 300 of a resolution independent system for the synthesis of a recorded instrument, constructed in accordance with the principles of the present invention. The resolution independent system of block diagram 300 includes storage unit 305, mapping circuit 310, editor 315, waveform shaper 320, output device 325, and speaker 330. The storage unit 305 corresponds to FIG.
And records and dictionaries generated by sampling, framing, and analyzing the sound source as described in FIG.

The mapping circuit 310 applies a time quantization map to the record. As mentioned earlier, the present invention applies a wide range of conventional and later developed sound manipulation techniques, allowing the record to be synthesized with an altered tempo or key, and Provides a function to synchronize playback performance with an external clock. Further manipulation of the recorded sound is provided by the editor 315, which can alter the contents or order of the record to provide effects not included in the original sound source. Editor 3
15 can also be used to reconstruct the sequence of records relative to the frequency contours as described.

The waveform shaper 320 includes a storage unit 305,
It is coupled to a mapping circuit 310 and an editor 315. The waveform shaper 320 receives the basic frequency, applies a waveform shaping conversion function, and stores the stored records,
Generate waveforms from either mapped or edited records. Waveform shaper 320 may also generate a waveform using some combination of these three. Also, the waveform shaper 320 can select from some of the waveform shaping transfer functions stored in the waveform shaper 320 to adapt to the waveform shaping process. The waveform shaper 320 is driven by an external clock in this embodiment, so that the waveform is synchronized with external events. The waveform is then converted to output sound using output device 325 and speaker 330. The synthesis process defined here can ensure that the originally recorded sound is played with the proper fidelity, or that the originally recorded sound is modified as appropriate for the user. Can be This does not preclude the use of other synthesis techniques such as FFT or direct sine waves.

Referring now to FIG. 4, there is shown a flowchart 400 of a resolution-independent method of synthesizing recorded instruments that can be performed in the system of FIG. By the method shown in flowchart 400, it is possible to apply a quantization map of time or change the content or order of selected records in the generation and playback of sound recorded in accordance with the present invention. .

The method begins at start step 405, where a decision to synthesize a record is made, and that record is selected at step 410. The record is then processed at step 415 using the time quantization map. Next, the record is
At 20 it is changed by the editing function. The selected, mapped, and edited record is then
The waveform is shaped in step 425, and the shaped record is then delivered for playback in step 430. The method ends at step 435.

Referring now to FIG. 5, there is shown a block diagram 500 of a communication infrastructure capable of distributing resolution independent recordings for remote playback. The block diagram 500 includes the wireless server 505 and the interactive music player 510 shown in FIG. 5A, and a random access play list 515, a download function 520, and the like for sound reproduction shown in FIG. 5B.
It includes a player 525 and a speaker 530.

The present invention further provides an infrastructure for distributing recordings for remote playback. One infrastructure includes a radio server 505, depicted as a radio station, which includes a recording database associated with it, a plurality of recordings contained within the recording database, and a plurality of recordings. Each recording includes a selection of a common spectral structure and a common formant structure corresponding to each individual instrument in the recording, or a record of the vocalist, which is re-synthesized and combined in real time during playback.

The interactive music player 510 makes a request to a corresponding request receiver coupled to a recording database associated with a wireless server 505 that receives remote requests for various ones of the plurality of recordings. Occurs.
Further, a transmitter coupled to the recording database is also associated with the wireless server 505 and transmits a plurality of recordings in response to the request. There is an additional one-way mode in which the receiver waits until the desired selection has been transmitted before downloading and playing it.

Thus, the present invention provides an audio-on-demand equivalent in which the formatted audio file is transmitted to a remote radio or player (acting as a "client" receiver). These remote radios are capable of synthesizing their audio on the fly. In this embodiment of the invention, the infrastructure further includes a plurality of remote digital radios capable of receiving and digitally operating the plurality of recordings. This infrastructure is in stark contrast to the traditional analog AM or FM radio infrastructure where the remote radio simply demodulates and amplifies the received radio waves. This infrastructure can work with any of the currently proposed or later developed digital transmitter and receiver standards. The subject matter of the program for the recording may include, but is not limited to, weather forecast, news, stock information or other topic information.

The random access performance list 515 represents a plurality of recordings embodied in a plurality of bit stream files. The bit stream file contains data relating to the fundamental frequency and the selected frequency as described above. A bit stream file, which may represent a set of choices or a set of suggestions, can occur in a single serial loop. The user can select any of these and download and play them. Alternatively, downloads 520 can be generated into a set of parallel loops that can be executed more quickly by the user. A transmitter associated with wireless server 505 can broadcast any of the multiple recordings to all remote radios. Alternatively, any of the multiple recordings can be directed only to individual remote radios.
The recording database may include a record of the request. This allows for tracking the populity of the song or the popularity of the advertisement and automatically calculates the exact royalty payment amount.

[Brief description of the drawings]

FIG. 1 illustrates a block diagram of a resolution independent system for recording a musical instrument constructed in accordance with the principles of the present invention.

FIG. 2 shows a flowchart of a resolution-independent method of recording an instrument that can be performed in the system of FIG.

FIG. 3 illustrates a block diagram of a resolution-independent system for synthesizing a recorded instrument constructed in accordance with the principles of the present invention.

FIG. 4 shows a flowchart of a resolution independent method for the synthesis of recorded instruments that can be performed in the system of FIG.

FIG. 5A shows a block diagram of a communication infrastructure that can distribute resolution independent recordings for remote playback.

FIG. 5B shows a block diagram of a communication infrastructure that can distribute resolution independent recordings for remote playback.

Claims (25)

[Claims] 1. A system for recording sound, comprising: a frame generator for extracting a fundamental frequency and a spectrum envelope from the sound and generating a frame therefrom; a common spectral structure and a common formant structure in the frame A frame analyzer for identifying a selected one of the above and generating a record including the fundamental frequency and the selected one. 2. The system according to claim 1, wherein said frames are discrete. 3. The system according to claim 1, wherein a musical instrument produces said sound. 4. The system of claim 1, wherein the frame generator samples the sound after extracting the fundamental frequency from the sound. 5. The system according to claim 1, further comprising a mapping circuit for applying a time quantization map to said record. 6. The system according to claim 1, further comprising an editor for changing a selected one of the contents and order of the record. 7. The system of claim 1, wherein the frame analyzer identifies the selection from a common spectral structure and a common formant structure by performing a Fourier analysis on the frame. System to do. 8. A method for recording a sound, comprising: extracting a fundamental frequency and a spectral envelope from the sound; generating a frame from the fundamental frequency and the spectral envelope; Identifying a selected one of a structure and a common formant structure; and generating a record containing the fundamental frequency and the selected one. 9. The method according to claim 8, wherein the frames are discrete. 10. The method of claim 8, wherein said sound is generated by a musical instrument. 11. The method according to claim 8, further comprising the step of sampling said sound before extracting said fundamental frequency from said sound. 12. The method of claim 8, further comprising applying time quantization to the record. 13. The method of claim 8, further comprising the step of changing a selected one of the contents and order of the record. 14. The method of claim 8, further comprising performing a Fourier analysis on the frame. 15. An infrastructure for distributing recordings for remote playback, comprising: a radio station having an associated recording database; and a plurality of recordings contained within the recording database. Each of the plurality of recordings includes a fundamental frequency and a corresponding one of a selected common spectral structure and a common formant structure, and further receives a remote request for any of the plurality of recordings. A request receiver coupled to the recording database and a transmitter coupled to the recording database for transmitting any of the plurality of recordings in response to the request. 16. The infrastructure of claim 15, wherein the transmitter broadcasts any of the plurality of recordings to a receiver. 17. The infrastructure of claim 15, further comprising a plurality of remote radios capable of receiving and digitally operating any of the plurality of recordings. . 18. The infrastructure of claim 15, wherein the plurality of recordings are generated by a musical instrument. 19. The infrastructure of claim 15, wherein any of the plurality of recordings is embodied in a plurality of bitstream files. 20. The infrastructure of claim 15, wherein the recording database includes a record of the request. 21. A receiver for receiving a recording including a fundamental frequency and a selected one of a corresponding common spectral structure and a common formant structure, and transmitting a waveform shaping based on the selected one. A waveform shaper coupled to the receiver for applying a function to the fundamental frequency to generate a waveform; and a speaker coupled to the waveform shaper for converting the waveform to an output sound. And radio. 22. The radio according to claim 21,
Further applying time quantization to the selected one;
A radio comprising a mapping circuit coupled to said receiver. 23. The radio according to claim 21,
A radio, wherein a transfer function of the waveform shaper is selected from a plurality of waveform shaping transfer functions stored in the waveform shaper. 24. The radio according to claim 21,
A radio wherein the waveform shaper is externally clocked. 25. The radio according to claim 21,
A radio comprising an editor coupled to said receiver for altering the content of said recording.