JP2001517814A - Sound effect system - Google Patents

Abstract

(57) [Summary] A sound effect system for generating sound effects, which has a set of behavioral characteristics and is modeled on a phenomenon that generates a sound of the system when the phenomenon is included. A computing device for processing a model for each of the sound effects, wherein the synthesis means for synthesizing sound and the integration is controlled by the computing device for the sound effect system; An output for outputting a sound effect, having the characteristics of the computing device including the central processing unit, a memory and an input device, a sound effect model, and a behavior of a phenomenon that generates a sound corresponding to the essence of the model; Means and one command generation, said from said sound representative, said the transmitted command to generate a stream of synthesizer commands, said sound integration being said Such sound-generating phenomena, including commands to control sound events and synthesizer parameters, including the commands that initiate and end, cause the output of speech to be representative of the behavioral characteristics of the synthesizer command in question, meaning integration that means Said to mean: and the way in which the system relates to the relevant behavioral characteristics of the phenomenon of generating the relevant sound in a manner that is effective in a wide range of real sound effects and their differences A sound effect system comprising means for controlling user parameters by which the production of said sound may be changed.

Description

DETAILED DESCRIPTION OF THE INVENTION

FIELD OF THE INVENTION The present invention relates generally to the field of interactive media and sound production. In particular, it relates to the production of modeling and interactive model-based sound effects and systems that can create a wide range of sound effects.

BACKGROUND OF THE INVENTION Machine inventions have been used for creating acoustic effects, for example, during the centuries in which metal lightning paper and barrel + rope "lion roars" were used in theater. In the twentieth century, film pioneers created an excess of knicks and tricks, including a "foley" technique of making sound effects by treating physical objects versus making sound effects. Such professionals (noise, the art of 1916) create noise in musical or semi-musical performances and have created the use of electromechanical devices for the sound effects synchronized with the cinematic event.

[0003] Since the arrival of digital audio, the sound effects professionals have used computers to edit and the process has recorded sound effects using tools such as digital audio editors. Hardware and software (hardware like MIDI keyboard & slider bank, software like sequencer) designed primarily for musical purposes are also widely used in the production of sound effects.

[0004] Sound effects are widely used in traditional media such as film, television, radio and theater, as well as in interactive media such as computer games, multimedia titles, and on world wide websites. And while effects are often created in similar ways, they may vary greatly depending on the way in which they are used, the media. This is because while the interactive media requires the sound effects to change according to predetermined conditions, the sound effects remain in a form fixed in traditional media. The sound effects in a movie, for example, remain the same from one viewing to the next. There is no interaction between the sound effects and the audience in the case of interactive media, such as computer games, there are numerous possibilities or "passes" through production relying on user behavior at each moment in time . Therefore, both of them are visual and audible elements built from the models that are currently based, if the user's experience is highly interactive.

[0005] In most computer games and virtual environments today, the visual elements are primarily made from models that provide a high degree of interactivity with the visual experience. Model-based images differ from pre-recorded images in that the former is made from adjustable parameters from the set. The latter are fixed and cannot be easily manipulated. For example, when the user is commanded to do so by the angle of view of the user and the scene will, the object resembles the way the user changes the view when he turns his head in the real world. When you turn to a virtual space, you move in a virtual space. This is possible, although not all views have ever been pre-recorded.

[0006] In the case of sound effects, however, the solution today is to recover from one second to a portion of the typically pre-recorded audio, typically one second to several minutes. For example, consider a video game simulating a moving car. Whenever a car experiences an event that would cause sound in reality, for example, hitting obstacles at high speeds, pre-recorded sounds are played at the exact moment when a collision occurs. Typically, the relationship between sound and event, i.e., collision, is very straightforward. That is, the sound effects do not take into account the environment or the like, for example, the speed of the car, the material of the object it collides with, the angle of influence, or the reverberation characteristics. By playing different pre-recorded sound parts corresponding to a particular situation at hand, it is possible to take into account different factors while accurately responding to all possible scenarios Collecting a complete continuum of likely sounds is impractical or often impossible.

[0007] By recalling pre-recorded sound portions, creating sound effects can be easily combined with existing sound portions to carefully alter the original sound portions that simulate the delicacy of real-world conditions. Or has the limitation that it cannot be manipulated in a smooth manner. It consists of the parameters of the phenomenon that produces sound because pre-recorded sound is not based on a model.

[0008] If one wanted to create sound effects that accurately (for that reason) design real-world events where sound portions are not available, the annoyance that an entirely new portion of sound is recorded Must be, by combining or manipulating existing parts, it cannot be created. Prerecording sound is a way that almost all sound effects are now both produced beating and interactive media. In interactive media, as discussed above, the main drawback of this method is the lack of realism. For traditional media, the drawback is increased labor. For example, if a series of footprints is required in a movie, if the sound effects professional would make a particular scene in the film, the actual footprint sound corresponding to what appears on the exact scene (however he Make and record a series of whether you decide to do so, whether by actually walking or banging a physical object together, if for any reason. If you can change the site,
From walking to moving scenes, a new recording session will have to be performed to produce a footprint corresponding to the increased speed of the steps. By manipulating previously recorded footsteps from the walking site, it would not be possible to produce just a realistic survey footprint.

[0009] One solution to this problem in traditional media has been to have a large collection of samples or CD-ROMs usually supplied on a CD. This solution has raised a number of problems, though these collections can contain thousands of samples. Inflexible due to one: the sample cannot be easily manipulated. In addition, finding the correct sample is often time consuming, since one has to go through many samples and determine acceptability. But the most serious problem is that the number of differences in sound effects is the best used under the condition of infinity, but the bargain that is the exact sample sound effect that is suitable for the purpose The number that is so is that it is still difficult. Therefore, in the sound effects industry, it is seen that there is a great need to have model-based methods and systems by creating sound effects that can achieve realism together as well as save time and effort be able to.

[0010] Although several synthetic techniques exist, no suitable one has been described above as necessary. Currently there are several different gatherings of digital sound integration technology. One is a dedicated synthesizer with specialized electrical circuits and / or DSP code driven by hardware and / or software controllers for real-time generation and interaction. The second collection is a general purpose software synthesizer that can execute any imaginable integration algorithm. They are generally not real-time, but have some kind of compiling stage that causes sound files.

[0011] Computer sounds experienced by as many people as possible are produced using a sound card on a desktop computer. Typically, a small number of integration methods are available. And control is by the MIDI protocol, with the host CPU, another, a networked computer, or an external device such as a music keyboard acting as a controller.

There are three integration methods commonly used on today's commercial sound card,
Whereas frequency modulation (FM) is a wavetable, and a wave guide (a kind of "body modeling") or integration of a wavetable typically relies on recorded material (eg musical instrument notes) Thus, PM and waveguide integration are purely synthetic. Purely synthetic algorithms provide more flexibility in the sounds they create, but imitation of natural sounds tends to be not as realistic as wavetable methods based on.

[0013] The recording tone of the designed sound source shifts, and the flexibility and time commonly used in designing the capabilities of a range instrument based on only a few different recorded notes and these Wavetable integration involves some treatment of the sound recorded during execution, such as a section that is usually annular, with the standard techniques of adjusting the sign and decay provided. Simple unprocessed playback of pre-recorded cutouts is simply "integrated" but it is the method most commonly used today to create sound effects within a software environment.

A person can make a distinction between several different time scales for sound synthesis. One is the rate at which samples must be delivered to the audio output device (eg, 44.1 kHz for a "CD quality" sound channel). The other is a control rate at which the parameter values controlling integration must be updated. This can often be slower than the sample rate. For example, at that rate the pitch and volume changes usually occur in instrumental and natural sounds, routine integration is updated with these parameters, usually 2 or slower than the sample, and still smooth changes You can make your understanding. Eventually, one can consider an event-time scale based on that rate at which algorithms (eg, notes) are started and stopped in the synthesis.

The distinction between different time scales is a paradigm, but it helps many,
Somewhat optional for certain integrations, and even more unsuitable. The MIDM protocol is based on the distinction between controllers and synthesizers, and the most common MIDM bus architecture at a transfer rate of approximately 3 Kbytes / sec. Similarly, in a shared virtual environment and an environment of computers connected by a network in which multi-user games are common, communication bandwidth is also in great demand. Client computers are powerful and can handle a significant amount of integration. If the client shares a common library of integrations, the routine, and then the "event and controller parameters" display of the sound is very efficient, places a low demand on communication bandwidth, and High sound quality can be arbitrarily generated depending on the ability.

The considerations outlined above have led to sound effects modeling based on wavetable event patterns. The fact that the individual events are still based on recorded sound samples allows the current system to maintain a connection with the replay and recording of recorded audio and integration of wavetables to a remarkable degree of realism. But the movement towards the composite building in the pattern representative allows the system to move up the curve with flexibility.

Since the ordinary desktop shape includes a sound source card with the capability of a wavetable,
The host is charged with sound construction only at the event controller rate so that much of the mixing of sound processing and operation takes place on the card. Furthermore, the pure software integration environment, Wavetable's event pattern technique, is more computationally efficient than "sample-by-sample" integration and more flexible and memory efficient than audio playback and recording. This is important when the sound is embedded in CPU intensive applications.

Another advantage of an efficient and expressive modeling approach is the breadth of sound that can be efficiently represented within its boundaries. There is no sound losing the sound effects domain, so universality is important. The event pattern representative of the wavetable falls between the two extremes. There is one long event on one end, which clearly does not prove the validity of the representative. There are a number of very short events on other extremes. Coarse integration is a common technique that can represent time-domain signals. However, this end of the spectrum does not achieve the desired reduction in computation, desired bandwidth and memory. The rate of events and synthesizer commands is the lowest (sounds seldom use more than 25 / s) to keep computation and bandwidth requirements easy to handle, especially when some sound processes are running quickly Is not desirable). On the other hand, the shorter the atomic event component, the more we can provide more flexibility in controlling the sound. Very effective interactivity can often be achieved at event rates such as 10 per second per sound effect.

Consider footprints as an example. Desirable controls may involve the rate of walking or running, the weight of the person, the type of shoes being carried and the surface characteristics. The event rates are clearly lower (though we actually use several events to represent a single step). The number of wavetables is a function of some of the desired variability, but a significant amount of variability can also be achieved with a combination of standard wavetable parameters such as volume and pitch.

Of course, other event-oriented sound effects designed in this way are applause, returning machine guns and objects. The sounds being made from the faster discrete events are probably less obvious. This category includes shuffled cards, certain satellites shouting patterns, and some types of ratchet-like machines, rock drills and engines, and the like.

Another class of sound types that can be designed with these techniques is textured sounds. In this category waterfall, wind, room firmness, crowd, traffic, scratching and spinning sounds. Generating a statistically stable sound in most multimedia applications today is done by making loop points in short recordings. Unfortunately, the ears are very sensitive to the presence of such loops whose discovery destroys any sense of realism fast. Algorithms that generate statistical events can be used efficiently in most of the situations where loops are currently used, but they also keep the amount of stored data to a minimum, but distracting Does not create an annular effect.

The sound perceived as one event (eg, a fired or broken pencil) is either simultaneous or partial, with flexibility also being a major advantage with respect to simple audio playback and recording. Can be designed using multiple wavetable events matching the memory (memory usage could be recorded even more serious than one event). More difficult, but not impossible sounds to model, are those that do not have enough texture change to allow a change in an undiscovered event (eg, a crackle). Sounds that are impractical to design in this way are those that have a complex characteristic time, continuity across many attributes (eg speech).

The sound distance at which the event pattern representation of this wavetable is efficient with respect to flexibility, storage usage and sound quality is therefore very wide. Object of the invention It is the object of the present invention to provide a sound effects system for responding to user actions in real time.

It is the subject of the present invention to provide a sound effects system capable of producing another realistic survey sound effect. It is yet another object of the present invention to provide a sound effects system that can save time and labor in making sound effects. It is still yet another object of the present invention to provide methods and systems for creating model-based sound effects.

It is still the subject of the present invention to provide another method and system that is constrained to produce a defined class or subclass of sound effects, but can produce a wide range of sound effects. is there. SUMMARY OF THE INVENTION Current sound effects systems produce sound effects that are based on parameterized models of sound-producing phenomena such as footprints, earthquakes, and the like. When creating models to guarantee realism, the behavioral characteristics of phenomena are taken into account. The system allows the user greater flexibility and interactivity because the sound effects are created from the model. By controlling a set of parameters, the user can interact with the system in real time. The system is versatile in that it can create a wide range of sound effects and is not limited to only creating a particular sound effect or class of sound effects.

The system incorporation of a central processing unit is a random access memory (RA
M), connected to one or more input devices such as a keyboard, a mouse, a controller, a MIDI controller; a visual display device; a sound synthesizer and an audio digital-to-analog converter (DAC), an amplifier,
Output system including loudspeakers or headphones (or alternatively, a sound card integrating some of these individual devices can be used) and a hard disk such as non-destructive storage. These components are inventions,
An interactive sound effects computer program (SFX program) consisting of a sound effects software engine (SFX engine) and, optionally, a graphical user interface program (GUI) that plays an auxiliary role in the central element. In one mode of operation, the GUI can be replaced by a dominant program that replaces the GUI. Also currently,
An operating system program, such as a standard operating system in any personal computer system.

The CPU shares its processing power with the SFX engine, GUI or dominant program, operating system and possibly other program's S in accordance with multitasking, such as the well-known slicing technique. And executes the instructions stored in the memory in the program. Under the direction of the SFX engine, it delivers these commands to a sound synthesizer that delivers a stream of events, or produces digital audio in response. The output of the synthesizer is a digital audio signal that is then expanded and delivered to the user by means of an amplifier and loudspeakers or headphones and converted to analog form by a digital-to-analog converter (DAC). Optionally, the digital audio signal may also be delivered to a CPU that allows it to be further processed or stored for trace recovery.

The SFX engine is not directly controlled at the same time by GUI means or by external dominant programs such as computer games, or rarely both. Under the control of the GUI, the user can efficiently control it by means of one or more input devices, such as an alphanumeric computer keyboard, pointing device, control device, and interact with the linear SFX program. When operating, a specialized controller such as a slider bank or a music keyboard is provided by means such as an instrument digital interface (
MIDI) standard or other physical control means. This mode useful, a GUI program uses a display strategy that provides the user with visual information about the status of the SFX engine, including which sound effects the model is currently calling, the structure of these sound models, their Information on parameter settings and other information.

The sound model generates interface functions, parameters for external control, private data supporting the state, bank wavetables and events used by the model while the process is stopped to share the CPU. Consists of an index into the code. The sound model has many sound classes directly from the bass class,
It is arranged as an object-oriented class hierarchy. This structure has many attributes and a common way to all sounds (eg for location, volume. On the other hand most other attributes are common to one model or otherwise With other models that have only a few (eg surface features of footprints).

The model is based on the phenomenon of producing any sound. Some examples of phenomena are running footprints, earthquakes, running air conditioners, jumping ball games, not moving cars and the like. The phenomenon can be virtually anything, as long as it has some sound associated with it. Indeed, phenomena need not necessarily be real phenomena, even in the sense that they need not actually exist in the real world.

The sound modeling process begins by identifying behavioral characteristics associated with particular sound phenomena related to sound production. Behavioral features distinguish acoustics from other acoustics, including those where a naive listener defines how it changes or evolves in response to different conditions Can be defined as a set of properties that would be considered.

For some of these situations, it is useful to analyze how the sound is generated from the phenomenon. Using the footprint as an example, the sound being generated from the footprint is primarily the result of two separate events, shortly thereafter, then the impact of a heel striking the surface, the impact of a toe striking the surface . Due to standard human footprints, the entire sound generated is the result of heel toe movements of one foot followed by heel toe movements of other feet. As one walks faster, the time interval between one foot heel action and the sound produced from the other foot heel action decreases. However, it is important to realize that the time interval between the sound also produced from the heel and then the toes also decreases with respect to any relationship to the heel time interval.

As soon as the behavioral features are identified, some sound sump or procedurally generated sound is needed, which will be the basis for many differences. Samples are a well-known integration technique by recording the real-world phenomena where a sample portion of the actual sound is found (or just taking a portion from a pre-recorded existence). It may be obtained by making the part more convenient or desirable under the conditions of certain sound effects.

The choice of the length of a sound sample depends on a number of factors. As a general rule, the smaller the sample, the greater the flexibility. On the back, the smaller the sample, the more difficult it is to achieve realism. A good approximation is that by changing the user parameters of the model, most of the perceptual limitations of the possibility of the equivalent sound wave in reality can be achieved without loss of useful flexibility, i.e. You should have the longest possible sample. The choice of sound sample depends to some extent on the limits of the parameters, and also on the behavioral characteristics of the phenomenon.

Once the behavioral characteristics have been analyzed and a sample has been obtained, it is necessary to select the user parameters, ie the parameters to be made available to the user of the model to control the sound effects. is there. The parameters represent various factors that need to be controlled to produce the designed sound. Although the parameters can be structured in many ways to provide a sound effect, it is useful for the preferred embodiment of the present invention to see that the parameter structure has three layers: top, center and bottom. is there.

The top layer consists of user parameters, which are the interface between the user and the sound effects system. The middle layer consists of the SFX engine, or parameters simply called "engine parameters". The bottom layer consists of synthesizer parameters, which are well-known parameters found in either current sound or music synthesizers.

Each of the user parameters affects a combination of engine and synthesizer parameters, though in the simpler case the user parameters may control only the synthesizer parameters or the engine parameters. Any combination of engine and synthesizer parameters is theoretically possible. However, the way they are combined will depend on how the user parameters are defined taking into account the behavioral characteristics of the particular phenomenon.

User parameters are defined for the desired sound effect. For example, for footsteps, the user parameters may be location, walking speed, sluggish style, weight, severity, surface type, etc. Although these parameters can be virtually defined in any way, they are often most useful if they directly reflect the characteristics of the phenomenon and the desired behavior for which the sound effects are being produced.

Intermediate layer parameters, or engine and bottom layer parameters, or synthesizer parameters, work together to create a sound effect defined by user parameters. Bottom layer parameters include sound manipulation techniques such as volume control, pan, pitch, filter cutoff, amplitude envelope, and many others well known to those skilled in the art. When,
And how the middle and bottom layer parameters are combined is controlled by the SFX engine that takes into account the behavioral characteristics of the particular phenomenon. In essence, the middle layer can be considered as a layer that "models" the sound using the basic sound handling parameters provided by the bottom layer. Although the role of the parameters in the middle layer is complex, the parameters can be broadly classified into timing, selection, and patterning. Timing parameters basically control the length of the time interval between triggering and stopping several pieces of sound within a particular sound effect and the time interval between other commands sent to the synthesizer. The selection parameters include the order in which the samples are selected, and the patterning parameters that control which sound sample is selected at a given moment control the relationship between these factors. By adjusting these three classes of engine parameters appropriately along with the synthesizer parameters, a large set of sound effects can be generated.

In general, the behavioral features will have some behavior in the choice of the synthesizer parameters used, but there are no hard and fast rules on how these parameters should be selected. Since sound is somewhat defined by human comprehension, and also with many subtle differences, studies of the behavioral characteristics of phenomena should see how synthesizer parameters should be used for a given state May not always reveal enough information to decide. Therefore, some empirical experiments may need to be performed. DETAILED DESCRIPTION OF THE INVENTION System Configuration The overall system is conceptualized as a number of layers, and the graphical user interface, which is the application in which the top layer will use sound, is rather a special one. (Although there is) one such application. The next layer is a collection of algorithm sound models. These are the objects that encapsulate the data and describe the sound behavior. The model provides an interface that applications can use to control the sound by playing, stopping, and sending messages, or by updating parameters.

The sound model generates commands and passes them to the synthesizer at the appropriate time. The synthesizer is the rear where the audio waveform for each sample is produced, mixed and sent to an audio output device. FIG. 1 is a block diagram showing the main elements of a preferred embodiment of the present invention. The central processing unit 1 includes a random access memory (RAM) 2, one or more input devices such as a keyboard 3, a mouse 4, a control device 5, a MIDI controller 6; a visually appealing display device 7; a sound synthesizer 8 digital. −
An audio output system including an analog converter (DAC) 9, an amplifier 10, a loudspeaker or headphones, which (or alternatively, a sound card integrating some of these individual devices can be used); Hard disk 12
Is connected to a non-volatile storage device such as These components play an auxiliary role to the central element of the present invention. The core elements are a sound effects software engine (SFX engine) 13 and, optionally, a graphical user interface program (GUI) 14. In one mode of operation, the GUI may be replaced by a dominant program 15 that replaces the GUI. Also, there is an operating system program 16, such as a standard operating system in any personal computer system.

The CPU 1 controls the SFX engine 13, the GUI 14, or the dominant program 1
5. Execute the stored instructions of the program in memory, sharing its processing power according to a multi-task configuration such as a well-known time slicing technique between the operating system 16 and possibly other programs.
Under the direction of the SFX engine, it delivers a stream of commands to the sound synthesizer 8. And it produces digital audio in response to these commands. The output of the synthesizer is a digital audio signal which is converted to analog form by a digital-to-analog converter (DAC) 9 and then an amplifier 10
Is passed to the user by the loudspeaker or the headphone 11. Optionally, the digital audio signal may also be returned to the CPU for further processing or storage for later restoration.

The hard disk or other non-volatile storage device 12 provides a means for storing the following items indefinitely. A digital audio output from the synthesizer 8 under the control of the SFX program, as an option of setting the parameters and other variables of the SFX program itself, including data and instructions representing a number of sound effects.

The SFX engine 13 is directly controlled by means of a GUI 14 or from an external dominant program such as a computer game 15 or, in rare cases, both simultaneously. Under the control of the GUI, when the user efficiently controls it with one or more input devices such as an alphanumeric computer keyboard 3, pointing device 4, joystick 5 and interacts directly with the SFX program, the slider bank A specialized controller such as a musical keyboard or a musical keyboard connected by means such as Musical Instrument Digital Interface (MIDI) Level 6, or other physical dominant means. In this useful mode, the GUI program 14 provides visual information about the status of the SFX engine, including the sound effects that the model is currently calling 13, the structure of these sound models, their parameters Use the display device 7 to be provided to the user with settings and other information.

When the control device is by means of a pointing device, the display device 7 also provides feedback to the user on the logical name position of the pointing device in the usual way. By observing the display 7 and / or listening to the audio being output while manipulating the input 3 to 6, the user changes the sound effect until satisfied with the result. It is possible. This mode of operation is to create an audio from the general sound effects model of the SFX system, start it on, or select the sound effects that the model causing the output from the system and adjust the parameters of the model It is intended to allow the user to create specific sound effects according to his / her needs by selecting models and other movement elements.

In an alternative mode of operation, the SFX engine 13 operates under the control of an external dominant program, such as a computer game, 15, network,
Resident information site (website) program, virtual
Reality programs, video editing programs, multimedia authoring tools, or any other program that requires sound effects. In this mode, the user interacts with the dominant program 15 by means of the devices 3 to 6 and the display device 7. The SFX engine 13 creates sound effects under its control,
This is accomplished by allowing the dominant program, which acts as a slave, to send data to the SFX engine 13, this data being translated by the SFX engine as the dominant message. In this mode of operation, the SFX engine will typically not be visible to the user on the display 7 and can be supported by the user only through indirect I dominant program aspects that affect the SFX engine. There will be. The control method and degree that the user has on the SFX engine is a completely dominant program function,
And decided by the dominant program designer. Logical Structure The logical structure of the current system is shown in FIG. The main element is the sex engine 1, which may be under the control of the GUI 2, as described above, or in an alternative mode of operation, under the control of an external dominant program. Synthesizer 4 leading to an audio output system is also shown. These elements are the same as the corresponding elements of FIG. 1, but are shown here in a way that emphasizes their logical interrelationships.

In the mode of operation in which the invention is used directly by the user, the user controls the system by means of GUI2, which accepts a user interface (such as a computer keyboard keystroke or a pointing device movement). And act together to inform the user of the status of the system and his effect / her action. User actions that affect the production of sound effects initiate a sound effect from the GUI, generate a control message that is sent to the SFX engine 1 for firing and control. These messages are determined by the SFX engine and are in a format known to the GUI. In response to these messages, S
The FX engine 1 designs the behavior of a currently-active sound effect and generates a stream of events or commands sent to the synthesizer 4, which in turn generates a production of voice. Certain information affecting the display method used by GUI2 is SFX
Included in the engine 1: for example, the way in which the controller parameters of the sound effects model should be indicated varies from one model to another,
The information is then provided by the SFX engine for the currently-active sound effect model. So the information needs to be returned from the SFX engine to the GUI, and allows the SFX engine to send display information to the GUI or the GUI
This is accomplished by allowing the SFX engine to retrieve display information from the SFX engine.

In an alternative mode of operation in which the SFX engine is controlled from a program outside the invention, the user interacts with the external dominant program 3 in a way that is completely independent of the invention. The dominant program of the three transmission controls tells the SFX engine one to start, fire and control sound effects. These messages were determined by the SFX engine and are in a format known to the controlling program, and are typically used by the GUI in similar or similar modes of operation as described above. They are subgroups of them. In response to these messages, the main purpose of the SFX engine 1 is currently-acti
It is to design the behavior of the ve sound effects and to generate a stream of events or commands sent to the synthesizer 4, which in turn generates a production of sound.

The main internal elements of the SFX engine 1 are a set of interactive sound effect models (SFX models) 5; an application programmer interface (API) 7
Message processor 8; parameter linker / mapper 9; timing and synthesizer command processor (TSCP) 10. A library or a set of interaction sound effect models (SFX models) 5, each model representing a sound feature and a sound effect action or a sound effect class, and is composed of data and program learning. These models may be sued continuously or simultaneously, so the system is typically acoustically isolated or coupled after an imperceptible or almost imperceptible delay (so-called "real-time") Can cause. Each SFX
One or more control parameters that the model may be used to alter the sound produced by the SFX model are provided, while the system is breaking sound effects These control parameters may also be modified in real time, producing audible changes in output. In certain cases a complex sound effects model is a hierarchy where any number of levels or wheels make up the model, so that it can be made arbitrarily complex from many simpler models. May consist of other sound effects negotiated in.

An application programmer interface (API) 7 receives data translated by the SFX engine as dominant messages, these messages arriving from the GUI 2 or an external dominant program 3. The API decrypts the message to establish which type of message was sent and forwards the message to the message processor 8.

The message processor 8, which includes starting and stopping specific sound effects, loading and unloading sound effects, and taking action directed by a dominant message, SFX effect of modification of control device parameters
Modify the settings of the SFX engine that apply to the model to affect its overall behavior, and otherwise control the SFX engine and design from RAM.

The parameter linker / mapper 9 provides a means for donating one or more alternative sets of controller parameters or meta-parameters to the SFX model, where these meta-parameters are organized in a hierarchy of parameters. Reset to the SFX model's original controller parameter set or other meta-parameters. It also provides a means for applying mathematical transformations of control parameters and meta-parameters to values. Specific SFX
When the original controller parameters of the model are controlled by an external dominant program 3, for example, where the SFX engine has its own design constraints, or the SFX model is described above, Typical S
When forming part of an FX model, the parameter linker / mapper is useful because it is not always the most appropriate or useful in all cases.

The timing and synthesizer command processor (TSCP) 10 provides many functions related to the processing of events sent to the synthesizer 4 in timing and other command towers h. The invention is not limited to a particular method of integration,
And the details of this element remarkably depend on the design of the type synthesizer.
However two general functions are identified: The SFX engine works by creating a stream of commands, such as MIDI commands, that are delivered to a synthesizer to create the sound.
And typically this process occurs in real time. Most synthesizers operate by creating or modifying the output sound at the moment an event or command is received. So it may be a signal that a simple implementation of the SFX engine will trigger the synthesizer commands, but at the moment they are needed by the synthesizer or the CPU will not be able to send multiple SFXs fast enough to avoid audible interruptions of the output sound. This is vulnerable to timing interruptions, as it may not be possible to handle the complex command stream of the model. Therefore, an even more sophisticated implementation is to generate short interval commands earlier than the current time, queue them with a mechanism such as a data buffer, and deliver them to the synthesizer at the appropriate time. Annotations can achieve greater consistency in timing. The TSCP may be adjusted so that the intervaf (by which the command is generated earlier than the current time) may also be differently scheduled for different SFX models. How to provide this function. Optimal conditions are a compromise between the need to avoid timing interruptions that are responsive to changing the control parameters and the need to build a system.

[0054] 2. If more than one SFX model is active, or if one SFX model is complex, each synthesizer channel will not be delivered to a different channel of the prepared synthesizer, producing a different sound component of the sound effect. There is a need to create a large number of command streams. In a typical implementation, these paths are limited resources and must be carefully managed and dynamically allocated on demand, for example. TSCP
Plays the channel manager as a composition.

As mentioned above, a hard disk or other nonvolatile
One purpose of storage (12 in FIG. 1) is to permanently store parameters and SFX
The provision of a means for storing the settings of other variable elements of the program.
While the system is in a mode that is directly controlled by the user using a GUI that is then recalled when the system is in an alternate mode under the control of an external governing program, such parameters and Other elements may be saved. This is because the parameters of the sound effects are most suitable for the application and the SFX engine is external (the.e.
GUI saves then found to recall this same set of values, while under control of the xternal) system is the dominant program to cause the same or almost the same sound effects Allow users to experiment directly with the use of. In this way, we store sound effects,
And remembering that by changing the parameters, by means of which, after it is remembered, it is changed, it passes by for it and is completely based on the model of sound effects, and may do so Unlike saving and recalling digital sound signals with sound effects.

As the model produces realistic sounds and responds to parameter changes in a realistic and / or predictable manner, the sound model is carefully tailored to the physics or other behavior of the real world object. May be shaped. With these closely related to the behavioral characteristics of the sound-generating phenomena that are closely designed, are the acoustic effects that the model might be most important for the particular acoustic effects in question? Or assigned a set of control parameters deemed appropriate. This set of parameters may include parameters that are specific to a particular model, general parameters for a similar set of models, and general parameters for all models of the system. For example, the parameters for the walking style in which the model of human footprints would be specific to this model, set the parameters to other parameters such as all human and animal footprint models and volume, or the echo depth common to all models. You may have another parameter for walking speed that would be normal.

The system models the program, with dramatic effects that have no real-world counterparts and pure imagination, with sound-producing entities and natural photorealistic simulations taking place. Other sound effects that are exaggerated in character due to other sound effects of a particular nature can be included. Their natural generation (nat
The sound effect model automatically provides an exact reproduction of the sound, either in a sound continuous scene of the correspondingly-occurring counterpart or in other audible features, the case of a realistic simulation and the exaggeration of the real sound. As will be appreciated, the sound effects models have their natural generation at any chosen degree of accuracy.
-Occurring) may be shaped according to the behavior of the counterpart.

The system can also support “complex sounds”: these are sound models that consist of a hierarchy of other sound models at any number of levels in the tier. Typically they may represent an entire scene that is made up of many sonic components. At the top level the user can make changes to the entire site (eg, change the overall volume), but control over individual elements is also possible, and these lower level elements are optional When making adjustments to get them released (heard "alone").

The system includes general support for “connecting parameters” where parameters may be related to combinations of other parameters according to mathematical relationships. This may mean, for example, that a higher level parameter makes a total change of a number of lower level parameters, or adjusts the magnitude to suit other parameters, or some other Allows it to be used to create complex sets of parameter changes.

In order to make the sound as realistic as possible, the system avoids exact repetition and creates a sound (typically random or semi-random) to produce a natural effect. Natural) fluctuations can be introduced. Techniques for introducing fluctuations include:-changing the timing of commands or events sent to the synthesizer-changing the value of parameters with commands sent to the synthesizer-if based on the sample the synthesizer is playing If so, randomly select a sample from a collection of similar but not identical samples.

The interval is such that if the time-critical commands are generated at the moment they are needed, but it is long enough to overcome the system interruption, the system is insensitive or almost insensitive The system generates a stream of commands to the short interval synthesizer earlier than the current time, in a defined manner to respond to changes in its controller parameters after.

The system provides two modes of triggering. In one mode, sound effects,
In general, a continuous, evolving or repetitive nature starts once and continues continuously until explicitly stopped. In other modes, they are visual events in computer games, files, video productions, or animations, thus allowing precise synchrony, as well as sound effects, typically short non-continuous properties Is caused.

The system encodes the behavior of the class of sound effects, selects the options of the generic model, sets the values of the variables of the generic model to specific values, and itself And a general sound effects model that provides a way for a user of the system to create a particular sound model by providing the sample to the synthesizer. Sound Effects Integration Techniques Sound Representation The sound model consists of interface functions, parameters for external control, private data that maintains state while the process is stopped to share CPU time, is a bank wave. Index into the code that generates the data and events used by the model in a table or integration.

The sound model is based on the fact that many sound classes are obtained directly from the bass class.
It is arranged as an object-oriented class hierarchy. This structure is due to the fact that it has many qualities and a way common to all sounds (eg location, volume),
On the other hand, most other features were common to one model, or had other models (eg, footprint surface features) that otherwise had little in common. Sound models have a computer-ahead window of time, which is the mechanism by which they share the CPU. This window can be different for different sound models, and is usually at the limit of 100-300 milliseconds. The sound model process is recalled at this rate, and all event ups calculate that so, and a little beyond the next expected callback time. Events are time-stamped at their desired output, and sent to the output manager (see FIGS. 4A, 4B and below for more details).

Some aspects of the sound model representative have been learned as a direct result of the way multimedia application developers need to control them. The first is that the parameters are pre-adjusted. These come from the frequent need to use the model with several different and separate parameterizations. Rather than explicitly burden the developer with having to write code to change each of the many parameters, their effects on pre-set and then settled sounds that can be recalled by the application Parameters may be adjusted using a graphical user interface ("GUI") while monitoring the. In addition to developer convenience, the advantage of swapping parameter settings in an application in one instance of a model in this way is that rather than using two different instances of the model, rather than breaking the walk, generating events This means that the algorithm you are using can continue (taking into account footprints).

Another representative problem is the need for two different and mutually exclusive methods of control over many sounds. Conceptually, there are two different types of parameters:
Those that the application will use to interact with the sound in real time and those that select specific parameterizations of the sound from a database. These two groups may be different in different contexts for the same sound.

Consider a footprint model. One of the natural controls of sound footprints if virtual environmental applications are designed so that the world sight is through the eyes of the user
One would be the rate parameter. The faster the user moves through the space, the faster the footprint rate. However, if sound footprints are attached to the visible feet, the individual sounds need to be tuned to clearly graphic events. In the first case, there is no graphic event, which is presenting a significant burden on the application to have on time,
And will send a message to the sound model for every step. In the second case, the rate parameter is meaningless.

The current system is based on the alternative control method, if it is an event-by-
Event control is required, the standard play functions of the model are not provoked, but provide for either or both if the object provides a message exchange. All other supports (eg continuous sound, statistical variability of parameter controls) are still available. For sound models that use rates to control other properties, meaningful rates must be measured from event triggers.

An even more complex problem of control is that, for example, with the applause model to be controlled in real time, using parameters for the number of bell tongues,
Can be illustrated. The parameters typically start at zero, many people are cornered to the corresponding level that is in a virtual audience, remain at that level for some time, and then gradually decay to zero. Would. No matter how specific the application may not require such private control for a particular purpose, just affect the number of people and in turn the time envelope of the "number of people" parameter It may be desirable to specify the "enthusiasm" level (a parameter of "meta time") that has been created. The application will only have to worry about the "enthusiasm" parameter at which (or before) the applause is started. The two methods of control are mutually exclusive.

In the case of footsteps, the example of applause is different from the example of footprints, since both types of control (individual footprint versus rate) discussed are real-time. The contrasting method of control in the applause case lies between the meta-time specification of the time trajectory and the real-time control of the trajectory. We believe that the most useful way to support these controller choices is to record the parameter trajectory created by the developer using the GUI, and then after the trigger event from the application
Use orbit during playback recording.

These control problems do not arise from the process of designing the sound itself. But rather it comes from the context in which the model is embedded. The same problem must be considered for all sound models, but the implementation of different control methods depends heavily on sound representation. The following is a description that provides steps to the sound modeling process. These steps are generic in the sense that they can be used to create a wide range of sound effects, and are not limited to producing just a particular set. However, in order that these principles may be more readily understood, various examples are provided and will be discussed from time to time for illustration purposes.

The present methods and systems generate sound effects that simulate sound with certain phenomena. Some examples of the phenomenon are footprints, earthquakes, air conditioners, moving cars. The phenomenon can be virtually anything, as long as it has some sound associated with it. Indeed, a phenomenon need not necessarily be a real life phenomenon, even in the sense that it need not actually exist in the real world. For example, the phenomenon could have been the firing of a futuristic phaser gun. Although such guns may not currently exist (hence the phenomenon cannot exist), this fact is irrelevant, as long as there is some awareness of what "
The sound associated with the "phenomenon" may be of the same type or something that may be acceptable to the listener. It is also useful to have some awareness of how "the sound will vary depending on various hypothetical factors.
For example, one may hear that the sound associated with shooting a phaser gun is more powerful, and it depends on what type of objects of various types are struck by the gun's rays. You may think it is noisier and sharper to be followed by.

The sound modeling process begins by identifying behavioral characteristics associated with a particular sound phenomenon that is related to sound production. What distinguishes a sound effect from other sound effects, including those in which the behavioral characteristics of a naive listener define how it changes or evolves in response to different conditions Can be defined as a set of properties that would be considered. In many cases, they have a one-to-one correspondence with the terms that a layman would use to describe sound effects. "In the case of sound effects that do not correspond to objects that actually exist or geniuses"
A real world feature is, for example, a phaser, as described above, a naive listener shooting a behavioral object might expect such an object or phenomenon to possess if it existed It is a characteristic.

The degree of things like speed, drag and stagger, weight (for the person making the footprint) are audibly surrounded by type (eg cement, grass, mud)
For example, in the case of footprints, the features of the action painting, location (location with respect to the listener), may include. It is easily appreciated that these features define the sound for a particular set of situations. For example, the sound will be different from those produced by casual walks from the sound produced from footprints; footsteps on hard marble will sound different than footprints on wet mud. Is standing.

For some of these situations, it is useful to analyze how the sound is generated from the phenomenon. Using the footprints again as an example, the sound being generated from the footprints is mainly immediately after two other events, then the impact of the heel hitting the surface, the impact of the toe hitting the surface Resulting. Due to standard human footprints, the entire sound generated is the result of heel toe movements of one foot followed by heel toe movements of other feet. As one walks faster, the time interval between one foot heel action and the sound produced from the other foot heel action decreases. However, it is important to realize that the time interval between the sound also produced from the heel and then the toes also decreases with respect to any relationship to the heel-heel interval. At some point, the sound produced from the heel and the sound actually produced from the toes partially overlap. And, as another thing, it becomes difficult to distinguish).

In addition, the time from heel to toe is affected by another parameter. When marching, the foot falls quickly and so when vertical, from heel to toe is very short. In contrast, the heel is far from being vertical and the toe touches the ground, so a long walk causes a longer heel to toe time and is relatively long before it touches the ground To travel. So, with the current method of modeling, the time from heel to toe (long walk vs. march) is the end result of both walking speed and "walking style". The general principle is that the parameters inside the model may be influenced by many of the appearance or "user" parameters in a mathematical relationship of arbitrary complexity.

In certain cases, when attempting sound modeling, knowledge of this mechanism of the sound generation is important for two main reasons. First, it allows one to change the correct set of parameters to create the most realistic sound effects.
If someone said, "If the heel and toe did not change the time interval equally, but changed the time interval between the sound produced from one heel and toe to another, In the footprint example given above, the resulting sound effect will not sound very realistic ... but the second main reason to analyze the mechanism is that it is needed It is important to have independent control of the heel and toe sounds individually, for example, again using the footprint example, with the concept of some loudness and sound type. There is, and for that, another sample for each of the sounds is needed, it is not enough to have the heel toe samples as a combined pair.

Of course, the behavioral characteristics of any particular phenomenon in a rather large range, of course, the degree of choice of selection and the analysis of these behavioral characteristics may be primarily sound effects. The nature of the sound-producing phenomena and the degree of realism that depended on the use desired. However, it is generally true that some identification and understanding of the behavioral characteristics of any phenomenon requires that sound effects be properly measured.

As soon as the behavioral features have been identified, some sound samples or other procedurally generated sounds are needed that will be the basis for many differences. sample,
Record the actual sound sample parts found in real-world phenomena and identify (or just take parts from some existing pre-recording) or well-known integration techniques, designed It may be obtained by creating the part either through it, which is more convenient or desirable under the condition of the sound effect of the camera. For example, in the case where the sound of a phaser gun is being designed, just synthesizing a sample, given, may be more convenient than no such phaser gun actually exists in the real world. In the case, however, of recording footprints, just that sound is produced from actual footprints, or recording individual searches of footprints (ie, heel taps and toe taps recorded separately) Or, it will be easier in most cases to record a simulation of these elements (eg, by tapping different types of rigid objects together until the desired sound is achieved). . When recording samples for use with this type of model, it is generally better to prevent the inclusion of sounds that are not closely related to a particular phenomenon, ie, to separate the sounds.

The selection of the length of a sound sample depends on many factors. As a general rule, the smaller the sample, the greater the flexibility. On the backside, the smaller the sample, the greater the work it is to achieve realism and the harder it is. A good approximation is that by changing the user parameters of the model, most of the perceptual limitations of the possibility of the equivalent sound wave in reality can be achieved without loss of useful flexibility, i.e. You should have the longest possible sample. For example, in the case of footprints, if one wanted to produce footprints of different speeds, include a set of samples for the reasons provided above, including heels and toes. It will be necessary to get. However, this means that one always has to record two sounds separately from the current edit taking into account the technique inherits and other forms of edit to split one record into many samples I do not. "But the heap technique may be difficult or impossible for cases where the sounds partially match.

The choice of the sound sample depends to some extent above the limits of the parameters (the parameters are discussed in detail below), and also depends on the behavioral characteristics of the phenomenon. Again using the footprint example, it should be pointed out that some sound effects do not require some additional white samples. For example, to pretend to be a walk, just the need for timing that can be changed, and therefore to change the style, this can finish any existing sample to change the surface on which the footprint is created It is easier to get a sample of footprints just on each of the surfaces, rather than trying to manipulate an existing sample. For example, it would not be easy to create footstep sounds on a soft muddy surface using only footprint samples on concrete surfaces. How many samples are needed for a given phenomenon can, of course, be determined by one person from one sound effect
We want people and rely on the range of change and the distance of sound effects.

In many cases, a number of similar but not identical samples are collected. It has two purposes. First, it supplies and quells a means about impersonating the subtle, ever-changing nature of real-world sound. For example, two footprints are not identical in realization, and a model that creates two identical footprints in succession is immediately recognized as artificial. Much of the naturalness may be imitated, with some samples to choose and a randomly selected set of similar samples (except for the same which is also triggered twice in immediate succession) Absent.

A second reason for collecting large numbers of samples is that by collecting points along the spectrum, a continuous spectrum can often be disguised. For example, there is no well-known integration or sound processing technique for changing a quiet giggle in heartfelt laughter (or vice versa), but the "power of laughter" differs, The parameters may be constructed by collecting a set of laughter samples in a "roaring degree" choosing individual samples according to the setting of the "force" parameter. Typically,
This technique is combined with any of the options described above. Sound modeling parameter structure Select the user parameters, i.e. the parameters that should be made available to the user of the model to control the sound effects, once the behavioral features have been analyzed and samples are obtained It is necessary to do.
The parameters represent the various factors that need to be controlled to produce the designed sound. Although the parameters can be structured in many ways to provide a sound effect, it is useful to look at the parameter structure, as illustrated in FIG. 5, for a preferred embodiment of the present invention.

Referring to FIG. 5, the top layer is “system”, consisting of user parameters that are the interface between the user and the sound effects. The middle layer consists of parameters employed by the SFX engine, or just stated to be "engine parameters". The bottom layer consists of synthesizer parameters, which are well-known parameters found in either current sound or music synthesizers.

As indicated by the arrows, in general, each of the user parameters may, in the simpler case, control only a synthesizer parameter or only an engine parameter, but the engine parameter and the synthesizer parameter Affects the combination.
A combination of engine and synthesizer parameters is theoretically possible; however, the way in which they are combined, as should be explained in detail below, the user parameters are characteristic of the behavior of a particular phenomenon Will be taken into account.

[0086] User parameters are defined with respect to the desired sound effect. For example, in the case of footprints, user parameters include location, walking speed, sluggish style,
It can be weight, severity, surface type, and the like. Although these parameters can be defined in virtually any way, they are often most useful if they directly (and therefore) reflect the phenomena in which the sound effects are being created and the characteristics of the desired behavior. In many cases, they are obvious, easy-to-use laypersons may use to describe sounds, whereas user parameters such as surface type may be useful parameters for phenomenon footprints Are the parameters understood by
If the surface type probably doesn't make sense in a terrestrial environment, it probably won't help for phenomena like earthquakes.

The user parameters can be defined in many ways to give the user control access. However, in a preferred embodiment, it is represented in the form of a "slider" on the graphical user interface (GUI), FIG. 3, where the user slides the slider bar to effect control. Can be. For example, because of the phenomenon footprint for the speed slider, the slider is moved to the right and the walking speed is increased. Because of the loose slider, the amount of drag on the walk is increased when the slider is moved. To combine the effects, some user parameters can be quickly appealed, for example, by wishing for both a speed slider and a loose slider, any combination of dragging speeds can be achieved.
Some combinations are clearly undesirable, but may be possible. For example, one would probably not combine the surface type "metal" with "marble". In contrast, "vacation" is more likely to be combined with "garbage" to achieve a footprint effect on the leaves on the garbage.

[0088] Intermediate layer parameters, as defined by user parameters
Alternatively, engine parameters and bottom layer parameters, or synthesizer parameters, work well in combination to create a sound effect. Bottom layer parameters can include sound handling techniques such as volume control, filter pan, pitch, filter truncation, and many others familiar to those skilled in the art.

When and how the median and bottom parameters are combined are controlled by the SFX engine that takes into account the behavioral characteristics of the particular phenomenon. In essence, the middle layer can be considered as the layer over which the "model" sound provided using the basic sound handling parameters provided near the bottom. Although the role of the intermediate layer parameters is complex, the parameters can be broadly categorized as selecting and simulating timing. These parameters are defined here separately and are different, but it should be by conceptual art tools that illustrate the modeling warrant by those skilled in the art or by the SFX engine. Understand the techniques used and do not have to exist as separate and separate components in current sound effects systems.

In describing the role of each class of parameters individually now, cause a few pieces of sound within a particular sound effect and time interval during other commands sent to the synthesizer, and During the pause, the timing parameter basically controls the length of the interval. Selecting the parameters at which the sound samples are selected at a given moment, including the order in which the samples are selected. The simulating parameters control the relationship between these factors. By properly adjusting these three classes of engine parameters, along with the synthesizer parameters, a large set of sound effects can be produced. The role and the effect of each of these parameters will be more clear in the example as provided below.

The characteristics of footprint behavior related to speed are described above, and reference is made once again to footprint examples. It has been described that as a speed increases the interval between the heel and toe, as well as another decrease in one heel to toe interval, as well as the interval. Here, the user parameter (top layer) is speed. When the user adjusts the speed slider to increase the speed, the hour parameter is reduced between the heel and toe, as well as the interval between one other heel and toe action. As described above, this timing is also affected by the "style" parameter. However, when speed and style are changed, the footprint pattern or behavior does not change. The heel sound is always followed by a toe sound or the like.

If, however, the sound effect was that of a moving horse, the behavioral features would be more complex. And therefore, additional parameters need to be accompanied. For example, consider the user parameter "speed" with respect to horse hooves. Timing parameters are adjusted to reflect the interval between shorter on average and affecting hooves on a given surface, so that the average of the time intervals between events is shorter as the speed increases. Need to be done. But to add
While walking, going fast, running lightly and galloping, the imitations and ordering of events change as horse switches. The exact pattern, of course,
It must be determined empirically using the characteristics of actual horse behavior.

As a final example, if the user parameters were surface type due to footprint phenomena, the only affected engine parameter class would be selected, since the imitation of timing and aspects does not change. The people who are involved. Here, it will be chosen which surface or combination of surfaces is a selected different set of samples. But timing and imitation do not change.

For some of these examples, there may be examples where synthesizer parameters would have to be sued alone or with engine parameters. For example, and also using the footprint example, from the time when typically heavier people are more immersed in the tone (although this may not always be true in reality) and produce louder footprints, etc., the synthesizer parameters, tone , Volume, etc. need to be controlled according to user parameters, weight (for the person making the footprint).

In general, although the behavioral features will have some attitude in selecting the synthesizer parameters to be used, it is difficult to see how these parameters should be selected. And there are no fast rules. Since the sound is defined somewhat by human comprehension, and also with many subtle differences, studying the behavioral characteristics of phenomena should be how synthesizer parameters should be used for a given state May not always reveal enough information to decide. For example, it has been found that it is a change that helps to increase the quality of the tune as speed, in order to produce the best survey effect for horse hooves. The relationship between tone and speed is not always obvious. Therefore, some empirical experiments may need to be performed.

To further illustrate the principles provided above, FIGS. 6-14 are tables listing the various parameters used for some actual sound files. Considering the table in FIG. 6 as an illustrative example, the first column lists the user parameters plus "any variation" (see above for description for "any variation") . The next column has a heading at the top showing the engine and synthesizer parameters, the engine parameters including the first three columns after the user parameters column. An "X" in the box indicates that the parameter in that column was used for sound modeling for the user parameter found in that particular row.

These tables show how user parameter changes affect a) event mimicry and b) aspects of different synthesizer parameters. Also (on the first row of each table) they are small random variations introduced to provide naturalness and great variety in sound effects, even where there are no changes to user parameters Influenced by, indicate.

In FIG. 8, the user parameters, brake, clutch and gas pedal are two “
Controls "internal" variables, car speed and engine speed. The two internal variables are governed by the 2-D differential gear equation with the pedal settings as input. Car speed and engine speed in turn control the synthesizer event generation. Engine speed is the event (hundreds per second) in which each ignition controls the ignition of the piston separately.

The speed of a car controls the “roaring” of a car running slowly along the road. In FIG. 14, the wind noise model consists of a very small number of events. Most parameters where user parameters (and any fluctuations) affect 25, ie volume, the same set of integrations to sell, leak, etc., but they affect them in different ways. give. For example, "power" controls the average value of a parameter (stronger winds have higher volume, tone, filter Q, etc.). "Width of change" (for the same parameters)
Controls the deviation from the average, and "gusts" control the rate of change of the parameter. "Wind" is also the number of layers in the sound (eg "whistle"
Number). Development Environment For embedding the sound model in the application, the development environment is a traditional person with recorded sound, select from CD, load in the environment, play,
And we need to be richer than we stopped. This is partly because sound selection involves selecting a track, but rather involves adjusting parameters. So there is an important role played by the interaction environment that is independent of the target application.

The developer tool consists of two parts: the GUI, SFX engine of FIG. Using the GUI, the developer checks a database of sound models, selects the sound model that will be used in the application, and adjusts the sound parameters while listening to the sound in real time. Code compilation listen (c
There is no mode-compile-listen cycle; the sound is selected and adjusted by ear. The parameters and their names are intuitive and closely related to the behavioral characteristics of the real world sounds that are designed at a high level (for example,
Rather than "AM broadcast depth and regularity,""roughness." As soon as the sound is selected and parameterized, the developer creates a file name where the customized sound model (ie, so that the user parameters can be used by the developer) will be stored. The API requirements described below are then used to play and control the sound model from within the application. Application programmer interface (API) functions initialize the audio engine, load, play and stop sound, create a mapper between application variables and sound parameters, and user and sound parameters Provided to applications that send messages to

Sounds can be loaded as needed and unrolled off. Request to play, FIG. 4A, request to stop with computer sound system, FIG. Will start playing sound over 4D. Will play and end the sound. If the unload was not requested after the sound was stopped,
It can be played again, in which case the sound model is removed from memory.

If interactive control of sound is not desired, play and stop are the only routines that need to be called. So, when exactly digital audio files are always used by the application, the sound model can be used. Developers are not faced with anything that forces them to deal with the fact that the sounds are generated algorithmically in real time from the file rather than played, otherwise they are each time they are played You may notice that the sounds are (generally) more natural and fun because they are not the same.

If interactive control of the sound model parameters is desired, the call to the API function accomplishes the task called SetParam (FIG. 4C). Some user parameters can be set for all sounds (eg sound
All user parameters are updated through the same bass note class function, though common to both (and relative location of the listener) and others are specific to a particular note. The GUI now plays a further sound documentation role in that the various parameter IDs that the sound accepts by the API are the same as compared to the name displayed with the slider in the GUI.

Typically, an application will control sounds using application variables that have a wide range of possible values determined by their role in the application. On the other hand, sound model user parameters have their own limits of possible values determined by the user parameters used in the sound object. The distance of the application variable is set param map (Se
A call to a function called tParamMap) can be automatically mapped on user parameter limits. This call only needs to be made once for any given user parameter after which any call to the set parameter will automatically perform the appropriate mapping. Sound location also "sets" with unit to update sound emitter and handset locations
Is controlled by a single call to the "mapping" function to establish that

Finally, FIG. Some sounds of 4F take a trigger message that can be sent from the application. Again, the messages that sound accepts are unique, while all invoke the same bass note method (just do nothing in response to messages that a particular sound does not process). These messages usually have the effect of immediately causing any element of the sound, such as a train whistle, when a steam train sound is being played. This message exchange method is also the means by which “event by event” is performed (eg, the individual footprint causing), and the method by which the parameters are preset is replaced. Execution Details To illustrate the execution details, FIG. 4A shows that what happens is arriving from a game or other application program or interacting message when play is initiated in response to the user, as well as through the GUI. After many initialization tasks, the system picks up the current settings of the user parameters, including requests for the necessary synthesizer paths. It then generates a set of commands such as MIDI messages sent to the synthesizer. It produces these in a short way in the future. When the delay in the system is long enough to ensure that there will never be a gap in speech production, but short enough to cause a change in the user parameters to produce a result that is perceived by the user to be almost immediate [ X + safety margin]. The system then starts a system timer that calls back the SFX engine after time X (ie, to return control to do so). Meanwhile, the computer is free to process other programs, such as games, in which the SFX engine may be embedded therein. Time X is typically on the order of 10-100 milliseconds. And the safety margin is typically on the order of 5-10 milliseconds.

FIG. 4B shows what happens after hour X. At this point, as described above, most of the buffered stream of generated synthesizer commands is (before it underruns, the amount of time equal to the safety margin left) Supported commands). Since these last calls may include the internal consequences of any of the user parameters, the system will first process any changes in the parameters and change (in some models) due to time advances. I do. This is followed by a small arbitrary change in certain internal parameters that may be needed for natural or fun effects to introduce sound-sensitive fluctuations. It then generates a command stream for another next time X, and leverages another call
Arrange back. This generate-callback cycle will continue until the system is shut down.

FIG. 4C shows what happens when one of the user parameters is changed. As described above, the system includes an elaborate mapping scheme in which a change in a user parameter can cause a change in any number of parameters inside the model. (This is needed to ensure that the user parameters behave in a manner that is consistent with the well-designed behavioral characteristics of the phenomenon.) And provide a non-linear transformation. And these transformations may be separate for each of as many mappings of user parameters as possible to parameters internal to the model. After mapping, a parameter change may also result in sending one or more synthesizer commands immediately. In other cases, these parameter changes (see above) are triggered on the next callback. FIG. 4
D arrives from the game or other application program and interacts through a GUI or message to indicate to the user what happens when a stop sound message is received in response as well. The system aborts the impending callback first. It then generates a synthesizer command in the future, as required to produce the desired end to the sound (for some sounds this may be a sudden stop. It is a natural or fun effect for those who need to be more gentle; in some cases, it may involve modifying the synthesizer parameters). Eventually it releases the corresponding synthesizer channel when the sound is started, and performs many other cleaning functions. The next play message resets, for example, the synthesizer path and system variables ready for reuse.

[0108] Flowchart 4E illustrates that when the trigger message is received in response to the user, as well as being received, what happens is arriving from the game or other application program and interacting through a GUI or message. Generates a finite length (typically a few seconds, or even less) sound effect (or part of a sound effect) that stops on its own agreement after the trigger message is intended to produce. So they act basically like playing with automatic stops, and this flowchart should be self-explanatory taking into account the description above. Process management Process management introduced by TSCP is one function. As described above and in FIG. It arbitrates both the fate order to become a synthesizer and the callback request of the process. Reduce and test those test cycles for interrupts generated by the system timer. Make them expensive resources. Servicing interrupts could significantly reduce performance if maybe each of many simultaneous sound processes were calling for callbacks directly from the system. Instead, a sorted queue of processes waiting for callbacks is maintained by the process manager.
And although the system timer is in use, the next sound processing only runs.

A similar mechanism is used for commands that are time-stamped for output to the synthesizer and sent to the process manager. Although each sound process generates commands in chronological order, when many processes are running at the same time, the process manager schedules the commands to sort, queue, and exit the timer. Provide only for the next person who has been.

Some commands, such as interactive controller messages, are queued, and rather serviced immediately. The backend synthesizer uses these commands to affect any new events that the process manager sends, even if the events are queued prior to the "flash-through" controller update. So, the user parameters have an incubation period that can be as large as compared to the callback period for the sound process.
However, the parameters controlling integration have shorter control rate latencies.

The process manager is also responsible for allocating synthesizer resources to sound processes and interfacing with specific backend synthesizer APIs. Without departing from the spirit or its essential features, the present invention may be embodied in other specific forms. The currently published embodiment should therefore be considered in all respects illustrative and open-ended. All changes that come within the scope and meaning of the invention as indicated by the appended claims and the equivalent limits of the claims should therefore be accepted therein.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a preferred embodiment of the present system.

FIG. 2 is a functional block diagram showing the logical structure of the system.

FIG. 3 shows a graphical user interface showing user parameters for a footstep sound effect in which the user parameters are represented in the form of a graphical slider.

4A to 4E are flowcharts illustrating the various steps that the system follows for various commands.

FIG. 5 shows a parameter structure employed in a preferred embodiment of the present system.

6 to 14 are tables illustrating the parameters used for the actual sound model.

[Procedure amendment]

[Submission Date] April 6, 2000 (200.4.6)

[Procedure amendment 2]

[Document name to be amended] Drawing

[Correction target item name] Fig. 1

[Correction method] Change

[Correction contents]

FIG.

[Procedure amendment 3]

[Document name to be amended] Drawing

[Correction target item name] Figure 2

[Correction method] Change

[Correction contents]

FIG. 2

[Procedure amendment 4]

[Document name to be amended] Drawing

[Correction target item name] FIG. 4A

[Correction method] Change

[Correction contents]

FIG. 4A

[Procedure amendment 5]

[Document name to be amended] Drawing

[Correction target item name] FIG. 4B

[Correction method] Change

[Correction contents]

FIG. 4B

[Procedure amendment 6]

[Document name to be amended] Drawing

[Correction target item name] Fig. 4C

[Correction method] Change

[Correction contents]

FIG. 4C

[Procedure amendment 7]

[Document name to be amended] Drawing

[Correction target item name] FIG. 4D

[Correction method] Change

[Correction contents]

FIG. 4D

[Procedure amendment 8]

[Document name to be amended] Drawing

[Correction target item name] Fig. 4E

[Correction method] Change

[Correction contents]

FIG. 4E

[Procedure amendment 9]

[Document name to be amended] Drawing

[Correction target item name] Fig. 5

[Correction method] Change

[Correction contents]

FIG. 5

[Procedure amendment 10]

[Document name to be amended] Drawing

[Correction target item name] Fig. 6

[Correction method] Change

[Correction contents]

FIG. 6

[Procedure amendment 11]

[Document name to be amended] Drawing

[Correction target item name] Fig. 7

[Correction method] Change

[Correction contents]

FIG. 7

[Procedure amendment 12]

[Document name to be amended] Drawing

[Correction target item name] Fig. 8

[Correction method] Change

[Correction contents]

FIG. 8

[Procedure amendment 13]

[Document name to be amended] Drawing

[Correction target item name] Fig. 9

[Correction method] Change

[Correction contents]

FIG. 9

[Procedure amendment 14]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG. 10

[Procedure amendment 15]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG. 11

[Procedure amendment 16]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG.

[Procedure amendment 17]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG. 13

[Procedure amendment 18]

[Document name to be amended] Drawing

[Correction target item name] FIG.

[Correction method] Change

[Correction contents]

FIG. 14

[Procedure amendment 19]

[Document name to be amended] Abstract

[Correction target item name] Full text

[Correction method] Change

[Correction contents]

A sound effect system for generating sound effects, the sound effect system having a set of behavioral characteristics, the sound effect system being modeled on a phenomenon in which the phenomenon is included to generate a sound of the system. A synthesis unit for processing a model for each of the sound effects, the synthesis meaning for synthesizing sound, and the integration being controlled by the calculation unit, a central processing unit Said computing device, including a memory and input device, a sound effect model, an output means for outputting a sound effect, having characteristics of a behavior of a phenomenon that generates a sound corresponding to the model essence, Obtained from the relevant sound representative,
One command generation, said transmitted command to generate a stream of synthesizer commands, is a phenomenon in which the sound integration produces the sound, initiates and terminates commands that control sound events and synthesizer parameters. Said synthesizing, which means to cause a typical audio output for the behavioral characteristics of the synthesizer command in question, including such means: and so that the system in question covers a wide range of realities in an efficient way Means for controlling user parameters by which the production of the sound may be altered in a manner related to the behavioral characteristics of the sound effect and the phenomenon of generating the sound causing the difference It is a sound effect system provided with.

──────────────────────────────────────────────────続 き Continuation of front page (72) Inventor Kerok, Peter Rowan Singapore, Singapore 129793, Blok Dee Kent Vale, Clementi Road 113 # 09-06 F-term (reference) 5D378 BB15 BB22 XX07 XX16 XX26 The sound production is altered in a way that is related to the behavioral characteristics of the sound-producing phenomena that cause the wide range of realistic sound effects and their differences in a specific way Sound effects system with means to control user parameters that may

Claims (38)

[Claims] 1. A sound effect system having a set of behavioral characteristics, the sound effect being created by modeling a sound-producing phenomenon of the system including the phenomenon. A system for processing a model for each of the sound effects, wherein the synthesis implies that the synthesis is controlled by the relevant computing device for synthesizing sound, a central processing unit. Said computing device, including a memory and input device, a sound effect model, having a behavioral characteristic of a phenomenon that generates a sound corresponding to the model essence, output means for outputting a sound effect, Obtained from the relevant sound representative,
One command generation of the transmitted command to generate a stream of synthesizer commands is a phenomenon in which the sound synthesis produces the sound, including commands to start and end sound events and commands to control the synthesizer parameters. The integration, which means that the behavioral characteristics of the synthesizer command of interest cause a typical audio output, means that such a system allows for a wide range of realistic sound effects and Sound comprising means for controlling a user parameter by which the production of the sound may be altered in a manner related to the behavioral characteristic of the sound-generating phenomenon causing the difference Effect system. 2. The sound effects system according to claim 1, comprising a graphical user interface, said graphical user interface allowing a user to control said sound effects. 3. The sound effects system of claim 1 including an application programmer interface, said application programmer interface allowing a process executing on a task or processing computing device to control said sound effects. . 4. The sound effects system according to claim 2, wherein the sound effects corresponding to the different sound-producing phenomena are controlled by an ordinary graphical user interface. 5. The sound effect according to claim 3, wherein the sound effects corresponding to different sound producing phenomena are present therein, wherein the sound effects controlled by the system to ordinary application programmers interact. 3 sound effect system. 6. The sound effects system of claim 1, wherein there are as many such input devices selected from the group consisting of keyboards, computer pointing devices, trackballs, controls, buttons, knobs, sliders and communication ports. The sound effect system according to claim 1. 7. The sound effects system according to claim 1, wherein said sound event is present therein, wherein the sound effects resulting from the part being played by the system recorded audio data. 8. The sound effects system of claim 1, wherein the sound event is there, wherein the system obtained from the procedural generation of audio data. 9. The sound effects system of claim 1, wherein said sound implying integration is peripheral device communicability. 10. The sound effects system according to claim 1, wherein said sound means integration on the central processing unit of the computing device is a process. 11. The sound effects system of claim 1, wherein a mechanism for introducing the sound and sound fluctuations therein provides output model incorporation,
2. The sound effects system of claim 1 including fluctuations occurring in non-synthetic sounds thereby effecting and mimicking fluctuations, thereby making the sound output more realistic. 12. The sound effects system of claim 11, wherein the mechanism includes means for changing the timing of the synthesizer command, including a command to start and end a sound event and a command to control synthesizer parameters. 13. The sound effects system of claim 11, wherein the mechanism comprises means for randomly selecting sound events from a collection of similar, but non-identical sound events. 14. The sound effects system of claim 11, wherein the mechanism includes means for changing a value of the synthesizer parameter. 15. A sound effect system further comprising claim 1: means for negotiating a plurality of similar, but not identical, sounds, in an order determined by the power of the order, the characteristic. Means for characterizing the event, including the acoustic properties, and means for selectively initiating and ending the sound event, the value of the user parameter and the power of the property. The sound effects system of claim 1, wherein the relationship between the user parameters has the effect that the user parameters are sufficiently controlling the behavioral characteristics of the sound effects along the spectrum. 16. The sound effects system of claim 1, wherein the synthesizer command is generated earlier than a current time by a predetermined time interval;
And the interval, long enough to make up for the delay while the command is generated, but short enough so that the system reacts to the change of the user parameter after a delay that is not fully perceived, 2. The sound effects system of claim 1 wherein the sound effect is held in a buffer until it means that the sound has to go ahead in time. 17. The sound effects system of claim 1 wherein the sound effects model the same sound effects system of claim 1 wherein the model includes a plurality of other sound effects arranged in a hierarchy of levels. 18. The sound effect system of claim 1, wherein the synthesizer parameters pan and market selected filter truncation, filter Q, and amplitude envelope from the next group of volumes. Sound effects system. 19. The system of claim 1, wherein said stream of synthesizer commands is generated by a process, wherein said system is an engine-dominated sound effect parameter, said dominant engine parameter sound effect system comprising: Start sound events and commands with synthesizer parameters,
Select any synthesizer command, including the command to terminate, time any subscriber or synthesizer command, including the command, and end the sound event and command to control the synthesizer parameters; and start and end the sound event 2. The sound effect system according to claim 1, wherein the synthesizer command includes a command to be executed and a command to control a synthesizer parameter. 20. The first parameter selected from the group consisting of the second parameter, the user parameter and the engine parameter, the second parameter selected from the group including the user parameter. For parameters, engine parameters and synthesizer parameters, then any change in the value of one of the first parameters in question will cause a change in the value of the parameter in question and that first parameter in the synthesizer will change in value. To
20. The sound effects system of claim 19, wherein the first parameter is mapped to the second parameter. 21. The sound effects system of claim 20, wherein a transformation selected from a group comprising a linear transformation includes a relation between a mapping parameter and a first parameter included in a second parameter. 21. The sound effects system of claim 20, including a non-linear transformation. 22. The sound effects system of claim 20, wherein the cartographic relationship exists in a plurality of levels, with one level at one second parameters also the first parameter at the next lower level. The sound effect system according to claim 20. 23. The sound effect system according to claim 20, wherein the second parameter corresponding to the parameter belongs to a different sound model. 24. The sound effects system of claim 1, wherein the mode of operation in which the sound effects are performed at each time begins when a particular stimulus is received by the system and received by the particular stimulus. The particular stimulus whose sound effect originates from the source, produced by the communication of the timed stream about the synthesizer comment on the sound implied by the integration, which is received in an unperceivable time Comprising said computing device and said input device of an external computer program, an external medium selected from the group by which said sound effects may be present and synchronized with the event, Such external media music, sound works, including motion videos, animations,
2. The sound effects system of claim 1, including computer games, graphic computer presentations, multimedia works, theater lighting works, and physical environments interacting with the works. 25. The system of claim 1, further including recording as well as recording the parameter trajectory or more, and sound recording the live recording means for replay. 2. The sound effects system of claim 1, wherein the effect indicates a parameter trajectory at a later time where the parameter trajectory comprises a timed continuation of the value of the used parameter. 26. A sound effects system according to claim 1 or more,
Means for allowing said user of the system to select the options of behavior, the options of unbranded behavior to design from the plurality, the options of behavior to limit the characteristics of behavior of the general sound effects of the general sound effect. A selection to design the behavioral characteristics of a particular sound-generating phenomenon; assigning values to predetermined variables in a well-known general sound effects model; Means for variably setting the behavioral characteristics to be restricted to the behavioral characteristics of the particular sound-producing phenomenon, and for storing the values of the options and the variables for later recalls Storage means, whereby the user of the system creates a specific sound effect from the general sound effect model as if it had been obtained by the model. Sound effect system. 27. The sound effects system of claim 1, wherein the means for providing a sound integration means for a user of the system proceeds with reading data of the sound implied by the additional sound data. And means for saving later recalls the audio data storage for which the audio data was indicated, thereby identifying the user of the system combining the general sound effects behavior A sound effect system, wherein the sound effect model of the sound effect model may be arranged in the model having the data of the substitute sound. 28. The sound effect system according to claim 1, wherein one of the phenomena generating the sound is a footprint. 29. The sound effects system of claim 1, wherein one of the phenomena producing the sound is gun firing. 30. The sound effects system of claim 1, wherein one of the phenomena producing the sound is operating. 31. The sound effect system of claim 1, wherein one of the phenomena producing the sound is applause. 32. The sound effect system of claim 1, wherein one of the phenomena producing the sound there is flowing little by little in a stream. 33. The sound effects system of claim 1, wherein one of said sound producing phenomena continues to talk monotonically about the machine. 34. The sound effect system according to claim 1, wherein one of the phenomena generating the sound is laughter. 35. The sound effects system of claim 1, wherein one of the phenomena producing the sound is a cheering crowd. 36. One of the phenomena that produces the sound there is wind.
Sound effect system. 37. The sound effects system of claim 1, wherein one of the phenomena that produces the sound is a non-real world phenomenon. 38. A method for producing a sound effect, comprising a set of behavioral features, modeled on a phenomenon that produces a sound, said phenomenon including said system. Identify a set of behavioral features for the sound-generating phenomena, represent the behavioral features of the corresponding sound-generating phenomena, build a sound effects model, model the model, and communicate the model. Keeping in a possible computing device connects to a sound synthesizer, and generating a stream of synth synthesizer commands results from the sound assertion, and the behavioral characteristics of the phenomenon in which the synthesizer generates the sound. Commands that are transmitted to the synthesizer concerned, say commands, and start and end sound events to control, so as to produce a typical voice production for the Providing the model with at least one user parameter, the user parameter, and the synthesizer parameter that said the command including the synthesizer command is related to the sound-generating phenomenon; Manipulating the user parameters to create the difference is the style, the method of creating the sound effect model that brings correspondence.