WO2007090273A1 - Digital media simulation tool and method therefor - Google Patents

Digital media simulation tool and method therefor Download PDF

Info

Publication number
WO2007090273A1
WO2007090273A1 PCT/CA2007/000171 CA2007000171W WO2007090273A1 WO 2007090273 A1 WO2007090273 A1 WO 2007090273A1 CA 2007000171 W CA2007000171 W CA 2007000171W WO 2007090273 A1 WO2007090273 A1 WO 2007090273A1
Authority
WO
WIPO (PCT)
Prior art keywords
recited
simulation tool
res
module
control board
Prior art date
Application number
PCT/CA2007/000171
Other languages
French (fr)
Inventor
Simon Ashby
Nicolas Michaud
Jacques Deveau
Original Assignee
Audiokinetic Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Audiokinetic Inc. filed Critical Audiokinetic Inc.
Publication of WO2007090273A1 publication Critical patent/WO2007090273A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data

Definitions

  • the present invention relates to digital media such as text, audio, image, video, or a combination thereof. More specifically, the present invention is concerned with a tool which allows both simulating and authoring such digital media for example before their implementation in an application to be deployed on a device provided with computational capability.
  • Simulation tools which are usually part of an application development tool or authoring package, and which allow developers to test how newly created elements blend together and interact with other components from the application are also known.
  • a frequent drawback of these conventional simulation tools is that changes made during a simulation are not reflected in the application development tool.
  • those simulation tools which are specifically targeted to simulating digital media usually present the information as a media graph, wherein the media objects to simulate are presented connected separately to the modifiers that affect those media. This limits the playback and visualisation possibility of the media and also the possibility to make changes during simulation.
  • Figure 1 is a perspective view of a game audio authoring system, including a simulation tool according to a first illustrative embodiment of the present invention
  • Figure 2 is an example of a user interface for a digital media simulation tool according to a first illustrative embodiment of the present invention
  • Figure 3 is an example of a user interface for a module from the simulation tool from Figure 2;
  • Figure 4 illustrates the links between an audio file, audio sources and a sound object
  • Figure 5 is an example illustrating the use of Events to drive the sound in a game
  • Figure 6 is an example of a Platform Selector List from the simulation tool from Figure 2;
  • Figure 7 is a close up view of the module area of the simulation tool from Figure 2, illustrating the module insertion indicator;
  • Figure 8 is a module similar to the module form Figure 3, further illustrating the game object list
  • Figure 9 is a module similar to the module form Figure 3, further illustrating a Reset menu
  • Figure 10 is an example of a user interface for a digital media simulation tool according to a second illustrative embodiment of the present invention.
  • Figure 11 is a first example of a first level hierarchical structure according to the second illustrative embodiment of the present invention.
  • Figure 12 is a second example of the first level hierarchical structure from Figure 11 , illustrating the use of containers to group sound objects;
  • Figure 13 illustrates a third example of a project hierarchy, including Master-Mixer and Actor-Mixer hierarchies;
  • Figure 14 is an example of a switch container for footstep sounds
  • Figures 15A-15B are flow diagrams illustrating an example of use of a switch container;
  • Figure 16 illustrates the use of a switch container in authoring a game;
  • Figure 17A illustrates an example of use of relative state properties on sounds
  • Figure 17B illustrates an example of use of absolute state properties on sounds
  • Figure 18 illustrates an example of use of states in a game
  • Figure 19 is an example of a user interface for the States panel of the simulation tool from Figure 10;
  • Figure 20 is an example of a user interface for the Switches panel of the simulation tool from Figure 10;
  • Figure 21 is an example of a user interface for the Real-time
  • RTPCs Parameter Controls
  • a digital media simulation tool comprising:
  • a control board for receiving at least one digital media-related element (DM-RE); the at least one DM-RE being associated to and characterized by at least one modifying value; each of the at least one modifying value being from a corresponding modifier; and [0031] at least one module included in the control board i) for displaying information related to the at least one DM-RE including the at least one modifying value associated thereto.
  • DM-RE digital media-related element
  • a method in a computer system for simulating digital media comprising:
  • DM-RE digital media-related element
  • a computer-readable medium containing instructions for controlling a computer system to generate:
  • a control board for receiving at least one digital media-related element (DM-RE); the at least one DM-RE being associated to and characterized by at least one modifying value; each of the at least one modifying value being from a corresponding modifier; and
  • DM-RE digital media-related element
  • At least one module included in the control board i) for displaying information related to the at least one DM-RE including the at least one modifying value associated thereto, and ii) for selectively simulating the at least one DM-RE with the at least one modifying value associated thereto.
  • the computer-readable medium can be a CD-ROM, DVD-
  • ROM read only memory
  • USB universal serial bus
  • computer application is intended to be construed broadly as including any sequence of instructions intended for a computer, game console, wireless phone, personal digital assistant (PDA), multimedia player, etc. which produces sounds, animations, display texts, videos, or a combination thereof, interactively or not.
  • PDA personal digital assistant
  • computer system will be used herein to refer to any device provided with computational capabilities, and which can be programmed with instructions for example, including without restriction a computer, a game console, a wired or wireless phone, a PDA (Personal Digital Assistant), a multimedia player, etc.
  • a digital media simulation tool 20 according to a first illustrative embodiment is described in Figure 2.
  • the simulation tool 20 is part of a game audio authoring system 10 which produces or modifies audio objects to be used by a video game (not shown).
  • the simulation tool according to illustrative embodiments of the present invention will sometimes be referred to herein simply as "Simulator” or "Soundcaster”.
  • the system 10 comprises a computer 12 programmed with instructions for generating an Authoring Tool including the simulating tool 20, a display 14, conventional input devices, in the form, for example, of a mouse 16A and keyboard 16B, and six (6) sound speakers 18A-18F, including a sub- woofer 18A, configured to output sounds to discreet channels that can optionally be encoded in a 5.1 DolbyTM Digital setup.
  • the system 10 is of course not limited to this setup. The number and type of input and output devices may differ and/or the sound output setup may also be different.
  • the system 10 also includes a conventional memory (not shown), which can be of any type.
  • the system 10 can also be further configured for network connectivity 19. Since it is believed to be well-known in the art to provide a computer or other similar devices with network connectivity, such implementation will not be described herein in further detail.
  • system 10 is illustrated as a desktop type workstation, it is of course not limited to such an embodiment and can take any forms such as a laptop, a portable device, a tablet PC, etc. operating on any operating system.
  • the simulation tool 20 allows: • prototyping and experimenting with the audio objects
  • the Simulator 20 includes a graphical user interface (GUI) allowing controlling the following aspects of a game audio:
  • the Simulator 20 comprises a control board 22 including a master controller 24, and modules 26 received in a module area 28 of the control board 22 for simulating and editing audio-related objects provided from the game audio authoring tool.
  • an audio simulation is built as follows:
  • the simulation tool 20, which is part of the authoring tool, is called from a conventional menu (not shown) of the Authoring Tool.
  • the control board 22 is then displayed.
  • the simulation tool 20 is configured so that adding an audio-related object therein causes the creation of a corresponding module 26, which is then displayed in the module area 28 thereof.
  • This newly-created module 26 represents the audio-related object that has been inserted in the control board 22 for simulation and which is associated thereto.
  • modules 26 will be briefly presented. Since all the modules 26 share a common form or template and differ only by the audio-related element contained therein and the associated properties associated thereto, we will refer to the module 26 herein as to any one of the modules 26 indistinctively.
  • the module 26 is in the form of a display pane comprising a descriptive portion in the form of a title bar 30, module playback controls 32, module playback indicators 34 and module property controls 36.
  • the title bar 30 includes information for identifying the audio- related element which is represented in the module 26. It includes for example the name of the audio-related element and may also include an icon categorizing the element.
  • the module 26 includes module playback controls 32 to manage the playback of the audio-related element represented by the module 26.
  • the control board 22 includes master controls to simultaneously manage the playback of all the modules 26, and the audio-related elements they represent, included in the control board 22.
  • the module playback indicators 34 are provided to indicate when certain predetermined behaviors of the audio-related element occur during playback.
  • the module property controls 36 display property values associated to the element and allows selectively changing these values during simulation.
  • the audio- related objects are sound objects or events used to drive the sound in the game.
  • the sound objects are in the form, for example, of imported audio files.
  • the original audio files can be of many known types such as any type of well known PCM (Pulse-Code Modulation) audio file format such as the well-known WAV format.
  • the original audio files can also be in a compressed format such as the well known MPEG-1 Audio Layer 3 (MP3) format.
  • MP3 MPEG-1 Audio Layer 3
  • the audio files can alternatively be streamed directly to the system 10 via, for example, but not limited to, a well-known voice-over-ip, protocol.
  • the Authoring Tool is configured so that the audio file importing process yields two copies of each audio file: the original copy and a work copy.
  • the original copy remains untouched, but can be referenced to.
  • the work copy is used for experimentation.
  • the importing process generally includes converting the audio files and creating audio sources for each audio file.
  • the importing process further includes the creation in the Authoring Tool of the sound objects that contain the audio sources.
  • the importing process includes the following subs-steps:
  • a work copy of each audio file is then created for each of a series of game platform versions. This file is based on the corresponding Imported folder file and has been converted for the specific platform.
  • a created sound object references audio sources as they are created in the Authoring Tool from an imported audio file. These sources contain the conversion settings for each project platform.
  • the Authoring Tool is configured so that the audio conversion process retains the same pitch and duration as the original files.
  • the following properties can be defined for the conversion: Channels, R-L Mix, Sample Rate, Audio Format, Bit Depth, Bit Rate and Filename Marker. Since these properties of audio files are believed to be well-known in the art, and for concision purposes, they will not be described furtherin.
  • the present Simulator is of course not limited to the import and audio conversion processes described above.
  • the Authoring Tool is further configured to modify the sound objects with modifiers.
  • modifiers can take any form allowing to be associated to and characterize the sound objects with modifying values, which can modify the sound objects or characterize them qualitatively or quantitatively.
  • modifying values which can modify the sound objects or characterize them qualitatively or quantitatively.
  • modifiers include for example, and without limitations, properties and behaviors which can be associated to the sound objects.
  • the properties are characterized by property values which are assigned to the sound objects so as to characterize and modify them. Since the above-mentioned properties are believed to be well-known in the art, and for concision purposes, they will not be described herein in more detail.
  • the Authoring Tool is further programmed with a Randomizer to randomly modify selected property values of an object each time it is played. Sliders, input boxes and/or any other GUI input means are then provided to allow the user inputting a range of values for the randomizing effect. [0081] Behaviors
  • each object can be characterized by behaviors.
  • the behaviors determine for example how many times a sound object will play each time it is called or the order it will play, and whether the sound is stored in memory or streamed directly from an external medium such as a DVD, a CD, or a hard drive.
  • the Authoring Tool is configured such that different types of behaviors are made available from one object to another.
  • the Authoring Tool is configured so that, by default, sound objects play once from beginning to end. However, a loop can be created so that a sound will be played more than once. In this case, the number up to an unlimited number of times the sound will be looped is defined.
  • the Authoring Tool further allows setting which sounds will be played from memory and which ones will be streamed from the hard drive, CD, or DVD.
  • an option is also available to avoid any playback delays by creating a small audio buffer that covers the latency time required to fetch the rest of the file.
  • the size of the audio buffer can be specified so that it meets the requirements of the different media sources, such as hard drive, CD, and DVD.
  • Events are provided to drive the sound in-game. Each event can have one or more actions or other events that are applied to the different sound objects to determine whether the objects will play, pause, stop, etc.
  • the Authoring Tool is configured so that Events can be integrated into the game engine even before all the sound objects are available. For example, a simple event with just one action such as play can be integrated into a game. The event can then be modified and objects can be assigned and modified without any additional integration procedures required.
  • a variety of actions are associated to an Event to drive the sound in-game.
  • the actions are grouped by category and each category contains a series of actions that can be selected.
  • Each action can be assigned a set of properties that can be used to fade in and fade out incoming and outgoing sounds.
  • the following table describes examples of event actions that can be assigned to an Event in the Authoring Tool:
  • Reset Volume Returns the volume of the associated object to its original level.
  • Reset LFE Volume Returns the LFE volume of the associated object to its original level.
  • Reset Pitch Returns the pitch of the associated object to its original value.
  • Set LPF Changes the amount of low pass filter applied to the associated object.
  • Reset LPF Returns the amount of low pass filter applied to the associated object to its original value.
  • An Event can perform one action or a series of actions assigned thereto.
  • each event action is characterized by a set of related properties that can be used to further refine the sound in-game, which fall into, for example, one of the following possible categories:
  • the Authoring Tool according to the first illustrative embodiment is configured so that when sound properties are defined for a particular platform, these properties are set across all predetermined platforms by default. These properties are said to be linked across platforms. This streamlines creating projects across platforms.
  • the Authoring Tool allows customizing the properties and behaviors for a specific platform by unlinking selected properties and/or behaviors, and defining new values.
  • the following modifiers can be linked and unlinked:
  • the Authoring Tool includes means for selecting and indicating the current platform.
  • This means can for example be in the form of a Platform Selector List 38 (see Figure 6) provided in the tool bar.
  • the modifying values assigned to an object are linked across all platforms unless a selected modifier is unlinked for the current platform selected.
  • the Authoring Tool includes excluder elements (not shown) associated to objects and allowing for including or excluding the corresponding object from the current platform.
  • the possibility to selectively exclude sounds from a project allows optimizing each platform for the available resources, customizing content for each version, optimizing content based on the limitations and strengths of certain platforms, optimizing performance based on the limitations and strengths of platforms, sharing of certain properties across versions, and creating content that will be delivered in different formats.
  • the Authoring Tool allows switching from one platform to another at any point in the development cycle by accessing the Platform Selector List 38 and selecting the desired platform. The platform versions of the sound objects in the hierarchy are then displayed.
  • a module 26 is created when audio-related objects, in the form of a sound object or an event, is added to the Simulating Tool 20.
  • the module 26 represents the object or event together with its properties and behaviors.
  • the module controls 32-36 allow modifying properties and playing back sounds.
  • the Simulator is configured so that an object or event can be added by dragging the audio-related object therein for example.
  • GUI means and user-menu means can also be used to import an object into the Simulator.
  • control board 22 is configured to display a module selection indicator 40 at the location where the audio-related object is dragged.
  • the mouse button is released, the selected object/event is added to the simulator as a module 26 at the location selected by the user, as indicated by the indicator 40.
  • the Simulator 20 is configured so that the order of the objects and events is not important when they are dragged into the Simulator 20 for their playback.
  • the playback depends on which module 20 is selected to play as will be described furthering in more detail.
  • the Simulator 20 allows re-ordering of the modules to recreate sequences or facilitate testing. More specifically, the Simulator is configured so that a module 26 can be selected in the module area 28 of the control board 22 and positioned at a new location therein. The module 26 formerly in that position is shifted down in the module area 28. Conventional dragging functionalities are provided in the Simulator 20 to move the modules 26. More specifically, a module 26 can be dragged to a new location. When the module is moved, the module formerly in that position is shifted down in the Simulator GUI dedicated window.
  • the Simulator 20 is further configured so that a module 26 that is no longer needed can be removed from the control board 22.
  • each module 26 is provided with a dedicated GUI element in the form of a conventional icon button 42 in the title bar 30, which the user can "click” to remove the module 26.
  • the master controller 24 also includes an icon button 44 for removing all modules 26 included in the Simulator 20 (see Figure 2).
  • GUI elements provided to remove modules can take other forms.
  • the Simulator 20 includes the following non restrictive playback options:
  • the Simulator 20 can be used to further discriminate which sounds to play.
  • the Simulator 20 includes tools to specify which sounds will play back based on:
  • Platforms As it has been described herein above, certain sounds are included or excluded in the audio based on platform during development. In a simulation it is possible to choose to play only the sounds that are in the current platform, or play all sounds in the module;
  • Converted Sounds Also, as described hereinabove, when imported audio files are converted, an original version of the audio files is maintained. By default, the simulator plays the converted sounds; however, the original version can also be played back as will be described hereinbelow;
  • Game Objects As part of the game integration process, sounds are associated with game objects so that the correct sound will play in the game. Game objects are entities in game development to which an interface, a sound, a trigger and so on can be attached. The Simulator allows specifying that only the objects in each module that are associated with a specific game object will play back.
  • control board 22 includes a platform discriminator, in the form of a first icon button 46, allowing enabling a Current-platform play mode, wherein only the objects and events in the current platform can be auditioned and modified.
  • a platform discriminator in the form of a first icon button 46, allowing enabling a Current-platform play mode, wherein only the objects and events in the current platform can be auditioned and modified.
  • the desired platform for playback is first selected as the current platform using the platform selector list 38.
  • the control board 22 further includes an original sound selector 48, in the form of a second icon button, for selecting the original pre- converted sound for playback.
  • each module 26 includes a game object list 50 and a connection to the game allows the user to select a game object for auditioning the associated sound.
  • connection to the game can be achieved via a Remote connector (not shown) provided with the Authoring Tool. Since it is believed to be within the reach of a person skilled in the art to perform a remote connection between applications, this will not be described herein in more detail.
  • the platform discriminator 46 and original sound selector 48 are in the form of on/off switches allowing switching between modes by clicking on the corresponding icon.
  • GUI elements can also be used for the platform discriminator 46, original sound selector 48, and game object list 50.
  • the control board 22 is configured so that modules 26 can be played back in any order, consecutively, asynchronously, overlapping and so on. For example, changing the order for playback allows testing and experimenting and helps finding a play back sequence.
  • the playback controls 32 of the module 26, which are in the form of Play, Pause and Stop buttons, can be used to cause the module to start, pause or stop playing respectively.
  • the master controller 24 of the control board 22 further includes pause and stop icon buttons 52-54 to cause to simultaneous pausing or stopping of all modules 26 in the module area of the control board 22.
  • the Simulator 20 further allows events and objects to be mixed at the same time they are being auditioned. Object properties can also be modified prior to mixing.
  • the Simulator 20 includes property indicators 56- 60 associated to some of the properties and playback indicators 61.
  • the property indicators 56-60 are interactive GUI elements associated to properties, including but not limited to volume, pitch and low-pass filter, and positioning, each having a property value assigned to the object- related element represented by the module 26 and that modifies that object- related element.
  • These GUI elements are in the form of a conventional input box 62 associated to a conventional slider 64, providing two means to visualize and edit the property value. Modifying a default value using the slider 64 causes the range between the new value and the default value to become highlighted.
  • GUI elements can alternatively differ from one property to another and of course take other forms.
  • link/unlink indicators 66 are provided to show whether the property value is linked to other platforms, and randomizer indicators 68 are also provided to indicate whether a Randomizer has been applied on the property value.
  • Partial Unlink The property value for the current platform is (half yellow) linked, but one or more corresponding values of other platforms are unlinked.
  • (yellow) Randomizer A property value to which a Randomizer On effect has been applied.
  • a Randomizer allows for defining a range of possible values that can be used randomly each time the object is played.
  • Randomizer - A property value to which no Randomizer Off effect has been applied.
  • the Simulator 20 includes playback indicators 61 which become highlighted when particular behaviors or actions occur during playback. They are displayed in the master controller 24, and in the individual modules.
  • playback indicators 61 which become highlighted when particular behaviors or actions occur during playback. They are displayed in the master controller 24, and in the individual modules.
  • the following table lists additional property and action parameter indicators in the Simulator 20:
  • Fade A fade has been applied to an event or a sequence container
  • Set Volume A set volume action has been applied to a sound object in an event.
  • Set Pitch A set pitch action has been applied to a sound object in an event.
  • Set LFE Volume A set LFE volume action has been applied to the sound object in an event.
  • Enable Bypass An Enable Bypass action has been applied to the sound object in an event.
  • playback indicators 61 can be used to indicate the state of any audio-related object or property associated thereto during simulation.
  • the property indicators 56-60 can be used to modify the value of the property associated thereto. According to the first illustrative embodiment, this is achieved by using the slider 64 or input box 62 to modify the value.
  • the Simulator 20 is further configured so that the user can access editing functionalities of the Authoring Tool for modifying the property values associated to the audio-related object during a simulation, for example by double-clicking the title bar of the corresponding module 26. It is believed to be within the reach of a person skilled in the art to provide other means to access the editing functionalities of the Authoring Tool or to another calling application.
  • the playback controls 32 and property indicators 56-60 define module simulation controls allowing re-creating the audio-related object represented by the module 26 as characterized by the different current property values set for example by the property indicators 56-60.
  • Event actions can temporarily change the properties of objects.
  • the Simulator 20 allows applying the event actions to all the objects that are associated with the event.
  • Reset functions available through a Reset menu 70 via a corresponding button 72 on the property control area of the module 26 (see Figure 9), are provided to return the values for objects associated with an event to their initial values. With these functions, event actions can be reset in each module and for all modules in the master control area.
  • the Reset menu which is associated with the reset function, includes the following non-restrictive options:
  • Reset Volume to reset the volume for the objects associated with the event
  • Reset LFE volume to reset the LFE volume for all the objects associated with the event
  • the Simulator 20 is further configured to allow event actions to be reset in the master controller area 24 of the control board 22.
  • a "Reset all” menu similar to the "Reset” menu 70 available through the modules can be accessed on the master controller 24 via a "Reset All” button 74.
  • a difference between the functions available though both menus is that all functions available through the "Reset all” menu effect all actions that have been triggered for the objects. Since both the "Reset" and “Reset all” menus are very similar, and for concision purposes, the Reset All menu and associated functionalities will not be described furthering.
  • the Simulator [00148] According to the first illustrative embodiment, the Simulator
  • the Simulator 20 includes a "Test” button (not shown) allowing to selectively prevent changes made in the Simulator 20 to be made in the Authoring Tool.
  • the Simulator allows saving the contents of a simulation, including the modules 26. This allows some of the simulations to be consulted and reused at a later time to audition changes, test, and validate work.
  • an input box 75 is provided to assign and then display a name to the current simulation session.
  • a "New" icon button 76 is also provided to open a New Soundcaster Session dialog box where the name of the current session can also be entered.
  • the Authoring Tool includes a Soundcaster Manager (not shown) which allows to create and open already created sessions.
  • a session manager can of course take many forms. Since such session manager is believed to be beyond the scope of the present invention, it will not be described herein in more detail.
  • the Simulator 20 is not limited to receive Audio objects which have been imported and converted, and can be modified to receive and play original files.
  • the layout and interface elements of the module 26 may also be different.
  • the simulation tool 100 is part of a game audio authoring system 10 which produces or modifies audio objects to be used by a video game (not shown).
  • the Simulator 100 comprises a control board 22 including a master controller 24, and modules 102 received in a module area 28 of the control board 22.
  • the Simulator 100 further comprises audio/game interaction panels in the form of game syncs panels 104-108 displayed in a game syncs area 110 of the control boards 22.
  • Authoring Tool is further provided with elements that are mapped to the game to allow the sound to match conditions in the game. This allows the audio to react to what is going on in the game.
  • These elements which will be referred to herein as game syncs, react when the game triggers a shift in action or environment, and provides a corresponding shift in the audio.
  • RTPCs Real-time Parameter Controls: game parameters that are mapped to audio properties so that when the game parameters change, the mapped audio properties will also reflect the change.
  • the Simulator 100 allows handling audio-related objects from a hierarchical structure, which originates for example from the Authoring Tool, to edit and manage the sound assets and which will now be described.
  • the sound objects can be grouped and organized in a first hierarchical structure, yielding a tree-like structure including parent-child relationships whereby when properties and behaviors are assigned to a parent, these properties and behaviors are shared by the child thereunder.
  • the hierarchical structure includes containers (C) to group sound objects (S) or other containers (C), and actor- mixers (AM) to group containers (C) or sound objects (S) directly, defining parent-child relationships between the various objects.
  • sound objects (S), containers (C), and actor-mixers (AM) all define object types which can be characterized by properties, such as volume, pitch, and positioning, and behaviors, such as random or sequence playback.
  • Sounds Objects that represent the individual audio asset and contain the audio source There are two kinds of sound objects:
  • Containers A group of objects that contain sound objects or other containers that are played according to certain behaviors. Properties can be applied to containers which will affect the child objects therein. There are three kinds of containers:
  • Random Containers group of one or more sounds and/or containers that can be played back in a random order or according to a specific playlist.
  • Sequence Container group of one or more sounds and /or containers that can be played back according to a specific playlist.
  • Switch Container A group of one or more containers or sounds that correspond to changes in the game.
  • Actor-Mixers High level objects into which other objects such as sounds, containers and/or actor-mixers can be grouped. Properties that are applied to an actor-mixer affect the properties of the objects grouped under it.
  • containers define the second level in the Actor-Mixer Hierarchy.
  • Containers can be both parent and child objects.
  • Containers can be used to group both sound objects and containers.
  • nesting containers within other containers, different effects can be created and realistic behaviors can be simulated.
  • Actor-mixers sit one level above the container. Actor-mixers can be the parent of any number of sounds, containers, and other actor-mixers. They can be used to group a large number of objects together to apply properties to the group as a whole. [00174] Figure 12 further illustrates the use of actor-mixers to group sound objects, containers, and other actor-mixers.
  • Actor-Mixer hierarchy The above-mentioned hierarchy, including the sound objects, containers, and actor-mixers will be referred to herein as the Actor-Mixer hierarchy.
  • the Master-Mixer Hierarchy is a separate hierarchical structure of control busses that allows re-grouping the different sound structures within the Actor-Mixer Hierarchy and preparing them for output.
  • the Master-Mixer Hierarchy consists of a top-level "Master Control Bus" and any number of child control busses below it.
  • Figure 13 illustrates an example of a project hierarchy including Master-Mixer and Actor-Mixer hierarchies. As can also be seen in Figure 13, the Master-Mixer and control busses can be identified by specific icons.
  • the child control busses allow grouping the sound structures according to main sound categories within the game as defined by a user or as pre-defined.
  • control busses create the final level of control for the sound structures within the project. They sit on top of the project hierarchy allowing creating a final mix for the game. [00180] Since the control busses group complete sound structures, they can further be used to troubleshoot problems within the game. For example, they allow muting the voices, ambient sounds, and sound effects busses, to troubleshoot the music in the game. An object within the hierarchy is routed to a specific bus.
  • the properties of an object can be divided into two categories:
  • Object behaviors determine which sound within the hierarchy will be played at any given point in the game. Unlike properties, which can be defined at all levels within the hierarchy; behaviors can be defined for sound objects and containers.
  • the Authoring Tool is also configured so that the types of behaviors available differ from one object to another as will be described furtherin.
  • Containers Since different situations within a game may require different kinds of audio playback, the hierarchical structure allows for grouping objects into different kinds of containers such as the following three different types of containers:
  • Each container type includes different settings which can be used to define the playback behavior of sounds within the game. For example, random containers play back the contents of the container randomly, sequence containers play back the contents of the container according to a playlist, and switch containers play back the contents of the container based on the current switch, state or RTPC within the game. A combination of these types of containers can also be used.
  • Step to play only one object in the container each time the container is played; Continuous: to play the complete list of objects in the container each time the container is played. This mode further allows looping the sounds and creating transitions between the various objects within the container.
  • Switch containers are provided to group sounds according to different alternatives existing within a game. More specifically, they contain a series of switches or states or Real-time Parameter Controls (RTPC) that correspond to changes or alternative actions that occur in the game.
  • RTPC Real-time Parameter Controls
  • a switch container for footstep sounds might contain switches for grass, concrete, wood and any other surface that a character can walk on in game (see Figure 14).
  • Each switch/state includes the audio objects related to that particular alternative. For example, all the footstep sounds on concrete would be grouped into the "Concrete” switch; all the footstep sounds on wood would be grouped into the "Wood” switch, and so on.
  • the sound engine verifies which switch/state is currently active to determine which container or sound to play.
  • FIGs 15A-15B illustrate what happens when an event calls a switch container called "Footsteps".
  • This container has grouped the sounds according to the different surfaces a character can walk on in game.
  • a random container is used to group the footstep sounds within the switch so that a different sound is played each time the character steps on the same surface.
  • switches represent the alternatives that exist for a particular element in game, and are used to help manage the corresponding sounds for these alternatives.
  • sounds are organized and assigned to switches so that the appropriate sounds will play when the changes take place in the game.
  • the Authoring Tool allows creating a switch called "concrete” and assigning a container with footstep sounds that match the concrete surface to this switch. Switches for grass, gravel and so on can also be created, and corresponding sounds assigned to these switches.
  • the sounds and containers that are assigned to a switch are grouped into a switch container.
  • the switch container verifies the switch and the correct sound is played.
  • switches are first grouped in switch groups.
  • Switch groups contain related segments in a game based on the game design. For example, a switch group called “Ground Surfaces” can be created for the "grass” and “concrete” switches illustrated in Figures 15A-15B and 16 for example.
  • States are provided in the Authoring Tool to apply global property changes for objects in response to game conditions. Using a state allows altering the properties on a global scale so that all objects that subscribe to the state are affected in the same way. As will become more apparent upon reading the following description, using states allows creating different property kits for a sound without adding to memory or disk space usage. By altering the property of sounds already playing, states allow reusing assets and saving valuable memory.
  • a state property can be defined as absolute or relative. As illustrated in Figures 17A and 17B, and similarly to what has been described hereinabove, applying a state whose properties are defined as relative causes the effect on the object's properties to be cumulative.
  • Figure 18 This example concerns the simulation of the sound treatment that occurs when a character goes underwater in a video game.
  • a state can be used to modify the volume and low pass filter for sounds that are already playing. These property changes create the sound shift needed to recreate how gunfire or exploding grenades would sound when the character is under water.
  • states are first grouped in state groups. For example, after a state group called Main Character has been created, states can be added that will be applied to the properties for the objects associated with the Main Character. From the game, it is for example known that the main character will probably experience the following states: stunned, calm, high stress. So it would be useful to group these together.
  • states After states have been created, they can be assigned to objects from the hierarchy.
  • the first step is to choose a state group.
  • the Authoring Tool is configured so that by default all states within that state group are automatically assigned to the object, and so that the properties for each individual state can then be altered.
  • RTPCs Real-time Parameter Controls
  • RTPCs are provided to edit specific sound properties in real time based on real-time parameter value changes that occur within the game.
  • RTPCs allow mapping the game parameters to property values, and automating property changes in view to enhancing the realism of the game audio.
  • using the RTPCs for a racing game allows editing the pitch and the level of a car's engine sounds based on the speed and RPM values of an in-game car. As the car accelerates, the mapped property values for pitch and volume react based on how they have been mapped.
  • the parameter values can be displayed, for example, in a graph view, where one axis represents the property values in the project and the other axis represents the in-game parameter values.
  • the Authoring Tool is configured so that the project RTPC values can be assigned either absolute values, wherein the values determined for the RTPC property will be used and ignore the object's properties, or relative values, wherein the values determined for the RTPC property will be added to the object's properties. This setting is predefined for each property.
  • RTPCs can be used to assign the game parameter (speed) to the project property (volume). Then a graph view can be used to map the volume levels of the footstep sounds to the speed of the character as it changes in game.
  • RTPCs can also be used to achieve other effects in a game, such as mapping low pass filter values to water depth, low frequency effect values to the force of an explosion, and so on.
  • the game syncs panels 104-108 allow access to the game syncs. The panels 104-108 can be used to enable game syncs during playback of audio-related objects as will now be described.
  • States, switches, and RTPCs can be auditioned in the application and in game. For example, by connecting to the game and adding the appropriate modules 102 to the Simulator 100, audio-related objects can be auditioned, tested, and mixed in real time with the game action using game syncs.
  • the Simulator 100 is configured to receive any object from the hierarchy, including: events, sound objects, random containers, sequence containers, switch container, and control buses.
  • the Simulator 100 is so configured that, when an object is dragged therein, the state groups and states enabled for the sounds in the module 102 corresponding to the object are added in a States list menu 105 which is included in the States panel 104 (see Figure 19).
  • the States list menu includes a first list of state groups 112 enabled for any one of the objects represented by one of the modules included in the Simulator 100.
  • a list of states 114 is provided for each of these state groups 112.
  • the list of states 114 is for example in the form of a drop down menu.
  • the States panel 104 further includes a "Show All" button 115 to force the States list menu 105 to show all state groups and states that have been created in the entire project and not just the states associated with the module 102.
  • the States panel allows enabling a state during playback. A selected state is applied to all sounds in the Simulator 100 that subscribe to the selected state during playback.
  • the master controller 24 of the Simulator 100 includes a user interface element for returning to the default or initial state specified for example by the Authoring Tool.
  • This interface element is for example in the form of a "Reset All States" option from a shortcut menu which is available from the (») button 116 on the master controller 24.
  • the Switch panel 106 includes a Switches list menu 107 which includes the list of all switches 116 that are assigned to objects presented by any one of the modules 102 (see Figure 20). These switches, which populate the list 116 as audio-related objects are inserted in the Simulator 100, can be used during playback. The switch containers in the simulation will play the sounds that correspond with the switches that are selected.
  • the Switches panel 106 further includes a "Show All" button
  • a "Reset All Switches" option is available from the Master controller to return the Switch Container to a predetermined default switch.
  • the Simulator 100 includes a user interface, in the form of the RTPC panel 108, which allows controlling the game parameter values using a range of values.
  • the Simulator 100 is configured so that when an audio-related object is inserted therein, the game parameters 118 and the associated mapped objects and properties assigned to the game parameters 118 in the module are added in the RTPCs panel 108.
  • the RTPCs panel 108 includes a "Show All" button 119 to force the RTPCs panel 108 to display all game parameters that have been created in the entire project, and not just those associated with the module 102.
  • the RTPCs panel 108 includes a user interface element 120 associated to each game parameters 118 to allow to change the game parameters values to audition the mapped sound properties.
  • the user interface element 120 includes a conventional slider, which represents the range of game parameter values. Since these values have already been mapped to object property values, the property values automatically change as the parameter values change.
  • the Simulator 100 is configured so that a game parameter
  • RTPCs panel 108 of the Simulator 100 can be modified and auditioned during playback by selecting it in the RTPCs panel 108 of the Simulator 100.
  • the game parameters that have been mapped to the module objects selected 102 will be applied during playback.
  • the simulator is also configured with a GUI command to return the game parameters to their original settings. This GUI command is for example included in the master control area.
  • the modules 102 include RTPC indicators 122 adjacent to the property indicators 56-60. Each of these indicators 122 is associated to the adjacent property displayed and changes its appearance when the adjacent property is associated to an in-game parameter value using RTPCs.
  • the RTPC indicator is grey and becomes blue when associated to an in-game parameter.
  • the hierarchical structure according to the second illustrative embodiment has been found particularly suitable to manage, author and simulate audio-related objects.
  • Other hierarchical structure can be provided for other types of media.
  • the game syncs panels 104-108 may take other form.
  • the number and nature of the game syncs can be different than those described herein.
  • any parameters that change with the application into which the media object to simulate are to be integrated can be used.
  • simulation tool 20 has been described as being part of an authoring tool, it can be implemented as an independent application.
  • an independent application receives digital media-related elements from an external source, such as another application running on a same computer system or through a network such as Internet, and may output modified version of the digital media-related elements after simulation for example through an output file to be loaded an used by the other application.
  • the present Simulation Tool is not limited to a game audio simulator or the likes having a user interface as illustrated hereinabove with reference to the first and second embodiments.
  • Other layout and graphical means, including different browsers or user-menu objects can be provided without departing from the spirit and nature of the present invention.
  • the present Simulator has been described with reference to a digital game, it is believed to be within the reach of a person skilled in the art to modify the present Simulator for use in or for other digital media applications.
  • the present Simulator is also not limited to audio-related objects. It is indeed believed to be within the reach of a person skilled in the art to use the present teaching and to modify the Simulator according to the first or second illustrative embodiment for use in simulating other media objects, including without limitations animations and video.
  • a Simulator according to embodiments of the present invention can be adapted to simulate, audition, play, view, listen, or perform other media specific actions on media objects or on media-related objects.
  • Such media-related objects may include objects linked to or related to media objects in any way.
  • the digital media simulation tool according to the present invention has been described with reference to illustrative embodiments including examples of user interfaces. These user interfaces have been described for illustrative purposes and should not be used to limit the scope of the present invention in any way. They can be modified in many ways within the scope and functionalities of the present system and tools. For example, shortcut menus, input boxes, icon buttons, etc, can be provided interchangeably.

Abstract

A digital media simulation tool according to present invention comprises a control board including a master control and a module area for receiving media objects from an application, the media objects being characterized by properties. Inserting a media object in the control board causes the creation of a corresponding module which displays the properties and allows their modification during simulation. The modules further include playback buttons for causing the playing or the displaying of the media object. The modules can be played asynchronously and in any sequence. The master controller also includes playback buttons to cause the simultaneous pausing and stopping of all modules. According to some embodiments, the control board further includes application syncs panels which allow selecting elements from the application which trigger some change in any one of the media objects.

Description

TITLE
DIGITAL MEDIA SIMULATION TOOL AND METHOD THEREFOR
FIELD
[0001] The present invention relates to digital media such as text, audio, image, video, or a combination thereof. More specifically, the present invention is concerned with a tool which allows both simulating and authoring such digital media for example before their implementation in an application to be deployed on a device provided with computational capability.
BACKGROUND
[0002] Computer tools allowing for monitoring or profiling parts of a computer application by collecting performance data are well known in the art. These performance data can then be used to customize the application for a targeted hardware and software platform.
[0003] By nature, these profilers usually only allow collecting information about the application being developed and do not allow any modifications during the profiling process.
[0004] Simulation tools which are usually part of an application development tool or authoring package, and which allow developers to test how newly created elements blend together and interact with other components from the application are also known. A frequent drawback of these conventional simulation tools is that changes made during a simulation are not reflected in the application development tool. [0005] Moreover, those simulation tools which are specifically targeted to simulating digital media usually present the information as a media graph, wherein the media objects to simulate are presented connected separately to the modifiers that affect those media. This limits the playback and visualisation possibility of the media and also the possibility to make changes during simulation.
BRIEF DESCRIPTION OF THE DRAWINGS
[0006] In the appended drawings:
[0007] Figure 1 is a perspective view of a game audio authoring system, including a simulation tool according to a first illustrative embodiment of the present invention;
[0008] Figure 2 is an example of a user interface for a digital media simulation tool according to a first illustrative embodiment of the present invention;
[0009] Figure 3 is an example of a user interface for a module from the simulation tool from Figure 2;
[0010] Figure 4 illustrates the links between an audio file, audio sources and a sound object;
[0011] Figure 5 is an example illustrating the use of Events to drive the sound in a game;
[0012] Figure 6 is an example of a Platform Selector List from the simulation tool from Figure 2; [0013] Figure 7 is a close up view of the module area of the simulation tool from Figure 2, illustrating the module insertion indicator;
[0014] Figure 8 is a module similar to the module form Figure 3, further illustrating the game object list;
[0015] Figure 9 is a module similar to the module form Figure 3, further illustrating a Reset menu;
[0016] Figure 10 is an example of a user interface for a digital media simulation tool according to a second illustrative embodiment of the present invention;
[0017] Figure 11 is a first example of a first level hierarchical structure according to the second illustrative embodiment of the present invention;
[0018] Figure 12 is a second example of the first level hierarchical structure from Figure 11 , illustrating the use of containers to group sound objects;
[0019] Figure 13 illustrates a third example of a project hierarchy, including Master-Mixer and Actor-Mixer hierarchies;
[0020] Figure 14 is an example of a switch container for footstep sounds;
[0021] Figures 15A-15B are flow diagrams illustrating an example of use of a switch container; [0022] Figure 16 illustrates the use of a switch container in authoring a game;
[0023] Figure 17A illustrates an example of use of relative state properties on sounds;
[0024] Figure 17B illustrates an example of use of absolute state properties on sounds;
[0025] Figure 18 illustrates an example of use of states in a game;
[0026] Figure 19 is an example of a user interface for the States panel of the simulation tool from Figure 10;
[0027] Figure 20 is an example of a user interface for the Switches panel of the simulation tool from Figure 10; and
[0028] Figure 21 is an example of a user interface for the Real-time
Parameter Controls (RTPCs) panel of the simulation tool from Figure 10.
DETAILED DESCRIPTION
[0029] In accordance with a first aspect of the present invention, there is provided a digital media simulation tool comprising:
[0030] a control board for receiving at least one digital media-related element (DM-RE); the at least one DM-RE being associated to and characterized by at least one modifying value; each of the at least one modifying value being from a corresponding modifier; and [0031] at least one module included in the control board i) for displaying information related to the at least one DM-RE including the at least one modifying value associated thereto.
[0032] According to a second aspect of the present invention, there is provided a method in a computer system for simulating digital media comprising:
[0033] providing at least one digital media-related element (DM-RE) to simulate; the at least one DM-RE being associated to and characterized by at least one modifying value; each of the at least one modifying value being from a corresponding modifier; and
[0034] creating a module for each of the at least one DM-RE for displaying information related to the at least one modifying value and including at least one module simulation control with the module to allow re-creating the at least one DM-RE as characterized by the at least one modifying value.
[0035] According to a third aspect of the present invention, there is provided a computer-readable medium containing instructions for controlling a computer system to generate:
[0036] a control board for receiving at least one digital media-related element (DM-RE); the at least one DM-RE being associated to and characterized by at least one modifying value; each of the at least one modifying value being from a corresponding modifier; and
[0037] at least one module included in the control board i) for displaying information related to the at least one DM-RE including the at least one modifying value associated thereto, and ii) for selectively simulating the at least one DM-RE with the at least one modifying value associated thereto.
[0038] The computer-readable medium can be a CD-ROM, DVD-
ROM, universal serial bus (USB) device, memory stick, hard drive, etc.
[0039] The expression "computer application" is intended to be construed broadly as including any sequence of instructions intended for a computer, game console, wireless phone, personal digital assistant (PDA), multimedia player, etc. which produces sounds, animations, display texts, videos, or a combination thereof, interactively or not.
[0040] Similarly, the expression "computer system" will be used herein to refer to any device provided with computational capabilities, and which can be programmed with instructions for example, including without restriction a computer, a game console, a wired or wireless phone, a PDA (Personal Digital Assistant), a multimedia player, etc.
[0041] Other objects, advantages and features of the present invention will become more apparent upon reading the following non restrictive description of illustrated embodiments thereof, given by way of example only with reference to the accompanying drawings.
[0042] A digital media simulation tool 20 according to a first illustrative embodiment is described in Figure 2.
[0043] According to the first illustrative embodiment, the simulation tool 20 is part of a game audio authoring system 10 which produces or modifies audio objects to be used by a video game (not shown). [0044] For concision purposes, the simulation tool according to illustrative embodiments of the present invention will sometimes be referred to herein simply as "Simulator" or "Soundcaster".
[0045] The game audio authoring system 10 will now be briefly described with reference to Figure 1.
[0046] The system 10 comprises a computer 12 programmed with instructions for generating an Authoring Tool including the simulating tool 20, a display 14, conventional input devices, in the form, for example, of a mouse 16A and keyboard 16B, and six (6) sound speakers 18A-18F, including a sub- woofer 18A, configured to output sounds to discreet channels that can optionally be encoded in a 5.1 Dolby™ Digital setup. The system 10 is of course not limited to this setup. The number and type of input and output devices may differ and/or the sound output setup may also be different.
[0047] The system 10 also includes a conventional memory (not shown), which can be of any type. The system 10 can also be further configured for network connectivity 19. Since it is believed to be well-known in the art to provide a computer or other similar devices with network connectivity, such implementation will not be described herein in further detail.
[0048] Even though the system 10 is illustrated as a desktop type workstation, it is of course not limited to such an embodiment and can take any forms such as a laptop, a portable device, a tablet PC, etc. operating on any operating system.
[0049] As will be described hereinbelow in more detail, the simulation tool 20 allows: • prototyping and experimenting with the audio objects;
• auditioning audio objects simultaneously;
• auditioning audio objects asynchronously;
• mixing and testing audio objects.
[0050] The Simulator 20 includes a graphical user interface (GUI) allowing controlling the following aspects of a game audio:
• Master controls;
• Objects and events.
[0051] With reference to Figure 2, the Simulator 20 comprises a control board 22 including a master controller 24, and modules 26 received in a module area 28 of the control board 22 for simulating and editing audio-related objects provided from the game audio authoring tool.
[0052] As will be described furtherin in more detail with reference to the components 22-36 of the simulation tool 20, an audio simulation is built as follows:
• adding and removing audio-related objects into the control board 22;
• selecting and adding audio-related objects; • defining simulation settings; and
• auditioning.
[0053] Each of the components of the simulation tool 20 will now first be described briefly.
[0054] The simulation tool 20, which is part of the authoring tool, is called from a conventional menu (not shown) of the Authoring Tool.
[0055] The control board 22 is then displayed. The simulation tool 20 is configured so that adding an audio-related object therein causes the creation of a corresponding module 26, which is then displayed in the module area 28 thereof. This newly-created module 26 represents the audio-related object that has been inserted in the control board 22 for simulation and which is associated thereto.
[0056] Turning now to Figure 3, the modules 26 will be briefly presented. Since all the modules 26 share a common form or template and differ only by the audio-related element contained therein and the associated properties associated thereto, we will refer to the module 26 herein as to any one of the modules 26 indistinctively.
[0057] The module 26 is in the form of a display pane comprising a descriptive portion in the form of a title bar 30, module playback controls 32, module playback indicators 34 and module property controls 36.
[0058] The title bar 30 includes information for identifying the audio- related element which is represented in the module 26. It includes for example the name of the audio-related element and may also include an icon categorizing the element.
[0059] The module 26 includes module playback controls 32 to manage the playback of the audio-related element represented by the module 26. As will be described hereinbelow in more detail, the control board 22 includes master controls to simultaneously manage the playback of all the modules 26, and the audio-related elements they represent, included in the control board 22.
[0060] The module playback indicators 34 are provided to indicate when certain predetermined behaviors of the audio-related element occur during playback.
[0061] The module property controls 36 display property values associated to the element and allows selectively changing these values during simulation.
[0062] The components of the module 26 and their operation will be described hereinbelow in more detail.
[0063] According to the first illustrative embodiment, the audio- related objects are sound objects or events used to drive the sound in the game.
[0064] Sounds
[0065] The sound objects are in the form, for example, of imported audio files. The original audio files can be of many known types such as any type of well known PCM (Pulse-Code Modulation) audio file format such as the well-known WAV format. The original audio files can also be in a compressed format such as the well known MPEG-1 Audio Layer 3 (MP3) format.
[0066] The audio files can alternatively be streamed directly to the system 10 via, for example, but not limited to, a well-known voice-over-ip, protocol.
[0067] The Authoring Tool is configured so that the audio file importing process yields two copies of each audio file: the original copy and a work copy. The original copy remains untouched, but can be referenced to. The work copy is used for experimentation.
[0068] The importing process generally includes converting the audio files and creating audio sources for each audio file. The importing process further includes the creation in the Authoring Tool of the sound objects that contain the audio sources.
[0069] More specifically, the importing process includes the following subs-steps:
• the original audio files remain unmodified and are moved into an Originals folder;
• copies of the imported files are created and are stored for example in an Imported folder;
• audio sources are created for and from the audio files and sound objects that contain the audio sources are created and made available for editing in the Authoring Tool (see Figure 4).
[0070] A work copy of each audio file is then created for each of a series of game platform versions. This file is based on the corresponding Imported folder file and has been converted for the specific platform.
[0071] In summary, as illustrated in Figure 4, a created sound object references audio sources as they are created in the Authoring Tool from an imported audio file. These sources contain the conversion settings for each project platform.
[0072] The Authoring Tool is configured so that the audio conversion process retains the same pitch and duration as the original files. However, the following properties can be defined for the conversion: Channels, R-L Mix, Sample Rate, Audio Format, Bit Depth, Bit Rate and Filename Marker. Since these properties of audio files are believed to be well-known in the art, and for concision purposes, they will not be described furtherin.
[0073] The present Simulator is of course not limited to the import and audio conversion processes described above.
[0074] The Authoring Tool is further configured to modify the sound objects with modifiers. These modifiers can take any form allowing to be associated to and characterize the sound objects with modifying values, which can modify the sound objects or characterize them qualitatively or quantitatively. [0075] These modifiers include for example, and without limitations, properties and behaviors which can be associated to the sound objects.
[0076] Properties
[0077] Examples of properties that can be associated to the sound objects include:
• volume;
• LFE (Low Frequency Effect);
• pitch; and
• LPF (Low Pass Filter).
[0078] Other and/or additional properties can of course be supported by the Simulator 20.
[0079] The properties are characterized by property values which are assigned to the sound objects so as to characterize and modify them. Since the above-mentioned properties are believed to be well-known in the art, and for concision purposes, they will not be described herein in more detail.
[0080] The Authoring Tool is further programmed with a Randomizer to randomly modify selected property values of an object each time it is played. Sliders, input boxes and/or any other GUI input means are then provided to allow the user inputting a range of values for the randomizing effect. [0081] Behaviors
[0082] In addition to properties, each object can be characterized by behaviors.
[0083] The behaviors determine for example how many times a sound object will play each time it is called or the order it will play, and whether the sound is stored in memory or streamed directly from an external medium such as a DVD, a CD, or a hard drive. The Authoring Tool is configured such that different types of behaviors are made available from one object to another.
[0084] The Authoring Tool is configured so that, by default, sound objects play once from beginning to end. However, a loop can be created so that a sound will be played more than once. In this case, the number up to an unlimited number of times the sound will be looped is defined.
[0085] The Authoring Tool further allows setting which sounds will be played from memory and which ones will be streamed from the hard drive, CD, or DVD. When media is streamed from the disk or hard drive, an option is also available to avoid any playback delays by creating a small audio buffer that covers the latency time required to fetch the rest of the file. The size of the audio buffer can be specified so that it meets the requirements of the different media sources, such as hard drive, CD, and DVD.
[0086] Events
[0087] Events are provided to drive the sound in-game. Each event can have one or more actions or other events that are applied to the different sound objects to determine whether the objects will play, pause, stop, etc. [0088] The Authoring Tool is configured so that Events can be integrated into the game engine even before all the sound objects are available. For example, a simple event with just one action such as play can be integrated into a game. The event can then be modified and objects can be assigned and modified without any additional integration procedures required.
[0089] After the events are created, they can be integrated into the game engine so that they are called at the appropriate times in the game.
[0090] The icon illustrated in the following table is used both to facilitate the reference in the present description and also to help a user navigate in the various GUI provided in the Authoring Tool, including the Simulation Tool 20.
Icon Represents
Ei Event
Table 1
[0091] An example of use of events will now be provided with reference to Figure 5 which concerns a first person role-playing game. According to this game, the character will enter a cave from the woods in one level of the game. Events are used to change the ambient sounds at the moment the character enters the cave. At the beginning of the project, an event is created using temporary or placeholder sounds. The event contains a series of actions that will stop the ambient "Woods" sounds and play the ambient "Cave" sounds. After the event is created, it is integrated into the game so that it will be triggered at the appropriate moment. Since no additional programming is required after the initial integration, different sounds can be experimented with, actions can be added and removed, and action properties can be changed until it sounds as desired.
[0092] A variety of actions are associated to an Event to drive the sound in-game. The actions are grouped by category and each category contains a series of actions that can be selected.
[0093] Each action can be assigned a set of properties that can be used to fade in and fade out incoming and outgoing sounds. The following table describes examples of event actions that can be assigned to an Event in the Authoring Tool:
Event Action Description
Play Plays back the associated object.
Break Breaks the loop of a sound or the continuity of a container set to continuous without stopping the sound that is currently playing.
Stop Stops playback of the associated object.
Stop All Stops playback of all objects.
Stop All Except Stops playback of all objects except those specified.
Mute Silences the associated object.
Unmute Returns the associated object to its original
"pre-silenced" volume level.
Unmute All Returns all objects to their original "pre- silenced" volume levels.
Unmute All Except Returns all objects, except those specified, to their original "pre-silenced" volume levels.
Pause Pauses playback of the associated object. Pause All Pauses playback of all objects.
Pause All Except Pauses playback of all objects except those specified.
Resume Resumes playback of the associated object that had previously been paused.
Resume All Resumes playback of all paused objects.
Resume All Except Resumes playback of all paused objects, except those specified.
Set Volume Changes the volume level of the associated object.
Reset Volume Returns the volume of the associated object to its original level.
Reset Volume All Returns the volume of all objects to their original levels.
Reset Volume All Except Returns the volume of all objects, except those specified, to their original levels.
Set LFE Volume Changes the LFE volume level of the associated object.
Reset LFE Volume Returns the LFE volume of the associated object to its original level.
Reset LFE Volume All Returns the LFE volume of all objects to their original levels.
Reset LFE Volume All Except Returns the LFE volume of all objects, except those specified, to their original levels.
Set Pitch Changes the pitch for the associated object.
Reset Pitch Returns the pitch of the associated object to its original value.
Reset Pitch All Returns the pitch of all objects to their original values.
Reset Pitch All Except Returns the pitch of all objects, except those specified, to their original values. Set LPF Changes the amount of low pass filter applied to the associated object.
Reset LPF Returns the amount of low pass filter applied to the associated object to its original value.
Reset LPF All Returns the amount of low pass filter applied to all objects to their original values.
Reset LPF All Except Returns the amount of low pass filter applied to all objects, except the ones that are specified, to their original values.
Table 2
[0094] An Event can perform one action or a series of actions assigned thereto.
[0095] As briefly mentioned hereinabove, each event action is characterized by a set of related properties that can be used to further refine the sound in-game, which fall into, for example, one of the following possible categories:
• delays;
• transitions; and
• volume, pitch, etc.
[0096] Customizing Object Properties per Platform
[0097] The Authoring Tool according to the first illustrative embodiment is configured so that when sound properties are defined for a particular platform, these properties are set across all predetermined platforms by default. These properties are said to be linked across platforms. This streamlines creating projects across platforms.
[0098] As will now be described in further detail, the Authoring Tool allows customizing the properties and behaviors for a specific platform by unlinking selected properties and/or behaviors, and defining new values. For example, the following modifiers can be linked and unlinked:
• Effects;
• Volume;
• Pitch;
• Low Pass Filters;
. LFE;
• Streaming;
• Etc.
[0099] The Authoring Tool includes means for selecting and indicating the current platform. This means can for example be in the form of a Platform Selector List 38 (see Figure 6) provided in the tool bar. As it has been mentioned hereinabove, the modifying values assigned to an object are linked across all platforms unless a selected modifier is unlinked for the current platform selected. [00100] Excluding Objects and Event Actions from a Platform
[00101] The Authoring Tool includes excluder elements (not shown) associated to objects and allowing for including or excluding the corresponding object from the current platform.
[00102] The possibility to selectively exclude sounds from a project allows optimizing each platform for the available resources, customizing content for each version, optimizing content based on the limitations and strengths of certain platforms, optimizing performance based on the limitations and strengths of platforms, sharing of certain properties across versions, and creating content that will be delivered in different formats.
[00103] Switching Between Platforms
[00104] The Authoring Tool allows switching from one platform to another at any point in the development cycle by accessing the Platform Selector List 38 and selecting the desired platform. The platform versions of the sound objects in the hierarchy are then displayed.
[00105] Managing Objects or Events in the Simulating Tool
[00106] As described hereinabove, a module 26 is created when audio-related objects, in the form of a sound object or an event, is added to the Simulating Tool 20. The module 26 represents the object or event together with its properties and behaviors. The module controls 32-36 (see Figure 3) allow modifying properties and playing back sounds. [00107] The Simulator is configured so that an object or event can be added by dragging the audio-related object therein for example. Of course, other GUI means and user-menu means can also be used to import an object into the Simulator.
[00108] As can be seen in Figure 7, illustrating an audio-related object being inserted into the control board 22, the control board 22 is configured to display a module selection indicator 40 at the location where the audio-related object is dragged. When the mouse button is released, the selected object/event is added to the simulator as a module 26 at the location selected by the user, as indicated by the indicator 40.
[00109] The Simulator 20 is configured so that the order of the objects and events is not important when they are dragged into the Simulator 20 for their playback. The playback depends on which module 20 is selected to play as will be described furthering in more detail.
[00110] The Simulator 20 allows re-ordering of the modules to recreate sequences or facilitate testing. More specifically, the Simulator is configured so that a module 26 can be selected in the module area 28 of the control board 22 and positioned at a new location therein. The module 26 formerly in that position is shifted down in the module area 28. Conventional dragging functionalities are provided in the Simulator 20 to move the modules 26. More specifically, a module 26 can be dragged to a new location. When the module is moved, the module formerly in that position is shifted down in the Simulator GUI dedicated window.
[00111] It is believed to be within the reach of a person skilled in the art to use the present teaching so as to provide a similar simulating tool having a different module inserting scheme in the control panel 22 or inserting scheme when a module 26 is moved therein.
[00112] The Simulator 20 is further configured so that a module 26 that is no longer needed can be removed from the control board 22.
[00113] More specifically, with reference to Figure 3, each module 26 is provided with a dedicated GUI element in the form of a conventional icon button 42 in the title bar 30, which the user can "click" to remove the module 26.
[00114] The master controller 24 also includes an icon button 44 for removing all modules 26 included in the Simulator 20 (see Figure 2).
[00115] According to further embodiments, these GUI elements provided to remove modules can take other forms.
[00116] Managing Simulation Playback
[00117] The Simulator 20 according to the first illustrative embodiment includes the following non restrictive playback options:
• audition platform-specific objects and events;
• audition the original pre-converted audio files;
• audition objects and events associated with specific game objects when connected to the game; • play back modules in any order;
• manage playback for one or all modules at once;
• modify properties for modules; and
• modify event actions for objects.
[00118] These playback options will be described hereinbelow in more detail.
[00119] Specifying Sounds to be Played
[00120] Before using the control board 22, and more specifically one or more of the modules 26 to audition corresponding sounds, the Simulator 20 can be used to further discriminate which sounds to play.
[00121] The Simulator 20 includes tools to specify which sounds will play back based on:
• Platforms: As it has been described herein above, certain sounds are included or excluded in the audio based on platform during development. In a simulation it is possible to choose to play only the sounds that are in the current platform, or play all sounds in the module;
Converted Sounds: Also, as described hereinabove, when imported audio files are converted, an original version of the audio files is maintained. By default, the simulator plays the converted sounds; however, the original version can also be played back as will be described hereinbelow;
• Game Objects: As part of the game integration process, sounds are associated with game objects so that the correct sound will play in the game. Game objects are entities in game development to which an interface, a sound, a trigger and so on can be attached. The Simulator allows specifying that only the objects in each module that are associated with a specific game object will play back.
[00122] With reference to Figure 2, the control board 22 includes a platform discriminator, in the form of a first icon button 46, allowing enabling a Current-platform play mode, wherein only the objects and events in the current platform can be auditioned and modified. Of course, the desired platform for playback is first selected as the current platform using the platform selector list 38.
[00123] The control board 22 further includes an original sound selector 48, in the form of a second icon button, for selecting the original pre- converted sound for playback.
[00124] To play back sounds associated with specific game objects, each module 26 includes a game object list 50 and a connection to the game allows the user to select a game object for auditioning the associated sound.
[00125] As can be better seen from Figure 8, this list allows selecting the game object whose associated sounds is to be auditioned. [00126] The connection to the game can be achieved via a Remote connector (not shown) provided with the Authoring Tool. Since it is believed to be within the reach of a person skilled in the art to perform a remote connection between applications, this will not be described herein in more detail.
[00127] The platform discriminator 46 and original sound selector 48 are in the form of on/off switches allowing switching between modes by clicking on the corresponding icon.
[00128] Of course, other GUI elements can also be used for the platform discriminator 46, original sound selector 48, and game object list 50.
[00129] Auditioning the Sounds
[00130] After the sounds have been selected, they can be played back. The control board 22 is configured so that modules 26 can be played back in any order, consecutively, asynchronously, overlapping and so on. For example, changing the order for playback allows testing and experimenting and helps finding a play back sequence. The playback controls 32 of the module 26, which are in the form of Play, Pause and Stop buttons, can be used to cause the module to start, pause or stop playing respectively.
[00131] The master controller 24 of the control board 22 further includes pause and stop icon buttons 52-54 to cause to simultaneous pausing or stopping of all modules 26 in the module area of the control board 22.
[00132] Fine-Tuning Sound Properties during Simulation [00133] The Simulator 20 further allows events and objects to be mixed at the same time they are being auditioned. Object properties can also be modified prior to mixing.
[00134] To ease the mixing of the sounds and modifications of the properties associated thereto, the Simulator 20 includes property indicators 56- 60 associated to some of the properties and playback indicators 61.
[00135] The property indicators 56-60 are interactive GUI elements associated to properties, including but not limited to volume, pitch and low-pass filter, and positioning, each having a property value assigned to the object- related element represented by the module 26 and that modifies that object- related element. These GUI elements are in the form of a conventional input box 62 associated to a conventional slider 64, providing two means to visualize and edit the property value. Modifying a default value using the slider 64 causes the range between the new value and the default value to become highlighted.
[00136] These GUI elements can alternatively differ from one property to another and of course take other forms.
[00137] In addition, adjacent to some of the properties, link/unlink indicators 66 are provided to show whether the property value is linked to other platforms, and randomizer indicators 68 are also provided to indicate whether a Randomizer has been applied on the property value.
[00138] The following table describes examples of indicator changes which may occur in each of these situations: Indicator Name Description
Link A property value that is linked to the corresponding values of other game platforms.
Unlink A unique property value that is not linked to
(yellow) the corresponding values of other game platforms.
Partial Unlink The property value for the current platform is (half yellow) linked, but one or more corresponding values of other platforms are unlinked.
(yellow) Randomizer - A property value to which a Randomizer On effect has been applied. A Randomizer allows for defining a range of possible values that can be used randomly each time the object is played.
Randomizer - A property value to which no Randomizer Off effect has been applied.
Table 3
[00139] In addition to these property indicators, the Simulator 20 includes playback indicators 61 which become highlighted when particular behaviors or actions occur during playback. They are displayed in the master controller 24, and in the individual modules. The following table lists additional property and action parameter indicators in the Simulator 20:
Icon Name Indicates
Cl Delay A delay has been applied to an event or a sequence container.
Fade A fade has been applied to an event or a sequence container
Set Volume A set volume action has been applied to a sound object in an event. Set Pitch A set pitch action has been applied to a sound object in an event.
# Mute A mute action has been applied to a sound object in an event.
Set LFE Volume A set LFE volume action has been applied to the sound object in an event.
\ Set Low Pass Filter A set Low Pass Filter action has been applied to the sound object in an event.
Enable Bypass An Enable Bypass action has been applied to the sound object in an event.
Table 4
[00140] These icons and indicators are for illustrative purposes only.
Other icons or indicators can be used. Also, other playback indicators 61 can be used to indicate the state of any audio-related object or property associated thereto during simulation.
[00141] At any time during a simulation, including prior, after or during playback, the property indicators 56-60 can be used to modify the value of the property associated thereto. According to the first illustrative embodiment, this is achieved by using the slider 64 or input box 62 to modify the value. The Simulator 20 is further configured so that the user can access editing functionalities of the Authoring Tool for modifying the property values associated to the audio-related object during a simulation, for example by double-clicking the title bar of the corresponding module 26. It is believed to be within the reach of a person skilled in the art to provide other means to access the editing functionalities of the Authoring Tool or to another calling application.
[00142] The playback controls 32 and property indicators 56-60 define module simulation controls allowing re-creating the audio-related object represented by the module 26 as characterized by the different current property values set for example by the property indicators 56-60.
[00143] Resetting Event Actions
[00144] Event actions can temporarily change the properties of objects. The Simulator 20 allows applying the event actions to all the objects that are associated with the event. Reset functions, available through a Reset menu 70 via a corresponding button 72 on the property control area of the module 26 (see Figure 9), are provided to return the values for objects associated with an event to their initial values. With these functions, event actions can be reset in each module and for all modules in the master control area.
[00145] An example of a Reset menu is illustrated in Figure 9.
[00146] As can be seen in Figure 9, the Reset menu, which is associated with the reset function, includes the following non-restrictive options:
• Reset all: to reset all objects to their original settings;
• Reset Mute: to reset the mute for the objects associated with the event;
• Reset Pitch: to reset the pitch for the objects associated with the event;
• Reset Volume: to reset the volume for the objects associated with the event; • Reset LFE volume: to reset the LFE volume for all the objects associated with the event;
• Reset Low Pass Filter: to reset the low pass filter actions for all objects associated with the event; and
• Reset Bypass Effect: to clear all Bypass actions that have been triggered for the objects.
[00147] The Simulator 20 is further configured to allow event actions to be reset in the master controller area 24 of the control board 22. A "Reset all" menu similar to the "Reset" menu 70 available through the modules can be accessed on the master controller 24 via a "Reset All" button 74. A difference between the functions available though both menus is that all functions available through the "Reset all" menu effect all actions that have been triggered for the objects. Since both the "Reset" and "Reset all" menus are very similar, and for concision purposes, the Reset All menu and associated functionalities will not be described furthering.
[00148] According to the first illustrative embodiment, the Simulator
20 is so configured and integrated to the Authoring Tool that changes made to a property value assigned to one of the module 26 yields the same changes in the Authoring Tool even after the Simulator 20 is closed.
[00149] The Simulator 20 includes a "Test" button (not shown) allowing to selectively prevent changes made in the Simulator 20 to be made in the Authoring Tool.
[00150] Simulation Sessions [00151] According to a further aspect of the first illustrative embodiment, the Simulator allows saving the contents of a simulation, including the modules 26. This allows some of the simulations to be consulted and reused at a later time to audition changes, test, and validate work.
[00152] For this purpose, an input box 75 is provided to assign and then display a name to the current simulation session. A "New" icon button 76 is also provided to open a New Soundcaster Session dialog box where the name of the current session can also be entered.
[00153] The Authoring Tool includes a Soundcaster Manager (not shown) which allows to create and open already created sessions. Such a session manager can of course take many forms. Since such session manager is believed to be beyond the scope of the present invention, it will not be described herein in more detail.
[00154] Even though the Simulator 20 has been described as supporting a plurality of platforms, such functionality can be omitted.
[00155] Also, the Simulator 20 is not limited to receive Audio objects which have been imported and converted, and can be modified to receive and play original files.
[00156] The layout and interface elements of the module 26 may also be different.
[00157] Turning now to Figure 10, a digital media simulation tool 100 according to a second illustrative embodiment will be described. [00158] Since the simulation tool 100 is very similar to the simulation tool 20, and for concision purposes, only the differences between the two tools will be described herein.
[00159] According to this second illustrative embodiment and similarly to the Simulator 20, the simulation tool 100 is part of a game audio authoring system 10 which produces or modifies audio objects to be used by a video game (not shown).
[00160] The Simulator 100 comprises a control board 22 including a master controller 24, and modules 102 received in a module area 28 of the control board 22.
[00161] The Simulator 100 further comprises audio/game interaction panels in the form of game syncs panels 104-108 displayed in a game syncs area 110 of the control boards 22.
[00162] Simulating with Game Syncs
[00163] According to the second illustrative embodiment, the
Authoring Tool is further provided with elements that are mapped to the game to allow the sound to match conditions in the game. This allows the audio to react to what is going on in the game. These elements, which will be referred to herein as game syncs, react when the game triggers a shift in action or environment, and provides a corresponding shift in the audio.
[00164] Three types of game syncs are provided to streamline and handle the audio shifts that are part of the game: • States: global property changes in the existing sounds that reflect the environment of the game;
• Switches: the change in the actual sounds in the game audio to reflect the environment of the game;
• RTPCs (Real-time Parameter Controls): game parameters that are mapped to audio properties so that when the game parameters change, the mapped audio properties will also reflect the change.
[00165] In addition to these game syncs, the Simulator 100 according to the second embodiment allows handling audio-related objects from a hierarchical structure, which originates for example from the Authoring Tool, to edit and manage the sound assets and which will now be described.
[00166] According to this hierarchical structure, the sound objects can be grouped and organized in a first hierarchical structure, yielding a tree-like structure including parent-child relationships whereby when properties and behaviors are assigned to a parent, these properties and behaviors are shared by the child thereunder.
[00167] As illustrated in Figure 11 , the hierarchical structure includes containers (C) to group sound objects (S) or other containers (C), and actor- mixers (AM) to group containers (C) or sound objects (S) directly, defining parent-child relationships between the various objects.
[00168] As will be described hereinbelow in more detail, sound objects (S), containers (C), and actor-mixers (AM) all define object types which can be characterized by properties, such as volume, pitch, and positioning, and behaviors, such as random or sequence playback.
[00169] Also, by using different object types to group sounds within a project structure, specific playback behaviors of a group of sounds can be defined within a game.
[00170] The following table summarizes the objects that can be added to a project hierarchy:
Object Icon Description
Sounds Objects that represent the individual audio asset and contain the audio source. There are two kinds of sound objects:
Sound SFX- sound effect object
Sound Voice— sound voice object.
Containers A group of objects that contain sound objects or other containers that are played according to certain behaviors. Properties can be applied to containers which will affect the child objects therein. There are three kinds of containers:
Random Containers— group of one or more sounds and/or containers that can be played back in a random order or according to a specific playlist.
Sequence Container-group of one or more sounds and /or containers that can be played back according to a specific playlist. Switch Container— A group of one or more containers or sounds that correspond to changes in the game.
Actor-Mixers ||j] High level objects into which other objects such as sounds, containers and/or actor-mixers can be grouped. Properties that are applied to an actor-mixer affect the properties of the objects grouped under it.
Master-Mixers m Master Control Bus/Control Bus
Table 5
[00171] The icons illustrated in the above table are used both to facilitate the reference in the present description and also to help a user navigate in the control board 22 of the Simulator 100. Other icons can of course be implemented.
[00172] With reference to Figure 12, containers define the second level in the Actor-Mixer Hierarchy. Containers can be both parent and child objects. Containers can be used to group both sound objects and containers. As will be described hereinbelow in more detail, by "nesting" containers within other containers, different effects can be created and realistic behaviors can be simulated.
[00173] Actor-mixers sit one level above the container. Actor-mixers can be the parent of any number of sounds, containers, and other actor-mixers. They can be used to group a large number of objects together to apply properties to the group as a whole. [00174] Figure 12 further illustrates the use of actor-mixers to group sound objects, containers, and other actor-mixers.
[00175] The characteristics of the random, sequence and switch containers will be described hereinbelow in more detail.
[00176] The above-mentioned hierarchy, including the sound objects, containers, and actor-mixers will be referred to herein as the Actor-Mixer hierarchy.
[00177] An additional hierarchical structure sits on top of the Actor-
Mixer hierarchy in a parent-like relationship: the Master-Mixer hierarchy. The Master-Mixer Hierarchy is a separate hierarchical structure of control busses that allows re-grouping the different sound structures within the Actor-Mixer Hierarchy and preparing them for output. The Master-Mixer Hierarchy consists of a top-level "Master Control Bus" and any number of child control busses below it. Figure 13 illustrates an example of a project hierarchy including Master-Mixer and Actor-Mixer hierarchies. As can also be seen in Figure 13, the Master-Mixer and control busses can be identified by specific icons.
[00178] The child control busses allow grouping the sound structures according to main sound categories within the game as defined by a user or as pre-defined.
[00179] These control busses create the final level of control for the sound structures within the project. They sit on top of the project hierarchy allowing creating a final mix for the game. [00180] Since the control busses group complete sound structures, they can further be used to troubleshoot problems within the game. For example, they allow muting the voices, ambient sounds, and sound effects busses, to troubleshoot the music in the game. An object within the hierarchy is routed to a specific bus.
[00181] Properties
[00182] According to this second illustrative embodiment, the properties of an object can be divided into two categories:
• relative properties, which are cumulative and are defined at each level of the hierarchy, such as pitch and volume. The sum of all these values determines the final property; and
• absolute properties, which are defined at one level in the hierarchy, usually the highest.
[00183] Behaviors
[00184] Object behaviors determine which sound within the hierarchy will be played at any given point in the game. Unlike properties, which can be defined at all levels within the hierarchy; behaviors can be defined for sound objects and containers. The Authoring Tool is also configured so that the types of behaviors available differ from one object to another as will be described furtherin.
[00185] Containers [00186] Since different situations within a game may require different kinds of audio playback, the hierarchical structure allows for grouping objects into different kinds of containers such as the following three different types of containers:
• Random containers;
• Sequence containers;
• Switch containers.
[00187] Each container type includes different settings which can be used to define the playback behavior of sounds within the game. For example, random containers play back the contents of the container randomly, sequence containers play back the contents of the container according to a playlist, and switch containers play back the contents of the container based on the current switch, state or RTPC within the game. A combination of these types of containers can also be used.
[00188] Defining How Objects Within a Container are Played
[00189] Since both random and sequence containers usually consist of more than one object, one of the following two play modes can be associated thereto:
• Step: to play only one object in the container each time the container is played; Continuous: to play the complete list of objects in the container each time the container is played. This mode further allows looping the sounds and creating transitions between the various objects within the container.
[00190] Switch Containers
[00191] Switch containers are provided to group sounds according to different alternatives existing within a game. More specifically, they contain a series of switches or states or Real-time Parameter Controls (RTPC) that correspond to changes or alternative actions that occur in the game. For example, a switch container for footstep sounds might contain switches for grass, concrete, wood and any other surface that a character can walk on in game (see Figure 14).
[00192] The icons illustrated in the following table are used both to facilitate the reference in the present description and also to help a user navigate in the Simulator 100.
Figure imgf000040_0001
Table 6
[00193] Each of these three game syncs will now be described in further detail. [00194] Each switch/state includes the audio objects related to that particular alternative. For example, all the footstep sounds on concrete would be grouped into the "Concrete" switch; all the footstep sounds on wood would be grouped into the "Wood" switch, and so on. When the game calls the switch container, the sound engine verifies which switch/state is currently active to determine which container or sound to play.
[00195] Figures 15A-15B illustrate what happens when an event calls a switch container called "Footsteps". This container has grouped the sounds according to the different surfaces a character can walk on in game. In this example, there are two switches: Grass and Concrete. When the event calls the switch container, the character is walking on grass (Switch=Grass), so the footstep sounds on grass are played. A random container is used to group the footstep sounds within the switch so that a different sound is played each time the character steps on the same surface.
[00196] Switch
[00197] The concept of switches will now be described in further detail.
[00198] As mentioned hereinabove, switches represent the alternatives that exist for a particular element in game, and are used to help manage the corresponding sounds for these alternatives. In other words, sounds are organized and assigned to switches so that the appropriate sounds will play when the changes take place in the game.
[00199] Returning to the surface switch example which began with reference to Figures 14, 15A and 15B, the Authoring Tool allows creating a switch called "concrete" and assigning a container with footstep sounds that match the concrete surface to this switch. Switches for grass, gravel and so on can also be created, and corresponding sounds assigned to these switches.
[00200] In operation, the sounds and containers that are assigned to a switch are grouped into a switch container. When an event signals a change in sounds, the switch container verifies the switch and the correct sound is played.
[00201] With reference to Figures 15A-15B and 16, when the main character of a game walks on a concrete surface for example, the "concrete" switch and its corresponding sounds are selected to play, and then if the character moves from concrete to grass, the "grass" switch is called by the sound engine.
[00202] Before being used in a switch container, switches are first grouped in switch groups. Switch groups contain related segments in a game based on the game design. For example, a switch group called "Ground Surfaces" can be created for the "grass" and "concrete" switches illustrated in Figures 15A-15B and 16 for example.
[00203] The icons illustrated in the following table are used both to facilitate the reference in the present description and also to help a user navigate in the Simulator 16.
Figure imgf000042_0001
Table 7 [00204] States
[00205] States are provided in the Authoring Tool to apply global property changes for objects in response to game conditions. Using a state allows altering the properties on a global scale so that all objects that subscribe to the state are affected in the same way. As will become more apparent upon reading the following description, using states allows creating different property kits for a sound without adding to memory or disk space usage. By altering the property of sounds already playing, states allow reusing assets and saving valuable memory.
[00206] A state property can be defined as absolute or relative. As illustrated in Figures 17A and 17B, and similarly to what has been described hereinabove, applying a state whose properties are defined as relative causes the effect on the object's properties to be cumulative.
[00207] Applying a state whose properties are defined as absolute causes the object's properties to be ignored and the state properties is used.
[00208] An example illustrating the use of states is shown in
Figure 18. This example concerns the simulation of the sound treatment that occurs when a character goes underwater in a video game. In this case, a state can be used to modify the volume and low pass filter for sounds that are already playing. These property changes create the sound shift needed to recreate how gunfire or exploding grenades would sound when the character is under water.
[00209] Similarly to switches, before being usable in a project, states are first grouped in state groups. For example, after a state group called Main Character has been created, states can be added that will be applied to the properties for the objects associated with the Main Character. From the game, it is for example known that the main character will probably experience the following states: stunned, calm, high stress. So it would be useful to group these together.
[00210] The icons illustrated in the following table are used both to facilitate the reference in the present description and also to help a user navigate in the Simulator 100.
Figure imgf000044_0001
Table 8
[00211] After states have been created, they can be assigned to objects from the hierarchy. The first step is to choose a state group. The Authoring Tool is configured so that by default all states within that state group are automatically assigned to the object, and so that the properties for each individual state can then be altered.
[00212] RTPCs
[00213] Real-time Parameter Controls (RTPCs) are provided to edit specific sound properties in real time based on real-time parameter value changes that occur within the game. RTPCs allow mapping the game parameters to property values, and automating property changes in view to enhancing the realism of the game audio. [00214] For example, using the RTPCs for a racing game allows editing the pitch and the level of a car's engine sounds based on the speed and RPM values of an in-game car. As the car accelerates, the mapped property values for pitch and volume react based on how they have been mapped. The parameter values can be displayed, for example, in a graph view, where one axis represents the property values in the project and the other axis represents the in-game parameter values.
[00215] The Authoring Tool is configured so that the project RTPC values can be assigned either absolute values, wherein the values determined for the RTPC property will be used and ignore the object's properties, or relative values, wherein the values determined for the RTPC property will be added to the object's properties. This setting is predefined for each property.
[00216] An example of use of RTPCs to base the volume of the character's footstep sounds on the speed of the character in game will now be provided with reference to a first shooter game. For example, when the character walks very slowly, it is desirable according to this example that the footstep sounds be very soft and that when the character is running, that the sounds be louder. In this case, RTPCs can be used to assign the game parameter (speed) to the project property (volume). Then a graph view can be used to map the volume levels of the footstep sounds to the speed of the character as it changes in game.
[00217] RTPCs can also be used to achieve other effects in a game, such as mapping low pass filter values to water depth, low frequency effect values to the force of an explosion, and so on. [00218] Returning to Figure 10, the game syncs panels 104-108 allow access to the game syncs. The panels 104-108 can be used to enable game syncs during playback of audio-related objects as will now be described.
[00219] States, switches, and RTPCs can be auditioned in the application and in game. For example, by connecting to the game and adding the appropriate modules 102 to the Simulator 100, audio-related objects can be auditioned, tested, and mixed in real time with the game action using game syncs.
[00220] The Simulator 100 is configured to receive any object from the hierarchy, including: events, sound objects, random containers, sequence containers, switch container, and control buses.
[00221] Enabling States during Playback
[00222] The Simulator 100 is so configured that, when an object is dragged therein, the state groups and states enabled for the sounds in the module 102 corresponding to the object are added in a States list menu 105 which is included in the States panel 104 (see Figure 19). The States list menu includes a first list of state groups 112 enabled for any one of the objects represented by one of the modules included in the Simulator 100. A list of states 114 is provided for each of these state groups 112. The list of states 114 is for example in the form of a drop down menu.
[00223] As can be seen in Figure 10, the States panel 104 further includes a "Show All" button 115 to force the States list menu 105 to show all state groups and states that have been created in the entire project and not just the states associated with the module 102. [00224] The States panel allows enabling a state during playback. A selected state is applied to all sounds in the Simulator 100 that subscribe to the selected state during playback.
[00225] The master controller 24 of the Simulator 100 includes a user interface element for returning to the default or initial state specified for example by the Authoring Tool. This interface element is for example in the form of a "Reset All States" option from a shortcut menu which is available from the (») button 116 on the master controller 24.
[00226] Assigning Switches During Playback
[00227] Similarly to the States panel 104, the Switch panel 106 includes a Switches list menu 107 which includes the list of all switches 116 that are assigned to objects presented by any one of the modules 102 (see Figure 20). These switches, which populate the list 116 as audio-related objects are inserted in the Simulator 100, can be used during playback. The switch containers in the simulation will play the sounds that correspond with the switches that are selected.
[00228] The Switches panel 106 further includes a "Show All" button
117 to force the Switches list menu to show all switch groups and switches and not just those that have are associated with the module 102.
[00229] A "Reset All Switches" option is available from the Master controller to return the Switch Container to a predetermined default switch.
[00230] Simulating Changes in Game Parameters [00231] As can be seen from Figure 21 , the Simulator 100 includes a user interface, in the form of the RTPC panel 108, which allows controlling the game parameter values using a range of values.
[00232] The Simulator 100 is configured so that when an audio- related object is inserted therein, the game parameters 118 and the associated mapped objects and properties assigned to the game parameters 118 in the module are added in the RTPCs panel 108.
[00233] The RTPCs panel 108 includes a "Show All" button 119 to force the RTPCs panel 108 to display all game parameters that have been created in the entire project, and not just those associated with the module 102.
[00234] The RTPCs panel 108 includes a user interface element 120 associated to each game parameters 118 to allow to change the game parameters values to audition the mapped sound properties. The user interface element 120 includes a conventional slider, which represents the range of game parameter values. Since these values have already been mapped to object property values, the property values automatically change as the parameter values change.
[00235] The Simulator 100 is configured so that a game parameter
118 can be modified and auditioned during playback by selecting it in the RTPCs panel 108 of the Simulator 100.
[00236] The game parameters that have been mapped to the module objects selected 102 will be applied during playback. [00237] The simulator is also configured with a GUI command to return the game parameters to their original settings. This GUI command is for example included in the master control area.
[00238] A " Reset All Game Parameters" option is available from the
Master controller to return the game parameters to their original settings.
[00239] The modules 102 include RTPC indicators 122 adjacent to the property indicators 56-60. Each of these indicators 122 is associated to the adjacent property displayed and changes its appearance when the adjacent property is associated to an in-game parameter value using RTPCs.
[00240] According to the second illustrative embodiment, the RTPC indicator is grey and becomes blue when associated to an in-game parameter.
[00241] Even though the simulation tool according to the second illustrative embodiment has been described with reference to audio-related objects part of a hierarchical structure and which can react to changes in the application through game syncs, a person skilled in the art would understand that these two aspects are not necessarily dependable upon each other. For example, it is believed to be within the reach of a person skilled in the art to use the above teaching to conceive a simulation tool similar to the Simulator 20 where the media objects are part of a hierarchical structure as described herein, or where the media objects can react to changes through game or application syncs.
[00242] The hierarchical structure according to the second illustrative embodiment has been found particularly suitable to manage, author and simulate audio-related objects. Other hierarchical structure can be provided for other types of media.
[00243] The game syncs panels 104-108 may take other form.
[00244] Also, according to further embodiments of the present invention and depending for example on the application of the Simulator, the number and nature of the game syncs can be different than those described herein. For example, any parameters that change with the application into which the media object to simulate are to be integrated can be used.
[00245] Even though the simulation tool 20 has been described as being part of an authoring tool, it can be implemented as an independent application. Such an independent application receives digital media-related elements from an external source, such as another application running on a same computer system or through a network such as Internet, and may output modified version of the digital media-related elements after simulation for example through an output file to be loaded an used by the other application.
[00246] The present Simulation Tool is not limited to a game audio simulator or the likes having a user interface as illustrated hereinabove with reference to the first and second embodiments. Other layout and graphical means, including different browsers or user-menu objects can be provided without departing from the spirit and nature of the present invention.
[00247] Even though the present Simulator has been described with reference to a digital game, it is believed to be within the reach of a person skilled in the art to modify the present Simulator for use in or for other digital media applications. [00248] The present Simulator is also not limited to audio-related objects. It is indeed believed to be within the reach of a person skilled in the art to use the present teaching and to modify the Simulator according to the first or second illustrative embodiment for use in simulating other media objects, including without limitations animations and video.
[00249] A Simulator according to embodiments of the present invention can be adapted to simulate, audition, play, view, listen, or perform other media specific actions on media objects or on media-related objects. Such media-related objects may include objects linked to or related to media objects in any way.
[00250] The digital media simulation tool according to the present invention has been described with reference to illustrative embodiments including examples of user interfaces. These user interfaces have been described for illustrative purposes and should not be used to limit the scope of the present invention in any way. They can be modified in many ways within the scope and functionalities of the present system and tools. For example, shortcut menus, input boxes, icon buttons, etc, can be provided interchangeably.
[00251] Although the present invention has been described hereinabove by way of illustrated embodiments thereof, it can be modified, without departing from the spirit and nature of the subject invention as defined in the appended claims.

Claims

WHAT IS CLAIMED IS:
1. A digital media simulation tool comprising: a control board for receiving at least one digital media-related element (DM-RE); the at least one DM-RE being associated to and characterized by at least one modifying value; each of the at least one modifying value being from a corresponding modifier; and at least one module included in the control board i) for displaying information related to the at least one DM-RE including the at least one modifying value associated thereto.
2. A simulation tool as recited in claim 1 , wherein the at least one module is created and displayed in the control board when the at least one DM-RE is received therein.
3. A simulation tool as recited in claim 1 , wherein the at least one module includes at least one property indicator for displaying the at least one modifying value.
4. A simulation tool as recited in claim 1 , wherein the at least one module being further ii) for selectively simulating the at least one DM-RE with the at least one modifying value associated thereto.
5. A simulation tool as recited in claim 4, wherein the at least one module includes at least one module control for at least one of starting, pausing and stopping said simulating the at least one DM-RE.
6. A simulation tool as recited in claim 4, wherein the control board further includes at least one playback indicator indicative of a state of at least one of the at least one DM-RE and at least one modifying value associated thereto during said simulating the at least one DM-RE with the at least one modifying value associated thereto.
7. A simulation tool as recited in claim 1 , wherein the at least one module is further ii) for selectively changing the at least one modifying value from an initial value to a new value.
8. A simulation tool as recited in claim 7, wherein the control board is coupled to an application for receiving the at least one DM-RE therefrom.
9. A simulation tool as recited in claim 8, wherein said changing the at least one modifying value in the simulation tool causes the at least one modifying value to be changed in the application.
10. A simulation tool as recited in claim 8, wherein the application is a game.
11. A simulation tool as recited in claim 7, wherein the at least one module includes at least one modifier control for said selectively changing the at least one modifying value.
12. A simulation tool as recited in claim 7, wherein the at least one module includes at least one property indicator associated to the at least one modifier control for displaying the at least one modifying value.
13. A simulation tool as recited in claim 7, wherein the at least one module includes a reset function for returning the at least one modifying value to the initial value.
14. A simulation tool as recited in claim 1 , wherein the at least one DM-RE including a plurality of DM-REs; the at least one module including a plurality of modules; each one of the modules being created for a corresponding one of the DM-REs.
15. A simulation tool as recited in claim 14, wherein the control board further includes a master controller to simultaneously manage the plurality of modules.
16. A simulation tool as recited in claim 15, wherein the master controller includes master controls for simultaneously stopping or pausing at least two of the modules being simulated.
17. A simulation tool as recited in claim 14 for a computer application.
18. A simulation tool as recited in claim 17, wherein the DM- REs include at least one event for driving at least one of the other DM-REs in the computer application.
19. A simulation tool as recited in claim 14, wherein the plurality of modules being further ii) for selectively simulating the DM-REs, each with a corresponding one of the at least one modifying value associated thereto.
20. A simulation tool as recited in claim 19, wherein some of the DM-REs include different platform versions thereof; the control board including a platform discriminator for enabling a current-platform mode wherein only selected DM-REs from the plurality of DM-REs corresponding to a platform selected by the platform discriminator are simulated.
21. A simulation tool as recited in claim 19, wherein each of the DM-REs includes an original version and a converted version thereof; the control board including an original version selector for selectively simulating the original versions of the DM-REs.
22. A simulation tool as recited in claim 19, wherein the DM- REs are to be inserted in an application.
23. A simulation tool as recited in claim 22, wherein the control board being further for receiving at least one sync parameter associated to at least one of the DM-REs; the at least one sync parameter being representative of a change in the application.
24. A simulation tool as recited in claim 23, wherein the at least one sync parameter includes a plurality of sync parameters; each of the sync parameters being associated to at least one of the DM-REs; the control board including a sync panel for displaying the sync parameters and for allowing selecting one therefrom; whereby, in operation, the plurality of modules selectively simulating the DM- REs associated to the selected sync parameter.
25. A simulation tool as recited in claim 24, wherein the sync parameter is an application state causing at least one of the modifying values associated to the DM-REs to change.
26. A simulation tool as recited in claim 24, wherein initial sync parameters from the plurality of sync parameters are associated to selected one of the DM-REs prior to said receiving at least one sync parameter associated to at least one of the DM-REs; the control board including a reset function to force the initial sync parameters to be associated back to the selected one of the DM-REs.
27. A simulation tool as recited in claim 23, wherein the at least one sync parameter includes a plurality of application parameters, each having a mapping with one of the DM-REs; the control board including a parameter panel for displaying the application parameters and for modifying the mapping therebetween.
28. A simulation tool as recited in claim 14, wherein the DM- REs are part of a hierarchical structure including at least one container element for grouping at least one selected DM-REs among the DM-REs so that when a selected one the at least one modifying value is assigned thereto, the selected one the at least one modifying value is shared to the at least one selected DM- REs.
29. A simulation tool as recited in claim 1 , wherein the at least one DM-RE is in the form of a sound-related object.
30. A simulation tool as recited in claim 29, wherein the sound- related object is from a file or from a stream.
31. A simulation tool as recited in claim 1 which is part of an authoring tool.
32. A simulation tool as recited in claim 1 which is part of a computer system.
33. A method in a computer system for simulating digital media comprising: providing at least one digital media-related element (DM-RE) to simulate; the at least one DM-RE being associated to and characterized by at least one modifying value; each of the at least one modifying value being from a corresponding modifier; and creating a module for each of the at least one DM-RE for displaying information related to the at least one modifying value and including at least one module simulation control with the module to allow re-creating the at least one DM-RE as characterized by the at least one modifying value.
34. A method as recited in claim 33, wherein providing at least one DM-RE includes providing a plurality of DM-REs.
35. A method as recited in claim 34, further comprising simulating at least some of the DM-REs.
36. A method as recited in claim 35, wherein the at least some of the DM-REs are simulated asynchronously, consecutively or simultaneously.
37. A method as recited in claim 34 wherein the DM-REs are to be used in a computer application.
38. A method as recited in claim 37, wherein the DM-REs include at least one event for driving at least one of the other DM-REs in the computer application.
39. A method as recited in claim 33, further comprising using the at least one simulation control to select another modifying value from the corresponding modifier.
40. A method as recited in claim 33, further comprising saving information related to the module for each of the at least one DM-RE and to the information related to the at least one modifying value in a Simulation Session.
41. A computer-readable medium containing instructions for controlling a computer system to generate: a control board for receiving at least one digital media-related element (DM-RE); the at least one DM-RE being associated to and characterized by at least one modifying value; each of the at least one modifying value being from a corresponding modifier; and at least one module included in the control board i) for displaying information related to the at least one DM-RE including the at least one modifying value associated thereto.
PCT/CA2007/000171 2006-02-09 2007-02-07 Digital media simulation tool and method therefor WO2007090273A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US77143606P 2006-02-09 2006-02-09
US60/771,436 2006-02-09

Publications (1)

Publication Number Publication Date
WO2007090273A1 true WO2007090273A1 (en) 2007-08-16

Family

ID=38344830

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA2007/000171 WO2007090273A1 (en) 2006-02-09 2007-02-07 Digital media simulation tool and method therefor

Country Status (1)

Country Link
WO (1) WO2007090273A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1994028480A1 (en) * 1993-05-24 1994-12-08 Media Station, Inc. Interactive multimedia development system and method
JPH07168864A (en) * 1993-12-16 1995-07-04 Ricoh Co Ltd Simulating device and method thereof

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1994028480A1 (en) * 1993-05-24 1994-12-08 Media Station, Inc. Interactive multimedia development system and method
JPH07168864A (en) * 1993-12-16 1995-07-04 Ricoh Co Ltd Simulating device and method thereof

Similar Documents

Publication Publication Date Title
US20070185909A1 (en) Tool for authoring media content for use in computer applications or the likes and method therefore
US20070157173A1 (en) Method and system for multi-version digital authoring
US8438482B2 (en) Interactive multimedia content playback system
Stevens et al. The game audio tutorial: A practical guide to creating and implementing sound and music for interactive games
US10062367B1 (en) Vocal effects control system
CN101542588B (en) Mashing-up data file, mashing-up device and contents making-out method
US9076264B1 (en) Sound sequencing system and method
IES20130120A2 (en) A system and method for generating an audio file
JP2014520352A (en) Enhanced media recording and playback
Lanham Game Audio Development with Unity 5. X
Paterson et al. Interactive digital music: enhancing listener engagement with commercial music
Comunità et al. Web-based binaural audio and sonic narratives for cultural heritage
WO2007090273A1 (en) Digital media simulation tool and method therefor
Pachet et al. MusicSpace: a Constraint-Based Control System for Music Spatialization.
Arrasvuori Playing and making music: Exploring the similarities between video games and music-making software
Jago Adobe Audition CC Classroom in a Book
US20010042145A1 (en) Method for combining multimedia data with audio data from a compact disk
US11922910B1 (en) System for organizing and displaying musical properties in a musical composition
Adobe Creative Team et al. Adobe Audition CS6 Classroom in a Book
Skjulstad Circuit bending as an aesthetic phenomenon
Calegario et al. Sketchument: Empowering users to build DMIs through prototyping
Dorigatti et al. DESIGNING A LIBRARY FOR GENERATIVE AUDIO IN UNITY
Sukoinen Audio implementation methods in Unity
Kjeldskov et al. Spatial mixer: Cross-device interaction for music mixing
Collins Choosing and Using Audio and Music Software: A guide to the major software applications for Mac and PC

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 07701763

Country of ref document: EP

Kind code of ref document: A1