JP2007293338A - Control of multiple user environment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To dynamically control action of one attendant or a plurality of attendants by a plurality of users in multiple user environment. <P>SOLUTION: A multiple user system for bi-directionally controlling action of at least a first attendant in one environment, comprises: a first device which a first user can operate so that action of the attendant in the environment may be dynamically controlled; and a second device which a second user can operate at the same time so that the environment may be dynamically changed. A multiple user bi-directional control method of at least the first attendant in the environment, comprises: a step for dynamically controlling action of the first attendant in the environment; and a step in which the second user dynamically changes the environment. A plurality of described embodiments include music environment in which the attendant is one musical instrument, and game environment in which the attendant is a character of a certain game. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  Embodiments of the present invention relate to a multi-user environment in which the actions of one participant or multiple participants can be dynamically controlled by multiple users.

  As an example of such an environment, there is a game environment. In this game environment, individual users control participating characters in the game, and participating characters compete with each other. The user can communicate in real time through a personal computer connected to the Internet or using a portable device.

  According to one aspect of the present invention, a system for interactive music production is provided, the system comprising a first portable device with a first user input for controlling music synchronization information. And an input means operable to receive the music synchronization information, a second user input unit for controlling the content of the first music control data, the first music control data and the music synchronization information And a second portable electronic device with a synthesizer for producing music.

  In accordance with another aspect of the present invention, a method is provided for interactively producing music that changes music synchronization information at a first portable device in response to input from a first user. Receiving the music synchronization information at the second portable device; creating first music control data at the second portable device in response to an input from the second user; Producing music in the second device according to the first music control data and the music synchronization information.

  For a better understanding of the present invention, reference will now be made by way of example only to the accompanying drawings in which:

  FIG. 1 shows a network 4 comprising a first portable electronic device 1, a second portable electronic device 2 and a third portable electronic device 3. The network can be a local network so that device users may be 'face-to-face' or users may be remote from each other. The network 4 may be formed by any suitable mechanism with direct or indirect wireless or physical connections of two or more devices. One connection mechanism uses a low power radio frequency transceiver (such as Bluetooth), which allows the portable device to move without restriction. The network 4 enables cooperation and execution of the device as an ensemble.

  FIG. 2 is a diagram schematically illustrating a portable electronic device 1 that synthesizes music. The portable electronic device 1 includes a user input interface (UI) 11, a processor 12, a memory 13, a display unit 14, an input unit 15 that receives data, an output unit 16 that transmits data, and an audio output unit. 20.

  The user input interface 11 is connected to the processor 12 and allows the user of the device 1 to control the operation of the device via the processor 12. The processor 12 is connected to the input unit 15 and the output unit 16. The processor 12 is operable to receive data received via the input unit 15, process this data, and output the data to the output unit 16 for transmission. The processor 12 is connected to the memory 13 and is operable to read from and write to the memory 13. The processor 12 is operable to control the display unit 14. In another embodiment, the display unit and the user input interface may be combined. The input unit 15 and the output unit 16 may be a radio frequency receiver and a transmitter, respectively. Alternatively, the input unit 15 and the output unit 16 may be part of a single physical interface (such as a USB port). The input unit 15 and the output unit 16 allow the device 1 to network with another device.

  The audio output unit 20 may be the same as that of a sound card of a personal computer. The audio output unit 20 includes a MIDI engine 22, is connected to a MIDI synthesizer 24, and is then connected to a speaker 26 (or another audio output unit such as a jack or a headset Bluetooth transmission / reception device). The MIDI engine 22 outputs the music control data 23 to the MIDI synthesizer 24 as a MIDI data stream in real time (during reproduction). The music control data 23 may include contents obtained from the MIDI file transferred to the MIDI engine 22 by the processor 12, or may include additional messages introduced by the MIDI engine 22 under the control of the processor 12. Alternative messages may also be included.

  The MIDI synthesizer 24 receives MIDI messages one at a time and responds to these messages by playing sounds through the speaker 26. These MIDI messages are received and processed in real time by a MIDI synthesizer.

  As shown in FIG. 1, the portable electronic device 1 may form a temporary network 4 comprising one or more other portable devices 2, 3. The network is preferably a local network so that the user of the device is 'face-to-face', but is not necessarily limited thereto.

  The network 4 enables device cooperation and execution as an ensemble. When a device operates as an ensemble, the music output of the ensemble is a combination of the music output of the individual devices. This music output can be changed in both directions by the user of the apparatus. This allows for two-way improved music composition by multiple users that are preferably face-to-face.

  Individual users may have exclusive responsibility for one or more aspects of music output. That is, only individual users can control the mode of music output.

  For example, add sound effects (echo, reverberation, etc.), change instruments, perform 'live' via background tracks, remove instrumentation such as drum beats, beats, melodies By removing the bus line, the predetermined music track can be changed in real time. The predetermined music track can be output as a MIDI file to the MIDI engine 22 by the processor 12. The user creates a sound effect by controlling the MIDI engine via the UI 11 and the processor 12, and adds to the MIDI file message input to the synthesizer 24, removes this message, or reads this message. Or adapt.

  As another example, separate predetermined music tracks may be mixed in both directions to be integrated. The processor 12 can provide individual predetermined music tracks to the MIDI engine 22 as MIDI files. The user controls the MIDI engine 22 via the UL 11 and the processor 12 to add to the MIDI file message input to the synthesizer 24, to remove this message, or to adapt this message.

  As another example, some or all of the users can be responsible for playing their instruments in the ensemble. The user then controls his device to change the synthesized sound of the instrument in the ensemble. That is, a musical instrument is played by one's own device. The user of the device uses the device's UI 11 (which may be a keypad or another MIDI controller such as an integrated integrated accelerometer that responds to device movement) in real time to play melody, rhythm, loop, etc. Will be able to demonstrate. The music output of the ensemble is a combination of instrument sounds controlled by the user. The ensemble may produce a pre-recorded accompaniment or accompaniment music. The output of the music ensemble is then a combination of the recorded accompaniment and the instrument controlled by the user.

  The network devices 1, 2, 3 may be mobile phones. Each mobile phone has an accelerometer that detects when the phone is shaken. Networked mobile phones form an instant samba band. The samba track is played as background music on one or individual phones. By transferring the stored MIDI file to the MIDI engine 22, background music can be produced by the processor 12. An individual phone will produce a percussion sound when it is shaken. The accelerometer output is converted to a MIDI message. These additional MIDI messages are added to the input of the synthesizer 24 by the MIDI engine 22, which controls the speaker 22 to generate additional percussion in time with the phone shaking. Percussion sounds controlled by a particular shaking of the phone may be generated only by that phone or may be generated simultaneously by all phones. In the latter case, the additional MIDI message is forwarded via the network 4 to another telephone.

  Bidirectional music control is achieved using a multi-music application.

  One type of music application allows the formation of one or more musical sound effects, such as echoes, reverberations, etc., and changes in real time. A specific musical instrument sound is output by another type of music application (instrument application).

  The music application can be used in an independent 'freestyle' mode, slave 'assist' mode, or master mode. A session is an ensemble of music applications. In any session, there will be only one music master application. A session will include multiple networked devices.

  In the independent mode, the music application is controlled as a conventional instrument. That is, the output will depend solely on the user's ability to use an available user input interface (such as a mobile phone keyboard).

  In the master mode, the music application plays a role of synchronizing the ensemble of the music slave application. The music master application provides music synchronization information to individual music slave applications regardless of whether the application is in the same device or another device.

  The tempo can be determined by this music synchronization information. Once the tempo is defined, the music slave application changes its timing and continuously matches the music master application during the session. This matching can also be performed in a slave application by changing the output of the MIDI engine 22 using the processor 12. The point in time when turning on / off the notes can be easily changed. Therefore, the music output by the slave is determined by the music synchronization information received from the master. A suitable master application is a drum beat application.

  Music synchronization information can also define important musical features and changes to the above features shared by music applications or devices.

  The music synchronization information can include the following: harmony (chord sequence), identifier of music application used in the ensemble, and identifiers of multiple background tracks used in the ensemble (if any identifier exists) ), A substantial amount of musical instrument applications, music keys, time signatures and overall instrument sound or music energy, or whether or not the music application is operated in slave or independent mode Application mode.

  If a background track is used, this background track may be played back as a MIDI file via the music master application. The user of the device will follow a predetermined example.

  In slave mode, the music application is synchronized with the music master application, and on the same device or on separate devices, the ensemble of slave music applications functions as a single system. The slave can synchronize with the music master application using music synchronization information transferred from the music master application to the music slave application via the network (if necessary).

  The music slave application may operate in an assisted performance mode. In the assisted performance mode, the output of the music application is not a reproduction of the music produced by the user. The music output is automatically modified / enhanced to ensure that the music produced complies with a certain subject. This music output is particularly useful for devices equipped with input devices optimized for other purposes, such as PDAs, mobile phone keypads, and the like. Therefore, knowledge and training about music are not required to play music using this system, and the user feels the illusion of playing better than he actually plays. The amount of assistance given can be varied and can be selected by the user and adapted to his ability.

  For example, the user's timing can be modified to maintain the tempo. By quantizing the tempo and changing it to a possible beat position, it is possible to automatically correct the timing for maintaining the tempo. For example, when a user inputs to a drum beat application that does not accurately reflect the correct timing, the timing is adjusted to the closest possible beat position. Timing corrections may be made by using a MIDI engine to change these MIDI messages before outputting them to the synthesizer. Another example is the creation of a correct chord, correction, and completion to preserve the harmony.

  Each device preferably has a copy of the individual music application used in the ensemble, and either device should be able to synthesize sounds created using the music application on another device.

  Furthermore, it is possible to control the music synchronization information using another application. The modifier application itself does not produce any music, but the modifier application is used to control music created by other people.

  In one embodiment, the modifier application is an ensemble music master application. The user can control and change the music synchronization information using the modifier sent to the music slave application.

  In another embodiment, the modifier application remotely controls the music master application to control and change the synchronization information sent by the music master application to the music slave application. The modifier application is in a different device connected to the music master application, and a control signal for controlling the music master application is sent across the network 4 by the modifier application.

  Therefore, the user can participate in the music ensemble by controlling the tempo, composition and chords used by other people without playing any musical instrument.

  Synchronization information is sent to the music slave application on a global scale in any of four different ways.

  The synchronization information includes tempo synchronization that guarantees simultaneous playback of applications. The application needs to function as a master application, and based on this master application, the slave application automatically and continuously matches its timing. The synchronization information also includes structural synchronization information that synchronizes with the application and is otherwise synchronized.

Any one of the following can be used to transmit structural synchronization information:
1) As standard MIDI file (SMF) meta event data 2) Real-time message dedicated to MIDI system 3) MIDI general-purpose controller 4) MIDI unregistered parameters

  Of the above number options, 3) or 4) is preferred.

  The music synchronization information can also be selected by the user of the modifier application from among a plurality of options for selection. The options given for selection may be automatically limited so that the produced music falls within an acceptable range of preferences.

  A modifier application can be used to change the amount of automated assistance received by the user for playing his instrument.

  As a result, the multi-user music system 4 for bidirectionally controlling the action in the music environment (at least the music output of the musical instrument of the first participant, etc.) is provided. This system dynamically changes the music environment with the first device 1 that can be operated by the first user so as to dynamically control the action (music output, etc.) of the instrument of the first participant in the environment. And a second device 2 that can be operated simultaneously by a second user. The first device 1 houses the music slave application, and the second device 2 houses the modifier application. The second device 2 cannot be operated to control the participant's action. The music system 4 may further include a third device 3 that can be operated simultaneously by a third user so as to dynamically control an action in the music environment (such as music output of the musical instrument of the second participant). .

  In addition to or in addition to the above, it is also possible to play a game using the portable device shown in FIGS. The portable electronic device 1 can form a temporary network 4 using one or more other portable devices 2 and 3. The network is preferably a local network so that the user of the device plays “face-to-face”, but is not necessarily limited thereto.

  The network 4 allows the devices to play the game together while collaborating. In this scenario, FIG. 1 will show a multi-user gaming system 4 that interactively controls at least a first participant's action in a gaming environment. In this system, the first device 1 that can be operated by the first user to dynamically control the action of the first participant in the game environment and the second user to change the game environment dynamically at the same time. And a second device 2 which can be operated. This second device is not operable to control the participant's actions.

  By way of example only, the game may be Nokia Snake (Trademark) II. In this game, the snake movement is directed by the first user and the snake moves through the maze while eating food. For example, the second user can control the maze layout and the snake movement speed.

  The system may further comprise a third device that can be operated simultaneously by a third user to dynamically control the actions of the second participant in the game environment. The game play status is displayed on the display of the first, second and third devices.

  Therefore, the first user can participate in the game alone or together with the third user, but the second user changes the game environment. . The second user does not participate in the game, but is still involved in the game.

  This game environment includes a virtual environment, the first participant can move by the action of the first user, and the second participant can be moved by the third user. It is. For example, the gaming environment may include a level layout in the game and / or a simulated weather forecast in the game and / or a level of difficulty of the game and / or a game tempo and / or participant characteristics. . The game environment may also include a transient event initiated by the second user. For example, a second user may create an obstacle in the race track that other players must drive or a bonus food worth the extra points that participants must try to eat first. It can be placed in the II maze.

  While embodiments of the present invention have been described above with reference to various examples, it should be understood that modifications can be made to the examples shown without departing from the scope of the claims of the present invention. Should.

  While efforts have been made in the above specification to draw attention to features of the present invention that are believed to have particular significance, the applicant hereof, regardless of whether or not particular emphasis has been made. It is to be understood that protection is sought for any of the patentable features mentioned above and / or shown in the drawings, or combinations of these features.

1 shows a networked portable electronic device. 1 schematically illustrates a portable electronic device.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 1st portable electronic device 2 2nd portable electronic device 3 3rd portable electronic device 4 Network

Claims (18)

A system for interactive music production,
A first device comprising a first user input for controlling music synchronization information;
A second device comprising:
Input means operable to receive the music synchronization information;
A second user input unit for controlling the content of the first music control data;
A system comprising: a second device including a synthesizer for producing music according to the first music control data and the music synchronization information.   The system of claim 1, wherein the first device produces music and controls the music synchronization information simultaneously.   The system of claim 1, wherein the first device either produces music or controls the music synchronization information, but not both at the same time.   The second device includes output means operable to transfer the first music control data to the first device, and the first device receives the first music control data. 4. The apparatus according to claim 1, further comprising: an input unit operable to perform the operation, and a synthesizer for producing music in accordance with the first music control data and the music synchronization information. The described system.   The system according to claim 1, wherein the first device has output means for sending music synchronization information to the second device.   6. The system according to claim 1, wherein the first device has input means for receiving the music synchronization information.   The input means of the second device is configured to receive second music control data from another device, and the synthesizer of the second device uses the first music control data and the second music control data. 7. The system according to claim 1, wherein the system is configured to produce music according to control data and the music synchronization information. A way to create music in both directions,
Changing the music synchronization information in the first device in response to input from the first user;
Receiving music synchronization information at the second device;
Creating first music control data in a second device in response to an input from a second user;
Producing music in the second device according to the first music control data and the music synchronization information.   9. The method of claim 8, further comprising the step of producing music in the first device according to the first music control data and the music synchronization information. A device that forms part of a system for interactive music production,
An apparatus comprising a user input unit for controlling music synchronization information.   The apparatus of claim 10, wherein the apparatus produces music and controls the music synchronization information simultaneously.   11. The apparatus of claim 10, wherein the apparatus either produces music or controls the music synchronization information, but not both at the same time.   The apparatus further comprises input means operable to receive first music control data, and a synthesizer for producing music according to the first music control data and music synchronization information. An apparatus according to any one of claims 10 to 12.   14. An apparatus according to any of claims 10 to 13, wherein the apparatus comprises output means for sending music synchronization information to another apparatus in the system.   15. The apparatus according to claim 10, further comprising an input unit that receives the music synchronization information. A device that forms part of a system for interactive music production,
Input means operable to receive music synchronization information;
A user input unit for controlling the content of the first music control data;
An apparatus comprising: a synthesizer for producing music according to the first music control data and the music synchronization information.   17. The apparatus of claim 16, further comprising output means operable to transfer the first music control data to another apparatus in the system.   The input means of the device is configured to receive second music control data from another device, and the synthesizer of the device has the first music control data and the second music control data. 18. The apparatus according to claim 16 or 17, wherein the apparatus is configured to produce music according to the music synchronization information.

Images