JP2012018329A - Session terminal and network session system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily hold a session of a musical instrument performance or a singing performance by giving a musical performance in a key suitable to a terminal user.SOLUTION: In a network session system of the present invention, a user terminal A gives a backing musical performance by reproducing backing musical performance information in a prescribed key desired by a user A, and as the user A plays a musical instrument or sings in the prescribed key to the backing musical performance, transmits to a user terminal B the backing musical performance information and user performance information which represents a content of musical instrument performance and singing performance of the user A. The user terminal B transposes the backing musical performance information and the user performance information to any set key desired by the user B and then reproduces them, and as the user B plays a musical instrument or sings while listening to a backing musical performance and a musical instrument performance or singing performance of the user A in this reproduction, transposes user performance information which represents a content of musical instrument performance and singing performance of the user B reversely to the prescribed key and then returns them to the user terminal A. On the user A's side, the performance of the user B is heard as music in the prescribed key desired by the user A, which enables a session without a feeling of strangeness.

Description

  The present invention relates to a network session system for performing a session such as a performance between a plurality of electronic music terminals connected via a communication network such as the Internet.

  2. Description of the Related Art Conventionally, there has been known a technique that enables musical sessions such as musical instrument performance (ensemble) and duet chorus via a communication network such as the Internet. For example, in the ensemble system of Patent Document 1, performance information representing a performance operation at the terminal (A) controls the delay time between the input timing and the processing timing, thereby enabling ensemble over a communication network. ing. In general, musical performance sessions, karaoke duets, and the like perform or sing a certain piece of music in a common key.

Japanese Patent No. 4259329

  However, when performing a duet in a musical session over a communication network, the sung range (key) is often different for each user, and it may not be possible to sing in the same key. In some cases, depending on the skill of the user, the musical instrument can be played only in a predetermined key (C major or A minor), and it is necessary for a high-skilled user to play the musical instrument in a key that matches the low-skilled user. was there.

  In view of such circumstances, the present invention is a network session in which a musical performance or singing session can be easily performed by performing in a tone suitable for a terminal user when performing a musical session via a communication network. The purpose is to provide a system.

[Summary of solution]
Briefly speaking, in the network session system according to the present invention, as shown in FIG. 1, information on back performance and user performance (instrument performance or singing) performed in a predetermined tone on the user terminal A side is transmitted to another user terminal B. When the user of the user terminal B performs the performance while listening to the back performance or the performance of the user A with the set key, the performance information is predetermined. Is transposed back to the user terminal A side. That is, the user terminal A plays back performance by playing back performance information in a predetermined key desired by the user A, and when the user A performs a musical instrument performance or singing in a predetermined key according to the back performance, User performance information representing the content of the musical instrument performance or singing of A is transmitted to the user terminal B. The user terminal B reproduces the back performance information and the user performance information after transposing to any setting tone desired by the user B, and the user B listens to the back performance by the reproduction and the user A's musical instrument performance or singing. On the other hand, when the musical instrument performance or singing is performed, the user performance information representing the contents of the user B's musical instrument performance or singing is reversely transposed in a predetermined tone and then returned to the user terminal A. On the user terminal A side, the performance of the user B can be heard as music of a predetermined tone desired by the user A, so that the user A can continue the performance in a predetermined tone and a session such as an ensemble or a duet without any sense of incongruity becomes possible.

[Configuration of Solution]
Accordingly, the configuration of a specific solution of the present invention will be described. According to the characteristics of the terminal configuration, the network session system together with the other session terminals (TMa) that are communicably connected via the communication network (CN). Terminal (TMb,..., TMd), which is configured to communicate with the other session terminal (TMa) and received from the other session terminal (TMa) through the communication means (10). Transposing means (B1sn, B1sp; B3sn, B3sp) for transposing other terminal user performance information (Upa) based on the back performance information (Bp) to be played and user performances (13-6, 14-7) of other session terminals Back performance information (Bp) transposed by transposing means (B1sn, B1sp; B3sn, B3sp) and other terminals -Performance playback means (B1 to B3; 9) for playing back the user performance information (Upb) based on the user performance (13-6, 14-7) of the session terminal while playing back the user performance information (Upa) And reverse transposing the own terminal user performance information (Upb) and transmitting the reverse-translated own terminal user performance information (Upb) to the other session terminal (TMa) through the communication means (10). , B3rp), a session terminal (TMb,..., TMd) is provided. Note that the parentheses indicate reference symbols, places, etc. of the embodiment, and the same applies to the following.

  Further, according to the characteristics of the system configuration, a network session system including a first session terminal (TMa) and a second session terminal (TMb,..., TMd) that are communicably connected via a communication network (CN). The first session terminal (TMa) includes first terminal communication means (10) for communicating with the second session terminals (TMb,..., TMd), back performance information (Bp) and the first session terminal. Performance information transmitting means for transmitting the first user performance information (Upa) based on the user performance (13-6, 14-7) to the second session terminals (TMb,..., TMd) through the first terminal communication means (10). (A1, A2a; A4, A5c) and the second user through the first terminal communication means (10) from the second session terminal (TMb, ..., TMd). Performance information receiving means (A5) for receiving performance information (Upb), and first terminal performance reproduction means for reproducing back performance information (Bp), first user performance information (Upa), and second user performance information (Upb) (A3 to A6; 9), and the second session terminal (TMb,..., TMd) includes the second terminal communication means (10) for communicating with the first session terminal (TMa), and the first session. Transposition means (B1sn, B1sp; B3sn, B3sp) for transposing back performance information (Bp) and first user performance information (Upa) received from the terminal (TMa) through the second terminal communication means (10), and transposition means The back performance information (Bp) and the first user performance information (Upa) transposed by (B1sn, B1sp; B3sn, B3sp) are reproduced, and the second session terminal The second terminal performance reproducing means (B1 to B3; 9) for reproducing the second user performance information (Upb) based on the user performance (13-6, 14-7) and the second user performance information (Upb) are reversed. A network session comprising reverse transposition means (B3rn, B3rp) for transposing and transmitting reversely transposed second user performance information (Upb) to the first session terminal (TMa) through the second terminal communication means (10) A system (claim 2) is provided.

  In the network session system according to the present invention, among the plurality of session terminals (TM: TMa, TMb,..., TMd) connected by the communication network (CN) and constituting the system, other session terminals (TMa) called the first session terminal ) Includes back performance information (Bp) created in a predetermined key desired by the user (referred to as a predetermined key). This back performance information (Bp) is used for back performance of the system during a network session. It is used and is reproduced not only by the first session terminal (TMa) but also transmitted to session terminals (TMb,..., TMd) called second session terminals (A1, A2a). In the second session terminal (TMb,..., TMd), an arbitrary key (setting key) desired by the user is set, and the received back performance information (Bp) of the predetermined key is transposed to the setting key. (B1sn, B1sp) is reproduced (B1). Accordingly, the user of the first session terminal (TMa) listens to the back performance sound based on the predetermined back performance information (Bp) (A3), and performs the user performance (13-6, 14-7) such as musical instrument performance or singing. ) Can be performed in a desired predetermined tone (A4 to A6), and the user of the second session terminal (TMb,..., TMd) can perform the back performance based on the back performance information (Bp) transposed to the desired setting tone. While listening to the sound (B1), user performance (13-6, 14-7) such as musical instrument performance or singing can be performed in a desired set tone (B2-B3).

  Also, user performance information (Upa) generated based on the user performance at the first session terminal (TMa) is sent to the second session terminal (TMb,..., TMd) and transposed to the set key (B3sn, B3sp) is reproduced (B3), and the user performance information (Upb) generated based on the user performance at the first session terminal (TMb,..., TMd) is reversely transposed to a predetermined key (B2rn, B2rp), Returned to one session terminal (TMa) and played back (A5 to A6). Therefore, the user of the first session terminal (TMa) can listen to the user performance sound such as musical instrument performance or singing by the user of the second session terminal (TMb,..., TMd) in a desired predetermined tone (A6). The user of the second session terminal (TMb,..., TMd) can listen to the user performance sound such as musical instrument performance or singing by the user of the first session terminal (TMa) in a desired set tone (B3).

  As described above, in the present invention, the back performance information is transmitted from the first session terminal to the second session terminal via the communication network, and the second session terminal performs musical instrument performance or singing (user performance) while listening to the information. In addition, when performing a musical network session by returning the musical instrument performance or singing information (user performance information) in the second session terminal to the first session terminal, the second session terminal receives the received back performance information. Is transposed by the desired shift amount, singing or performing while listening to the back performance after the transposition, and this information is reversely transposed by the desired reverse shift amount and then returned to the first session terminal. Therefore, according to the present invention, when performing a musical session via a communication network, it is possible to perform a performance in a tone suitable for the terminal user and easily perform a session such as playing an instrument or singing.

It is a figure for demonstrating the outline | summary of this invention. 1 shows a configuration of a network session system according to an embodiment of the present invention. It is a figure for demonstrating the communication delay countermeasure of a network session. Various embodiments relating to measures against communication delay in a network session are shown. 6 illustrates an example operation of a network session including transposition according to an embodiment of the present invention. 6 illustrates another exemplary operation of a network session including transposition according to an embodiment of the present invention.

[Overview of system configuration]
FIG. 2 shows a system configuration of a network session system according to an embodiment of the present invention. As shown in FIG. 2 (1), this network session system is composed of a session management server SV and a plurality of session terminals TM (the symbol “TM” representatively represents session terminals): TMa to TMd. The server SV performs connection support between the session terminals TM as members, for example, a connection procedure between the session terminals TM performed prior to the start of the session. After connection is established between the session terminals TM, for example, between the session terminals TMa to TMd, performance information such as MIDI data and audio data is transmitted between the session terminals TMa to TMd without using the session management server SV. Sent and received. Since the operation of the session management server SV is well known, the description of the more detailed operation is omitted here.

  The plurality of session terminals TMa to TMd that are members of the network session are electronic music devices capable of playing musical instruments and / or karaoke, respectively. The session terminals TMa to TMd are further connected to musical instrument performance and / or karaoke. There is a session terminal having a back performance function. Of the session terminals having the back performance function, any one of the session terminals arbitrarily set at the time of the member configuration is called a “first session terminal”, and the back performance function of the first session terminal is used as the network session at that time. It can be used for back performance. In this case, the back performance information of the first session terminal is reproduced by the first session terminal, and is transmitted to and reproduced by the remaining other session terminals called “second session terminals”. In the following description, it is assumed that session terminal TMa behaves as a first session terminal and other session terminals TMb to TMd behave as second session terminals. The individual session terminals TMa to TMd are represented by “session terminals A to D”, respectively.

  FIG. 2 (2) is a block diagram showing a hardware configuration example of a session terminal constituting the network session system. A session terminal TM constituting a network session system according to an embodiment of the present invention is a kind of computer having an electronic music information processing function, and is a personal computer (PC) installed with an electronic musical instrument and a music information processing application. Such an electronic music device can be used. As shown in FIG. 1 (1), the session terminal TM includes a central processing unit (CPU) 1, a random access memory (RAM) 2, a read only memory (ROM) 3, a storage device 4, a setting operation detection circuit 5, A performance operation detection circuit 6, an analog-digital (A / D) conversion circuit 7, a display circuit 8, a sound source / effect circuit 9, a communication interface (I / F) 10 and the like are provided. Are connected to each other.

  The CPU 1 constitutes a data processing unit together with the RAM 2 and the ROM 3, and executes various information processing including network session processing according to a predetermined control program including a network session program. The RAM 2 is used as a storage management area for holding data necessary for these processes in an available manner and a work area for temporarily storing various data, and the ROM 3 is used for executing these processes. Various control programs including network session programs, preset data, and the like are stored in advance.

  The storage device 4 includes a storage medium such as an HD (hard disk), an FD (flexible disk), a CD (compact disk), a DVD (digital multipurpose disk), a flash memory, and the like, and a driving device for the storage device. Programs, MIDI performance data including back performance data, and the like can be stored in an arbitrary storage medium. The storage medium may be detachable or may be built in the session terminal TM.

  The setting operation detection circuit 5 constitutes a setting operation unit (panel operation unit) together with a setting operation unit (panel operation unit) 12 such as a switch and a mouse, and detects an operation of the setting operation unit 12 to correspond to the setting operation. Operation information is introduced into the data processing units (1 to 3), and the data processing unit performs various settings based on the setting operation information. The performance operation detection circuit 6 constitutes a musical instrument performance input unit together with a performance operator 13 such as a keyboard, detects operation of the performance operator 13 by a terminal user, and introduces performance operation information corresponding to the performance operation to the data processing unit. Then, the data processing unit creates event description format in which the content of the musical instrument performance is represented by an event sequence based on the performance operation information, that is, MIDI format performance data (referred to as MIDI performance data) and sends it to the sound source / effect circuit 9. The A / D conversion circuit 7 constitutes a singing input unit together with the microphone 14, converts the singing voice signal input from the microphone 14 by the performance of the singing by the terminal user into a digital signal, and introduces it into the data processing unit for data processing. The section sends audio format performance data (referred to as audio performance sound data) representing the content of the singing voice as a waveform to the sound source / effect circuit 9. The display circuit 8 includes a display 15 such as an LCD for displaying various screens necessary for performance input such as various settings and musical instrument performances and singing, and indicators / lamps (not shown). Control is performed according to commands from the data processing unit, and display assistance is provided regarding settings and performance input.

  The sound source / effect circuit 9 includes a sound source unit that generates audio data from MIDI format data, and an effect unit that performs various audio data processing including a DSP, and functions as a performance reproduction unit together with the sound system 16. For example, the tone generator unit generates audio performance sound data based on MIDI performance data obtained from performance operation information from the performance operation detection circuit 6 or MIDI performance data obtained from the ROM 3, the storage device 4, or the communication I / F 10. The effect unit generates a predetermined effect on the audio performance sound data input from the microphone 14, the audio performance sound data generated by the sound source unit, or the audio performance sound data obtained from the communication I / F 10 or the like. Audio output data is generated by adding or mixing. The sound system 16 includes a D / A converter, an amplifier, a speaker (including headphones), and the like, and generates musical sounds (instrument performance sound and singing sound) based on audio output data from the sound source / effect circuit 9.

  The communication I / F 10 is a music-only wired I / F such as MIDI, a general-purpose short-distance wired I / F such as IEEE 1394, a general-purpose network I / F such as Ethernet (registered trademark), a wireless Lan, Bluetooth (registered trademark), or the like. Includes one or more general-purpose short-range wireless I / F, etc., and exchanges MIDI performance data and audio performance sound data with other session terminals via a communication network CN such as the Internet, or externally from a server computer or the like A control program, data, and the like can be received from the device and stored in the storage device 4.

  When the electronic musical instrument is used as the session terminal TM and karaoke is not performed, the singing input unit 14-7 is not necessary, and when only the karaoke is performed using the PC with the music information processing application as the session terminal TM, the musical instrument performance is performed. The input unit 13-6 is not necessary. The hardware configuration of the session management server SV is substantially the same as that shown in FIG. 2B, but the musical instrument performance input unit 13-6 and the performance playback unit 9-16 are not necessary.

[Network session communication delay countermeasures]
FIG. 3 is a diagram for explaining a communication delay countermeasure applied to the network session system according to one embodiment of the present invention. First, an outline of information exchange in this network session system will be described. In this system, back performance information data reproduced on the session terminal A functioning as the first session terminal is reproduced on the session terminal A (A1 to A1). A3) The other session terminals B to D, which are session members, are transmitted as the back performance information Bp in the MIDI performance data format or converted into the audio performance sound data format. When the back performance information Bp is sent from the session terminal A to each of the session terminals B to D, each of the session terminals B to D reproduces the received back performance information Bp by each terminal user (B1 to D1). Musical instrument performance and / or singing is performed in accordance with the reproduction of the back performance information (audio) Bp (B2 to D2). The user performance data representing the contents of the musical instrument performance and / or singing and generated at each session terminal B to D is MIDI performance data or audio performance sound data in the case of musical instrument performance, and audio performance sound data in the case of singing. And transmitted to the session terminal A as the user performance information Upb to Upd of the session terminals B to D.

  In the session terminal A, the terminal user of the session terminal A performs the musical instrument performance and / or singing in accordance with the back performance reproduced by the own device (A4), and the musical instrument performance and / or the other session terminals B to D. Or mixed with singing (A5 to A6). That is, the performance information Bp, Upa representing the content of the back performance and the musical instrument performance and / or singing by the user is the effect unit in the sound source / effect circuit 9 and the user performance information Upb- received from the other session terminals BD. It is mixed with Upd and played. Thereby, the session music performed by the musical instrument performance and / or singing in each session terminal AD according to the back performance is completed.

  The ensemble data representing the completed session music may be only played back at session terminal A, recorded, or transmitted to another device (not shown) via communication network CN, Examples of transmission include uploading a recorded file to a file sharing server or transmitting it in real time to a live broadcast listener terminal using a network live broadcast application (A6). Furthermore, not only recording and transmitting ensemble data, but also recording or transmitting a moving image together.

  Now, this network session system is in an environment where a certain amount of communication delay exists between the session terminals A to D. Here, the communication delay time Tc between session terminals, as shown in FIG. 3 (1), in addition to the communication delay time Tn on the pure network CN between the transmission side and reception side session terminals TMs and Tmr, Performance data transmission and reception drive unit (audio driver for audio performance sound data, MIDI driver for MIDI performance data) DRs, DRr transmission and reception operation delay times Tds, Tdr at terminals TMs, Tmr It is. In particular, with respect to the transmission operation delay time Tds, the ratio of the operation delay of the transmission buffer BUs in the performance data transmission drive unit DRs receiving the transmission instruction of the transmission side session processing unit SPs is high, and the reception operation delay time Tdr is also a performance. The ratio of the operation delay of the reception buffer BUr in the performance data reception driving unit DRr that performs the operations from the reception of data to the start of reproduction in the reception side session processing unit SPr is high.

In this system, a session terminal A that functions as a first session terminal and prepares back performance data communicates with other session terminals B to D that function as a second session terminal and use back performance data from the session terminal A. Thus, the pure communication delay time Tn between the session terminal A and the other session terminals B to D can be measured. Specifically, the session terminal A transmits a delay measurement packet with a packet transmission time ta1 to each of the session terminals B to D, and each of the session terminals B to D receives the delay measurement packet. A response packet with tk1 and response time tk2 (provided that the subscript “k” is “b” to “d”, indicating that each terminal B to D is involved) is returned to the session terminal A. . Then, based on the time ta2 at which the response packet is received at the session terminal A and the above-described times ta1, tk1, and tk2, the round-trip delay time dk between the session terminals B to D is calculated by the following formula <1>. To do.
dk = (ta2-ta1)-(tk2-tk1) ... <1>

  Here, times ta1 and ta2 are times managed in session terminal A, and times tk1 and tk2 are times managed in session terminals B to D. Therefore, the absolute times at the terminals A to D do not need to be exactly the same. This is because the time difference from packet transmission to packet reception at session terminal A or the time difference from packet reception to packet transmission at session terminals B to D is obtained as is apparent from equation <1>. This is because no consideration is required.

  A value obtained by halving the round-trip delay time dk obtained in this way can be regarded as a one-way pure communication delay time Tn. Strictly speaking, the communication delay time is not necessarily the same between the forward and backward paths of the packet, but since there is no difference in the implementation, in this embodiment, it is a half of the round-trip delay time. As a one-way pure communication delay time Tn.

  Then, the one-way communication delay time Tc is obtained by adding the transmission operation delay time Tds and the reception operation delay time Tdr to the obtained pure communication delay time Tn, and this one-way communication delay time Tc is set as the monitor delay time Tm. To do.

In addition, although the value of communication delay time with each measured session terminal BD may differ, in that case, what is necessary is just to employ | adopt one of the following values suitably.
(1) Average value.
(2) Maximum value.
(3) Although the maximum value is basically adopted, the threshold value Tt is adopted when a predetermined threshold value (delay time in a range where a session can be practically performed) Tt is exceeded.
(4) A different delay time is set for each of the session terminals B to D (specific examples will be described later).

  At the start of the session, as shown in the functional block diagram of FIG. 3 (2), if there is a back performance playback instruction at the session terminal A (A1), the desired back performance data to be played back is read from the storage device 4. The back performance information Bp corresponding to this is transmitted to the session terminals B to D, and the reproduction of the back performance sound in the own device is delayed by the set monitor delay time Tm (A2 to A3). That is, the session terminal A emits the back performance sound based on the reproduction of the back performance data to the performance reproduction unit 9 when the monitor delay time Tm has elapsed (A2) after the back performance reproduction is instructed (A1). Is started (A3).

  The session terminals B to D receive the back performance information Bp corresponding to the back performance data reproduced at the session terminal A with a delay of the communication delay Tc from the reproduction instruction, and reproduce the received back performance information Bp. The back performance sound is emitted (B1 to D1). Therefore, when the user of the session terminals B to D performs the instrument performance or (= and / or) sings while listening to the back performance (B2 to D2), the corresponding user performance sound is emitted ( B3 to D3), user performance information Upb to Upd representing the content of the musical instrument performance or singing is transmitted to the session terminal A.

  Also in the session terminal A, the user of this terminal can perform the musical instrument performance or singing while listening to the back performance sound reproduced by the own device (A4), and the user performance information Upa representing the contents is also stored in the session terminal B. To D. Note that the back performance sound at the terminal A is emitted with a delay of the monitor delay time Tm from the playback instruction of the back performance, and this delay (Tm) causes the session terminal A to function as the second session terminal. This is almost equal to the delay (Tc) in the case of the session terminal, and the effect is not felt for the user of the session terminal A. The user performance sound data (Upa) corresponding to the musical instrument performance or singing content by the user of the terminal A is the effect unit in the sound source / effect circuit 9, and the back performance sound data (Bp) or other session terminal B Are mixed together with user performance sound data corresponding to the user performance information Upb to Upd received from -D (A5), and a session performance sound combining these performance sound data is emitted, recorded or transmitted (A6).

  In this way, in the other session terminals B to D, the sound played or sung by the session terminal A is emitted almost after the monitor delay time Tm, which is delayed from the release of the back performance sound (B1 to D1). (B3 to D3), the users of the session terminals B to D further perform musical instrument performance or singing while listening to this. For the users of the session terminals B to D, the musical instrument performance or singing of the user of the session terminal A is delayed from the back performance by the monitor delay time Tm which is substantially equal to the communication delay time Tc. If it is within a predetermined time (for example, within 30 ms), musical instrument performance or singing is possible without much discomfort. In session terminal A, the user performance sound sung or sung by session terminals B to D is emitted after communication delay time Tc, which is delayed from sound emission of back performance (A3) (A6). If the communication delay Tc is within a predetermined time (for example, within 30 ms), musical instrument performance or singing can be performed without much discomfort, as with the session terminals B to D.

  If the monitor delay is not inserted in the session terminal A, as described in FIG. 1 (2), the sound of the user's own user performance sound by the user of the session terminal A's musical instrument performance or singing, and the session terminal B ~ D user's musical instrument performance or singing other machine user's performance sound is emitted by the round trip of the communication delay Tc compared to the performance of these performance sounds at session terminals B to D almost simultaneously. (For example, about 60 ms), the delay of the other-user user performance sound becomes too large, and the feeling of strangeness is great. In this system, by inserting the monitor delay Tm, the delay of the user's own user performance sound at the session terminal A is halved, and the delay of both user performance sounds at the session terminal A and the session terminals B to D is equalized. (The same delay occurs when each of the session terminals A to D functions as the first session terminal and the second session terminal), so that the uncomfortable feeling can be reduced.

  A specific setting example will be described for the case where different delay times are set for each of the session terminals B to D [in the case of (4) above]. As the monitor delay time Tm in the session terminal A, the maximum value of the measured communication delay time with each session terminal or the above-described predetermined threshold value Tt [see (3) above] is set. In addition, to each session terminal B to D, “monitor delay time Tm−each communication delay time Tc” (however, when a predetermined threshold value Tt is set, when the communication delay time Tc exceeds the threshold value Tt, The pack performance information Bp is transmitted after waiting for the threshold value Tt instead of the communication time Tc.

  For example, when the communication delay times Tc with the session terminals B, C, and D are 20 ms, 40 ms, and 60 ms, respectively, the monitor delay time Tm at the session terminal A is 60 ms (the threshold value Tt = 50 ms). If set, 50 ms) is set, and is transmitted to session terminal B after being delayed by 60-20 = 40 ms (when threshold Tt = 50 ms is set, 50-20 = 30 ms), and to session terminal C, 60−40 = 20 ms (50−40 = 10 ms if threshold Tt = 50 ms is set) and then transmitted, and then to session terminal D, 60−60 = 0 ms (if threshold Tt = 50 ms is set, 50− 50 = 0 ms), that is, transmission without delay. In this way, it is possible to play back performance almost simultaneously at each session terminal.

[Various Embodiments Regarding Countermeasures for Communication Delay]
The network session communication delay countermeasure has been described above, but various embodiments can be adopted. FIG. 4 is a functional block diagram for explaining various embodiments of the network session system according to the present invention. For example, when the back performance is prepared at the session terminal A or the musical instrument performance is performed at each of the session terminals A to D, the back performance data prepared at the own terminal A is converted to the MIDI format as shown in FIG. In this MIDI format, the back performance information Bp is transmitted to the other terminals B to D, or the user performance information Upa of the MIDI format generated by the own terminal A; B to D; Upb to Upd is transmitted to the other terminals B to D; 4 (2), or as shown in FIG. 4 (2), the performance sound data (A2a, A5c; B3c) may be transmitted to the other terminals B to D; A as back performance information Bp or user performance information Upa; Upb to Upd.

  When transmitting in MIDI as shown in FIG. 4 (1), each session terminal on the receiving side converts the performance information Bp; Up received in MIDI into an audio format by the sound source unit (9) (A3a, B1a; A5b, B3b) Sound is emitted (A3, B1; A6, B3). Further, when the monitor terminal Tm is inserted in the session terminal A, if the back performance data is in the MIDI format, it may be delayed as it is as shown in FIG. 4 (1) (A1 → A2) or FIG. As in 2), the back performance sound data converted into the audio format by the sound source unit (9) may be delayed (A2a → A2).

  In addition, when singing at each of the session terminals A to D, headphones are used to emit the back performance sound so that the emitted back performance sound is not collected by the microphone 14. desirable. For example, in session terminal B, if a back performance sound to be emitted is collected by a microphone and transmitted to session terminal A, session terminal A will play back the original back performance sound that has been reproduced. This is because the user performance information Upb from the session terminal B delayed by the communication delay Tc is mixed, which is not preferable.

  Further, the insertion mode of the monitor delay Tm may be selected by the user. For example, the monitor delay time Tm may be automatically set according to the measured communication delay time Tc, or may be set by the user with reference to the measured communication delay time Tc. In the case of automatic setting, half of the round trip delay time (2 × Tc) of the communication path may be set as the monitor delay time Tm as described above, or the one-way delay time may be set as the monitor delay time Tm as it is. May be. In addition, when the communication delay time Tc is longer than the time at which an uncomfortable feeling occurs, the user may be notified through the display 15 or the like that the session is difficult to be established.

  Further, when each of the session terminals A to D is an electronic musical instrument, as shown by broken line frames A5a, B3a; A5d, B3d marked with * in FIGS. From the performance operation 13 (A4, B2) until the user performance sound data generation (A5b, B3c) is started by the sound source unit in the sound source / effect circuit 9, or the user performance sound data generation (A5c, B3b) ) After the user performance sound is emitted (A6, B3), a slight performance delay Tp whose upper limit is about the monitor delay time Tm or the communication delay time Tc may be inserted. In this way, the time difference between the generation of the user performance sound corresponding to the musical instrument performance of the other terminal user received from the other session terminal and the generation of the user performance sound (monitor sound) corresponding to the musical instrument performance of the user terminal user is calculated. Can be shortened. However, since there is a delay from the performance operation to the sound generation, there is a possibility that some users feel uncomfortable, so it is desirable that the value of the performance delay Tp is not increased as described above.

[Transpose at each session terminal]
When performing a user performance at each session terminal of the network session system, a certain user may be able to perform only in a predetermined key such as C major or A minor in musical instrument performance due to performance skills. In addition, in singing, the range of singing is different for each user, and depending on the key, it may not be possible to sing. Therefore, in the network session system according to one embodiment of the present invention, the back performance and the user performance are transposed at each session terminal according to the musical ability and preference of each user regarding the key, and suitable for each user. By playing in the same tone, a session without a sense of incongruity can be easily realized. 5 and 6 show an example of the operation of the network session system including transpose according to one embodiment of the present invention, and "communication delay" in each figure indicates that there is a communication delay similar to that in FIG. . Further, “monitor delay Tm” (A2) and “performance delay Tp” (A5a, B3a; A5d, B3d) indicate that the same communication delay measures as in FIG. 4 can be taken, but are not necessarily required. .

  In this system, each of the session terminals B to D has an arbitrary transpose value (also referred to as a transpose amount) [= note number shift value (also referred to as note number shift amount) or a pitch according to a key desired by the user. Shift value (also referred to as pitch shift amount)] Ts and reverse transpose value (also referred to as reverse transpose amount) [= reverse note number shift value (also referred to as reverse note number shift amount) or reverse pitch shift value (reverse pitch shift amount) It is also possible to set Tr (Tr = −Ts), and the predetermined back performance information Bp and user performance information Upa sent from the session terminal A to the own session terminals B to D are set. The transposition value (amount) Ts can be converted into a key (set key) that has been transposed. For the set-tone user performance information Upb to Upd generated by the terminals B to D (note that the symbols Upc and Upd are not shown and are replaced by the symbol Upb), the set reverse transpose value (quantity) It is possible to reverse-transform the predetermined tone by reverse transposing only Tr. It is preferable to automatically set the reverse transpose value (amount) Tr in conjunction with the setting of the transpose value (amount) Ts.

  That is, in each session terminal B to D, when the predetermined back performance information Bp and user performance information Upa are received from the session terminal A, the received back performance information is transposed (B1sn, B1sp; B3sn, B3sp), and the user Perform a user performance (musical instrument performance and / or singing) while listening to the back performance (B1) or the user performance (B3) of the terminal A in the set tone after the transposition (B2), the user performance of the own terminals BD The user performance information Upb having a set key representing the contents is reversely transposed and reversely converted into a predetermined key (B2r: B2rn, B2rp) and transmitted to the session terminal A. Thereby, in each session terminal BD, although a musical instrument performance or singing is performed in a tone that is easy to perform or sing, in a state where the session terminals A are collected, the tone is in a uniform state.

  As for the transposition mode of the performance information, if the received back performance information Bp or user performance information Upa is in the MIDI format (event description data format), the note number is shifted by the set note number shift value (amount). In the case of (B1sn; B3sn) and audio format (waveform data format), the pitch is shifted by the set pitch shift value (amount) (B1sp; B3sp). Further, when the user performance information Upb generated by each terminal B to D is in the MIDI format, the note number is shifted by the set reverse note number shift value (amount) (B2rn), and in the audio format, it is set. The pitch is shifted by the reverse pitch shift value (amount) (B2rp).

  FIG. 5 shows a functional block diagram in the case where performance information is communicated in the MIDI format as in FIG. 4 (1), and musical instrument performances are performed as user performances at the session terminals A to D. First, when playback of the back performance is instructed at the session terminal A (A1), the MIDI back performance data of a predetermined tone is read from the storage device 4, and the session terminals B to D are stored as the back performance information Bp in the MIDI format. After the monitor delay Tm, the back performance sound is emitted in a predetermined tone through the sound source / effect circuit 9 and the sound system 16 (A2 to A3). When the user of the terminal A performs the musical instrument performance in a predetermined tone while listening to the back performance sound (A4), MIDI user performance information Upa indicating the content of the musical instrument performance is also transmitted to the session terminals B to D. Further, the back performance information Bp and the user performance information Upa are received by the session terminals B to D after the communication delay Tc (A1 → B1sn, A4 → B3sn).

  In each of the session terminals B to D, the received back performance information Bp is shifted in note number and converted into a set tone (B1sn), and then the back performance sound is emitted in the set tone through the sound source / effect circuit 9 (B1a). ~ B1). When the user of the terminal BD performs a musical instrument performance in a setting tone while listening to the back performance sound (B2), the corresponding user performance sound is emitted in the setting tone through the sound source / effect circuit 9 and the sound system 16. (B3a-B3). Also, the MIDI user performance information Upb having a set key representing the content of the musical instrument performance is reversely converted into a predetermined key by reverse note number shift (B2rn), and then transmitted to the session terminal A. After the communication delay Tc, Received by terminal A (B2rn → A5b).

  In the session terminal A, back performance sound data based on the back performance data is generated by the sound source unit in the sound source / effect circuit 9 (A3a), and the user performance sound data and session corresponding to the musical instrument performance by the user of the terminal A User performance sound data based on the user performance information Upb from the terminals B to D is generated (A5b), and these performance sound data are mixed by the effect unit in the sound source / effect circuit 9 (A5), and the mixed session performance is performed. The sound data is in a state of tone, and is emitted through the sound system 16 and recorded (stored) in the storage device 4 or transmitted to the session terminals B to D (A6).

  FIG. 6 shows a functional block diagram in the case where performance information is communicated in audio format as in FIG. 4 (2). As a user performance, a musical instrument performance is performed at session terminal A, and singing is performed at session terminals B to D. . In the session terminal A, when playback of the back performance is instructed (A1), the back tone sound data in the audio format is based on the predetermined MIDI back performance data read from the storage device 4 by the sound source unit (9). Is generated (A2a), and this back performance sound data is transmitted to the session terminals B to D as the back performance information Bp, and after the monitor delay Tm, the back performance sound is transmitted through the effect section (9) and the sound system 16. A tone is emitted (A2 to A3). When the user of the terminal A performs the musical instrument performance in a predetermined tone while listening to the back performance sound (A4), the user performance sound data in the audio format generated by the sound source unit (9) according to the musical instrument performance is (A5c). It is transmitted to the session terminals B to D as user performance information Upa. Also, the back performance information Bp and the user performance information Upa are received by each session terminal B to D after the communication delay Tc (A2a → B1sp, A5c → B3sp).

  In each session terminal B to D, the received back performance information Bp is pitch-shifted and converted into a set tone (B1sp), and then the back performance sound is emitted in the set tone through the effect unit (9) and the sound system 16. (B1). When the user of the terminal B to D sings in the set tone while listening to the back performance sound (B2), the corresponding user performance sound is emitted in the set tone through the effect unit (9) and the sound system 16 ( B3d-B3). In addition, the audio user performance information Upb having a set key representing the content (waveform) of the singing is reversely converted into a predetermined key by reverse pitch shift (B2rp), and then transmitted to the session terminal A. After the communication delay Tc, Received by the terminal A (B2rp → A5).

  In the session terminal A, by the mixing function of the effect section (9), the back performance data (A2a), the user performance sound data (A5c) corresponding to the musical instrument performance by the user of the terminal A, and the user performance information from the session terminals B to D The user performance sound data based on Upb is mixed (A5), and the mixed session performance sound data is in a state of adjustment, and is emitted through the sound system 16 and recorded (stored) in the storage device 4. Or, it is transmitted to the session terminals B to D (A6).

  In FIGS. 5 and 6, the user performance mode of the session terminal A and the session terminals B to D and the performance information communication mode between the terminal A and the session terminals B to D are “musical instrument performance + musical instrument performance, bidirectional MIDI communication”. (Fig. 5) and "Musical instrument performance + singing, two-way audio communication" (Fig. 6) have been described, but "Singing + singing, two-way audio communication", "Musical instrument performance + instrument performance, two-way audio communication" "Musical instrument performance + singing, musical instrument performance is MIDI, singing is communicated by audio", and so on.

  Regarding the transposition, the session terminal A may be capable of transposing. In that case, the session terminals B to D are notified that the session terminal A has transposed, and the session terminals B to D set their own transpose values in consideration of the transpose value of the terminal A. . Alternatively, the transpose values of all terminals may be designated by the session terminal A and sent to each terminal in a transposed state at the time of sending.

  Although the user performance information Upb in each session terminal B-D has been shown as an example transmitted only to the session terminal A, the user performance information may be transmitted and received between the session terminals B-D. If it does in this way, also in each session terminal BD, the sound which mixed the user performance information (Upa and Upb) in all the session terminals can be heard.

  When a predetermined processing time is required for transposition and reverse transposition, it is preferable to add the time required for transposition and reverse transposition to the communication delay time Tn.

TM: TMa to TMd; TMs, TMr session terminal,
Bp, Up (Upa to Upd) Back performance information and user performance information,
B1sn, B3sn; B1sp, B3sp Note number and pitch shift part (transposition part),
B3rn, B3rp Reverse note number shift unit and reverse pitch shift unit (reverse transposition unit).

Claims (2)

A session terminal that constitutes a network session system together with another session terminal communicably connected via a communication network,
Communication means for communicating with other session terminals;
Transposition means for transposing back performance information received from other session terminals through communication means and other terminal user performance information based on user performance of other session terminals;
A performance reproducing means for reproducing the back performance information and other terminal user performance information transposed by the transposition means, and reproducing the own terminal user performance information based on the user performance of the session terminal,
A session terminal comprising reverse transposition of reverse transposition of own terminal user performance information and transmitting the reversely transposed own terminal user performance information to another session terminal through communication means. A network session system composed of a first session terminal and a second session terminal that are communicably connected via a communication network,
The first session terminal
First terminal communication means for communicating with the second session terminal;
Performance information transmitting means for transmitting the first user performance information based on the back performance information and the user performance of the first session terminal to the second session terminal through the first terminal communication means;
Performance information receiving means for receiving second user performance information from the second session terminal through the first terminal communication means;
First performance playing means for playing back performance information, first user performance information, and second user performance information;
The second session terminal
A second terminal communication means for communicating with the first session terminal;
Transposition means for transposing back performance information and first user performance information received from the first session terminal through the second terminal communication means;
Second terminal performance reproduction means for reproducing the back performance information and the first user performance information transposed by the transposition means, and reproducing the second user performance information based on the user performance of the second session terminal;
A network session system comprising reverse transposition means for transposing the second user performance information and transmitting the reversely transposed second user performance information to the first session terminal through the second terminal communication means.

Images