JP3838152B2 - Performance support system by visualizing performance network in music performance system using communication network and server therefor - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a technology for visualizing a performance network and supporting session performance in a music performance system that performs a music session via a communication network.
[0002]
[Prior art]
When a large number of musical instruments participate in an ensemble through a network, it is difficult to generate a full mesh connection, and a configuration in which the number of connections is limited is necessary. For example, one server is prepared, and all musical instruments generate connections to the server. Each musical instrument transmits performance data to the server, and the server distributes the performance data to all musical instruments. With such an approach, it is difficult to realize a large-scale ensemble due to the heavy load on the center server.
[0003]
In order to solve this problem, the present applicant has proposed a method of connecting all musical instruments by directly connecting musical instruments to each other and repeating them (Japanese Patent Application No. 2002-127463). In this configuration, musical instrument performance data is transmitted via a plurality of musical instruments, so the influence of delay cannot be ignored. However, an orderly music expression can be realized by incorporating the delay as one of the music expressions and aligning the delay with a good time (for example, four bars).
[0004]
In such a system, the connection state of musical instruments in the network has a great influence on music expression. For example, a musical instrument connected via a line with a large delay causes a larger delay than that connected via a high-speed line. This shows that the connection state of musical instruments is an important element in ensembles over a network, just as the arrangement of musical instruments is important in a concert stage.
[0005]
In the existing system, the performance cannot be optimized because there is no means for the performer to grasp the connection state or to change the connection state. For example, if there is a performer who has only a slow line in the center of the network, a large number of performers will pass through the line, and many performances will have a delay effect (called bandwidth disturbance). There is no way to avoid it. In addition, when there are performers of the same musical instrument nearby, performances of the same timbre overlap, so it is preferable that these performers are separated from each other, but there is a disadvantage that there is no means for separating them.
[0006]
[Problems to be solved by the invention]
In view of such circumstances, in the performance system in which a musical instrument is connected by a communication network such as the Internet and the performer enjoys ensemble through the communication network, each performer places his / her instrument in an optimal position. An object is to provide a visual performance support interface for connection.
[0007]
[Means for Solving the Problems]
According to the present invention, a performance network (CN) is formed by a two-way communication network that interconnects a plurality of user terminals (CL) and these user terminals, and performance data is exchanged between the user terminals. In a music performance system using a communication network, a server (SV) that is communicably connected to each user terminal (CL) is provided, and the server (SV) indicates a connection state of the user terminal in the performance network (CN). The information storage means (3S) for holding the graphical information to be represented and the control means (1S) for changing the connection state in response to the notification from the user terminal, each user terminal (CL) is transmitted from the server Based on the schematic information, the display means (7C) for displaying the connection state of the user terminal in the network in a graphic form and the notification are performed. Because of the input means (5C) performance support system according to visualize performance network and a [claim 1] is provided. Note that the parentheses indicate reference symbols or terms in Examples described later, and the same applies to the following.
[0008]
In the performance support system according to the present invention, the schematic information is a matrix that expresses a connection relationship between the user terminals by a network matrix connecting nodes corresponding to the user terminals (CL) by links corresponding to the communication network transmission paths. The display means (7C) is configured to display the connection state of each user terminal in a matrix figure based on the matrix information.
[0009]
Further, according to the present invention, a performance network (CN) is formed by a bidirectional communication network that interconnects a plurality of user terminals (CL) and these user terminals, and performance data is exchanged between the user terminals. Schematic information representing a connection state of user terminals in a performance network (CN), which is a server (SV) that is communicably connected to each user terminal (CL) with respect to a music performance system using a communicative communication network A performance unit comprising information storage means (3S) for holding information, control means (1S) for changing the connection state in response to notification from the user terminal, and communication means (2S) for transmitting graphical information to the user terminal A server for claim support by network visualization is provided.
[0010]
[Effects of the Invention]
In the performance support system according to the present invention, each of the plurality of user terminals (CL) functions as a musical instrument played by the user (performer), and bidirectionally communicates with the plurality of user terminals via a communication network such as the Internet. In a music performance system in which a performance network (CN) is configured to be connected and a ensemble is enjoyed by a plurality of performers, the server (SV) is connected to the user terminals (CL) by the schematic information of the information storage means (3C). The connection state is managed, and the performer of each user terminal (CL) grasps the connection state between the user terminals by the graph display (7C) based on the schematic information, and further the user by the input means (5C) such as a controller. By setting the own terminal position at the desired position, the connection state can be easily set via the control means (1C) of the server (SV). It is possible to further.
[0011]
According to the present invention, the graphic information representing the connection state of the musical instruments of the performers participating in the performance is shared between the performers and visualized as a graph. It can change dynamically and optimize performance. Accordingly, each performer can enjoy a desired ensemble by connecting his musical instrument to an optimal position.
[0012]
In the performance support system of the present invention, a net-like model called a matrix is used as graphical information for visualization, and the player's connection relation is expressed according to this. Then, the performer can freely move on the matrix by input means such as a controller, change the connection relationship with other performers, and change the musical expression more dynamically.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. The following embodiments are merely examples, and various modifications can be made without departing from the spirit of the present invention. In the following description, an instrument played by the performer (and a client terminal including the instrument) and the performer may be simply referred to as “performer”.
[0014]
[System Overview]
FIG. 1 shows a network configuration of an entire music performance system to which a performance support system according to an embodiment of the present invention is applied. This music performance system includes a plurality of client terminals (also simply referred to as “clients” or “user terminals (performer terminals)”) CL corresponding to a performer, and one or more servers SV that manage a connection state between clients. It is possible to communicate bidirectionally between the client CLs and between the client CL and the server SV via a communication network such as the Internet. As shown on the right side of FIG. 1, these clients CL form a client network (performance network) CN, and each client CL is connected to other clients in the network CN to mutually notify performance data. In addition, it is possible to always connect to the server SV and notify each other of information regarding the connection state between clients.
[0015]
Each client CL exchanges performance data based on musical instrument performance with an adjacent client terminal that includes an instrument played by a performer (user) and is directly connected through a transmission path (link) of a communication network. Can also be called "node". Such inter-client connection (connection) is sequentially repeated to link a plurality of clients CL, and a plurality of linked clients CL form one ensemble system called “group” in the client network CN. Composed. In one group, between the clients that are not directly connected, each performance data is relayed via a client located in the middle, and each client CL performs at every other client terminal in the group. Data can be acquired.
[0016]
In other words, in this music performance system, each client interactively exchanges performance data with other clients connected on the communication network (the performance data of a client not directly connected is passed through an intermediate client). In theory, a session (ensemble) can be performed with an infinite number of clients. Therefore, since performance data based on the performance of each client CL is transmitted via a plurality of clients by such bidirectional communication between clients, the influence of the transmission delay cannot be ignored. It is possible to construct an ensemble system that realizes an orderly musical expression by incorporating the performance data string into a musical delimiter time by incorporating it as one of the expressions.
[0017]
The server SV manages the entire state of the music performance system configured as described above, particularly the connection state between clients, and can communicate with each client terminal CL at any time via a communication network such as the Internet. is there. For example, the server SV holds matrix information for visually representing the connection state between clients, and accepts connection, disconnection or termination requests from each client CL to update the matrix information, Can be sent to each client CL. Each client CL can know the connection state between clients in the system by the matrix displayed on the display device according to the matrix information from the server SV.
[0018]
2 and 3 show a block configuration of each client and server according to an embodiment of the present invention. As shown in FIG. 2, each client CL includes a control unit 1C, a network device 2C, a matrix database 3C, a “musical instrument” 4C, a “movement controller” 5C, an acoustic output device 6C, a display device 7C, and the like.
[0019]
The control unit 1C is a control device for the entire client device CL including a central processing unit (CPU) and a storage device (including a read-only memory (ROM), a random access memory (RAM), an external storage device, etc.). The CPU controls each part of the client CL in accordance with various control programs stored in. For example, according to the performance data input / output control program, the network device 2C corresponding to one or more other clients connected to the client CL is activated, and the devices 2C, 4C, 6C related to performance data input / output are activated. Relay performance data sent and received. Further, according to the client connection management control program, connection between clients such as matrix information and connection / disconnection / termination request information between the connection device (2C) with the server SV and the matrix database 3C and the movement controller 5C. Relay information.
[0020]
The network device 2C is a network interface (I / F) device for communicating information with the server SV and other clients via the communication network, and information (matrix information, A connection device for exchanging connection / disconnection or termination request information) with the server SV and a connection device for exchanging performance data with other clients (both connection devices share one connection device). May be.) The matrix database 3C stores the latest matrix information notified from the server SV, and is used to manage the connection state between clients related to the client CL.
[0021]
The “musical instrument” 4C is a performance data input unit for inputting performance data by the performance of the performer (user) of the client CL. When an electronic musical instrument is used, the “musical instrument” 4C indicates a performance operation unit or a performance data generation unit. However, any device that can generate performance data (music data) may be used. The performance data from the “musical instrument” 4C is transmitted to other performers via the connection device (2C) with the client. The “movement controller” 5C is a connection information input unit used to exchange connection information with the server SV. For example, the “movement controller” 5C is used to move the node position of the client CL in the client network CN. Use something similar to a game controller. Note that the input device includes a panel input unit (not shown) for performing other setting operations such as the operating state of each unit in the client CL.
[0022]
The performance output device 6C is a music reproduction unit for reproducing performance data input from the performance input device 4C by the performer of the client CL and performance data input from the adjacent client through the network device 2C. The performances of the performer and other performers can be heard through the keyboard. The display device 7C is a visual information output unit for displaying a matrix on a display (not shown) based on the matrix information, and displaying the operation state of each unit in the client CL.
[0023]
On the other hand, as shown in FIG. 3, the server SV includes a control unit 1S, a network device 2S, a matrix database 3S, and the like. Like the client CL, the control unit 1S includes a CPU and a storage device, and controls each unit of the server SV by the CPU according to a management program stored in the storage device. For example, between the network device 2S and the matrix database 3S, information on connection between clients such as matrix information and connection / disconnection / termination request information is relayed.
[0024]
The network device 2S is a connection device (I / F) for communicating information with each client CL in the client network CN via the communication network, and all the client CLs in the network CN managed by the server Information related to the connection is exchanged with each client CL. This information includes matrix information, connection / disconnection / termination request information, etc. in addition to node information of each client. The matrix database 3S stores the latest matrix information of all clients, and is used to manage the connection state between the clients.
[0025]
[Matrix]
In one embodiment of the present invention, a “matrix” is defined as shown in FIG. 4 so that the connection relationship between clients in the client network CN can be visually grasped. That is, the matrix is composed of “nodes” (◯) indicated by circles and “links” (−) connecting the nodes, the nodes correspond to the clients (players), and the links are connected to the transmission path of the network CN. Correspond. Each performer (client) CL can use the movement controller 5C to select a free position in these nodes and establish a valid connection with a client of another desired node. it can. This position is not related to the actual network position. That is, the performer CL belonging to the client network CN can freely select a position on the matrix regardless of the actual (geographic) location.
[0026]
Connections necessary for actual communication are generated between adjacent nodes on the matrix. For example, in the case of the Internet, connections such as TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) are generated. The nodes and links in the matrix visualize the clients that are valid for actual communication and the connections between them. Matrix information representing the contents of such a matrix is transmitted from the server SV to each client terminal CL and displayed on the display of the display device 7C.
[0027]
FIG. 5 is a display example of ensemble organization on a matrix according to one embodiment of the present invention. In FIG. 5, a node (• etc.) with a predetermined fill is a valid node indicating a client terminal CL (hereinafter simply referred to as “player”) from which performance data is generated based on the performance of the performer. Yes, a bold link is an effective link indicating a connection actually generated.
[0028]
In the example shown in the figure, “Piano”, “Drum”, “Guitar”, “Sax”, “Bass” [the names of these musical instruments (timbres) are the performer ( Client terminal) It is also used to represent CL and its performance data. ] Of five musical instruments (tone colors) are appended, indicating that there are performers who generate performance data of these five musical instruments (tone colors) at each node. In addition, the four effective links that connect the effective nodes to each other have an effective connection (connection) between adjacent performers, and these five performers “Piano” to “Bass” are one ensemble system. It shows that it constitutes.
[0029]
In addition, when displaying a matrix on the display device 7C of each performer CL, a node (effective node) in which the performer actually exists is painted or illustrated with a different color as shown in FIG. Are displayed separately from other nodes (blank display nodes) that do not have a performer. In this case, among the valid nodes, the player CL's own node is highlighted, for example, by blackening or dark coloring, or a notation added to the valid node. When emphasized by frame display or color display, each player CL can clearly grasp his / her node.
[0030]
FIG. 5 also serves as an example of highlight display when the player CL is “Piano”. The matrix figure displayed on the display device 7C of the player “Piano” is as shown in FIG. “Own node is painted black, and the musical instrument (tone color) notation“ Piano ”attached to this node is framed.
[0031]
Hereinafter, the operation relating to the matrix will be described with reference to FIG. In this description, the case where performance data is transmitted with a certain delay and change in volume every time one connection is passed is considered as an example. It is not an essential part of the invention.
[0032]
The performance data of each performer is transmitted to the performers connected by the connection. At this time, it is possible to add changes such as delay, volume, and pitch to the performance data. For example, every time performance data passes through one connection, if the performance data timing is delayed by one measure and the volume is reduced by one unit, the performance data “Drum” is displayed to other performers. 6 is transmitted.
[0033]
[Performance of performers, group formation and merger]
Further, the performer can freely move the place on the matrix. In this case, processing for terminating the existing connection and newly generating a connection based on the new position is performed. For example, when the performer “Piano” is shifted to the connection from the performer “Guitar” to “Sax” in the state of FIGS. 5 and 6, the valid node and the valid link are in the state as shown in FIG. become.
[0034]
That is, in this case, on the matrix, the connection changes as shown in FIG. 7A, and the delay and volume change with the change of the connection. For example, the performance data “Drum” as in the example of FIG. The transmission status is shown in FIG. 7B. Note that the player “Piano” highlights his / her node in the same manner as described above, as shown in FIG. 7A.
[0035]
Next, FIG. 8 shows an example relating to formation and combination of ensemble groups. Here, the outline of the performance support according to the present invention will be summarized with reference to FIG. In the system of the present invention, a performance network CN is formed by a two-way communication network that interconnects a plurality of performers (user terminals) CL, and performance data is exchanged between performers CL that are effectively interconnected. Provides a visual performance support interface for a suitable music performance system. In the server SV that is communicably connected to each performer CL, matrix information (schematic information) representing the connection state of the performer CL that is effective in the performance network CN is created by using nodes and links. It is changed according to the notification from the person CL. In each performer CL, the connection state between performers CL is displayed in a matrix figure based on such matrix information. In the example of FIG. 8, performers “Piano” to “Bass”, “Violin” to “Viola” are displayed. "(A); indicates that the interconnection between" Piano "to" Viola "(B) is effective.
[0036]
Now, according to the example of FIG. 8, the formation and combination of ensemble groups will be further described. In this example, as shown in FIG. 8A, an ensemble composed of performers “Piano”, “Drum”, “Guitar”, “Sax”, “Bass” and a performer “ There are two groups of ensembles that are formed by linking "Violin", "Cello", and "Viola". The performance data of each group is not transmitted to different groups without communication connection.
[0037]
However, when “Violin” moves to the lower left in the connected state of FIG. 8A, two groups can be made into one as shown in FIG. 8A. That is, when the data is combined into one group, the performance data of the two groups are transmitted to each other.
[0038]
Contrary to the above-described example, the movement of the performer causes the separation of the group, and also causes the performer to leave. In any of these cases, the transmission path of the performance data changes, and the music expression changes greatly.
[0039]
[Operation algorithm]
9 to 11 are flowcharts showing processing procedures when starting, moving, and ending a client according to an embodiment of the present invention. Hereinafter, each operation algorithm will be described with reference to each drawing.
[0040]
(1) Processing at client startup (participation)
When a certain client CL is activated and participates in the ensemble, as shown in FIG. 9, the client CL first starts connection with the server SV based on the operation of the movement controller 5C, and requests connection. Is notified to the server SV (A11), and node information such as the name of the performer, tone color, and IP (Internet Protocol) address is notified to the client (A12).
[0041]
On the other hand, the server SV receives a connection request from the client CL (S11), selects the initial position of the client CL on the matrix managed by the server SV, and registers the received node information. (S12). Then, the server SV transmits the matrix information including the initial position of the client CL and the required node information to all the clients in the client network CN (S13), and is adjacent to the client CL on the matrix. A client is selected and a connection request is transmitted to all clients designated to be adjacent (S14).
[0042]
The client CL that has started the connection with the server SV displays the matrix information received from the server SV on the display of the display device 7C (A13), and receives the connection request (A14). Connection (connection) with the selected client is started (A15).
[0043]
All other clients display the matrix information received from the server SV on the display (B11). Of these, when the client selected so as to be adjacent to the client CL that has transmitted the connection request further receives a connection request from the server SV (B12), the client CL that is designated as adjacent to the client CL is in accordance with this connection request. Connection is established and connection is started (B13). (The broken line frame on the right side of FIG. 9 indicates processing of only other clients adjacent to each other, and indicates that other clients not adjacent to each other are not executed.)
[0044]
(2) Processing when moving the client
When the client CL moves on the matrix, as shown in FIG. 10, the client CL changes the movement direction (movement request) of the client position (node) on the matrix by the input operation of the movement controller 5C. In accordance with this, the server SV is notified of the movement request (A21).
[0045]
On the other hand, the server SV receives a movement request from the client CL (S21), and on the matrix, a connection to be disconnected and a connection to be newly generated (new connection) as a result of the client position movement. Are specified, a client related to disconnection and new connection is designated, a disconnection request is transmitted to all corresponding clients (S22), and a connection request is transmitted to all clients related to the new connection (S23). Then, the matrix information stored in the matrix database 3S is updated according to the disconnection of the existing connection or the generation of the new connection, and the updated matrix information is transmitted to all clients (S24).
[0046]
When the client CL that has transmitted the movement request receives a disconnection request and a connection request from the server SV (A22), the client CL disconnects the connection with the specified other client according to the disconnection request (A23), and specifies according to the connection request. The connection with the other client is started (A24), and the matrix information received from the server SV is displayed on the display (A25).
[0047]
On the other hand, of all other clients, all clients related to disconnection receive a disconnection request from the server SV (B21), and disconnect the connection with the specified other client according to the disconnection request (B22). (The broken line frame on the right side of FIG. 10 shows the processing of only the client to be disconnected.) When all the clients related to the new connection receive the connection request from the server SV (B23), they are designated according to the connection request. A new connection is started with another client (B24). (The dashed-dotted line frame indicates processing of only newly connected clients.) Then, all other clients including clients related to disconnection and new connection display the matrix information received from the server SV on the display. (B25)
[0048]
(3) Processing when the client ends (leaves)
When a certain client CL leaves the performance system and ends the role of the valid node in the matrix, as shown in FIG. 11, the client CL sends an end request to the server SV by operating the movement controller 5C. Notify (A31).
[0049]
On the other hand, the server SV receives a termination request from the client CL (S31), checks the connection to be disconnected when the client CL leaves the performance system and disappears from the matrix, and relates to this disconnection. The client is designated, and a disconnection request is transmitted to all corresponding clients (S32). In addition, information (node information and the like) related to the client CL that has notified the disconnection request is deleted from the matrix, an end display is returned to the client CL (S33), and this matrix information is further transferred to all other clients. (S34).
[0050]
Upon receiving the disconnection request from the server SV (A32), the client CL that has notified the termination request disconnects the connection with the client designated for disconnection according to the disconnection request (A33). When the client CL further receives an end display (A34), the client CL disconnects from the server S (A35).
[0051]
Further, of the other clients, all clients related to disconnection receive a disconnection request from the server SV (B31), and disconnect the connection with the specified other client according to the disconnection request (B32). (The broken line on the right side of FIG. 11 shows the processing of only the other clients to be disconnected.) Then, all other clients including the clients related to the disconnection display the matrix information received from the server SV on the display ( B33).
[0052]
In the above-described embodiment, an example in which each node can be connected to a maximum of four other nodes is shown. However, as long as each node can be connected to two or more other nodes, any number of connections is possible. Good. If the number of connectable nodes is different, the display form of the schematic information indicating the connection state is also different. For example, if each node can be connected to a maximum of three other nodes, the display shape is such that each node is located at each intersection where hexagons are arranged in a honeycomb shape. Further, the display form is not limited to a display form in which each node is arranged on Hiraoka, but may be a display form in which the nodes are arranged three-dimensionally.
[0053]
【The invention's effect】
As described above, according to the performance support system of the present invention, when a musical instrument is connected via a communication network such as the Internet and the performer enjoys an ensemble via the network, the connection state of the musical instrument in the network is visually displayed as a graph. Since the diagram information that can be converted into a chart is prepared and shared among the performers, each player can use the graphic figure displayed based on the diagram information to perform the player. Can be visually grasped. In addition, the performer can easily change the connection state by using an input device such as a controller and dynamically change the music expression. Therefore, each player can connect his musical instrument to the optimal position.
[0054]
Further, according to the present invention, a network-like model called a matrix is used for the graphical information for visualization, and the connection relations of the performers are expressed accordingly. It is possible to move freely, change the connection relationship with other performers, and further facilitate the connection to the optimal position.
[Brief description of the drawings]
FIG. 1 is a network configuration diagram of an entire music performance system to which a performance support system according to an embodiment of the present invention is applied.
FIG. 2 is a block diagram of each client according to an embodiment of the present invention.
FIG. 3 is a block diagram of a server according to an embodiment of the present invention.
FIG. 4 is a diagram for explaining a matrix according to an embodiment of the present invention.
FIG. 5 is a diagram showing a display example of ensemble organization on a matrix according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating a transmission state of performance data “Drum” to each player in the example of FIG. 5;
FIG. 7 is a diagram showing a matrix display and performance data “Drum” being transmitted to each player when the player “Piano” is moved on the matrix according to one embodiment of the present invention;
[Fig. 8] Fig. 8 is a diagram for explaining the formation and combination of groups (ensemblers) according to one embodiment of the present invention.
FIG. 9 is a flowchart showing a processing procedure when a client is started according to an embodiment of the present invention.
FIG. 10 is a flowchart showing a processing procedure when a client moves according to one embodiment of the present invention.
FIG. 11 is a flowchart showing a processing procedure when a client leaves (ends) according to an embodiment of the present invention.
[Explanation of symbols]
Client (terminal) [user / performer (terminal)] that becomes a CL valid node,
CN client network (performance network),
"Musical instrument" (performance data input unit) and "movement controller" (connection information input unit) constituting the 4C, 5C input device,
6C, 7C A performance output device (music reproduction unit) and a display device (visual information output unit) constituting the output device.

Claims (3)

In a music performance system using a communication network in which a performance network is formed by a bidirectional communication network that interconnects a plurality of user terminals and these user terminals, and performance data is exchanged between the user terminals.
A server that is communicably connected to each user terminal is provided,
The server includes information storage means for holding schematic information representing the connection state of the user terminal in the performance network, and control means for changing the connection state in response to a notification from the user terminal.
Each user terminal includes display means for displaying a connection state of the user terminal in the network in a graph based on schematic information transmitted from the server, and input means for performing the notification. Performance support system by visualizing the performance network. The schematic information is matrix information that represents a connection relationship between each user terminal by a network matrix connecting nodes corresponding to the user terminals by links corresponding to the communication network transmission path,
The performance support system by visualizing a performance network according to claim 1, wherein the display means displays a connection state of each user terminal in a matrix figure based on the matrix information. A music performance system using a communication network in which a performance network is formed by a two-way communication network that interconnects a plurality of user terminals and these user terminals, and performance data is exchanged between the user terminals. A server that is communicably connected to each user terminal,
Information storage means for holding graphical information representing the connection state of the user terminal in the performance network;
Control means for changing the connection state in response to a notification from the user terminal;
A server for performance support by visualizing a performance network, comprising: communication means for transmitting graphical information to a user terminal.

Images