JP2003202862A - Device capable of exchanging information between music and visual sensation and server device communicable with the device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device which enables a plurality of persons to share music and visual sensation and a server device which can communicate with the device. <P>SOLUTION: When a client A specifies a server and requests session member information of the server, the server sends the current session member information to the client A. The client A when receiving the session member information from the server displays the session member information and when a user selects one of unused sound panel colors, inputs a user name, and selects a timbre, a log-in request is sent to the server together with sound panel color information, user name information, and timbre information. The server when receiving the log-in request from the client A registers the client A as a new member of the session and sends updated session member information to a client B. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device capable of mutually converting music information and visual information for visualizing the music information, and a server device capable of communicating with the device.

[0002]

2. Description of the Related Art An apparatus for visualizing and displaying music information and converting the visualized music information into music information for pronunciation is conventionally known.

[0003]

However, the above-mentioned conventional device operates independently and is not one in which a plurality of people can share music and vision.

The present invention has been made in view of this point, and an object of the present invention is to provide a device capable of sharing music and vision with a plurality of people and a server device capable of communicating with this device. .

[0005]

In order to achieve the above object, a device capable of converting information between music and vision according to claim 1 is transmitted from an input device for inputting visual information and an external device. Receiving means for receiving music information, first conversion means for converting the input visual information into music information using a predetermined music-visual conversion algorithm, and the converted music information to an external device. A transmitting means for transmitting, a second converting means for converting the received music information into visual information by using the music-visual conversion algorithm, and a combination of the converted visual information and the input visual information. It is characterized in that it has a display means for displaying the converted music information and a sounding means for producing a sound by combining the converted music information and the input music information.

In order to achieve the above object, a device capable of converting information between music and visual sense according to claim 2 receives visual information transmitted from an input means for inputting visual information and an external device. Receiving means, converting means for converting the input visual information and the received visual information into music information using a predetermined music-visual conversion algorithm, and transmitting the input visual information to an external device Transmitting means for displaying, the display means for displaying the input visual information and the received visual information together, and the combined pronunciation of the input and converted music information and the received and converted music information And a sound producing means for producing.

Preferably, the display means displays the input visual information and the visual information from the external device in different display modes (claim 3).

Further, it is preferable that the sounding means pronounces the music information from the device and the music information from the external device in different music modes (claim 4).

Furthermore, preferably, the music-visual conversion algorithm is a conversion algorithm using a sound plane (claim 5).

Here, the sound plane forms one set of sound panels by arranging, for example, sound panels to which one pitch is assigned so as to correspond to respective constituent sounds forming a predetermined scale. Then, a plurality of units are arranged along the scale axis, the chord axis, and the bass axis with one set of this sound panel as one unit (this situation is the same even if the claims are changed hereinafter).

For example, by selecting a target sound panel from a plurality of sound panels arranged on the formed sound plane, music is visualized on the sound plane. The selection of the sound panel is performed by coloring the sound panel, and the characteristic of the pitch of the pitch assigned to the sound panel is changed according to the characteristic of the color to be colored. , The color itself, the characteristic of the sound is a timbre, and by providing sound panels of different colors, a plurality of parts of music are visualized on the sound plane. Further, the sound panel is selected by coloring the sound panel, and the reproduction order of the pitches of the pitches assigned to the sound panel is specified according to the density of the color to be colored. , Below, even if the claims are changed).

In order to achieve the above object, a server device according to a sixth aspect is a server device capable of communicating with a plurality of the device according to the first aspect, and the server device according to the first aspect. When the music information output from a certain device is received, the music information is transmitted to another device among the plurality of devices according to claim 1.

In order to achieve the above object, a server device according to a seventh aspect is a server device capable of communicating with a plurality of the device according to the second aspect, and the server device according to the second aspect. When the visual information output from a certain device is received, the visual information is transmitted to another device among the plurality of devices according to claim 2.

[0014] Preferably, the connection state of a plurality of the devices communicating with the server device is managed, and when the connection state of a certain device changes, the fact is notified to the other devices (claim). Item 8).

In order to achieve the above object, claims 9 to 12 are provided.
The program according to claim 1 can be realized by the same technical idea as those of claims 1, 2, 6 and 7.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings.

The device capable of converting information between music and vision according to one embodiment of the present invention (hereinafter abbreviated as “information conversion device”) may use one device alone,
It is configured so that it can be used in cooperation with other devices. A specific example of the latter case is a server / client system including a plurality of information conversion devices of this embodiment as clients and at least one server device (server computer).

FIG. 1 is a block diagram showing a schematic configuration of the information conversion apparatus of this embodiment.

As shown in the figure, the information conversion apparatus of this embodiment is mainly equipped with a keyboard 1 for inputting character information.
A mouse 2 which is a pointing device, a key operation detection circuit 3 which detects an operation state of each key of the keyboard 1, a mouse operation detection circuit 4 which detects an operation state of the mouse 2, and a CPU 5 which controls the entire apparatus. And the CPU 5
A ROM 6 for storing a control program executed by the CPU, table data, etc., a RAM 7 for temporarily storing performance data, various input information, calculation results, etc., a timer 8 for measuring an interrupt time in timer interrupt processing and various times,
A display device 9 for displaying various information and the like, for example, including a large liquid crystal display (LCD) or a CRT (Cathode Ray Tube) display and a light emitting diode (LED), and a floppy (registered trademark) disk (FD) as a storage medium. A floppy disk drive (FDD) 10 that drives 20, a hard disk drive (HDD) 11 that drives a hard disk (not shown) that stores various application programs including the control program and various data, and the control program CD-ROM drive (CD-ROMD) 12 that drives compact disk-read only memory (CD-ROM) 21 that stores various application programs and various data, and input MIDI signals from the outside and MID A MIDI interface (I / F) 13 that outputs as an I signal to the outside, and a communication interface (I / F) 1 that transmits / receives data to / from the server computer 102 or another client computer 103 via the communication network 101.
4, a tone generator circuit 15 for converting performance data input via the MIDI I / F 13 or the communication I / F 14 or preset performance data into a tone signal, and various effects on the tone signal from the tone circuit 15. Effect circuit 16 for applying
And converting the tone signal from the effect circuit 16 into sound,
For example, it is composed of a DAC (Digital-to-Analog Converter), a sound system 17 such as an amplifier and a speaker.

The above components 3 to 16 are connected to each other via a bus 18, and a timer 8 is connected to the CPU 5.
Another MIDI device 100 is connected to the MIDI I / F 13, a communication network 101 is connected to the communication I / F 14, an effect circuit 16 is connected to the tone generator circuit 15, and a sound system 17 is connected to the effect circuit 16. ing.

As described above, the hard disk of the HDD 11 can also store the control program executed by the CPU 5, and when the control program is not stored in the ROM 6, the hard disk stores the control program and stores it. ROM by reading to RAM7
The CPU 5 can be caused to perform the same operation as when the control program is stored in 6. By doing so, it is possible to easily add or upgrade the control program.

CD-ROM of the CD-ROM drive 12
The control program and various data read from 21 are
It is stored in the hard disk in the HDD 11. This makes it easy to newly install or upgrade the control program. In addition to the CD-ROM drive 12, a device for utilizing various forms of media such as a magneto-optical disk (MO) device may be provided as an external storage device.

As described above, the communication I / F 14 is connected to the communication network 101 such as a LAN (Local Area Network), the Internet, or a telephone line,
It is connected to the server computer 102 and another client computer 103 via the communication network 101. When the above programs and various parameters are not stored in the hard disk in the HDD 11, the communication I / F 14 is used to download the programs and parameters from the server computer 102. The computer serving as a client (in the present embodiment, the information conversion device) transmits a command requesting the download of programs and parameters to the server computer 102 via the communication I / F 14 and the communication network 101. The server computer 102 receives this command and sends the requested program and parameters to
The download is completed by distributing the program to the computer via the communication network 101 and receiving the program and parameters via the communication I / F 14 and storing them in the hard disk in the HDD 11.

In addition, an interface for directly exchanging data with an external computer or the like may be provided.

As can be seen from FIG. 1, the information conversion apparatus of the present embodiment is composed of a general-purpose computer, but the invention is not limited to this, and it may be composed of a dedicated apparatus composed of minimum components. You may

In the present embodiment, there is only one server computer 102, and the information conversion device of this embodiment and another client computer 103 are connected to this one server computer 102. Not limited to this, there may be a plurality of units. Further, in the present embodiment, the number of other client computers 103 is one, but the number is not limited to this, and there may be a plurality of units.

Further, in the present embodiment, as described above,
The other client computer 103 is configured by a general-purpose computer similarly to the information conversion apparatus of this embodiment, and therefore, the hardware configuration thereof is almost the same as that of the information conversion apparatus described above. Further, since the server computer 102 does not need a specific configuration, that is, it may be configured by a general-purpose computer, it has substantially the same configuration as the information conversion device described above.
Of course, the configuration may be different among the devices. For example, the server computer 102 may omit the components having a secondary function of producing a musical sound, that is, the tone generator circuit 15, the effect circuit 16, and the sound system 17. When the server computer 102 is provided with the HDD 10 and the CD-ROM drive 12, the client computer can share the HDD 10 and the CD-ROM drive 12, and therefore the client computer has the HDD 10 and the CD-ROM drive 12. -The ROM drive 12 may be omitted.

The control processing executed by the information conversion apparatus configured as described above will be described separately for the case where one apparatus is used alone and the case where it is used in cooperation with a plurality of apparatuses.

FIG. 2 is a diagram showing a configuration of a sound plane used by the information conversion apparatus according to the present embodiment to visualize music.

As shown in the figure, the sound plane is formed by arranging hexagonal sound panels without a gap.
An infinite plane is formed by arranging an infinite number of sound panels, and a finite plane is formed by arranging a finite number of sound panels.

One pitch is assigned to each sound panel, and each sound panel is arranged along three kinds of axes, that is, a scale axis, a chord axis and a bass axis.

The scale axis is an axis that runs in the left-right direction in FIG. 2, and the sound panels are arranged along this axis in the order of scale. In the illustrated example, the 7-tone scale is adopted as the scale, but the present invention is not limited to this, and other scales such as 12, 8, 6 or 5 scales may be adopted.

The chord axis is an axis that runs diagonally from the lower right to the upper left (or from the upper left to the lower right) in FIG. 2, and each sound panel is arranged along this axis so as to be a diatonic chord. ing. Specifically, in FIG. 2, the sound panel at the position where the chord axis and the scale axis intersect, that is, the sound panel with the note name "D" is used as the starting point, and three sound panels are selected at the upper left along the chord axis. Then, the chord composed of these three note panels becomes Dm, which is one of the three diatonic chords. Similarly, if the four sound panels are selected at the upper left along the chord axis with the sound panel with the note name "D" as the starting point, the chord formed by these four sound panels becomes Dm7, and the diatonic 4 chord. It is one of the. In other words, if you select three sound panels in the upper left along the chord axis with a certain sound panel as the starting point, the chord formed by these three sound panels becomes one of the diatonic 3 chords, and When four tone panels are selected along the upper left along the chord, the chord formed by these four tone panels becomes one of the diatonic 4 chords.

The bass axis is an axis that runs diagonally from the lower left to the upper right (or from the upper right to the lower left) in FIG. 2, and the sound panels are arranged along this axis so as to form the fifth circle. . That is, the pitch between the sound panels adjacent to each other along the bass axis is 5 degrees downward (or 4 degrees upward).
Degree). Therefore, if you move the specified chord by one note panel along the bass axis, II-V
(Two Five) Chord progression of progression. Thus, by providing the base axis, it is possible to visually grasp the chord progression.

FIG. 3 is a diagram showing an example of a unit forming the sound plane of FIG. As can be seen by looking closely at FIG. 2, the sound plane is formed by a plurality of units with the seven sounds highlighted in FIG. 3 as one unit.

FIG. 4 is a diagram showing an example of a method for forming a sound plane having a scale different from that of FIG. 2, and as shown in FIG. 4, a major assigned to each sound panel constituting one unit. By changing the pitch of the scale to the pitch of the minor scale, it is possible to change from the major scale to the minor scale as the entire sound plane.

Next, an algorithm for arranging sounds on the sound plane will be described.

FIG. 5 is a diagram showing an example in which a timbre is assigned to the sound panel. In the illustrated example, the upper sound panel is
A case where a piano tone is assigned and a bass tone is assigned is shown in the lower tone panel. in this way,
On the same sound plane, although there are a plurality of sound panels having the same shape but different tones, the difference in the timbre between the sound panels is expressed by the background color of the sound panel in this embodiment. There is. However, since the drawing cannot be displayed in color, for convenience, the red background color is represented by the diagonal line from the upper right to the lower left, and the blue background color is represented by the diagonal line from the upper left to the lower right.

FIG. 6 is a diagram showing an example in which a music (song) composed of a plurality of parts (timbres) is displayed on the sound plane of FIG. 2, and a predetermined music is displayed on the sound plane. .

In FIG. 6, since the same tone color is assigned to the sound panels having the same direction of the background diagonal lines as described above, in the illustrated example, two parts (tone colors) of music are displayed. There is. The thickness (density) of the diagonal lines indicates the color depth.

FIG. 7 is a diagram for explaining the relationship between the shade of color and the music element, and the progress of music and the note value are both represented by the shade of color.

In the figure, the upper diagram is a diagram showing the relationship between the shade of color and the progress of music, and the pitches assigned to the sound panels of the same color are changed from dark to light. It shows that the pronunciation is done sequentially.

Further, in the figure, the lower diagram is a diagram showing the relationship between tone of color and tone value. In adjacent sound panels of the same color, the tone value becomes longer as the density change becomes larger. It has become.

When the algorithm of FIG. 7 is applied to the sound plane of FIG. 6, it can be seen that music progresses in the direction of the arrow for each part.

In the present embodiment, the progression of music and the tone value are expressed according to the shade of color, but the present invention is not limited to this, and a hue change using a color band (hue circle) is used. You may do it.

FIG. 8 is a diagram showing an example for expressing a chromatic scale. In the illustrated example, the chromatic scale is expressed by a drum-shaped sound panel (highlighted). As a method of expressing the chromatic scale, in addition to this, a part of the unit of the musical plane is changed to a chromatic scale unit configured by a musical scale panel to which a chromatic scale pitch is assigned, or the chromatic scale unit is It is conceivable to form it on a layer different from the sound plane layer, and to overlay this different layer on the sound plane layer.

In the present embodiment, the pitches assigned to the respective sound panels constituting the sound plane are within one octave. Therefore, the larger the area of the sound plane, the more the same pitch. There will be more sound panels. Therefore, in order to express music on such a sound plane, a rule is required to determine which sound panel is selected from the sound panels having the same pitch.

As the rule, this embodiment adopts a rule of selecting a sound panel closest to the sound panel selected immediately before the sound panel to be selected.
However, if the target sound panel has already been selected, the next closest position is selected. When there are a plurality of sound panels at the same distance, the one on the left side is preferentially selected.

When predetermined music is expressed on the sound plane according to the above rules, it is expressed as shown in FIG.

In this way, music can be expressed on the sound plane, but conversely, new music can be composed by drawing some pictures using each sound panel on the sound plane. it can. In other words, by using the above rules, that is, expressing the part according to the type of color and expressing the progression of the music and the note value by the shade of the color in reverse, it corresponds to that picture. You can reproduce the notes that you want. Then, if the reproduced notes are sequentially converted into performance data such as a MIDI signal and sent to the tone generator circuit 15, the composed music can be played.

Further, the music already expressed on the sound plane can be visually edited.

FIG. 9 is a diagram showing an example of editing to add harmony to music on a sound plane.

In the figure, before editing, the sound panel of the red background color (represented by the diagonal line from the upper right to the lower left) and the blue background color (represented by the diagonal line from the upper left to the lower right) on the sound plane. To the music represented by the sound panel, a sound panel with a yellow background color (represented by a grid mesh) is added to thicken the track along the progression of the music indicated by the arrow. If you edit it, the harmony will be added.

FIG. 10 is a diagram showing an example of editing so that a bass is added to music on a sound plane.

In the figure, before editing, for the music in which only the melody was expressed only on the sound panel of the red background color (represented by the diagonal lines from the upper right to the lower left) on the sound plane, this red If you edit to add a sound panel with a blue background color (represented by the diagonal line from the upper left to the lower right) to support the sound panel of the background color, the bass is added.

FIG. 11 is a diagram showing an example of editing so as to add progress sounds to music on a sound plane.

In the figure, before editing, the melody progress is made for the music in which only the melody was expressed on the sound plane only by the sound panel of the red background color (represented by the diagonal line from the upper right to the lower left). If you edit to add a sound panel with a yellow background color (expressed in a grid pattern) in the middle of, the progress sound will be added.

Although not shown, when a decoration is drawn at a position on the sound plane of the sound panel which is already selected before editing, a decoration sound is added.

FIG. 12 is a diagram showing an example of the chord form expressed on the sound plane of this embodiment. By designating the chord form shown in FIG. 12, chord performance can be performed without using keys. it can.

FIG. 13 is a diagram showing an example when a certain musical score is converted into a picture on a sound plane. By applying the above rules to the notes described in the score, it is possible to easily convert them into pictures on the sound plane.

FIG. 14 is a diagram showing an example in which a photograph of a person is converted into a picture on the sound plane. As shown in FIG. 14, the shades of colors expressed in the picture are directly converted into a picture on the sound plane. Since it is converted, by applying the above rule, it is possible to generate music peculiar to the photograph from the picture after conversion. The conversion target is not limited to a portrait photograph, and may be a landscape photograph or a picture instead of a photograph. In short, any one can be used as long as it can express the shade of color.

The information conversion apparatus of this embodiment can be modified as follows by making a simple change in its control processing. That is, (1) When the rhythm is determined by the template or algorithm and the user specifies the sound panel on the sound plane,
(2) Each time the sound panel is specified, the corresponding sound is output and the mode for recording the position and the sound panel are specified. There is a mode (= practice mode) in which only the sound to be played and the position is not recorded, and either of them is switched (3) The color is displayed each time one sound is played, and the picture is gradually made (4) Display multiple pictures on the sound plane and select one from each picture (for example, randomly select) to make a sound and hit the picture, or display nothing on the sound plane. Instead, listen to the sound and apply the picture that corresponds to the sound, or conversely, apply the music from the picture.The picture is a maze, and if you follow the correct route, you will get the correct music. Use it for quizzes, such as or. (5) When you input characters on the sound plane, it becomes music, and when you input music, it becomes characters. (6) When you change the shade of the picture, the rhythm of the music changes, and vice versa. When you change the rhythm (for example, by replacing the template or changing the algorithm), the shade of color changes and the picture changes.

Next, when the information conversion apparatus of this embodiment is used in cooperation with a plurality of apparatuses, one server computer (hereinafter abbreviated as "server") and two client computers ( Hereinafter, two client computers will be referred to as “client A” and “client B”.
Abbreviated) and a network connecting the computers,
An example will be described in which a session is configured between the two client computers using the sound plane (common sound plane).

FIG. 15 shows the client of this embodiment.
It is a flowchart which shows the procedure of the main routine of the real-time session which each computer which comprises a server system each performs.

In the same figure, since the data exchange between the client A and the server is the same as that between the client B and the server, the processing procedure of the client B is not shown. Then, in the following, only the data exchange between the client A and the server will be described, but this is for convenience of description, and if the client A is read as the client B (conversely, the client B is the client A). The same applies to the client B (or the client A) as it is.

First, the server determines the clients A and B.
Before it stands up, it is supposed to be standing up.

When the client A attempts to start a session, the client A designates a target server by designating an IP address or the like (step S1).
In this way, the target server is designated because there is a high possibility that a plurality of servers are usually present on the communication network 101, and any one of them is specified. . In other words, by using different servers, multiple sessions can be launched and the user of each client can join either session.

Client A requests session member information from the server (step S2). The session member information includes the name of the user participating in the session, the tone panel color selected by this user, and information regarding the tone color selected by this user.

The server receiving the request for session member information transmits the current session member information to the client A (steps S101 → S10).
2).

Upon receiving the current session member information from the server, the client A displays this information on the display device 9 (step S3). Thereby, the user of the client A can grasp the user name, tone panel color, and tone color of the current session member. Then, as shown in FIG. 16, the user of the client A selects any tone panel color from the unused tone panel colors, inputs the user name, and selects a tone (tone is
You may have the same tone among the session members,
(You may choose different tones) (Step S4),
In response to a subsequent login instruction, a login request is sent to the server together with tone panel color information (which is represented by a MIDI channel), user name information and tone color information (which is represented by a program change). (Step S5).

Upon receiving the login request from the client A, the server registers the client A as a new member of the session (steps S103 → S104) and updates the session member information to other clients.
For example, it transmits to the client B (step S10).
5). Accordingly, the client B can know the participation of the new session member.

When another client participates in the session, the client A receives the session member information (steps S6 → S7), and the session member information on the client A is updated.

When the client A performs a performance operation on the sound plane, the information is sent to the server (step S8). Processing of this step S8 (performance operation processing)
Will be described later in detail with reference to FIG.

Upon receiving the performance operation information from the client A, the server sends the performance operation information to the client B.
(Step S106). As a result, the performance operation of the client A is reproduced on the sound plane of the client B. The details of the processing (performance operation processing) in step S106 will be described later with reference to FIG.

In the client A, the session member update process (step S
6 and S7) and the performance operation process (step S8) are repeated, and when a logout instruction is issued, a logout request is output to the server (step S10) and the session is exited.

Upon receiving the logout request from the client A, the server excludes the client A from the session members (steps S107 → S108), and sends the updated session member information, that is, the session member information excluding the client A to the client B. It is transmitted (step S109).

FIG. 17 shows steps S8 and S10 described above.
6 is a flowchart showing a detailed procedure of a performance operation process of No. 6.

When a performance operation is performed on the sound plane in the client A (for example, the mouse 2 is used to indicate a predetermined sound panel among a plurality of sound panels forming the sound plane), the operated sound panel The color assigned to the client A is drawn (step S21 → S22).

Then, the pitch M corresponding to this sound panel is M.
Generate an IDI note event (step S23),
Based on the MIDI note event thus generated, a musical tone is generated with the tone color assigned to the client A (step S24). The MIDI channel is set corresponding to the tone panel color of each client (for example, 1ch for red, 2ch for blue, etc.), and the MIDI channel corresponding to the tone panel color of the client that generated the note event. A note event occurs.

Then, the generated MIDI note event is transmitted to the server (step S25).

When the server receives the MIDI note event, the server receives the MIDI note event.
To (step S121 → S122). As a result, in the client B, the performance operation of the client A is reflected on the sound plane, a musical tone is generated with a corresponding tone color, and a real-time session between the client A and the client B is realized.

When the client A receives the MIDI note event generated by the client B from the server,
A tone panel corresponding to the pitch of this note event is drawn with a color corresponding to the MIDI channel of this note event (steps S26 → S27), and based on this note event, a musical tone is produced with the tone color corresponding to this MIDI channel. Generated (note that the tone panel color (= MIDI channel) and the tone color of the musical tone are associated with each other, M
The tone color is also specified from the IDI channel) (step S
28).

FIG. 18 is a flow chart showing the detailed procedure of the performance operation processing different from the performance operation processing of FIG. In the performance operation process of FIG. 17, a musical tone is generated by designating a sound panel on the sound plane, whereas in the performance operation process, predetermined performance pattern data (= loop data) stored in advance is stored. The difference is that the music is played back automatically and repeatedly, and the drawing is performed on the sound plane. Since this performance operation processing is configured in this way, it is possible to easily realize a performance that is difficult or difficult to perform by manual operation, and the user can concentrate on his or her own manual performance. Both 17 and the performance operation processing of FIG. 18 can be executed in parallel on the main routine of FIG.

In FIG. 18, at least one loop data is stored in each client, and when any one of the loop data is selected by the client A (steps S31 → S32), this loop data itself is stored in the server. It is transmitted (step S33). The loop data itself may be stored in the server and only the data designating the loop data may be transmitted from the client A.

The server transmits the loop data to all the clients based on the received loop data (or the data designating the loop data) (steps S131 → S132). In addition, when each client stores the loop data, data specifying the loop data may be transmitted.

When the client A receives the loop data (or the data designating the loop data) from the server, the loop data is automatically and repeatedly reproduced according to a predetermined tempo (the MIDI note event is read).
(Steps S34 → S35). The MIDI channel of this loop data is the MIDI channel of the color assigned to the original client who specified the loop data. The color corresponding to this MIDI channel is M
A sound panel corresponding to the pitch of the IDI note event is drawn, and a tone color corresponding to the MIDI channel is generated based on the note event. In this way, the loop data instructed by the client A is also reproduced by the client B in the same manner, and the sound panel drawing and the generation of the musical sound are performed. It should be noted that the progress of the music is managed in each client, and it is desirable that the reproduction of the loop data is matched to this music (for example, matched to the bar timing).

If there is a loop data deletion instruction instructed by the client A, this deletion instruction is transmitted to all clients (steps S133 → S).
134).

When the client A receives the instruction to delete the loop data, the corresponding loop data is deleted (the data may be actually deleted, or the reproduction may be ended without changing the data) (step S40 → S4
1).

As described above, when the client A gives an instruction to send desired loop data, the loop data is also reproduced at the client B, and when the deletion instruction is given, the loop data is also deleted at the client B. User can freely control the loop data.

In this embodiment, the server computer and the client computer are separately configured. However, the present invention is not limited to this, and the server computer may also serve as a client computer. That is, a computer may operate as a client computer having a server function.

Further, in the present embodiment, the MIDI note event is transmitted and received between the server computer and the client computer, but information indicating the sound panel position on the sound plane may be transmitted and received. The same sound plane is used by each client computer, and if the origin is also set at a common position, the same number can be assigned to each sound computer on each sound panel. By transmitting and receiving this number instead of the note event, the same panel can be designated among the client computers.

Further, the client computer is identified by the MIDI channel of the MIDI event in the present embodiment, but the client user name may be given at the time of transmitting / receiving the data and identified by this.

Furthermore, the drawn sound panel display may be held for a certain period of time and then suddenly disappeared, or the color may be gradually faded (in this case, the operation is performed). Is displayed with the feeling that the afterimage of remains). For example, as described above, when the time corresponding to the tone value has elapsed, the color is lightened to the depth corresponding to the tone value.

In this embodiment, the sound panel has a hexagonal shape in order to form the sound plane without a gap.
Not limited to this, for example, polygons other than hexagons, circles, etc.
Any shape may be adopted.

A storage medium recording the program code of the software that realizes the functions of the above-described embodiments is supplied to the system or apparatus, and the computer (or the CPU 5 or MPU) of the system or apparatus stores the storage medium in the storage medium. It goes without saying that the object of the present invention can also be achieved by reading and executing the executed program code.

In this case, the program code itself read from the storage medium realizes the novel function of the present invention, and the storage medium storing the program code constitutes the present invention.

As the storage medium for supplying the program code, for example, the floppy disk 20,
Hard disk, optical disk, magneto-optical disk, CD-
The ROM 21, CD-R, magnetic tape, non-volatile memory card, ROM 6 or the like can be used. Further, the program code may be supplied from the server computer 102 via another MIDI device 100 or the communication network 101.

Further, by executing the program code read by the computer, not only the functions of the above-described embodiment are realized, but also the OS and the like running on the computer are executed based on the instruction of the program code. It goes without saying that a case where some or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included.

Further, after the program code read from the storage medium is written in the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, based on the instruction of the program code, The function expansion board or the CPU 5 provided in the function expansion unit performs a part or all of the actual processing,
It goes without saying that the processing includes the case where the functions of the above-described embodiments are realized.

[0100]

As described above, claim 1 or 9
According to the invention described in (1), the visual information input through the input means is converted into music information using a predetermined music-visual conversion algorithm, and the converted music information is transmitted through the transmitting means. Music information transmitted to the external device and received from the external device via the receiving unit is converted into visual information using the music-visual conversion algorithm, and the converted visual information and the input visual information are converted. Is also displayed on the display means, and the converted music information and the input music information are pronounced together, so that it is possible for a plurality of people to share music and vision.

According to the invention described in claim 2 or 10, the visual information input via the input means and the visual information received from the external device via the receiving means are the predetermined music-visual conversion algorithm. Are converted into music information respectively, and the input visual information is transmitted to an external device via a transmission means, and the input visual information and the received visual information are combined on the display means. Is displayed and the input and converted music information and the received and converted music information are pronounced together, so that it is possible for a plurality of people to share music and vision.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of an information conversion device according to an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a sound plane used by the information conversion apparatus of FIG. 1 to visualize music.

FIG. 3 is a diagram showing an example of a unit forming the sound plane of FIG.

FIG. 4 is a diagram showing an example of a method for forming a sound plane having a different scale from that of FIG.

FIG. 5 is a diagram showing an example in which a timbre is assigned to a sound panel.

6 is a diagram showing an example in which music composed of a plurality of parts is displayed on the sound plane of FIG.

FIG. 7 is a diagram for explaining the relationship between color shades and music elements.

FIG. 8 is a diagram showing an example for expressing a chromatic scale.

FIG. 9 is a diagram showing an example of editing to add harmony to music on a sound plane.

FIG. 10 is a diagram showing an example of editing so that a bass is added to music on a sound plane.

FIG. 11 is a diagram showing an example of editing so that progress sounds are added to music on a sound plane.

FIG. 12 is a diagram showing an example of a chord form expressed on the sound plane of FIG.

FIG. 13 is a diagram showing an example in which a musical score is converted into a picture on a sound plane.

FIG. 14 is a diagram showing an example in which a portrait image is converted into a picture on a sound plane.

FIG. 15 is a flowchart showing the procedure of a main routine of a real-time session executed by each computer constituting the client / server system of the present embodiment.

FIG. 16 is a diagram showing an example of a state in which self information is being input based on current session member information displayed on the display device 9.

FIG. 17 is a flowchart showing a detailed procedure of a performance operation process of FIG.

18 is a flowchart showing a detailed procedure of performance operation processing different from the performance operation processing of FIG.

[Explanation of symbols]

1 keyboard 2 mice 5 CPU 6 ROM 7 RAM 9 Display device 14 Communication I / F 15 Sound source device 101 communication network 102 server computer 103 Other client computer

Claims (12)

[Claims] 1. An input unit for inputting visual information, a receiving unit for receiving music information transmitted from an external device, and a predetermined music-visual conversion algorithm for converting the inputted visual information into music information. First converting means for converting, transmitting means for transmitting the converted music information to an external device, and second converting means for converting the received music information into visual information using the music-visual conversion algorithm Means, display means for displaying the converted visual information and the input visual information together, and sounding means for producing a sound by combining the converted music information and the input music information. A device that can convert information between music and sight. 2. Input means for inputting visual information, receiving means for receiving visual information transmitted from an external device, and said input visual information and said received information using a predetermined music-visual conversion algorithm. Conversion means for converting the respective visual information into music information, a transmitting means for transmitting the input visual information to an external device, and a display for displaying the input visual information and the received visual information together. An apparatus capable of converting information between music and visual sense, comprising: means, and a sounding means that pronounces the input and converted music information and the received and converted music information together. 3. The music according to claim 1, wherein the display means displays the input visual information and the visual information from the external device in different display modes. A device that can convert information between sight. 4. The music according to claim 1, wherein the sound producing means produces music information from the device and music information from the external device in different music modes. A device that can convert information between sight. 5. A device capable of converting information between music and vision according to claim 1, wherein the music-visual conversion algorithm is a conversion algorithm using a sound plane. . 6. A server device capable of communicating with a plurality of devices according to claim 1, wherein when receiving music information output from a device among the devices according to claim 1, A server device for transmitting the music information to another device among the plurality of devices according to claim 1. 7. A server device capable of communicating with a plurality of the devices according to claim 2, wherein when the visual information output from a device among the plurality of devices according to claim 2 is received, A server device for transmitting the visual information to another device among the plurality of devices according to claim 2. 8. The connection state of a plurality of the apparatuses communicating with the server apparatus is managed, and when the connection state of a certain apparatus changes, the fact is notified to the other apparatuses. Or the server device according to any one of 7 above. 9. A first conversion procedure for converting visual information input through the input means into music information by using a predetermined music-visual conversion algorithm, and transmitting the converted music information. And a second conversion procedure for converting the music information received from the external device via the receiving means into visual information using the music-visual conversion algorithm, and A display procedure for displaying on the display means the combined visual information and the input visual information together, and a pronunciation procedure for producing the combined musical information of the converted music information and the input music information in a computer A program to run. 10. A conversion procedure for converting visual information input via the input means and visual information received from an external device via the receiving means into music information using a predetermined music-visual conversion algorithm, respectively. The input visual information, a transmission procedure of transmitting to an external device via a transmission unit, a display procedure of displaying the input visual information and the received visual information together on a display unit, A program for causing a computer to execute a pronunciation procedure in which the input and converted music information and the received and converted music information are pronounced together. 11. A program for causing a computer to execute a control method for controlling a server device capable of communicating with a plurality of devices according to claim 1, the control method comprising: A program which, when receiving music information output from a device among the devices described in the above, transmits the music information to other devices among the plurality of the devices according to claim 1. 12. A program for causing a computer to execute a control method for controlling a server device capable of communicating with a plurality of the device according to claim 2, wherein the control method is according to any one of the plurality of claims 2. A program for transmitting visual information to another device among the plurality of devices according to claim 2, when the visual information output from a device among the devices described in the above is received.