JP2000235388A - Method and device for visualizing music, and recording medium where music visualizing program is recorded - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate music understanding in appreciation and education of music by expressing music by visualization. SOLUTION: This music visualizing device is provided with an input means 100 for obtaining music information, a separating means 110 for separating the input information into structure components of music, conversion means 120a-120n for executing processing in parallel to convert sound features into color features by referring to respective conversion tables 150a-150n to the information on each structure components of the music, a display composing means 130 for composing plural conversion information of the respective music structure components, and a display means 140 for displaying the composed conversion information synchronized with the reproduction of the music. Thus, since it become possible, for example, to visualize music, music education which has required comprehension by reading and listening to plural parts (playing instruments) using a score, is facilitated in reading and understanding the score.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for visualizing music (including singing).

[0002]

2. Description of the Related Art Conventionally, as a method of visualizing music at the same time as playing a sound or music, a waveform is displayed like an oscilloscope, or a sound being played is divided for each frequency like a graphic equalizer. Methods for visualizing the strength are known. These methods show the waveform of the sound of the music being played, the frequency of the sound and the state of the sound, and although it was possible to visualize the characteristics of the physical sound,
He could not say that he was visualizing the musical expression itself.

[0003]

In the prior art,
No consideration was given to the structural visualization of the music. For this reason, music appreciation in music appreciation and music education relies heavily on hearing, and in order to understand the details of musical expression, special training such as color listening training, scorebook reading comprehension, and listening comprehension training was required. Further, as a method of visualizing and expressing music, there is an image video such as a television or a video, but this is not a method of visualizing and expressing a music structure.

An object of the present invention is to facilitate music understanding in music appreciation, music education, and the like by diversifying visualization of the structure of music expression.

[0005]

The present invention utilizes the phenomenon of color hearing, which is one of the synesthesias in psychology. Synesthesia is a phenomenon in which, when one sensation is stimulated among the five senses of a human, the other sensations are also evoked. In the case of color hearing, color sensation is evoked when sound is stimulated by hearing. The present invention uses the phenomenon of color hearing to visually express the structure and composition of music.

More specifically, sound characteristics are converted into expression methods such as color characteristics, display positions, and display area sizes.
Display in synchronization with music presentation. Music information is a MIDI (Musical Instrument) that can identify music composition information for each part (playing instrument) or chord (chord) that composes the music.
Digital Interface) and PCM data (PulseCode)
Modulation) information. From this input information, the information is separated into parts, etc., and the characteristics such as strength, length, short and high of each sound of music are extracted from the separated information,
Using a conversion table, they are converted into features such as light and shade of color and light and dark and expressed. At this time, different conversion results can be obtained by using different conversion tables for each input information. Finally, the composition of the entire music is expressed by synthesizing the conversion results of all the separated information.

[0007] By expressing music with colors, the structure of the music can be visually expressed, and music appreciation and music education can be performed not only from the auditory sense but also from the visual sense, and the music understanding can be simplified. . In addition, music education, which requires reading and listening to a plurality of parts (playing instruments) using a conventional scorebook, also allows music to be visualized, thereby simplifying the task of reading the scorebook.

[0008] Further, by enabling expression for each individual part (playing instrument), not only is it possible to visualize the expression that the musician tried to express in each individual part, but also for all parts or their parts. By integrating and visualizing parts, it is possible to assist the musician in trying to express the music as a whole.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of an example of an apparatus configuration for realizing a music visualization method according to the present invention. In the figure, 100 is an input means for inputting music information, 11
0 is separating means for separating input music information for each music component, 120a to 120n are converting means for extracting each sound feature for each music component and converting it into color features, and 130 is each converting means 120a to 120n. Display synthesizing means for synthesizing the converted information converted in step 1; display means 140 for displaying the synthesized display information;
A conversion table 160 used in the conversion tables 20a to 120n is an expression adjusting unit for setting and changing the conversion rules of the conversion tables 150a to 150n.

FIG. 2 is a processing flowchart for explaining the operation of FIG. Hereinafter, the operation of FIG. 1 will be specifically described with reference to FIG.

[Acquisition of music information] First, the input means 100
Thus, music information is obtained (step 200). The music information described here is a description of music, and music information such as parts (playing instruments) and chords (chords) can be separated into music components, music data, MIDI data, and PCM data obtained by sampling music. And so on.

[Separation by music component] Separation means 110
Separates the music information input from the input unit 100 into music components such as parts (playing instruments) and chords (chords) (step 210). For example, the music represented in FIG. 3 is 310 part 1, 320 part 2, 33
One part 3 (23) and three parts express one music. In the separating means 110, the information is
A part is recognized and separated as separate information from tone color and part (performance musical instrument) designation information included in note data such as IDI and PCM.

FIG. 4 shows an example of the structure of music information. In the data, from the beginning, a part attribute 410 indicating which part the note is information about, a part 411 indicating a tune of the music, a chord information 412, a dynamic level 413 of the sound. , The length 414 of the sound, the pitch 415 of the sound, and the reproduction start time 416.
The separating means 110 reads the part attribute 410 therein, determines the part from this data, and
0, a portion 411 indicating the key of the music, chord information 412 when the part is a chord, strength of the sound 413, length of the sound 41
4. Conversion means 120a-1 for converting the information of the pitch 415 of the sound and the reproduction start time 416 for each part.
Hand over to 20n.

[Extraction of Sound Features and Conversion to Color Features] The conversion means 120a to 120n receive the music information separated by the separation means 110 and perform conversion work in parallel for each part (step 220).

First, the conversion means 120a to 120n perform one by one of the sounds included in the music information acquired for each music component.
Features such as strength, short and long, high and low are extracted. That is,
The converting means 120a to 120n convert the information of the part attribute 410, the portion 411 indicating the key of the music, the chord (chord) information 412, the strength of the sound 413, the length 414 of the sound, and the pitch 415 of the sound, Read from.

Next, the conversion means 120a to 120n convert the conversion tables 150a to 150
With reference to 50n, the feature of the sound is converted into a color and a color expression method. The conversion tables 150a to 150n correspond to the conversion unit 1
The conversion rules (rules for associating features such as strength, length, shortness, high and low of sound with color shading, expression method, expression position, etc.) used by 20a to 120n are stored.

FIG. 5 shows an example of the conversion tables 150a to 150n. The conversion table 500 of FIG. 5 has a hierarchical structure, and includes one sub-table of conversion information about color expression methods and one sub-table of conversion information about expression methods other than a plurality of colors. Each sub-table further stores some sub-tables. Taking FIG. 5 as an example, the color sub-table 510
Are three sub-tables, that is, a sub-table 511 of the entire music piece having conversion information of the tone and color of the music piece such as a major key.
0, a chord subtable 5120 having chord and color conversion information, and a note subtable 5130 having per note note and color conversion information. Further, the note sub-table 5130 includes a plurality of conversion tables 5131, 5132, and 513 that hold the correspondence between each sound and color in the sense of color hearing, such as the rule of correspondence between sound and color.
.. Are stored.

The conversion means 150a to 150n perform conversion work according to the conversion information of the conversion table as shown in FIG. For example, the correspondence rule 1 (5131) between sound and color is as follows.
A conversion rule for converting specific sound pitch information into a “specific color” is stored. Taking the data of FIG. 4 as an example, it is assumed that the conversion unit extracts information on the pitch of the sound (415) and refers to the rule 1 (5131) for correspondence between sound and color. If the conversion rule written therein is (B) of FIG. 6 and the information of the pitch (415) is the information on the sound of “mi”, the display color is determined to be “color 533”. . Similarly, the key information (211) and the chord information (212) of the music can be converted in accordance with the conversion rules in the sub-table 5110 of the entire music and the sub-table 5120 of the chord having the chord and color conversion information.

The sub-table of the expression method other than the color also has a hierarchical structure of the airplane, and stores conversion information from the characteristic of the sound to the expression method of the background or the display time.

As shown in FIG. 5, various conversion rules can be stored in the conversion table 500, and a user can arbitrarily specify a conversion rule to be used for each piece of separated information.
The conversion units 120a to 120n perform conversion work in accordance with a conversion table specified by the user, and create sound visualization expression information by synthesizing the conversion results. Referring again to FIG. 4 as an example, the user sets the part attribute 410 to the conversion table 5312 and the sound length 414 to the conversion table 52.
32, the pitch 415 of the sound is designated as the conversion table 5132 as the conversion information, and the visualization expression information of the sound can be obtained. Further, the conversion rules can be changed or added at any time by the expression adjusting means 160 described later.

[Synthesis of Visualized Expression for Each Music Component] The display synthesizing means 130 collects the results of the conversion work performed for each part (playing instrument) by the conversion means 120a to 120n, and displays one display. Combined as data and displayed
(Step 230). FIG. 7 shows an example of composition of visualization for each part. Here, the visualization expression 710 of part 1 is the expression information of part 1 (310) in FIG. In the same machine, the visualization expression 720 of part 2 is an example of the expression information of part 2 (320) in FIG. 3 and the visualization expression of part 3 (730) is an example of the expression information of part (performance instrument) 3 (330) in FIG. . The display combining unit 130 combines these three visualized expressions and creates display information as one overall image 740.

The expression synthesizing means 130 reads the reproduction time from the input information, and synthesizes the expression information and manages the reproduction time. In the example of FIG. 4, by synthesizing the expression information by reading and managing the reproduction start time 416, the display information is passed to the display unit 140.

[Display] The display means 140 receives the display information from the display synthesizing means 130 and displays it in synchronization with the reproduction of the music (step 240). The display means referred to here is a 2D shown on a screen such as a display of a personal computer.
-It can be expressed by colors such as an 8D screen and a lighting device.

The expression adjusting means 160 is used to adjust the visualized expression of the music displayed by the above-described conversion operation, and to assist the expression closer to the emotion intended by the musician. That is, using the expression adjusting means 160, the user rewrites or newly creates the conversion tables 150a to 150n referred to by the converting means 120a to 120n, and finely adjusts the visualization.

Although the embodiment of the present invention has been described above, the apparatus configuration shown in FIG. 1 is actually constructed using a so-called computer. Also, the procedure and algorithm of the processing flow as shown in FIG.
It is described in a language executable by a computer and is called a music visualization program, and can be provided by being recorded on a computer-readable recording medium, for example, a floppy disk or a compact disk (CD-ROM).

[0026]

As described above, according to the present invention,
Music appreciation and music education are possible not only from the auditory sense but also from the visual sense, and music understanding is simplified. Also, music education, which required reading and listening to multiple parts (playing instruments) using a scorebook, can now be used to visualize music.
Reading the scorebook becomes easier. In addition, by enabling expression for each part, not only is it possible to visualize the expression that the musician tried to express in each part, but also to express all parts or a part of them in an integrated manner This will assist the musician in trying to express the whole piece of music.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an apparatus configuration according to an embodiment of the present invention.

FIG. 2 is a processing flowchart for explaining the operation of FIG. 1;

FIG. 3 is an example of music to be converted.

FIG. 4 is an example of a data structure of music information.

FIG. 5 is a configuration example of a conversion table.

FIG. 6 is an example of a conversion rule stored in a conversion table.

FIG. 7 is an example of composition of visualization for each part.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 input means 110 separation means 120 a to 12 n conversion means 130 display synthesis means 140 display means 150 a to 150 n conversion table 160 expression adjustment means

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Yasuhito Hayashi 3-19-2 Nishishinjuku, Shinjuku-ku, Tokyo Inside Japan Telegraph and Telephone Corporation (72) Inventor Hiroshi Otani 3-192-1, Nishishinjuku, Shinjuku-ku, Tokyo No. Nippon Telegraph and Telephone Corporation F-term (reference) 5D082 AA03 AA04 AA05 AA14 AA15 AA24 AA27

Claims (5)

[Claims] 1. A method for visualizing and expressing the structure of music, wherein music information is input, the input information is separated for each component of the music, and a sound feature is color-coded for each of the separated components. A music visualization method characterized by converting the converted information into different features, synthesizing each converted information, and displaying the converted information in synchronization with the reproduction of the music. 2. An apparatus for visualizing and expressing the structure of music, comprising: input means for acquiring music information; separating means for separating information for each music component from the input information; A conversion unit that performs a process of converting a sound feature to a color feature in parallel with reference to a conversion table for each piece of information, and a display synthesis unit that synthesizes a plurality of pieces of conversion information for each component of music. And a display means for displaying the synthesized conversion information in synchronization with the reproduction of the music. 3. The music visualization apparatus according to claim 2, wherein the conversion table has at least a conversion rule between a sound, a song, a chord, a note, and the like and a color and a characteristic in a hierarchical structure. A featured music visualization device. 4. The music visualization device according to claim 2, further comprising an expression adjusting unit for setting and changing a conversion rule of a conversion table. 5. A computer-readable recording medium for recording a music visualization program for visualizing and expressing a structure of music, comprising: a process for inputting music information; A separate separating process, a process of converting the sound characteristics of each separated component into a color feature, a process of synthesizing each conversion information, and a process of displaying the synthesized conversion information And a recording medium recording the following.