JP2008233576A - Method for making musical piece and musical score into video and image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method of converting a musical piece and a musical score available in an auditory creation system (music) into video (motion image) and an image (still image) to be used for a visual creation system (art, design, video media or the like). <P>SOLUTION: Factors which are common to both an auditory creation system and a visual creation system among factors constituting the both systems are used as conversion parameters to be exchangeable between both the systems, and substitution algorithm complying therewith is used to solve the problem. Especially, in claim 1, a musical sound generated by a musical instrument 1 playing a musical piece or an audio device 2 reproducing a musical piece is detected by a sound sensor (microphone) 3 and then converted into a corresponding color plane by an arithmetic processor 5 in which the algorithm of the conversion parameters 4 are programmed. The color plane obtained by the conversion is projected by a visual system output device 6 on a display screen 7. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a method for visualizing or imaging music and musical scores.

  The present invention relates to a method for converting a musical piece and a score in an auditory creation system (music) into a video (moving image) and an image (still image) used in a visual creation system (art, design, video media, etc.). However, inventions that are considered to be related to this include Japanese Patent Laid-Open No. 2000-172252 “Media Conversion Device” and Japanese Patent Laid-Open No. 2006-330921 “Music Device with Image Display”. In addition, one of the examples of the conversion method between the color constituting the hue and the scale constituting the temperament, which is one of the implementation methods of the present invention, was first filed by the inventor in Japan. This is realized by applying a method described in Japanese Patent Application Laid-Open No. 2004-213383, “Method of Replacing Color, Scale, and Emotion Element” (2002.2.14 application). In addition, there is Japanese Patent Application No. 2007-060111 “Performance apparatus and performance method using coloring tools” filed in Japan by the inventor of the present invention in the same invention as the present invention.

The “media conversion device”, which is a conventional technology, is a device for measuring the similarity for a music plagiarism authorization work, and is an embodiment in which a musical score of a song is imaged (still image) It does not convert the music being played or played back immediately (in real time). Similarly, in the conventional “music apparatus with image display”, the image generation means uses the component extracted from the music information as the Fourier transform means, that is, the amplitude of the frequency component of the musical sound constituting the music and It is derived by calculating the phase. That is, music information is converted into image information in accordance with a mathematical (artificial) conversion rule.
JP 2000-172252 A JP 2006-330921 A JP 2004-213383 A Japanese Patent Application No. 2007-060111

  As described above, the “media conversion device” has a problem of immediacy, and does not show a method for immediate (real-time) conversion of music being played or reproduced. Further, in the present invention, there is a problem that the correspondence method (conversion law) between the scale and the color information is not specifically shown. In addition, in the “music apparatus with image display”, the law of conversion of music into images is a mathematical means, which is based on artificial conversion rules, and is applied to the visual and auditory systems respectively. It cannot be said that it takes into account related events (obtained from factor analysis based on color, size, or perceptual system in the living body), or historically inherited attributes or subjects.

  It has been clarified that sensory organs such as vision and auditory sense and their perceptual system were born by the process of evolution of organisms, and it is common between visual system and auditory system. The idea that there is a rational order (system) is valid. And if the function of the rotation axis (transformation parameter) that makes it possible to go back and forth between the visual system and the auditory system is given to the common rational order, elements in different perceptual systems can be changed to different perceptual systems. Will be available in

  The present invention based on the above hypothesis (the invention based on the above hypothesis is Japanese Patent Application No. 2007-060111 “Performance apparatus and performance method using coloring tools” filed in Japan by the present inventor) Analyze the elements that make up the auditory system and visual system, that is, the attributes that make up music, the attributes that make up music, and the attributes that make up art and art, or subjects, and find elements that share common characteristics. Connected, they are positioned as a rational order common to the visual and auditory systems, and are positioned as rotation axes, ie transformation parameters, that allow each other's systems to cross and go in both directions.

  The present invention intends to provide a method that makes it possible to convert a musical piece and a musical score into an image or an image using the conversion parameter expressed above.

  In order to achieve the above-mentioned object, the present invention is a circular ring having circulation transition properties, both of which are related to wave motion, and the frequency of one round of the circular ring is double (1 octave). Since many features such as the tone (specifically color and tone) obtained by the combination of the constituent elements are common to human emotions, the hue in the visual system and the temperament in the auditory system, It was positioned as a rational order system common to both perceptual systems, that is, as a transformation parameter of the main axis that makes it possible to cross both systems.

  Furthermore, the size of the color plane was assigned as an attribute of the visual system corresponding to the length of the sound in the auditory system, and it was positioned as one of the other conversion parameters common to the auditory system and the visual system.

  The conversion parameters (rotation axes) up to the above can be summarized as “color-scale”, “hue-temperament”, and “color surface magnitude—sound length”.

  Further, as a solution to the problem of enabling immediacy with performance or playback, the size of the display screen to be projected or projected is made to correspond to the length of one measure constituting the music. In other words, one measure of a song corresponds to the size of a screen (display screen or projector screen) corresponding to the visual output device, and the musical sounds constituting the song are successively displayed in time series on the screen. It takes a method of projecting or projecting.

  The second claim of the present invention is an integrated use of the conversion parameters expressed above, "color-scale", "hue-temperament", and "size of color surface-length of sound", And each musical sound in a musical score as a picture (still picture).

  Making it possible to convert elements in the auditory system into a visual system not only shows new possibilities in music creation activities, but also the visual creation system (art) that has been divided in the past. It can be expected that it will promote the emergence of a new creative system by fusing each other's domain of music and auditory creative system (music).

  Furthermore, the present invention related to the perceptual system not only shows a new possibility as a tool or means for emotional education, but also effective communication that has not been possible in the past for people with a perceptual system disorder. It can be expected to be a tool or support for its development.

  According to the first aspect of the present invention, each musical tone of a musical piece being played or played is made to correspond to the tone level (tone) of the tone and the length of the tone to the size of the color surface, and the color surface pattern is output to the visual system. A method for instantly projecting from a device onto a display screen is shown.

  The method for rendering each musical tone in a musical piece and a musical score as an image (still image) using the conversion parameters shown above is independently an invention according to the second claim.

  In the present invention, a conversion parameter that enables the auditory system and the visual system to be exchanged, that is, a rational order system common to the auditory system and the visual system is used (specifically, the pitch of the sound is expressed by hue, The conversion method which makes the length of the sound correspond to the size of the color surface), the artificial conversion method in the prior art is not taken. According to the first aspect of the present invention, the color plane pattern made to appear by the above method is composed of music on the display screen whose screen size corresponds to the length of one measure of the music. By continuously projecting or projecting the total number of measures, the music can be converted immediately (in real time).

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram of the main part of the first claim, in which a musical instrument 1 or an audio reproduction device 2, a sound sensor (microphone) 3, an algorithm of conversion parameters 4 are programmed, and an arithmetic processing unit 5 and a visual system. It shows that it is composed of an output device 6 and a display screen 7.

  Musical sounds generated by the musical instrument 1 that plays the music or the audio device 2 that reproduces the music are detected by the sound sensor (microphone) 3, and then a conversion algorithm is programmed with the conversion parameter 4 as the conversion axis. It is converted into a corresponding color plane by a certain processing unit 5. The color plane is projected toward the display screen 7 by the visual system output device 6. At this time, when the musical instrument 1 that plays a musical piece is an electronic musical instrument, or when the audio device 2 to be reproduced is a device that can output sound generated as electronic information, the electronic information of the generated sound is converted into the above-described conversion. It is relayed to a relay with an electrical circuit that conforms to the algorithm and converted directly to the color plane. Since the subject of the present invention is a factor analysis conversion algorithm between the visual system and the auditory system, description of optics (sensor), electronics, and mechanism system is omitted.

  FIG. 2 shows a hue 1 that is a rational order (system) in the visual system and a temperament 2 that is a rational order (system) in the auditory system. Incidentally, one example of a method for converting the color constituting the hue 1 and the scale constituting the temperament 2 is disclosed in Japanese Patent Application Laid-Open No. 2004-213383 “Color, Scale and Emotion Element” previously filed in Japan by the present inventor. This method is realized by applying the method described in “Method of Substituting” (2002.2.14 application). The outline will be described below with reference to FIGS.

  Figure 3 shows that hue is a three-step generation process (red, green, blue), counter-response (cyan, magenta, yellow), and categorization (tri-primary and counter) This indicates that the color system is a cyclically transitioned color system established by a color that fills the space between six colors generated by the response.

  FIG. 4 shows that the temperament is composed of harmonics of the fundamental tone that is played at the same time when the fundamental tone is played, but the generation process is the order of the long triad, counter-response, and other semitones. It shows that it is completed through. FIG. 5 shows the generation order of harmonics constituting the temperament in detail from the viewpoint of frequency.

  FIG. 6 is an example of correspondence between colors and scales according to the method described in “Method of Replacing Colors, Scales, and Emotion Elements”. When converting a color into a scale, according to the process of generating hue and temperament, firstly, the three primary colors of hue are made to correspond to the major triad 4 in C major, and the opposite response 2 in the second stage of hue generation The tone is generated by the counter-response 5 in the second stage of the temperament generation, and further, the categorized three color group in the third stage of the hue generation is the six semitones in the third stage of the temperament generation To correspond to.

  Specifically, red, green, and blue, which are the three primary colors of hue 1, correspond to de, mi, and so, which are the major triads 4 (in C major) of the temperament, and cyan, magenta that are counter-response 2 of hue. , Yellow corresponds to Les, Fah, and La, which are counter-response 5 of temperament, and categorized 3 groups in hue correspond to 6 semitones of temperament.

  In the method described in “Method of Replacing Colors, Scales, and Emotion Elements”, the tone of the music appropriately corresponds to the fundamental tone used and the difference between the major scale and the minor scale.

  For example, illuminate the various colors corresponding to the seven harmonics in C major (“De”, “Le”, “Mi”, “Fa”, “So”, “La”, “Shi”) in hue 1 of FIG. When viewed, “red”, “yellow”, “green”, “blue-green (category color between green and blue when illuminated in categorization 3 of FIG. 6)”, “dark blue (FIG. 6 "Category color between cyan and blue when illuminated in categorization 3"), "purple (category color between blue and magenta in categorization 3 of FIG. 6)", "red ( The categorization color between magenta and red in the case of categorization 3 in FIG. 6). Further, for example, if the fundamental tone is a minor scale in which the fundamental tone is S (= G minor), the colors corresponding to the “seven harmonics” will be “blue (basic color corresponding to the fundamental tone) when viewed in hue 1 of FIG. ”,“ Magenta ”,“ red (category color between magenta and red when illuminated in categorization 3 of FIG. 6) ”,“ orange (red when illuminated in categorization 3 of FIG. 6) “Categorized color between yellow and green” ”,“ Yellow green (categorized color between yellow and green in the case of categorized 3 in FIG. 6) ”,“ green ”,“ cyan ” .

  When carrying out the present invention, the color plane using the color and scale replacement algorithm shown above, that is, the conversion algorithm based on the conversion parameters “color-scale” and “hue-temperament” is used. Next, the size of the color plane must be specified. For this means, it is necessary to use the conversion parameter “length of sound—size of color surface”. The utilization method will be described with reference to FIGS.

  In FIG. 7, the length of each musical sound played or played by an instrument or audio playback device is related to the size of the corresponding color plane. FIG.

  7-1 is four quarter notes in one bar, 7-2 is two half notes in one bar, 7-3 is a whole note, 7-4 is an eighth note in one bar Eight and 7-5 indicate the relative sizes of the color planes corresponding to the case where there are two quarter notes and eight sixteenth notes in one bar. 7-6 to 7-10 indicate that the color plane is not output for rests.

  FIG. 8 shows that the color plane distribution method in the display screen relates to the size of the color plane corresponding to the length of the unit note (this is called “unit area”) and the number (= beat). FIG.

  For example, when the unit note is equivalent to an eighth note, the area of the color plane is 8-1. When the unit note is equivalent to a quarter note, the area of the color plane is 8-2. Also, the unit note is equivalent to a half note. When the area of the color plane is relativized as shown in 8-3 (this is about 141 percent in terms of the ratio corresponding to each unit note, with the eighth note as a reference) If the unit note is a quarter note and one measure is the length of four quarter notes, that is, one quarter measure is to be visualized. The area is 8-4, and the unit note is a quarter note, and one measure is the length of two quarter notes, that is, the area of one measure when attempting to visualize a two-quarter beat song. Is 8-5, the unit note is a half note, and one measure is the length of two half notes. In other words, the area of one measure when trying to visualize a song with a half beat is 8-6, the unit note is an eighth note, and one measure is the length of six eighth notes, that is, The area of one measure when trying to visualize a six-eight beat song is 8-7.

  The display time of the color plane whose size is specified as described above conforms to the length of the corresponding sound. A specific example of the relative display time of each color plane projected or projected on the display screen is shown in FIG. The example shows (9-1) excerpts from the beginning of "Moon Desert" to the ninth measure (9-1) and shows them in time series (9-2).

  10-1 in FIG. 10 is an example of the color plane that appears in the bar (within the display screen) in order that the size of the display screen projected or projected from the visual system output device corresponds to one bar of music. Is a figure showing from the beginning of "Moon Desert" to the ninth bar. The circled numbers on the upper left of each screen indicate the ranking of the bars that make up the song.

  When the above is performed, the position of each color plane displayed on the display screen is defined in the order of the scale (in order of the pitch of the sound). At this time, the correspondence of the color planes of portions where a plurality of musical tones are pronounced at the same time, such as chords, are made to appear overlapping.

  The color plane 10-1 in FIG. 10 is arranged in the order of lower to higher scales from the left to the right of the display screen. 10-2 is an enlarged display of this. 10-3 shows an example in which the scales are arranged in an annular shape. The broken lines in 10-1 and 10-2 represent the center positions of the color planes corresponding to the assigned scales, and the size of the arranged color planes (corresponding to the length of the sound) is centered here. Increase or decrease. If there is an accompaniment part in the music to be visualized according to the present invention, a new display screen is added, or the entire display screen is divided into a melody part (melody part) and an accompaniment part. .

  FIG. 11 shows an example in which each musical scale (temperament) is arranged in an order that circulates around the periphery of two square display screens in which a melody portion and an accompaniment portion are arranged in parallel. In addition, the size (size) of each color surface is in accordance with the definition in FIG. That is, the area corresponding to the length of each note of 32 minutes, 16 minutes, 8 minutes, 4 minutes, and 2 minutes is changed every 2 times (about 141 percent as a ratio).

  The color surface in the left melody portion of 11-1 represents the quarter note of the scale “do”, and the color surface in the right accompaniment portion represents the eighth note of the scale “si”. The color plane in the left melody portion of 11-2 represents the quarter note of the scale “do”, and the color plane in the right accompaniment portion represents the half note of the scale “La #”. The color surface in the left melody portion of 11-3 represents the half note of the musical scale “mi”, and the color surface in the right accompaniment portion represents the thirty-second note of the musical scale “la”. Similarly, the color plane in the left melody portion of 11-4 represents the sixteenth note of the scale “Fa”, and the color plane in the right accompaniment portion represents the quarter note of the scale “So #”.

  FIG. 12 shows an example of a part of a song (from the second bar to the second bar of the fifth bar of the music “TSUNAMI”) that has undergone video conversion according to the scale arrangement defined by FIG. This is an example of a frame taken (extracted) for each time signature.

  The second claim of the present invention is implemented in accordance with the color and scale replacement algorithm and the sound length and color plane replacement algorithm described above. The implementation method will be described with reference to FIGS.

In carrying out the second claim, first, the tone of the song, the length of the song, and the length of one measure are detected from the target song or score, and the corresponding color, the size of the color plane, and the drawing plane are detected. The size will be specified.

FIG. 13 shows that the drawing surface (the region to be drawn) for drawing the color plane is within the size (size) corresponding to one measure specified from the music or the score (this drawing surface is referred to as “ Called the "basic score").

The size of the color plane to be drawn is drawn corresponding to the length of the sound, but the detailed method is performed according to the contents of FIGS. 7 and 8 described in the first embodiment.

The musical sounds constituting the music or the score appear continuously in chronological order, but when they are converted into images (still images) in the second claim, they are described for each measure (for each basic score), After that, a basic score (one bar area) is drawn in a time series sequence. The total drawing area corresponds to the length of the song.

The arrangement position of the color plane drawn in the basic score is not particularly defined even if it is specified in the musical order. At this time, the part where the chord appears is basically based on selecting the root of the chord.

The shape of the basic score is not specified. In addition, when it is determined that the continuation of the basic score is more attractive in view of the entire drawn, it is not necessary to draw the basic scores in chronological order. The arrangement may be changed as appropriate). Further, even if the music to be handled is an excerpt of only one bar portion, that is, even if the total drawing area is one basic score, it is included in the embodiment of the second claim of the present invention.

FIG. 14 is a specific example of the second claim shown above, and the score (14-1) of the song “Ladybug samba” is imaged for each basic score (one bar). The color planes corresponding to the length of each musical sound are distributed to each of them (14-2), and the colors corresponding to the scales are colored with respect to the respective color planes (14-3, 14-4), 14-5. FIG. 3 is a diagram showing an example of designing a tissue box package by appropriately changing the arrangement of basic scores.

It is a block diagram of the principal part of a 1st claim. It is a figure showing hue and the temperament in the auditory system in the visual system. It is a figure showing the production | generation stage of the hue. It is a figure showing the production | generation stage of the temperament. It is a figure of the production | generation order of the temperament seen from the viewpoint of the frequency. It is the figure which showed the response | compatibility with the color which comprises the hue, and the scale which comprises a temperament. It is the figure which showed that the length of the sound was related to the magnitude of the corresponding color surface. It is the figure which showed the distribution method of the color surface in a display screen. It is the figure which showed the display time of each color surface projected on time series. The figure which showed the example in which each color surface projected shows in the display screen sequentially. The figure which showed the example by which each musical scale is arrange | positioned in the order which goes around a display screen. An excerpt of the music that has been converted to video according to FIG. It is the figure which showed the "basic score." It is the figure which showed the Example of the 2nd claim.

Claims (2)

  Each musical tone of a musical piece that is played or played by a musical instrument or audio playback device, the pitch of the sound (tone) is assigned to the hue, the length of the sound is assigned to the size of the color, and the color pattern is represented by one measure of the song. A method for instantly projecting or projecting onto a display screen whose length corresponds to the screen size.   A method for rendering each musical sound in a music piece and a musical score as an image (still image) by associating the pitch (tone) of the sound with a hue and the length of the sound with the magnitude of the color plane.