KR100322347B1 - Method and apparatus for harmonizing colors by harmonic sound and converting sound into colors mutually - Google Patents

Abstract

The present invention relates to a harmonic color selection method and apparatus using the harmonic method, a tone conversion method and apparatus, and a color tone conversion method and apparatus. In particular, the harmonic color selection method of the present invention selects a scale division ratio and a harmonic method code, and selects a reference color. Select, calculate brightness of the selected reference color, determine an octave corresponding to the calculated brightness, and from the reference color frequency based on the harmonic frequency ratio of the selected harmonic code in response to the scale frequency ratio within the determined octave If the harmonic color frequency is calculated, and the calculated harmonic color frequency is out of the visible frequency band, the harmonic color frequency divided by 2 or twice is selected as the harmonic color frequency, and the calculated harmonic color frequency of the reference color is calculated. Display in color. Therefore, in the present invention, by selecting the harmonic color corresponding to the harmonic frequency of the harmonic method, it is possible to select the harmonious colors very easily during the color design work.

Description

Harmonizing Color Selection Method and Apparatus Using Harmonic Method, Method and Apparatus for Harmonizing Colors and Harmonizing Colors by Harmonizing Sound and Converting Sound Into Color Mutually}

As color television is rapidly popularized, the economic value of color is increasing. In addition, discordant colors due to indiscriminate supply of colors due to the commercialization of colors pose a new color pollution problem in the human living environment.

Therefore, harmonious color selection is very important for web designers, artists, architects, interior decorators, industrial designers, landscape architects, costume designers, stage designers, creators and buyers of various fields.

Color Harmony means that two or more colors are used next to each other to produce a good effect.

Until now, tonalization is based on human visual experience and selects a reference color by referring to a book in a color printing form in which the reference color and the color matched to the reference color are printed in natural colors. Standards include the CIE color system proposed by the International Illumination Organization (CIE) for accurate visible light in displays such as coloring books, Munsell color systems and cathode ray tubes.

However, the conventional color harmonization method has no objectivity as a harmonization method based on human experiences or experiments, and the color accuracy is deteriorated because the harmonic color must be selected based on the human visual sense based on the printed data. In addition, the reference data in the form of a print has a limitation that can not be directly applied to the desktop top publishing system (DTP) using a computer.

And, in tone conversion presented as a method of deriving harmonious colors from harmonically already harmonized music, color and sound differ only in the bands of frequencies applied to the visual and auditory. The same is true of color and negative reversible transformations based on this.

The conventional music display device represented by the graphic equalizer (Graphic Equlizer) that visualizes the change of sound simply displays the magnitude of the sound pressure for each frequency band, so the displayed image is simple and the application range is narrow. have.

U. S. Patent No. 6,046, 724 discloses a method and apparatus for converting sound into light. According to the patent, the sound is controlled through six filter means corresponding to three octaves to eight octaves, respectively, to control the three-color light source of RGB according to the frequency and the sound intensity to display the light corresponding to the sound.

Accordingly, a first object of the present invention is to provide a harmonic color selection method and apparatus using a harmonic method for deriving harmonious colors from harmonically already harmonized music in order to solve the above problems.

It is a second object of the present invention to provide a tone conversion table that enables to see the mutual conversion of sound and color at a glance by making a 10-octave 12 scale and a 10 luminance 12 color correspond to 1: 1.

It is a third object of the present invention to provide a musical instrument capable of recognizing each scale in color when playing music by coloring the scale position using a tone conversion table.

A fourth object of the present invention is to provide an objective and valid formulated frequency conversion criterion using color and sound waves, i.e., frequency characteristics, and to easily convert visible and audible frequencies into audible and visible frequencies based on this criterion. The present invention provides a method and apparatus for converting sound and color tones that enable visualization of colors and visualization of sounds.

It is a fifth object of the present invention to provide a sound source position determining method and apparatus capable of determining the position of a sound source in order to express the sound source position of stereo sound during tone conversion.

In order to achieve the above object, the method of selecting a harmonic color according to the present invention includes selecting a division ratio and a harmonic code, selecting a reference color, calculating brightness of the selected reference color, and calculating the brightness. Determining a corresponding octave, and calculating a harmonic color frequency from the reference color frequency based on the harmonic frequency ratio of the selected harmonic code in response to the scale frequency ratio in the determined octave, wherein the calculated harmonic frequency is a visible frequency. And converting the harmonic color frequency to the visible frequency band when the band is out of band, and displaying the calculated harmonic color frequency as the harmonic color of the reference color.

The scale division rate is an average rate or a pure rate. In the case of the average ratio, the harmonic color can be obtained by the following formula.

-------(One)

Where F _h is the harmonic frequency to be obtained, F _r is the input frequency, k is the sound coefficient, and the octave frequency is equally divided by the equal ratio, and n is the harmonic frequency ratio, where 1≤n≤ (k-1) K and n are natural numbers.

An apparatus for selecting a harmonic color according to the present invention includes selecting means for selecting a scale division ratio, a harmonic code and a reference color, storage means for storing a harmonic frequency ratio table of the harmonic scale based on the harmonic division code according to the division ratio, and the selected reference color. Calculates the brightness of, calculates the octave corresponding to the calculated brightness, and calculates the harmonic color frequency from the reference color frequency by referring to the harmonic frequency ratio of the selected harmonic code in response to the scale frequency ratio within the determined octave And a calculating means for converting the harmonic color frequency to the visible frequency band when the calculated harmonic color frequency is out of the visible frequency band, and display means for displaying the calculated harmonic color frequency as the harmonic color of the reference color. Characterized in that.

The tone conversion table of the present invention is arranged on the first coordinate axis with 12 scales and 12 color systems 1: 1 corresponded at intervals of 12 equal intervals using the scale 'degree' and the color system 'red' as coordinate origins, It is characterized in that the octave and the 10-step brightness is arranged in a 1: 1 correspondence. The tone conversion table constitutes a circular coordinate system or a rectangular coordinate system. The color scale 'red, green, blue' is approximated to the color scale 'red, green, blue'.

The circular coordinate system has a first coordinate axis in a circumferential direction and a second coordinate axis in a circumferential direction. The Cartesian coordinate system may have a saturation proportional to the scale or a luminance directly proportional to the scale.

The musical instrument of the present invention is characterized in that the color scale corresponding to the color of the tone conversion table is colored.

The tone conversion method of the present invention comprises the steps of: Fourier transforming the input sound, sampling at least one specific audible frequency signal of the Fourier transformed signal, and at least one audio frequency signal sampled by the following formula And converting each signal into a signal and displaying a color corresponding to the converted at least one visible frequency.

----------------(2)

(here, Where F is the visible frequency, F _l is the reference visible frequency, the integer part of X is the octave value, B _{F is} the luminance _{of the} color, and x * is the fraction of the decimal point of x to determine the scale value within one octave. Variable, f _i is the sampled input audio frequency, f _l is the reference audio frequency, and C 'is a constant that satisfies 0≤C'≤1.)

The sampling may include inputting the Fourier transformed signal in frame units at predetermined time intervals, obtaining at least one or more peak values for each frame, arranging the obtained peak values in a sound pressure magnitude order, and the sound pressure magnitude. Sampling a predetermined number of peak values in order.

The displaying step may be displayed as an image colored in the converted color having a magnitude proportional to the sound pressure of each peak value at a frequency position of each sampled peak value. In addition, the displaying may be performed by displaying an image colored in the converted color, having an animation movement proportional to the sound pressure of each peak value at each position of the animation corresponding to the frequency of each sampled peak value. In addition, the display step may be displayed by overlapping images having a magnitude proportional to each sound pressure of all peak values sampled at a predetermined position.

The tone conversion method of the present invention comprises the steps of: Fourier transforming a plurality of sounds input through a plurality of channels, sampling at least one specific audio frequency signal of each Fourier transformed signal, and Calculating sound source positions between the plurality of channels using audio frequency signals of frequencies corresponding to each other, and at least one audio frequency signal sampled for each sound among the plurality of sounds by Equation (2) And converting each color into a visible frequency signal, and displaying a color corresponding to the converted at least one visible frequency according to a frequency and a sound source position.

The generation position of the sound source between the sound source input positions of the plurality of channels is calculated based on the following equation.

(Where a is a sound source input position value to obtain a sound source input position of a specific channel '0', d is a distance between the sound source input position of the first channel and the sound source input position of the second channel, s is The vertical distance between the line between the sound source input position of the two channels and the actual sound source position, k is a constant of k> 0, I _diff is the sound pressure value of the second channel relative to the sound pressure value of the first channel for a specific peak Chime.)

The tone converting apparatus of the present invention comprises an input means for inputting a sound, an amplification means for amplifying the input sound, a Fourier transform means for Fourier transforming the amplified sound, and a specific audio frequency signal of the Fourier transformed signal. Sampling means for sampling at least one sample; tone conversion means for converting each sampled audio frequency signal into a visible frequency signal; and a color corresponding to at least one visible frequency converted from each of the sampled audio frequencies. And display means for displaying.

The present invention provides a tone conversion apparatus comprising: input means for inputting sound through a plurality of channels, amplification means for amplifying the input sound, Fourier transform means for Fourier transforming the amplified sound, and a Fourier transformed signal. Sampling means for sampling at least one audio frequency signal, calculation means for calculating sound source positions between the plurality of channels using audio frequency signals of frequencies corresponding to each other for the plurality of sounds, and the sampled angles. A tone converting means for converting an audible frequency signal into a visible frequency signal, and display means for displaying a color corresponding to at least one visible frequency converted from each of the sampled audible frequencies according to a sound source position do.

The display means may be configured as an image display device such as a CRT or LCD, or an illumination device such as a full color display lamp or a color laser light source device.

The color-to-tone conversion method of the present invention comprises the steps of obtaining a visible frequency (F _i ) and a luminance (B _Fi ) corresponding to the input color, converting the obtained visible frequency into an audible frequency by the following formula, and the converted audible And outputting the frequency as a sound.

-------------(4)

(here, , f is the frequency of the sound you want to find, f _l is the

Quasi-audible frequency, F _i is the input visible frequency, F _l is the reference visible frequency, B _Fi is an integer that satisfies 1≤B _Fi ≤10, C is a constant and -1≤C≤1.

The color-tone conversion apparatus of the present invention includes input means for obtaining a visible frequency and luminance corresponding to an input color, color-to-sound conversion means for converting the obtained visible frequency into an audible frequency by Equation (4), and the converted audio frequency. And output means for outputting sound.

The sound source positioning method according to the present invention comprises the steps of inputting sound from a sound source through a plurality of channels spaced apart from each other, Fourier transforming the sound input through the respective channels, and the peak value of each Fourier transformed signal Sampling a; and calculating a sound source generation position between a plurality of channels by using sound pressure of a plurality of channels at each of the sampled frequencies.

The generation position of the sound source between the sound source input positions of the plurality of channels is calculated based on the following equation.

------------ (5)

(Where a is a sound source generation position value to obtain a sound source input position of a specific channel as '0', d is a distance between the sound source input position of the first channel and the sound source input position of the second channel, and s is the two The vertical distance between the line between the sound source input position of the channel and the actual sound source position, k is a constant of k> 0, I _DIFF is the sound pressure value of the first channel for the particular peak—the sound pressure value of the second channel).

The generation position of the sound source between the sound source input positions of the plurality of channels is determined based on the following equation.

---------- (6)

If {E} _ {diff}> 0, the source of sound source is close to the first channel

If {E} _ {diff} <0, the source of sound source is close to the second channel.

Where E _diff is the sound source location value with the middle point between the two channels as '0', P _n is the sound pressure value for each peak location, n is the number of detected peaks, and M1 _energy is the sound pressure of the first channel. Energy value, M2 _energy is the sound pressure energy value of the second channel.)

The sound source position determining means of the present invention comprises input means for inputting sound from a sound source through a plurality of channels spaced apart from each other, Fourier transform means for Fourier transforming the sound input through the respective channels, and each Fourier transformed signal. Sampling means for obtaining the highest point and sampling the frequency and level of the highest point, and calculating means for calculating the sound source generation position between the plurality of channels using the respective sound pressure of the plurality of channels at each sampled frequency.

The present invention relates to a method and apparatus for selecting a harmonic color using the harmonic method, and a method and apparatus for converting sounds and chromatic tones, and more particularly, to select a harmonic color accurately tuned to an arbitrary reference color based on the harmonic method, The present invention relates to a method and an apparatus for enabling color imaging of sound and sounding of an image by mutual frequency conversion of a sound and a color having a sound.

1 is a view showing a tone conversion table of the circular coordinate system according to the present invention.

2 is a view showing a first tone conversion table of the rectangular coordinate system according to the present invention;

3 is a view showing a second tone conversion table of the rectangular coordinate system according to the present invention;

4 and 5 is a view showing the musical instrument color scale position by the tone conversion table according to the present invention.

6 is a block diagram of a harmonic color selection device according to the present invention;

FIG. 7 is a diagram showing the configuration of a first table (tone division ratio and harmonic frequency ratio) stored in the storage means of FIG. 6; FIG.

FIG. 8 is a diagram showing the configuration of a second table (chemical method code) stored in the storage means of FIG. 6; FIG.

9 is a screen state diagram provided on a computer monitor for explaining an example of the method of selecting a harmonic color according to the present invention.

10 is a screen state diagram provided on a computer monitor for explaining another example of the method of selecting a harmonic color according to the present invention;

Figure 11 is a block diagram of a tone conversion apparatus according to an embodiment of the present invention.

12 is a flowchart for explaining a first embodiment of a sound source generation position calculation algorithm according to the present invention;

Fig. 13 is a schematic diagram showing the sound source input position and the position of the sound source of each channel according to the first embodiment.

14 is a frequency spectrum diagram of FFT calculation for each L and R channel in a specific frame.

Fig. 15 is a schematic diagram showing an example of a screen on which sound is displayed as an image according to the first embodiment.

16 is a flowchart for explaining a second embodiment of the sound source generation position calculation algorithm according to the present invention;

Fig. 17 is a schematic diagram showing the sound source input position and the position of the sound source of each channel according to the second embodiment.

Fig. 18 is a schematic diagram showing an example of a screen on which sound is displayed as an image according to the second embodiment.

19 is a flowchart for explaining a third embodiment of the sound source generation position calculation algorithm according to the present invention;

Fig. 20 is a schematic diagram showing an example of a screen displaying a sound image according to the third embodiment.

FIG. 21 is a schematic diagram showing a distribution state of a sound source generating position to be utilized as a key frame for applying to a human body as an example of a screen which the output means can display in the first embodiment; FIG.

FIG. 22 is a schematic diagram illustrating only a sound source to be used as a key frame among sound sources distributed in FIG. 21;

FIG. 23 is a schematic diagram showing a process of mapping based on keyframe information of FIG. 22; FIG.

24 is a schematic diagram illustrating a state in which the mapping process of FIG. 23 is completed;

Fig. 25 is a view showing an example of a command input means in the tone color conversion mode according to the present invention and a screen display state in which sound is imaged in the fix mode;

Fig. 26 is a diagram showing a screen display state in which sound is imaged in the bit mode.

Fig. 27 is a diagram showing a screen display state in which sound is imaged in the tone mode.

28 to 33 are diagrams showing the screen state of the image expression by the tone conversion according to the present invention.

34 is a block diagram showing a preferred example of a color tones conversion apparatus according to the present invention.

35 is an original image for chromatic conversion

36 is a screen state diagram for explaining a sub-region applied to the original image.

37 is a screen state diagram in which a plurality of sub-regions is applied to the front of an original image.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 shows a tone conversion table of a circular coordinate system according to the present invention. Fig. 1 shows an example in which the 12 scales of the harmony method correspond to the color, and an example in which the 'degree' sound (denoted C in the figure) corresponds to red (682 nm) is shown. As shown, the color is arranged so that the value of taking the log of the visible frequency from the low frequency is directly proportional to the angle of 0 degrees with respect to the point where the original 'degree (C)' is located, and from red to orange, yellow, and green Appears in the order of blue and purple. And note that the coefficients on the circle (i.e., the number of octaves equally divided into logarithmically regular intervals) are shown in Figures (C), (#), (D), (Eb), Based on 12 scales represented by beauty (E), wave (F), wave # (F #), sole (G), sole # (Ab), la (A), Ra # (Bb), and poem (B) Placed. In the direction of the center, the brightness of the color system and the octaves of the musical scale correspond to 1: 1 and are arranged in the color wheel. Therefore, the luminance and the octave become higher toward the center of gravity, and the luminance and the octave become lower toward the center of gravity.

2 shows a first tone conversion table of the rectangular coordinate system according to the present invention. The horizontal axis of the first tone conversion table represents pitch and color, and the vertical axis represents octave and luminance. In the first tone conversion table, the saturation of the color is directly proportional to the tone height. Thus, in the same octave, the luminance of the colors is different but the saturation is the same.

3 shows a second tone conversion table of the rectangular coordinate system according to the present invention. The horizontal axis of the second tone conversion table represents pitch and color, and the vertical axis represents octave and luminance. In the second tone conversion table, the luminance of the color is directly proportional to the pitch. Therefore, in the horizontal direction, the luminance of the colors is the same but the chroma is displayed differently.

4 and 5 show musical instruments whose scale positions are colored by the tone conversion table according to the present invention. FIG. 4 shows a case where a color scale of a guitar and a piano is colored based on a second tone conversion table whose luminance is proportional to the pitch, and FIG. 5 shows a case where a color tone is colored based on a first tone conversion table.

Therefore, in the musical instrument of the present invention, the player can recognize the scale position by color, and can easily distinguish between the octave and the pitch of the scale.

6 shows a block diagram of a harmonic color selecting device according to the present invention. The harmonic color selection device of FIG. 6 includes a selection means 10, a storage means 12, a calculation means 14, and a display means 16.

The selecting means 10 selects a reference color for finding a harmonic color, and selects a scale division ratio and a harmonic code.

The storage means 12 store the first and second tables of FIGS. 7 and 8. Fig. 7 is a first table showing the scale division ratio and harmonic frequency ratio, and Fig. 8 shows a second table showing the harmonic frequency ratio corresponding to the harmonic code.

The calculating means 14 calculates the harmonic color by the following equation when the scale division rate is selected as the average rate.

Where F _h is the harmonic frequency to be obtained, F _r is the input frequency, k is the sound coefficient, and the frequency of one octave is equally divided by the equal ratio, and n is the harmonic frequency ratio of 1≤n≤ (k-1). K and n are natural numbers.

In general, since 12 scales are mainly used, the case where k is 12 and n is 1 to 11 will be described. Since k is 12 in the above equation Is about 1.0594, which is the ratio between neighboring frequencies when a frequency of one octave is divided into 12 equal parts of a logarithmic ratio. In other words, in the harmonic method, when one octave is divided into 12 scales, it is equal to the frequency ratio between neighboring notes (for example, 'degree' and 'degree #').

Thus, the harmonic frequency ratio n means equal intervals between the sounds constituting a particular chord. For example, a major chord consisting of `` Domisol '' is based on the frequency corresponding to the `` degree '' sound. Since the equal ratio intervals between the frequencies corresponding to the sole sound are 4 and 7, respectively, the harmonic frequency ratio values of the major codes are shown as 4 and 7 in the above table. Therefore, when the input frequency corresponds to the 'degree' sound, the plurality of harmonic frequencies that match the input frequency are the frequencies corresponding to the 'mi' sound and the 'sol' sound, respectively.

As an example, when the input frequency provided from the input means 11 is about 690 THz (wavelength of 435 nm) corresponding to blue, and the harmonic frequency corresponding to the minor code is calculated, the harmonic frequency ratio n is 3 and 3, respectively. 7, THz = 820.55 THz (wavelength of 365.61 nm), THz = 1033.83 THz (wavelength of 290.18 nm).

However, since F _h2 is out of the visible frequency range, when the calculated harmonic frequency is out of the visible frequency range, the calculated first harmonic frequency is divided by two so as not to be out of the visible frequency range. Therefore, F _h2 is 516.92 TH _Z (wavelength of 580.36 nm) divided by two.

In addition, the existing 12-tone scale with the average ratio divides the frequency of one octave by the logarithmic ratio at equal intervals as described above. In the case of pure rate, the frequency interval between neighboring sounds is not constant because the sound is created in the form of fraction on the basis of 'degree' sound. However, the pure rate is still well harmonized compared to the average rate so that it can be used as a tuning method.

Therefore, in order to calculate the harmonic frequency in the case of dividing the musical system into pure parts, the fractional frequency ratio corresponding to the harmonic frequency ratio n is referred to based on the main frequency (the frequency corresponding to the 'degree' sound). (Referred to as 'pure rate frequency ratio'), and FIG. 7 shows the pure rate frequency ratio corresponding to each scale based on the main frequency.

As an example, when the input frequency provided from the input means 11 is 460 THz corresponding to red, and the user selects a major code and the harmonic frequency ratio n derived from FIG. 8 is 4 and 7, respectively, the harmonic frequency As pure rate frequency ratio corresponding to ratio 4 and 7 is 5/4 and 3/2 respectively, THz = 575 THz, THz = 690 THz.

The display means 16 converts the received harmonic frequency into visible light and displays the harmonic color on the screen of the monitor.

9 is a screen state diagram provided on a computer monitor for explaining an example of the method of selecting a harmonic color according to the present invention. As shown in the screen of FIG. 9, when the chemical method code list is clicked in the left window, the chemical method code list is displayed in the center window. For example, when the 'SUS4' code is selected from the displayed harmonic code list, three harmonic colors are displayed at ± 1 octave and ± 3 octave selected in the right window, and a harmonic frequency figure of 'SUS4' is displayed at the bottom of the color wheel. When the reference color is selected in the color wheel, the color matching the upper color matching window is changed and displayed.

Fig. 10 shows a screen state diagram provided on a computer monitor for explaining another example of the method for selecting a harmonic color according to the present invention. In another example of Fig. 10, the harmonic color selection screens are shown in the harmonic codes 'AUG' and '7 / ALT', respectively. On each harmonic color selection screen, a harmonic color display window, a luminance selection bar, a saturation selection bar, and a color wheel of the selected harmonic code are displayed on the left side, and the harmonic colors of all the harmonic codes written on the table are displayed on the right side. Selecting a reference color from the color wheel on the right changes the colors of the harmonic display.

Fig. 11 shows a block diagram of the timbre converting apparatus of the preferred embodiment of the present invention. The apparatus of Fig. 11 includes microphones 22 and 24, amplifiers 26 and 28, Fourier transducers 30 and 32, sampling means 34 and 36, tone conversion calculating means 38 and 40, sound source position calculating means ( 42, display control means 44, display apparatus 46, command input means 48, storage means 50, and control means 52.

The tone converting apparatus of FIG. 11 inputs the sound of the L and R channels through the plurality of microphones 22 and 24 for inputting sound from the sound source 20, respectively. The sound signals of the input L.R channels are amplified by the amplifiers 26 and 28, and the amplified signals are fast Fourier transformed on a frame-by-frame basis by the Fourier transformers 30 and 32, respectively. The Fourier transformed signal is filtered by a predetermined filtering algorithm in the sampling means 34 and 36, and the filtered peak values are sorted in sound pressure order and then sampled to an appropriate number of peak values, for example, 30 or less. Each sampled peak value signal has a frequency value and a sound pressure value, respectively. This signal is converted into the visible frequency by the tone conversion formula described above in the tone conversion calculation means 38, 40.

As an example, the input audible frequency f _i is 329.6 H _Z corresponding to a 4 octave 'mi' sound, the constant C 'represents a value of 0.29 by a reference frequency corresponding to' red ', and the lowest audible If the frequency f _l and the lowest visible frequency F _l are set to 20 Hz and 350 THz, respectively, the visible frequency derived by the above formula is 441 THz, and the luminance of the color to be obtained is 4.33 ( 43.3%).

On the other hand, the sampled peak value signals are provided to the sound source position calculating means 42. The sound source position calculating means 42 calculates sound source positions of two signals of the L. R channel at the same frequency.

12 is a flowchart for explaining the detailed operation of the sound source generation position calculation step according to the first embodiment.

FIG. 12 is a flowchart illustrating two channels of a plurality of channels, namely, assuming that a location of an actual sound source is known and a region in which a sound source moves is defined between the first channel and the second channel, as shown in FIG. 13. It is for calculating the generation position of the sound source between the sound source input position of one channel and the sound source input position of the second channel.

In the drawing, d is the distance between the sound source input position of the first channel and the sound source input position of the second channel, s is the vertical distance between the line between the sound source input position of the two channels and the actual sound source, a is a specific channel ( For example, it represents a sound source generation position value to be obtained by setting the sound source input position of the first channel) to '0'.

In order to calculate the sound source generation position value a, a sound source signal for each channel is input as described above (100), and an audible frequency input through each channel based on the control signal and the sound source signal for each channel is a predetermined frequency. A Fast Fourier Transform (FFT) is calculated at intervals (101), and one or more peak points are detected for each channel based on the result of the FFT operation (102).

More specifically, FIG. 14 is a frequency spectrum diagram showing the results of FFT calculation of audible frequencies for each channel in one frame, in which the horizontal axis and the vertical axis are the frequency axis and the sound pressure level axis (the decibel value indicated in the figure is a negative value). The results of analyzing the electrical signal of the received sound by the magnitude of the sound pressure for each frequency. Based on the FFT calculation result, a plurality of peak points such as a first peak point P ₁ , a second peak point P ₂ , and a third peak point P ₃ are detected in the first channel for the same frequency. In the second channel, a plurality of peak points such as a first peak point P ′ ₁ , a second peak point P ′ ₂ , and a third peak point P ′ ₃ are detected.

Then, the sound pressure level value converted in units of the corresponding frequency and dB (decibel) at each peak point detected for each channel is detected, and the sound pressure difference of the peak point for each frequency is calculated and output as I _diff (103). That is, when comparing the first peak point in the first and second channels, I _diff means _the sound pressure level value of P ₁ -P ' ₁ sound pressure level value, and the output I _diff and the preset s and d Based on the value, the generation position a of the sound source for each peak generation frequency is calculated based on Equation 4 below.

Here, a is a sound source generation position value to be obtained by setting the sound source input position of the first channel to '0', k is a constant of k> 0, and s and d are the same as defined above.

On the basis of the above formula, the occurrence position a of the sound source for each peak generation frequency is calculated (104), and the detected value a is compared with the value d (105).

It is determined that the first channel is close (106), and when a> d / 2, it is determined that the generation position of the sound source is close to the second channel (107). In operation 108, the frequency value and the sound pressure level value of each detected peak point and the calculated position of the sound source are output.

The display control means 44 outputs visible light based on the input sound source generation position, the frequency value of each peak point, and the sound pressure level value, and as shown in FIG. 15 as an example of the display screen, Correspondingly set both ends of one axis to the first channel and the second channel, respectively, the position where the sound source is generated on the first axis is the coordinate of the first axis, and the frequency value of each sound source is the coordinate of the second axis corresponding to the vertical axis. According to Equation 3, the color calculated from the frequency of the sound source is represented by the area proportional to the sound pressure.

Accordingly, the display control means 44 can easily display the position of the sound source for each peak generation frequency, the color corresponding to the sound source, and the magnitude of the sound pressure on the two-dimensional plane in which the first axis and the second axis are combined.

As described above, the present embodiment calculates the position of the sound source for each corresponding frequency at which the peak point is detected. Therefore, even if the frequency input from the sound source generated at one point is analyzed, the sound source generation position value is different for each peak generation frequency within one frame. However, by applying the following second embodiment, the same sound source generation position value can be derived within one frame.

16 is a flowchart for explaining the detailed operation of the sound source generation position calculation step according to the second embodiment.

In the flowchart of FIG. 16, as shown in FIG. 17, the distance d between two input channels of the plurality of channels, that is, the first and second channels, is not given, and the position of the actual sound source and the sound source are moved. The region to be defined is not limited between the first and second channels, and if the sound source moves on an axis perpendicular to the axis connecting the first and second channels, the intermediate point between the two channels is set to '0'. This is for detecting the sound source generation position.

First, a sound source signal for each channel is input (110), and an audible frequency input through each channel is FFT calculated at a predetermined frequency interval (111), and a plurality of peak points for each channel based on the FFT calculation for each channel. The process of detecting 112 is the same as that of the first embodiment.

The sound pressure level value at each peak point detected for each channel is detected, and the sound pressure energy of the first channel and the sound pressure energy of the second channel are respectively calculated based on Equation 5 below (114). Comparing and detecting the sound pressure energy to calculate the generation position of the sound source.

At this time,

Here, E _diff is a sound source generating position value having a midpoint between two channels as '0', P _n is a sound pressure level value for each frequency at which a peak point is detected, n is the number of detected peaks, and M1 _energy is a first channel. Is the sound pressure energy value, M2 _energy represents the sound pressure energy value to the second channel.

Based on the above equation, M1 _energy and M2 _energy are calculated according to the sound pressure level value for each frequency detected as peak point in the first and second channels, and E _diff is detected according to the result value of M1 _energy and M2 _energy . . And if to determine the positive, negative, whether or not the detected E _diff (115) the E _diff> If zero, determines that the generation of the sound source position closer to the first channel _{(116), E diff <0} , the sound source It is determined that the generation position is close to the second channel (117), and the corresponding frequency value and sound pressure level value and the sound source generation position at each detected peak point are output to the display means 44 (118).

18 shows an example of a screen that can be displayed by the display means, and the screen configuration is the same as that of the first embodiment, and the frequency displayed as a circle as one sound source generation position value is calculated in one frame. The generation position of the other sound source is arranged in a straight line parallel to the second axis.

19 is a flowchart illustrating a detailed operation of the sound source generation position calculation step according to the third embodiment. The present embodiment is characterized in that the generation position of the sound source is calculated for each frequency band. Assumptions for the moving area are the same as in the second embodiment, and steps 120 to 122 in the flowchart of FIG. 19 are the same as in the first embodiment.

In step 122, sound pressure level values are detected at a plurality of peak points detected for each channel, and frequency is divided into bands based on a predetermined frequency division criterion (123). For example, when frequency is divided into three regions. Separate the audible frequency into low frequency band, mid frequency band and high frequency band.For example, 20 HZ ~ 200 HZ section as low frequency band, 200 Hz ~ 2 kHz section as mid frequency band, and 2 kHz ~ 20 kHz section as high frequency Can be separated into bands.

The sound pressure energy M1 _{energy / low} of the first channel and the sound pressure energy M2 _{energy / low} of the second channel are respectively calculated in the _low frequency band according to the sound pressure level value of each peak point detected in the low frequency band (124). ), Based on the sound pressure energy detected for each channel

Computing the E _{diff / low,} and determines whether or not positive and negative, and 125, and determines that if the E _{diff /} to determine a _low of E _{diff / low>} 0, the generation position of the sound source in the low-frequency band closer to the first channel If E _{diff / low} &lt; 0, it is determined that the generation position of the sound source is close to the second channel in the low frequency band (127).

Based on the same principle as described above, M1 _{energy / mid} and M2 _{energy / mid} and M1 _{energy / high} and M2 _{energy / high} are respectively calculated according to sound pressure level values of each peak point detected in the medium frequency and high frequency band (128). , 132), respectively, calculates E _{diff / mid} and E _{diff / high} from the sound pressure energy of each channel, and determines whether the sound source is generated in the medium and high frequency bands by determining whether it is positive or negative (129-131 and step 132). ~ 135). Then, the sound source generation position value detected for each frequency band, the frequency and the sound pressure level value at each peak point are output to the display control means 44 (136).

20 shows an example of a screen that the output means can display, and the screen configuration is the same as in the previous embodiment, and is calculated as a circle by calculating the generation position of the sound source for each frequency band within one frame. Since the generated position of the sound source has the same value for each frequency band, it is arranged in a straight line parallel to the second axis for each frequency band.

Here, when a MIDI signal is received from a MIDI device without receiving sound information as a sound source, the MIDI signal uses a unique protocol, and the location information, musical instrument information, and scale of the sound source having 256 or more channels are different from each other. Since it has information, etc., it is not necessary to calculate the frequency of each sound source and the generation position of the sound source by performing frequency analysis (FFT calculation) for each channel.

Thus, when the MIDI signal is received and reproduced and visualized, intrinsic harmonic structure information corresponding to musical instrument information (overtones generated at a specific interval, a specific size, and a specific number in the high frequency portion centered on the main sound) is stored in the storage unit Therefore, by generating the harmonics according to the scale information, the original sound of the instrument is reproduced, and the position information of the sound source is displayed on the first axis, and the main sound (scale information) and the harmonic (instrument information) on the second axis. According to the display, the scale information of the predetermined value) is displayed, and the corresponding size and corresponding color are displayed at the position.

Therefore, when receiving a MIDI signal can be displayed by changing the shape for each channel (that is, for each instrument), the unit cost is low because no FFT operation is required, it can be easily added to the existing MIDI signal playback device There is this.

In addition, the display control means 44 may display the information of each input peak by the visible light of the color corresponding to the frequency and the area proportional to the magnitude of the sound pressure at the position where the sound source is generated for each frequency. As another embodiment of the method for visually displaying the above, the distribution state in the coordinates based on the sound source generation position and frequency of each peak is used as a keyframe of the animation, and the mapping is performed based on the keyframe. You can display the animation motion in response to the sound source's origin and sound level.

That is, when the sound source generation position of each peak is calculated, two-dimensional coordinate values based on the sound source generation position and frequency of each peak are input as keyframe information, and mapping is performed based on the received keyframe information to generate sound sources. The image is converted into an animation motion (such as a dancing doll) indicating the position and height of the sound and output to the display control means 44. As a result, the display control means 44 displays the animation motion based on the motion information described above.

FIG. 21 is a schematic diagram illustrating a distribution state of a sound source generation position to be used as an animation keyframe for application to a human body as an example of a screen that the output means may display in the first embodiment of the present invention, and FIG. 22 is shown in FIG. 21. It is a schematic diagram showing only the sound source to be used as a key frame among the distributed sound sources.

FIG. 23 is a schematic diagram illustrating a mapping process based on the keyframe information of FIG. 22, and FIG. 24 is a schematic diagram illustrating a state where the mapping process of FIG. 23 is completed.

As shown in the figure, only the part similar to the human body shape is taken as the animation keyframe data, and based on this, the human body and the specific shape are mapped and the animation motion is displayed.

Fig. 25 shows an example of the command input means of the tone color conversion mode according to the present invention and a screen display state in which sound is expressed in the fix mode. The tone conversion input means of the present invention shown in Fig. 25 is presented on the screen.

The tone conversion command means is the music file OPEN key, music file list window, play key, pause key, stop key, fast rewind key, fast play key, repeat key, Fourier transform resolution adjustment key, resize key, scaling adjustment Keys, peak number adjustment keys, and the like.

The music file open key is used to load a music file from another directory.

The music file list window provides a list of imported music files.

The Fourier transform resolution adjustment key adjusts the resolution of the vertical axis of the screen.

Fourier transform adjusts the sampling resolution in each frame unit.

The resize key adjusts the size of the sampled peak value.

The scaling key adjusts the scaling of the sampled peak value linearly or nonlinearly.

The peak number adjustment key adjusts the number of selected peak values among the total sample values sampled for each frame.

Tone conversion command means image file OPEN key, image file list window, screen mode selection key, sound input selector, screen left and right width adjustment key, screen up and down height adjustment key, sound frequency band setting key, screen output mode selection key, setting Contains value memory keys.

The image file open key is for importing a basic image file to be expressed for each frequency.

The image file list window displays a list of selected image files.

The screen mode selection keys provide a full screen, a reduced screen shown, a vertical screen, a horizontal screen, a horizontal frequency screen, and the like.

The sound input selection key selects the CD mode for music files on the hard disk of the computer and the line mode for inputting real sound through a microphone in real time.

The left and right width adjustment keys are displayed on the left side in the center of the sound source, and on the right side, they are displayed in the left and right directions. Therefore, at the left end, the image is displayed only at the center vertical axis like the mono sound source. To exaggerate stereo, you can adjust it to the right end.

The height adjustment key of the screen is displayed on the horizontal axis because the frequency position is gathered on the center horizontal axis when it is in the left direction. In the right direction, the frequency position is displayed up and down because it is expanded up and down.

The screen output mode selection key selects fixed mode, beat mode, and tone mode.

In the fixed mode, as shown in FIG. 25, an image is displayed as one selected image file.

In the bit mode, as shown in Fig. 26, an image file is displayed while the image file is changed according to the condition. The change condition of the image file uses the keys displayed on the left and right of the screen. The right key is for setting the frequency band to detect the condition and the left key is for setting the threshold for detecting the size of the image file within the set frequency band. Therefore, when an image having a size exceeding the threshold hold is displayed in response to a large sound pressure within the set band, the image is displayed by changing to the next order image.

In the tone mode, as illustrated in FIG. 27, an image of a wave shape over time is represented along a circumference. Therefore, in the tone mode, the waveform change according to the input time may be known.

28 to 33 show examples of tone conversion images by various image files. Therefore, in the present invention, it is possible to express music in various images by various shapes of the basic image file.

Fig. 34 shows a preferred example of the color tone conversion apparatus according to the present invention. The color-tone conversion apparatus includes image input means 60, image scanning means 62, 64, color extraction means 66, color-tone conversion calculation means 68, sub-tone synthesis means 70, L-channel synthesis means 72. And R channel synthesizing means 74, amplifiers 76 and 78, and speakers 80 and 82.

The image input means 60 inputs a video or still image picked up by a video camera, a digital camera, or the like. The input image is provided to the image scanning means 62, 64 at 30 frames per second.

The image scanning means 62 and 64 are composed of a subregion 62 and a unit cell region 64. In the exemplary embodiment of the present invention, as shown in FIG. 36, the subregion 62 has a scanning bar in which a plurality of unit cells are vertically arranged. Each unit cell includes a plurality of pixels of size m × n. Therefore, the 2D image is divided into a plurality of sub-regions as shown in FIG. 37.

The color extraction unit 66 inputs a plurality of pixel values from each unit cell 64 to extract the color frequency and luminance of the unit cell as representative values of the pixels, for example, an average value, a median value, and a maximum value.

The color-tone conversion means 68 generates an audible frequency based on the color-tone conversion formula with the color frequency and luminance provided from the color extraction means 68.

The sub-region synthesizing means 70 synthesizes the audible frequencies corresponding to each generated unit cell to generate one composite sound signal.

Each of the plurality of synthesizing means outputs an L-channel compound sound L _i and an R-channel compound sound R _i , respectively, corresponding to the position values of the sub-regions arranged in the original image. For example, the leftmost subarea has an L position value of 10 if its L position value is 10, and the rightmost subregion has an L position value of 0 and an R position value of 10.

The L channel synthesizing means 72 synthesizes a plurality of L channel composite sounds L ₁ to L _n to generate L channel sounds, and the R channel synthesizing means 74 generates a plurality of R channel composite sounds R ₁ -R _n . Synthesize to produce R channel sound.

The generated L and R channel sounds are output through the loudspeakers 80 and 82 through the amplifiers 76 and 78, respectively.

Therefore, the color of the image is output as stereo sound in each frame unit.

When the image is to be expressed in mono sound, it is also possible to output the sound of each subregion in response to the scanning speed while scanning the subregion from the left to the right of the image.

In this way, this nickname provides a method of harmonizing colors using the Mars method, suggesting a more objective and scientific tonalization theory, and can derive a color system that is precisely tuned to an arbitrary reference color. And by providing an objective and valid frequency conversion criterion, it is possible to visualize the audio and sound of color.

In addition, in the case of visualizing sound, the present invention provides various outputs of visible light by calculating a location of a sound source and displaying the visible light converted from the sound as an area proportional to the magnitude of sound pressure for each frequency at the location of the sound source. The device is not only able to derive harmonious colors from the harmonious music already harmonized, but also can be applied to lighting of performance stages, landscaping of urban architecture and fields for the hearing impaired, and to visualize visual information. In this case, it has an effect that can be widely applied to all of you such as composition using image, environmental music, field for the visually impaired.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited thereto, and various modifications and changes can be made within the scope of the claims and the detailed description of the invention and the accompanying drawings. It is natural to fall within the scope of

Claims (36)

Selecting a harmonic code; Selecting a target color; Calculating brightness of the selected reference color; Determining an octave corresponding to the calculated brightness; A harmonic color frequency is calculated from the reference color frequency on the basis of the scale ratio of the scale in the octave and the harmonic frequency ratio of the selected harmonic code. Where Fh is the harmonic color frequency, Fr is the reference color frequency, k is the scale factor corresponding to the harmonic color frequency, n is the harmonic frequency ratio, 1≤n≤ (k-1), and k and n are natural numbers. Calculating according to; Converting the harmonic color frequency into a visible frequency band by halving the harmonic color frequency when the calculated harmonic color frequency is out of the visible frequency band; And displaying the calculated harmonic color frequency as a harmonic color of the reference color. Selection means for selecting a scale division ratio, a harmonic code and a reference color; Storage means for storing a harmonic frequency ratio table of harmonic scales by the harmonic code according to the division ratio; The harmonic color frequency is calculated from the reference color frequency based on the ratio of the sound field frequency in the determined octave and the harmonic frequency ratio of the selected harmonic code based on the selected sound field division ratio. Where Fh is the harmonic color frequency, Fr is the reference color frequency, k is the scale factor corresponding to the harmonic color frequency, n is the harmonic frequency ratio, 1≤n≤ (k-1), and k and n are natural numbers. Calculating means for converting the harmonic color frequency into 1/2 the visible frequency band when the calculated harmonic color frequency is out of the visible frequency band; And And display means for displaying the calculated harmonic color frequency as the harmonic color of the reference color. Generating a sound signal by inputting a sound; Fourier transforming the sound signal; Inputting the Fourier transformed signal in units of frames at predetermined time intervals; Obtaining at least one peak value for each frame; Arranging the obtained peak values in order of sound pressure magnitude; Sampling a predetermined number of peaks in order of magnitude of the sound pressure; The sampled at least one audio frequency signal is (here, Where F is the visible frequency to be obtained, Fl is the reference visible frequency, the integer part of x is the octave value, B _{F is} the luminance _{of the} color, x * is the negative height within one octave, f _i is a sampled audio frequency, fl is a reference audio frequency, and C 'is a real number between 0 ≦ C ′ ≦ 1 as a constant); And And displaying a color corresponding to the converted one or more visible frequencies. The method of claim 3, wherein the displaying step And a color proportional to the sound pressure of each peak value at a frequency position of each sampled peak value and displaying an image colored with the converted color. The method of claim 3, wherein the displaying step Corresponding to each movement position of the animation corresponding to the frequency of each sampled peak value, the magnitude of the movement at each movement position is proportional to the sound pressure of the corresponding peak value, and the coloring of the animation is represented by the converted color Tone conversion method characterized in that. The method of claim 3, wherein the displaying step And overlapping images having a magnitude proportional to each sound pressure of all peak values sampled at a predetermined position. The method of claim 3, wherein the sound is generated from a plurality of sound sources, respectively, and the sounds generated from the respective sound sources are respectively generated as a plurality of sound signals through a plurality of channels, respectively, and an input position of the plurality of sound signals. Tone conversion method characterized in that for calculating the position of the sound source in accordance with the following formula. (Where a is a sound source generation position value to obtain a sound source input position of a specific channel '0', d is a distance between a sound source input position of the first channel and a sound source input position of the second channel, and s is the two Vertical distance between the line between the sound source input of the channel and the actual sound source position, k is a constant of k> 0, I _diff is the difference between the sound pressure value of the first channel and the sound pressure value of the second channel for a specific peak .) Input means for inputting a sound to generate a sound signal; Amplifying means for amplifying the sound signal; Fourier transform means for Fourier transforming the amplified sound signal; The Fourier transformed signal is input in frame units at predetermined time intervals, at least one or more peak values are obtained for each frame, and the obtained peak values are arranged in a sound pressure magnitude order, and a predetermined number of peak values are determined in the sound pressure magnitude order. Sampling means for sampling with; For each sampled audio frequency signal, the following equation (here, Where F is the visible frequency to be obtained, Fl is the reference visible frequency, the integer part of x is the octave value, B _{F is} the luminance _{of the} color, x * is the negative height within one octave, tone conversion means for converting each of the visible frequencies by f _i is a sampled audio frequency, fl is a reference audio frequency, and C 'is a real number between 0 ≦ C ′ ≦ 1 as a constant; And And display means for displaying a color corresponding to at least one visible frequency respectively converted from each of the sampled audible frequencies. Obtaining a visible frequency (F _i ) and a luminance (B _Fi ) corresponding to a representative color representing each unit cell region for each of the plurality of unit cell regions of the 2D color image; Obtained visible frequency with the following formula (here, , f is the negative frequency to be obtained, f _l is the reference audio frequency, F _i is the input visible frequency, Fl is the reference visible frequency, B _Fi is an integer between 1≤B _Fi ≤10, and C is a constant- A real number between 1 ≦ C ≦ 1); And And outputting the converted audio frequency as a sound corresponding to a representative color of the corresponding unit cell region. 10. The color-tone conversion method of claim 9, wherein the representative color value is any one of an average value, a median value, or a maximum value of a plurality of pixel values of a unit cell area. The color-tone conversion method of claim 9, wherein the converted negative distance weights are added corresponding to the distances from the reference point of the 2D color image to the position of each unit cell region. Image scanning means including a plurality of sub-regions having a scanning bar shape in which a plurality of unit cells are vertically arranged; Color extraction means for inputting a plurality of pixel values from each unit cell to obtain a visible frequency (F _i ) and a luminance (B _Fi ) of the unit cell from representative values of the pixels; Obtained visible frequency with the following formula (here, , f is the negative frequency to find, f _l is the reference audio frequency, F _i is the input visible frequency, Fl is the reference visible frequency, B _Fi is an integer between 1≤B _Fi ≤10, C is a constant -1 A real number between ≤ C ≤ 1); And And output means for outputting the converted audible frequency as a sound corresponding to a representative color of the corresponding unit cell region. The method of claim 12, wherein the output means Sub-region synthesizing means for synthesizing audible frequencies corresponding to each unit cell and generating one compound sound signal; Channel synthesizing means for outputting an L-channel composite sound Li and an R-channel composite sound Ri corresponding to the position values of each sub-region arranged in the original image; And And a speaker for outputting the generated L and R channel sounds. delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete