JP4456423B2 - Color-sound converter - Google Patents

Description

  The present invention relates to a color-sound conversion device used in an apparatus for recognizing the color of an object by timbre or voice, and recognizes the color of the object surface from the reflected light of the object surface detected by an optical sensor. The present invention relates to an apparatus for outputting information.

  People with visual disabilities often have inconveniences in everyday life because they do not know the color or pattern of objects. For example, the right and left color matching of socks and the selection of appropriate clothing at the ceremonial occasion. In addition, people with color blindness struggle to talk by color name, and it is also important to know the color of the object by color name.

An apparatus for receiving reflected light from the surface of an object and causing a person to recognize a color by voice is known, and Patent Document 1 proposes an apparatus that utters a color of an object by a color name. Documents 2 and 3 propose an apparatus that generates a sound by converting the color of an object into a timbre.
JP 2002-22537 A JP-A-56-137235 JP 63-163120 A Mitsuru Tsukito: Research on color identification system for color handicapped persons, Nagoya City Industrial Technology Research Report, 81 (1996) 13. Ren Takahashi: Color sensor that reads out sound by voice, transistor technology, 37, 78 (2000) 260.

  However, this type of conventional device only outputs a single color of the object as a timbre or sound, and when it is desired to know the color arrangement or change of the surface of the object, the light receiving unit of the apparatus is repeatedly applied to the surface of the object. The only way was to guess the color distribution and change from the sound and voice of each point.

  Also, if the detected color is output as a color name, the color name changes when the threshold is exceeded even when the color is gradually changing, so whether the color is changing gradually or different color areas There is a problem that it is not possible to know whether the two are adjacent to each other along the boundary line.

  In addition, those that output colors as timbres cannot be used unless the relationship between colors and timbres is mastered, and visually impaired persons who cannot recognize colors visually can assist in learning the relationship between colors and timbres. When a person is required and the relationship between color and tone is forgotten or ambiguous, there is a problem that the color cannot be recognized accurately.

  Furthermore, when a visually impaired person performs FAX or copying, it is necessary to specify which side of the sheet is a printed side on which information is described. However, it is difficult to identify only by recognizing one color, and it has recognized multiple colors, but when the information description area is small, many operations are required to identify the front and back, There was also a problem that it was mistakenly judged to be blank.

The present invention calculates repeatedly the color element values such as RGB values and HLS values specifying the color of the received light, it generates sound data to correspond to color element value computed in each iteration calculation, the sound data When a command signal is received, a color name having a color element value closest to the calculated color element value is selected from a plurality of color names registered in advance. The above-mentioned problem is solved by providing a color-sound conversion device that utters the selected color name.

Color according to the present invention - the sound conversion apparatus as claimed in claim 1, an optical sensor 3 for receiving light, color component calculation means for calculating a color element value identifying the color of the light from the received light 33, 35, a sound data generation means 31 for generating sound data corresponding to the calculated color element value, a color name table 34 in which a plurality of color names and color element values corresponding to each color name are registered, and a calculation A color name acquisition means 37 for acquiring a color name having a color element value closest to the color element value from the color name table; speakers 4, 9, 42; and a command button 21; The generated sound is always output to the speaker, and the color name acquisition means can be realized as a configuration that outputs voice information of the selected color name to the speaker when receiving a command signal.

The color according to the present invention - the sound conversion apparatus as claimed in claim 2, an optical sensor 3 for receiving light, color computes the color element value identifying the color of the light from the received light elements Calculation means 33, 35, sound data generation means 31 for generating sound data corresponding to the calculated color element value, a color name table 34 in which a plurality of color names and color element values corresponding to each color name are registered, Color name acquisition means 37 for acquiring a color name having a color element value closest to the calculated color element value from the color name table, and speakers 4, 9, 42, and the sound data generation means Is always output to the speaker, and the color name acquisition unit is configured to output voice information of the acquired color name to the speaker when the acquired color name is changed by repeatedly acquiring the color name. it can.

Various mappings between colors and sounds are possible. For example, the hue can be mapped to an infinite scale signal, and at this time , sound data including the infinite scale signal is generated.

Further, when each element for determining color (for example, RGB) is associated with a specific musical instrument and the value of each element is expressed by the sound level of the corresponding musical instrument, at least three types as described in claim 3 Sound data including the timbre of the instrument is generated.

As for the brightness, for example, when the light color is converted to a high sound and the dark color is converted to a low sound, the overall frequency increases as the lightness increases corresponding to the lightness of the color as described in claim 4. Is generated.

Also, with regard to the saturation, the turbid color is converted by adding noise that increases the volume corresponding to the low saturation, and corresponding to the saturation of the color as described in claims 1 and 2. Sound data including noise whose volume increases as the saturation decreases is generated.

The apparatus of the present invention can continuously convert color information into sound information and output the sound while tracing along the object. Further, since the color name is output together with the timbre as necessary or automatically, the following which has been impossible until now can be performed.
(1) Color pattern recognition A pattern with a border such as a striped pattern suddenly changes in tone when moving the device along the surface of the object. On the other hand, in gradation, since the timbre changes continuously with the movement of the apparatus, the state of color change can be understood by the difference. Also, depending on which direction the timbre changes, it is possible to recognize what color is arranged and how, and by listening to the change of the timbre when the moving direction is changed, what kind of pattern is It is possible to guess if there is.
(2) Confirmation of presence / absence of information By tracing the surface of the object with the apparatus of the present invention, the color change and the boundary can be understood, so the presence / absence of information such as characters on the object surface can be recognized as the color change, It becomes easy to specify.
(3) Acquisition and confirmation of timbre and color name When the timbre corresponding to the color is output, the color name corresponding to the timbre is uttered by changing the color name or pressing a button. Therefore, the correspondence between the timbre and the color name can be confirmed at all times, so that erroneous recognition can be avoided. Also, the device can be used without assistance from the beginning.

  In the following description, the voice information of a word representing a color name is called a voice, and the voice information having no meaning as a word is called a sound. In addition, recognizing a color change state and boundary is called color pattern recognition, and specifying a color by sound type is called color name recognition.

1. Stand-Alone Color Recognition Device FIG. 1 is a diagram showing a configuration of a stand-alone color recognition device used in a color pattern recognition confirmation test used in the present invention. In the figure, reference numeral 1 denotes a known voice name utterance device, which includes a light sensor 2 having a light source 2 and RGB photodiodes, and a speaker 4 for outputting a voice. Reference numeral 5 denotes a personal computer. This personal computer is provided with conversion means 6 for converting a color RGB signal or HLS signal into sound attribute data by a predetermined algorithm, and the obtained sound data is converted into MIDI data. 8 is a MIDI synthesizer, and 9 is a speaker for outputting sound.

  The signal of the optical sensor 3 is converted into a digital value by an A / D converter. The optical sensor 3 has an error in the detection value depending on conditions such as temperature. For this reason, in the color name utterance device 1 of FIG. 1, color correction reference data is stored in a storage memory, and a reference color (not shown) incorporated in the device for performing color measurement is also read at the same time to obtain color correction reference data and In comparison, the colorimetric value of the object 10 is corrected based on the difference.

  In order to measure colors continuously, it is necessary to stabilize the color measurement by operating an automatic calibration function during color measurement. The color measurement interval can be adjusted according to the user, for example, by changing the interval from 0.1 to 1.0 seconds at intervals of 0.1 seconds.

  The color name utterance device 1 in FIG. 1 is operated by a command button (not shown), and when the command button is pressed, the RGB value has three attributes of color (hue: hue, lightness: lightness, saturation: saturation). Conversion to HLS values, and further conversion of HLS values to color name data. The color name data is data obtained by rounding the HLS value and rounding the color name table having audio data corresponding to the HLS value to the number of colors that can be searched.

  Then, referring to a color name table (not shown) with the color name data, audio sampling data of the corresponding color name is acquired and uttered from the speaker 4 through the D / A converter 11. After that, if the command button is kept pressed, the process switching means (not shown) transmits the RGB value or HLS value continuously measured and calculated to the personal computer.

  Data received by the personal computer is converted into sound by a conversion algorithm according to the color-sound mapping method proposed in the present invention. The sound corresponding to the color is transmitted from the personal computer to the MIDI sound source 8 as a MIDI (Musical Instrument Digital Interface) standard message, and is generated from the speaker 9.

2. Color-sound mapping method The following two methods are proposed as a mapping method for converting color to sound, taking into account the three methods of color expression and sound.

3. Infinite scale system 3.1 Hue-to-sound mapping Hue mapping, which is the basis of three attributes based on typical psychological attributes in the color representation method, is examined.

  When hue is gradually changed, there is a continuity that forms a ring. Elements having such continuity in sound are scales of C (do), D (le), E (mi)... C (do). From C to the next C is called one octave, and when the frequency is increased by one octave, the frequency is doubled. A method can be considered in which one octave is mapped to a color by 12 equal temperament that equally divides 12 octaves. However, C that is one octave higher does not coincide with C at the base point, and a spiral structure is formed instead of a ring, so color continuity is impaired. Therefore, as a method of solving this contradiction, mapping using an infinite scale is performed. This method is called an infinite scale method.

  The infinite scale is a scale that perceptually feels as if it is infinitely high with several sounds. Although it has been said that there is no practical use so far, we will focus on the point that scale continuity can be realized and map the hue circle and infinite scale.

3.2 Lightness / Saturation and Sound Mapping By mapping hues with an infinite scale, only hues can be expressed, and subtle color changes such as gradation cannot be recognized. Mapping to other attributes of color, lightness and saturation, is also necessary.

  To map lightness and saturation, consider possible color changes for an infinite scale. First, the change of the pitch with respect to the infinite scale is that the infinite scale itself is the pitch of the sound, so that the peak of the frequency distribution of the infinite scale is changed up and down in octaves instead of up and down in scale units. Next, the change in timbre is generally a change in instrument sound, but it does not change because there is a risk that the instrument sound affects the recognition of the hue, and instead of the instrument sound, noise (white noise or pink) By combining (noise), the timbre can be changed. From the above, the sound element mapped to lightness and saturation with the infinite scale as the hue is the vertical change of sound and the addition of noise in octave units.

3.3 Correlation experiment between lightness / saturation and sound The experiment confirmed whether the addition of white noise and the vertical change in octave units as a sound change factor had a strong correlation with lightness or saturation. The experimental method is based on three types of white noise free (OdB), 40 dB and 60 dB and C4 (261.6 Hz) which is a general de (C), C6 (1046.5 Hz) on 2 octaves, 2 Nine kinds of sounds were combined and evaluated, with three kinds of sounds of C2 (65.4 Hz) under octave. The sound was generated by the color recognition apparatus shown in FIG. 1, and an oboe sound was generated from the MIDI sound source, and the volume was adjusted to 70 dB at a location 50 cm away from the speaker 9. Here, since white noise is a sound that is not included in the MIDI sound source, it is generated in the WAV format and reproduced simultaneously.

  From this experiment, it was recognized that the white noise volume had a great effect on the saturation when compared with the octave as a reference, and that when the volume was increased, it felt a dull color. It was recognized that the octave change had a large effect on the brightness when compared with the white noise volume as a standard, and that when the octave was raised, it felt a bright color. Also, when white noise is added, a dull image is added and the image appears to be slightly dark, but the change in the volume of white noise has little effect on the brightness.

3.4 Overview of the infinite scale method Figure 2 shows an overview of the infinite scale method. Hue was mapped to 12 equal temperament infinite scales and divided into 12 parts. The base point C was red, which was highly common in color hearing research. Next, lightness is mapped in octave steps up and down (for example, -2, -1, 0, +1, +2), and saturation is mapped to noise in steps (for example, 0, 40, 60 dB). did. As for the achromatic color, the volume was gradually increased from white to black as noise only. Originally, white should be silent, but in order to show that it is in a colorimetric state, the volume is set to a low level of about 35 dB. According to this method, a color / sound conversion algorithm is developed, and a sound corresponding to 184 colors can be generated. The sound is reproduced by transmitting this sound as a MIDI message to the MIDI sound source.

4. Trio system 4.1 Mapping of RGB and sound Color can be expressed by additive color mixture of RGB which is the three primary colors of light. It is possible to express a color by associating and playing an instrument sound with each of RGB. Since the number of necessary instrument sounds is three, it is called a trio system. For example, when red = piano, green = oboe, and blue = horn, the sound is red if it is a piano-only sound, and yellow if the piano and oboe have the same volume and the maximum value. Ideally, three instrument sounds corresponding to RGB are selected as instrument sounds that are easy to imagine each color. However, it is difficult to specify the instrumental sound because the color hearing ability varies greatly between individuals. Therefore, the five types of trio shown in Table 1 were used as candidates so that the instrument sounds can be easily separated and identified without causing discomfort.

  Since the color changes according to each of the RGB stimulus amounts, it is necessary to map the stimulus amount and the sound. “Height” and “magnitude” can be considered as elements of the sound to be mapped, but it is desirable to hear only one type of instrument sound for the three primary colors such as red. ”(FIG. 3).

4.2 Outline of the trio system The three instrument sounds corresponding to RGB can be selected from the combinations shown in Table 1. In addition, it is considered that it is easier to discriminate the three sounds when the three instrument sounds are different from each other than the same scale, and C, E, and G are assigned to each instrument sound to obtain a chord.

  If the volume balance between the instruments is poor, the image color will be biased. The actual volume of MIDI varies depending on the instrumental sound even if the numerical values specifying the volume are equal. Therefore, the volume of each instrument was measured and the volume correction coefficient of each instrument was obtained and corrected so that the same volume was obtained for all instrument sounds. The volume was changed in 64 steps. The RGB light amounts are r, g, b (0 ≦ r, g, b <256), and the MIDI volume designation values of the three types of instrument sounds are Va, Vb, Vc (0 ≦ Va, Vb, Vc <64). Then, the trio system can be expressed by the following mathematical formula.

ここで、ｋは定数、α、β、γは音量補正係数である。

Here, k is a constant, and α, β, and γ are volume correction coefficients.

  A MIDI message for generating a sound is generated by a color-sound conversion algorithm according to this method, and is transmitted to a MIDI sound source using a multitimbral function to reproduce the sound. Thereby, a continuous sound can be generated rather than an infinite scale.

5). Verification Test Results As a method of presenting colors with timbres, a verification test was performed in which the infinite scale method and the trio method proposed in the present invention were symmetric. The test subjects were 8 sighted people in their 10s and 30s (5 men, 3 women) and 2 visually impaired people (30s women, 50s men). Move the colorimetric part along the line and hear the change in the sound as it moves. (1) Is it gradation or striped? (2) If it is a striped pattern, what is the number of colors and the color name? (3) If it is gradation, how many colors will it change? It was done by getting answers to such questions. The results obtained from the verification test are as follows.
-The trio system was more effective than the infinite scale system in identifying color patterns. However, the infinite scale method has a high identification rate and is considered to be a very effective method.
-There was no difference in color pattern identification between the two mapping methods due to visual impairment or scale identification ability.
・ In the color name recognition by the infinite scale method, the scale identification ability was greatly affected, and the higher the scale identification ability, the higher the correct answer rate.
-The trio method was more effective in recognizing color names for people with low scale identification ability.
・ It was confirmed that visually impaired persons can recognize color patterns with this system.

6. Portable Color Recognition Device A stand-alone type device as shown in FIG. 1 is inconvenient for practical use. FIG. 4 is a diagram showing an example of a basic configuration of a portable color recognition apparatus according to the present invention. The apparatus is roughly divided into a color measurement unit 20, a command button 21, an audio data generation unit 30, and an output unit 40. The color measurement unit 20 includes an optical sensor 3, an A / D converter 22, a memory 23 for storing color correction reference data, and a comparison calculator 24 for color correction.

  The audio data generation unit 30 includes a sound data generation unit 31 and a voice data generation unit 32. The sound data generation unit outputs sound data corresponding to the measured color by the mapping method as described above, and the voice data generation unit 32 outputs voice data corresponding to the color name of the measured color. The output unit 40 includes a D / A converter 41 and a speaker 42, and sounds and voices corresponding to the measured colors are output from one speaker 42.

  FIG. 5 is a diagram showing the configuration of the voice data generation unit 32, which is the same configuration as that used in the color name utterance device shown in FIG. That is, the computing unit 33 converts the RGB value into a color three-attribute HLS value, and further converts the color three-attribute data into color name data. Here, the color name data is a color defined in JIS Z8102, and the three attribute information of the color is rounded down to the number of colors in the color name table 34 having the color name data corresponding to the audio data.

  Then, referring to the color name table 34 with the color name data, voice information (sampling data) of the acquired color name is uttered from the speaker 42 through the D / A converter (see FIG. 4).

  FIG. 6 is a diagram showing an example of the configuration of the sound data generation unit 31 when using the trio system. That is, the calculator 35 calculates the volume of each musical instrument part from the RGB values and instructs the synthesizer 36 to set the volume of each instrument.

  The synthesizer 36 has a general configuration, and is shown in the figure as having a voltage controlled oscillator (VCO), a voltage controlled filter (VCF), a voltage controlled amplifier (VCA), a noise generator, a tone color storage memory, and a mixer. However, it is a diagram schematically shown to the last. Actually, instead of sending a signal to each, it is converted into a command held by the synthesizer 36 and sent. The present invention is not limited to the processing shown in the figure, and can be applied to all devices that generate a plurality of sounds having a scale and a timbre.

  A typical synthesizer generates a reference frequency of the target sound with the VCO, filters the target tone with the VCF, and uses a value called ADSR (Attack, Decay, Sustain, Release) with the VCA to produce volume and sound. Controls the rise and decay of The VCO controls the pitch, the VCF and VCA control the tone, and the VCA controls the volume. Each of these sounds is one sound (one part).

  In the case of sounding with RGB sounds, for example, a piano tone is set for R, a trumpet tone is set for G, and a violin tone is set for B, and each detected RGB value is set as the volume of the corresponding instrument. Sound is generated by instructing tone color data and volume data to a channel for processing each tone color.

  FIG. 7 is a diagram showing an example of the configuration of the sound data generation unit 31 when the infinite scale method is used. That is, the computing unit 33 converts the RGB value into a three-attribute HLS value of color, and commands the scale, pitch, and noise volume according to the values of hue H, saturation L, and brightness S, respectively.

  Hue values are C (do), C # (do #), D (le), D # (le #), E (mi), F (fa), F # (fa #), G (so) , G # (So #), A (La), A # (La #), and B (B) are assigned to 12 scales.

  The lightness value is octave so that it sounds low octave at low lightness and high octave at high lightness. The saturation value is the noise volume, and the noise volume increases as the saturation decreases. The musical scale data created in this way is pronounced by commanding the synthesizer 36 as shown in the trio system.

  The command button 21 is an operation switch and operates as a switch for switching between sound and voice. When the command button 21 is operated, the command button 21 sends a signal to the voice data generation unit 32 to utter the color name once and then the sound data. Returning to the generation unit 31, the signal of the color measurement unit 20 is continuously sent to the sound data generation unit 31 while the command button is kept pressed. Therefore, when the command button 21 is pressed, the color name is first uttered. When the command button is continuously pressed and the object surface is traced, the sound is generated while changing according to the change in color, and the command button 21 is released once when appropriate. If pressed again, the name of the color detected at that time is spoken.

  The basic method of using the device by operating the command button 21 is to press the colorimetric unit, press the command button to hear the name of the starting color, and continue to press the command button to make a sound corresponding to the color. Therefore, the user can recognize the color pattern by listening to the color change as a sound change while tracing the object. At the initial stage, when you are not used to color and sound mapping, press the timely command button when the sound changes, and recognize the color change while listening to the color name each time. In addition, by learning, it becomes possible to recognize how many colors it is with only sound, and more efficient color recognition becomes possible.

  When the color is recognized by sound, the color name can be automatically uttered. FIG. 8 is a diagram showing a configuration of the voice data generation unit 32 that automatically utters a color name when the color name changes when the apparatus is moved along the surface of the object. As shown in the figure, after the three color attributes are converted into color name data, the color name data recorded last time is compared, and when the color name data changes, a signal of the color measuring unit 20 is output.

  In the said embodiment, the infinite scale system and the trio system were illustrated as a color and sound mapping system. These methods are preferable methods adapted to human perception, but the present invention is of course not limited to these methods. For example, as another method, the hue corresponds to multiple instrument sounds, the color continuity is played simultaneously by neighboring instruments, and the volume is changed by changing the volume from high to low on one side and from small to high on the other side. There is also a method of adding continuity of noise, adding noise to chroma expression, brightness expression changing the frequency ratio of sound frequency or changing the ratio of high and low frequency, and natural noise such as wind and waves as noise Various methods such as use are conceivable.

Diagram showing an example of a stand-alone color recognition device Figure showing an overview of the infinite scale method Diagram showing the outline of the trio system The figure which showed the example of the basic composition of the portable color recognition device The figure which showed the example of the voice data generation part The figure which showed the example of the sound data generation part at the time of a trio system The figure which showed the example of the sound data generation part at the time of the infinite scale method The figure which showed the example of the voice data generation part which utters a color name automatically

Explanation of symbols

3 Optical sensor 4 Speaker 9 Speaker
21 Sound-voice switch
31 Sound data generator
32 Voice data generator
34 Color name table
42 Speaker

Claims (4)

A light sensor (3) for receiving light, color element calculation means (33, 35) for calculating a color element value for specifying the color of the light from the received light, and sound data corresponding to the calculated color element value A sound name generating means (31) to generate, a color name table (34) in which a plurality of color names and color element values corresponding to each color name are registered, and a color element value closest to the calculated color element value A color name acquisition means (37) for acquiring a color name from the color name table, a speaker (4, 9, 42), and a command button (21), and the sound data generation means adjusts the color saturation. Correspondingly, sound data including noise that increases in volume as the saturation is lowered is generated and output to the speaker, and the color name acquisition unit receives the command signal from the command button and selects the selected color. A color-sound conversion device that outputs voice information of a name to a speaker. A light sensor (3) for receiving light, color element calculation means (33, 35) for calculating a color element value for specifying the color of the light from the received light, and sound data corresponding to the calculated color element value A color name table (34) in which sound data generation means (31) to be generated, a plurality of color names and color element values corresponding to each color name are registered, and a color element value closest to the calculated color element value A color name acquisition means (37) for acquiring a color name from the color name table; and a speaker (4, 9, 42), wherein the sound data generation means has a low saturation corresponding to the saturation of the color. The sound data including noise that increases the volume is generated and output to the speaker, and the color name acquisition unit repeatedly acquires the color name when the acquired color name changes. A color-sound conversion device that outputs voice information to a speaker. The color-sound converter according to claim 1 or 2, wherein the sound data is sound data including at least three kinds of musical instrument tones . The color-sound conversion apparatus according to claim 1 or 2, wherein the sound data is sound data having an overall frequency that increases as the lightness increases corresponding to the lightness of the color.

Images