IL125296A - Acoustic imaging - Google Patents

Abstract

Frequency processing apparatus comprising a signal input device for receiving a dynamic signal, a wavelet table for storing a base of sample wavelets, a correlating unit for correlating input frequencies with the sample wavelets stored in the table to produce a matrix of wavelet coefficients elements, and an output device for producing a dynamic graphical representation of said elements. 3078 י' בטבת התשס" ד - January 4, 2004

Description

ACOUSTIC IMAGING YAACOV GUGENHEIM C: 30639 30639spe.doc 02/07/99 Acoustic Imaging Field Of The Invention The present invention relates to acoustic imaging, and more particularly but not exclusively to acoustic imaging suitable for music.

Background Of The Invention Acoustic imaging of a form is currently used today in speech intelligibility and voice analysis in the form of "sonograms" or "spectrograms" giving a two dimensional representation of sounds examples are shown in Fig. la (wideband) and Fig. lb (narrowband). The concept of acoustic imaging may be symbolized by a literal reading of Exodus 20-15 "And all the people had seen the voices".

In Fig. 1 the abscissa X is time and the ordinate Y shows frequency. The gray level (or shading) at each ordinate in the graph is proportional to the sound level at this frequency. Fig. 1 thus demonstrates a two dimensional representation of a three parameter process, but the intelligibility and the accuracy are poor because : a) two different frequencies with the same acoustic power level would have the same gray level in this representation, so that the only distinction therebetween is in the Y position, b) due to the Fast FOURIER Transform processing, generally used in these apparatus, the frequency resolution is constant along the frequency scale giving either wideband analysis or narrowband analysis.

A constant frequency scale of the type shown in Fig. 1 is inadequate for music and for hearing as follows: For music: because musical instruments work in the equally tempered scale where the frequencies ratio between notes (semitones) are constant so the frequencies intervals between notes are not linear but logarithmic. For example in the piano keyboard Fig. 2, the sharper interval do7- ti6= 4186-3951 = 235 Hz , ut t e ass nterva o t o = . - . = . z on y ,so t at process ng cannot be used .

For hearing : because the basilar membrane in the human ear can be represented by a set of 30 filters with logarithmic (non-linear) frequency spacing .

A spectrogram of MATLAB (Fig 3 ) from THE MATHWORKS INC. is a time-dependent frequency analysis , with a pleasant colored representation but also based on FFT. It needs a large CPU time and presents a static picture. It cannot produce a real time image.

Summary of the Invention An object of the present invention is to overcome the disadvantages mentioned above in order to provide a pleasant and accurate representation of music (or speech). Embodiments of the invention may take in account dynamic signals (musical movements ,voice pitches) and represent them in a real time dynamic image taking into account changes in time-frequency resolution.

According to a first aspect of the present invention there is provided frequency processing apparatus comprising a sound input device, a table of sample wavelets, and a correlator for correlating input frequencies with the sample wavelets stored in the table, that is to say to perform a wavelet transform and to produce thereby an output signal which can be interpreted by image processing apparatus to form a dynamic output image. Preferably the sample wavelets are at frequency intervals on a logarithmic scale. The processing, including the correlation may be carried out using all of the samples taken. Alternatively, to save on CPU loading and thus to enable real time processing even on machines of modest processing power, the number of samples used can be reduced, particularly on lower pitched sounds.

The sample sounds may be the notes of the musical scale, each note being characterized by belonging to an octave and having a position within said octave.

In an embodiment there may further be provided image construction apparatus operative to translate notes into positions on a two-dimensional surface in accordance with their amplitudes, frequencies and timing.

Image construction apparatus may alternatively be operative to translate notes into positions in a three-dimensional volume in accordance with their amplitudes, frequencies and timing.

Preferably the image construction apparatus may be operative to assign a first color to a note in accordance with its position within its octave and a second color in accordance with its octave, the two colors being mixable to produce a color for the note.

Embodiments of the invention may further provide brightness control, said brightness control being operative to assign a brightness level to each note in accordance with the intensity of the note. Each note is assigned a group of pixels on the screen and the brightness control may operate by ^ setting the brightness of all of the pixels in the group. Alternatively it may be possible only to alternate the pixels between two brightness levels. In this case the brightness control would operate by randomly assigning the high brightness level to a number of pixels within the group in accordance with the desired level of brightness.

Preferably the dynamic output representation that is produced is a real time representation of the initial input signal.

According to a second aspect of the present invention there is provided an acoustic imaging device comprising an input for receiving a succession of musical notes, each note being associated with one of the musical octaves and having a position within its associated octave, and image construction apparatus operative to assign a first color to each note in accordance with its position within its octave and a second color in accordance with its octave, the two colors being mixable to produce a color for the note.

The above embodiments may be used in association with a display device. Suitable display devices include for example flat panel displays, projectors and screens, holographic display devices and cathode ray tubes.

Brief Description Of The Drawings For a better understanding of the invention and to show how the same may be carried into effect, reference will now be made, purely by way of example, to the accompanying drawings in which, Fig. 1 a shows a wide band spectrogram Fig lb shows a narrow band spectrogram Fig 2 is a representation of the acoustic spectrum, Fig 3 is an example of a spectrogram produced using Matlab, Fig 4 shows a speech sample, Figs. 5 to 8 show wavelet coefficient values in different channels, Fig. 9 shows a processed speech sample displayed in '¾asic-landscape" mode 3D, g. s ows e requency se ec v y o appara us accor ng o e present nvent on, Figs. 11 show speech samples displayed in a contour mode, Figs. 12 show displays of different samples in versions of the "notes-landscape" mode, Fig 13 shows a music sample in the notes submode of the picture mode, Fig. 14 is a display of a music sample in the wavelet picture mode, Figs. 15 and 16 are wavelet diagrams of two different notes on the musical scale, Fig. 17 is a block diagram of apparatus according to the invention, and Fig. 18 is a flow chart of a method for using the apparatus of fig. 17.

Description Of The Preferred Embodiments Fig. la shows a wideband spectrogram of a voice signal and Fig. lb shows a narrowband spectrogram of the same voice signal.

Fig. 2 shows a full spectrum of the musical range with ranges marked for common musical instruments and for the human voice.

As will be seen from Fig. 2 the piano note range covers all the other musical instrument and can be used to describe a complete concert.

An embodiment of the invention provides an acoustic imaging system that is able to deal effectively with the musical scale in a manner that is as close as possible to the way in which the scale is dealt with by the human ear. It is thus first of all necessary to, measure the level and identify each note from do0 (32.7 Hz) to ti6 (=3951 Hz) There are 7 octaves from do0 to ti^ . The note corresponding to a given note in the succeeding octave is twice the frequency of the given note : e.g. d0i=65.4 Hz , do2 = 130.8 Hz ,do3 = 261.6 ..etc..

In the equally tempered scale there are 12 adjacent semitones per octave having frequencies in the ratio of ,2V 2 = 1.05946 x o o . o s ese or "sharp"). Thus a method that is to resolve this note correctly from its neighbors must select 34.65-32.7=1.95 Hz at low frequency ( In order to reach that selectivity a FFT needs 0.51s time record length). Such a resolution is overdesigned for high frequencies because between tie and do7 we have intervals in the range of 4186-3951=235 Hz .

This is characteristic of the equally tempered scale, in which it was observed that each note may be viewed as a constant percentage bandwidth filter (Af/f = Cte , not Af ) Digital filtering (Butterworth or Chebyshev filters) or FFT cannot simultaneously solve the problems of high resolution at low frequencies, short response time at high frequencies and matched selectivity . However Wavelet analysis has a resolution cell (AfxAt) which is constant along the entire time-frequency plane.

An embodiment of the present invention is shown as a software listing in Matlab, in table 1.

Operation of the embodiment is as follows: Firstly there is generated a base of 84 matched wavelets representing the 7 octaves >< 12 notes per octave = 84 piano notes. This base provides natural functions for correlation with the input signal . The choice of piano notes as a base solves the scaling problem and is the most appropriate base to use to represent music.

Each wavelet consists of a carrier at the frequency note modulated by a "Mexican Hat" wavelet, known as a "paquet wavelet" in technical literature.

W(a, b)=(l-T2).exp(-T2/2) > An example is shown in table 2. The index k is the note number , a is the scaling factor normalized at d00=32.7 Hz and the frequency note is f (12 per octave) The sampling frequency for the highest note:( ti6=3951Hz) is 2.28><3951=9 KHz.

In practice this is fs= 10kHz ,so the sampling time is t=0.1 mS Because the reference tone in music is 440Hz = la3 ,we define the primary function at this frequency on 512 samples (see mathematical expression in Table 1).

The scaling factor for normalization is now a=2 W-oW , with n=0 at f0=440 Hz giving an observation time (wavelet duration at each frequency) of Tobs = 2 (9+ I2) / fs , from n=45 (do0) to n= -38 (tie) * to increase speed and limit buffer size the wavelet is "cut" into two parts and the correlation with the increasing or decreasing part is tested, (the signals at these frequencies are very slow).

Ns is the sample number reference per octave. It represents the resolution cell of the octave. At frequencies lower than I¾ (do, do# ,re ,..), the wavelet is stretched by the scaling factor so that the number of samples describing the complete wavelet [called here N^ = round(Tots/0.1 mS)] is greater than Ns (for 9 frequencies) . See, for example the wavelet la3, Fig. 15 (duration 51.2 mS) and compare with the wavelet do3 Fig. 16, (duration 86.1 mS). In this case N¾f = 861>Ns = 512.

For higher frequencies (only two per octave) : Nsof Note that the "life time" of musical signals is always longer than the lengths of wavelet chosen and that they also match the speech timing. b) Ίη-phase wavelets" A continuous wavelet transform gives the best results, but : - it is computationally expensive to calculate wavelet coefficients at every sampling point and we may be limited by available computational power, and - the final representation of the picture is limited in scaling freedom as it comprises small resolution cells Δ£χΔί of N discrete pixels, so that the difference between this and the wavelet algorithm mentioned above may not be visible to the eye.

If the life time of musical signals is always longer than the frequency- time matched cells the algorithm can be simplified by searching the vector norm of the wavelet coefficient projected on it's sinus and cosinus components. This is the meaning of "in phase wavelets", see table 1.

The coefficient 0.3276 normalizes the Cw values to 1 when the input signal is unity. 1 c) The 1024x768 screen [X, Y] (for example) is now filled with optimal resolution cells. Musical and physiological considerations related to time length of musical movement or breathing duration, and the physical layout of the screen in 1024x768 pixels, leads to the optimal cell resolution of 32 (or 64) pixels per note : ΔίΔΐ=32 (or 64). For example, using the 32 pixel solution with y = 768 , we have Ly/2=384 pixels =32 pix/note, giving the 12 notes for octave 6 Ly/4=192 pixels =16 pixels (2 columns in X) for octave 5 Ly/8=96 pixels for octave 4 (12x 8 pixels in Y) X4 columns Ly/16=48 pixels in Y for octave 3 (12x4 pixels in Y)X8 columns Ly/32=24 pixels in Y for octave 2 (12x2 pixels in Y)xi6 columns Ly/64=12 pixels in Y for octaves 1 and 0 (one row of 32 pixels per note This choice is a compromise between signal dynamic range and musical history time, dynamic range when the signal amplitude is represented by the number of active pixels in the resolution cell in the screen (in this case 64 is better than 32) and on the other hand the musical history is displayed on the whole screen because 32 strips of 32 pixels correspond in our case to 32X0.2048 sec. =6.55 sec frame time. The second possibility, 64 pixels per cell, gives a better dynamic range but a shorter musical history : 3.27 sec.

Thus, to compute the wavelet coefficients we translate the wavelet W; (firstly centered by b=Tobs 2 ) by steps (k.Ns) computing the integral over the entire wavelet (described by Nsof samples) but representing it as Ns samples because of the cell definition. We then compute the mean value of the product . input Signal by called "W in the LISTING (the "paquet wavelet" mean value is zero without signal). When Nsof>Ns the last step always ends at sample 2048.

Figs. 4 to 9 show the results of testing the algorithm on the words "chicken little" in the data file accompanying the book "C Algorithms for real time processing" by Paul Embree.

Fig. 4 shows the first 2048 samples of the raw sound .

These are processed and give : a) the two values in Fig. 5 in the output of the la2 channel b) the four values of the la3 channel Fig. 6 c) 8 values Fig. 7 in the output of the la4 channel d) 16 values in Fig. 8 for tis in octave 5, (and so one.. 32 values for the octave 6).

The dyadic repartition of the Wavelet coefficients is thus verified.

With two strips of 0.2s, 64 coefficients on 4096 samples are obtained.

Selecting the 4096 samples at the end of the speech file gives the phonemes "--ken little".

Three strips allows the inclusion of all the phonemes in "chicken little" into 0.6 s of speech and can represent them in 3 dimensions : time-frequency-amplitude (mode LANDSCAPE), as shown in Fig. 9.

SELECTIVITY A fundamental requirement for such a system is its selectivity : the ability to resolve and select between two adjacent semitones (the note and its sharp ,like do and do# or its flat ,like mi and mi flat) without crosstalk between channels .

Digital filtering, for example using Butterworth or Chebyshev bandpass filters, requires a large number of coefficients and this introduces delays ithe time response ,so for niinimum distortion in the time-frequency image they have poor selectivity .

FFT requires a lengthy recording time to reach the required selectivity and cannot reduce the selectivity at high frequency (mismatching).

The advantage of the wavelets transform method is that once the principal wavelet length is chosen for the required selectivity, it remains optimal at each frequency because it depends only on the number of signal cycles in the observation time, (for example in the present embodiment T0b_ x f0 = 22.5 ).

The results of a selectivity test with a pure tone in channel 39 are presented in Fig. 10. Note the output levels of the others channels (cross talk) . n As mentioned above, the number of carrier periods in a wavelet is constant. For example fo=440 Hz, gives a period T0=2.27 mS for the wavelet carrier and a wavelet duration Tobs = 51.2ms, then gives 22 periods in the wavelet .(That remains roughly true for all the wavelets ). But the number of samples per wavelet changes with frequency, thus for octave 6 we have only 64 samples but in octave 1 the stretched wavelet represented by 2048 samples is oversampled, so we can downsample the low frequencies by a scaling factor Nd (depending on Anti-aliasing): for octaves 6 , 5 and 4 the scaling factor Nd = 1 for octave 3 the scaling factor Nd = 2 for octave 2 the scaling factor Nd = 2 for octaves 1 and 0 the scaling factor Nd = 4 With the above simplification the number of operations computing a strip of 32 pixels (horizontal) corresponds to 0.2 s , and with 756 pixels in the vertical direction this gives roughly Megaflops for the basic modes (Wavelets ). See below, definition of the modes.

Each of the 32 strips forming the image appears every 0.2048 s from left to right. As each new strip is ready the image is shifted one strip right allowing the new strip to appear on the left side of the screen. The skilled person will appreciate that equivalent embodiments are feasible in which the processing of the strips proceeds in any other direction.

These operations can be completed in 0.2 s on a standard home computer, using a C version of the program , running on a suitable computer, using the process discussed above including the optimization for speed. The process including the optimization may be referred to as the simple and fast wavelet transform. As each strip represents 0.2s of music it can be seen that the calculation may be performed in real time.

If the reduction in CPU loading is not needed then a continuous wavelength computation may be used in which the above simplifications are not made. The result is smoother, that is to say the continuity is improved. The calculations are more likely to pick up high frequency harmonics that may be missed by the simplified version. 1 Λ FREQUENCY TO COLOR RELATION The twelve crescendo notes per octave can be related to combinations of the three fundamentals colors based on the relation : freq. = c/wavelength (c= light velocity). Colors may be assigned to the notes such that the lowest frequency is assigned red and the highest frequency is assigned violet. The skilled person will realize that the assignment of colors to frequencies is arbitrary and other assignment patterns are as appropriate.

The three fundamental colors in additive mixing for television, computer screens, and even printing, are : red, green, and blue. Most other colors can be obtained by suitable combination of these three fundamental colors, for example, cyan = blue and green in equal quantities, yellow = green and red in equal quantities, and magenta = blue and red in equal quantities.

This gives us six of the twelve colors that are needed for the 12 notes per octave.

A second order combination is as follows,: (Y+R)/2 ; (Y+G)/2 (C+G)/2 : (C+B)/2 (M+R)/2 ;(M+B)/2 so after classification in the rainbow order that is the frequency order ,we have the 1/1 relationship [12 colors, 12 notes] required for one octave.

In order to represent the different octaves the "note's color" may be mixed with an octave's color. Each octave is assigned a color chosen from the same palette described above (from red for the bass to violet for the treble). The effect is that dot and do7 have different hues because doi is mixed with red and do7 with violet but the eye .like the ear ,can recognize the similarity between them because "do" is always red ( the note's color).

The effective [R G B] colormap used in our prototype is shown in table 3 It is thus possible to represent graphically the physiological law that two sounds with octave intervals have consonance (harmony by similarity).

Each note thus has a single color and position, as it has a single acoustic tone and the image is thus preferably able to convey a symphony of forms and colors.

RELATION between ACOUSTIC LEVEL and COLOR BRIGHTNESS In order to represent the acoustic level in graphical form, the brightness of the pixels on screen is preferably modulated. The skilled person will be aware of numerous methods of achieving this. Two preferred embodiments are described below by way of example only. a) In the first embodiment each individual pixel can only be given two brightness states, low and high. The overall brightness is set by selecting the number of pixels within each resolution cell that are in each state. The number of excited (brightened) pixels per resolution cell on screen in the picture modes is preferably given by: z (nf, ncel) = NP x (Cw(nf, ncel)/ Normalization factor, where the normalization factor is given by the function = f{Mean[Cw(nf, ncel)], σ [Cw(nf, ncel)]} NP is the number of pixels in the resolution cell AfxAt : 32 or 64 which sets the dynamic range, σ represents standard deviation of the coefficient, Mean is the average value of frequency and time of the coefficients, Cw is the wavelets coefficients matrix, and Cw(nf, ncel) the element at frequency nf, in time cell : ncel. b) In the second embodiment the intensity of each individual pixel called "value" in Hue-Saturation-Value coordinates is modulated in accordance with an intensity scale. The brightness of the color is intensified by z (nf, ncel) = Intensity Scalex(Cw(nf, nceI)/NormaIization Factor ( ormalization Factor like above) MODES OF REPRESENTATION Three modes of imaging are available. In each mode two sub-modes of representations are also available. The sub-modes are: "Basic" i.e. Wavelets, where the resolution cell (Af.At) is constant over the display. This uses the SDRPI subroutine given in table 1 page 3.

"Piano" or "Notes" where the frequency axis is simply the well-known piano keyboard. This mode is particularly useful for teaching music.

The output matrix built from the wavelets coefficient Cw(nf, ncel) is called Cvu(i, j) in page 2 of Table 1.

The main Modes are : - A fast 2 dimensional "contour" mode, exemplified by fig. 11, - a 2 dimensional "picture" mode, exemplified by fig. 13, and 14, and - a 3 dimensional "landscape" mode, exemplified by fig. 9 and 12.

The 2D basic and piano modes differ only in the frequency representation.

In the basic wavelet modes the number of hertz per mm of screen is constant.

In the piano modes each note has 1/72 for 6 octaves(or 1/84 for 7 octaves) part of the axis; for the 2D modes the time axis is the same 32 strips giving a musical history of 6.5 s (or 16 strips giving 3.27s for 64 pixels per cells).

Examples of Visualizations -Figure 1 1 show the basic CONTOUR mode. Sounds are drawn to form bonding regions using isoclines that are iso-amplitudes. It is very fast mode and can provide real time processing even on a low cost machine . Two different thresholds are used in each of these figures.

Fig. 12 shows the LANDSCAPE mode. Landscape mode is a three dimensional mode where the x axis is time, the y axis is frequency , and the z axis is amplitude .

After a frame time interval (say 3 to 6 s), the screen becomes full and succeeding strips are then replaced every 0.2 s, thus building a surrealistic landscape of colored mountains and valleys which is constantly updated by the music. Note the similarities between this "notes" sub-mode and Fig.. 9 which shows the "chicken little" sample in the c¾asic" sub-mode .

Picture Mode The principle is that each octave from άθ\ to tie has 12 colors, one per note, which color is the same throughout each octave. However each octave has its own "octave color" so the actual color assigned to any given note is the resultant produced by mixing the octave color with the note color. Each note thus has its own specific color and yet the relationship, between notes in the same octave and the same notes in different octaves, is rendered clearly perceptible to the eye.

Figs. 13 and 14 show the submodes of the picture mode and give an idea of a still frame from the complete picture seen on the screen. The former is in "notes" submode and the second is in the "basic" wavelet submode.

General Fig. 17 is a block diagram of a device operative in accordance with an embodiment of the present invention. An audio (or more generally a frequency) signal is provided by an audio source 10 to a AGC -filter 12. AGC-Filter 12 provides amplitude gain control and frequency correction based on the frequency response of the rest of the device, that is to say it increases the amplitude at frequencies to which there is low sensitivity and reduces the amplitudes at frequencies to which there is high sensitivity. The corrected signal is then passed through an aliasing filter 14 and then through an A-D converter 16. The A-D converter 16 is set to sample at least at twice the Nyquist frequency. It will be appreciated by the skilled man that the above functions are provided as standard in sound cards or can be done by digital processing and thus the use of the invention in conjunction with a computer having a sound card does not require that these features are provided explicitly.

The digital output of the A-D converter 16 is stored in a buffer 18. The stored signal is then cross-correlated in correlator 20 with the contents of wavelet table 22. As discussed above, wavelet table 22 contains sample sinus and cosinus waveforms for all of the notes of the musical scale. The sample wavelet coefficients values are then output to an image construction block 24. In the image construction block the notes are given a spatial assignation depending on the timing and frequency of the note, and the mode, by spatial assignation unit 26. The spatial assignations are then assigned color and brightness, depending on the octave, the position within the octave and the intensity, by the color assignation block 28 The output of the image construction block 24 is then displayed on display device 30. The display device 30 may be any standard display device known to the skilled man and includes CRT devices, flat panel displays, HGED devices, mosaic devices, plasma screens and projector based screen devices. It may also include holographic displays or RGB Laser projector. The display may, for example, be used on a home computer or may be provided as part of a concert performance or used in a discotheque. In the notes mode the system is advantageous for teaching music.

In the landscape mode the point of observation can be varied using readily available 3D imaging technology. For example a perspective may be taken from far above the landscape or alternatively a lower perspective can be taken so that the observer appears to be flying in between the peaks. In this case the point of perspective can weave in and out to create the illusion of the flight path of a light aircraft.

A device according to the present invention may preferably be implemented using an add-on card to a standard personal computer. The add-on card may contain filters 12 and 14 and A-D converter 16. The buffer 18, correlator 20, wavelet table 22 and image construction block 26 would then be implemented in software.

Alternatively the add-on card may additionally contain a dedicated correlator 20, and preferably also the buffer 18 and wavelet table 22. This provides considerable performance improvement. The correlator preferably differs from the standard correlator in that it is not limited to operating on a fixed interval. Rather the interval is variable, depending on the interval being compared.

Fig. 29 is a flow chart showing a method of using an embodiment according to the present invention.

In step slO a predetermined number of signal samples are stored in a buffer. The wavelets stored in the wavelet table 22 are then loaded into the correlator 20 and a cross correlation is carried out to find the necessary wavelet coefficients. An image is then built up from the coefficients output by the correlator, depending on the mode selected, SI 8 - S20, and if the screen is full, S22, then the oldest strip falls off at one side of the screen and the newest strip is added at the other side of the screen.

It is appreciated that various features of the invention which are, for clarity, described in the contexts of separate embodiments may also be provided in combination in a single embodiment. Conversely, various features of the invention which are, for brevity, described in the context of a single embodiment may also be provided separately or in any suitable subcombination.

It will be appreciated by persons skilled in the art that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention is defined only by the claims that follow: LIST OF FIGURES FIG 1A 1 B : Sonograms (narrow - wide band) FIG 2 : piano keyboard FIG 3 : Spectrogram MATLAB FIG 4 : Speech Sample FIG 5 TO 8 : Algo outputs Ia3 Ia4 do5 fa6 FIG 9 : "Chicken Little" mode LANDSCAPE-WAVELET FIG 10 : SELECTIVITY FIG 1 1 : Mode Contour FIG 12 : "Chicken Little" mode LANDSCAPE-NOTES FIG 13 : "Yeruchalaim" mode PICTURE-NOTES FIG 14 : "Yeruchalaim" mode PICTURE-WAVELETS FIG 15 -16 : WAVELETS Ia3 - do3 FIG 17 : BLOCK DIAGRAM FIG 18 : FLOW CHART TABLE 1 : MATLAB LISTING 4 pages : pages 1 -2 : Main page 3 : subroutine SDRPI (idem SDR) page 4 .'Wavelets tables TABLE 2 : Wavelets frequencies and scaling coef a (2 pages) TABLE 3 : RGB coefficients for the color palette defining the acoustical frequency to color relation mod=input ( 'mode : wavelets = 0 , otes =1 ') ninit=input ( 1 nstrip init') ul=cputime; Cnor=0.3276; Cvu=zeros (72, 32) ; #=zeros(756, 1024) ; r nstrip=ninit : ninit+31, - ndeb=nstrip*2048; sinput=1000*y (ndeb+1 : ndeb+2048 ) ; clear C for nf=l:72, n=nf-39; Nsof=round (tobs (nf) / . le-3) ; if n>=-38 & n<-26, Ns=64;Nd=l; elseif n>=-26 & n<=-15 , Ns=128 ;Nd=l ; elseif n>=-14 & n<=-3 , Ns=256;Nd=l ; elseif n>=-2 & n<=9, Ns=512 ;Nd=2 ; elseif n>9 & n<=21, Ns=1024 ;Nd=2; elseif n>21 & n<=45, Ns=2048 ;Nd=4 ; end for ncel=l: (2048/Ns) -1, nl=(ncel-l) *Ns+l; n2=(ncel-l) *Ns+Nsof; hout=sinput (nl :Nd:n2) .*sima{nf} (l:Nd:Nsof) ' ; hout2=sinput (nl:Nd:n2) .*sima2{nf} (l:Nd:Nsof) '; C=norm( [mean (hout) mean (hout2 ) ] ) ; Cw (nf, ncel) =C/Cnor; end ncel=2048/Ns; % last time cell if Ns<2048, nl=2048-Nsof+l; n2=2048; hout=sinput (nl:Nd:n2) . *sima {nf } ( 1 :Nd:Nsof) ' ; hout2=sinput (nl :Nd:n2) .*sima2{nf } (l:Nd:Nsof) ' ; else%Ns=2048 nl=l;n2=min(2048,Nsof) ; if NsNsof, hout=sinput (l:Nd:Nsof) .*sima{nf) (l:Nd:Nsof) '; hout2=sinput (l:Nd:Nsof) .*sima2{nf} (l:Nd:Nsof) '; elseif Ns==Nsof, hout=sinput (l:Nd:Nsof) .*sima{nf } (l:Nd:Nsof) ' ; hout2=sinput (l:Nd:Nsof) .*sima2{nf) (l:Nd:Nsof) else end end end C=norm( [mean (hout) mean (hout2) ] ) ; Cw (nf, ncel) =C/Cnor; end Cvu (1: 12, : ) =Cw(l: 12, : ) ; for k=2:2:32, Cvu(13:24,k-l:k)=Cw(13:24,k/2)*ones (1,2) ; end for k=4:4:32, Cvu(25:36,k-3:k)=Cw(25:36,k/4) +ones(l,4) ; end for k=8:8:32, Cvu(37:48,k-7:k)=Cw(37:48,k/8) *ones(l,8) ; end end Cvu(61:64, 1:32) =Cw( 61:64, 1) *ones(l,32) ;%calibr Cvu ( 65 : 72, 1 : 32 ) =Cw ( 65 : 72 , 1 ) *ones ( 1, 32 ) /1.1 ; if mod=0, sdr; jlse, end, ^lf nstrip==ninit, if mod==0, Cb=C; else, Cb=Cvu; end, else if mod==0, Cb=[Cb C] ;else, Cb=[Cb Cvu],-end, end end image ( fliplr ( 10*Cb) ) colormap (hot) cputime-ul Sin the dyadic rectangles = resolution cells clear C ind=32 ; for nf=l:12, for ncel=l:32, • C( (nf-1) *ind+l:nf*ind,ncel )=Cvu(nf,ncel) *ones (32, 1) ; 'end end ind5=16; for nf=13:24, for ncel=l:32, C(385+(nf-13) *ind5 : 384+ (nf-12 ) *ind5, ncel ) =Cvu (nf, ncel ) *ones ( 16, 1 ) end end ind4=8; for nf=25:36, for ncel=l:32, C (577+ (nf-25) *ind4 : 576+ (nf-24 ) *ind4 , ncel ) =Cvu (nf, ncel ) *ones ( 8 , 1 ) ; end end ind3=4 ; for nf=37:48, for ncel=l:32, C(673+(nf-37) *ind3 : 672+ (nf-36) *ind3, ncel) =Cvu (nf, ncel ) *ones ( 4 , 1 ) ; end end ind2=2; for nf=49:60, for ncel=l:32, C(721+ (nf-49) *ind2 : 720+ (nf-48 ) *ind2 , ncel ) =Cvu (nf, ncel ) *ones ( 2 , 1 ) ; end end for ncel=l:32, C(745:756,ncel)=Cvu(61:72,ncel) ; end tobs (nf) =2Λ (9+n/12) /10000; Nsof=round (tobs (nf) / . le-3 ) ; f0(nf)=440* (2Λ (-n/12) ) ; if n>=-38 & n<-26, Ns=64;Nd=l; elseif n>=-26 & n<=-15,Ns=128;Nd=l; ^lseif n>=-14 & n<=-3 ,Ns=256;Nd=l; Lseif n>=-2 & n<=9, Ns=512 ;Nd=2; ¾lseif n>9 & n<=21, Ns=1024;Nd=2; elseif n>21 & n<=45, Ns=2048;Nd=2; end t=[0. : 0. le-3.-tobs (nf) ] ; %sinput=y(ndeb+1: ndeb+2048) ; sl=sin(2*pi*f0(nf) . *t); s2=cos (2*pi*f0 (nf) .*t); a=2A(-(5+(2-n)/12)); b=tobs (nf) 12; T=(t-b) /a; %if rem(nf,12)==12,kT=0.5; %else kT=l;end kT=l; ima=(l-T . *T/kT) .* exp(-T . *T .12); simafnf}=sl . *ima; sima2 {nf}=s2. *ima; end 1. OO Oe+00 1. OOO e+00 3.2703e+01 2. OOOOe+00 1.0595e+00 3.4648e+01 3. O OOe+00 1.1225e+00 3.6708e+ 1 4. OOOOe+00 1.1892e+00 3.8891e+01 . OOOOe+00 1.2599e+00 4.1203e+ 1 oc-T o 6. OOOOe+00 1.3348e+00 4.3053e+01 7. OOOOe+00 1. 142e+00 4.6249e+ 1 8. OOOOe+00 1.4983e+00 4.8999e+01 9. OOOOe+00 1.5874e+00 5.1913e+01 1. OOO e+ 1 1. £81Se+00 5.50 e+01 1. lOOOe+Ol 1.7818e+00 5.8270e+0i 1.2000e+01 1.8877e+00 6.1735e+01 ans = 1.300Oe+ 1 2. O OOe+00 6.5406e+ 1 1.4000e+01 2.1139e+0O 6.9296e+01 1.500Oe+0i 2.2449e+00 7.3416e+ 1 1.6000e+01 2.3784e+00 7.77S2e+01 1.700Oe+01 2.5198e+00 8.2407e+01 1.8000e+01 2.6697e+00 8.7307e+01 1. 000e+01 2.82S4e+00 9.2498e+01 2. OOOOe+ 1 2. 966e+00 9.7999e+0.t 2. lOOOe+01 3.1748e+00 1.0383e+02 2.2000e+01 3.3636e+00 1.1000e+02 2.3000e+01 3.5636e+00 1.1654e+02 2.4000e+01 3.7755e+00 1.2347e+02 2.5000e+ 1 4. OOOe+OO 1.30Sle+02 2.6000e+ 1 4.2379e+00 1.3859e+02 2.700Oe+01 4.4898e+00 1. 083e+02 2.8000e+01 4.7568e+00 1.555άβ+02 2.9000e+0i 5.0397e+00 1. 481e+02 3. OO Oe+O 1 5.3394e+00 1.7401e+02 3.100Oe+01 5.6569e+00 1.8500e+02 3.2000e+01 5.9932e+00 1. 600e+02 3.3000e+01 6.3496e+00 2.0765e+02 3.4000e+01 6.7272e+00 2.2000e+02 3.5000e+01 7.1272e+00 2.3308e+02 3.6000e+ 1 7.5510e+00 2.4694e+02 3.7000e+0l 8. OOe+00 2.6l03e+02 3.8000e+ 1 8.4757e+00 2.7718e+02 3.900Oe+01 8.9797e+00 2.9366e+02 4. OOOOe+01 9.5137e+00 3.1113e+02 4. l OOe+Ol 1.0079e+ 1 3.2963e+02 4.2000e+ l 1.0679e+01 3.4923e+02 .300Oe+ 1 1.1314e+01 3.6999e+02 4.4000e+01 1.19S6e+ 1 3.9199e+02 4.500Oe+ 1 1.2699e+01 4.1530e+02 4. 000e+01 1.3454e+ 1 4.4000e+02 4.7000e+01 1.4254e+01 4.6016e+02 4.8000e+01 1.51 2e+01 4.9388e+02 ans = f 4.9000e+01 1.6000e+01 5.2325e+02 5. OOOOe+O 1 1.6951-5+01 5.5436e+02 5. l OOe+ 1 1.79595+01 5.8733e+02 5.2000e+01 1.9027e+01 6.2225e+02 5.3000e+01 2.0159e+01 6.5925e+02 5.4000e+01 2.13575+01 6.9846e+02 5.5000e+ 1 2.2627e+01 7.3999e+02 5.6000 e+ 1 2.3973e+01 7.3399e+02 5.7000e+01 2.5398e+ 1 8.30 1e+02 5. SOOOe+ 1 2- 6909e+ 1 8. S OOe+ 2 5.9000e+ 1 2.8509e+01 9.3233e+02 . OOO e+O 1 3.0204e+ 1 9.8776e+02 6. lO Oe+01 3.2000e+01 1.0465e+0. 6.2000 e+01 3.3903e+01 1.1087e+0. 6.3000e+01 3.5919e+01 1.1747e+0. 6.4000e+01 3.8055e+01 1.2445e+0. 6.5000e+ 1 4. 317e+01 1.3185e+0. 6. 000e+01 4.2715e+01 1.396 e+0." 6.7000e+01 4.5255e+ 1 1.4800e+0-' 6.8000e+01 4.7946e+01 1.56SOe+0- 6.9000e+01 5.0797e+01 1.66l2s?+0: 7. OOOOe+Ol 5.3S17e+01 1.7600s+0" 7.1000e+01 5.7013e+01 1.8647e+0 7.2000e+01 6.0408e+ 1 1.9755e+0.: 7.3000e+01 .4000e+ 1 2.0930e+03 7.4000e+ 1 6.7806e+01 2.2175e+03 7.5000 e+01 7.183Se+01 2.3493e+03 7. 000e+ 1 7. 109e+ 1 2.4890e+03 7.7000e+ 1 8.0635e+01 2. 370e+03 7.8000e+01 8.5430e+01 2.7938e+03 7.9000 e+01 9.0510e+01 2. 0 e+ 3 8. OO e+O 1 9.5892e+01 3.1360e+03 8.1000 e+01 1.0159S+02 3.3224 e +03 8.2000e+01 1.0763e+02 3.5200e+03 8.3000e+ 1 1.1404e+02 3.7293e+03 8.4000e+ 1 1.2082e+02 3.951 le+03 . 0 0. 1667 0. 8056 0 0. 3333 0. 8333 0 0. 5000 0. 8611 0.1667 0. 5000 0. 8889 d") 0.3333 0. 5000 0. 7500 0.5000 0. 5000 0. 6111 0.5000 0. 5000 0. 4722 0.5000 0. 3333 0. 5000 0.5000 0. 1944 0. 5000 0.4167 0. 0556 0. 5000 0.2500 0. 0833 0. 5000 0 0. 1111 0. 9167 0 0. 3056 1. 0000 0 0. 5000 1. 0000 0 0. 6944 1. 0000 0.1667 0. 7222 1. 0000 0.3333 0. 7500 0. 8333 0.5000 0. 7778 0. 6667 0.5000 0. 8056 0. 5000 0.5000 0. 6667 0. 5000 0.5000 0. 5278 0. 5000 0.4167 0. 3889 0. 5000 0.2500 0. 4167 0. 5000 0 0. 4444 0. 9167 0 0. 6389 1. 0000 0 0. 8333 1. 0000 0.0278 1. 0000 1. 0000 0.2222 1. 0000 0. 9722 0.4167 1. 0000 0. 7778 0.6111 1. 0000 0. 5833 0.6389 1. 0000 0. 3889 0.6667 0. 8333 0. 3611 0.6944 0. 6667 0. 3333 0.6389 0. 5000 0. 3056 0.5000 0. 5000 0. 2778 0.2778 0. 5000 0. 6667 0.3056 0. 6667 0. Ί ? 9 0.3333 0. 8333 0. 6944 0.3611 1. 0000 0. 6667 0.5556 1. 0000 0. 6389 0.7500 1. 0000 0. 4444 0.9444 1. 0000 0. 2500 0.9722 1. 0000 0. 0556 1.0000 0. 8333 0. 0278 1.0000 0. 6667 0 0.9167 0. 4722 0 0.5000 0. . 0.5000 0.6944 0.5000 0.5000 0.8333 0.5000 0.6667 0.8056 0.5000 0.8333 0.7778 0.3333 1.0000 0.7500 0.1667 1.0000 0.7222 0 1.0000 0.5278 0 1.0000 0.3333 0 0.9167 0.1389 0 0.7500 0.1111 0 0.5000 0.0833 0.4167 0.5000 0.2222 0.5000 0.5000 0.3611 0.5000 0.5000 0.5000 0.5000 0.6389 0.5000 0.5000 0.7778 0.5000 0.3333 0.9167 0.5000 0.1667 0.8889 0.5000 0 0.8611 0.3333 0 0.8333 0.1667 0 0.7222 0 0 0.5278 0 0

Claims (13)

Claims

1. Frequency processing apparatus comprising a signal input device for receiving a dynamic signal, a wavelet table for storing a base of sample wavelets, a correlating unit for correlating input frequencies with the sample wavelets stored in the table to produce a matrix of wavelet coefficients elements, and an output device for producing a dynamic graphical representation of said elements.

2. Frequency processing apparatus according to claim 1 wherein the sample wavelets are at frequency intervals on a logarithmic scale.

3. Frequency processing apparatus according to claim 1 wherein the sample wavelets are at dyadic intervals.

4. Frequency processing apparatus according to claim 1 wherein the sample wavelets are the notes of the musical scale, each note being characterized by belonging to an octave and having a position within said octave.

5. Frequency processing apparatus according to claim 1 wherein the sample wavelets are elements of speech.

6. Frequency processing apparatus according to claim 4, further comprising image construction apparatus operative to translate elements of the matrix into positions on a two-dimensional surface in accordance with their frequencies and timing.

7. Frequency processing apparatus according to claim 4, further comprising image construction apparatus operative to translate elements of the matrix into positions in a three-dimensional volume in accordance with their frequencies and timing.

8. Frequency processing apparatus according to either of claims 6 and 7 according to claim 4, the image construction apparatus further operative to assign a first color to a note in accordance with its position within its octave and a second color in accordance with its octave, the two colors being mixable to produce a color for the note.

9. Frequency processing apparatus according to claim 6, 7, or 8„ further comprising brightness control, said brightness control being operative to assign a brightness level to each note in accordance with the intensity of the note.

10. Frequency processing apparatus according to any preceding claim wherein the dynamic graphical representation is a substantially real time representation.

11. 1 1. Display apparatus when attached to the frequency processing apparatus of claim 1.

12. Display apparatus according to claim 11, being one member of a group comprising a flat panel display, a projector and screen, a holographic display device, a plasma display, an HGED display, and a cathode ray tube.

13. Frequency processing apparatus substantially as hereinbefore described with reference to the accompanying drawings. Yakov Gugenheim Inventor Madar 13/13 , Rehovot Israel