NO843858L - SIGNAL TREATMENT AND SYNTHETIZATION PROCEDURE AND DEVICE - Google Patents

Description

This invention relates to the processing and synthesizing of complex signals, and more particularly it relates to a new method and device for processing and identifying the components of a complex signal and for synthesizing complex signals in which signal components are identified expressed by discrete locations in an either multidimensional representation, derived from the measurement and/or generation of at least the time period or duration and attenuation of the signal components that make up the complex waveform being processed or generated.

The analysis and synthesis of complex waveforms is well developed and used in a large number of areas. E.g. the analysis of acoustic signals having complex waveforms produced by one or more separate sources is well known. In the same way, analyzing signal waveforms of electrical signals, seismic signals and other signals is well known for use in speech processing, environmental sensing, biomedical signal analysis and the like. Equivalent systems are known to be used for generating complex waveforms in e.g. areas such as music synthesis, production of artificial speech and the like.

In the present known acoustic systems, it is possible to obtain the desired information only from simple waveforms and from single source objects such as a single loudspeaker against a low background noise. The influence of factors such as amplitude, dynamic range, pronunciation and the gender of the speaker will cause complexity and unacceptable results in speech analysis equipment. Similar problems exist in waveform analysis of signals produced by sources other than the human voice. These systems that exist are relatively expensive and use significant input energy during operation due to their complex structure and complex processing algorithms.

In reality, all existing complex signal analysis systems are based on some form of Fourier or spectral analysis or, more generally, on the application of least squares calculations of polynomial functions. Such methods have a common disadvantage in that important information is lost during the processing sequence due to the need for a relatively long measurement interval which is necessary to estimate or calculate the polynomial coefficients of a Fourier series. Since the measurement interval must be relatively long, the measurement process loses information that is related to the "time section" that relates to the time at which a new individual signal within the waveform has started. Because of this limitation, the Fourier type analysis technique cannot identify multiple source objects appearing simultaneously within the observation interval or the measurement of a complex waveform produced by multiple objects.

If the interval for calculating the polynomial coefficients is reduced in a spectral analysis system, in order to improve the time section accuracy, the accuracy of the mean square calculation of the source function is reduced. Consequently, with Fourier analysis the spectrum will become more and more unclear as the selection window is reduced in time.

The difficulties with the existing systems for analyzing stochastic signals are summed up by a known time-frequency uncertainty which specifies that accuracy in a frequency range requires low resolution in the time range and vice versa. This is described e.g. in the article by D. Gabor, entitled "Theory of Communication", Journal of IEEE, vol. 39, Part III, No. 26, November, 1946. The problem mentioned above is also described by S.M. Kay and S.L. Marple, "Spectrum Analysis - A Modern Perspective", IEEE-Proceedings, vol. 69, November, 1981, pp. 1380-1419 and by H. Bremerman, "Pattern Recognition, Functions and Entropy", IEEE, Transactions on Biomedical Engineering, vol. BME-15, No. 3, July, 1968.

It is interesting to note that the human ear has solved the waveform analysis problem, although the operation of the ear in this analysis function is not understood. In older theoretical analyzes of the workings of the ear, it was assumed that the hearing process uses spectral processing. Thus, an early theoretical model of hearing was made by Von Helmholtz in 1863. The Von Helmholtz model was based on a Fourier or frequency domain signal analysis. Both Von Helmholtz and his successors concentrated on studying well-defined and relatively simple signal states, especially with only one signal at a time. These simplifications have provided a reasonable basis for a theoretical study, but we ignore how the ear processes frequency discrimination over a wide logarithmic range, how it reacts to transients and its ability to detect and identify sounds over a wide amplitude and dynamic range range of near-optimal detection sensitivity and how it can discriminate multiple sources in space and temporarily even when they are separated or interleaved with large amplitude ranges. Acoustic sources can also be identified by the ear despite background noise and extreme disturbance of amplitude and phase. This is the well-known "cocktail party effect" whereby the human ear can pick out the voice of an invisible speaker from a background of other voices of equal or higher amplitude. The application of Fourier analysis type techniques cannot explain many of the functions of the human ear herein including the functions described above expressed by a consistent signal processing theory.

Signal processing methods and devices currently available are extremely complex and expensive, although their functions do not come close to the efficiency and wide range of applications of the human ear. The present invention provides for signal processing that more closely approximates the results obtained by the human ear and that is more complex and

less expensive than currently available equipment.

The present invention provides a new signal processing method and an equivalent signal generation method for the analysis and generation of complex waveforms, and uses a non-spectral analysis method and can be performed with reliable and relatively inexpensive processing circuits.

In accordance with the invention, the complex waveform is analyzed by performing sequential measurements at either one or both the real and complex zero points of the waveform. The directly measurable parameters at the real and complex zero point of the waveform are the waveform energy level, the time period or the start of new wave fronts and derivatives of the waveform. Direction to arrival can also be measured. These parameters are called "waveform information vectors" and are also sometimes hereinafter referred to as "WIVs". These WIVs are then processed using relatively simple digital or analog circuitry to derive the measured information that characterizes and locates each signal source object within the collection of arriving waveforms. The primary identification device lies in the formation of the pattern in the vector space formed by the coordinates of periodicity versus residue versus amplitude. This vector space is called "Aural Retina" because of its analogy with the ear and the hearing process. It has been shown that human listeners identify sounds expressed by pitch and timbre. In particular, the pitch is an expression of the periodicity in the time range, and the timbre is an expression of instantaneous harmonic content, or residue (rest) in the frequency range. This relationship between pitch and periodicity and timbre and residue allows a special definition of pattern space inside the ear and the hearing process within which the objects of aural perception are analogous to the objects of visual perception. In other words, objects are defined as sets of points in the coordinates of periodicity, residue and amplitude of the aural retina analogous to objects received in the visual or visual retina as expressions of height, width and brightness.

To perform this function, WIVs are extracted at the time of each real or complex zero point of the waveform and compared to a set of previous patterns to derive time slice and waveform information to the individual source of objects that produced the entire waveform. By following each subsequent analysis interval, additional information regarding the waveform can be made available to continuously improve the analysis process.

Thus, with the present invention, acoustic sounds can be described with the help of a pattern in a two-dimensional space by plotting periodicity (pitch or pitch) as a function of residue (timbre).

Spatial or directional information between sources that give a given complex waveform at a sensor is determined by using spatially directed channels that measure the wave from a given direction. Aural retina measurements taken in the different directions are then compared and the signals of the multiple sources are separated and individually identified.

In an analogous way, the new process and device according to the invention can be used to synthesize complex waveforms such as synthesizing the human voice or the like with relatively simple equipment.

The particular processing circuit used in a given application depends on the frequency range of the waveform. For frequency ranges from seismic frequencies up to ultrasonic frequencies, standard LSI and VLSI technologies and components can thus be used. In this frequency range, the waveform processing can be used for such applications as oil exploration, underwater submarine detection and tracking, speech instruction and hearing research, motor-driven machine trimming and other acoustic analyses, assessment of musical instruments, bioelectrical monitoring, speech recognition and the like.

The invention can also be used at higher frequencies which can be in the far ultrasonic band up to the radio band and up to 200 MHz for use in radio detection in the broadcast band, interference detection and localization in the radio band and the like. The invention can also be used for detection in even higher frequency ranges up to 1000 MHz, where the circuits are preferably made with surface acoustic wave devices for use in spread spectrum communication systems and the like.

The theoretical basis for the new aural retina design in the respective to the present invention will now be described in connection with the processing of an existing complex waveform as opposed to the generation of the complex waveform which will be described later. Any complex waveform produced by the overlapping reactions of a plurality of objects, each of which represents a respective source of information, can be described in the following way:

where

A-^= k the amplitude echo coefficient;

S-^= k complex frequency (+ je^)

t^= time period (start) of the k excitation function.

It is assumed in equation (1) that t >_ t-^. The function f(t) is 0 when t < t^.

As expressed in equation (1), the composite waveform f(t) is the sum of a sequence of exponential small waves of different amplitudes, acting as reaction or response functions to excitations occurring at different time periods (or instants). This process is shown schematically in fig. 1. Thus, fig. 1 schematically on its left side n excitation sources 20-23 each of which produces a pulse-type excitation at a time t. The sources 20-23 can have any relative spatial localization and any desired size. When the objects are stimulated by the unit pulse excitation pulse

<6>(<T>n), their response signals may have any desired form, as shown by the various waveforms above blocks 24-27 in which the waves are oscillating waves or critically damped waves, or the like, illustrated by the values A- |ch(T^). The response signals of each of the objects 20-23 can be stated generally as

where 1-^k represents the location.

The output signals from the blocks 24-27 are summed in a summing device 28 which gives the output f(t) which is the function of equation 1 above. Note that the complex waveform shown in fig. 1 contains the time sections t^, t2, t3 and t of the respective generators 20-23.

From the above it appears that the waveform emitted by each object contains potential information about spatial and temporal patterns for each object. To analyze the function shown in fig. 1, the signal analysis system according to the invention has the ability to sort, compare and associate the measurements of time section, amplitude and response function for each of the sources 20-23 with a high degree of analysis reliability. Analysis reliability refers to the amount of recoverable information in relation to the available source information. The signal sources involved are normally characterized by random and unpredictable parameters. Thus, waveforms produced by environmental and biological sensors are at best random, stochastic or transient. E.g. will human speech and various animal sounds of highly variable periodic components mixed with random and transient events. In some cases, many signal sources will coexist in time and space.

Because of this randomness and the variation possibilities, the signal analyzer according to the invention must extract information within very small time intervals. Otherwise, the ability to associate the time slice information of the objects 20-23 with the corresponding system response to the system response blocks 24-27 respectively will be lost. If this information is lost, there is no way to identify the components of a source. Note that this problem is also related to the ability to detect a signal in a background of noise or other signals.

From the above, it is obvious that a fundamental requirement for the analysis system is that the system must identify the time period x k as accurately as possible. The time section tk in equation 1 is the pulse that produces the individual response function to a small wave, where small waves can extend from an overdamped case whereby a >> 1 to the oscillation case where 0 _< a 1, where a is the real part of the complex frequency a + je.

In the oscillation state, the time interval is the fitting of a periodic function at the complex frequency a ^j^. In a complex signal, the sum of the different small waves occurring in different time periods will cause the different peaks and zero points in the waveform.

Therefore, each ripple in the waveform of FIG. 1 a property resulting from either an excitation pulse or an oscillatory response. These properties can be represented as real and complex zero points in the time domain as described, e.g. in the publication of H.B. Voelcker, "Toward a Uniform Theory of Modulation" Part I: Phase-Envelope Relationships, IEEE Proceedings, Volume 54, March 1966, pp. 340-353 and Part II, Zero Manipulation, IEEE Proceedings, May, 1966, pp. 735- 755. See also US Patent No. 3,510,640 issued to H.B. Voelcker, entitled "Method and Apparatus for Interpolation and Conversion of Signals Specified by Real and Complex Zeros".

In accordance with the invention, a device is provided to identify and measure the complex zero points in a complex waveform in order to thereby be able to measure both the time period and the period of the exponential small waves that make up the complex waveform.

Since any individual zero point in the complex waveform can be either the start of a new small wave or a response from a previous small wave, a facility is provided to measure and store the information at each zero point so that this can be related to previous and /or subsequent data. Directly measurable parameters at the zero points are energy level A, time period, and derivatives or derived values. The direction of the arrival of the energy is also an instantaneous parameter and is determined by comparing the instantaneous energy and time sections of spatially separated sensors. As previously mentioned, these parameters measured at the zero points are referred to as waveform information vectors or "WIVs".

The derivative or velocity change vector is useful after it has been normalized for absolute signal strength. This is done by dividing the derivative f'(t) of equation (1) with respect to f(t) to produce the amplitude-independent time derivative of the logarithm of f(t). Thus is

In equation (3), the function r(t) can be interpreted as the instantaneous "residue" of the spectral pattern of the signal f(t). Consequently, the function r(t) is often in what follows appropriately named the "residual function". The function r(t) also represents the residuals of the responses from past and present exponential functions in the signal f(t).

It is instructive to consider the case where f(t) is a simple exponential small wave of the form

f(t) = Ae "at sin Bt (4)

In equation (4), the expression a is the holding time of the exponential small wave, while the expression 6 is the ringing frequency. Note that these are also the real and complex components of the complex frequency f-^in equation (1). By taking the derivative of f(t) in fig. 4 and then dividing this by f(t) in equation (4), the residue function r(t) is produced in the following way

This simply reduces to:

The residual function as it is represented in equation (6) can be interpreted in many ways. The purpose, in accordance with the present invention, is to identify a function that characterizes the exponential coefficient ot in selected real or complex zero points f(t) or to r(t). The zero points of f'(t) are the same as the zero points of r(t). The advantage of sampling the function r(t) at its zero points is that the discontinuities of the function caused by the real zero points in f(t) can be avoided. This process will then lead to extremely simple circuitry to extract the desired information.

A number of methods are possible to achieve this result and two are described below:

In the first method, only the real zero points of the function r(t) in equation (6) are considered. It will be seen that when r(t) = 0, then:

As before, the imaginary frequency B defines the ringing frequency of the simple exponential wavelet under consideration, while a defines the real axis frequency and specifies the damping or fall time of the exponential function. The purpose of the electronic system will then be to determine ° and ^ using the information derived from the zero points of r(t). Both a and B are first measured as functions of the zero points of r(t) using conventional measuring devices. The residual wave that forms r(t) is then processed to produce either positive or negative half-periods and the time period of the ringing (assumed that a is less than 6 ) is then measured. This period is called T ^. An appropriate way to obtain the quantity a is to measure the period between the leading edge and the peak of the waveform so that

The parameter<«> then becomes the definition of residue. When the period Ta described above is equal to Tg /4 it corresponds to the duration of a quarter period of a pure sine wave, which then, at any frequency, gives the reference zero-point bandwidth IBW or, in the opposite case, the maximum possible rise time. Therefore, the measurable parameter Ta is related to the residual r(t) by equation (6) above. Note that T a is derived from the logarithmically derived waveform which is not variable with the signal amplitude, thereby removing certain practical problems from the system design.

The relationship between T a and Tg is shown in fig. 2 for values of a.

In fig. 2, the value Ta is the timbre equivalent, while the quantity l/Tg is the periodicity or pitch of the signal for different amounts of residue a in the signal.

The diagram in fig. 2 forms the basis for the Aural Retina concept according to the invention. Thus elementary signal functions are instantly identified by their point positions within the existence diagram bounded by a = 0 for a sine wave signal state. Note in fig. 2 that no signal can exist to the left of a= 0. Between the values of the residue from a = 0 to a =Tr/4Ta (a critically damped line), the signals will oscillate and they will thus have separable periodicities. A simple pulse response would appear in fig. 2 as a single point. Repetitions of pulse responses will produce similar patterns of points that depend on the interference pattern of the excitation period with the ring period.

In the overdamped region to the right of<«>= ^/4Ta, single pulses will appear on the zero pulse repetition frequency axis at the measured rise time. Note that the rise time is essentially the composition of all the exponential functions that occur at a given pulse section. However, a train of periodic pulses will occur at a point given by its rise time Ta and its repetition rate Tg.

Noise pulses will tend to propagate through the plane of existence in fig. 2, and with its density concentrated near the periodicity of the upper bandwidth cutoff of the system.

The above procedure has assumed that T g, the period to the ringing frequency, can be measured. In this elementary example, the measurement of Tj does not present any problem, but in general the measurement of Tg requires a special measurement system which will be described later.

Above is described a type of Aural Retina which can be used in accordance with the invention and which depends on the measurement at real zero points in the function r(t). It is also possible to derive the residual function r(t) at the complex zero points of the function since the complex zero points would correspond to the peak values of f(t). Thus, it will be seen from equation (6) that:

Of the above, when f(t) = sin 8 t = 1 (a peak), r(t) = -a. In general, r(t), when this is sampled at the peaks of f(t) using the measuring equipment, will produce a measure that is related to the value a.

By using this type of measurement, residual a will provide an instantaneous estimate of the waveform's rate of change or instantaneous bandwidth. This method will then define a rectangular existence area instead of the tri-angular plane existence area in fig. 2.

It should be noted that the pattern derived in fig. 2 for Aural Retina is similar to an existence diagram derived by Schouten on the basis of psycho-acoustic experiments on humans as described in the publication by R. Plomp and G.F. Smoorenburg, Editors, "Frequency Analysis and Periodicity Detection in Hearing", Lydon: A.W. Sijthoff, 1970. Thus, there is a correlation between the above mathematical analysis based on the availability of waveform information vectors in an exponential waveform and the actual measured response of the ear as shown in fig. 2A which is a reproduction of Schouten's existence diagram.

The diagram in fig. which represents the Aural Retina of the present invention will have the various instantaneous WIV data imprinted on it, where WIV data can be relatively easily measured. It will later be shown how WIV samples or samples will be processed to derive the desired information relating to the source objects of the complex waveform.

The term "Aural Retina" has been chosen because of the functional similarity between the spatial object analysis of the visual retina and the analyzes of source objects in the space defined by the waveform information vectors. This is shown in fig. 3a and 3b which schematically show a visual retina and an aural retina respectively. In fig. 3a, the visual retina will reveal object shapes in a plane formed by a height and a width axis. A third dimension to the visual retina can be represented by the lightness of the object shapes. In the aural retina according to the invention, object shapes can be thought of as lying in the plane of the aural retina in fig. 2 and related to pitches and timbres, as the third dimension of the aural retina can be loudness if the waveform being analyzed is a sound wave.

Signal detection in the Aural Retina, which is like in the eye related to two functions: (1) the sensing of an object pattern expressed by means of an energy threshold; and (2) the selection of a specified object pattern from a background of other patterns. In the aural retina of fig. 3b are the primary source object patterns shown in the existence space delimited by residue and periodicity, as in fig. 1 where the patterns represent the signal identification. Acoustic sounds are described in a similar way in the ear by means of pitch and timbre as shown by means of the existence diagram in fig. 2a which was recorded on the basis of psychoacoustic experiments carried out on humans. In prior art devices where Fourier analysis type processing was applied to complex signals, it will be noted that the same signal can be represented by Aural Retina with lower dimensions than is possible by using the necessary high number of Fourier coefficients. The lower dimension allows the use of faster and cheaper computer to process the signal as will be described, and requires less memory capacity than the spectral analysis technique.

An essential characteristic of the invention is that the method described above is reversible. Thus, waveforms with random content can be synthesized by means of appropriate programming of the time sections and other waveform information vectors in the signal generation circuits.

The invention is specified in more detail in the attached patent claims.

The invention will now be described with reference to the drawings, where: Fig. 1 is a schematic drawing illustrating a plurality of signal sources and their summed response functions. Fig. 2 schematically illustrates a two-dimensional plane of existence in which specific signals can be presented in accordance with the invention. Fig. 2A is a diagram produced by psychoacoustic experiments on the performance of the human ear. Figs. 3A and 3B compare the characteristics of the visual retina and the aural retina according to the invention, as shown in Figs. 2. Fig. 4 schematically illustrates analysis intervals used to process a complex waveform in accordance with the invention. Fig. 5 is a functional block core of a system for

to perform the analysis intervals on fig. 4.

Fig. 6 is a block core of a preferred circuit arrangement constructed in accordance with the invention. Fig. 7 is a block core of an analog residue processor that can be used in fig. 6 and in connection with the present invention. Fig. 8a to 8f show a digital residue processor that can be used instead of the analog system in Fig. 7. Fig. 9A schematically illustrates an interleaved pulse train of overdamped small waves to be analyzed. Fig. 9B schematically illustrates a waveform to be analyzed which has a periodic excitation with an underdamped response function. Fig. 10A schematically illustrates an input signal which

must be processed.

Fig. 10B shows the clock signal produced from the signal of Fig. 10A. Fig. 11 schematically illustrates a logic map of a new periodicity deinterleaving system for deinterleaving the signal train of fig. 10B. Fig. 11A schematically illustrates the reasons for false recognition of a pulse in higher octaves of a plurality of octaves. Fig. 12 is a logic map of a periodicity sorting matrix for sorting a plurality of octaves. Fig. 13 is a logic diagram of a simplified periodicity sorting matrix that can be used with the circuit of the present invention. Fig. 14 is a block diagram of a quasi-analog processor arrangement made in accordance with the present invention. Fig. 15 is a segmented existence diagram useful for describing the outer product of the periodicity and residual which is measured in accordance with the invention and displayed in an Aural Retina type-

showcases.

Fig. 16 is a block diagram of an aural retina processing and automatic signal recognition circuit that can be manufactured in accordance with the invention. Fig. 17 shows an alternative method for recognizing the signal pattern which does not require the RAM memory of fig. 16.

Fig. 18 schematically illustrates a logic map for a

simple octave time division detector circuit.

Fig. 18a schematically illustrates the manner in which fig. 18 can be extended

to a multi-octave system.

Fig. 19 is a block diagram of a waveform synthesizer

carried out in accordance with the invention.

Fig. 20 is a diagram of a zero location matrix

used in the waveform synthesizer of fig. 19.

Fig. 21 is a timing diagram for the zero location matrix on

fig. 20.

Fig. 22 illustrates a method for residual modulation in a synthesizer arrangement. Fig. 23 is a typical signal waveform to be analyzed. Fig. 24 shows the spectrum analysis of the signal in fig. 23. Fig. 25 is the waveform of fig. 23 together with pulses representative of various aspects of the waveform and the Aural Retina display corresponding to the waveform of FIG. 25. Fig. 26 shows a second waveform added to the waveform

on fig. 25.

Fig. 27 shows the spectrum analysis of the combined waveforms of fig. 26. Fig. 28 shows the Aural Retina display corresponding to the combined waveforms of Fig. 26.

Reference is now made to the drawings of which fig. 1, 2a, 3a and 3b have just been referred to. A block diagram of a circuit for analyzing WIV data sensed by at least two different sensors is shown in fig. 4. In general, the analysis process according to the invention, as shown in fig. 4, the storage of each WIV as it is received. WIV is then compared with previous WIV. Decisions are then made regarding their understanding and action is taken where necessary. The analysis can be carried out in at least five analysis intervals. In fig. 4, a right sensor 30 and a left sensor 31 are shown, each of which may be of any desired type, and each of which receives waveform energy from one or more energy sources. The sensors 30 and 31 are connected to WIV data measurement circuits 32 and 33 respectively, which will be described later, which are able to extract from the received signals information regarding time period t , energy A and residue R. By appropriate processing of the data the direction from which the signal came can also be determined. Note that larger numbers of separate sensors can be used to locate a source object within a large angle such as by using the trilateration process described in applicant's prior US Patent 3,886,553, entitled "Coordinate Locating Method and System", issued May 27, 1975 However, the direction vector is not always necessary to identify the source object or its message. Note also that in this measurement the time period T is the time at which the signal is received, the energy A is the instantaneous level of the energy and the residual

R corresponds to residue r(t) in equation (8).

The five analysis intervals shown in fig. 4 is defined by the limits of the information available in the collected WIV data. At the moment data is received, the sample or sample at each waveform's zero point will represent the first interval. The arrival angle of the information can be immediately determined by comparing the amplitude and/or time section in the sensors 30 and 31.

The information in the first interval from the measuring devices 32 and 33 is processed to derive the spatial WIV information in the processing block 34. The immediately received information can be "flash" information which can provide information with regard to the clarity of the received signal . Thus, in the first interval, the system can feed rudimentary or flash answers to simple questions regarding the meaning of each WIV event. For example, if the question is: "Is the signal representative of a dangerous condition?", then an amplitude threshold, rate of change or direction can be established to specify an immediate response if danger is detected. Thus, it is not necessary in the first interval to identify the source, but only its potential threat.

In the second analysis interval, circuits are arranged to extract the periodicity (T^) and to compare time sections of the instantaneous signal with a previous signal. This circuit is shown in block 35 in fig. 4. If in the second processing interval events are observed that have equal time intervals, it can be determined that periodic signal components are present in the composition of f(t). Such components can be caused by excitation functions, underdamped system response or simply by random phenomena in a stochastic waveform. However, periodic excitation components will repeat continuously. The second interval adds periodicity information to the WIV data, but only over a range of three instantaneous WIV events, which is the minimum to determine a periodic event. This added information usually does not add sufficient data to allow identification of a source function so that the response effect will be the same as for the first interval and is identified simply as more "flash" in Fig. 4.

In the third processing interval, the WIV data together with added periodicity is averaged over a short time interval within the processing circuit 36. The short time interval is chosen to provide a reasonable estimate of the source objects in the waveform received by the sensors 30 and 31. This can be considered a "snapshot" of the source that allows the identification of some aspects concerning the nature of the source such as who, what, where and when qualities.

The fourth analysis interval has circuit 37 adapted to examine the subsequent snapshots from the third interval to determine variations in time and space in the source objects. This is analogous to the successive production of individual images that make up a film of moving images. The impact of the additional analysis in the fourth analysis interval will provide answers to questions such as: "What do the objects do?".

In the fifth analysis interval, circuit 38 is arranged which allows the analysis of syntactic patterns which can answer a question such as: "What do the source objects say?". There may also be subsequent analysis intervals which will be discussed later.

The analysis interval sequence shown in fig. 4 represents a basic procedure for extracting information from any source according to the invention. This procedure can be used to design a desirable system to provide the specified results. Thus, the analysis intervals and their circuits 34 to 38 can be arranged as necessary depending on the specific purpose of the system. Thus, the circuit blocks 32 to 38 can be designed for speech processing, environmental sensing, biomedical signal analysis and for processing other related signals. Note that the system that uses a kind of Fourier analysis or, more generally, a least squares estimation of polynomial functions, cannot use the analysis sequence shown in Fig. 4.

Fig. 5 is a block diagram showing the functional data flow in the system according to the invention. The system of fig. 5 uses the same sequence with the analysis interval referred to in FIG. 4. Facilities are provided to first identify WIV at signal waveform nulls. These zeros can be the real zero crossings, the complex zero points, or both real and complex zero points. In the most general case, which is shown in fig. 5, there are two sets of WIVs to show the importance of determining the direction of arrival at each selection zero. Thus, there is an Aural Retina for each side of the system and a combined spatial Aural Retina in which the left and right WIV are compared to determine directional parameters.

The signals from the transducers 30 and 31 in fig. 5 are first sampled and processed by logarithmic analog-to-digital converters 40 and 41 respectively, which convert the received waveforms into logarithmic amplitude coded digital data. This is done to allow the sampling of a larger amplitude dynamic range in the instantaneous variation of the waveform than would have been possible with standard automatic gain control methods. The logarithm functions are also needed to produce the logarithmic derivative from which the residue is derived. The peak amplitude, as a digital value, directly forms an element in the WIV space. Digital differentiators 42 and 43 respectively produce the real and complex zero points of the signal as their outputs, which outputs are processed to provide the residue vector of the WIV.

Thus, in the first analysis interval in fig. 5 an amplitude output from the logarithmic analog-to-digital converters 40 and 41 and a residual output from the digital differentiators 42 and 43. Information regarding the zero points carrying time slice information is also connected to respective periodicity sorting matrices (PSM) 44 and 45, the details of which will be described later. Periodicity sorting takes place during the second analysis interval in fig. 4 and 5 by comparing the time interval between zero points in such a way that all periodic components are detected and digitized.

Aural Retina circuits 46 and 47 are then provided for the right and left sides of the system respectively, and constitute circuits for performing the first estimates of the waveform content received by transducers 30 and 31 respectively.

Since the aural retinas 46 and 47 represent multidimensional space, if visual display is required, appropriate display devices can be driven by the Aural Retina circuits 46 and 47. The visual interpretation or display of the vectors can be performed in different ways and the method chosen is dependent on the information of interest. It is generally not necessary to observe all the vectors at the same time. In particular, the primary vectors are contained in the plane of existence bounded by residue and periodicity which are the two parameters present in the main waveform identification information, as shown in fig. 2. This allows the observer to receive "flash" data quickly. This means that he can understand the pattern that has particularly important immediate or momentary content.

In fig. 5, a spacious Aural Retina comparison circuit 48 is also arranged which receives all the information fed into the Aural Retina circuits 46 and 47, but makes comparisons between the left transducer vectors and the right transducer vectors. The comparison circuit 48 may use conventional amplitude comparison and/or time-difference direction finding circuits such as those used in radio direction finding applications. The observation of amplitude and direction is useful in certain operating situations, but is most useful in providing a signal object sorting function which, analogous to the binaural guidance, allows listeners to ignore disturbances and other sounds.

Although the Aural Retina circuits 46 and 47 perform signal analysis, the circuits are not used to perform the main decisions about the contents of the collection. The analogy between the visual and aural retina in fig. 3A and 3B are suitable for this purpose. Thus, the Aural Retina can essentially be considered the storage location for spatial patterns and only subsequent recognition processes can identify the patterns and interpret their meaning.

The third analysis interval in fig. 5 somehow relates to the interval in fig. 4 the storage and averaging of the raw data of each arriving WIV. This treatment is carried out to set a smoothed WIV which can be observed visually in selected situations and which can then be stored and recalled as desired. In the third interval display, a circuit 50 is arranged for WIV storage and averaging to obtain composite fine-resolution data. Suitable microprocessors can be used in the circuit 50>. A stock CRT can also be used for storage and averaging.

Since the patterns are now under the control of the system, "templates" of desired WIVs can be provided to enable automatic recognition of different input signal situations. Thus, a source of stored signal patterns 51 can be arranged which can be applied to an appropriate comparator 52 which will determine whether the received WIV in the storage 50 correspond to predetermined signal patterns in the system 51. If a "hit" (hit) is determined, can circuit 52 produces an appropriate decision or output signal related to the signal waveform content that has been processed in circuit 50.

In the Aural Retina concept, this function is equivalent to signal detection. Note that unlike the conventional method of frequency-selective energy detection, no decision is made to reject or select energy during the process. Thus, when a signal is detected by a recognition decision, it does not necessarily remove the WIV pattern from another signal. The arbitrary and simultaneous detection by recognition of a wide range of interleaved signal patterns can thus be performed and the adaptive selection process is easily completed.

In the fourth interval in fig. 5, circuitry is provided for comparing smoothed vectors, which circuitry is shown as circuit 53. In this circuit 53, compositions of smoothed WIVs containing certain common characteristics are compared and variation patterns are identified. This process can e.g. recognize diphthongs (pitch or format variation) in speech. Circuit 54 may be arranged to produce output signals related to decisions regarding variable patterns. Circuit 53 can also be used to drive a projector such as can be of the type with strip drawing.

Relatively few signal sources other than human speech will require more than three or four processing intervals as shown in fig. 5. However, other steps are possible and e.g. a fifth processing interval can be used which interval contains circuit 55 for normalization for automatic recognition of a voice and for speech processing. Circuit 55 could also be used, e.g. for sonar target recognition if desired. A stored template for a particular pattern variation is shown at circuit 56 and may be connected to block 54 for automatic recognition of a predetermined pattern variation.

Fig. 5 shows the many possibilities with the Aural Retina concept, and also shows that there will be many common characteristics among any number of special applications of the concept. Any suitable circuits can be used to perform the various functions shown in fig. 5.

A special Aural Retina device that can be used on fig. 5 is shown in fig. 6 in block diagram form for the channel in fig. 5 which extends from the signal sensor 30. The circuit which extends from the signal sensor 31 in fig. 5 will essentially be identical to the circuit that will be described in connection with fig. 6.

The purpose of the system of fig. 6 is to perform the calculations indicated by equations (8) and (9). The general procedure performed by the block diagram of FIG. 6 will be to transform the signal waveform into a unipolar logarithmic form from which the amplitude, residue and time period can be found. It may be desirable in certain applications, such as when processing seismic signals used in geophysics, to analyze both positive and negative polarities. This will require a double LAD. Output signals from the double LAD will be treated identically and then brought together where appropriate into the PSM for periodicity analysis. The periodicities are then measured and linked together with the WIV data in a short-term storage device. This interim storage allows the accumulation of recurring data patterns that characterize individual signals. These patterns are then periodically scanned into a microprocessor-controlled warehouse where smoothing, display and automatic signal recognition are carried out. This process completes the functions of the first three treatment intervals of fig. 5. Subsequent processing in four or more intervals will be performed primarily in conventionally designed computer algorithms.

In fig. 6 are components that are identical to the components in fig. 5 given identical reference numbers. It will be seen that the sensor signal 30 is first applied to a preamplifier 60 and then to the logarithmic analog-to-digital converter 40. A conventional clock 61 is connected to the converter 40, residue processor 62, periodicity sorting matrix 45 and time slice frame circuit 63. After amplification and digital conversion, the residue processor 62 will , which is part of the digital differentiator 42 in fig. 5, extract and encode the amplitude and residual information as well as the real and complex zero points containing the time slice data. The zero information and the amplitude information are applied to circuits 64 and 65 respectively and the residue information is applied to circuit 66. The zero information of circuit 64 is then applied to the periodicity sorting matrix 45 and the output signals from circuits 45, 66 and 65 are applied to a WIV format circuit 67. Note that the periodicity sorting matrix 45 has analyzed the zero points and sorted the periodic sequences to identify their arrival times.

An outer product matrix circuit 68 receives WIV format information from circuit 67 and time slot frame information from circuit 63 and works to assemble the WIV for instantaneous identification. The outer product matrix circuit 68 then transfers each recognition event to a temporary storage circuit 69 where averaging and interpolation of the outer product matrix can be performed. Automatic recognition of selected source objects can also be performed at this level in the automatic source object recognition circuit 70. The output signal from the outer product matrix 68 is also applied to the visual Aural Retina display circuit 71 and information from the interpolation and averaging circuit 69 is also sent out to the Aural Retina forward -the pointer 72 which provides the smoothed information.

In general, the signal processing functions of the blocks shown in FIG. 6 is carried out by analogous methods. That is, they could work directly on the signal waveform as shown in fig. 7. With reference to fig. 7 which shows an analog residue processor, it is seen that the signal f(t) is connected to a rectified logarithmic amplifier 80. If the output signal is connected to a differentiator 81 having an appropriate RC time constant circuit. Output signals from the differentiator 81 are the logarithmic derivative of f(t) which contains the desired residual function r(t). Since this constant only appears at the peaks of f(t), it is only necessary to extract the logarithmic derivative at these moments by choosing the peaks of f(t). This is done by differentiating the function f(t) in the differentiator 82 and then the zero crossings are detected in the conventional zero crossing detector circuit 83. These zero crossings are then used to drive gate 84 and gate 85 to control the residual function and the logarithm of the amplitude respectively into the subsequent processing circuit only at the complex zero points. A zero crossing detector 86 is also connected to the output of the logarithm amplifier 80 in order to be able to apply the output logarithm f(t) only at the time real zero. Note that real zero points are detected from the edges of the logarithmic f(t) signal. Note also that the peaks in the signal correspond to first-order complex zero points and are used for further analysis.

While the analog circuit in fig. 7 is relatively simple in its structure, the logarithmic amplifier 80 and the differentiator 82 and the peak detector circuit are expensive. Furthermore, a digital processing circuit would lead directly to the output form necessary for subsequent processing by a digital microcomputer used for pattern recognition.

A logarithmic analog-to-digital converter showing a total block diagram showing the paths of the various digital time waveforms as they relate to the various system functions; a detailed logical design kernel for the three basic blocks or PC units of the present operating system; and time waveforms that show how the various functions are performed by digital processing.

Fig. 8a shows the various timing control lines derived from the base clock 90. There are three basic control areas separated by dashed lines: (1) the LAD selection switch 91; (2) the digital differentiator and data code circuits 92; and (3) the PSM data input and output 93 .

A time divider 94 provides a set of time pulse trains which are used to derive the appropriate durations of the control signals. The LAD switch pulse output by timing logic circuit 95 is designed so that its period coincides with the basic shift register clock inside PSM 93. The switch pulse width is chosen as small as possible in accordance with the RC charge and discharge requirements of resistor 96 and capacitor 97 in the switch circuit.

The LAD selection switch circuit 91 has a signal fed into it which is the signal f(t). This signal is applied to preamplifier 800 and an appropriate LAD switch circuit 801 which is connected through RC circuit 96-97 to comparator 802. The output signal from comparator 802 is then connected to an input to gate 803 and to an input to a real zero selection circuit 804 which will be described. The second input to real zero selection circuit 804 is derived from the LAD switch pulse and the second input to gate 803 is derived directly from the fast clock output of time divider 94. A control logic circuit 805 receives input signals from the divide by two output of time divider 94; from divide by 64 the output from the time logic circuit 95 and from the fast clock output from the time divider 94. The control logic output signals from circuit 805 are then connected to data counter 810, which also receives the output signal from gate 803, to complex zero point selection logic circuit 811 and to latch 812. The output signals from the data counter 810 are applied to the subtraction circuit 813 and latch 812 as shown and the output signals from the subtraction circuit 813 are applied to the residual latch 814 and the amplitude latch 815. The output signals from the complex zero point selection logic 811 are applied to the zero mode combination selection circuit 816 and the various circuits are connected to each other as shown. The output signal from the residual latch 814 consists of the residual function, while the output signal from the amplitude latch 815 is the logarithm output function. The PSM data input and readout section 93 comprises a readout scan select logic circuit 820 which receives the output signal from the fast clock readout scan clock and shift clock respectively, and its eight output lines (typically) are connected to the periodicity sort matrix 45. The periodicity sort matrix has an output connected to counter 821 as in in turn drives latch 822 which outputs data related to digital periodicity in the input signal. A polling pulse logic control circuit 826 is provided and receives an input signal from the shift clock and from the zero mode combination selector circuit 816, and outputs an output signal to the periodicity sort matrix 45.

Wave the worms in fig. 8b illustrates the operation of the LAD 91. The waveform f(t) is the output signal from the preamplifier 800 which feeds the switch 801 controlled by the switch pulse. At each switch pulse, the electronic switch circuit (usually a CMOS semiconductor chip such as the CD4066) is closed, causing the capacitor 97 to charge up to the level E of the signal f(t) at that instant. When the switch pulse is removed, the capacitors 97 discharge through the resistor 96. The discharge voltage appears at the input of the amplitude comparator 802 (an LM311 chip). Comparator 802, which was turned on by the charging voltage, remains on until the discharge falls below the threshold level ET. The threshold Erp is usually chosen as close to zero as possible to thereby obtain a large amplitude dynamic range. The duration of the comparator pulse is given by T = RC logE/ET, where RC is the time constant. To obtain maximum dynamic range, <E>m<a>x/ET and the value of RC are chosen so that T is less than the period of the switch pulse.

In fig. 8b shows a "cleaned" comparator pulse output where the switch pulse is used to cancel its own contribution to the width of the comparator signal and to eliminate spurious transients. From there, the comparator pulse as shown in fig. 8b used to drive fast clock pulses into data counter 801.

The cleaned pulse is also sent to the real zero selection logic 804, which detects when the shift clock interval in which the waveform f(t) goes from negative to positive. This clocked zero point pulse is shown in waveform (f) in fig. 8(b).

The controlled pulses of waveform (c) in fig. 8b is sent to data counter 810 which converts the sequence of pulses in each sample into a binary number. During the sequence of operations determined by the control logic for each output period, shown in FIG. 8(e), and these samples or samples representing the logarithm of the instantaneous amplitude of the waveform are subtracted, compared, latched (stored), and cleaned. In the subtraction operation, the stored two complementary numbers B of the preceding sample are added to the present number A in the binary counter 810. This operation is equivalent to a discrete derivative of the logarithm of the waveform f(t), and thus represents the logarithmic derivative. At the same time as this subtraction operation, the comparison decision which determines whether the current number A is greater or less than the previous number B is carried out. This decision is detected by sensing the state of the "sign bit" output from subtraction circuit 813 of the two's complementary differences. If there is a change in the sequence of decisions, this indicates the presence of a complex zero point in the form of a zero or a peak. This peak/zero decision is then converted by circuit 811 into a clocked pulse for analysis in the PSM. Different combinations of real and complex zero points can be selected in experimental situations in signal analysis.

After the subtraction operation, a control pulse is sent from logic circuit 805 which decrements the count data from counter 810 by two and inverts the data to produce the two's complementary numbers, which are then blocked in latch 812 and cleared from the counter in preparation for the next selection interval. Although the subtraction occurs in each sample interval, only the result is stored in the latch register for those samples or samples where zero has been detected. This selection represents the instantaneous residual function f(a). in a similar way, logarithm amplitude information (before the two's complement values) can be stored at peak complex zeros. These time-coincident samples of amplitude and residual now constitute two of the waveform information vectors or "WIV". (As described earlier, the amplitude samples can be compared to derive a direction-related vector).

Note that each sample interval corresponds to the main shift clock interval that controls the delay shift register in the periodicity sort matrix 45. Fig. 8a shows the wiring carrying the timing signals and the function blocks associated with the PSM 93. The PSM requires the shift clock timing signal to control its delay shift registers, a readout scan select signal to control the sequence of reading out the detected periodicities and a means for digitally encoding the periodicity measurements as they are read out by the PSM. The interrogation pulse logic 826 accepts the zeros from the digital differentiator circuit and ensures the timing for correct input to the PSM shift register.

It should also be noted that there is a block that allows the selection of any desired combination of waveform zeros as follows:

real

top

zero

real + top

real + zero

top + zero

real + top + zero

In the above, the expression (+) is defined as a logical OR. The ability to perform this selection is desirable in order to achieve optimal processing of different types of input signals. For example currently it seems that the best analysis of human speech will use the real + peak mode.

It can be seen from fig. 8a that the binary data regarding the periodicity measurements are stored in block 822 so that they can be processed in accordance with the methods described above for the aural retina.

Fig. 8c shows a detailed circuit diagram of a ready-to-use LAD circuit 91. This circuit carries out the necessary component identification and performs the functions of the amplifier 800, switch 801 and comparator 802 described in Fig. 8a. Component pin elements are also shown. An LM326 precision voltage regulator 803 supplies -12 volts to the LM387 feedback inverting amplifier 831 provided.

The CD4066 switch 832 is controlled by the positive switching pulse derived from the 2N3904 transistor amplifier 833. Amplifier 833 is required to convert the 5 volt TTL logic level to the 15 volt level for proper switching of the switch 832. A low pass filter 833 is used to assist in isolate switching or switching effects from the 831 preamplifier.

The discharge time constant RC of resistor 96 and capacitor 97 is chosen at 10K ohms and 220 pf to give the desired dynamic range. The output of the switched waveform is isolated from comparator 802 using a 2SK43 field-effect transistor source follower with high input resistance and low output resistance. The LM311 comparator 802 is triggered on positive voltage and recovers when the signal falls below the threshold set by the variable resistor 840. Bypass capacitors are used to reduce tripping due to stray voltage and to stabilize the tripping action. The SN7408 gate is used to buffer the comparator from the various subsequent load funct ions.

The feedback input amplifier 831 contains a voltage balance adjustment. It also has a 7.5 volt zener diode regulator in the -12 volt supply. This is necessary to reduce the total voltage supply to the CD4066 so that it lies within the manufacturer's 15 volt limit.

Returning to fig. 8a where the logic used to generate the timing signals necessary to drive the PSM 93 and the digital differentiator 92 is then described. This circuit is shown in detail in FIG. 8d, and the waveforms seen at the various points in the time sequence are shown in fig. 8e. The main clock 90 is an oscillator that uses an SN74LS04 hexagonal inverter with the possibility of a 2:1 frequency adjustment at 2 MHz. The output is labeled "fast clock" and is used directly in the following logic. From here the clock speed is divided by two in successive steps labeled A, B, C, D and E in fig. 8e. The waveforms are obtained from the two SN7493 integrated circuits M2 and M3. At point "E", the waveform is again divided by two into two channels to produce a two-phase clock. The E-clock is then inverted before driving dividers M6(A), from which the main PSM shift clock is derived. The non-inverted E clock is divided by Mg(B). The set of waveforms now allows the generation of some necessary control signals. The LAD switch pulse is derived by AND gates M4 and<M>11 (pins 8, 9, 10), which perform AND on the waveforms B, C, D, E and G of FIG. 8e. The inverted output signal is shown as curve (i) in fig. 8e. The scan pulse gate J in fig. 8e is derived from M-q (pins 4, 5, 6), which performs AND on waveform H and the shift clock. J signals then perform an AND on the A clock signal so that 24 pulses occur for each shift clock interval through AND gate Mg (pins 11, 12, 13), as in fig. 8(a).

The purge pulse is generated by the D flip-flop M-^ and AND gate M5 (pins 8, 9, 10) in fig. 8d. The purpose of this circuit is to extend the switch pulse somewhat so that unwanted switch transient effects can be removed from the comparator output. The D flip-flop provides a delayed fast clock pulse which is combined with the switch pulse to give the waveform shown as curve (p) in fig. 8e. The readout scan clock must then be shifted to discrete channels to provide a sequential readout shift clock to be used in the PSM 45. This is done by the counter M7 which generates a 3-bit sequential binary code. This code is demultiplexed by the Mg (74138) to provide a pulse with a shift clock interval duration that alternates over the eight octave channels. This switch gate thus selects the correct PS channel so that it receives the readout search clock pulses using inverters Mg and AND gates M15 and M-^. This sequence is illustrated by curves (q) and (r) in fig. 8e. The selected readout scan clocks for octaves 8 and 7 are chosen to illustrate the beginning of the eight-octave scan cycle which proceeds from 8, 7, 6 ...2, 1; and then repeats to 8, etc. This allows periodicity data to be scanned from the lowest to the highest frequencies. Digital encoding of the recognized periodicity sample is provided by the binary counter ^10 °9 ^11* This counter accumulates the number of readout scan pulses for each eight-octave scan cycle. When a periodicity detection pulse occurs, as illustrated by curve (s) in FIG. 8e, the serial data pulse corresponding to the recognition channel will drive out the binary number in an 8-bit parallel data bus. The example in fig. 8e shows a periodicity recognition on channel 36 with the corresponding binary number in curve (t) in fig. 8e. ;It should be noted that this method of data readout was chosen due to certain practical system considerations. Several alternative methods can be used which would be obvious to skilled binary logic constructors. ;As shown in fig. 8a, control signals for the digital differentiator section 92 are also derived from the timing logic circuit 95. This control logic is shown in detail in the circuit diagram of fig. 8 f. The control signals in fig. 8f is formed by the logic sequences produced by AND gates contained in M5, M7 and by the inverters of M-^. They are three successive plus signals shown as curves (j), (k) and (1) in fig. 8e and are called subtraction, complement and block/clear. ;The controlled clock pulses (see curve (e) in Fig. 8b), which are produced by AND gates M2(11, 12, 13) and M2(8, 9, 10) are fed to the eight-bit up-down counter M-q and M4. The total count is always reached before the end of the control pulse time cycle. During the count cycle, count output signals are sent through the exclusive OR gates M-^4 and M-^5 to the 8-bit binary adders M-^g and Mg. These adders are also connected to the 8-bit latches M-^q and M3 which hold the two's complementary results of the early preceding count cycle. Thus, the 8-bit output of the adders will present the continuous difference between the count cycle and the preceding frame count. At the end of the counting cycle, the difference remains static. ;The control pulse sequence continues from this point. The first control pulse is the "subtraction" pulse, which transfers the frame difference to the storage Mg and M-^*The subtraction pulse also times the monostable J-K flip-flop and causes its state to change depending on the presence or absence of a sign bit in latch M12 . This decision determines whether the current count A is greater than or less than the previous count B. Using this data, the two dual D-type monostable flip-flops and AND gates produce pulses corresponding to the complex zeros of the signal waveform, which will be described later.

The second control pulse, the complement, has two effects: it decreases the up-down counter by a bit and it causes the exclusive OR gates to invert, or complement, the binary number from the counter. These are the two's complement numbers which the blocking pulse (the third in the sequence) ensures to be stored in the latches M-^q and M3 in fig. 8 f. The blocking pulse also activates a monostable multivibrator Mg which produces a very short pulse with a width of about 125 nanoseconds. This slightly delayed pulse clears the counters and prepares them for the next counting cycle. The purpose of the delay is to ensure that the number is stored before it is purged.

The true zero instant is selected by using the purge pulse 5(p) to time the D flip-flop pair M-^ with the LAD comparator output feeding the D input. This circuit only selects the positive transitions of the signal envelope as this passes through the true zero axes.

The duration is just a shift clock interval.

An illustration of the null selection process derived from the waveform selections is shown in Fig. 8b. This timing diagram shows the comparator pulse (c), the cleaned comparator pulse (d) and the controlled fast clock pulses used to encode data (e). As the waveform f(t) crosses the zero axis, the LAD comparator is triggered and remains H (high) until the waveform falls below zero. This zero crossing thus produces a pulse which has the width of a shift clock period (f). As the waveform increases in amplitude, the decision A > B continues in the H state until sample pulse No. 4. Since the comparator width (digital amplitude) of No. 4 is smaller than No. 3, the sign bit decision A < B becomes high, causing a transition in the J-K seesaw. The illustration in fig. 8b shows the resulting pulse in waveform (g), which identifies the peak of waveform f(t). Note that the decision is necessarily delayed by two shift clock intervals, but this is a constant that is taken into account in the subsequent processing.

Note that the zero mode, which detects the transition to A > B will occur at both the real positive zero points and any zero point in the waveform. If thus only zero zero points are desired in subsequent processing, it will be necessary to combine the separate real zero detection with the real zero part of the zero mode detection so that the real zero is cancelled.

The periodicity sorting matrix 45 in the description is the circuit which recognizes and sorts all periodic elements in the signal composition simultaneously. The signal composition that is printed on the periodicity sorting matrix can generally consist of different mixtures of random or periodic excitation of overdamped and underdamped small waves. With regard to the category of the periodic signals

There are two basic situations:

(1) Two or more interleaved unsynchronized pulse trains of which at least one is periodic (small waves are overdamped); or (2) A periodic excitation with oscillatory response functions (small waves are underdamped).

These two signal states are shown in fig. 9A and 9B respectively. Fig. 9A shows two interleaved pulse trains having periods T-1 and T2 respectively. Fig. 9B shows a waveform which is typical for e.g. voice speech where the rate of repetition defines the pitch of the voice. The ring microwave responses are known as formants. The essential problem of measuring the individual repetition periods of the waveforms of FIG. 9A and 9B can be seen from the figures. It is obvious that no method based only on measuring successive time intervals of the zero crossings or peaks could succeed in sorting interleaved signal sources. In accordance with the present invention, however, a new periodicity sorting matrix is arranged which is schematically illustrated in fig. 11, which performs this function.

The device shown in fig. 11 is based on the continuous analysis of a signal for periodic components as it propagates through a tapped delay line. An array of logic gates is connected in such a way that when a signal pulse arrives at the input, a simultaneous comparison is made with all previous pulses in the delay line. This takes place in such a way that the event with two equal time intervals can be recognised. This event is defined here as a periodicity. Periodicities do not have to occur in a contiguous sequence as pulses in a pulse train. For example a series of randomly spaced pulses will, for example, produce three pulses having equal intervals. In the purely random or random series, these false periodicity detections will be distributed over the range of detection gates with equal probability. The statistical rate of false recognition would depend on the pulse density of the composition. When the distribution is not completely random, but is centered near a particular repetition rate, as in a "jittering" pulse train, the number of responses is greatest at the gate corresponding to the average pulse repetition rate surrounded by a Gaussian-type distribution of responses. The spread of this distribution will depend on the signal-to-noise ratio. Fig. 11 schematically shows the topological relationship to an arrangement of AND gates and the delay shift register. Fig. 11 is a "logic map" of the circuit arrangement. The illustrative circuit of FIG. 11 contains 12 AND gates each illustrated schematically with a respective one of horizontal lines 100-111. Each of the AND gates 100-111 is connected to the schematically illustrated tapped shift register delay line 112 which has 25 positions. The ports 100-111 are connected to the various positions at points represented by diagonal lines along the length of the horizontal lines representing the ports 100-111. The signal that enters the shift register 112 is the signal that is derived, e.g. from an output signal from the zero selection logic 811 in the digital residue processor of FIG. 8a. This signal is connected to a logarithmic A-D converter 113, digital differentiator 114 and zero selection logic 811. Thus, the input signal which may have the form shown in fig. 10A be converted to the signal shown in fig. 10B. Note fig. 10B that the interleaved pulses to pulse train A and to pulse train B are marked separately as An and Bn respectively.

A shift clock 113 timing diagram is schematically shown in the lower part of fig. 11 with a matrix schematically illustrating the manner in which the pulses A and B enter the shift register and are then propagated down the shift register delay line 112. The AND gate connections shown in fig. 11 are arranged sequentially and respond to any periodic event occurring within a 2:1 range. Alternating pairs of connections to the center pin of each periodicity gate 100-111 are necessary to ensure recognition of all possible combinations of intervals between the sequential pulse pairs. These combinations result from the 24 discrete clock time intervals of the shift register delay gain 112. For convenience in describing the system, the time scale of the signal shown is quantized in the same 24 units as the units of the tapped shift register delay line.

The two pulse trains A and B shown in fig. 10B is also shown timed on the same scale as the shift clock 113 and has periods of 6 and 10 time units respectively. When the first pulse of wave train A, labeled pulse A±, enters the delay line 112 there are no other pulses so no recognition occurs. Similarly, when pulses B-^, B2 and A2 enter sequentially there will be no recognition of periodicity and they continue through the system. When pulse B3 arrives in time unit 15, however, pulse B2 in time unit 8 and B-^ in time unit 1 will be in the system and the three simultaneous pulses are produced, thereby producing a recognition response for pulse train B which has a period of seven time units. In a similar way, when pulse A 3 arrives there will be another recognition from the ten unit port. In subsequent operation, each pulse train will be continuously deinterleaved in the discrete channels of periodicity recognition register 115.

In the circuit of fig. 11, it is extremely important to use the 2:1 ratio of sub-multiples in order to produce a logical barrier function which will prevent false recognition in the system. Thus, it is necessary to eliminate false responses at submultiples of the real periodicity on the assumption that it is desirable to recognize periodicities over a range greater than one octave. The key to removing these false responses is the 2:1 to ratio referenced above.

The first AND gate lines represent and are labeled the first octave of signal pulses; the second octave of AND port lines are connected and labeled as shown; and the third octave which is the lower eight AND gates are also labeled. A periodicity recognition register 121 which corresponds to the periodicity recognition register 115 in FIG. 11 are connected to the outputs of the 24 AND gates.

The propagation delay line 120 in FIG. 12 has a non-uniform delay characteristic in which the relative delays in each stage are indicated by the spacing of the taps. The time between units 0 to 16, which has the highest time resolution, corresponds to the time quantization of the highest expected periodicity of the signal. In units 18-32, the relative delay is two time units; in delay units 36-64 it is four time units. The octave ratio of the ports is adjacent so that there is a continuous periodicity detection capability over a three octave periodicity range.

The submultiple blocking function referred to earlier is performed by feeding back the output signal from each gate to the delay line increment in which the detection occurs. This feedback is labeled "recognition prevention signals" in fig. 12. Thus, the propagation of a submultiple signal pulse is terminated before it enters the lower periodicity recognition octaves.

It was previously noted that in the device of fig. 12, the distance between the delay line taps is proportional to the delay time. Designing an appropriate system for the necessary seven or eight octaves of an acoustic signal environment thus in a practical situation requires devices to reduce the physical distance along the delay line per delay unit as the delay increases. A device for reducing this distance is shown in the digital circuit diagram of fig. 13 which performs the functions previously described in the logic maps of fig. 11 and 12.

Fig. 13 shows how the logic map in fig. 12 can be transferred to conventional circuit design. This diagram represents all the essential characteristics of a drivable system and, except for the base module octave, is identical in construction. Each octave is equipped with respective stock shift registers 140, 141 and 142 for octaves 1, 2 and 3 respectively. Only four elements of the shift registers, rather than eight elements, as shown in FIG. 12, is shown in fig. 13 for reasons of simplicity. Of course, any number of elements can be used depending on the desired resolution.

The stock shift registers 140, 141 and 142 function like the register 121 in fig. 12 and provides an output signal in response to the detection of a periodicity "beater". This output signal can take any desired form, such as an electronic output control signal produced during the serial reading of all contents of the register. The output signal can also take the form of an LED output signal for each location within the stock shift registers.

Each element of each stock shift register is provided with a three-input AND gate shown as AND gates 150-161 having its outputs connected to the A, B, C and D inputs of the stock shift registers 140, 141 and 142 in the manner shown. The clock pulse signal that is fed in like the input signal in fig. 10B is then imprinted a clocked shift register having sections 170 and 171 in the first octave and sections 172 and 173 in octaves 2 and 3 respectively. Similar clock shift register sections will be arranged for each of the other octaves. Thus, as many octaves as desired can be used to cover as large a frequency range as desired for the signal analysis function.

The shift register elements are timed or clocked by a clock 180 which runs at the same rate used to clock the signal pulse train. (This circuit is referred to as the data shift clock in Fig. 8a). The delay time for each successive octave is increased by a factor of 2 by dividing the clock speed of clock 180 by 2 for each successive octave. Thus, 2:1 division circuits 181, 182 and so on are provided to reduce the clock rate fed in by two for each successive shift register 172, 173 and so on.

In order to ensure the transfer of data between octave delay lines, an additional shift register element, designated Tx, is inserted in each octave delay line 171, 172 and 173 to take care of the quantized time position of the period of the pulse pairs that occur when a sample is made of the analog periodic signal of the shift clock.

The signal from the Tx~ element in each clock shift register is connected to carry OR gates 190, 191 and 192 respectively in the respective octaves 1, 2 and 3. The other input signal to the OR gates 190, 191 and 192 is the signal from the previous delay element Tg, T12°9T16 for OR gates 190, 191 and 192 respectively, so that the signal pulse can always be transferred to the input element of the next shift register in the next octave.

The first input signal for each of the AND gates 150-153 is an input signal from a TQ position to the clock shift register 170. A second input signal to the AND gates 150-153 is taken from the clock shift register positions T5, Tg, Ty and Tg respectively. In a similar manner, other input signals to AND gates 154-157 are taken from shift register positions Tg, T10, T-q and T12ti-1 clock shift register 172.

The second input signal to AND gates 158-161 is taken from positions T-^, T-^, T<->^ and T-^g to register 173. The third input signal to gates 154-161 consists of output signals from OR- ports 200-207 respectively. The third input signal to AND gates 150-153 is shown as the output signal from OR gate 210, the T2or T3 output signal from shift register 171, the output signal from OR gate 211 and the output signal from position Counter T4 to shift register 171 respectively.

The arrangement described above will provide a periodicity output in one of the stock shift register positions of the stock shift register 140, 141 and 142 in the manner shown by the diagonal lines previously discussed in connection with FIG. 11 and 12.

Because of the discrete nature of the periodicity selections, an interrogation port 220 is provided to double the width of the interrogation pulse in all octaves lower than the base module. Thus, the first input signal to each of AND gates 154-161 consists of the output signal from OR gate 220, the input signals which are connected to clock shift register positions Tq and T-L respectively.

Fig. 13 shows the digital circuit device by means of which detected periodicities can be stored and transferred to a subsequent viewer and/or an automatic processing step. Thus, the output signals from the recognition ports 150-161 are sent to the pre-set inputs of the stock shift registers 140, 141 and 142. The stock shift register records the occurrence of a periodicity in its stock element after which the prevention function described in connection with fig. 12 works to remove the pulse from its associated delay shift register element. Thus, in fig. 13 each of the clock shift registers T5, Tg, Ty, Tg of 171 and each of the registers of 172 and 173 receive a clear barrier connection from the output signal to AND gates 150-161 respectively.

As an operational example for the circuit of FIG. 13, it can be assumed that triple periodicity takes place in the register locations TQ, Tg and Tg. The event is recorded in register D-^. The pulse in Tg is then cleaned by the obstacle pulse that appears on the Tg cleaning input of the shift register 171.

Each storage shift register 140, 141 and 142 is equipped with an additional delay element marked X. This additional element is used to store data to be shifted into the next register if a serial reading of data is desired. As stated earlier, data stored in each storage register can also be observed directly by connecting each storage element to a light emitting diode. Alternatively or simultaneously, data can be transferred out in serial form using standard shift register methods. If desired, data in the stock change registers can be decoded in binary numerical form to directly identify the item recorded. Any other form of data readout can be chosen as desired by the designer of the system.

Periodicity sorting matrices are known per se, but are significantly more complex than the new device shown in fig. 13. A known design is described in the publication "PRF Signal Analyzer", IBM Technical Disclosure Bulletin, vol 10, no. 11, April, 1968, p. 1781. The system shown uses a different form of octave submultiple rejection and has worse performance and is more complex than the system in fig. 13 and requires many more delay elements and logic components. This is especially true when the periodicity sorting system requires more than three or four octaves.

A characteristic feature of the circuit of FIG. 13 is that it can use as many octaves as desired and the number is only limited by the time response speed of the logic element. Thus circuits using eight octaves have been successfully used and with a resolution of 24 increments.

The periodicity sorting matrix in fig. 13 can be performed with many different circuit techniques. Thus, the delay function can be performed by devices such as analog tapped delay lines, cascaded monostable multivibrators, charge-coupled integrated delay circuits, digital shift registers and surface acoustic wave devices. However, when processing signals in the lower frequency ranges, which are used in seismic systems and normal audio/acoustic frequency ranges, the digital shift register arrangement in fig. 13 be the preferred circuit arrangement.

The general characteristics of a system that uses analog or quasi-analog methods that can use charge-coupled delay devices or surface acoustic wave devices are shown schematically in fig. 14 showing a circuit that performs the first and second analysis intervals of FIG.

5 and 6. While the function of the circuit of Fig. 14 can be performed by a fully digital processing system, the system is simplified by using an analog or quasi-analog system to provide the necessary information to the Aural Retina circuit. More particularly, as shown in FIG. 14, the signal f(t) to be analyzed is applied to a suitable delay line such as the dispersive delay line 200 which has n taps T^-T^. This gives a discrete sampling waveform of f(t) along the length of the delay line which has an increasing delay per unit of length. Each tap in the delay line 200 is connected to a respective circuit such as one of the circuits 201-204 which converts the stored function f(t) to log f(t) in a known manner. The output signal log f(t) of the function in each of the taps is then connected to a suitable subtraction circuit 205 which is arranged to subtract the output of each log-shaped delay tap value from the value of the preceding delay tap. This gives a discrete derivative of log f(t) which is the function r(t) = d£log f(t)^J/dt which is the "residue" described earlier.

The residual function r(t) is then applied to respective threshold detectors 206 which are provided for each delay cell and which work to determine the time interval of selected waveform features such as peaks or zeros. At the instants that correspond to these properties, the values of r(t) or residual are controlled into the Aural Retina circuit 207 via the residual ports 208.

At the same time, the output signals from the threshold detectors 206 are applied to the periodicity sorting matrix 209, which may be identical to the circuit in fig. 13. The output signal from PSM 209 is also applied to the input of the Aural Retina circuit 207. The Aural Retina circuit 207 then has inputs that take care of the periodicity of the waveforms, residue and amplitude so that it now has the information necessary to produce the necessary waveform recognition patterns. Note that the general arrangement shown in fig. 14 can be performed with charge coupled delay or surface acoustic wave device technology.

There is a significant advantage to the arrangement in fig. 14 since the arrangement eliminates the need for intermediate storage, addressing, averaging and parameter association logic that is necessary in a fully digital design.

The manner in which the outer product matrix 68 in fig.

6 can be performed on to perform the third treatment interval described in fig. 5 in which the WIV is assembled into a vector space in which the signal patterns can be recognised. A method by which this is accomplished is shown in Applicant's US Patent 3,886,553, entitled "Coordinate Locating Method and System", issued May 27, 1975. The method shown in the patent has as its primary purpose locating emitters in geographic coordinates by recognize the time-difference patterns of the received radiation. However, the circuits can be used for any coordinate system.

The following describes a novel method and circuit for identifying the waveform pattern using the two-dimensional space of residual a versus periodicity t . In general, the waveform information vectors comprising a four-dimensional space constituted by residue, periodicity, direction and amplitude can also be used.

In the new arrangement according to the invention, the residual data derived from the logarithmic derivative of the peak sample output from the register 98 of FIG. 8 first converted to an analog time difference. The periodicity reading from the periodicity sorting matrix in fig. 13 is also converted into an analog time difference by means of the serial readout mode described in connection with fig. 13. Periodicity and residue then form the coordinates of a rectangular Aural Retina existence space, which was described in fig. 2. This plane can be divided into a set of straight tangles which are bounded by reference points, as illustrated in fig. 15.

The reference points on fig. 15 are named with no. 00-03; 10-13; 20-23; and 30-33. Each of these reference points can be represented as points in time as a function of the continuous time variables a and T. Each point therefore defines a particular relative time delay pattern. By arranging a device to recognize the time delay pattern between the two input variables a and T, it is possible to locate corresponding points in the existence diagram. Since each point or set of points represents a waveform, the ability to locate these points is equivalent to identifying the signal.

Fig. 16 shows as a block diagram a circuit which is designed to perform the function mentioned above and which constitutes the outer product matrix circuit 68 in fig. 6. The circuit in fig. 16 serves the outer product manipulation of the WIV that has been measured in the waveform. In fig. 16, the two time-difference coded parameters for periodicity P and residue a are fed to respective shift register tapped delay lines 300 and 301. A third identical register 302 is also provided. Each of the registers 300-302 consists of conventional flip-flops. Registers 300 and 301 have four taps which are each marked 3, 2, 1 and 0 respectively. Each of these taps is connected to a respective AND gate, each of which is schematically shown by one of the horizontal lines numbered 01 to 33. Each of these AND gates corresponds to a position in the existence diagram of fig. 15. Each of the gates has three inputs and the third input signal comes from the last tap on delay line 302 shown as line 304 in FIG. 16.

A reference delay circuit is also provided in fig. 16 to allow recognition of all points in a plan. For each delay line, timed (clocked) countdown systems 305, 306 and 307 are provided which allow the input time delay pulse to be shifted through each delay line with higher resolution than that provided by the relatively coarse division of the discrete delay line taps into delay lines 300 -302. This count-down method allows a fine-grained interpolation of the pattern localization by comparing the relative values of responses to adjacent reference points as shown in applicant's aforementioned patent No. 3,886,553.

The operation of the circuit in fig. 16 is as follows: At any instant, a signal's location in the plane of FIG. 15 correspond to a specific AND gate 01-33 which is activated by the output signals of three coincident delay line taps. Only one port can be activated at a time and then give an output pulse. The array of gates is then connected to a conventional logical priority encoder 310a which converts the position of the activated gate into a binary code number on lines Ag, A^, A2 and A3. This number exists for the duration of the gate pulse.

The coded recognition pulse then addresses a direct storage 310. For each recognition event, a count will be added to the memory at the location specified by the port address. After each readout scan, the direct storage 310 is cleared and the cycle begins again. The scanning recognition cycle thus contains time section information for all events within the resolution of the period of the highest frequency signal.

Two functions that can be performed when processing the contents of the direct storage 310 are: (1) search out the composite Aural Retina plane of existence for examination of its contents which constitute a

"learning" process, and

(2) perform automatic recognition of preselected patterns as a result of the examination of the stock or mine contents.

The reading of the storage 310 is controlled by means of a suitable microprocessor 311 which in turn addresses all the elements in the storage 310. The accumulated count at each address can then be compared with adjacent reference points and, by means of a suitable interpolation algorithm such as that described in the applicant's aforementioned patent no. 3,886,553, the pattern point can then be calculated in the resolution provided by the original time measurement. These high resolution points then form the pattern of the existence diagram as represented in a simple periodicity sort matrix octave readout scan cycle. By accumulating in an adder these interpolated patterns in the microprocessor memory over an appropriate integration period, a complete snapshot of the Aural Retina signal pattern is obtained. That is that the adder adds each new recognition event from OPM to the number of beats already stored in RAM. It then creates a "histogram" of pattern responses in the stock interval. After each such interval, the accumulated patterns may be read out into a viewer or into temporary storage for subsequent analysis in the fourth processing interval searching for variable patterns, which was described in connection with FIG. 5.

The scan-scan transformation process can be used to identify the pattern corresponding to particular signal waveforms and their individual source objects. It is actually a learning process. By sampling/examining and recognizing the pattern coordinates produced by various signal source objects, a set of "templates" can be formed to search for the presence of desired incoming signal patterns. A template of this kind is shown in fig. 16 as the block 320. The template will actually consist, e.g. of a read storage encoded in a sequence of addresses corresponding to preselected waveforms. To search for a particular pattern, the addresses of the templates or read memory 320 are printed onto the direct storage address data bus in the readout mode after each periodicity sort matrix readout octave scan cycle. If the examined pattern is present in the direct storage 310, the examination will produce an event that can be recorded in a suitable recognition response counter 321. After a predetermined number of recorded events in a certain template examination pattern, a threshold comparator 322 can be activated to indicate the presence of the source object in the signal composition. This information can then be used to activate further analysis systems or control devices.

Fig. 17 shows a circuit which is an alternative to the circuit in fig. 16 for the Aural Retina processing and automatic recognition and which does not require the direct storage 310. With reference to fig. 17, the shift register delay lines 305, 306 and 307 of fig. 16, and the AND gates 00 to 33 are also shown. Each of the time reference inputs from register 307 is equipped with pattern selector switches 350 which can be alternatively opened or closed, as desired, either manually or remotely, to thereby be able to preselect one or more patterns that can be recognized. The output of ports 00 to 33 is then connected to a suitable response summation circuit 351 which in turn drives a counter 352 so that it accumulates the number of responses, and the threshold circuit 353. Circuit 353 will then provide a suitable recognition signal when the desired signal is present, as selected by switches 350 in a composite of mixed input signals. That is that a valid signal recognition signal will only be produced when a predetermined number of responses have taken place.

Methods have been described for scan transformation and pattern recognition in two dimensions, as shown in fig.

15. The same methods can be used for the recognition of signal source objects in the four-dimensional WIV space, as described for the generalized system in fig. 5. This is done by expanding the outer product matrix in fig. 16 into four dimensions using two additional tapped delay shift registers containing amplitude and direction input data in time-coded form. Additional recognition AND gates will be required for the process, as indicated by the product of the number of discrete elements in each vector. The number of ports can thus be impractically large for satisfactory pattern discrimination. However, an interpolation process, such as described in applicant's prior US Patent No. 3,886,553, can be used to improve the resolution by approximately 100:1 using relatively few gates. In this way, the four-dimensional periodicity sorting matrix can be carried out economically.

The outer product function described earlier can be performed by using the microprocessor method. However, due to the high number of multiplications and comparisons on the array of reference points that must be performed at each clock data interval, the computer processing becomes difficult and requires significant computational speed and energy.

The scan transform processor and waveform recognition system allows identification and localization of the set of source objects in the signal environment as described in connection with FIG. 16. They therefore comprise the devices for receiving the objects of the third analysis interval in fig. 5. Expressed in terms of the information content, the data speed has been contracted. With this lower data rate, the use of conventional microprocessor methods becomes appropriate so that the requirements for the remaining processing intervals to determine pattern changes and their significance can best be performed by means of computer algorithms.

It will be noted that the concept of the present invention is generally valid for any signal processing device regardless of the frequency range concerned. The only limitations in the circuits are derived from the response speed of the individual devices used. While acoustic processes have been enlarged for application of the invention, it will be obvious that the invention can be used in higher frequency applications than those in the acoustic range.

It is thus described in fig. 18 a new process and circuit for detecting time segments or increments of a timed series of events. In fig. 18, the signal pulse train that is fed in is thus applied to an appropriately tapped delay register 400. The taps of register 400 are connected to three input AND gates, each of which is represented by a horizontal line leading to pins A to H of recognition register 401. The outputs from recognize The NOR gate 401 is then connected to a NOR gate 402 and the output of the NOR gate 402 is connected to an input to AND gate 403. The other input to gate 403 is taken from the zero delay element of delay register 400.

The operation of the system in fig. 18 is as follows: The criterion for timing is the event of the presence of an interrogation pulse in the zero element of the delay register 400 without a corresponding periodicity recognition. The NELLER gate 402 provides a low output signal when a periodicity detection is recorded in the detection register 401. At the same time, the event of a periodicity detection requires that a pulse logic level H must exist in the zero-delay tap of register 400. However, a time segment is defined as the event of no detection when an interrogation pulse or the output pulse to drain zero occurs. This event can occur either when an aperiodic pulse is received or during the first two pulses of a periodic sequence.

The logical events from AND gate 403 are listed as follows:

The table above shows that time slots are identified by logic H when an unrecognized query exists. Note that a time slot detection event coincides with the arrival time of each pulse, regardless of the length of delay necessary to detect its associated periodicity.

The time slice detection method described in relation to FIG. 18 is extended to include a multi-octave system of fig. 18 a. With reference to fig. 18a, the clocked pulse train which is fed in is applied to each of n ok-tav circuits which are shown schematically as three circuits 410, 421 and 422.

Each octave 420, 421 and 422 has an OR gate 423, 424 and 425 respectively, instead of the NOR gate 402 of FIG. 18. OR gates 422, 423 and 424 each operate to detect periodicity within their range. A multi-input OR gate 426 is attached to all octave OR gates 423 through 425. When a signal appears in delay circuit 427, but when no periodicity detection is detected in any of the octaves, the Gloal time slice gate assumes a logic H. This indicates a time section event as previously described for the single octave in fig. 18.

From this point, the application of time slice information determines the information obtained in fig. IB how it should be treated. Time slice data can be separated into two groups: (1) addition of periodic sequences specified by pulse pairs; (2) arrivals of random events specified by single-celt pulses.

Although the above may seem to provide a fairly obvious distinction, the mere presence of two time-slice pulses will not necessarily indicate the type of event that has taken place (a periodic spike or two random events). This can be solved by arranging the device to associate the time section pulses with the periodic recognition data. Thus, if any pair of pulses is associated with a subsequent periodicity measurement, the time slice event must be a spurt; if not, then it's random. E.g. the on time section will always be followed by a sequence of periodic pulses (at least one) having a periodicity equal to the distance of the pulse pair. The particular logic necessary to perform this arrangement will be apparent to those skilled in the art.

As previously described, the new method according to the invention can be used for waveform synthesis, as well as for waveform processing and detection. It is possible to synthesize the function f(t) into equation (1) when the A^, sk and t k parameters are given. Expressed by Aural Retina, s-^ is defined loosely by a^ as a function of residue, and by t as a function of the periodicity of the ringing frequency if a is less than the critical attenuation. By thus supplying residue and amplitude at each zero point, defined by time period t k, it is possible to synthesize an approximation of any assumed waveform. Basically, then, the synthesis is equivalent to defining the response to the set of hypothetical source objects that form the complex waveform. Spatial information is carried in the differences of A^ and t^ which are inherent in physical localizations of sources relative to the position of the receiver or source. This will then allow synthesis of e.g. stereophonic sound. However, as in the Aural Retina signal analyzer, the synthesis method is applicable to synthesize random excitation signals as well as periodic signals.

When synthesizing a signal, the structure of the signal must first be known and this can be achieved with the help of an appropriate composition program or the like. The program will specify the desired location of zero points in time (time section, periodicity and duration), together with residue, amplitude and direction (WIV) at each zero point. In the case of voice response systems, e.g. where a computer produces speech information, the synthesizing program will contain WIV and null pattern data representing a specified speech vocabulary. In the bandwidth, compressed speech system known as "voice codes", the composition program will be specified using the coded speech data that has been sent through a communication channel. Another use of the synthesizer may be to reproduce music, voice or facsimile data that has been stored in a computer data storage device which may be portable.

In order to perform the synthesizing operation, a synthesizing matrix is provided which contains the means to place the waveform zeros, both real and complex, periodic or aperiodic, in a desired sequence in accordance with the defined time periods and durations of the postulated source objects. Furthermore, as each null point is placed in the sequence, it is demodulated by WIV information (residue, amplitude and direction) associated with the source object, which is determined by the compositing program. One should also note that the degree of waveform reliability can be controlled by adjusting the resolution of the residual, amplitude and time periodicity components. For example, by eliminating the residual and amplitude vectors, a representation of truncated speech can be generated for those communication applications where only intelligibility is important. After the synthesizing matrix function, a suitable spectral shape filter can be used to reduce the effects of selection and non-linear processing.

A block diagram of a novel waveform synthesizer made in accordance with the above stated features of the present invention is shown in FIG. 19. The purpose of the system in fig. 19 is to reverse the Aural Retina operation described above. Thus, the periodicities are locations in time produced by the zero point location matrix 500 in FIG. 19 which is physically the opposite of the periodicity sorting matrix previously described.

As shown in fig. 19, datums are generated in block 502 under the influence of composition program 501 and are imprinted on datum location matrix 500 in terms of time period, periodicity and duration. The zero point location matrix 500 converts information regarding time period, periodicity and duration into a suitable set of pulses representing the zero points that define the zero crossings, peaks and zero points of the waveform. The residue vector of the composition program 501 is applied to the residue processing block 503 and the residue vector information generated in block 503 is applied to the residue modulator 504. The residue modulator 504 and its subsequent functions 505 to 510 perform the necessary operations to synthesize at each zero point, including: a) integration to get log f(t); b) take anti-log to log f(t) to get f(t); and c) multiply f(t) by A to obtain the appropriate amplitude at each complex zero point and to

define the real zero points, where A=0.

As each pulse is emitted from the zero pulse location matrix 500, the pulse output from modulator 504 is "modulated" in width by the residual function in modulator 504. The waveform emitted from modulator 504 is then a square wave approximation of the waveform f(t) that it is desired to produce .

The square wave approximation output from modulator 504 is then conveniently split into two or more channels allowing the interpolation of amplitude and time delay information content, e.g. stereophonic information about the source objects. Thus, after division, the signals are applied to interrogators 505 and 506. They are then passed through anti-log circuits 507 and 508 respectively to perform the anti-log derivation operation described earlier.

The output signal from each of circuits 507 and 508 is then modulated in amplitude by means of modulators 509 and 510 respectively such that: (1) relative amplitudes Ak to source object exponentials are consistent with waveform requirements, and (2) relative differences in amplitude between channels are in compliance with the necessary acoustic propagation laws.

Amplitude modulation circuits 509 and 510 are controlled by relative amplitude outputs to direction monitoring circuit 511 which is controlled by composition program 501. Direction circuit 511 also has time-delayed outputs for both signal channels, labeled right and left channels, which are applied to time delay circuits 512 and 513 respectively. This provides the necessary time delays required for acoustic propagation of the signal. The output signals from the time delay circuits 512 and 513 are then applied to suitable filters 514 and 515 respectively, which are essentially bandpass filters which remove DC components and high frequency selection components. The output signals from these two filters then define the left waveform and the right waveform of the synthesized signals. The absolute amplitude of the signals can be suitably controlled in the absolute amplitude control block 517 which is connected, as shown to the direction circuit 511.

It will be obvious that many variations are available when it comes to the design of the system of fig. 19. For example if only positive and negative zero points are used, the waveform becomes a rectified signal. In many cases, the noise produced can be tolerable. The stereophonic effect shown in fig. 19 is also often unnecessary.

Figs. 20 and 21 schematically illustrate a zero point positioning circuit which can be used as circuit 500 in Fig. 19. An examination of fig. 20 and 21 between the zero point location matrix and the pulse sorting matrix has previously been described in connection with fig. 13. In the zero point location matrix of fig. 20, however, each periodicity combination for each quantized pitch frequency is represented by a separate line instead of using OR gates as in the periodicity sort matrix of FIG. 13. This allows the programmer or synthesizer to select pitches that lie between the quantized increments. With reference to fig. 20 shows an example of a zero location matrix which can provide six discrete periodicities or pitches per octave. In fig. 20, each of the horizontal lines represents an inverse or opposite of the AND gates used in PSM. Each of the inverted AND gates has three outputs which are connected to the "pre-selection" input of selected shift registers 530, 531 and 532. The connection is schematically shown with a dashed line representing a suitable connecting device such as a resistor, diode or direct wire connection.

A shift clock 533 times the operation of each shift register with the time division by two for each successive register as with the countdown circuits 534 and 535. Note that the shift registers work in the opposite direction to those previously described for the periodicity sort matrix. Clock pass-through gates 536 and 537 tie delay lines 530, 531 and 532 together.

Each of the inverse AND gate inputs is connected to respective line address decoders 540, 541 and 542 for the three octaves shown in FIG. 20. Decoders 540, 541 and 542 are used to select the desired periodicity pulse group by connecting to the various delay line taps as shown. Appropriate line addresses are generated by the assembly program depending on the input information to the system.

The function of the line address decoders 540, 541 and 542 can be performed in many ways. As a first possibility, the conventional line decoder using binary address numbers to select a line can be used. In another embodiment, a shift register containing a bit at the selected line can be used. A desirable feature of the use of a shift register for the function is that it can address multiple lines in parallel and simultaneously, whereas the demultiplexer can only address one at a time.

In the system described above in fig. 20 it will be seen that there are six discrete periodicities or pitches A, B, C, D, E and F per octave. The basic octave has two possible combinations for A, C and E, while other octaves, due to the clock countdown, have three for A, C, E, with two each for B, D and F. In general, this is 5n/2 lines per octave for all but the fundamental octave and 3n/2 for the fundamental octave.

The delay line taps 0-6 supply the "center pulse" of the three-pulse group that defines a periodicity. The base or basic octave is the only octave in which the center pulse requires its own set of shift register pre-taps. For example, note that taps 7-12 provide not only the third pulses of the base octave, but also the center pulses of the second octave. Similarly, taps 13-18 provide the third pulses of the second octave and the center pulse of the third octave.

Two important requirements for the synthesizer are shown in fig. 20.

The first is that AND gates 536 and 537 are used to prevent the upper octave clock from switching the data pulse from the lower octave before the entire lower clock period has been completed. Without this feature, the periodicity would lose one clock increment for each octave above the correct pulse period.

The second is that the entry of the zero pulse into the selected periodicity line must be timed or clocked to the leading edge of the clock pulse of the octave into which it is sent. This is shown in the labeled "line ready" control in the line address decoder of Fig. 20.

The operation of the zero location matrix of FIG. 20 can be further understood from the timing diagram in fig. 21. Fig. 21 shows the correct clock pulse trains for addressing the lines as well as the transition barrier for the clock pulses when the signal pulse is transferred from a lower to an upper octave. In fig. 21 is a selected pulse shown on line B to the third octave of FIG. 20. The pulses are introduced into the spaces in the shift register at Bq, B-j^ and B2q. The switching clock transitions are shown between octaves. These pulses represent the desired periodic zero points and are then propagated step by step through the shift register chain to form the specified output signal.

As a further illustration, a line in FIG. 21 shown as addressed by hy, A3 and Aq simultaneously with the Bq, B-^, B2q pulse. This shows that the system can provide a set of interleaved zero points that would specify a com-

plex waveform.

Although fig. 21 shows the shift register element selected by means of "preselection" inputs, it will be obvious that it can also be performed by direct connection with the '^" input. The form of the line itself can also consist of direct wire connection or any desired type of logical addressing method .

Although the block diagram in fig. 19 shows separate function blocks 504 for residual modulation and 509 and 510 for amplitude modulation, it is possible to combine the operations to thereby achieve a simpler circuit design. Thus, a method for inserting residual information based on the idea that the period between the real zero point and the first peak is equal to T/4 is described below. Expressed by the damped exponential concept of the Aural Retina existence diagram of fig. 2, this T/4 period of the rise time represents a residue of zero. As the residual increases towards infinity, the rise time approaches zero.

By means of a monostable multivibrator it is thus possible to form a period of T/4 for each periodicity representing the sine wave or the zero residual state. By then reducing the multivibrator period as a function of the residual amplitude, it is possible to generate a signal containing the required residual at each zero point.

A schematic arrangement using a monostable multivibrator 600 in this way is shown in fig. 22. The multivibrator 600 of FIG. 22 has a residual input connected to the line comprising resistor 601 and has a variable input pulse repetition frequency weighting time constant determined by the RC circuit comprising resistor 602 and capacitor 603. An input from this circuit is applied to the multivibrator pulse trigger input, while the residual signal is connected to the reset input. The output from the monostable multivibrator 600 then consists of the residue-modulated pulse as indicated schematically under device 600.

In operation, the fundamental T/4 width of the monostable for a sine wave at a given periodicity is defined by the existence diagram and is governed by the PRF weighting time constant. The composition program can control this weighting by varying the resistor and/or the capacitor in an appropriate way. The residual signal resets the monostable multivibrator in accordance with the magnitude of the specified residual. E.g. in the case of a sine wave condition, the period T car can be set to T/4. The period T can be made T/8 for critically damped exponentials and approximately equal to zero for broadband overdamped exponentials.

The output signal from the multivibrator 600 is then integrated so that the spectrum gets e.g. a roll-off of 6 db per octave. The antilogarithm function can then be performed using various well-known techniques using e.g. operational amplifiers or digitally controlled attenuators.

The next step in the processing sequence is to assign amplitude levels to each pulse in accordance with the composition program of FIG. 19. This can be done using conventional amplitude modulation methods such as diode bridges, digitally controlled attenuators or amplifier-controlled amplifiers.

The zero-pulse matrix part of the synthesizing system can be performed with digital techniques, where the final parts of the circuit can use analog signal processing. However, it is possible, if desired, to use fully digital methods. This can be particularly useful in the modulation and antilog functions that require non-linear processing of

the signal.

To demonstrate the advantages of the invention, a comparison of waveform analysis using the new invention compared to spectrum analysis techniques is now described, and to explain and illustrate the ways in which the Aural Retina and the Fourier analyzes each yield similar information to identify a signal. It will also be shown how Aural Retina can extract this information (a) faster than is practical with spectrum analysis, (b) with fewer dimensions (less data processing) and (c) in the presence of overlapping signals by using time-slice information which is lost by spectral processing. Furthermore, it will be shown how processing functions equivalent to correlation and deconvolution (unwrapping) can be performed.

Fig. 23 shows a simple periodic series of damped sinusoidal oscillations representing an elementary form of signal. In this example, the ring frequency is 1000 Hz and the repetition frequency for each ring pulse is 200 Hz. Expressed in voice processing, this would represent a pitch of 200 Hz and a formant of 1000 Hz. The Fourier transform is the conventional method to process this type of signal for the purpose of identifying these parameters. The Fourier method works excellently if the signal remains unchanged for a certain time interval which is necessary to get the measurements carried out with a specified precision. This is due to the previously described, well-known time-frequency inequality A t Af >^!• This restriction is unrealistic in practical applications where the signal waveform is time-limited, variable and composed of several sources.

We can derive an illustration of the problem with this time-frequency limitation by showing how the Fourier spectrum develops as a function of time when applied (time-averaged) to the signal waveform in fig. 23. Thus, with the spectral envelope taken at intervals relative to the impact, tg is shown in fig. 24. For convenience reasons in the later comparison with Aural Retina these spectral sampling intervals will correspond to the waveform peaks even though the spectral sampling windows are generally completely arbitrary with respect to the waveform.

At t-L in fig. 24 corresponds to the spectral envelope of the rise time of the exponential function from the on-off to the first peak. In reality, this can be defined as the bandwidth of the signal. Note that the "bandwidth" only has an empirical definition with respect to rise time. Thus, the spectrum is that given by a pulse whose peak is defined at the 1000 Hz ring frequency (or pole) of the exponential-function ion. It is "flat" from 0 Hz to the point where it starts to "fall off" at 1000 Hz.

At the second positive peak at time t2 in fig. 23, a complete cycle of 1 millisecond has taken place and the spectral energy begins to center around the ring frequency. When applying the A t Af inequality, the bandwidth around the peak is given by Af=l/At=l/lxl0~<3>=1000Hz.

In a similar way, in the third and fourth selection in fig. 24 envelope grow sharper around the ring top with bandwidths of 500 and 330 Hz respectively. So at the beginning of the second pulse response the spectrum starts a new function. The repetition of the pulse occurs at 1/200 Hz=5 milliseconds from tg and causes "ripples" in the previously smooth spectral envelope. Peaks in this ripple occur at 200 Hz intervals. As the number of spectral sampling intervals increases, the widths of the pulse repetition rate ripple become sharper in accordance with the AtAf inequality. At the end of the second response repetition, the spectral resolution A f will thus be 1/10 msec=100 Hz. Finally, after four repetitions, the resolution is 25 Hz. It is obvious that in order to be able to measure either the ring rate or the repetition rate with a certain accuracy, one has to "wait" a particular interval. But more importantly, the spectrum envelope must be obtained before the parameters can be determined. This requires calculation of the Fourier transform within the specified interval if "real time" operation is desired. This transformation is a well-known consumer of computational energy despite the many improvements in the FFT algorithm.

A similar analysis of Aural Retina can be described for the signal shown in fig. 23. In this case, the procedure is to extract a set of measurements at each datum (real or complex), which are independent. This independence is an essential characteristic that distinguishes Aural Retina from all forms of "least square" polynomial approximation methods, of which the Fourier transform is only one. Fig. 25 shows the time sequence for the samples taken at each peak. At each peak, the residual (or instantaneous bandwidth) and amplitude are measured simultaneously. These are the basic measurements of the waveform. From them, by comparing amplitude and time between two spatially separated transducer channels, a third instantaneous vector is derived: direction (©). By using the periodicity sorting matrix, the zero selections can also be established to identify periodicities, coincidences and time segments of time series.

In the example of fig. 25, the "residue" vector becomes a measure of the instantaneous bandwidth. In reality, it is the centroid (center of mass) of the spectral distribution that corresponds to the time interval between real and complex zero points or normalized rise time. It is also an estimate for the ringing frequency of the system, and is thus a "predictor" function. The amplitude vector, as extracted in the Aural Retina system, is a logarithmic function of the amplitude at each peak. Direction would be a function of the spatial position of the signal source.

The sequence of measurements on periodicity is a function of the device with which this parameter is measured. It takes three equally spaced zero points to detect a periodic event. As thus shown in fig. 25, the first two zeros related to the ring response periodicity are not recognized as periodic. They are an indication of the impact of a time series and are defined as the time period event that has zero periodicity.

When the third ring zero point arrives, this is identified as periodic at 1000 Hz rate. Then when the second pulse function is received the process begins again because the periodicity of the pulse repetition rate has not yet been identified. Finally, when the third pulse arrives, we have identified the repetition rate of the pulses. The pattern then becomes a pair of pairs forming the time section followed by a pair of repetition periodicities and a series containing the ring periodicities.

When comparing the set of measurements derived from the waveform of FIG. 23 using the spectrum patterns of fig. 24 with those derived from Aural Retina (Fig. 25), it is seen that with Aural Retina is achieved: 1) the ringing frequency is estimated directly from a simple waveform measurement, residual, rather than from a complex

analysis of the pattern of a spectral envelope;

2) the repetition rate is measured exclusively at the precise moment when such information is available, rather than from an estimate of the ripple distance on the spectral envelope, or by some other complex transformation of the spectrum into a "calculated"

or an "autocorrelation" pattern;

3) since the vector measurements performed at each waveform's zero point are independent, they can be processed by statistical procedures to complete optimal detection processes commonly associated with frequency domain techniques.

Although certain fundamental advantages of Aural Retina over Fourier processing can be seen in the processing of this simple waveform, the most significant advantage is found when the signal source contains two or more overlapping time series. To illustrate this situation, fig. 26 another simple damped sine wave added to the rows of fig. 23. The problem now is to calculate the description of both signals, including their exact time points for switching on (time period).

By applying Fourier analysis, we now have two spectral patterns that overlap during the lifetime of the two signals. The left curve in fig. 27 illustrates the spectral pattern of the second attenuated sine wave, which has a ring period of 4 msec. and produces a spectral peak at 250 Hz. The resultant spectrum at any time after the application of signal 2, tg2 / is the sum of both spectra.

The middle curve in fig. 27 shows the effect of overlapping spectral envelopes. The envelope of the spectral pattern at the end of the ringing of signal #2 shows two peaks: one at 250 Hz and the other at 2000 Hz. However, the effects of the repetition harmonic ripple on the 250 Hz signal pattern will cause interference in the selection of the real peak of the 1000 Hz signal.

At the end of the train of five pulses, the repetition harmonic ripple pattern will completely misplace the real peak of the 1000 Hz signal; this is shown on the right in fig. 27. Also note that there is no indication of the time period of the 250 Hz signal. In reality, the spectral patterns have either made it difficult or eliminated the possibility of sorting the two signals.

Since the example of fig. 27 is based on the simplest type of signal, it is obvious that the processing complexity becomes much more difficult when typical real-world signals are used.

To illustrate how Aural Retina solves the problem of sorting the two signals, refer to fig. 26. The figure shows the application of the periodicity sorting matrix's de-interleaving process to sort individual samples by analyzing their time domain relationships. After the waveform analyzer extracts the real and complex zero points with their associated measurements of amplitude and residual, the PSM time slices and periodicities of both signals are identified. As shown in fig. 26, the combined real and complex zero points are sorted one by one, so that the two signals, S-^ and S2 form two identifiable sets of measurements. Although the residual (related to the rate of change) can be measured at both real and complex zero points, the amplitude is measured only at the peaks. The time sequence diagram in fig. 26 shows the times at which the periodicities of the two signals are identified. Thus, the onset of the small wave S2 is identified in the middle of the zero points of the ringing and repetition of sequence S-^.

Although the two signals S-^ and S2 in fig. 26 can be identified by using only the periodicity and temporal conditions, a more complete description of the signals can be obtained by using the residual and amplitude parameters in the form of either a two- or three-dimensional space. This is shown in fig. 28. The primary recognition space applies the periodicity to residual parameters. Since the pattern in this plane is independent of absolute amplitude, automatic identification is greatly simplified. In general, the amplitude parameter is needed to describe a signal only as a second sorting parameter (and for possible later synthesis).

Fig. 28 illustrates the description of signals 1 and 2 in the Aural Retina recognition space, in which the parameter vectors in fig. 26 are combined to show the interconnected patterns that identify each signal. For convenience, each of these patterns is shown as an accumulation over the set corresponding to the duration of each pulse response to signal No. 1. The patterns are analogous in function to the sequences of the spectrograms of Figs. 27. Each point in space represents an accumulation of events. If the parameters are non-varying, the point position is unchanged and represents all events over the observation time. Thus, by deriving additional descriptive dimension by summing the events at each point location as in a multivariate histogram.

The illustrations above demonstrate the statistical advantage of the new signal processing method. With this method, one can achieve the purposes of signal identification and detection on the basis of classical statistical theory. Furthermore, it is now possible to electronically duplicate many psychoacoustic phenomena to the ear; e.g. the ear's ability to sort overlapping signal sources, as in the "cocktail party effect" and apply the results to devices used in automatic voice recognition equipment. The invention also allows high interpretability of "truncated speech" expressed by the fact that periodicity analysis of the real and complex nulls contains the principal components of the speech information, and excludes the residual vector, which contains only the "pitch" characteristics of the speech sound.

From a comparison between the two processing methods, it can be seen that besides the inevitable difficulties that the Fourier methods have in processing overlapping time-limited signals, there are clear differences in the complexity of processing the signals. The spectral pattern is a two-dimensional construct, but the same information is contained in the Aural Retina in the form of a small set of point measurements. E.g. is a damped sine wave represented by a pair of points in the periodicity/residual plane, while the Fourier domain requires a line of points whose shape characterizes the ring peak and "Q". The reduction in dimension is equal to a reduction in the information needed to describe the signal. Thus, the invention allows lower data rates, lower costs, lower energy consumption and lower system complexity.

Since the identification properties of the Aural Retina method are independent of the signal amplitude, the effects of large signal amplitudes will not inevitably mask the lower amplitudes. Nor is there a need to maintain a constant amplitude to be able to "recognize" a spectral pattern, which is necessary in Fourier analysis. Furthermore, in the Aural Retina system, signal recognition is made relatively simple by simply setting up a "template" that accepts only the specified set of points that characterize the desired signal.

Although the present invention has been described in connection with preferred embodiments, many variations and modifications will now become obvious to those skilled in the art. It is therefore preferred that the present invention is not limited by the particular description, but only by the accompanying patent claims.

Claims (10)

1. Method for analyzing a signal waveform f(t), characterized in that it comprises the steps of applying said complex signal f(t) to a logarithmic converter to produce an output signal with amplitude related to log f(t); applying said output signal related to log f(t) to a differentiator to produce an output signal related to a residual function, d[log f(t)3/dt; generating a zero-related output signal that is related to at least one of either the real zero point or complex zero points in said residual function when said function is either zero or a peak or zero value; applying said zero-related output signal to a periodicity sorting device to determine signal periodicities; and examination of said residual function d [log f(t)3 /dt and said amplitude output signal which is related to log f(t) in the times of said signal periodicities in an at least two-dimensional plane of existence in which each set of one or more discrete points in said plane of existence represents a particular waveform. 2. Method for analyzing a signal waveform f(t), characterized in that it comprises the steps of applying said signal f(t) to a logarithmic converter to produce an output signal with an amplitude related to log f(t); applying said output signal which is related to log f(t) on a differentiator device to produce an output signal which is related to a residual function, d £log f(t)J/dt; and output means for producing a given instantaneous output signal which is related to a predetermined value of said residual function. 3. Method according to claim 1, characterized in that it comprises a second signal waveform input f-^ it) and means identical to the means described for processing f(t) for processing f]_(t); and comparison device for comparing said two-dimensional plane of existence produced by processing said waveforms f(t) and f]_(t) to determine relative relationships to said signals. 4. Method according to claim 3, characterized in that said signals f(t) and f-j ^t) are received in first and second spatially separated signal channels. 5. Method according to claim 1, characterized in that it further comprises the step of normalizing said residual function by dividing by f(t). 6. Method according to claim 1, characterized in that said complex waveform f(t) is the result of individual signal sources and in which Ak = k amplitude coefficient; S k= k complex frequency (a k <+> j 6k ) t^ = time section (start) of the k excitation function. 7. Procedure for analyzing a complex waveform in which Ak = k amplitude coefficient; Sk= k complex frequency (ak <+> j <B>k )T^ = time section (start) of k excitation func- john, characterized in that said method comprises the step of determining at least two waveform information vectors (WIV) to waveform information vectors for time period (x), energy (A), residue (R) or direction (9) until arrival at said waveform (wherein said waveform information vectors are measured at the time at which at least one of f(t) or df*f(t)3/dt is equal to zero, and determining the existence of a given instant information characteristic of said wave -shape by comparing at least two WIVs with predetermined values. 8. Method according to claim 7, characterized in that it further comprises the step of determining a periodicity in said at least three WIV. 9. Method according to claim 8, characterized in that it further comprises the further step of comparing said WIV with subsequently produced WIV. 10. Method according to claim 8, characterized in that it further comprises the further step of comparing said WIV with a predetermined pattern.