CA1154874A - Word comparator device - Google Patents

Abstract

Associative Memory Circuit System and Method Abstract The associative memory circuit (10) is a word comparator device (100) that provides a numeric measurement of the degree of word similarity between the compared words as defined by a mathematical formula. The circuit is preferably an electrical digital circuit. The system (10) is an improved associative memory retrieval device and includes the use of the word comparator device connected in a storage loop to locate and extract records that are very similar to the supplied query. Inexact queries (70, 104) will rapidly locate records similar with respect to word, numeric and mask related measurements of similarity. The method of word comparison and processing serial data in the improved associative memory retrieval device (100) preferably of parallel configuration can provide rapid response to queries, while processing a large number of simultaneous requests.

Description

37~L "`

Associative Memory Circuit System and Method Technical Field My invention is an electronic associative memory circuit that includes a word comparator device for providing numeric measurement of the degree of word similarity between the compared words.
In operation the comparator device is con-nectable in a computer storage loop to locate and extract records that are similar to the supplied query. Inexact queriès can locate records.

Background Art An associative memory is a special kind of storage device. Whereas most memories are numerically addressed, associative memories are addressed via their contents. For example, one might ask an associative memory to rekurn all records containing the letters "ZXU" in columns one, two and three respectively. The address applied to an associative memory is the query.
If some record exactly satisfies the query, then the query is said to be exact, otherwise it is said to be inexact. Conventional associative memories provide no informatio,n in response to an inexact quèry. Inexact qu'eries are merely rejected. If a record exists within the associative mem~ry that is-only slightly different from the supplied query, then in many cases it would be desirable to know of its existence. Such records are minor corruptions of the query; Alternative-ly~ they are very similar to the query. Some related concepts were discussed in the following articles:

- . ~
:
~ . . . . .

,. . .

\ -(1) "The application of a pattern matching algorithm to searching medical record text," in the proceedings of the second annual symposium on computer applications in medical care, p. 308-313, I~EE 7~CH 1413-3, by the inventor et al, a copy of which is attached hereto and made a part hereof;

(2) The Ramon D. Faulk article in communica-tions of the ACM, Vol. 7/Numb~er II/November 1964, pages 647-653;

(3) The A.J. Szanser article, Mathematical Linguistics Error-Correcting Methods in Natural Language Processing, Information Processing .68-North-Holland Publishing Company-Amsterdam (1969), pages 1412-1416; and

(4) The A. J. Szanser article, The Computer Journal, Vol. 16, Number ~, pages 132-134;

(5) United States Patents 3,333,243; 3,651,459 and 4,084,260 show the state of the prior art.
A storage loop is formed when a storage device repeatedly and sequentially transmits its entire contents to external devices over a high speed data bus. An associative memory may be formed by attaching to this bus a device whose function it is to scrutinize in a passive manner the data stream originating from the stQrage device as it passes by on the bus. This a~tached device senses data appearing on the bus that is related in some predetermined fashion to a supplied query.
Central to associative memories of this type is some sort of word comparator device. In the prior art/ a simple digital comparator distinguishes two cases: equal and unequal. Other devices can : . . . ..

detect certain special corruptions such as the transposition of two characters, the deletion of a single character. Still other devices compare two words to arrive at an indication of how similar.they are. This device recodes the woxds as binary strings and then measures the Hamming distance between them. Devices in the prior art do not, however, seem to compare words in a general way that approaches the sort of similarity recog-nizing ability found in humans. United StatesPatents 3,333,243; 3,651,4i9; and 4,084,260 show the state of the prior art.

Disclosure of Invention This invention relates to a new and improved word comparator device referred to hereinafter as a word associator circuit that provides a numeric measurement of the degree of a word similarity between the compared words as defined by mathemati-cal formula. This invention also relates to an associative retrieval system and method for retrieval of inexact queries in a quick and expeditious manner. The circuit may be an electrical digital circuit or other type of circuit-ry that will provide an output conforming to amathematical formula to prov~e an improved word comparator function. An associative memory is normally thought of as a device which responds only to exact queries. Some may respona in a limited manner to inexact queries, for example see "Backend Processors is REM thé Answer" in Datamation, March 1978, pg. 206-207. The system and method of this invention uses the improved :

. :
. .

~L~5~7~

word comparator device to form an associative memory which responds to inexact queries in a new and useful way. This word comparator device is used to rapidly locate records most similar to a query. Similarity is defined by certain mathe-matical formula and is measured using a high speed digital circuit referred to hereafter as a wor~
Associator Circuit in the preferred embodiment.
Similarity has three components; word, numeric, and mask. Most central to this invention is the notion of word similarity. Words are strings of symbols from some alphabet and word similarity is a measurement of the similarity between two words. Several variable parameters are involved in the definition of word similarity, making it very flexible. Of significance is the ease with which the degree of word similarity may be deter-mined using digital circuits. Using existing technology word associator circuits may be built to process serial data at rates in excess of 20,000,000 characters per second. Such a circuit is described hereinbelow.
The system includes the use of one or more' associator circuits in storage loops to locate and extract records most similar to the supplied query. The system's architecture is preferably totally parallel so that it may be configured to process any number of simultaneous requests. Its memory is partitioned into storage loops so that through suitable configuration, arbitrarily large . . .

.

`,:l' ~
Qi ~ 5~374 ~.

datasets may be processed. Attached to each stor-age loop via a dynamlcally variable network are multiple associator circuits to locate and extract the records most similar to the supplied query.
By varying the number of storage loops versus the number of associator circuits, configurations may achieve a wide range of cost/response-time possi-bilities. In use, the associative retrieval system with assoeiator circuitS eould off load from a host processor, the task of searehing for database records. In doing this, it could provide improved services as well as entirely new services.
The method for retrieval as described in detail herein that confoxms to a mathematical forrnula also provides a new and improved invention over and above the prior art.
When queries of words are made seeking records, exaet records as well as slightly differ-ent records or similar records, are produced as an output. Such outputs are said to be a minor cor-ruptions of the query or are said to be in close similarity with the query. A measurement of the degree of similarity between two strings of symbols from some alphabet is defined ln "The definition~
computation and application of symbol string similarity functions", Emory University, M.S.
Thesis, Department of Mathema'ties, 1978, by the inventor. This measurement agrees well with intuition while remaining mathematieally simple and easy to compute, see "The application of a pattern matching algorithm to searching medical record text", in the proceedings of the ~'~ 's . . ~ . .
. .

~. .. :, .

~1~;i4~

second annual symposium on computer applications in medical care, p. 308-313, IEEE, 78 CH 1913-4, by the inventor et al. Using this measure-ment, minor corruptions may be located, thus ex-tending the conventional fu~ction of an associative memory into the area of a similarity memory. Of significance is the fact that this measurement may be trivially computed using the new and improved digital circuitry disclosed herein. Circuits to compute it are described herein and may be built to achieve very high processing ra~es. Therefore, in some applications, minor corruptions may be located without additional overhead.
The basic mathematics of word similarity was developed by the inventor and constituted his Master's thesis at Emory University, Atlanta, Georgia, June, 1978 referred to herein above. The inventor is also primary author of a paper presented to, and published in the IEEE reference herein above.
This paper describes the application of word similarity to searching raw narrative medical record text.
It is an object of this invention to provide an associative memory circuit that is a new and improved word associator circuit.
It is another object of'this invention to provide a word associator circuit that provides a measurement of the degree of word similarity.
It is another object of this invention to provide a word associator circuit`that provides a nu~eric measurement of the degree of word similar-ity between the compared words as defined by a mathematical formula.

~ .

~Si413~

It is another object of this invention to provide a word associator circuit in the form of an electrical circuit for providing a word com-parison conforming to the mathematical formula.
It is another object of this invention to provide an associative retrieval system by connecting a~ least one word associator circuit into at least one storage loop by dynamically variable networks.
A further object of this invention is to provide a method of processing data for word comparison that conforms to a particular mathe-matical formula.
An additional object of this invention is to provide a system method of retrieving similar words, n~ ers, and/or masks from inexact queries.
In accordance with these and other objects which will be apparent hereinafter, the instant invention will now be described with particular reference to the accompanyiny drawings.

Brief Description of Drawings Figure 1 is a block diagram of an associative retrieval system.
Figure 2 is a block diagram of the associator circuit illustrated in Figurevl.
Figure 3 is an illustra~ion of the timing of and ~.
Figure 4 is a block diagram of the basic word associator or comparator circuit.
Figure 5 is a block diagram o~ another word associator or comparator circuit~
Figure 6 is a schematic diagram of the selec-tor circuit shown in Figure 4~

.

, .
- : :
, ~s~
`

Figure 7 is a schematic diagram of the $ally memory circuit shown in Figure 4.
Figure 8 is a schematic diagram of the add circuit shown in Figure 4.
Figure 9 is a schematic diagram af the latch circuit shown in Figure 4.
Figure 10 is a schematic diagram of the test ci~rcuit shown in Figure 4.
Figure 11 is a schematic diagram of the add-latch circuit shown in Figure 4.
Figure 12 is a schematic diagram of another 'add-latch circuit shown in Figure 4.
Figure 13 is an illustration of the operation of the word comparator.
Best Mode for Carrying Out the Invention This invention is a new and improved associa-tive'memory circuit that is a word comparator ` device which provides a numeric measurement of the degree of word similarity between the compared words as defined by mathematical formula. The associative memory circuit includes a word associa-tor circuit shown in detail as an electrical digital circuit in Figures 6 through 12. The system or associative memory circuit is! shown in Figure 1.
This associative memory circui,t is an improved associative rètrieval device'that includes the use of the word associator or comparator circuit connected in a storage loop to locate and extract records that are most similar to the supplied query. Inexact queries will rapidiy locate records similar with respect to wo d, numeric and mask related measurements of similarity~

-. "
- . ~

. . ,: ~ ~
.. .. .

~548~

The new and improved method that is set forth below in detail provides a method of word compari-son and a method of processing in the improved associative memory circuit or associative retriev-al device. The processing is preferably in aparallel configuration that provides rapid response to queries, while processing a large number of simultaneous requests.
Referring now to Figure 1, most internal data traffic within the associative memory circuit 10 passes through shared memory 12, a time multiplexed multi-port random access read~
write memory of any well known design. Each of the many ports of the shared memory 12 is allotted a brief time slice on the order of one millisecond.
A port may disconnect prior to this time e'lapsing.
The associative memory circuit 10 communicates with the outside world through its communications modules 14 and 14' of any a well known design. A
plurality of communications modules may be connect-ed as illustrated by numeral 18 and the small circles or dots. I'he communication modules 14 and 14' are microcomputer based flexible interfaces responsible for decoding requests and then super-vising the operations of the associative memorycircuits 10 to satisfy the re~uest. The communica-tions modules'or circuits may communicate with the other associative memory circuits 10 using shared memory through buses 16 and 16' in any well known manner and by use of any well known design.
These communication modules might ~also perform sonsiderable preprocessing before~ passing a query onto the other system components.

: . . . :

' . . ' , . ~,, ~ '' ' :

:.: . : ~

- ~5~37~

The main storage units (MSU~ 20 and 20' of any well known design are devices that contain the actual records to be searched in memory units of any well known design. The main storage units contain any of a variety of well known control circuits to transmit these records in a fixed format over a bus. A plurality of main storage units may be used as illustrated by number 22 and the dots. The transmission format requires the simultaneous transmission of record characters taken sequentially from the r~cord moving from right to left and from left to right, see Figure 13 and the in-use description set forth herein-below. Numeric portions of a record are trans-mitted separately. The bus or lines 24 and 24'also contain control and timing signals, error correction codes and a data path of well known design for use in'the communications between associator circuits 42, 44, 42' and 44' and extractor circuits 56 and 56'. Within an MSU
20 and 20', data might be compressed to conserve resources by any well known means. The main storage units (MSU) 20 and 20' might be formed using virtually any of today's data storage devices. Following the trans'mission of each record along lines 2,4 and 24~5 a short blanking period is required to permitithe associator circuits to initialize themselves for another record. Prior to the transmission of each records data, the MSU 20 and 20' must transmit an internal record number for the `record that follows. These numbers should be a,ssigned sequentially by the ~SU 20 and 20f .
-~lS~1379~

The control circuit 30 is connected to theMSU devices 20 and 20' by bus 26. The control circuit 30 is responsible for all update and control of the MS~'s. The control circuit may consist of one or more simple microcomputers of well known design. Control circuit 30 communi-cates through shared memory 12 over bus 40.
Optionally, a direct interface 32 of well known design might be attached by bus 34. This would permit a direct data path from an MSU 20 and 20' to an external high speed device. This would facilitate the rapid loading of an entire MSU 20 and 20' as might occur at bootstrap time.
All data storage loops generated by the MSU devices 20 and 20' feed into network switch-ing 36 by bus 24 and 24'. The network switching circuit 36 is responsible for routing through bus 38 or 38' data from an MSU 20 and 20' to a vacant associator circuit 42 or 44 as well as 42' or 44' to satisfy a query. Additional associator circuits may be connected between 42 and 44 and 42' and 44' as illustrated by numerals 46 and 46' and the dots. Additional parallel circuits may also be interconnected as illus-25 ~ trated by numeral 48 and the!dots. Networkswitching circuit 36 is conneated to a control device 50 of well known design by bus 52 which processes requests communicated through shared memory 12 by connection bus or line 54. Control device 50 decodes these requests and decides which requests are to be processed and in what ~ ~ , ,~ .
order. Then contr~l device 50 co~unicates to network switching 36 over line 52, a specific order to reconfigure the network.

.: ; ~ . ~ :

::

~5~4 The associator circuits (ac) 42, 44, 42' and 44', are an important part of this invention.
The associator circuits 42, 44, 42' and 44' are connected in strings terminated at one end by a single or multiple extractor circuits 56 and 56' respectively by continuations of bus 38 or 38' respectively and at the other end by the netwoEk switching module 36. Data from a selected MSU
passes through network switching 36 and then through an associator circuit 42, 44, 42' and 44'.
This circuit scruitnizes the data as it passes, looking for records that are very similar to the query provided. Of significance here is the ~act that the word associator circuits, a part of ~2, 44, 42', and 44' (within each associator circuit) can look for similar records at very high ~ata rates. It is expected that data rates on the bus in excess of 20,000,000 characters per second are quite possible using today's standard technology.
The associator circuits 42, 44, 42', and 44' flag the most similar records and they are then extracted from the data stream by the extractor circuits 56 and 56' and eventually passes back through shared memory 12 over bus 60 to the commun-ications circuits 1~ and 14'.
The diagnostic computer ,6,4, also of any well known design, is connected in'a well known manner to the associative memory 10 to provide system performance statistics and maintenance information in a well known manner.
Referring now to Figure 2~ thè basic module - is referred to by numeral 43 whi;c-h is a more detailed block diagram of ass;ociator circuit 42 ' ~
.

1~5~87~

and 42' of Figure 1. Eaeh pair of associator eireuits in Figure 1 is similar to the Figure 2 illustration. Figur^ 2 shows the associator eircuit 43 along with the basie interconneetions.
The query is stored in qu~ry storage 70. As reeords pass by on the data bus 38, the records are received by the assoeiator circuit on inter-face 72 over bus 7~ where the records are merged with query eharacters,transmitted over bus 76 in an appropriate manner and then forwarded through buses 78, 80, and 82 to the thxee types of assoeiator eireuitry. The three types of assoeiator eireuitry are: (1) a word associator cireuit 84, t2) a number assoeiator eireuit 86, and (3) a mask assoeiator eireuit 88. Within the word assoe-iator eireuit 84 exists two cireuits designated by numerals 100 and 100', one of whieh is shown in greater detail in bloek diagram in Figure 4 whieh is illustrated in sehematie form in Figures 6 through 12. The word associator eireuit 84 eombines the output of eireuits 100 and 100' at the end of each reeord to arrive at the degree of word similarity. If the basie eireuit of Figure 4 is used then at the end of each record, the M output from eaeh copy of the eireuit are added together by~any well known means to arrive at the numerator of the fraetion that equals the degree of word similarity. The denominator is eomputed by any well known means ineluding table lookup by circuit 84 and is equal to L~L+l) where L is the,length of the eom-pared words. If the more compiex circuit ~f Figure 5 is used, then the numerator i5 eomputed -- ~ : ;: ' 3L~S~8~4 by any well known means and is equal to twice the sum of the M quantities output from the two copies of the circuit. The denominator is com-puted by any well known means and is equal to the sum of the TOTM quantities output from each copy of the circui-t. The word associator circuit 84 may or may not actually perform a division to arrive at the degree of word similarity.
Instead, the ranker 96 and the other associator circuits 80 and 88 might work entirely with fractional representations of similarity. Using the basic circuit of Figure 4 as 100 and 100' in Figure 2, computes the basic form of word similarity given by the mathematical formula disclosed herein.
It should also be noted the Figure 5 is an enhanced version of the circuit in Figure 4.
The word associator circuit 84 is mainly made up of circuits 100 and 100l and interconnect-ing circuitry of well known design.
Again, referring to Figure 2, at the end of each record, the three associators forward their "opinion" of how similar the record and the query were over bus 90, 92, and 94 respectively to a ranker 96 of well known design. If the record was a perfect match, then it is marked by the - ranker 96 for immediate extraction. Otherwise it is ranked relative to the pri~or records processed.
Only the N highest ranking records are maintained in the ranker 96 by their internal record numbers.
Here, N is an integer design parameter of well known design. Loading of the many`'paxameters in-; - volved in the association process`-is controlled by an onboard microprocessor ox controller 98 of well known design. The microprocessor is connected to the basic module in a well known manner. When all records have been observed, the ranker waits for the highest ranking records to appear again in the storage loop. ~s they appear, the ranker 96 maxks them for extraction in any well known manner.
~ he ~asic method for computing the degree of word similarity in a non-complex and expeditious manner involves processing symbols as they occur in a bidirectional serial data stream. By bidirectional serial, we mean that successive positions from each of the two words under com-parison and from their flips are simultaneously transmitted. For example, imagine two observers of the data stream performing the procedure or method as illustrated in Figure 12. Of signifi-cance is the fact that each observer needs knowleage only of the data instantaneously before - him in the data stream. We describe the procedure from the standpoint of these observers. We describe the information they must reme~er, com-putations they must perform and ~ecisions they must make. Each observer performs an iae~tical method, the first observes the transmission of the words, the second observes the tranSmission of the flips of the words. ~e~pre starting, each observer must per~form certa~n initial tasks.
After all data has passed, they combine their knowledge to arrive at the degree of word similar-ity between the transmitted words. Hereafter, ~ the block diagram shown in Figurè 4 will be ; described, second the method për`~rmed by each :
.: - - : . ~ :~: . : : .

~41~74 observer will be described and then the method in which their knowledge is eventually,combined will be described.
- Referring now to Figure 4, the one word associator circuit in block diagram form in illus-trated as numeral 100. Two of these circuits 100 are included in the word associated circuit 84 in , Figure 2. The data selector 102 illustrated in ' Figure 4 is shown in detail in Figure 6 and may utilize two quadruple bus gates such as Texas instruments, Incorporated's circuit 7412S and 74126. It has two input buses 104 and 106 entering from above. One of these -two in one ,~
time frame is routed to a single output 108 exiting below. Figure 6 discloses the schematic for a single bus line. Clock input potent'ial 110 is connected to the selector 102.
Two,clocks are utilized in,circuit 100, Figure 4, They are referred to as 0 and ~ and graphically illustrated in Figure 3 to disclose their interrelationship. Their interrelationship and purpose is described and disclosed in the diagrams illustrated in Figure 3. The purpose of 0 is to select either the read or write mode of , 25 operation in the tally memory 114 in Figure 4 and - to provide certain triggers in latches 170 and 136 of Figure 4. When 0 is low, 'the read mode of operation in 114 is selected. When 0 is high, the write mode of operation in 114 is selected.
30 The event consisting of a low to high transition ' of 0 may serve as a trigger to latch components 170 and 136 of Figure 4O -'~ , - - :

.
. - ~ . .

r ~5~

The purpose of ~ is to define whether a query or a bus character is currently being pxo-cessed. A complete cycle of 0 corresponding to a read/write cycle in 114 occurs during each half cycle of ~ as shown in Figure 3. Another purpose of 0 is to provide a trigger to latch 182 of Figure 4. The event consisting of a high to low transition of 0 and may serve as a trigger to latch 182. The clock 0 is used as a control in-10 put in blocks 144,126 and 102 of Figure 4.
The main portion of the Figure 9 circuit isdesignated by numeral 112 and includes a random access read/write memory 114 referred to as tally or a tally memory. The memory address enters from above through bus 108. The read/write mode of operation is selected by the read/write poten-tial 0 entering from the right through line 116 from a clock means of well known design not shown.
When read/write potential is low, the read mode is selected. When read/write potential is high, the wri-te mode is selected.. Data exists from the lower left on bus 120 and 0 enters from the upper riyht.
In the preferred embodiment, tally is organ-ized as 256 8 bit words. Thus, addresses and data are 8-bit quantities. Figure,7 shows the schematic for each bit of a tally word' The tally may in-clude one or mor~ Texas Instrument Incorporated Hex inverter 122 No. 7404, and 256-bit read/
30 write memory 124, No. 74200. It should be designed so that its contents may rapidly be set to zero by clear means 118.

.

. .~

.. . .
~3 5~79L

Simply setting all cells simultaneously to ~ero, may however not be practically feasible due to power surge and overheating considerations.
Therefore, several cycles may be necessary to clear the memory. Each cycle would then clear a fixed portion of the memory. Also, it is not necessary to actually set all of the cells to zero. An extra bit associated with each word could be maintained. I`o clear the memory, only those extra bits would be cleared. Then when a memory word is read, this extr~a ~it is tested.
If it is zero, then zero is read out instead of the ac-tual memory contents. This extra bit is set only when its associated loca-tion is written 15 into. Now, if one is read out, then the actual `~
memory contents are presented as usual to the outside world. In this way, the extra bits affect a logical clearing of the memory while avoiding a physical clearing of all of the cells. ~;
Techniques such as these serve to significantly expedite the clearing operation, but such pro-cedures are not necessary because there~are well known standard procedures available.
The add block Figure 4 is an incrementer/
decrementer 126. Data enters from the right.
Depending on the state of in~ut potential, the input potential value of the'en~ering data is either incremented ~r decremented before exiting the two four bit binary full adders circuit 130 30 on bus 132, as shown in Figure 8. The adders 130 may include a Texas Instrument Incorporated 4-bit binary full adder No. 7483. Hex inverters 134 ! - `` `

.~

': : : '. . :
;
', -: ', . ~ . :, ' 1~54~8~

No. 7404, Exhibit AA is connected between the input potential over bus 128. The incrementer/decrementer 126 in Figure 4 is actually an adder in which one of the summands is restricted to either plus or minus one. If input potential of 128 is zero, then plus one is added. If input potential of 128 is one, then minus one s added.
A positive edge triggered data latch 136 in Figure 4 is connected to bus 132 by bus 138. Data enters through bus extension 138 from the left and exits from the right on bus 140 to tally 114.
.On the positive going edge of read/write potential actuated by input 0, the contents entering latch 136 are latched and become the output from the latch over bus 140. In the preferred embodiment shown in ~igure 9, the latch is a two-bit b-type register 5 with 3-state output 142 shown in Figure 9, ~exas Instrument Incorpora-ted Number 74173.

A conver-tible sign tester 144 in ~igure 4 has an input through bus 132 for data entering from the right and the tester 144 determines if this da-ta is non-positive or non-negative, depending on the sta-te of the input potential on bus 146 enter- -iny from below. The test is performed relative to two's complernent arithmetic. ~f the input poten-tial is low, then test will o~tput high on output bus 148 to the left, provided that its input is greater than or equal to zero. In the preferred embodiment, tester 144 i5 shown in Figure 10 as an 8-bit device. The input 132 is connected to two dual 5-input positve no ga-te 15~ and 152 Texas Instrument No. 74260 connected .

; . i , . ... :, ., : :- :

to one gate of a quadruple 2-inpu-t positive and gate 154, Texas Instrument No. 7408. Gate 154 is connected to one gate of a quadruple 2-input positive or gates 156 Texas Instrument Incorporated No. 7432.
Gate 156 Texas Instr~ent No. 7432 ~' is also connected to and gate 158. And gate 158 Texas Ins-trument No. 7408 is connected to one of the input lines 132 and line 196 through inverter "
10 160, a Texas Instrument No. 7404 The output of 156 is connected to the input of 162 that is the same as 156 having another input Erom and gate 169 with input from bus 146 and the output of inJerter 168, a Texas Ins-tru~ent Incorporated 15 7909.

A clearable edge trigge,red ]atch 170 is shown in Figure ~ and shown in detail in Figure 11 with an adder attached as described b~`low. Input 172 inserts a one (1) into latch 170. The output is 20 transmitted on bus 174 to add lat,ch 182. The adder 170 has two inputs on busses 148 and 172 and a single output on bus 174 which is the sum of the inputs. The output 178 of the adders 176, ' such as a 4-bit binary full adder of Texas Instrument, Incorporation NO.,r7483 'become the input to the latches 180 such as 4-bit D-type registers with 3-state outputs, ~xhibit D. One of the inputs to the adder is the output of the latch. The other input enters from above in Figure 11 and is wired permanently to be equal to 1. The latches 180 are~si,milar to item 142 and are triggered from bus 148 when it is high '~

, , :. . - . -. - . ~

- : . - . ,, , . : ~ .

8~4 at the positive going edge of the read/write potential. The current latch contents exit from below over bus 174, to an add-latch 182. In the preferred embodiment, latch 170 is an 8-bit latch coupled to an ad~er with an ~-bit output and two 7-bit inputs as shown in Figure 11. The clearing connections are not shown in Figure 11, but may be accomplished by any well known manner.
Add-latch 182, shown in Figures 4 and 12, is like item 170. Its input enters from bus 174 from above. Add-latch 182 is triggered on the negative going edge of the read/write potential ~. In the preferred embodiment, add-latch 182 in-cludes 4-bit binary full adders 184 such as Texas Instruments Corporation No. 7483, is a 13-bit latch coupled, that is connected to a 4-bit D-type register with 3-stage outputs 186 such as Texas Instrument Corporation No. 74173. Indicator/
readout devices of well known design may be connect-ed thereto. The output of item 182 is an electrical output-signal that may be translated to readable or other indications by any well known device.
Clearing connections are not shown in Fiyure 1~, but any well known devices or procedures may be used.
Referring now to illustr~ion Figure 13, the memory 190 contains the words~ABC and ABB which are to be compared. They are transmitted via two transmitters 192 and 194 over a data stream four characters wide, as illustrated. The top half of the stream contains the transmission of the unaltered words and the bottom half contains the transmission of the flips of the words. On each ..
: .

~54~

side of the stream sits an observer 196 and 200.
Each observer is watching a single column at a time as columns flow from left to right. The memory 150 and transmitters 192 and 19~ correspond -to the MSU 20 of Figure 1 and the data stream roughly for illustration purposes corresponds to the data bus 38 of Figure 2 ~although query characters do not occur on the data bus or in the ~SU). The two observers correspond for illustration purposes to the two copies of the circuit 100 shown in Figure 4 contained within the word associator 84 of Figure 2. To perform his appointed task, each observer must "remember" a numeric quantity associated with each alphabet member. In this example there are but three; A, B, and C. This collection of quantities corresponds for illustra-tion purposes to tally 114 of Figure 4. Each observer must also "remember" a numeric quantity R that is 170 and another M that is 182. These correspond for illustration purposes directly with Figure 4. At each instant in time, each observer notices two characters before him. He processes one at a time in some fixed order, say top to bottom. First, he increments the quantity corres-ponding to the top character.! Then if it is lessthan or equal to zero, he increments R. Next, he decrements thè quantity corr~sponding to the bottom character. Then if it is greater than or equal - to zero, he increments R~ Finally, now that both ~ -characters are processed, he updates M by adding R to it. This continues for each`column as they flow past. '~

- , . , , ~, -~. :

: .

.

":~
~lS9~8~

In the above, we have assumed that he observer started with all quantities equal to zero. Once the record has passed, the two ob-servers add together the values for M that they have arrived at. This result is then divided by LtL+l) which in this case is 3(3+1)=12. L is defined as the length of the compared words.
Th~s, we have 8/12= 2/3 as the final similarity between the two words. Figure 13 displays the final state of all numeric quantities involved~
A basic mathematical formula has been created.
If A is an alphabet, then words in~A are finite concatenations of members from A. If w is a word in A, then w denotes the flip of w. For example, the flip of the word "abcd" is "dcba". If x and y are numbers then (x,y) denotes the greater of the two quantities x-y and O. If w is a word in A, then n(a,w,i) denotes the number of occurances of alphabet member "a" in word w found in position i or beyond where position is measured canonically -from left to right. If w and v are words in A, both of length L, 'then the degree of word similarity between them is denoted Stw,v) and is given by the formula below:
L
L(L+l) -~ S (n(a,w,i,),n(a,v,i,)) ~ ~-o~fl ~I(n(~,w,'i),n(a,v,i~) S (w,v) = . '~
. ~
LtL+l) This formula, as set forth above and well understood by those skilled in the art, produces a number between 0 and 1 inclusive. It produces 1 if and '~3' . ~

- ~ . .
., ~ , . .. . . . .
.~ . , : .... ~ .
. ~ ~

only if w and v are identical. :rt produces 0 if and only if w and v share no co~non alphabet members. Intermediate values are interpreted as degrees of similarity between these two extremes.
Formula exist and are discussed in "The Definition, Computation and Application of Symbol String Similarity Functions" referred hereinabove, which do not presume equality of the lengths of w and v.
The above formula is, however, the most fundamental.
It equally weighs all alphabet members and word positions. It corresponds to the circuit of Figure 4.
In the equation above, the fundamental compu-tation is that involving the double summation.
Various forms of the equation might still perhaps produce a useful measurernent of word similarity.
For example, the whole equation might be raised to some positive integral power. Falk in the artical referred to hereinabove, discussed a function which also associated with each pair of words, a number between zero and one. But his function is consider-ably more complex and in practice is much more difficult to compute. This is discussed in the thesis referred to hereinabove. The disclosed formulà rests on a simpler and more rigorous mathematical foundation, as p~inted out in the thesis. Faulk also makes lit'tle attempt towards justifying his formula.
It should be noted that the circuit inven~ion evolved from the formula to the algorithm to the circuit. The following criteria are met:
1. Mathematical simplicity 2. Ease of computation : ~

3. Agreement with human intuition 4. Flexibility to permit varied applications.
The formula derived in the thesis and disclosed herein and made a part hereof is mathematically simple and produces results that appear to agree well wi~h human intuition as referred to in the IE~E article referred to hereinabove. Computation of the formula in a straight forward fashion re-quires quite a bit of work, primarily due to the double summation.
The next evolutionary step is the invention of an algorithm which quickly computes the formula. This algorithm is presented herein in the form of a Fortran function subprogram. It is called with three input parameters: IQ, IR and N.
IQ and IR each of which is a dimension N integer vector. Upon return, the variable Theta is the degree of similarity between the input words IQ
ancl IR. Alphabet members are integers between 1 ~0 and 256 inclusive.
To process a character from each of the two words under comparison, the algorithm implemented in machine-language on a modern general purpose CPU such as the IBM 370, requires the e~ecution of dozens of instructions each comprised o~ many micro instructions. s To perform this same ta~k, our circuit re-quires only two internal clock cycles. Each rep-resents approximately a read/write cycle pertaining to a high speed random access memory. Actually, the clock must be slightly slower than the memoryls maximum speed to permit other circuit components to operate~ In use, however, the multiple should .~. ,'~... . !
. . .
.

8~

be less than 2. Tnerefore, we see that our circuit is capable of computing our definition of word similarity much faster than any existant general purpose processor.
A complete similarity memory system may con-tain many such circuits. Therefore, the system would then be performing a search function beyond the capabilities of existant general CPU's. The basic algorithm as programed in Fortran is:
FUNCTION THETA tIQ, IR, N) C
C IQ AND IR ARE EACH WORDS OF LENGTH N IN THE
C ALPHABET CONSISTING OF THE NUMBERS 1 THROUGH 256.
C THEY ARE PASSED AS INTEGER VECTORS EACH OF
C DIMENSION N. VPON RETURN, THETA ASSUMES THE
C VALUE OF THE BASIC WORD SIMILARITY BETWEEN IQ
C AND IR.
C
` 20 INTEGER R
DIMENSION ITALLY (256), IQ(N), IR(N) DO 1 I=1,256 1 ITALLY(I)=O
- 25 MO=O
R=O
C
DO 2 I=l,N
ITALLY (IQ(I))=ITALLY~IQ(I))+l IF (ITALLY(IQ(I)).LE.0) R=R+l ITALLY (IR(I))=ITALLY(IR(I))-l - IF (ITALLY(IR(I)).GE.p) R=R~
2 M=M~R
C ..

'~

- .: :' ' : '' :

~L~S~7g ~ P
DO 4 I=1,256 4 ITALLY(I)=0 C

DO 3 J=l,N
I=N~l-J
ITALLY (IQ(I))=ITALLY(IQ(I))+l IF (ITALLY(IQ(I)).LE.0)R=R~l ITALLY(IR(I))=ITAI.LY(IR(I))-l IF (ITALLY(IR(I)).GE.0) R=R+l 3 M=M+R
C
THETA=FLOAT(M)/(N*(N~l)) C

RETURN
END
:
Referring now to Figure 5, the SEL is a random access read/write memory 210. Its address enters on bus 212 from abov~ and data output is to the right on bus 214 In Figure 5, .it is assumed that the read mode is selected as the device is only written to during master initiali-zation. In the preferred embodiment the read/
25 write memory 210 is organized as 127 2~bit words.
SYN consists of three random access read/
write memories, 216, 216', a~d 216''. In each, the address enters on bus 218 from above the data output transmitted on bus 220, 220', and 220'' from below. In the Figure, we assume that the read mode is selected as the devic.~ is only written .
- to during master initializatian`.; In the preferred embodiment, it is organized as thP~e memories, each çonsisting of 256 8-bit words. Select is a . . ,;, ~

:. ; . : : :, ::

:: .

data selector 222. It has three input busses 220, 220', and 220l' entering from above. One of these three is routed to a single output 224 existing below, depending upon the numeric value of the 2-bit value entering from the Left. If this value is 0, then Select ignores its input and outputs zero. If this value is 1, 2 or 3, ~hen the fil~st, second or third input data bus respectively is routed to the output. In the preferred embodiment, it has 8-bit inputs and outputs.
MV is a one bit latch 226. It is set/reset during master initialization. The current state of MV exists above.
Select 228 is a data selector. It has two input busses 224 and 230 entering from above.
One of these is routed to a single output éxisting below, depending upon the state of ~ entering from the right. If ~ is low, then the right input bus is selected. Otherwise, the left bus is selected.
In the preferred embodiment, Select has 8-bit inputs and outputs.
Double skip on zero is a circuit to block the propagation of 0 and ~, for the duration of two 0 `cycles, provided that ~=0, MV=l and the output from Select is zero, at the t!ime of a positive transition of 0. This effect~vely causes the later circuit stages to igno~e completely the current column. These altered versions of 0 and 9 -are then used by the central s-tage of the circuit.
PW is a random access read/write memory. Its address enters from the left and d`àta output is - to the right. In the figure, we~assume that the read mode is selected, as the device is only written .
. ' ':

' 2~

to during master initialization. In the preferred embodiment, it is organized as 127 2-bit words.
CW is a random access read/write memory.
Its address enters from above and data output is to the right. In the figure, we assume that the read mode is selected, as the device is only written to during master initialization. In the preferred embodiment, it is organized as 256 2-bit words.
Distributor is a data distributor. It has three outputs above and a 2-~it control input to the left. When this input is zero, all three outputs are zero. ~hen it is 1, 2, or 3, the first, second or third output goes high respectively, leaving the others zero.
Single skip on zero is a circuit to block the propagation of 0, for the duration of one 0 cycle, provided that the cumulative output from the gate circuits is zero at the! time of a positive transition of 0. This altered version of 0 is then used by the lowest circuit stage.
Each gate is a pair of logical and gates usQd to control the propagation of the data output from CW. Both bits leaving CW center each gate to the left. Inside gate there !are two 2-input and gates. One input from each becomes a common control input shown entering ~rom below. The remaining two inputs connect to the two entering data lines. The outputs from the gates are shown to the right. When the control input is low, the gate outputs zero. When it is higX, gate simply propagates its two bit input. ~
The combination of the three gate circuits and the distributor circuit forms a variable shift "

:: :
. ~ , ` '' ,. ' ~ ' :: , .
:
.

register which shifts the output of CW, depending upon the output of PW. This aff~!cts the computa-tion of the final weight.
TOTM is an add-latch device. Its input enters above. It is triggered on a negative ~ transition.
In the preferred e~bodiment, TOT~I is a 17-bit latch together with an adder having one 17-bit input and one 11 bit input.
R is an add-latch device. Its input enters above and its output exists below~. It is triggered on a positive 0 transition provided that T entering from the right is equal to one.
In the preferred embodiment, R is an ll-bit latch coupled to an adder having one ll-bit input and one four bit input.
TOTR is an add-l~tch device. Its input enters above and its output exists below. It is triggered on a positive transition of 0 provided that ~=1. In the preferred embodiment, it is an 11 bit latch together with an adder having one 11-bit input and one four-bit input.
M is an add-latch device. Its input enters above. It is triggered on a negative a transition.
In the preferred embodiment, it is a 17 bit latch together with an adder having!one 17-bit input and one 11 bit input. !S
; Sign test is defined as test of Figure 4. L
is defined as L of Figure 4. Tally is defined as tal,ly of Figure 4. Adder is defined as add of Figure 4.
Referring to Figure 5, this diagram illustrates ` how the basic circuit of Figure ~4 may be consider-ably enhanced with sacrificing speed of processing.

, .,, . :, .:

, ,,,. .. : -. . . :

:: ~ : -:: : ~ . :

379~

In Figure 5, before data reaches the core or data selector, circuit 102 and the basic word associator circuit 112 that is identical to that shown in Figure 4, several tasks are performed.
First, a memory word i5 fetched corresponding to the current column position being processed. If this word is zero, then the current column is ignored. This is accomplished by using the double skip on zero circuit 220 of well known design.
This circuit mexely blocks propagation of all timing signals during the current character pair.
Therefore, the circuit ignores the current column.
Whereas the circuit of Figure 4 processed every column unconditionally, this facility permits column selection in the circuit of Figure 5. If the fetched word is non zero, then it is used to select one of three tables to be used in trans-lating the data character from the record before it reaches the cire circuit 102'. This is called synonym processing and permits additional flexi-bility. The facilities above are implemented via the random access memory's SEL 208 and the three random access memory's labeled SYN 202, 204, and 206, and by the SEL 208 component which simply selects one of the three outputs from the SYN
memories 202, 204, and 206. ~e SEL 208 is of any well known design. The S~N 202, 204 and 206 is a set o~ three random access memories of a well known design. If the translated value of a record character is zero and the MV flag is set, then the entire current column~is lgnored as above. MV 222 is a one bit latch ff a well known design. This permits the definition of missing .
.: :
, : ,, ~ : ~ - , ,:
. .. ~ , . .

3L~5~

value fields so as not to detract from the measure of the similarity between the record and the query.
Position bus 224 is connected to SEL 20~ and PW
226 that is a random access memory of well known design. The enhancements we have discussed so far constitute simple preprocessing and are not crucial to the basic invention. We now discuss some more crucial enhancements.
In the circuit of Figure 4 you will note that the quantity "1" is always added to R 107.
This ~as the effect of weighing all alphabet members and column positions equally. Figure 5 implements a weighing scheme wherein the column number currently under consideration and the current character are used to determine a weight which is to be added to R instead of "1". In this way, one can weigh characters heavier than others and one can weigh columns heavier than others. GeneraIly speaking, many circuits might compute a weight to be used to update R. Figure 5 contains one such circuit. In this circuit, each alphabet character is assigned a two bit weight. This weighs 0, 1, 2, 3 are possible.
Each column position is also assigned a weight also two bits. But this weight'is used to control a shift reyister so that'here,,the possible weights are 0, 1, 2, 4. The characte'r weight is effective-ly multiplied by the column weight to arrive at the final weight. A complete word associator circuit must of course contain two copies of the circuit of ~igure 5. We observe that the positional weight '' t-ables defined For each copy mig~t differ. This might allow certain columns to be processed with , . . : . : :: :, - . , ~ .
; ~
':~ ': " ":, ' ~3l5~87~

more emphasis on initial or on final characters.
When the two tables agree t there is no such directional bias, It should be noted that when the final weight of a column/character pair is zero, the character is not processed. This is accomplished by the "single skip on ~ero" circuit 230 which blocks propagation of timing singals for the duration of a single character.
The weight scheme is implemented by the random access memories PW 226 and CW 242 of well known design and by the selectable shift register 244 formed by the distributor 246 and gate components 248, 250, and 252 and by the single skip on zero circuit 230, all of which are of well known designs.
In the circuit of Figure 4, the final result M needed to divide by N(N+L) where N is thé length of the words under comparison, to arrive at the measure of similarity. In the circuit of Figuxe 5, this denominator must be computed since it will depend upon the weights encountered during processing.
In Figure 5, TOTR 254 and TOTM 256 that are add-latches, compute a denominator term. A correspond-ing term is computed by the other copy of the circuit. The sum of these two terms is the final denominator. The final numerator is twice the sum of the,two M values read out,o~ the two circuit copies. The similarity betw~en the query and bus words is the quotient of the numerator and the denominator quantities. The above is just one way in which the information read out of ~he circuit may be interpreted to arrive àt a measurement of simpilarity. Other schemes mig~t-weigh various terms unequally. The circuit 112' in the lower right of : . - ~. . .
., ~ - . ~ - :

Figure 5 is xecognizable as very sim~lar to the circuit of Figure ~. The only difference is -that the selection component is llOW external and R
may now be updated by quantities other than "1".
The circuit of Figure 5 is divided by dashed lines into three stages designated by I, II and III. Note that busses passing from stage to stage are broken. This indicates that buEfers might be inserted to achieve a pipeline with three stages.
In this way the circuit can process data as fast as the circuit of Figure 4. Without pipeline buffers, the circuit is two to three times slGwer.
The timing signals are labeled identically in each stage but may vary from stage to stage both because of the optional pipeline and because of the skip on zero circuits.
The circuit of Figure 5 must be initialized before use. A master initialization must be performed once per search to establish weights, etc. This initialization must load the SEL 208, SYN 202, 204 and 206, PW 226, and CW 242 memories.
Also, the MV 222 flag must be set or reset.
Before each record is processed, additional initialization is required. The tally memory in 112' not illustrated must be,set to zero as must the R and M, not shown, and T!QTR 254 and TOTM 256 add-latches. After each recdrd is processed, the contents of M, not shown in 112', and TOTM 256 are read out of the circuit.
Connections describing initialization and readout are trivial and of well known design and are therefore not shown in the ~a~ings.

. . . ~ . ~

:, : : .::: : ..

Finally, note that in the circuit of Figure 5, the bus data character must be stable on the bus even during the processing of the query character. This permits the bus character to be translated while the query character, which does not pass through synonym translation, is processed.
The query translation is better left to software since the query is fixed during a search.
The instant invention has been shown and described herein in what is considered to be the most practical and preferred embodiment. It is recognized, however, that departures may be made therefrom within the scope of the invention and that obvious modifications will occur to a person skilled in the art.

Industrial Applicability . .
The associator or comparator circuit shown in Figure 4 may be constructed in electronic chip form for use as a circuit allowing inexact word queries in any system of other. Further this associator or comparator circuit may be utilized in an associative retrieval system as shown in Figures 1, 2 and/or 5. Or other industrial appli-cations as set forth hereinabove.
-' ` ' >
"~. :

. . .
, , .: . :, ~ ~

'- ` . - . : :~. . . ,:
.. ~

Claims (4)

THE EMBODIMENTS OF THE INVENTION IN WHICH AN EXCLUSIVE
PROPERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS:

1. A word comparator device for providing a numeric measurement of the degree of word similarity between each set of compared words in order to retrieve inexact queries in a quick and expeditious manner, comprising:
a selecting means having at least one input and an output, said selecting means for addressing and routing either query words and record words to an output at different times; control timing circuit means for selecting either the read or write mode and for defining whether a query word or record word is being processed, said control timing circuit connected to said selecting means for controlling the input of query words and record words; a read/write memory means having an input, output, and updating input, said input is connected to -the output of said selecting means, said read/write memory means for storage and retrieval of numeric information addressed by the output of said selecting means;
said control timing circuit connected to said read/write memory means, depending on control parameter means; an adder means including an input connected to said read/write memory means output and an output, said adder means for incrementing or decrementing the input for later control use and updating of said read/
write memory means; said control timing circuit connected to said adder means; latching means connected to the output of said adder means, said matching means for updating said read/
write memory means with the output of said adder means; said control timing circuit connected to said latching means; a tester means including comparator means, said tester means for deciding if the output of the adder means is non-positive or non-negative to produce an output; said control timing circuit connected to said tester means; and output latch means connected to said tester means, said output latch means for providing a numeric measurement of the degree of word similarity between the record word and the query word.

2. A word comparator device as set forth in Claim 1, wherein: said selecting means is a data selector means; said control timing circuit means include a query word, record word clock circuit, and a read/write clock circuit and operating at twice the speed of said query word/
record word clock; said read/write memory means including a random access read/write memory; said adder means including an incre-menter/decrementer in which one of the summands is restricted to plus or minus one; said latching means is a positve edge triggered data latch; said tester means is a convertible sign tester for determining if the input is non-positive or non-negative said output latch means includes a clearable edge triggered latch and an add-latch; said triggered latch connected to said adder means and said add-latch connected to said triggered latch.

3. A word comparator device as set forth in Claim 1, wherein: said word comparator device for comparison of a record word from right to left; a second word comparator device at least similar to said word comparator device with an output for comparison of a record word from left to right;
a memory storage means; a query storage means; an interface connected to said query storage means and said memory storage means; said interface connected to said word comparator device and said second word comparator device; and an adding means for producing a system output connected to said output latch means of said comparator device and said output of said second comparator device.

4. A word comparator device and word comparator system as set forth in Claim 3, wherein: said system output provides degree of word similarity in accordance with the formula:

wherein A is an alphabet, w is a word in A, x and y are numbers and (x,y) denotes the greater of the two quantities x-y and O, n(a,w,i) denotes the number of occurances of alphabet member "a" in word w found in position i or beyond where position is measured canonically from left to right, w and v are words in A, both of length L, and the degree of word similarity between them is S(w,v).