RECORDING OF TAGGED OBJECTS IN A CLOSED ENVIRONMENT Background of the Invention The invention relates generally to reliably recording tagged objects having access to a closed environment, and more particularly, to a method and apparatus for identifying, through an alphanumeric tag, objects entering and leaving a closed environment such as a parking lot, container terminal, toll road, etc. Numerous systems have been developed which attempt to read, exactly, a tag, such as a license plate number, associated with a vehicle, and compare that read number to a database for determining various information about the vehicle. Such systems have been used for controlling access and egress at the borders of the United States, and have also been used to identify vehicles entering and leaving closed environments such as a parking lot in order to automatically assess use fees. By "closed environment" is meant a bounded environment or space having a limited number of entry and exit points.
These systems typically attempt to read the tag associated with the vehicle. In particular, if the tag is not easily identifiable, various methods have been used to provide a best estimate of the tag's text. Accordingly, an object of the invention is a method and apparatus for tracking, tagged objects, such as vehicles entering and leaving a closed environment where identification of the alphanumeric characters constituting, a tag identifier, for example, a license plate, need not be precisely determined.
Other objects of the invention include the ability to reduce theft of vehicles from a closed environment by associating vehicle identification tags with parking tickets cards, etc. , provided and identified upon entry into a parking area, and also to verify revenue in a
closed environment, such as, for example, a toll highway or a parking area.
Summary of the Invention The invention relates to a method for accounting for tagged objects in a closed environment and features the steps of visually sensing a tagged object entering the closed environment, capturing an image of the entering object, the image including at least an entering object identification tag having a sequence of alphanumeric characters, and extracting, from the entering object identifier, an entering identifier designation representing, as completely as possible, the sequence of alphanumeric characters on the object identifier. The method further features storing the extracted entering identifier in a list of stored extracted identifiers, visually sensing a tagged object exiting the closed environment, again capturing an image of the exiting object, the exiting object image including at least an exiting object identification tag having a sequence of alphanumeric characters, and comparing the exiting object identifier to the stored list of extracted identifiers to determine a best match between the exiting object identifier and an identifier on the list. The exiting object is then identified as the object associated with the best match on the list. The comparing step features the steps of matching a succession of alphanumeric characters from the exiting object identifier with each sequence of alphanumeric characters represented on the list and scoring the success of each such match, the best match being a match having a best score.
In a preferred embodiment, the scoring step features the steps of assigning a likelihood value to each identified match of each pair of groups of
alphanumeric characters and multiplying the likelihood scores to obtain a score for each pair of groups. In other embodiments of the invention, the invention features the method of verifying the revenue derived from objects entering and leaving a closed facility by visually sensing a tagged object entering the closed facility, capturing an image of the entering object, the entering object image including at least an entering object identifier having a sequence of alphanumeric characters, extracting from the entering object identifier an entering identifier designation representing the sequence of alphanumeric characters, storing the extracted entering identifier designation in a list of stored extracted identifiers, visually sensing a tagged object exiting the closed environment, comparing the exiting object identifier to the stored list of extracted identifiers to determine a best match between the exiting object identifier and an identifier on the list, and storing the match, including a time interval for which the exiting object was in the closed facility, for verifying the revenue derived from the closed facility.
In yet another aspect of the invention, the identification system can be employed for identifying objects from among entries in a list stored in an electronic memory. In this aspect, once a representation of an identifier tag on an object has been obtained, and a best match of the object identifier determined, the object associated with the best match is selected as the object being sought.
In yet another aspect of the invention, the method further features associating each entering object designation with a second identifier not associated with the object, and automatically closing a transaction when the best match in the list is designated, by associating
the object identification provided by the matching process with the second identifier, and verifying the correspondence of the second identifier and the object exiting the environment. In this manner, thefts of vehicles from parking areas, such as airport parking, can be reduced by electronically associating the vehicle license plate with the second identifier associated with the vehicle when it enters, and preventing the use of other second identifiers in connection with the vehicle at the vehicle exit. An electronic system correlating the second identifiers and the license plate, will prevent the exiting, from the parking area, of vehicles having incorrect second identifiers.
In another aspect, the invention relates to a method for comparing a first and a second alphanumeric character stream and features the steps of performing an exhaustive ordered search character by character, of the characters of the respective alphanumeric strings, and identifying the maximum number of character matches in the strings. The method further features identifying, as a match, selected known mismatches of characters between the two strings, assigning a likelihood value to each character match, mismatch, deletion, and insertion, in each pair of alphanumeric character strings being compared, and multiplying the likelihood scores of a continuous sequence of characters to obtain a score representing the match between said pair of character strings.
Brief Description of the Drawings Other objects, features, and advantages of the invention will be apparent from the following description, taken together with the drawings in which:
Figure 1 is a schematic block diagram of a system in accordance with the invention;
Figure 2 is a flow chart illustrating the operation of the method of the invention for tracking entering and exiting vehicles in a closed environment; Figure 3 is a flow chart of an alternate embodiment of the invention in which receipt verification is imp1e ented;
Figure 4 is a flow chart of an alternate embodiment of the method of the invention in which vehicles are tracked to prevent theft; Figure 5 is a flow chart of an alternative embodiment of the invention in which toll charges can be automatically billed; and
Figure 6 is a graph showing an example of partial matching in accordance with a preferred embodiment of the invention.
Description of the Preferred Particular Embodiments of the Invention While the invention relates generally to the recording of events relating to objects having associated therewith an alphanumeric tag, and having access to a closed environment, in the particular applications which follow, the objects are vehicles, and the alphanumeric tags are described as license plates attached to the vehicle. This description is not to be considered as limiting the scope of the invention which is applicable to any type of object having an alphanumeric tag and which is found in a closed environment.
Accordingly, referring to Figure 1, in a typical application of the invention, vehicles 10 enter and leave a closed environment 12 such as a parking lot, container terminal, airport, toll highway, or other bounded closed environment or space having a limited number of entry and exit points. Upon entering the closed environment at 14, a camera 16, for example a video camera, acquires an image of a vehicle identifier. The vehicle identifier
can be any of a number of tags, but typically, for a vehicle, is a license plate 18. A similar event takes place when the vehicle 10 leaves the closed environment at 20. A camera 22, positioned in the manner comparable to that of camera 16, acquires an image of the vehicle identifier 24. The object of the invention is to provide a matching capability to identify the vehicle leaving the closed environment as the same vehicle as entered the closed environment. In performing this function, the system is then able to determine the length of time that the vehicle remained in the closed environment as well as to account for vehicles within the closed environment in an orderly and logical fashion.
In accordance with the invention, the output of camera 16 is directed to a digitizing element 30.
Digitizing element 30, which can be an analog-to-digital
(A-D) converter, frame grabber, or digitizer, followed by an appropriately programmed computer system, acquires an image from camera 16. The image includes the license plate 18. The output of digitizer 30 is directed to an analyzer 32, which can be implemented in a general purpose digital computer, and whose task it is to extract from the image an alphanumeric string representing the license plate 18. The output of the analyzer 32, the alphanumeric string, is stored electronically in a character string storage medium 34. The character stream storage memory can be, for example, the disk drive of a computer system, or computer RAM.
When a vehicle exits from the closed environment, a digitizer 36, comparable to digitizer 30, acquires and digitizes an image of the vehicle, including the license plate 24 of the exiting vehicle. The output of the digitizer is passed to an analyzer 38, comparable to analyzer 32, which extracts from the image an identification of the exiting vehicle by determining an
alphanumeric string which represents the license plate 24 of the exiting vehicle. The alphanumeric string from analyzer 38 is sent to a matching circuitry 40. Matching circuitry 40 also has access to the stored character strings representing entering vehicles and their vehicle identifiers, and the matching circuitry 40 defines a best match between the extracted identifier of the exiting vehicle 10 and the extracted identifiers stored in memory 34. (It should be noted that in some embodiments, the stored list may be preestablished and the match is made to find the closest identifier on the list.) The matching circuitry is preferably implemented in software in an appropriately programmed computer system in accordance with the description which follows. Once the best match has been found, the best match information, that is, for example, the duration of the vehicle in the closed environment, and other stored parameters and data regarding the vehicle, (or the vehicle identification itself) are processed by the best match processing circuitry 42. The best match processing circuitry 42 is also preferably implemented in software on a general purpose digital computer.
In various applications of the invention, the best match processing 42 can take several different forms. Referring to Fig. 2, in a parking lot system, the best match processing can determine the time duration that the vehicle was in the parking lot (block 44) and can provide, therefore, the charge for parking (block 46) . In another embodiment, referring to Figure 3, where a ticket and ticket number (or other identification indicia (such as credit card number, transponder identification, access cards, etc.) unassociated with the vehicle itself) are associated with the vehicle as it enters, the best match processing of Figure 2 can be extended, and can provide verification (for example, by ticket number) of
the charges assessed by a parking lot attendant when the vehicle exits the parking lot (block 46) . At the end of an accounting period, such as the end of the day, the system can provide a verification of the receipts reported by the parking lot attendant (block 50) . In yet another embodiment of the invention, referring to Figure 4, where a ticket, having a ticket number imprinted on it, or another identification indicia as noted above, is provided to or associated with the entering vehicle, the ticket number or other indicia is recorded by the system (block 52) , and the system can verify that the vehicle, when it leaves the system, such as an airport parking lot, is associated with the same ticket or indicia as was provided by the attendant or otherwise associated with the vehicle at entry (block 54) . This can decrease the number of stolen cars taken from parking lots, generally, and airport parking lots in particular. An alarm can be sounded if there is a mismatch. Referring to Figure 5, in another application of the invention, cars entering and leaving a toll road can be identified and the charges for using the toll road can be made available by identifying the entering location (block 58) and determining the toll charge (block 60) . Then, the charge can be paid either to the toll road attendant, when the car exits the toll road, or can be billed automatically to the vehicle (found in a list of preregistered vehicles) in accordance with a preregistration process so as to speed up egress from the toll road (block 62) .
In a perfect world, the alphanumeric character string found by the digitizer and analyzer 30, 32, respectively, when the car enters the closed environment, will identically match the license plate of the vehicle. Similarly, when the car leaves the closed environment the
alphanumeric string obtained by the digitizer and analyzer 36, 38 will again identically match the alphanumeric string on the license plate of the vehicle. The matching process is then a simple one. In practice, however, the license plate on a vehicle may make it difficult to accurately extract all of the alphanumeric identifiers on the vehicle identifier. Thus, there may be a variety of occlusions, for example trailer hitches or piled snow, or dirt which obscures the readability of characters. Lighting variations and deterioration of the identifier substrate material (that is the paint) can also adversely affect the accuracy of the system. In these cases, which are most common, part of the identifier string is misread or missed, and accordingly only a part of the string is accurately identified.
Due to these factors, in some instances, the entering identifier for a particular vehicle will not exactly match, according to the system, the exiting identifier for the same vehicle. However, a high degree of similarity between the two strings will provide sufficient information to enable the system to associate the two strings using a partial matching method as described below. In this manner, accurate records can be maintained even in the case of degraded reader performance.
In accordance with the invention, therefore, when an exiting identifier string is determined by analyzer 38, the task is to find the one string from a list of known vehicles stored in memory 34 that have been entered into the system, that matches the exiting identification string. Thus, the two strings, one from memory and one from analyzer 38, must be associated with the same vehicle and must be identified as well as possible. In mathematical terms, the problem can be stated as follows:
Given an alphanumeric string E (the exiting identifier) , what is the best search procedure through a list of strings L = {Si s2 s3...sn} (the stored entering identifiers) in order to find the best match s* to the string E given that there may not be an exact, one to one, match?
In accordance with the invention, the first step effected by the matching circuitry 40, is to find the string s in memory 34 having the largest number of ordered matching characters to the string E. This alone may be sufficient to find a unique string s* from the list L of strings stored in memory 34, if one such string gives a maximal number of character matches. However, in the case that a plurality of strings are found each of which has the same maximum number of matching characters, than a second step using a probabilistic approach is required in order to produce the single best match as s*. Partial matching requires that some of the characters in E match exactly with those in s while preserving an ordering of the characters from left to right. For example, with the strings
E= ABC123 and SL = ABG1Z3. (1)
There is a degree of partial matching evidenced by the characters A, B, 1 and 3 which occur in order. In the case of the strings
E = ABC123 and sL = 321CBA, (2) while there is a direct individual match between each of the characters in E and sif it is impossible to match more than a single character while preserving the left to right ordering of the character sequence. As a result, for the strings in (2) , at most only a single (non- unique) character match can be recorded.
When attempting to find a string s* that best matches E, the method first finds all strings si which provide the largest number of left to right ordered character matches. In each case, an exhaustive graph-
search strategy is used to find the largest number of matching characters in a left to right order. Given E and Sj^, a graph is represented by each pair of matching characters when comparing the two strings. The search method then finds the largest number of matching pairs for each starting (and matching) pair, given that the ordering from left to right must be preserved. Consider the following example.
E = RDFRET and S = FDRET (3)
Referring to Figure 6, the labels 50, 52, 54, 56, 58 and 60 correspond to graph connections between matching string characters (indicated by the solid lines) . To identify the largest number of matching characters in the strings, for a starting pair, the method starts with the left-most matching pair (in this case 50) , and then, moving to the right, finds all other matches that are consistent, based on a left-to-right ordering. (Note the intervening, non-matching characters are ignored.) In the above example, connection 52 is not consistent with connection 50 since it does not preserve left to right ordering (D follows R in the string E, yet it comes before R in the string s^) . The only other connections which are consistent with connection 50 are connections 58 and then 60. Since no other consistent matches may be added in this case, the longest complete path starting with the connection 50 consists of matches between the characters R E and T. Other complete paths are found by starting with the connection to the right of 50. For example, starting with connection 52, the connection paths 52, 56, 58, 60 can be constructed.
Connection 54 is not consistent with a path starting at 52 since it does not preserve left-to-right ordering. The complete path starting at connection 54 is given by connection paths 54, 56, 58, 60. Other paths starting at
connections 56 and 58 are also possible, however, since they are already contained in the match found starting at connections 52 and 54 they need not be considered.
This example demonstrates how a graph search process can proceed. In general, it is possible to have multiple paths which start from any given graph connection. A classical Artificial Intelligence technique - known as a Depth-First-Search strategy can be used to enumerate all valid graph paths and select the longest one. In this example, the two longest paths correspond to the connections 52, 56, 58, 60 and 54, 56, 58, 60.
In this example, there are a total of m==4 character matches in each of the two strings. After scanning through the list of all possible strings L, if a single string s is found such that the number of matches m is equal to the number of characters in both strings (E and si) , then there is a perfect (one- to-one) match, and the string s* = s^ is selected as the best match. In the case of incomplete matches, however, the number of matching characters (m) will be smaller than the number of characters in either E or s^ If there is a unique string that generates a maximum value of m, then it is chosen as the closest match for E. Otherwise, all strings in L which give rise to the same (maximum) number of matching characters (mroaχ) are selected and processed further as described below.
If multiple matching strings are found after this first stage of the method, then a second stage of processing is required. Starting with the exiting identifier string E, a second matching criterion uses a probalistic approach to identify a single best matching string s* from the set R (the set of strings that give rise to scores of 3 ^) .
As noted above, due to a variety of factors, there can be differences between the set of entering identifier strings L that are automatically read by the system, and the precise set of actual identifier strings A associated with vehicles in the closed system. For example, as noted above, differences can arise when the material of a given vehicle identifier is extremely dirty or degraded, or when there are occluding factors such as piled snow or a trailer hitch. The result is that one or more characters may be either misread or missed entirely by one or both of the analyzers 32, 38. By considering the common types of reading degradations, in accordance with the invention, probabilities can be assigned to quantify the likelihood of each type of degradation. When comparing the exiting identifier string E to the subset of entering identifiers R (having a maximum character match m.,^) , the best match is then the one with the most likely set of associated degradations.
Four types of degradation are considered (although others may be possible and included in the method) in the manner described below. The four degradations are: i. Character Deletion ii. Character Insertion iii. Random Character Misread iv. Typical Character Misread
A combination of these degradations can explain all differences between the set of actual identifiers A and the set of machine read entering identifiers L stored in memory. Each degradation is described in more detail below. i. Character Deletion. This occurs when one or more characters are missed in the reading process. This may arise for a variety of reasons - such as occlusion by an object or by severe soiling or degradation of the vehicle identifier material.
ii. Character Insertion. This occurs when the reading system inserts or adds a character which was not in the actual identifier string. iii. Random Character Misread. This occurs when the reading system misreads a given string character and replaces it with the unpredictable result. iv. Typical Character Misread. There is a class of character misreads that occur due to the similarity between various characters. Depending on the font type, there may be very subtle differences between characters such as D and O, or B and 8. As a result, under realistic imaging conditions involving, for example, dirt and variable lighting or low sensor resolution, the reading system may confuse characters that have similar forms. (Note that equivalences between characters that are commonly misread can also be incorporated in the first graph matching stage described above.)
Each of these reading degradations is assigned a probability based upon its frequency of occurrence (found in a series of experimental trials using a particular analyzing system.
Given the exiting identifier string E and a candidate matching string s^ the matching circuitry 40 compares the two strings to determine the minimum number of degradations that explain how one string can be transformed into the other string. Since each degradation has an assigned probability, the overall probability of making a set of independent degradations (the degradations are considered to be independent) is given by the product of the probabilities of each.
Thus, starting with the subset of strings R (from the first matching stage) , each string sL in R is compared with the exiting identifier E.
In example (3) above, the two longest paths found in the associated graph correspond to an ordered matching
of either the characters DRET or the characters FRET. In the first case (DRET) , assuming that the strings E and s arose from the same actual identifier string, one possible matching scenario is explained as follows. The first character in the actual identifier was misread, giving rise to two different first character interpretations (R and F) in the strings E and sL respectively; the second character was read correctly as a D; and the third character F was deleted when read on entry. Hence it is present in the string E but not in the string si.
This interpretation is illustrated in Fig. 3 where matching characters are indicated by joining lines.
If the probability of deleting a character is assigned a value Pd and the probability of random misreading a character is Pr, then the overall probability associated with this first interpretation is Px = Pd X Pr.
A similar interpretation is suggested by the insertion of an F into the string E. If the probability of insertion is assigned the value Pif then the overall probability in this second case would be P2 = P^ x Pr. This interpretation is illustrated in Fig. 4.
One final interpretation depends upon the ability to distinguish the characters R and F. For example, the reading system may have some known difficulty in distinguishing the characters R and F. Based on performance observations, an associated probability PR_F is assigned to rank the likelihood of such misreads in practice. This is an example of the fourth type of degradation (a typical character misread) . Assuming that the F in the string E was deleted when read on entry, then the overall probability of this third matching scenario (which confuses R's and F,s) is given. A similar analysis can also be made for the other matching
scenario (FRET) found in example (3) . To automate this process, a depth-first search algorithm is employed, as is well known in the art, in order to enumerate all possible degradation scenarios. For a given pair of strings E and sL, the most likely degradation (the one having the highest value P^ = max {Pj^, P2, P3...}) is selected.
By performing this process for all possible strings s in the subset R, a single likelihood value P is computed in each case. The string s^ having the largest overall value P* = max {Pi> is then selected to represent the best match s* for the exiting identifier E.
Consider now a numerical example of how a probabilistic strategy is used to find the best match. For a particular system, assume the following four types of optical character reader (OCR) misclassifications can occur with the associated probability likelihoods: a. D misread as an 0 - likelihood PD_0 = 10% b. B misread as an 8 - likelihood PP_8 = 12% c. Character Deletion - likelihood Pd = 2% d. Random Misread - likelihood Pr = 1% Then, given the exiting identifier string E as DOW 187 and two entering identifiers: s as DDW127 and s2 as D45W187, the string probability likelihoods (recognizing that m equals 5 for the two strings (the misreading of "0" for "D" being treated as a recognized error for this reader and being counted as a match for purposes of the first part of the matching process) ) can be determined as follows. (Note that the probability assigned to an exact character match is "1".) Comparing E and s^
D matches D
D has been misread as an O (type a misread = 0.1)
W matches W 1 matches 1
8 does not match 2 (type d misread = 0.01)
7 matches 7
Accordingly, the associated probability: P = 0.1 * 0.01 = 0.001 Similarly, comparing E and s2: D matches D
O is misread (type d misread = .01) either 4 or 5 are deleted (type c deletion = .02) W matches W l matches 1
8 matches 8 7 matches 7
Then the associated probability P = 0.02*0.01 =
0.0002 By choosing the highest probability score (.001), the string s is chosen as the best match for the string
E.
Additions, subtractions, and other modifications of the preferred embodiments of the invention, to other tagged, moveable objects in closed environments, will be apparent to those practiced in the field and are within the scope of the following claims.