AU2005203141A1

AU2005203141A1 - A method of detecting streams of postal items

Info

Publication number: AU2005203141A1
Application number: AU2005203141A
Authority: AU
Inventors: Belkacem Benyoub; Matthieu Letombe
Original assignee: Solystic SAS
Current assignee: Solystic SAS
Priority date: 2004-07-20
Filing date: 2005-07-19
Publication date: 2006-02-09
Anticipated expiration: 2025-07-19
Also published as: ATE432508T1; FR2873469B1; FR2873469A1; EP1622065B1; NO20053471D0; DE602005014588D1; PT1622065E; NO20053471L; ES2327853T3; EP1622065A1; AU2005203141B2; NO335868B1

Abstract

The method involves forming a digital image of a surface of each postal item having information of address, and binarizing the digital image. Subsidiary components (CC) of the image are extracted from the digital image of the items to perform automatic recognition operation of the address by optical code recognition (OCR) and to distinguish a group of homogenous postal items. An independent claim is also included for a sorting machine of implementing a postal item processing method.

Description

I

AUSTRALIA

Patents Act COMPLETE SPECIFICATION

(ORIGINAL)

Class Int. Class Application Number: Lodged: Complete Specification Lodged: Accepted: Published: Priority Related Art: Name of Applicant: Solystic Actual Inventor(s): Matthieu Letombe, Belkacem Benyoub Address for Service and Correspondence: PHILLIPS ORMONDE FITZPATRICK Patent and Trade Mark Attorneys 367 Collins Street Melbourne 3000 AUSTRALIA Invention Title: METHOD OF DETECTING STRAS OF P L A METHOD OF DETECTING STREAMS OF POSTAL ITEMS Our Ref: 747915 POF Code: 469254/458825 The following statement is a full description of this invention, including the best method of performing it known to applicant(s): -1- 600eq A METHOD OF DETECTING RUNS OF UNIFORM MAIL ITEMS The invention relates to a method of handling mail items, which method comprises forming a digital image of each mail item, which image includes address information, and performing an automatic address recognition operation by optical character recognition (OCR).

The invention relates more particularly to a mail handling method that makes it possible to detect "runs".

A "run" is typically a succession or a group of uniform mail items, often a very large number of uniform mail Citems, having visible physical characteristics that are identical. The mail items come from the same mail sender and are addressed to different destinations. Such mail items can be referred to as "major user" mail items or as "referenced customer" mail items.

Figure 1 very diagrammatically shows the mail items E of a run F. Figure 1 shows two mail items E more precisely, and it can be seen that the front faces E of said two mail items bear address information. In its top left corner, each of the two mail items also bears the name and address of the sender, indicated by reference x, in its bottom left corner, it also bears the name of the group (indicated by reference y) to which the sender belongs, and, in its center, slightly offset rightwards, it bears an address block indicated by the reference z, the address block being disposed in a framed region under a plastics film. All of the mail items E of the run F are constituted in the same manner, and each of them bears the elements x, y, z that are characteristic of the run F and that are placed in the same positions on the front faces of the mail items. Only the destination address in the block z of the mail item E changes from one mail item E to another in the run F, as can be seen in Figure 1.

Currently, runs are identified visually by the operators while they are feeding sorting machines with mail items.

2 More particularly, when an operator identifies a run, the operator can adjust the parameters of the automatic OCR recognition process for recognizing the destination address on the sorting machine. But often that parameterization is not optimum and is not used because it requires expertise in the use of OCR that the operator generally does not possess. When the operator does possess the expertise necessary for such manual parameterization, said parameterization is generally performed only on very long runs, i.e. runs made up of Sthousands of mail items, because implementing it requires actions of measurement, of optimization and of data input that slow down overall operation of the sorting machine.

The OCR parameters can, for example, be adjusted in the manner presented in European Patent Document EP 1 159 705.

Generally, mail items constituting runs are sent to post offices by referenced customers who enjoy "discount" contracts. Discount contracts make provision for a reduction in the postage charge to be granted that is a function of the outward sorting rate of reading and of the inward sorting rate of reading of the addresses by OCR. Generally the rates of reading are determined by means of test batches of mail items at the time of signing of the contract, and they are not checked throughout the entire term of the contract, because determining rates of reading is lengthy to perform and thus costly. Thus, post offices do not have any means of checking that the performance levels specified in the contract are indeed complied with by the referenced customers.

An object of the present invention is to provide a method of automatically detecting runs for the purposes of automatically parameterizing the OCR address reader and of automatically monitoring the discount contracts.

To this end, the invention provides a method of handling mail items, which method comprises forming a digital image of each mail item, which image includes address information, and performing an automatic address recognition operation by OCR, said method being C characterized in that it further comprises processing for automatically identifying groups of uniform mail items or "runs", which processing comprises a step consisting in extracting image attributes from the image of the mail M item, which attributes are suitable for distinguishing a (N group or "run" of uniform mail items.

In a particular implementation of the method of the Cinvention, the processing for identifying runs comprises a step consisting in extracting closely related components from the image of the mail item.

These closely related components are already extracted for performing the OCR address recognition.

With the method of the invention, the same information is thus used to detect runs and thus to add a new feature in simple manner to an automatic mail sorting process.

With the method of the invention, the OCR automatic address recognition system can be adjusted automatically each time a new run is identified. In the same way, with the automatic run identification of the invention, it is possible to organize systematic and automatic determination of the rate of reading of the identified runs for the purpose of managing discount contracts in post offices.

In another implementation of the method of the invention, runs are listed in a catalogue in the form of a set of closely related components that are predetermined for each run. Thus, during processing of a current mail item, the closely related components extracted from the digital image of the current mail item are compared with the predetermined closely related components of the runs in the catalogue for detecting whether or not the current mail item belongs to one of the runs.

SIn another implementation of the method of the invention, the processing for identifying runs comprises a step consisting in extracting gray scale image C attributes from the image of the mail item, and the runs are listed in a catalogue in the form of gray scale image attributes that are predetermined for each run.

In another implementation of the method of the Minvention, the catalogue contains OCR adjustment N parameters associated with the runs, which adjustment parameters improve automatically.

In another implementation of the method of the invention, a current mail item is identified as belonging to a run, and the digital image of the current mail item is recorded in a database with the results of the OCR address recognition operation for the purpose of checking the rate of reading for the run.

The present invention also provides a mail sorting machine specially arranged to implement the above-defined method.

The invention will be better understood on reading the following description and on examining the accompanying figures. The description is given merely by way of indicative example and is in no way limiting to the invention. In the figures: Figure 1 shows two mail items belonging to a run; Figure 2 is a very general flow chart showing run detection in the OCR address recognition process; Figure 3 is flow chart of the method of the invention for detecting a run; Figure 4 shows in more detailed manner the process of Figure 3 for comparing closely related components: Figure 5 shows the catalogue of Figure 3 in more detail; Figure 6 is a detailed flow chart of the various steps making it possible to decide that a run has been detected and to determine the actions to be taken; Figure 7 is a very general flow chart showing another implementation of run detection; and Figure 8 is a diagram showing extraction of attributes from a gray scale image.

Figure 2 very diagrammatically shows an implementation of the method of the invention for automatically detecting or identifying runs in an automatic OCR address recognition process.

The process of automatically handling a mail item in a sorting machine includes automatic address recognition Cby OCR, whereby a digital image 1 is formed of the surface of the current mail item bearing address information. The digital image of the mail item is binarized at 2, and, at 3, closely related components (CCs) are derived from the binarized digital image. The composition of said closely related components is described in more detail below. The closely related components (CCs) of the mail item are then used to make it possible to locate the address block, and the rows and characters of the postal address, and serve for automatic address recognition 4 by OCR.

The method of the invention also uses closely related components extracted at 3 as data suitable for distinguishing at step 5 a group or run of uniform mail items. As shown in Figure 2, run detection 5 can be performed in parallel with OCR automatic address recognition so that the process loops back at L for each subsequent mail item with the same result as for the run detection. The information obtained on the basis of the run detection and on the basis of the OCR address recognition performed on the current mail item is used during the OCR processing process performed on the subsequent mail item and during the associated run detection.

Figure 3 shows the run detection 5 of Figure 2 in more detailed manner and starting from step 6. Figure 3 also shows the forming 1 of the digital image of the surface of the current mail item, the binarization 2, and the extraction of the closely related components 3 from said digital image. In step 6, the closely related C components of the current mail item are compared with the closely related components of models that are prerecorded and listed in a dynamic catalogue 7 stored in a memory.

The term "model" is used to mean a set of predetermined closely related components that define (characterize) a (N run. The process 6 of comparing the closely related components is described in more detail below with reference to Figure 4.

When, at 6, it is found that the closely related components of the current mail item match 8 the closely related components of a model from the catalogue, the model in question is enriched 9 with the closely related components of the current mail item so as to consolidate the robustness of the model. When it is found 10 that the closely related components of the current mail item do not match the closely related components of any of the models of the catalogue 7, a new model is created 11 in the catalogue 7, e.g. by overwriting an old model, the closely related components of the new model then being the closely related components of the current mail item.

It is easily possible to imagine associating each model in the catalogue with a model identity number which is kept in a memory of the OCR 12. The OCR outcomes determined in step 4 can also be recorded in the OCR memory 12 in connection with the identity number of a run model.

In the method of the invention, a decision module of the OCR repeatedly analyzes the OCR memory 12 so as to detect a run, and decides at 13 whether to execute functions resulting from runs being detected.

More particularly, when the decision module has decided at 13 that the current mail item belongs to a current run, it can undertake actions 14 to optimize automatic address reading by OCR. When it is detected O that a current mail item is part of a run, the module 13 can also, at 15, cause information to be stored in a database 16 on the current mail item and optionally on C the preceding mail items also belonging to said current run for the purposes of monitoring major user discounts.

An implementation of the method of the invention is thus based on using the closely related components as image attributes suitable for distinguishing a group of uniform mail items so as to characterize runs. In a manner known per se, a closely related component in a Cbinary image is constituted by all of the adjacent black spots that can be associated immediately adjacently with one another without going via a white spot. In Figure 1, two closely related components 17, and "Raspail" (in joined-up writing), each of which respectively matches every one of the mail items E of the run F, are highlighted by being boxed-in by respective dashed-line frames 18. The extraction 3 of the closely related components from a binarized digital image thus consists in identifying each of the closely related components in a table of closely related components by a plurality of criteria, e.g. by the position of the rectangular frame 18 surrounding the closely related component 17 in the image, by the length and the width of the frame 18, by the density of black inside the frame 18, and by the number of closely related pixels.

Figure 4 is a more detailed flow chart of the process of comparing 6 the closely related components of Figure 3 so as to determine whether a current mail item can be associated with a model from the catalogue 7. At step 3, all of the closely related components in the image of the current mail item are extracted. Of all of said closely related components, only the most significant are kept, by eliminating the others in step 20, e.g. only the one hundred having the largest number of pixels are kept. The elimination at 20 of the smallest closely related components makes it possible to v accelerate the process of comparison at 6 and also sometimes to eliminate the destination address, which address is generally different on the different mail C items belonging to a run. At 21, each discriminating closely related component of the current mail item is compared with each discriminating closely related component of each of the models. For this purpose, a M cascade of filters are used with the various criteria of the table of closely related components, a compatibility threshold being given for each of the criteria.

For example, firstly the positions of the frames surrounding the closely related components are compared, and then, in the event of success relative to the chosen compatibility threshold, the lengths and widths of the frames are compared, etc. For each of the models in the catalogue, it is then known which closely related components and thus how many (step 22) closely related components of the current mail item match the closely related components of a model. At 23, the model having the largest number of closely related components that match is kept, and said number is compared with a threshold at 24 which is, for example, 30 matching closely related components. If this comparison step is conclusive, the current mail item is detected as being part of a run identified by a model from the catalogue 7.

Figure 5 shows an example of the structure of the catalogue 7 shown in Figure 2. The catalogue 7 is a structured file and mainly includes four elements of information for each model 30, namely, an identity number 31, a set of closely related components 32, optimization parameters for adjusting the OCR 33, and a utility indicator for indicating the utility of the model 34.

In addition, the catalogue 7 is made up of two portions. A first portion 35 of the catalogue is dynamic and its contents are continuously changing. It is in said first portion 35 of the catalogue 7 that a new model associated with a current mail item is recorded (created) at 11, when the closely related components of the current mail item do not match 10 the closely related components of the models in the catalogue 7. When said C first portion 35 of the catalogue 7 is full, it is possible to overwrite a model 30 in said first portion so as to create 11 a new model 30. The model that is overwritten is, for example, the model whose utility indicator 34 is the lowest. The utility indicator 34 of c- a model 30 depends on the utility of the model 30 in the method of the invention, i.e. on the time elapsed since it was last recognized, on the number of times it has been recognized, on whether it led to a decision to detect a run, on the utility of the OCR parameters of the model, etc. Recording a new model 30 in the first portion 35 consists in giving an identity number 31 to the model 30. For example, in correspondence with said identity number 31, the following are recorded: the one hundred most significant closely related components 31 of the current mail item; the OCR parameters 33 used for performing automatic address recognition on the current mail item; and the utility indicator 34.

In addition, when a current mail item has been recognized as being associated 8 with a model 30 from the first portion 35 of the catalogue 7, the information elements of the model are enriched by the current mail item. The identity number 31 of the model 30 does not change so as to enable the decision module to recognize the association of the current mail item with the model The set of closely related components 32 of the module is enriched 9 with the closely related components of the current mail item. The values of each of the criteria of the table of closely related components for the closely related components of the set of closely related components 32 of the model 30 that match the closely related components of the current mail item are averaged. The closely related components of the current mail item that do not match the closely related components of the model are recorded in the set of closely related components 32 of the model, until the set of the closely related components contains a maximum C number of closely related components for a model This enrichment 9 of the closely related components of the model 30 makes it possible to consolidate the model by making it more reliable and, for example, more Mtolerant to noise on the closely related components during binarization of the digital image. Thus, in the model 30, the OCR parameters 33 of the current mail item are recorded over the old ones, and the utility indicator 34 is updated.

Initially, models 30 of runs for referenced customers who have entered into commercial agreements, or models of runs that occur very frequently and that comprise very large numbers of mail items are recorded in a non-erasable second portion 36 of the catalogue. Once recorded in the second portion 36 of the catalogue 7, the information elements of the models 30 then do not change ever again. The set 32 of closely related components of said models is complete and cannot be enriched any further with closely related components. The OCR parameters 33 for the mail items associated with the models 30 are fully optimized, typically by sampling, and they are not changed. The models 30 of the second portion 36 of the catalogue 7 have consecutive identity numbers 31. The utility indicator 34 is unnecessary. In addition to the initially-recorded models 30, models coming from the first portion 35 of the catalogue 7 are recorded at 37 in the second portion 36 of the catalogue 7 when said models have OCR parameters that are fully optimized and recur frequently, i.e. when their utility indicators 34 have exceeded a certain threshold. These models 30 are recorded in spaces left vacant in the second portion 36 of catalogue 7 and they do not overwrite the previously-recorded models Figure 6 shows in more detail the steps whereby the information contained in the OCR memory 12 and in the catalogue 7 is analyzed by a decision module 40 of the C mail handling machine in order to detect a run at 13, to decide on performance of functions resulting from a run passing, and thus to decide on whether a run is detected, and to determine at 14 the suitable actions to be triggered. In step 41, and on the basis of the OCR memory 12, the decision module 40 analyzes the identity numbers 31 of the models 30 associated with the latest Cmail items. If the identity number 31 of the model associated with the mail item preceding the current mail item is different, at 42, from the identity number of the model associated with the current mail item, no action relating to a run is triggered in step 43. Otherwise, at 44, a search, in step 45, in the catalogue 7, makes it possible to determine whether the model 30 associated with the current mail item is a model 30 in the second portion 36 of the catalogue 7 or whether said model has already been the subject of a run. If it has, at 46, the decision module decides (detects) in step 13 that the current mail item belongs to a run and triggers performance of functions resulting from a run being detected. Otherwise, at 47, the decision module counts, in step 48, the number of consecutive mail items including the current mail item that are associated with the same model 30. If, as at output 49, this number is less than or equal to four, for example, no action relating to a run is triggered. If, as at the output said number is greater than four, the decision module decides, at step 13, that the current mail item belongs to a run, and triggers performance of functions resulting from a run being detected.

A function resulting from a run being detected is the function of monitoring discounts that involves, at transmitting instructions to the database 16, so that the identification number 31 of the model 30 associated with the mail items of the run, the OCR outcomes of all of the mail items of the run, and the images of a few mail items of the run, e.g. one in fifty, among other things, are retrieved and recorded in files identified, for example, by the date, hour, minute, and second of the recording.

Another function resulting from a run being detected is the function of automatically optimizing the OCR parameters for reading the addresses of the mail items subsequent to the current mail item. The OCR parameters S33 recorded in correspondence with the identity number 31 of the model associated with the mail items of the run are retrieved, and their level of optimization is checked at 52. If said OCR parameters are fully optimized 53 (this generally applies for runs of mail items from referenced customers), they are transmitted during the looping back L to the OCR for reading the addresses on the subsequent mail items. If the OCR parameters 33 are not fully optimized 54 (this generally applies to runs whose model has just been created in the catalogue), in addition to the OCR parameters 33 of the catalogue 7, the decision model 40 analyses the OCR outcomes and the OCR parameters of the mail items associated with said model and contained in the OCR memory, and determines, at 14, new OCR parameters for automatically improving the automatic address recognition by OCR. The new OCR parameters are then transmitted during the looping back L to the OCR for reading the subsequent mail item. The OCR parameters will probably be improved over numerous mail items before they enable optimum address recognition to be achieved.

More particularly, an OCR parameter making it possible to accelerate OCR reading is the parameter concerning the distinction between "handwritten" and "typed" text. After having located the address block and the rows, the OCR address recognition unit seeks to determine whether the writing in the address block is Shandwritten or typed. Sometimes, the decision is unambiguous as of this step, and further, unnecessary processing is stopped, but other times, the decision C between handwritten and typed writing is taken only at the end of the OCR processing, during the OCR outcome.

In the invention, analysis of the OCR outcomes of the current mail item and of the preceding mail items of the current run can lead to detecting that the writing is typed for the current run, and it is then no longer necessary to perform handwritten writing recognition, Sthereby avoiding read errors and rejects.

Thus, when the processing detects that the destinations are the same for all of the mail items of the current run, which is typically a case of error in locating the address block by the OCR (the OCR is probably reading a block containing the address of the sender of the run), then with the invention, it is possible to stop that error by preventing the OCR from reading the erroneous address block and to locate another address block in the image. The OCR outcomes of the mail items belonging to the current run that are previously processed can be re-considered and the mail items can be redirected when the sorting machine is equipped with a delay line that makes it possible not to sort a mail item instantaneously when its OCR outcome is known.

Optimizing the OCR parameters can also relate to: the position of the address block when no address has been read; the binarization parameters; the character font used; the order of the elements of the address (post code or "ZIP" code, city, street); and the processing mode to be used, e.g. domestic or international.

The method of the invention that is described above for optimizing OCR parameters is based on the assumption that the subsequent mail item belongs to the current run.

It is also possible to optimize the OCR parameters for reading the address of the current mail item by performing OCR processing in two OCR address recognition C passes.

The OCR parameters can also be optimized in the manner described in European Patent Document EP 1 159 705 by using video encoding. The optimized OCR parameters are then recorded in the corresponding model in the catalogue for possible subsequent use.

The information contained in the database 12 serves Sto determine the rates of reading of the mail items of referenced customers in order to check that the clauses established in the discount contracts are complied with.

Normally, this operation is performed independently of the mail handling, e.g. on a computer on the basis of data 12 stored in the memories of a plurality of sorting machines, which databases 12 can contain data identifying the sorting machine and the unstacker through which the runs have passed. The referenced customers can be put into correspondence with the detected runs directly if said runs have a model initially recorded in the second portion of the catalogue. Otherwise, the sender can be determined by an operator on the basis of the digital images of the mail items of the runs, as stored in the database 12.

In the above-described method of the invention, the concept of inter-mail-item context is introduced, and the basis for the detection of runs is a module that can compare two images and can determine whether or not they match. The comparison is based above on closely related components as image attributes suitable for distinguishing a group of uniform mail items for detecting runs. But the invention covers other types of image attribute such as the positioning and the size of the various blocks in the image, the density of black in the image, and the table of symbols resulting from OCR recognition. It is also possible to extract the attributes of a gray scale image, or to use the densities of colors on a color digital image to constitute said image attributes suitable for distinguishing runs.

C Figure 7 shows another implementation of the method of the invention for automatically detecting runs, in which implementation the image attributes suitable for distinguishing runs are attributes extracted from a gray scale image. The automatic processing process consisting (N in forming a digital image 1 of the current mail item, in binarizing said image, at 2, and in deriving closely Crelated components, at 3, from the binarized image, which components are used for automatic address recognition 4, remains unchanged. In step 60, the digital image formed at 1 is processed so as to extract the gray scale image attributes and said gray scale image attributes are used for run detection at 61. Run detection 61 on the basis of the gray scale image attributes is performed like run detection 5 on the basis of the closely related components in parallel with OCR address recognition so that the OCR processing process loops back at L for each subsequent mail item with the result of the run detection Figure 8 shows in detail the extraction of gray scale image attributes of the current mail item that is performed in step 60. The digital image 65 formed at 1 is a gray scale image of the current mail item. The image 65 of the current mail item is firstly sub-sampled in step 66 into a low-resolution gray scale image 67 of the current mail item. Global gray scale attributes 68 are extracted from the low-resolution image 67. The gray scale attributes 68 are more particularly the height and the width of the mail item, the minimum, the maximum, and the mean shade of gray of the pixels, the variance, and the entropy of the image. Various grids are then applied to the low-resolution image 67 in step 69 so as to obtain gridded images, e.g. a first image 70 that is 4 x 4 grid and a second image 71 that is 5 x 4 grid in Figure 8.

Local gray scale attributes 72 are extracted from the various gridded images 70, 71. The local gray scale attributes are, in particular, the minimum, the maximum, and the mean shade of gray of the pixels, and the variance and the entropy computed for each block 73 of the grid of each gridded image 70, 71. The global attributes 68 with the local gray scale attributes 72 of the current mail item form the gray scale image attributes 74 that serve for run detection 61.

S 10 The use of gray scale attributes 74 is substantially Cidentical to use of closely related components. The set of closely related components 32 of a model 34 of the catalogue 7 is then replaced with gray scale attributes 74. The comparison between the gray scale attributes of a model and the gray scale attributes of the current mail item takes place as follows. The global attributes of the current mail item and of the model are compared, and thresholding is used to determine whether the mail item matches the model. In the event that it matches the model, the local attributes are compared, correlation coefficients between the local attributes of all of the blocks corresponding in pairs are computed, and are averaged to give the mean and median correlation coefficients of each grid, and, by averaging the correlation coefficients of the various grids, a single correlation coefficient is obtained for correlation between current mail item and model, which single coefficient, by thresholding, indicates whether the model and the current mail item are associated. Enriching a model to make it more robust takes place by averaging the gray scale attributes of the model with the gray scale attributes of the current mail item.

Claims

1. A method of handling mail items, which method comprises forming a digital image of each mail item, which image includes address information, and performing an automatic address recognition operation by OCR 3, said method being characterized in that it further comprises processing for automatically identifying Sgroups of uniform mail items or "runs", which processing Scomprises a step consisting in extracting image attributes from the image of the mail item, which attributes are suitable for distinguishing a group or "run" of uniform mail items (17, 74)

2. A method according to claim 1, in which the processing for identifying runs comprises a step consisting in extracting closely related components (17) from the image of the mail item.

3. A method according to claim 2, in which runs are listed in a catalogue in the form o.f a set of closely related components (32) that are predetermined for each run.

4. A method according to claim 1, in which the processing for identifying runs comprises a step consisting in extracting (60) gray scale (GS) image attributes (74) from the image of the mail item.

A method according to claim 4, in which runs are listed in a catalogue in the form of gray scale image attributes that are predetermined for each run.

6. A method according to claim 3 or claim 5, in which the catalogue contains OCR adjustment parameters (33) associated with the runs.

7. A method according to claim 6, in which said OCR adjustment parameters contained in the catalogue improve Sautomatically.

8. A method according to any preceding claim, in which a current mail item is identified as belonging to a run, and the digital image of the current mail item is M recorded in a database (16) with the results of the OCR address recognition operation for the purpose of checking 1 0 the rate of reading for the run.

9. A mail sorting machine, characterized in that it is specially arranged to implement the method according to any one of claims 1 to 7. DATED: 18 July 2005 PHILLIPS ORMONDE FITZPATRICK Attorneys for: GOLYSTIC