EP0245330A1

EP0245330A1 - Image processing method particularly in a postal sorting installation

Info

Publication number: EP0245330A1
Application number: EP19860906351
Authority: EP
Inventors: Michel Gustave Jules Fardeau
Original assignee: Bertin et Cie SA
Current assignee: Bertin Technologies SAS
Priority date: 1985-10-25
Filing date: 1986-10-24
Publication date: 1987-11-19
Also published as: FR2589267B1; FR2589267A1; WO1987002602A1

Abstract

L'invention est destinée à faciliter l'examen par un opérateur de tri postal des adresses qui n'ont pas pu faire l'objet d'une lecture automatique. Selon le procédé de l'invention, après avoir établi une image binaire de la lettre contenant l'adresse à examiner, on délimite sur l'image un certain nombre de blocs rectangulaires (28), chacun de ces blocs comprenant un certain nombre d'éléments formant des mots, parties de mot ou nombre de l'adresse, et on met en mémoire uniquement des éléments qui se trouvent à l'intérieur des blocs, en éliminant par exemple les blocs (30, 31) qui ne peuvent correspondre à des éléments d'adresse en raison de leurs dimensions. Les blocs peuvent ensuite être déplacés pour fournir une image ergonomiquement satisfaisante au poste de lecture.The invention is intended to facilitate the examination by a postal sorting operator of addresses which could not be the subject of an automatic reading. According to the method of the invention, after establishing a binary image of the letter containing the address to be examined, a certain number of rectangular blocks (28) are delimited on the image, each of these blocks comprising a certain number of elements forming words, parts of a word or number of the address, and only elements which are inside the blocks are stored in memory, by eliminating for example the blocks (30, 31) which cannot correspond to address elements due to their dimensions. The blocks can then be moved to provide an ergonomically satisfactory image to the reading station.

Description

Image processing method, in particular in a postal sorting installation.

The present invention relates to an image processing method, in particular in a postal sorting installation.

The automatic processing of mail by post, in modern facilities, requires, prior to sorting, to affix routing and distribution indexations to the envelope. Usually these indexings are made in the form of coded sticks. Entering the postal code or distribution address can be automatic, manual, or both (mixed entry).

The latter case corresponds to equipment of the "SIRCA" type (Integrated Address Recognition and Coding System). In these devices, the images of the letters are subject to automatic recognition, and in the event of failure, they are presented to an operator on a video console (video coding).

The operation of the video coding system indicated above poses various problems:

- the distribution over time of the failures of automatic recognition is absolutely random, and consequently it is difficult to predict the number of operators which it is advisable to assign to the video coding consoles, and a variation of the rate of automatic reading involves either a lack of work by the operators, or a slowdown of the automatic reading machine by the same operators.

- Furthermore, the ergonomics of the workstations poses various problems and it is necessary to limit as much as possible the fatigue felt by the operators working on the video console.

Studies carried out among others by the applicant have shown in particular that operator fatigue and information processing times are a function of the quality of the image and of the centering of the address, this centering avoiding the eye search of the postal code.

A modern treatment process includes:

- The establishment of a binary image including the address to be examined,

- the search on this image of the position of the address lines,

- the elimination of certain areas of the image which are foreign to the address by obtaining optimal framing,

- storing the framed image, and

- sending it to the video screen of a reading station for examination by an operator.

The search for the position of the address lines is used to define the optimal framing, this framing responds to the ergonomic concern indicated above.

The fact that the image is put in memory makes it possible, provided that sufficient memory is provided, to compensate for the irregularities in appearance of the failures of the automatic reading of the addresses.

This procedure nevertheless has a number of drawbacks:

1º) The image is "dirty", which means that it contains parasitic information, which has nothing to do with the address that one wishes to examine: it can be either spots of origins various, but also of inscriptions, such as postmarks or others, which are outside the address area proper, but which are however reproduced on the video screen. This spurious information is an additional cause of fatigue for the operator and requires an increase in the memory capacity necessary to store the image.

2º) The framing does not allow to obtain by itself a perfectly ergonomic presentation of the image; in fact, the position of the postal code, which is the first data that the operator must enter, does not is not in a fixed location, hence, also, an additional fatigue to seek it.

It has been proposed (FE-A-2335890) to delimit on the image a certain number of distinct rectangular blocks not overlapping, and of uniform dimensions, but of variable positions, the position of these blocks being chosen so that each includes a part of the picture elements forming the address and together they contain the whole address, and then only process the content of these blocks in an automatic reading unit.

This system is ill-suited to reading handwriting, because the size of the blocks must then be very large to take account of the largest writes, and is consequently unsuitable for the tightest writes. In addition, the large size of the blocks leads to an increase in the load of the memories and of all the equipment.

The object of the present invention is to remedy these drawbacks, that is to say to provide an image processing method which makes it possible to obtain images as "clean" as possible, and in which the prior localization of the information to be examined on the screen is as regular and usable as possible, this process being specially adapted for the treatment of addresses which could not be read by an automatic reading device, for example handwritten addresses.

To this end, the invention provides a method of processing, in particular in a postal sorting installation, of images comprising an address, this method comprising the following steps:

a) the establishment of a binary image comprising said address,

- b) the elimination of at least one zone of the image which is foreign to the address by obtaining an optimal framing, - c) analysis of the remaining image to determine the dimensions x and y of a a certain number of blocks each comprising a certain number of picture elements capable of being part of the address, - d) elimination of the blocks comprising only picture elements too small to be considered as part of the address,

e) storage of the only image elements which are included in the preserved blocks,

f) processing of the image elements stored in memory, this processing comprising at least the sending of the image elements to the video screen of a reading station with a view to identifying an address which does not could be read by an automatic reading device.

Preferably, the blocks whose dimensions in the y direction, that is to say perpendicular to the direction of the address lines, are clearly less, or clearly greater, are not stored in memory. writing line determined beforehand on the image of the object.

Preferably also, the signal density of each block is determined, that is to say the proportion of pixels (elementary spaces of the image) having the same binary value corresponding to the presence of a read sign, and the blocks whose density deviates too much from the average density of a block containing writing parts are not stored in memory.

The fact that the image elements which are inside blocks of too small dimensions (for example because they are isolated elements, such as dots or marks of small dimensions) are eliminated, constitutes an important contribution to the "cleanliness" of the image. This therefore presents itself with improved contrast. On the other hand, the fact that the postal code is contained in a single block or, possibly, in a small number of blocks close to each other, makes it possible to easily place this postal code in the most suitable location.

The elimination of the blocks having a dimension in the direction y much lower than that of the lines of writing corresponds to the elimination of the lines or horizontal underlines, which are a parasitic element, which obstructs the reading more than it does promotes. The elimination of elements whose dimension is, on the contrary, clearly greater than the height of a writing line, also corresponds to the elimination of parasitic drawings, such as postmarks, postal stamps or the like. These two measures therefore also contribute to obtaining a "cleaner" image. They also facilitate the placement of blocks containing the "Postal Code" in the appropriate location. Elimination based on the density criterion makes it possible to ignore large spots, which lead to blocks of very high density, while elimination of blocks of low density, for example correspond to inscriptions or marks that are oblique to the reading lines. In both cases, it is again a question of eliminating parts of spurious images.

According to an advantageous modality, to delimit the blocks, the pixels (elementary parts) of the binary image are grouped in "rectangular blocks" each containing an equal number of pixels, this number being greater than one but at most equal to that which corresponds to the minimum dimension of the usual handwritten letters, and a first binary value is given to all the pixels of a block of which no pixel has the moaner corresponding to the presence of a read sign, and a second binary moaner opposite to all the pixels of the other adornments, and the blocks are delimited so that they contain all the blocks having this second binary moaner, which are joined to each other, and which may contain parts of address. Preferably, the blocks leaving said second binary value and which are isolated or form an isolated assembly comprising a number of paries less than a fixed groover are left outside the blocks.

Following other arantagantes modalities: We move the blocks relative to each other to obtain a constant gap between the lines and between the blocks. This modality obviously has the effect of improving the presentation of the image to the operator. Preferably, at least certain blocks of the reconstructed image obtained after moving the blocks are reduced or amplified ("Zoom" function) to occupy a defined space in a memory or on the screen of the reading station. Image reduction does not bring about significant losses of information, since it has been found that the most "voluminous" images correspond to writings of large dimensions, and can therefore be reduced somewhat without inconvenience. On the contrary, the image amplification coming from an address written with an exaggeratedly small writing, facilitates the work of the operator. Furthermore, it is preferable that the reconstructed image be subjected to a so-called "compression" treatment before it is stored and / or sent. at the reading station, preferably this compression treatment is done according to the Read method. The purpose of this compression operation, which is a conventional operation, is to reduce the amount of information to be stored and transmitted, and therefore to save transmission time, and to reduce the cost price of the installation. . It is not necessary to describe here in detail the method of Eead, which is well known to those skilled in the art, remember only that its principle consists after having transmitted all the information relating to a line of the image, to not take into account for the next line only the information concerning the differences of this line with the previous one. The invention will now be explained in more detail using a practical example illustrated by the drawings, among which:

Figure 1 is an overall diagram of the process. Figure 2 is a view of an envelope bearing a handwritten address.

Figure 3 is a view of the envelope portion taken into account by the image capture machine. Figure 4 is an enlarged view of part of Figure 3, schematically creating a block.

Figure 5 is a view similar to that of Figure 3, with indication of the blocks, Figure 6 is a view similar to that of Figure 5, corresponding to the image that finally appears on the operator's screen.

Figure 1 is a block diagram showing the entire process of the invention. The reference 1 designates an image capture and automatic reading machine, an interface 2 receives a binary video image from machine 1. The reference 3 designates a station for transforming columns into lines. The reason for this transformation is that the machine used in the practical example for capturing images operates in rertical lines, the letters scrolling in front of it horizontally. It is necessary, with this type of material, to transform the image into an image comprising horizontal lines, so as to be able to operate with a material of current type. It is clear that with other image capture equipment, this post 3 would possibly not have its raison d'être.

The images are then sent to an image buffer memory 4. The reason for this buffer memory is as follows: the image capture machine 1 supplies images of letters at a rate of 15 per second, in the practical example described. The input is made on a height of 64 mm on letters from 140 to 220 mm, with a resolution of 8 points to the millimeter in each dimension. The binary images provided have 512 x 1,760 dots, or 901,120 dots; the memory necessary to store such an image therefore has a capacity of 112,640 bytes. The automatic reading unit processes several images in parallel according to a "pipeline" structure, and can take up to a second to say if it has been able to recognize an address. If there is total recognition (routing and distribution), there is no need to memorize the images of the letters to process them by video coding, otherwise, they must be memorized. Given this one second delay and the throughput (15 letters per second), it is essential to provide a buffer of sixteen letters in the processing machine,

The simplest functionally solution would be to have a buffer of sixteen images and, on the order of the automatic reading unit, to store these images directly on a disc for later video coding. This solution is in fact unrealistic for several economic and technical reasons: the various works carried out on ergonomics have shown, as we said above, the importance of image quality. It is therefore excluded to present operators with the raw binary image. This image must therefore be processed to obtain the optimal ergonomic quality desired. And this is the purpose precisely of the present invention.

On the other hand, the planned storage capacity being 100,000 images, raw image storage would correspond to an enormous memory volume, of the order of 11 gigabytes. Furthermore, in the case of the maximum speed of 15 letters per second, the average speed of storage on disk would be of the order of 13.5 megabits per second. These two figures are incompatible with the current state of the art, and show that it is essential to reduce the images before their storage, this reduction can be done in three stages:

- research and storage of useful information,

- framing of useful information, - compression of the resulting images by a conventional method of the Read coding type.

The buffer memory 4 only ensures the storage of the images necessary for image processing as will be seen below. In fact, the memory 4 must only allow the storage of at least two images of 112 kilobytes, one of these images being in the process of acquisition, and the other being processed.

The images coming from the interface 2 and from the transformer station 3 are sent to a station 5 for searching for information blocks, connected to a microprocessor 6 for image processing. All the operations carried out at stations 5 and 6 will be explained using figures 2 to 6.

Figure 2 is a complete image of the back of a letter. The image capture machine 1 retains on this letter only a strip 64 mm high, which makes it possible to obtain the image of FIG. a circular stamp which constitutes a parasitic image element, and it is also observed that the left ends of the address lines are not aligned vertically. Figure 4 corresponds to a partial street, greatly enlarged, of Figure 3. A grid materializes the limits of the "trimmed" between which the image is distributed; in practice, the size of a trim is of the order of 1 mm by 4 mm. Hatching has been applied to the trim 22 which contains a part of the reading element, that is to say at least one pixel of which has a binary moan different from that of the background. The other blocks 23 have not been hatched in the figure. Horizontal lines 24, 25, and vertical lines 26, 27, delimit a block 28, which contains two letters and a point located at a distance of a letter less than the size of a block, A block 29 corresponds to a isolated point, which is outside a block. Indeed, the microprocessor software prohibits it from constituting a block with a single (maximum four) blocks. FIG. 5 shows all of the blocks which have been formed on the image represented in FIG. 3. It can be seen that these blocks correspond to all of the address elements, with however two parasitic blocks 30, 31, one corresponding to the circular stamp 20 located at the top left of the image, and the other to a horizontal line located at the bottom of the image, below the address. The microprocessor then eliminates these blocks by a comparison of their dimensions arec the height of the lines as it emerges from the other trimmed,

Figure 6 shows the reconstructed image after it has undergone an additional cropping operation consisting in moving the blocks with respect to each other, and with respect to the limits of the image, so that the block which contains the code postal 32, that is to say the first from the bottom and from the left, is troure in the location which was recognized the most terrible during the preliminary studies, and the other blocks were placed at suitable distances from each other, both vertically and horizontally. In fact, this last operation of placing the blocks selected according to the ergonomic criteria is done in an image reconstruction unit. This unit receives on the one hand, from the image processing microprocessor 6, the coordinates of the blocks which have been retained for the rest of the operations, to the exclusion of the other blocks, and on the other hand, it receives memory buffer 4 the information concerning the content of these blocks. From the information coming from these two sources, and according to the program of the microprocessor 6, the unit 7 for reconstructing the images proceeds to the positioning of blocks according to the criteria defined above.

The image reconstruction unit 7 also proceeds, on the order of the microprocessor 6, to reduce or dilate the blocks, so that the reconstructed or "cropped" image occupies in a memory space approximately 50,000 bytes, Let us call that the initial image occupies in a memory a space ranging between 71,680 and 112,640 bytes, these numbers corresponding respectfully to letters of 140 mm and 220 mm long.

The processing operations to be carried out in the image reconstruction unit 7 relate to a number reduced blocks, 10 to 15 blocks in general. The use of a fast microprocessor, of the "68000" type from the company THOMSON / EFCIS is possible, it leads to a calculation time which can be estimated at around thirty milliseconds, which is compatible with the rate 15 frames per second.

The images leaving the image reconstruction unit are sent to a buffer memory 8. From this buffer memory, they are sent to a clear image compression unit. This compression unit consists of two integrated circuits of the "AMD 7970" type from the company ADYANCED MICRO DEVICES, operating as a flip-flop. The reason for this arrangement is that the compression time of an AMD 7970 circuit is approximately order of 100 milliseconds, higher than the expected rate of 65 milliseconds per image. The compression ratio provided by this circuit is from 15 to 30. This results in compressed images having a size of the order of 3 kilobytes. The memory 8, which powers a compression unit 9 must make it possible to store at least three images of 50 kilobytes: one being received, the other two being processed by each of the two AMD 7979 circuits, the allocation of these memories being made by circular permutation.

The images leaving the compression circuit are sent to a compressed image buffer 10, which too must allow the storage of three images but of a smaller size: only three kilobytes each, two of these images being loading by the AMD 7970 circuits, and the third being in the process of transfer by the aral. The three memories are also assigned by circular permutation. The compressed images leaving the buffer memory 10 are sent to the system management computer 11. This computer is connected via a link 12 to twenty reading stations 13, of which only three were represented.

Each reading station comprises on the one hand a keyboard-screen assembly 14 and a decompression unit 15. This unit consists of a circuit of the "AMD 7970" type, already mentioned above; it ensures the decompression of the 3 kilobytes of the compressed image into the 50 kilobytes of the image restored on the screen. This restored image indeed includes 400 x 1000 points. The link 12 between the management computer and the reading stations is a "ETHERNET" type link. The arrangement according to which the images are decompressed at the reading station, and not at the level of the management computer, makes it possible to reduce in a very considerable proportion the load of the "ETHERNET" connection system.

The management computer 11 is also connected on the one hand to a disk memory unit 16, and on the other hand to the "ELIT" machine (Indexing and Sorting Reading Equipment) 17 so as to transmit to it either the routing code which was determined at the reading station for printing on the letter itself or the street delivery code for immediate sorting of the letter. The present invention made for postal sorting, is directly adaptable by the skilled person to other cases of applications that may require other ergonomic criteria for the presentation of images. It should also be noted that the inention can be part of an automatic character recognition system, advantageously constituting a preprocessing step of such a system. In the embodiment described, the method according to the invention is implemented by the collaboration of hardware means and software means. If we accept longer operating times, we can envisage a fully software implementation.

Claims

1. Process for processing, in particular in a postal sorting installation, images comprising an address, this process comprising the following steps:

a) the establishment of a binary image comprising said address,

b) the elimination of at least one zone of the image which is foreign to the address by obtaining an optimal framing,

c) analysis of the remaining image to determine the dimensions x and y of a certain number of blocks each comprising a certain number of image elements capable of forming part of the address,

- d) elimination of the blocks comprising only picture elements too small to be considered as part of the address,

2. Method according to claim 1, characterized in that it does not store the blocks whose dimension in the y direction, perpendicular to the address lines is significantly less or significantly greater than the height of a line of writing determined beforehand on the image of the object.

3. Method according to one of claims 1 or 2, characterized in that the density of each block is determined, that is to say the proportion of pixels having the binary value corresponding to the presence of a sign of reading, and the blocks whose density deviates too much from the average density of a block containing writing parts are not stored in memory.

4. Method according to one of claims 1 to 3, characterized in that, to delimit the blocks, the pixels (elementary parts) of the binary image are grouped in rectangular blocks each containing an equal number of pixels, this number being greater than one but at most equal to that which corresponds to the minimum dimension of the usual handwritten letters, a first binary value is given to all the pixels of a block of which no pixel has the value corresponding to the presence of a read sign, and a second binary value opposite to all the pixels of the other blocks, and the blocks are delimited in such a way that they contain all the blocks having this second binair value which are joined to each other, and which are likely to contain parts of the address.

5. Method according to claim 4, characterized in that the blocks outside the blocks having said second binary value and which are isolated or form an isolated set comprising a number of blocks less than a fixed value.

6. Method according to one of claims 1 to 5, characterized in that it constitutes a preprocessing step of an automatic character recognition system.

7. Method according to one of claims 1 to 5, characterized in that the blocks are sent to the memory or to the reading station after having replaced them in the image according to ergonomic criteria.

8. Method according to claim 7 - characterized in that the blocks are replaced to obtain a pre-localization of the routing code and / or distribution information.

9. Method according to claim 7 or 8, characterized in that the blocks are moved relative to each other to obtain a constant difference between the lines and / or between the blocks.

10. Method according to one of claims 7 to 9, characterized in that at least some blocks are reduced or amplified so that the reconstructed image obtained after moving the blocks occupies a space defined in a memory or on the screen of the reading station.

11. Method according to one of claims 1 to 10, characterized in that the reconstructed image is subjected to a compression treatment before it is stored and sent to the reading station.

12. Method according to one of claims 1 to 11, characterized in that it is implemented by hardware and software means acting in collaboration.

13. Method according to one of claims 1 to 11, characterized in that it is implemented by entirely software means.

14. Method according to one of claims 11 to 13, characterized in that the compression treatment is done according to the method of Eead.

15. Method according to one of claims 11 to 14, characterized in that the compressed images are sent to the reading station, the reading station itself decompressing before sending the image to the screen video.

16. Method according to claim 15, characterized in that several image reading stations are connected using a local network of the "ETHERNET" type or the like.