WO2009066297A2

WO2009066297A2 - A method of verifying the contents of bundles of paper currency

Info

Publication number: WO2009066297A2
Application number: PCT/IL2008/001531
Authority: WO
Inventors: Nir Avrahami; Ronen Lerner
Original assignee: Ioimage Ltd.
Priority date: 2007-11-21
Filing date: 2008-11-20
Publication date: 2009-05-28
Also published as: WO2009066297A3; IL187545A0

Abstract

The invention is a method of automatically verifying the denomination and number of packs in standard bundles of paper currency that are exchanged between banks. The method comprises placing a plurality of bundles on a table, taking an image of each of the bundles with an electronic camera, and using a computer comprising dedicated software to process the images. The denomination of the bank notes is determined from the color of the color coded straps used to bind the packs in the bundles and the software employs a global optimization technique to determine the number of packs in each of the bundles.

Description

AMETHOD OFVERIFYING THE CONTENTS OFBUNDLES OF

PAPER CURRENCY

Field of the Invention

The invention is related to the field of banking. In particular the invention relates to methods of automatically verifying the contents of the bundles of paper currency that are exchanged between banks.

Background of the Invention

Large amounts of money are transferred each day between banks, particularly between individual banks and the central banks, around the world. A typical deposit contains several bags of currency each of which contains several bundles of banknotes. Each central bank has its own regulations and, as an example, we can consider those of the Federal Reserve System (FRS) of the United States of America. In order to deposit money in one of the 12 regional Federal Reserve Banks (FRB) that make up the FRS it must be packaged as follows: 100 bills of the same denomination, known as a "standard strap" or simply as a "strap" are wrapped with a color coded strap. [Note that the word "pack" will be used interchangeably with the word "strap" herein to avoid confusion between the colored strap that holds the standard strap].

Counting of the bundles that make up each deposit is obviously a time consuming operation that requires the following of rigid procedures and use of stringent methods to eliminate, or at least minimize, errors, theft, and fraud. It would therefore be of considerable interest if a method could be provided, which would not only reduce the time required to perform the counting task but would also at least equal if not exceed the accuracy and security of traditional methods. Many methods and machines have been devised for counting individual bank notes. Machinery of many different designs for identifying the denomination of individual notes and sorting them, counting them, and creating straps is known and in everyday use not only in large central banks but also in the smallest of branch banks and in other establishments such as retail stores and sports arenas where large quantities of cash are handled.

There has been much less activity in developing methods or machinery for counting standard bundles of paper money. Some typical examples of different approaches to solving this problem are described in the following publications:

- JP03238579A2: This patent describes a system that counts the number of paper straps by using active computer vision. A slit light source projects a line of light on the side of the bundle and an image is photographed using a television camera. Next, the image is binarized and the line of light becomes a broken line wherein each segment corresponds to a single strap. The bundle of straps must be properly oriented (parallel to the slit of light) and the system process only one bundle at a time.

- US 5, 022,531: This patent describes a bundle processing apparatus for verifying the number of packs in a bundle. In this patent each pack is bound together by a band that is wrapped around the middle of the short dimension of the bank notes and a pack is comprised of ten packs that are held together by a band wrapped around them in a direction transverse to the straps on the packs. A number of bundles are loaded onto the machine and then each bundle is moved automatically by conveyor belts through the following stations: o The denomination detecting section comprises a scanner. Information extracted from the scanner measurements (it is not specified what information, but apparently information printed on either the bands or the banknotes) is compared with stored information to determine the denomination of the money in the bundle. o The length detecting section comprises a rotary mirror, laser light source, and a light detecting element. The length detecting section scans the length of the band that is wrapped around the bundle and the length of the bundle is determined from the waveform of the detected light. It is then determined if the measured value is equal to a preset value of the length. o The ten-packs counting unit comprises a rotary mirror, laser light source, and a light detecting element. In this case the unit scans the bands that are wrapped around the packs and the number of packs in the bundle is determined from the waveform of the electrical signals corresponding to the detected light and compared with a predetermined value. o If the length is not equal to the preset value and/or the number of packs is not equal to the predetermined value, then the bundle is rejected and conveyed to a rejected bundle stacking section, o If the length is equal to the preset value and the number of packs is equal to the predetermined value, then the bundle is conveyed to a temporary stacking unit from where it goes to other parts of the machine where five stacks are put together to form a case. As can be understood, the apparatus described in this patent is complex both mechanically and electronically, which in itself is a problem for an apparatus that is expected to provide perfect results while dealing with large numbers of bundles on a daily basis. Additionally the patent does not describe how the inventors deal with two basic problems that will only be mentioned here but will be described more fully with reference to the present invention hereinbelow. These problems are: (1) For used banknotes, the bundles can be up to on the order of 50% longer than bundles of new banknotes. (2) The number of packs in the bundle are determined by measuring the number of peaks (or valleys) caused by the difference of reflection at the edges of the straps. Usually the scanned line shows a considerable amount of noise and the number of peaks measured depends on the threshold condition adapted to eliminate the noise.

- US 6,502,052: In this patent the word "bundle" refers to a predetermined number of paper sheets or banknotes held together by a band, i.e. this is called a "strap" in the FRB nomenclature. In the apparatus of this invention, banded bundles of notes are stacked in a specially designed storage box. Inside the storage box is located a photoelectric detector system that can be moved up and down from the top of the storage box to its bottom by mechanical counting means. The detector system is positioned opposite the bands and as it is moved in a vertical direction past the bundles the electrical signals from the photocell corresponding to the intensity of the light reflected from the stack of bills is recorded.

Since the reflectance of the bands differs from that of the space between bands, the number of bundles in the storage box can be counted. The device described is only a part of a much larger and more complicated money handling system, nonetheless it can be seen that to count the number of stacks, they must first be inserted into the storage box and after they are counted they must be removed to allow another stack of bundles to be introduced. The method of this patent can not be advantageously adapted to solve the problem of a large bank such as a

FRB.

In view of the large expenditure of labor involved it is interesting that, despite the many efforts to provide machinery that can automatically count the bundles, the inventors do not know of any central banks which use a fully automated method to perform this task. Amongst possible reasons for this is that, in addition to providing a substantial savings in labor, it is essential that the counting machinery be essentially 100% reliable and error free. The value of a single standard bundle of 1,000 bank notes ranges from $1,000 to $100,000 so there is no room for inconsistent results.

It is therefore a purpose of the present invention to provide a method for quickly and accurately counting the number of standard straps in a standard bundle of banknotes that can be carried out using equipment commonly present at locations where money is counted with the addition of dedicated software.

It is another purpose of the present invention to provide a method for quickly and accurately counting the number of standard straps in a standard bundle of banknotes that is semi-automatic.

Further purposes and advantages of this invention will appear as the description proceeds.

Summary of the Invention

The invention is a method of automatically verifying the contents, i.e. the denomination and number of packs, of bundles of paper currency that are exchanged between banks. The method comprises the steps of: a. placing a plurality of bundles on a table; b. taking an image of the plurality of bundles with an electronic camera; and c. using a computer comprising dedicated software to process the image in order to: i. determine the denomination of the bank notes from the color of the color coded straps used to bind the packs in the bundles; and ii. use a global optimization technique to determine the number of packs in each of the bundles. The method of the invention can be applied to one of the bundles of paper currency at a time or simultaneously to all bundles on the table.

According to the method of the invention, a number of search regions, which are easily identified in the images taken by the camera and whose locations are known to the dedicated software are marked out on the top of the table and the bundles of paper currency are placed in each of the search regions, wherein no more than one of the bundles is placed in each of the search regions. The bundles can be placed on the table with arbitrary orientation, as long as the faces of the banknotes are perpendicular to the table and the straps on all of the packs are visible. Additionally, the color coded straps do not need to be perfectly centered on the packs. In preferred embodiments, with the exception of the dedicated software, the method is carried out using standard equipment that is already present at the location where the bundles of paper currency are being counted.

According to the method of the invention, a number of locations, which are easily identified in the images taken by the camera and whose locations are known to the dedicated software are marked out on the top of the table for color calibration boards and one color calibration board is placed at one or more of the locations. The color calib ration boards are used to normalize the colors in the images.

According to the method of the invention each bundle is imaged only once. Data comprising: information entered by the user including details of the contents of the bag comprising the bundles of paper currency as displayed on the shipping receipt; the image/s of the bundles; and the identity of the specific search regions in which each of the bundles is located and the counting data as determined by the dedicated software are displayed on the computer monitor and both are saved to a database. The saved data is used for one or more of the following: creating counting reports; comparing the actual contents of the bag with the shipping information supplied with the bag; summarizing the totals on a daily, weekly, monthly or a yearly basis; and investigating irregularities and suspicion of criminal activity.

According to the method of the invention the denomination of the currency in the bundles is determined by the dedicated software interrogating the images of the bundles to detect possible strap pixels, which are then classified to the various strap classes by comparison with a table comprising the known standard strap color distribution that has previously been learned and stored in the memory of the computer.

All the above and other characteristics and advantages of the invention will be further understood through the following illustrative and non-limitative description of preferred embodiments thereof, with reference to the appended drawings.

Brief Description of the Drawings

— Fig. 1 illustrates the initial set-up step of the system;

— Fig. 2 is a flow chart describing the steps of the method of the invention; — Fig. 3 shows a rectified image of a bundle and a smart projection used to determine the number of packs in the bundle;

— Fig. 4 is an example illustrating the threshold problem;

— Fig. 5 illustrates the step of computing the overlay for each possible number of packs in the bundle; and — Fig. 6 illustrates the step of determining the number of packs in the bundle.

Detailed Description of Preferred Embodiments

The invention is a method of verifying the contents, i.e. the number of packs, in standard bundles of money. As in the manual methods presently employed, the bank employees first open the sacks of money and place the individual bundles on a conveyor belt or table to be inspected. According to the invention, once the bundles are placed on the table, they are imaged using an electronic camera and then an image processing method, to be described hereinbelow, is used to provide solutions to two different problems. Firstly the denomination of the bank notes is determined from the color of the color coded straps used to bind the packs in the bundle. Secondly the number of packs in each bundle is determined in order to verify that each bundle is indeed a standard bundle comprising 10 packs. These two tasks can be carried out either one after the other, in which case the color information can be used in the process of determining the number of bundles or in parallel for each individual bundle. Also, in different embodiments, the method of the invention can be applied to one bundle at a time or multiple bundles can be processed simultaneously since there are no dependencies between the bundles.

Dedicated equipment can be provided but, with the exception of the software comprising the instructions for carrying out the calculations, the method normally can be carried out using equipment that is already present in most locations where large quantities of paper money are handled. Specifically, what is needed is a table; a video camera, which can be a security camera that is located or relocated above the table; illumination means; and a computer, e.g. a PC, for carrying out the image processing and recording the results.

The initial set-up step of the system is illustrated in Fig. 1. An electronic camera and lighting means (neither shown in the figure) are installed over the top of table top 10. The camera can be either a stationary video camera or a "still" camera, and is positioned such that its field of view covers the entire table top in a single frame. Alternatively the camera can focus on one or more search areas at a time and, after imaging, is moved to focus on the next search area/s. The only restriction is that all bundles are imaged exactly once, i.e. a bundle should not show in two different images. The camera is connected by cable or wireless connection to a computer (not shown). The images gathered by the camera are transferred to the computer for display, storage, and processing and the computer can be used to input instructions to the camera, e.g. to take another image and transfer it to the computer. The illumination means preferably provides uniform illumination that falls both vertically onto the entire surface of the table and also horizontally onto the sides of the bundles. Direct overhead illumination is important for the detection of the color of the straps, whereas side illumination provides better contrast for detecting the edge between adjacent packs, therefore they are both needed.

In the physical set-up stage of the system, a number of search regions 12 are marked out on table top 10. The search regions are predetermined areas on the table top 10 that are bordered and/or painted in a color that contrasts with the uniform background color of the table top so that they are easily identified in the images taken by the camera and whose locations are known to the dedicated software. The locations of one or more color calibration boards 14 are also marked out on the table top 10. Preferably, one color calibration board 14 is located at each corner as shown in Fig. 1.

Once the physical set-up of the system is completed, the counting routine can begin. A bag of money is selected, the details of the contents of the bag, e.g. number of bundles, denominations of the money in the bundles, and total value of the contents of the bag as displayed on the shipping receipt are entered into the computer and the user initiates a new count. Bundles are placed on the table in the search regions in an arbitrary orientation and the system acquires an image of the table top. The system identifies the denomination of the bills from the color of the straps and counts the number of packs in each bundle by use of a global optimization technique as described hereinbelow. The image and the counting data are displayed on the computer monitor and both are saved to a database. The saved data can be used for various purposes such as creating counting reports comparing the actual contents of the bag with the shipping information supplied with the bag, or summarizing the totals on a daily, weekly, monthly or a yearly basis. The saved data can also be used at any time to investigate irregularities and suspicion of criminal activity by viewing the images which were used for the counting.

It is to be noted that, although the invention is described herein in terms of a table top, the throughput of the system and automation can be further increased by using a suitable marked up conveyor belt to bring one group of bundles at a time into position under the camera and illumination means.

More specific details of the invention, including the image processing steps, are best described with reference to the flow chart shown in Fig. 2. After the bundles have been distributed on the table, with no more than one in each search region 12, an image is taken and input to the computer in step 100.

Fig. 1 is a typical image showing six bundles, each in its own search region

12, on the table top 10. From the figure a number of the more important advantages enabled by the usage of digital imaging in the present invention over the electro-mechanical systems of the prior art can be seen.

Specifically, multiple bundles can be handled simultaneously, the bundles can be handled with arbitrary orientation, and the straps do not need to be perfectly centered on the packs.

Following the input of the image in step 100 it is preprocessed in step 102 to remove noise and the colors are normalized to avoid illumination effects by use of the color calibration boards 14.

In step 104 each of the search regions 12 is investigated to detect the presence or absence of a bundle and its location, i.e. the identity of the specific search region 12 in which the bundle is located is noted and stored in the computer memory. In step 106, the image of the bundle is interrogated by the program to detect possible strap pixels. These pixels are classified to the various strap classes by comparison with a table 108 comprising the known standard strap color distribution that has previously been learned and stored in the computer memory. The most probable strap class is then chosen. If, for a predetermined proportion of the possible strap pixels, e.g. approximately 10% or more, a different most probable strap class is obtained, then this indicates the possibility of different types of straps in the bundle and an error message is displayed either visually on the computer display screen, audibly by means of a speaker, or in some other way to call the attention of the operator to the suspected problem. Once the class of strap, i.e. the denomination of all of the bills in the bundle, is unambiguously known this information is stored in memory, where it will be used in step 110 to compute the total sum of money in the bundle.

Once a bundle is detected in step 104 in a specific search region 12, the process of determining the number of packs in the bundle is started. In step 112 the orientation of the packs, i.e. the orientation of the bills in the pack is determined and a rectified image of the bundle is created in which the bills are lined up parallel to one of the axis of a two-dimensional Cartesian coordinate system. In Fig. 3 a rectified image of a typical bundle is shown with distances measured in pixels. Since this is a controlled application, i.e. the parameters of the camera, height above the table, viewing angle, etc. are all known and constant, the magnification, i.e. the number of pixels per centimeter, is known enabling easy conversion from physical dimensions to the corresponding number of pixels in the image and vice versa.

In order to determine the number of packs in the bundle a "smart one- dimensional projection" is generated in step 114. The smart one -dimensional projection is a one-dimensional signal whose length equals the height of the rectified image (measured in pixels along the vertical axis in Fig. 3) and whose value f(i) for the i-th row of pixels is proportional to the probability that this line separates between two adjacent packs. An example of a smart projection is shown on the left side of Fig. 3 wherein the function has been rotated, aligned and overlaid on the rectified image to match the rows, such that the distance of the curve to the left at each row represents f(i). The larger the value of f(i) the larger the probability that an interface between two packs occurs at the i-th row. Referring to Fig. 3, an example of how the probability is determined for the rows is: the uppermost interface (between the upper two packs) is around row 40. Note that there are many pixels along this row, whose gradient is directed vertically and whose color is almost black. The smart projection is generated by counting them. This is not the only possible "projection" and there are many other valid "projections" that can be used and it is to be understood that the invention is not to be limited to a specific method of making the smart projection.

In the prior art the number of packs is determined by counting the number of "peaks" in the smart projection. This approach to determining the number of packs in the bundle has been rejected by the inventors of the present invention for reasons that can be understood by considering the following example.

In Fig. 4 is shown a smart projection as described hereinabove of a bundle comprising ten packs, and hence nine interfaces. Along the horizontal axis distance in pixels is measured from one edge of the bundle. The y-axis shows a score which is proportional at the i^th location to the probability that the ith row is an interface point between two packs. The interfaces are numbered and marked with vertical arrows. A non-interface point is marked with the arrow labeled A. As can be seen, the value of interface 2 is lower than that of point A. Hence any threshold which detects the 2^nd interface, e.g. that marked Thresh 2, will also detect, mistakenly, point A as an interface. 1

- 13 -

Moreover, any threshold eliminating point A, e.g. Thresh 1, will also eliminate, mistakenly, the 2^nd interface. The present invention overcomes this problem by using a global optimization scheme based on the use of distance constraints derived from limitations on the pack size.

Fig. 5 illustrates step 116 of the flow chart of Fig. 2. The actual width of a pack of used bank notes can be up to 50% larger than the nominal width of a pack made up of 100 newly printed banknotes. Therefore any given pack can have an actual width that is greater than or equal to the minimum value and less than or equal to the maximum value. In step 116 the computer measures the overall width of the bundle by counting pixels on the rectified image in Fig. 3. Based on the known minimum and maximum pack widths, the minimum and maximum number of packs that can be present in a bundle having the measured width is determined. Now an overlay is created for each possible number of packs in the bundle. Determining the optimal location in the overlay of each of the markers that represents the location of an interface between packs in the bundle can be done in several ways. A preferred method of solving the placement is given in the following example:

A smart projection function with f(x), which is assumed to be strictly positive, is defined. It is assumed that the basic signal is of length L pixels, therefore f(x) is defined for {1,2, ...,L}.

The score of an overlay having markers at (x_l5X2, ...,Xn) is defined as score

The problem to be solved is to find the legal overlay that has the highest possible score. The phrase "legal overlay" means an overlay in which the locations of the markers obey the following pack size constraints:

If the minimal pack size is denoted by m and the maximal by M, then 1. the leftmost marker must be located at a distance > m from the left end of the bundle and at the same time at a distance < M from the same side;

2. the rightmost marker must be located at a distance > m from the right end of the bundle and at the same time at a distance < M from the same side; and

3. the distance between adjacent markers must be no smaller than m and no larger than M.

In mathematical terms a legal overlay T(x_n,n) with a fixed rightmost marker is defined as the score of the best placement of n markers, such that:

1. The rightmost marker (which is the n^th) is located at x_n

2. The leftmost marker (which is the 1^st) is located at X₁, where

3. All distances between adjacent markers obey | Xi-Xi+i I €[m,M]

Relaxing the constraint on the location of the rightmost marker to a range of valid positions with respect to a position j, an optimal placement of n markers is denoted by:

D(J₁ n) = max {T(k, n) \ k e [j - M , j - m], k > 0}

The optimal legal placement score is hence

optimal placement score = D(L + 1, n) =

= max{r(jfc,n) \ k <= [L + 1 - M, L + 1 - m\,k > θ}

Moreover it is to be noted that the following recursive relation holds: 1

- 15 -

- 1) > 0

TU,n)

The above relations yield a recursive algorithm that is used in step 116 to compute the best placement (overlay) in a straight forward manner. Using dynamic programming one can avoid exponential processing time, and achieve an efficient polynomial algorithm.

The above is illustrated Fig. 5. The length of the bundle is approximately 420 pixels and using the minimum and maximum pack widths it is determined that the bundle can consist of between 7 and 13 packs. The program now computes an overlay for each of the possible numbers of packs. The overlays are displayed graphically stacked one on top of the other and overlaid on the smart projection. The markers of each overlay are shown as dashed vertical lines in the respective level. Shown in the figure is the optimal placement of the markers as calculated using the above described recursive algorithm.

In step 118, the overlay that has the best placement score is chosen, i.e. the number of packs in the bundle is determined. This is illustrated in Fig. 6, which is a graph showing the best placement score for each of the overlays in Fig. 5. The highest value is for the overlay having ten packs, which verifies that this is a standard bundle of banknotes. In step 110 the results from steps 106 and 118 are combined and the sum of the currency in the bundle is computed. Finally, in step 120, various statistics concerning the bundle are outputted from the computer to be, for example, displayed on the computer screen, printed on a printer, or sent to a remote location for display, storage, or analysis.

Steps 104 — 120 are repeated for all the bundles on the table after which they are removed and replaced with another plurality of bundles. As described hereinabove, the method can be carried out by processing the bundles on the table either one after the other in series or simultaneously in parallel.

Although embodiments of the invention have been described by way of illustration, it will be understood that the invention may be carried out with many variations, modifications, and adaptations, without exceeding the scope of the claims.

Claims

1. A method of automatically verifying the denomination and the number of packs in standard bundles of paper currency that are exchanged between banks, said method comprising the steps of: a. placing a plurality of bundles on a table; b. taking an image of each of said bundles with an electronic camera; and c. using a computer comprising dedicated software to process said image in order to: i. determine the denomination of the bank notes from the color of the color coded straps used to bind the packs in said bundles; and ii. use a global optimization technique to determine the number of packs in each of said bundles.

2. A method according to claim 1, wherein said method is applied to one of the bundles of paper currency at a time.

3. A method according to claim 1, wherein said method is applied to two or more of the bundles of paper currency simultaneously.

4. A method according to claim 1, wherein the bundles can be placed on the table with arbitrary orientation, and the color coded straps do not need to be perfectly centered on the packs.

5. A method according to claim 1, wherein, with the exception of the dedicated software, the method is carried out using standard equipment that is already present at the location where the bundles of paper currency are being counted.

6. A method according to claim 1, wherein a number of search regions, which are easily identified in the images taken by the camera and whose locations are known to the dedicated software are marked out on the top of the table and the bundles of paper currency are placed in each of said search regions, wherein no more than one of said bundles is placed in each of said search regions.

7. A method according to claim 1, wherein a number of locations, which are easily identified in the images taken by the camera and whose locations are known to the dedicated software are marked out on the top of the table for color calibration boards and one color calibration board is placed at one or more of said locations.

8. A method according to claim 7, wherein the color calibration boards are used to normalize the colors in the images.

9. A method according to claim 1, wherein each bundle is imaged only once.

10. A method according to claim 1, wherein data comprising: information entered by the user including details of the contents of the bag comprising the bundles of paper currency as displayed on the shipping receipt; the image/s of said bundles; and the identity of the specific search region in which each of said bundles is located and the counting data as determined by the dedicated software are displayed on the computer monitor and both are saved to a database.

11. A method according to claim 10, wherein said saved data is used for one or more of the following: creating counting reports; comparing the actual contents of the bag with the shipping information supplied with the bag; summarizing the totals on a daily, weekly, monthly or a yearly basis; and investigating irregularities and suspicion of criminal activity.

12. A method according to claim 1, wherein the denomination of the currency in the bundles is determined by the dedicated software interrogating the images of said bundles to detect possible strap pixels, which are then classified to the various strap classes by comparison with a table comprising the known standard strap color distribution that has previously been learned and stored in the memory of the computer.