WO2007119049A1

WO2007119049A1 - A system and method to decode barcodes by use of wireless device and webcam

Info

Publication number: WO2007119049A1
Application number: PCT/GB2007/001358
Authority: WO
Inventors: Priyasantha Ariya
Original assignee: Priyasantha Ariya
Priority date: 2005-04-13
Filing date: 2007-04-13
Publication date: 2007-10-25

Abstract

The present invention describes a system and method of easily sharing digital content from user devices capable of using universal, single, abstract and persistent identifiers such as XRI based universal Identity for addressing the sender and receiver(s) for sharing digital content over communication network and providing fine grained control over access, privacy, usage, expiry, compliance and synchronization of the digital content. This standard based universal and persistent Identity is used for controlling access, privacy, usage, expiry, compliance and synchronization control of the digital content. Further the invention also stores the digital content on a network storage device to share and archive the data.

Description

A SYSTEM AND METHOD TO DECODE BARCODES BY USE OF WIRELESS DEVICE AND WEBCAM

Field of the invention The invention relates to bar code decoding mechanism. More particularly the present invention relates to a method and system to transmit, read and decode a barcode.

Description of the related Art Commercial transactions involve the use of bar code in many ways — promotions, tickets, vouchers, supermarkets, airport security, check-in, boarding areas, stadiums, libraries, test centers, conference centers, and in many other ways and places. The use of bar code enhances the speed, reliability and credibility of a transaction.

Traditionally we associate bar codes with the printed labels on goods. With the advent of advanced information technology, we see the barcodes going paperless . The bar codes are digitally and electronically displayed over PCs and mobile devices to make the transactions quick and convenient for the consumers. The use of mobile devices to receive and transmit a bar code in the commercial world is a well known prior art. Of course, there remains a limit for electronic bar codes to be deciphered at the point of sale or entry. With different bar codes formats, it may become difficult for scanners to read all kinds of formats — i.e. , the contrast ratio between the bars and spaces; and the physical dimension and/or resolution make job difficult for existing scanners at the point of sale.

Our prior art search has revealed, USPTO Patent No. 6,134,548 titled as "System, method and article of manufacture for advanced mobile bargain shopping" reveals a system that facilitates web-based comparison shopping in conventional, physical, non-web retail environments. A wireless phone or similar hand-held wireless device with Internet Protocol capability is combined with a miniature barcode reader (installed either inside the phone or on a short cable) and utilized to obtain definitive product identification by, for example, scanning a Universal Product Code (UPC) bar code from a book or other product. The wireless device transmits the definitive product identifier to a service routine (running on a Web server) , which converts it to (in the case of books) its International Standard Book Number or (in the case of other products) whatever identifier is appropriate. The service routine then queries the Web to find price, shipping and availability information on the product from various Web suppliers. This information is formatted and displayed on the handheld device's screen. The user may then use the hand-held device to place an order interactively.

International Application No. WO 2005/017812 titled as SYSTEM AND METHOD FOR DECODING AND ANALYZING BARCODES USING A MOBILE DEVICE is a system and method for decoding barcodes using a mobile device. Generally, the barcode image is acquired via a digital camera attached to the mobile device. After the barcode image has been acquired, software located on the mobile device enhances the barcode image and subsequently decodes the barcode information. The barcode information is then transmitted to a server via a wireless network. The server processes the barcode information and transmits media content related to the barcode back to the mobile device.

International Application no. WO 03/100528 titled as METHOD FOR THE WIRELESS TRANSMISSION OF IMAGES AND THE SUBSEQUENT RETRIEVAL THEREOF relates to a method for the wireless transmission of images and the subsequent retrieval thereof. The inventive method comprises steps consisting in acquiring the image using an image capture device, sending an identification document for the subsequent retrieval of the image by the user and processing said image in order to supply it to the user. The aforementioned identification document, which comprises encoding means, is sent directly by the image capture device that captures the image. The aforementioned image capture device transmits said image in a wireless manner to at least one image processing device where it is supplied to the user once the identification document has been used (passed through a barcode or card reader) . The image capture device can be a digital camera, a video camera, etc.

International Application No. WO 03/028273 titled as SYSTEM, METHOD, AND APPARATUS FOR COMMUNICATING INFORMATION BETWEEN A MOBILE COMMUNICATIONS DEVICE AND A BAR CODE SCANNER is a system, method, and apparatus for communicating information encoded in a bar code format between a mobile communications device and a bar code scanner. Transmission information data is encoded into a bar code format. A signal is generated from the bar code format to simulate a reflection of a scanning beam being moved across a static visual image of the bar code format; and the signal is transmitted as light pulses from the mobile communications device.

Our google search has also revealed a link: http://www.yangsky.com/barcode.htm with a product named BarSeer. BarSeer is a low-cost video barcode reader using only 640x480-pixel webcam! Anyone who has a USB webcam can turn a PC into a barcode scanner.

However we can see that despite the presence of efforts in this direction there is a lack of an efficient and cost effective way to decipher the bar codes. Prior art does not mention deciphering of bar codes from normal mobile phones by using a webcam. Moreover prior art also does not reveal any special bar code formats that are displayed by mobile devices for an easy reading into a webcam.

Thus there is an urgent need for an improved, adaptable and cost- effective system and method that allows for fast and efficient decoding of barcoding format when it is displayed on a mobile device.

SUMMARY An object of the present invention is to enable consumers conduct business transactions at the point of sale or entry, by enabling to redeem promotional vouchers such as tickets, loyalty cards or receipts through the use of mobile phones.

Another object of the present invention is to have barcodes specifically designed to be displayed and decoded directly off screens of mobile devices.

The present invention is a mechanism to decode a barcode for enabling consumers to conduct business transactions. A wireless device receives special barcode formats that are read by standard capturing device attached to a computer at a point of sale. In this manner, the consumer is able to redeem promotional vouchers such as tickets, loyalty cards or receipts through the use of mobile phones.

BRIEF DESCRIPTION OF THE DRAWINGS

The preferred embodiments of the invention will hereinafter be described in conjunction with the appended drawings provided to illustrate and not to limit the invention, wherein like designations denote like elements, and in which: The preferred embodiments of the invention will hereinafter be described in conjunction with the appended drawings provided to illustrate and not to limit the invention, wherein like designations denote like elements, and in which:

Fig 1 is a flow chart that depicts the method of the present invention to decode barcodes on a wireless device by use of a webcam/ capturing device.

Fig 2 depicts two tables with client side and server side activities (prior art) .

Fig 3 shows the schematic diagram of how the barcode image on the wireless device is decoded.

Fig 4 is an example of a specifically encoded barcode.

Fig 5 a shows the start and end bars;

Fig 5 b shows the locator blobs and

Fig 5 c shows the two 1-D barcodes

Fig 6a shows 1-D graphical representation of blob detector and Fig 6 b shows the points in the basins represented by thickened line. And Fig 6 c shows blobs of various sizes.

While the invention is susceptible to various modifications and alternative forms, specific embodiments of the invention are provided as examples in the drawings and detailed description. It should be understood that the drawings and detailed description are not intended to limit the invention to the particular form disclosed. Instead, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the invention as defined by the appended claims.

DESCRIPTION OF PREFERRED EMBODIMENTS

Fig 1 is a flow chart that depicts the method of the present invention to decode barcodes on a wireless device by use of a webcam/ capturing device. The figure shows the contemplated invention in the context of promotion, voucher, ticket and loyalty redemptions . The steps described in are as follows:

Step 1 : Consumer receives barcode on the wireless device, e.g. mobile phone. This can be done as part of a promotion or loyalty campaign where consumers Fig 1 is a flow chart that depicts the method of the present invention to decode barcodes on a wireless device by use of a webcam/ capturing device. The figure shows the contemplated invention in the context of promotion, voucher, ticket and loyalty redemptions. The steps described in are as follows:

Step 1 : Consumer receives barcode on the wireless device, e.g. mobile phone. This can be done as part of a promotion or loyalty campaign where consumers have subscribed to an offer and are subsequently sent the voucher as barcode on the phone. It can also be done whereby the consumer sends a text message or interacts via a WAP or WEB site to receive the voucher in the return message.

Step 2: Consumer holds mobile phone in front of a webcam at retail point of sale or entry. The consumer goes to the retail store to redeem the offer. At the point of sale, there is a standard webcam connected to a PC. The consumer holds the phone that is displaying the AMT Code barcode in front of the webcam. In some situations the retailer tiller person or ticket inspector can also do this.

Step 3 Barcode is decoded and the offer is redeemed through the PC application.

Fig 2 is a block diagram that depicts the method in which consumer interact to receive an AMT Code barcode on their wireless device. Interaction can be conducted by SMS, WAP interaction, MMS or HTTP over wireless bearers including GSM, GPRS, UMTS or WIFI. The wireless gateway receives the customer interaction and uses application logic to generate a number appropriate to the application. The number is then encoded using the mechanism detailed in this document to form the AMT Code barcode. This code is then encoded into a format appropriate for delivery back to the wireless device, e.g. SMS (picture message format) , MMS, HTTP, and WAP PUSH and then sent to the device.

Fig 3 shows the schematic diagram of how the barcode image on the wireless device is decoded. A standard imaging device such as a webcam is connected to a host machine, typically a standard personal computer (PC) or computer equipment capable of storing 8Mb of memory. A video grabber module reads the webcam images from the webcam, and grabs frames from the video stream at the fastest rate achievable by the camera. The still frames are then processed by the AMTCode decoder (described later in this document) and then the decoded number passed through to the host interface where it can interface to applications running on the host device.

Fig 4 is an example of a specifically encoded barcode -AMTCode barcodes are specifically designed to be displayed and decoded directly off screens of mobile devices. This barcode is designed to encode 20 numeric digits in a rectangular region of 72 x 14 pixels to allow this to be displayed almost all mobile devices.

The barcode format has the following compositions:

a) Fig 5 a shows the start and end bars, both consisting of 3 bars (dark, light, dark) of thickness 1 pixel, height 14 pixels.

b) Fig 5 b shows the locator blobs, 3 blobs of size 2x2 pixels and a centre blob of size 3x2 pixels, in positions shown above. These blobs provide position, scale and orientation information.

c) Fig 5 c shows the two 1-D barcodes

The bars are 5 pixels high and can take only 2 widths, wide (2 pixels) and narrow (1 pixel) . Each 1-D barcode is 66 pixels long. The first 60 pixels encode 10 digits. The last 6 encode a check number.

Each digit is encoded in 6 pixels, using either 1 wide bar and 4 narrow bars (5 combinations) or 2 wide bars and 2 narrow bars (6 combinations) . Each digit is encoded in 6 pixels, using either 1 wide bar and 4 narrow bars (5 combinations) or 2 wide bars and 2 narrow bars (6 combinations) , 3 wide bars (1 combination) are reserved for the check digit, and not use 6 narrow bars.

Each digit is represented using either 4 bars or 5 bars. Hence the total number of bars for the 10 digits is in the range 40 to 50 inclusive.

Check Number = number of bars - 40

In order to ensure the two locator bars are the same colour, each 1-D barcode must have an odd number of bars. We assign a symbol with an odd number of bars to represent a total that is even and vice versa.

The steps to decode the barcode are as follows:

Finding candidates for the locator blobs;

Finding candidate blob pairs to get approx position orientation and scale of the barcode;

Determining a line that goes through the centre of the barcode;

Moving up/down from this line following the start/end bars to read the 2 barcodes;

Detecting Blobs

Input: A grey scale image where intensity is in the range 0 - 255. Four integers SmallSize, LargeSize, IntensityDepth, IntensityHeight. Output: List of point and intensity (corresponding to the centre of gravity and lowest intensity of each blob detected) .

This blob detector is designed to find the 4 "locator blobs" along the middle of the barcode. The aim is to minimize the number of other blobs detected within the scene.

The method is based loosely on the 'watershed' algorithm, where we imagine the intensity to be a height. The dark blobs we are searching for are basins in this terrain.

Fig 6a shows 1-D graphical representation of blob detector and Fig 6 b shows the points in the basins represented by thickened line. The thickened line occurs at a particular intensity.

A valid blob is a basin whose maximum depth is greater than

(IntensityDepth + IntensityHeight) , and whose size is in-between

SmallSize and LargeSize.

Fig 6 c shows blobs of various sizes. To measure the size of a blob, we take the smallest area bounding box in the horizontal and diagonal orientations. The sides of this box must be in-between SmallSize and

LargeSize.

The following is an example of Algorithm

1) Sort points into order of increasing intensity. Start with a blank image.

2) Starting with the lowest intensity, add all the points with same intensity into the image. Any point that has no neighboring blobs (use 8 neighborhood), create a new blob. For any point with neighboring blobs, merge them altogether. Test to see if it is a valid blob.

3) A valid blob is very likely to merge with other blobs to produce another, larger, valid blob. All of them are not required. First the largest valid blob is located. Assuming this occurs at intensity level I. Construct blobs using all points within the blob with intensity less than I- IntensityHeight. Then select only those blobs that are larger than Smallsize and deeper than IntensityDepth.

4) Finally, for all valid blobs calculate the centre of gravity.

There are 5 test images. They are 24-bit bitmaps. Use the function, Intensity = 0.3 * r + 0.59 * g + 0.11 * b, to convert to grey-scale.

The barcode is 72x14, the centre blob is 3x2 and the other blobs are 2x2. So values for SmallSize and LargeSize can be determined by noting that if a blob is too small, then a bar of thickness 1 could not be read, similarly if the blob is too large then the barcode code not possibly fit within the image. Intensity Height and IntensityDepth are heuristically determined to minimize the number of false blobs detected.

Centre Blob Detector

Input: List of point and intensity (corresponding to the centre of gravity and lowest intensity of each blob detected) Output: List of point pairs For all pairs of points

Assume the first point is the centre point. Dismiss all point pairs that do not allow the barcode to be: entirely in the image too small to be read

Use the heuristic rule that the intensity of the points in-between the 2 blobs are higher than the highest intensity of the point pair. Also assume the intensity varies smoothly. Additional heuristic rules are used establish if there is an additional blob roughly half way between the pair, and also if there is an additional blob close to the leftmost blob of the pair.

Location Bar Detector and Decoder Input: List of point pairs

Output: 20 digit string or Fail string

For each point pair or until successful read:

Use decoder to read positions of blobs and start/end bars. Change orientation slightly if unsuccessful.

If successful, then move up (i.e. 90 degrees to the orientation of the point pair) carefully following the centre line of the start/end bars. use decoder to read barcode, continue moving up until a successful read, or stop if we have moved too far from the centre line, in which case the entire read has failed.

If we have a successful read, the repeat process moving down instead. If we have read the 2 barcodes successfully output digits, else output Fail.

Decoding the Barcode

Use method in "Barcode Waveform Recognition using Peak Locations" by Joseph and Pavlidis. This method does not measure the width of the bars, it measures the distance between the intensity peaks of 2 neighboring bars. Distance between peaks of 2 narrow bars = N Distance between peaks of 2 wide bars = W Distance between peaks of wide followed by narrow = M Distance between peaks of narrow followed by wide = M

The first 3 peak distances are almost sufficient to decode a symbol. To disambiguate the remainder, we look one bar further. For odd digits:

While the preferred embodiments of the invention have been illustrated and described, it will be clear that the invention is not limited to these embodiments only. Numerous modifications, changes, variations, substitutions and equivalents will be apparent to those skilled in the art without departing from the spirit and scope of the invention as described in the claims .

Claims

1. A mechanism to decode a barcode to enable consumers conduct a business transaction, such mechanism comprising, receiving a barcode format over a wireless device; reading the said barcode format by use of a standard capturing device attached to a computer, whereby the consumer holds the wireless device in front of the said capturing device at a point of business transaction; and decoding of the bar code by an application in the host attached to the said capturing device; whereby a consumer is able to redeem promotional vouchers such as tickets, loyalty cards or receipts.

2. A method as claimed in claim 1, wherein receiving a barcode format can be through SMS, WAP interaction, MMS or HTTP.

3. A barcode format of claim 1, wherein said barcode format are specifically encoded and designed to be displayed and decoded directly off screens of wireless devices.

4. A barcode format of claim 3, wherein the said barcode is designed to encode 20 numeric digits in a rectangular region of 72 x 14 pixels in order for it to be allowed to be displayed for almost all mobile devices.

5. A method of reading the said barcode format of claim 1 , further comprising a video grabber module that reads the images from the capturing device.

6. A method of reading the said barcode format of claim 5, wherein the frames from the video stream are read at the fastest rate achievable by the camera.

7. A method of decoding the barcode of claim 1, further comprising of the decoded number passing through to the host for interfacing to applications running on the said host device.

8. A system to decode barcodes, for enabling consumers to conduct business transactions, such system comprising, wireless device to receive specified barcodes for displaying; means of reading the said barcode format at the point of sale in order to decode it, by an application in the host attached to the said capturing device; whereby a consumer is able to redeem items such as vouchers, tickets, loyalty cards or receipts.

9. A system of claim 8, wherein a wireless device is a portable device capable of using bearer technologies (although not limited to) : GSM, GPRS, CDMA, UMTS or WIFI

10. A system of claim 8, wherein the means of reading the said barcode format is a standard capturing device, such as webcam, attached to a computer.

11. A barcode format of claim 8, wherein the barcode is encoded and designed to be displayed and decoded directly off screens of wireless devices.