WO2009148409A1

WO2009148409A1 - Method and system for extracting a barcode from a captured image

Info

Publication number: WO2009148409A1
Application number: PCT/SG2009/000194
Authority: WO
Inventors: Yiqun Li; Yue Wang; Kart Leong Lim; Hanlin Goh; Phuong Ngoc Nguyen; Ngan Meng Tan; Tat Jun Chin; Yilun You; Joo Hwee Lim
Original assignee: Agency for Science Technology and Research Singapore
Current assignee: Agency for Science Technology and Research Singapore
Priority date: 2008-06-03
Filing date: 2009-06-03
Publication date: 2009-12-10
Anticipated expiration: 2010-12-03

Abstract

A method and system for extracting a barcode from a captured image. The method comprises the steps of locating an object in the captured image; and extracting the barcode from the object.

Description

METHOD AND SYSTEM FOR EXTRACTING A BARCODE FROM A

CAPTURED IMAGE

FIELD OF INVENTION

The present invention broadly relates to a method and system for extracting a barcode from a captured image, to a barcode and to a method for interactive information access.

BACKGROUND

Using barcodes for encoding and subsequently recovering the encoded information is an existing technology that has been widely applied, e.g. in goods identification and tracking. There are many types of one-dimensional (1 D) or two- dimensional (2D) barcodes and respective barcode generation/recognition software currently in use. Examples of existing 1 D barcodes include the Universal Product Code (UPC) and the European Article Number (EAN). Examples of existing 2D barcodes include the Data Matrix code, the Quick Response (QR) code, the PDF 417 code and the Aztec code. Recognition of a barcode pattern is typically carried out as a line-by-line detection, which can be time-consuming.

It should be appreciated that most of the above codes were originally designed for being printed on blank and clear surfaces, e.g. paper. When the information encoded in the barcode is required to be recovered, a high-resolution scanner or camera is normally needed to capture the image of the barcode pattern. All the elements in the pattern are then recognized and the information is decoded. Since such barcodes are designed to be captured by a high-resolution imaging device, they usually contain many small elements so that a larger amount of information can be encoded.

There are many applications in broadcasting, publishing or advertising, etc. where the use of barcodes to provide interactive information is desirable. The information can also be delivered directly to an interested user via e.g. a mobile phone. In such instances, the user may first use a camera-enabled mobile phone to capture an image of a barcode for obtaining the interactive information.

However, there are also numerous challenges. Firstly, the surface containing a barcode, e.g. a billboard or a television (TV) screen, may be at a distance from the user and the barcode can occupy only a portion of said surface to avoid being obtrusive. As a result, the barcode may appear very small on the captured image, which usually has limited resolution, being taken from a mobile phone camera. Thus, detection of the barcode pattern can be difficult, especially as there can be complex background around the barcode. Secondly, the image may be captured within a short time, and may thus be blurred or out of focus. Thirdly, the surface containing the barcode may be a light source, e.g. an advertising board or a TV screen, potentially distorting the actual barcode pattern. Fourthly, the processing of the captured image may have to be carried out on the mobile phone, which has limited computing power. Therefore, complex algorithms are not suitable. There are a few prior art approaches in which a barcode pattern is used for encoding interactive information. In one approach, a proprietary 2D colour pattern comprising 4 colours and a 5x5 grid of cells, which can be of different shapes, is used. Said colour pattern can be printed on a newspaper or embedded in a TV program. However, the colours on the image captured by the mobile phone camera may be significantly different from the actual colours due to e.g. lighting conditions, TV display characteristics and camera optics. In addition, the colour pattern is required to be at least 50% of the entire captured image.

In some other approaches, the standard 2D Data Matrix code is used. However, these approaches usually require the barcode pattern to be clearly captured, in good resolution and to occupy at least 25% of the captured image. A need therefore exists to provide a method and system that seek to address at least one of the above problems.

SUMMARY In accordance with a first aspect of the present invention, there is provided a method of extracting a barcode from a captured image, the method comprising the steps of: locating an object in the captured image; and extracting the barcode from the object.

The method may comprise the steps of: binarizing the captured image; flood-filling the binarized image for removing substantially all background pixels having a first binary value surrounding the object having a second binary value; identifying a location of the object; and extracting the barcode from the object.

The method may further comprise converting the image into a grayscale image prior to the binarizing step.

The method may comprise the steps of: calculating an integral image of the captured image; identifying a location of the object in the captured image by applying a template of the barcode to the integral image, the template defining different regions having the same pixel value in the barcode; and extracting the barcode from the object.

The identification of the location of the object in the captured image by applying a template of the barcode to the integral image may comprise applying the template at different orientations.

The method may comprise the steps of: converting the captured image into a grayscale image; identifying a location of the object in the grayscale image using pixel-by-pixel processing of the grayscale image based on a template of the barcode, the template defining different regions having the same pixel value in the barcode; and extracting the barcode from the object.

The pixel-by-pixel processing of the grayscale image based on a template of the barcode may comprise a sequence of processing steps, such that a subsequent processing step is performed on a selected ones of the pixels based on a preceding processing step.

The method may comprise the steps of: binarizing the captured image; flood-filling the binarized image for identifying a location the object having a first binary value surrounded by a substantially continuous border having a second binary value; and extracting the barcode from the object.

The flood-filling of the binarized image for locating the object may comprise seeding the flood-filling based on user assisted localisation.

In accordance with a second aspect of the present invention, there is provided a barcode for use on a surface, the barcode comprising a pattern consisting of white and black cells, wherein the black cells are enlarged relative to the white cells such that spreading of the white cells into the black cells in a captured image of the barcode is reduced compared to having the white and black cells of substantially a same size.

The black and white cells may comprise a plurality of information cells and a plurality of validation cells, the validation cells being encoded based on the information cells.

The black and white cells may further comprise a parity cell, the parity cell being encoded based on the validation cells. A total number of black cells among the parity cell and the validation cells may be even.

In accordance with a third aspect of the present invention, there is provided a method for interactive information access, the method comprising the steps of: providing a barcode as defined in the second aspect on a surface; taking an image of at least a portion of the surface including the barcode; extracting the barcode from the image; decoding the barcode; and accessing information associated with the decoded barcode. . .

In accordance with a fourth aspect of the present invention, there is provided a method for interactive information access, the method comprising the steps of: providing a barcode on a surface; taking an image of at least a portion of the surface including the barcode; extracting the barcode from the image using a method defined in the first aspect; decoding the barcode; and accessing information associated with the decoded barcode.

In accordance with a fifth aspect of the present invention, there is provided a system^' for extracting a barcode from a captured image, the system comprising: ■ means for locating an object in the captured image; and means for extracting the barcode from the object.

The means for locating the object may comprise: means for binarizing the captured image; means for flood-filling the binarized image for removing substantially all background pixels having a first binary value surrounding one or more objects having a second binary value; and means for identifying a location of the object.

The means for locating the object may comprise: means for calculating an integral image of the captured image; and means for identifying a location of an object in the captured image by applying a template of the barcode to the integral image, the template defining different regions having the same pixel value in the barcode.

The means for locating the object may comprise: means for converting the captured image into a grayscale image; and means for identifying a location of an object in the grayscale image using pixel-by-pixel processing of the grayscale image based on a template of the barcode, the template defining different regions having the same pixel value in the barcode.

The means for locating the object may comprise: means for binarizing the captured image; and means for flood-filling the binarized image for identifying a location of the object having a first binary value surrounded by a substantially continuous border having a second binary value.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will be better understood and readily apparent to one of ordinary skill in the art from the following written description, by way of example only, and in conjunction with the drawings, in which:

Figure 1 shows a schematic diagram illustrating a system for providing interactive information according to an example embodiment.

Figure 2A shows a flow chart illustrating an implementation a Mobile Client Application according to an example embodiment.

Figure 2B shows a flow chart illustrating an implementation a Server Application according to an example embodiment. Figure 3A shows various elements of a barcode pattern according to an example embodiment. Figure 3B shows an arrangement of the cells in the data element of Figure 3A according to an example embodiment.

Figure 4 shows a conventionally designed barcode pattern and the corresponding camera-acquired pattern.

Figure 5 shows the barcode pattern of an example embodiment and the corresponding camera-acquired pattern. Figures 6A-6E show respective results after applying successive steps of a morphological operation process according to an example embodiment.

Figure 7 shows a gradient vector image of the pattern of Figure 6F according to an example embodiment.

Figure 8 shows a line accumulator histogram plot for finding the edges of the L- shape according to an example embodiment.

Figure 9A shows two substantially perpendicular lines formed on respective edges of the L-shape element according to an example embodiment.

Figure 9B show coordinates of three corner points of the barcode pattern according to an example embodiment.

Figure 10 is a schematic diagram illustrating the concept of Integral Image according to an example embodiment.

Figure 11A shows a diagram illustrating reference points for calculating an integral image of a black box in an upright position. Figure 11B shows a diagram illustrating reference points for calculating an integral image of a black box in a rotated position.

Figure 11C shows a diagram illustrating a decomposition of a rotated integral image into two complementary half integral images.

Figure 11 D shows a diagram illustrating sections used for calculating a rotated integral image according to an example embodiment.

Figure 12 shows a template of the barcode with the respective reference points and rectangular regions according to an example embodiment.

Figure 13 shows example results of barcode pattern detection using the integral images approach according to an example embodiment. Figure 14 shows a flowchart illustrating a method for barcode pattern detection using heuristic operations according to an example embodiment.

Figure 15A shows the sampling of each barcode cell center for barcode information extraction according to an example embodiment. Figure 15B shows the barcode pattern extracted after the sampling in Figure 15A.

Figure 16 shows a real-time image on the screen of the mobile phone according to an example embodiment.

Figure 17 shows a detailed flow chart 1700 illustrating a barcode pattern detection method based on user-assisted localisation of a coloured border element according to an example embodiment.

Figure 18A shows a captured image of a scene containing a plurality of barcode patterns disposed at various positions.

Figure 18B shows the image obtained after the colour binarization step based on the image of Figure 18A according to an example embodiment.

Figure 18C shows the image obtained after the flood-fill operation based on the image of Figure 18B. Figure 18D shows an isolated part corresponding to a desired barcode pattern extracted from the image of Figure 18C.

Figure 19A shows an isolated part with edges substantially parallel to the x- and y-axes. Figure 19B shows the isolated part of Figure 19A rotated at an angle for finding the corner points according to an example embodiment.

Figure 2OA shows a sampling grid for decoding barcode cell information according to an example embodiment. Figure 2OB shows an output of decoding the grid of Figure 2OA

Figure 21 shows a flow chart illustrating a method for decoding a barcode pattern according to an example embodiment. Figure 22 shows a flow chart illustrating a method for extracting a barcode from a captured image according to an example embodiment.

Figure 23 shows a block diagram of a computer system for implementing the method and system of the example embodiment.

Figure 24 shows a block diagram of a wireless device for implementing the method and system of the example embodiment.^'

DETAILED DESCRIPTION

Figure 1 shows a schematic diagram illustrating a system 100 for providing interactive information according to an example embodiment. In the example embodiment, the system is described in relation to a TV program broadcast. However, it should be understood by a person skilled in the art that the same system architecture can apply to a different medium, e.g. printed material or advertising board. In the example embodiment, a barcode pattern is first generated by an information provider. Said pattern is then embedded into a TV program video 102 during the post-production of the TV program using e.g. a computer server 104. When the TV program is being broadcast via e.g. a broadcasting station 106, a TV 108 receiving the broadcast signal displays the barcode pattern on a portion of the TV screen. A TV viewer can take a picture of the TV screen using e.g. a camera on a mobile phone 110, which has a client software preloaded therein. The mobile phone 110 extracts the barcode from the picture taken and information encoded in the barcode is sent to an Image Interpretation Server 114 via e.g. a GSM network 112.. The Image Interpretation Server 112 converts the information from the mobile phone 110 to a corresponding content link based on e.g. a predesigned look-up table, and sends the links back to the mobile phone 110 via said GSM network 112. The TV viewer may choose to access the contents provided by a Content Association Server 116 through the mobile phone 110.

In an alternate embodiment, the picture taken may be transferred to a computer server (not shown) having a barcode extraction software preloaded therein. The computer server extracts the barcode from the picture and information encoded in the barcode is sent to an Image Interpretation Server 114 via e.g. an Internet connection. The Image Interpretation Server 112 converts the information from the personal computer to a corresponding content link based on e.g. a predesigned look-up table, and sends the links back to the personal computer via said Internet connection. The TV viewer may choose to access the contents provided by a Content Association Server 116 through the personal computer.

In another alternate embodiment, the picture taken is sent from the mobile phone 110 via e.g. a GSM network 112 to the Image Interpretation Server 114 where the barcode extraction process is carried out, and the corresponding content link based on e.g. a predesigned look-up table is sent back to the mobile phone 110 via said GSM network 112. The TV viewer may choose to access the contents provided by a Content Association Server 116 through the mobile phone 110.

Some portions of the description which follows are explicitly or implicitly presented in terms of algorithms and functional or symbolic representations of operations on data within a computer memory. These algorithmic descriptions and functional or symbolic representations are the means used by those skilled in the data processing arts to convey most effectively the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities, such as electrical, magnetic or optical signals capable of being stored, transferred, combined, compared, and otherwise manipulated.

Unless specifically stated otherwise, and as apparent from the following, it will be appreciated that throughout the present specification, discussions utilizing terms such as "scanning", "calculating", "extracting", "computing", "generating", "initializing", "outputting", or the like, refer to the action and processes of a computer system, or similar electronic device, that manipulates and transforms data represented as physical quantities within the computer system into other data similarly represented as physical quantities within the computer system or other information storage, transmission or display devices. The present specification also discloses apparatus for performing the operations of the methods. Such apparatus may be specially constructed for the required purposes, or may comprise a general purpose computer or other device selectively activated or reconfigured by a computer program stored in the computer. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general purpose machines may be used with programs in accordance with the teachings herein. Alternatively, the construction of more specialized apparatus to perform the required method steps may be appropriate. The structure of a conventional general purpose computer will appear from the description below.

In addition, the present specification also implicitly discloses a computer program, in that it would be apparent to the person skilled in the art that the individual steps of the method described herein may be put into effect by computer code. The computer program is not intended to be limited to any particular programming language and implementation thereof. It will be appreciated that a variety of programming^" languages and coding thereof may be used to implement the teachings of the disclosure contained herein. Moreover, the computer program is not intended to be limited to any particular control flow. There are many other variants of the computer program, which can use different control flows without departing from the spirit or scope of the invention.

Furthermore, one or more of the steps of the computer program may be performed in parallel rather than sequentially. Such a computer program may be stored on any computer readable medium. The computer readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a general purpose computer. The computer readable medium may also include a hard-wired medium such as exemplified in the Internet system, or wireless medium such as exemplified in the GSM mobile telephone system. The computer program when loaded and executed on such a general-purpose computer effectively results in an apparatus that implements the steps of the preferred method.

As discussed above, the system according to the example embodiment comprises a Mobile Client Application and a Server Application. Figure 2A shows a flow chart 200 illustrating an implementation the Mobile Client Application according to an example embodiment. Figure 2B shows a flow chart 230 illustrating an implementation the Server Application according to an example embodiment.

The Mobile Client Application is installed in the user's mobile phone in a manner known to a person skilled in the relevant art. During the broadcast of the TV program, at step 202, the user activates the Mobile Client Application on the mobile phone in the same manner as other mobile phone applications. Once the application is activated, a screen of the mobile phone displays a real-time image 204 that is in the field of view of the phone camera.

When a barcode pattern that is of interest appears on the TV screen, the user aims the phone camera at the pattern and presses a key on the mobile phone to capture the image at step 206 in the example embodiment. At step 208, the image is captured in the mobile phone for further processing. At steps 210, 214, and 216 respectively, the application locates the barcode position on the captured image, recognizes the barcode and extracts the information from the barcode. In any one of the above steps 210, 214 and 216, if an output of the barcode pattern position, a recognition result or information extracted from the barcode pattern is not valid, an error message 212 is displayed on the screen of the mobile phone to prompt the user to take another image. If the outputs are all valid, the pattern information will be sent to the server 112 (Figure 1) at step 220, leaving a confirmation message 218 on the screen to inform the user that the extraction process is successful. The system of the example embodiment is advantageously not limited to a particular position of the barcode in the image.

In the example embodiment, the Server Application is installed in a computer server, in a manner known to a person skilled in the relevant art. At step 232, the server receives the pattern information from the mobile client. At step 234, the server associates the pattern information to a corresponding content link according to e.g. a look-up table provided. At step 236, the server sends the link back to the mobile client. The user would then be able to access the relevant content via said link.

Figure 3A shows various elements of a barcode pattern 300 according to an example embodiment. Since the barcode pattern 300 is embedded into the TV program video as a small portion (preferably less than about 6% of the entire video image) to avoid annoying the audience, the size of the barcode is very small after it is captured by a mobile phone camera. Therefore, the number of black and white cells in the barcode is preferably small such that the size of each cell can be at least 3x3 pixels. Ih the example embodiment, the barcode pattern 300 comprises a 6x6 lattice of black or white cells enclosed by a border element 310, which can be white or coloured (except black, to be discussed in detail below). The width of the border element 310 is preferably at least 3 pixels when captured. The lattice comprises a black L-shape identification element 320a, an inverted L-shape identification element 320b comprising alternating black and white cells, and a data element 330 comprising the central 4x4 cells.

Figure 3B shows an arrangement of the cells in the data element 330 of Figure 3A according to an example embodiment. Each cell represents a single bit of data and may be either white or black, corresponding to binary values of 0 and 1 respectively, as, will be appreciated by a person skilled in the art. In the data element 330 of the example embodiment with 16 binary coded cells, 11 information cells 332 are used to encode information, resulting in a representation power of 2048 unique combinations of codes. In addition, 4 validation cells 334 are used to validate the accuracy of the encoded information. The validation cells 334 are then further validated using a parity cell 336.

In the example embodiment, after the information cells 332 are encoded, the information cells 332 are validated. To perform validation of the information cells 332, a validation integer Vi₀ is first calculated by adding 2 to the number of white cells, i.e.:

where N = 11 is the size of the information string, /„ = 0 for a white information cell, and /„ = 1 for a black information cell.

Based on equation (1), the validation integer V₁₀ comprises an integer value between 2 and 13 (i.e. V₁₀ e {2, K, 13}), which is then converted to a four-bit binary string V₂. Each bit in the binary string V₂ corresponds to a validation cell 334 in the barcode 300, with white and black cells representing binary values 0 and 1 respectively. In addition, to detect errors in the validation cells 334, an even parity scheme is adopted in the example embodiment. The parity cell 336 is set to black if the number of black validation cells is odd, resulting in an even total number of black cells in the 5 validation and parity cells. The parity cell 336 is placed e.g. at the bottom left corner of the data element 330 in the barcode pattern 300.

Figure 4 show a conventionally designed barcode pattern 402 and the corresponding camera-acquired pattern 404. As is illustrated in Figure 4, the inventors have recognised that a problem faced with using barcodes on a TV screen is that the white portions tend to experience aberration. This is likely due to the barcode being displayed using a medium that is an active light source. The extent of the aberration may be dependent on both the camera optics and the camera sensor, e.g. cameras with low- quality lenses or sensors tend to acquire images with high levels of aberration. The aberration may be so severe that the white cells 408 spread extensively into black cells 406, possibly resulting in an erroneous decoding of the barcode pattern. The problem is especially apparent in the case of a black cell 406 being completely surrounded by 8 white cells 408, potentially resulting in the disappearance of the black cell 406. In a preferred embodiment, the size of the black cells is advantageously enlarged in the barcode pattern to reduce or minimise the problem described above. Figure 5 shows the barcode pattern 502 of an example embodiment and the corresponding camera-acquired pattern 504. As seen from Figure 5, by making the black cells 506 relatively larger than the white cells 508, when aberration is experienced and the white cells 508 spread into the black cells 506 after the image pattern is captured by a camera, the black cells 506 remain clearly . visible. Thus, the barcode design of the example embodiment can advantageously result in a more accurate extraction of the black and white cells even with aberration. Returning to Figure 2, after an image has been captured in step 206, the position of the barcode pattern in the image is located. In the example embodiment, location of the barcode pattern may be carried out using one of the following approaches:

- Morphological Operations and Hough Transform,

- Integral Images, - Heuristic Operations, or

- User-assisted Localisation of Coloured Border Element

Barcode Pattern Location using Morphological Operations and Hough Transform Figures 6A-6F show respective results after applying successive steps of a morphological operation process according to an example embodiment. As shown in Figure 6A, the original colour image taken by the mobile phone camera is first converted to a grayscale image. Next, a threshold is applied to binarize the grayscale image to obtain a binary image, as shown in Figure 6B. Since the barcode pattern is enclosed by a white border element 310 (Figure 3A), given a binary image of Ones' (white) and 'zeros' (black), a morphological reconstruction operation (i.e. flood-fill) with 4-connectivity to the image is used to fill holes of 'zeros' with 'ones' from all directions, resulting in the removal of all black background surrounding white objects. A border of the binary image of e.g. 1 -pixel width is preferably set to 'zeros' before performing flood-fill in order to avoid premature discontinuity in the flood-fill operation. As shown in Figure 6C, the background portions of the image surrounding the barcode pattern is almost completely converted to white after the flood-fill operation. Figure 6D shows a close-up view of the barcode portion in the image of Figure 6C. It can be seen from Figure 6D that some residual pixels 602, 604 remain after the flood-fill operation,

In the example embodiment, a series of morphological closing and opening is applied to clean up non-barcode pixel residues 602, 604 (Figure 6D) after the flood-fill operation. A close operation is used to remove fine pixels 604, while an open operation is applied to remove all connected pixels 602 that are fewer than P pixels, where P is prior information known by inspecting the minimum number of pixels of the L-shape identification element 320a (Figure 3A) of the barcode image taken at the maximum distance. As shown in Figure 6E, after a morphological close operation, fine pixels 604 are removed but connected pixels _.602 remain. As shown in Figure 6F, after a morphological open operation, even the connected pixels 602 are removed and only a possible barcode pattern remains. An advantage of the above operations is that by removing small residues and possibly some of the actual barcode pixels but essentially preserving the full L-shape structure, the efficiency of the detection task can be improved. The missing barcode pixels can be recovered after the L-shape structure has been detected by using the coordinates of the L-shape structure on the pre-open image . in Figure 6D.

Figure 7 shows a gradient vector image of the pattern of Figure 6F according to an example embodiment. A gradient direction Hough transform is employed in the example embodiment to detect the barcode orientation. By performing e.g. a 3x3 local gradient computation on the image, information on both the gradient magnitude and the vector directions (dx_n, dy_n) of each pixel is obtained. Further, by eliminating all gradients bearing zero magnitude, the edge points of the data are obtained in the example embodiment. Since each edge point has a gradient direction perpendicular to the line it lies on, votes of all the possible angles of each line can be accumulated, e.g. by increasing respective voting values by 1 in the bin. However, it should be appreciated that the 3x3 gradient computation window can only offer accuracy up to a specific angle resolution. In the example embodiment, a tolerance of ±20° is used for fine tuning. Thus, when performing Hough transform on all the edge points, the number of useless votes is reduced by restricting D_n = [Q_n ± 20°}, where the Hough equation is defined as:

X_nCOsD_n + y_nsinD_n = R_n (2) θ_n = tan-¹(dy_n ZcIx_n) (3) Figure 8 shows a line accumulator histogram plot for finding the edges of the L- shape according to an example embodiment. By progressively accumulating the votes of each potential line or Hough pair parameters (R_n, D_n) from each edge point into a one- dimensional line accumulator bin, after all the edge points are recorded, a first largest line from the dynamic accumulator bin is detected in the example embodiment by taking the highest voting value in the bin. A second line substantially perpendicular to the first line is then selected based on its highest vote at e.g. D₂ = {D_Ϊ+90°±10°}, whereby D₁ is the Hough angle of the first line.

Figure 9A shows two substantially perpendicular lines 902 and 904 formed on respective edges of the L-shape element according to an example embodiment. Figure

9B show coordinates of three corner points 906, 908 and 910 of the barcode pattern according to an example embodiment. After peaks corresponding to the two perpendicular lines are obtained, two arrays of coordinates are stored for each of the image data i.e. the black pixels having such Hough parameters. In addition, any edge points having very close pair parameters are merged to either of the two respective peaks. As a result, the coordinates of two substantially perpendicular lines 902 and 904 are obtained. In addition, the two sets of Hough parameters {R-,,Di & R₂,D₂) are used to solve for the coordinates of an intercept of the two lines 902 and 904 by manipulating Equation (2). Thus, the coordinates of the first corner point 906 in the example embodiment are obtained as follows:

The other two corner points 908 and 910 as shown in Figure 9B are solved by finding the largest magnitude of pixels from corner point 906 given the respective line parameters found above. Since line 904 lags behind line 902 by about 90°, the lines can be distinguished based on e.g. the difference in D_n. The 3 corner points 906, 908 and 910 can accurately describe the location of the barcode pattern in the image.

It will be appreciated that there can be cases where multiple L-shape elements are detected due to e.g. complex environment. In the example embodiment, false detections are removed e.g. by inspecting the inverted L-shape element 320b (Figure 3A) diagonally opposite the L-shape element 320a (Figure 3A) for alternating black and white cells.

Barcode Pattern Location using Integral Images

Figure 10 is a schematic diagram illustrating the concept of Integral Image according to an example embodiment. An integral image of an image l(x,y) can be defined as follows:

That is, Il(p,q) contains trie sum of intensities of all pixels at coordinates (x,y), where x < p and y ≤ q, i.e. above and to the left of (p,q) in the example embodiment in which the origin is at the top left corner of the diagram in Figure 10.

Given an input image l(x,y), its Integral Image can be calculated very efficiently in one pass over the image, as described by the equations below:

where RS(p,q) is the cumulative row sum of the p-th row until the q-th column. That is, to compute the Integral Image function at (p,q), the previously computed value of Il{p-\,q) is added to the cumulative row sum of the current row (the p-th row) until the q-th column.

Figure 11A shows a diagram illustrating reference points for calculating an integral image of a black box in an upright position. Figure 11B shows a diagram illustrating reference points for calculating an integral image .of a black box in a rotated position. Figure 11C shows a diagram illustrating a decomposition of a rotated integral image into two complementary half integral images. Figure 11 D shows a diagram illustrating sections used for calculating a rotated integral image according to an example embodiment.

Using integral image, the sum of intensities of pixels in a rectangle or box can be computed very quickly. For the black box shown in Figure 11 A, the sum of pixel intensities under the black box can be calculated as:

S = Il{xA,yA) - Il(x2,y2) - Il(x3,y3) + Il{x\,y\) (8) Equation (8) shows that computing the sum of intensities of pixels under the black box requires only 4 references of the Integral Image, regardless of the size of the box (i.e. regardless of the number of pixels under the box). The above computation may be significantly faster than a method of accessing every single pixel intensity and accumulating their values, where the computational time increases with each increase in the size of the box.

In the example embodiment, Integral Images are used to locate the L-shape identification element 320a (Figure 3A) in the image, since the barcode pattern comprises a configuration of boxes which make up the L-shape. Figure 12 shows a template 1200 of the barcode with the respective reference points and rectangular regions according to an example embodiment. The template 1200 is defined by a configuration of 16 points, represented by p1, p2... p16, which define 6 regions, i.e. Region 1 , Region 2... Region 6, respectively. When the template 1200 is applied onto the image at e.g. point p1 = (x1 ,y1), the sums of intensities of pixels under the 6 regions using the Integral Image of the image are as follows:

SR₁ = II(x7,y7) - II(x2,y2) - II(x3,y3) + II(x 1,y1) SR₂ =II(χ12,y12) - II(x4,y4) -II(x11,y11) +II(x3,y3) SR₃ = II(x9,y9) - II(x5,y5) - II(x8,y8) + II(x4,y4) SR₄ = II(x14,y14) - II(x7,y7) - II(x13,y13) + II(x6,y6)

SR₅ =II(x13,y13) -II(x10,y10) - II(x12,y12) +II(x8,y8) SR₆ = II(x16,y16) - II(x14,y14) -II(x15,y15) +II(x11,y11) where SR_n denotes the sum of pixel intensities under Region n, and (x1 ,y1 , (x2,y2), ..., (x16,y16) denote the coordinates of points p1, p2 p16 respectively.

With the sums of pixel intensities of the 6 regions, a Logo Response Value (LRV) of the image location p1 is calculated in the example embodiment as:' _LRV JSR_{1 +}SR_{2 +}SR_{4 +}SR₆) (SR_{3 +}SR₅) _(g)

The 1st term on the right hand side of Equation (9) corresponds to the regions in the barcode where the pixels are supposed to be white, while the 2nd term on the right hand side of equation (9) corresponds to the regions in the barcode where the pixels are supposed to be black. P_w and P_B correspond to the number of pixels in the template 1200 which are expected to be white and black respectively, and act as normalization factors to ensure that the 1st and 2nd terms are scaled between e.g. 0 and 255. It will be appreciated that if the template 1200 is placed on a location in the image where the barcode pattern is found, the LRV will be very high. Hence, in the example embodiment, given an input image, the template 1200 is applied at all positions in the image and the respective LRVs are calculated. The location having the barcode pattern (if it exists in the image), produces the highest LRV. The above operation can be very quick since all that is done is to compute the coordinates of 16 points and reference the integral image to obtain the respective sums of pixel intensities.

As the actual size of the barcode in the input image is not known, in the example embodiment, the template 1200 is re-sized and re-applied to all points in the image and the respective LRVs are re-calculated, for all . desired template sizes. It will be appreciated that only the point configuration of the template 1200 is re-sized at each round. The same integral images can be re-used.

Figure 13 shows example results of barcode pattern detection using the integral images approach described above. It can be seen from Figure 13 that multiple responses 1302 may be generated around the location of the barcode pattern. In the example embodiment, the barcode pattern is identified as the response having the largest bounding box. The above approach is described in relation to a substantially upright pattern. In situations where the pattern is rotated, the integral images approach can still be used by:

(a) rotating the input image at multiple angles and redo the operations above, or

(b) computing "rotated" integral images for multiple angles from the original image.

It will be appreciated that the angles of rotation to be considered by the system of the example embodiment do not have to span the whole range of 360 degrees, since the user is expected to capture the image at roughly upright orientations. Also, very small angle resolutions do not need be considered since the LRVs computed above have a degree of tolerance towards slightly rotated barcodes. For example, the angles of rotation in the range of e.g. [-35°, -30°, -25°, -20°, ..., 0° 20°, 25°, 30°, 35° ] are found to be compatible with the system of the example embodiment. The solution described in (a) should be apparent to a person skilled in the art and will not be discussed herein. For the solution described in (b), the integral images are computed at a desired angle without rotating the input image. Figure 11B illustrates an example of a rotated integral image. To process an image with rotated integral images, the barcode template has to be rotated accordingly. In the example embodiment, to calculate a rotated integral image, the rotated integral image is decomposed into two complementary half integral images as illustrated in Figure 11 C. A half integral image at angle can be calculated as: hir (x, y) = RS(x, y) + hll^a (x - L^a (y), y - \) (10)

Where L° denotes a ladder vector at angle or and is obtained e.g. based on an algorithm as shown in Table 1:

Table 1

Input: Desired angle a>+ dimensions of input image.

Initialize: g = tan(π/2 — a), e = g, f = 0_} r = 0,

L^α = [ ] fniill vector).

1 , while r =£ max{rows_a cols) do

2 . ϊTe < gβ

3 . c = e + g

4 , / = /+ 1 else

6. c = e — l

7. TF *— / (append)

8 . r = r + l

9. f = Q

10 , end if

11. «3nd. wMIe

Output: Vector L^α of length max(iDW5, cols).

Further, as shown in Figure 11 D, in the example embodiment, the relevant sections for calculating the rotated integral image are Q and R, which can be expressed as:

Q + R = (P + Q + R + S) - (P+Q) + Q - S (11) where P + Q + R + S is the upright integral image II(x, rows)

P + Q is the upright integral image H(x,y) Q is the half integral image Mf (x, y)

S = hll^a(x,y + ϊ) , which is the outcome of rotating the image clockwise by 90°, computing hlf and reversing the rotation. This can be achieve e.g. by changing the order of iteration from left-to-right-then-top-to-bottom to bottom-to-top-then-left-to- right when calculating the cumulative row sum and the half integral image.

Based on Equation (11), the rotated integral image is obtained in the example embodiment as:

ir (X₁ y) = JJ(JC, rows) - II(x, y) + hll^a (x, y) - hlf (x, y + 1) (12) In the method and system of the example embodiment, the algorithm first tries to detect the non-rotated (i.e. upright) barcode. If that fails, it then tries to detect the barcode for the rotated cases using 'rotated integral image' as described above, until a valid barcode is detected or all the rotation angles have been tried. Barcode pattern location using Heuristic Operations

Figure 14 shows a flowchart 1400 illustrating a method for barcode pattern detection using heuristic operations according to an example embodiment. The method exploits the design of the 2D barcode pattern as described in Figure 3A to detect and locate said barcode within an image. Since the barcode pattern is enclosed by a white border element 310 and comprises a black L-shape identification^' element 320a (Figure 3A), the method of the example embodiment first seeks to locate the bottom left corner of the L-shape element 320a. A best case is built and refined progressively for finding the correct pattern. At step 1402a, the captured imaged is converted to a grayscale image in the same manner as described above with respect to the morphological operations. For example, if a pixel has large variance between colour channels, the pixel is set to neutral gray level, else, the pixel is converted to grayscale based on Red-Green-Blue (RGB) information. As the barcode pattern only contains non-coloured pixels, the best case is initialised accordingly at step 1402b.

At step 1404a, all pixels are checked to determine if they are black. This is based on the heuristic that the pixel at the bottom left corner of the L-shape element 320a is black. Accordingly, the best case is updated at step 1404b to be the blackest pixel.

Starting at a predetermined angle, at step 1406a, the pixels adjacent to each black pixel found in step 1404a are checked to determine if they are white. This is based on the heuristic that the pixel at bottom left corner of the L-shape element 320a has at least 2 white pixels within a close range at an angle close to 90 degrees. At step 1406b, the best case is updated to be the black pixel with the largest mean white to black difference.

At step 1408, the L-shape is grown based on the current angle. At step 1410, the opposite corner (i.e. the top right corner) is found and checked to determine if it is white. This is based on the heuristic that the pixel at top right corner of the barcode pattern is white.

At step 1412a, the width:height ratio of the L-shape is determined. This is based on the heuristic that the barcode pattern is substantially square, i.e. the ratio is close to 1. Accordingly, at step 1412b, the best case is update to be the L-shape that is most square.

At step 1414a, the cells on the L-shape element and the diagonally opposite sides are sampled. This is based on the heuristic that the L-shape element 320a of the barcode pattern comprises black cells and the inverted L-shape element 320b of the barcode pattern comprises alternating black and white cells. The best case is accordingly updated at step 1414b.

Steps 1406 to 1414 are then repeated for all other angles within a desired range based on step increments. At step 1416, the best barcode pattern is returned. The heuristic method of the example embodiment is invariant to rotation and is able to handle slightly skewed barcodes. Furthermore, the operations are ordered by their respective computational simplicity such that simpler operations are performed before more complex one. For example, if a pixel intensity is detected as not close to black, it is unlikely that it is the pixel at the bottom left corner of the L-shape element, and further operations can be skipped.

Using any one of the above approaches, the exact location of the barcode pattern and the length of the L-shape element are found in the example embodiment. Figure 15A shows the sampling of each barcode cell center for barcode information extraction according to an example embodiment. Figure 15B shows the barcode pattern extracted after the sampling in Figure 15A.

In the example embodiment, the length of the barcode in each X and Y-axis of the L-shape element is divided by the number of barcode cells (i.e. six) in each axis for obtaining the average displacement changes, Vx and Vy. Both values are used to determine the average distance for moving from a particular cell to its adjacent cell in the image. Further, both Vx and Vy are used to calculate the center, e.g. 1502, of each cell.

From the average value of the 3x3 neighbourhood pixels of each center, the respective cell is determined as 'black' or 'white' in the example embodiment.

Barcode Pattern Detection with User-assisted Localisation of Coloured Border Element

In this approach, the localisation of the barcode pattern is assisted by the TV viewer such that the subsequent detection process can be simplified. Figure 16 shows a real-time image 204 (Figure 2) on the screen of the mobile phone according to an example embodiment. A dotted rectangle 1602 is provided such that the pattern to be captured is preferably located within the rectangle. In addition, a sign, e.g. a cross 1604, is provided approximately at the center of the real-time image 204 for guiding the user to position the barcode substantially at the center of the real-time image 204.

Figure 17 shows a detailed general flow chart 1700 illustrating a barcode pattern detection method with user-assisted localisation of a coloured border element according to an example embodiment. In addition or as an alternative to the approaches described above, the method of Figures 17 uses a border element 310 (Figure 3A) that is coloured.

As shown in Figures 17, at step 1702, an image of the barcode pattern is captured. In the method of the example embodiment, the image is preferably captured with the barcode pattern positioned substantially at the center of the image. This can be facilitated by the target-assisting cross 1604 (Figure 16) that guides the user to align the barcode pattern with the cross 1604 before capturing the image. The barcode pattern is thus localised approximately at the center of the image. This saves time and computational power in the subsequent steps, making the method of the example embodiment very suitable for implementation on mobile devices.

From the imaged captured in step 1702, a binarization based on colour is carried out in the example embodiment. In the binarization step 1606 all coloured pixels in the image are located. A coloured pixel is defined in the example embodiment as one that is not gray (i.e. RGB values are not equal) in the example embodiment. The binarization is implemented e.g. by applying the pixel's RGB values to the function C(R, G, B) as shown below: 1 |R - G| > Th_e or |R - B| > Th_c or |G - B| > Th₀ C(R, G, B) =

0 otherwise where Th₀ is a colour threshold 1704. For various lighting conditions, different colour thresholds 1704 are selected accordingly, e.g. if the picture is dark, the threshold 1704 should be small, etc. The colour threshold 1704 can be calculated e.g. based on the image histogram or by linear checking all of the thresholds in the desired range, e.g. [0, 255]. In the example embodiment, due to the limited computing power of the mobile phone processor, only one or few common thresholds are used.

At step 1708, a flood-fill operation is carried out from substantially the image center based on grayscale pixels to isolate the barcode pattern. In the example embodiment, a pseudocode for the flood-fill operation is as follows:

1. Initialize the queue: Q = O Initialize the isolated-part set: S_ip = {}

2. Find the nearest not-coloured point p₀ from the center (W/2, H/2) Enqueue it: Q <— p_c 3. If Q is empty go to step 7.

4. Dequeue the top element of Q, p_t

Add it into the isolated-part set: S_ip <— p_t

5. Find the set of neighbor points of p_t that is not-coloured and not in Sj_P: S_nb

6. Enqueue the set: Q <— S_nb Go back to step 3.

7. Return the isolated-part set.

At step 1710, the corners of the isolated part are detected in the example embodiment, e.g. by examining the minimum and maximum values of x- and y- coordinates of every point in the isolated part. If the boundaries of the isolated part are parallel with the horizontal/vertical axes, there can be many points with maximum/minimum x- or y-coordinates. In such cases, the isolated part is rotated by e.g. 45 degrees, before the maximum/minimum x- or y-coordinates are examined. Figure 18A shows a captured image of a scene containing a plurality of barcode patterns 1802a-e disposed at various positions. Figure 18B shows the image obtained after the colour binarization step based on the image of Figure 18A according to an example embodiment. Figure 18C shows the image obtained after the flood-fill operation based on the image of Figure 18B. Figure 18D shows an isolated part 1806 corresponding to a desired barcode pattern 1802a extracted from the image of Figure 18C.

The desired barcode pattern 1802a is positioned substantially at the center of the image of Figure 18A. As seen from Figure 18B, after the colour binarization step, the black and white pixels in the image of Figure 18A are converted to white, while the coloured pixels are converted to black in the example embodiment. Boundary regions 1804a and 1804b in the binary image correspond to the coloured border elements of barcode patterns 1802a and 1802b respectively. As shown in Figure 18C, the flood-fill operation in the example embodiment starts at approximately the center point of the image. The operation is stopped after reaching the boundary region 1804a. As a result, an isolated part 1806 corresponding to the desired barcode pattern 1802a is located on the image. From the isolated part 1806, respective corner points AO, A1 , A2, A3 are determined as described above.

Figure 19A shows an isolated part 1902 with edges substantially parallel to the x- and y-axes. Figure 19B shows the isolated part 1902 of Figure 19A rotated at an angle for finding the comer points according to an example embodiment. As described above, the isolated part 1902 is rotated by e.g. 45 degrees such that new corner points AO', A1',

A2' and A3' can be unambiguously determined.

Referring to Figure 17, at step 1714, the contents of the pattern isolated from the binarization and flood-fill steps is decoded to check if it is a valid barcode, based on a binary threshold 1712. If the decoding fails, a different binary threshold 1712 is selected and the checking is repeated. If the decoding is still unsuccessful after all available binary thresholds 1712 are used, a different colour threshold 1704 is selected and the color-based binarization and flood-fill steps are repeated, followed by the decoding of the barcode. If no barcode is obtained even after all colour thresholds 1704 are exhausted, a message is displayed e.g. to prompt the user to capture another image.

After the four corner points are obtained as described above, the coordinates of the corner points are used as reference points for barcode sampling in the an example embodiment based on the grayscale image obtained after the binarization step 1706 (Figure 17). Figure 2OA shows a sampling grid 2000 for decoding barcode cell information according to an example embodiment. Figure 2OB shows an output of decoding, the grid 2000 of Figure 2OA. In the example embodiment, grid 2000 comprises 6x6 squares. The value of each square is determined by calculating average grayscale level of every pixel in that square, in which:

1 grayscale < Th_bin .

I (grayscale) =

0 otherwise where Th^_n is the binary threshold 1712 (Figure 17). In the example embodiment, after sampling the barcode, a 2-dimensional array of size 6x6 is obtained. Further, in the validation step, the array is also checked for the presence of the black L-shape element, and an inverted L-shape element comprising alternate black and white cells. Further decoding steps are described below. The following describes a decoding process that can be applied to a barcode pattern obtained from any one of the approaches as described above.

Figure 21 shows a flow chart 2100 illustrating a method for decoding a barcode pattern according to an example embodiment.

At step 2102, parity error is detected based on the design rule the number of black cells in (Vi₁V₂, V₃, V_4, P₁) is even. At step 2104, the validation range is checked. In the example embodiment, the validation string {v-i_, v₂, V₃, v₄} is converted into the corresponding validation integer V_w, which is then checked for falling within the range 2 ≤ V₁₀ ≤ 13. At step 2106, the number of black information cells N_B is computed, i.e.:

N_B = N + 2-V₁₀ where N = 11 is the total number of information cells.

At step 2108, the number of black information cells extracted from the barcode pattern is compared with the value of N_B obtained from step 2106.

At step 2110, encoded information is extracted. In the example embodiment, the information cells are extracted to form an integer between 0 and 2047. The integer is then used to extract more information, e.g. a content link, from e.g. a look-up table.

Figure 22 shows a flow chart illustrating a method for extracting a barcode from a captured image according to an example embodiment. At step 2202, an object is located in the captured image. At step 2204, the barcode is extracted from the object.

The method and system of the example embodiment can be implemented on a computer system 2300, schematically shown in Figure 23. It may be implemented as software, such as a computer program being executed within ,the computer system 2300, and instructing the computer system 2300 to conduct the method of the example embodiment.

The computer system 2300 comprises a computer module 2302, input modules such as a keyboard 2304 and mouse 2306 and a plurality of output devices such as a display 2308, and printer 2310.

The computer module 2302 is connected to a computer network 2312 via a suitable transceiver device 2314, to enable access to e.g. the Internet or other network systems such as Local Area Network (LAN) or Wide Area Network (WAN). The computer module 2302 in the example includes a processor 2318, a

Random Access Memory (RAM) 2320 and a Read Only Memory (ROM) 2322. The computer module 2302 also includes a number of Input/Output (I/O) interfaces, for example I/O interface 2324 to the display 2308, and I/O interface 2326 to the keyboard 2304.

The components of the computer module 2302 typically communicate via an interconnected bus 2328 and in a manner known to the person skilled in the relevant art. The application program is typically supplied to the user of the computer system 2300 encoded on a data storage medium such as a CD-ROM or flash memory carrier and read utilising a corresponding data storage medium drive of a data storage device 2330. The application program is read and controlled in its execution by the processor 2318. Intermediate storage of program data maybe accomplished using RAM 2320. The method of the current arrangement can be implemented on a wireless device 2400, schematically shown in Figure 24. It may be implemented as software, such as a computer program being executed within the wireless device 2400, and instructing the wireless device 2400 to conduct the method.

The wireless device 2400 comprises a processor module 2402, an input module such as a keypad 2404, an output module such as a display 2406 and a camera module 2407. The camera module 2407 may comprise e.g. a Charge- Coupled Device (CCD) or a Complementary Metal Oxide Semiconductor (CMOS) image sensor (not shown) capable of capturing still images of objects.

The processor module 2402 is connected to a wireless network 2408 via a suitable transceiver device 2410, to enable wireless communication and/or access to e.g. the Internet or other network systems such as Global System for Mobile communications (GSM) network, Code-Division Multiple Access (CDMA) network, Local Area Network (LAN), Wireless Personal Area Network (WPAN) or Wide Area Network (WAN).

The processor module 2402 in the example embodiment includes a processor 2412, a Random Access Memory (RAM) 2414 and a Read Only Memory

(ROM) 2416. The processor module 2402 also includes a number of Input/Output

(I/O) interfaces, for example I/O interface 2418 to the display 2406, and I/O interface

2420 to the keypad 2404. The components of the processor module 2402 typically communicate via an interconnected bus 2422 and in a manner known to the person skilled in the relevant art.

The application program is typically supplied to the user of the wireless device 2400 encoded on a data storage medium such as a flash memory module or memory card/stick and read utilising a corresponding memory reader-writer of a data storage device 2424. The application program is read and controlled in its execution by the processor 2412. Intermediate storage of program data may be accomplished using RAM 2414.

It will be appreciated by a person skilled in the art that numerous variations and/or modifications may be made to the present invention as shown in the specific embodiments without departing from the spirit or scope of the invention as broadly described. The present embodiments are, therefore, to be considered in all respects to be illustrative and not restrictive.

Claims

1. A method of extracting a barcode from a captured image, the method comprising the steps of: locating an object in the captured image; and extracting the barcode from the object.

2. The method as claimed in claim 1 , the method comprising the steps of: binarizing the captured image; flood-filling the binarized image for removing substantially all background pixels having a first binary value surrounding the object having a second binary value; identifying a location of the object; and extracting the barcode from the object.

3. The method as claimed in claim 2, further comprising converting the image into a grayscale image prior to the binarizing step.

4. The method as claimed in claim 1, the method comprising the steps of: calculating an integral image of the captured image; identifying a location of the object in the captured image by applying a template of the barcode to the integral image, the template defining different regions having the same pixel value in the barcode; and extracting the barcode from the object.

5. The method as claimed in claim 4, wherein identifying the location of the object in the captured image by applying a template of the barcode to the integral image comprises applying the template at different orientations.

6. The method as claimed in claim 1, the method comprising the steps of: converting the captured image into a grayscale image; identifying a location of the object in the grayscale image using pixel-by-pixel processing of the grayscale image based on a template of the barcode, the template defining different regions having the same pixel value in the barcode; and extracting the barcode from the object.

7. The method as claimed in claim 6, wherein the pixel-by-pixel processing of the grayscale image based on a template of the barcode comprises a sequence of processing steps, such that a subsequent processing step is performed on a selected ones of the pixels based on a preceding processing step.

8. The method as claimed in claim 1, the method comprising the steps of: binarizing the captured image; flood-filling the binarized image for identifying a location the object having a first binary value surrounded by a substantially continuous border having a second binary value; and extracting the barcode from the object.

9. The method as claimed in claim 8, wherein flood-filling the binarized image for locating the object comprises seeding the flood-filling based on user assisted localisation.

10. A barcode for use on a surface, the barcode comprising a pattern consisting of white and black cells, wherein the black cells are enlarged relative to the white cells such that spreading of the white cells into the black cells in a captured image of the barcode is reduced compared to having the white and black cells of substantially a same size.

11. The barcode as claimed in claim 10, wherein the black and white cells comprise a plurality of information cells and a plurality of validation cells, the validation cells being encoded based on the information cells.

12. The barcode as claimed in claim 11 , wherein the black and white cells further comprise a parity cell, the parity cell being encoded based on the validation cells.

13. The barcode as claimed in claim 12, wherein a total number of black cells among the parity cell and the validation cells is even.

14. A method for interactive information access, the method comprising the steps of: providing a barcode according to any one of claims 10 to 13 on a surface; taking an image of at least a portion of the surface including the barcode; extracting the barcode from the image; decoding the barcode; and accessing information associated with the decoded barcode.

15. A method for interactive information access, the method comprising the steps of: providing a barcode on a surface; taking an image of at least a portion of the surface including the barcode; extracting the barcode from the image using a method according to any one of claims 1 to 9; decoding the barcode; and accessing information associated with the decoded barcode.

16. A system for extracting a barcode from a captured image, the system comprising: means for locating an object in the captured image; and means for extracting the barcode from the object.

17. The system as claimed in claim 16, wherein the means for locating the object comprises means for binarizing the captured image; means for flood-filling the binarized image for removing substantially all background pixels having a first binary value surrounding one or more objects having a second binary value; and means for identifying a location of the object.

18. The system as claimed in claim 16, wherein the means for locating the object comprises means for calculating an integral image of the captured image; and means for identifying a location of an object in the captured image by applying a template of the barcode to the integral image, the template defining different regions having the same pixel value in the barcode.

19. The system as claimed in claim 16, wherein the means for locating the object comprises means for converting the captured image into a grayscale image; and means for identifying a location of an object in the grayscale image using pixel-by-pixel processing of the grayscale image based on a template of the barcode, the template defining different regions having the same pixel value in the barcode.

20. The system as claimed in claim 16, wherein the means for locating the object comprises means for binarizing the captured image; and means for flood-filling the binarized image for identifying a location of the

■object having a first binary value surrounded by a substantially continuous border having a second binary value.