GB2451073A - Checkout surveillance system - Google Patents
Checkout surveillance system Download PDFInfo
- Publication number
- GB2451073A GB2451073A GB0713750A GB0713750A GB2451073A GB 2451073 A GB2451073 A GB 2451073A GB 0713750 A GB0713750 A GB 0713750A GB 0713750 A GB0713750 A GB 0713750A GB 2451073 A GB2451073 A GB 2451073A
- Authority
- GB
- United Kingdom
- Prior art keywords
- product
- surveillance
- boundary
- image
- scan
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G07G1/00—Cash registers
- G07G1/0036—Checkout procedures
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G07G1/00—Cash registers
- G07G1/0036—Checkout procedures
- G07G1/0045—Checkout procedures with a code reader for reading of an identifying code of the article to be registered, e.g. barcode reader or radio-frequency identity [RFID] reader
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
- G07G3/00—Alarm indicators, e.g. bells
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
Abstract
A checkout surveillance system comprises a scanner which scans product bar codes as they pass a first boundary. A video camera is provided to enable recording of images of a surveillance area in which the product scanning occurs. A signal processor receives notifications of positive scan events when a bar code is successfully scanned. The processor also processes captured images to track their movement and identify a 'no-scan' event when a product passes a second boundary in the absence of a positive scan. Upon a no-scan event, the processor may activate an alarm to notify the occurrence of an unsuccessful scan. The processor may also comprise means to aid locating the product in an image by removing background detail, noise, shadows and skin/clothing features.
Description
SURVEILLANCE SYSTEM
The present invention relates to a surveillance system, in particular for monitoring a point of sale in a retail outlet.
Retail outlets, such as shops and supermarkets are known to monitor the sale of goods by scanning a product, for example by scanning a barcode label on a product at the point of sale, for example at the checkout of the retail outlet.
The scanning process is also used in the product sale process. In addition, using a scanned barcode to monitor product sales enables retail outlets to monitor their stock levels and to re-order products when stocks run low.
However, some products, although they are passed across the point of sale, are not scanned in the product sale process. There may be several reasons for this, some of which are accidental and some of which constitute acts of theft, either by staff or customers of the retail outlet. Accordingly, there is a need for an unobtrusive surveillance system for detecting products which pass through a checkout but which are not scanned.
The present invention aims to overcome at least some of these problems by providing a surveillance system for monitoring retail outlet transactions in which a scanner scans at least one product during the transaction, comprising: a camera for recording images of a surveillance area where the scanning occurs, so as to generate a series of sequential images; and a signal processor: for receiving notification of scans made by such a scanner; for processing the images by tracking the movement of such a product in the images; and for identifying a no-scan event when such a product is tracked across an exit boundary within the images without an associated notification of a scan; wherein the exit boundary is located in each image at a position corresponding to a boundary within the surveillance area which is passed by such a product during or after it undergoes scanning. It has been found that events where a product is passed across a checkout and no bar code reading is made can be effectively identified by a surveillance system tracking the product over an exit boundary and correlating this with bar code readings made by the checkout scanner. It is preferred that the exit boundary is located in each image at a position corresponding to a boundary within the surveillance area which is passed by such a product after it undergoes scanning. The signal processor may process the images and identify no- scan events either in real time as the images are recorded or at a later time by pro-cessing pre-recorded images.
In order to improve the accuracy with which no-scan events are located, it is preferred that the signal processor comprises means for identifying a no-scan event when such a product is tracked sequentially across an entry boundary and then across the exit boundary without an associated notification of a scan and wherein the entry boundary is located in each image at a position corresponding to a boundary within the surveillance area which is passed by such a product before it undergoes scanning. It is preferred that the signal processor comprises means for identifying a no-scan event when such a product is tracked sequentially across an entry boundary and then across the exit boundary without an associated notification of a scan in the time period during which the product is between the boundaries.
There may be more than one entry boundary and more than one exit boundary. For example, there may be more than one route that a product travels across a surveillance area and so multiple sets of, entry and exit boundaries may be required to accommodate the different routes. In addition an interim boundary located between the entry and the exit boundary may be added in order to improve accuracy. In this case a product may be tracked over the entry, then the interim and then the exit boundary, before a no-scan event is identified.
The camera may be a video camera, in particular a closed circuit television (CCTV) camera. Preferably, the camera records a surveillance area substantially from overhead.
The surveillance system may additionally comprise a clock for recording a time at which each no-scan event is identified. The time recorded by the clock can be used to identify a set of images recorded by the camera, just before the identified time, so as to locate and view a recording of the no-scan event In order to reduce the amount of processing done by the signal processor, without prejudicing the accuracy with which no-scan events are identified, the signal processor may comprise means for stopping tracking of a product where the product is tracked across the entry boundary on receipt of an associated notification of a scan.
The signal processor may comprise means for locating such a product in each image. In order the locate the product the signal processor may comprise one or more of the following: means for removing the background of the surveillance area from the image; means for removing noise and shadows from the image; and means for removing areas showing skin and clothing from the image.
The surveillance system may additionally comprise the scanner, which may be a bar code reader. The surveillance area may be an area of a checkout.
The surveillance system may additionally comprise an alarm and the signal processor may activate the alarm when a no-scan event is identified.
There is also provided a method of surveillance for monitoring retail outlet transactions in which a scanner scans at least one product during the transaction, comprising the steps of: recording images of a surveillance area where the scanning occurs, so as to generate a series of sequential images; receiving notification of scans made by such a scanner; processing the images by tracking the movement of such a product in the images; and identifying a no-scan event when such a product is tracked across an exit boundary within the images without an associated notification of a scan; wherein the exit boundary is located in each image at a position corresponding to a boundary within the surveillance area which is passed by such a product during or after it undergoes scanning. This method of surveillance may have the same features as are described above in relation to the surveillance system.
There is also provided a method of surveillance for monitoring retail outlet checkouts in which a scanner scans at least one product passing over the checkout, comprising the steps of: recording images of a surveillance area where the scanning occurs, so as to generate a series of sequential images; and processing the images by tracking the movement of such a product in the images. The tracking of the product movement over the checkout provides information about the typical path products pass over a checkout. Different checkout operators may use different paths and the paths used by different operators may be compared so as to identify training needs for checkout operators.
The invention will now be described by way of example only and with reference to the accompanying schematic drawings, wherein: Figures lA to 1! show a series of images of a portion of a checkout, illustrating the signal processing steps for locating a product passing through the checkout process; Figure 2 shows a plan view of the checkout, a portion of which is shown in the images of Figures IA to II; Figure 3 shows an image of a portion of the checkout of Figures 1 and 2, illustrating the located product and the entry and exit boundaries used in the surveillance system according to the present invention; Figure 4 shows method steps used in a surveillance system according to the present invention; and Figure 5 shows components of the surveillance system.
Figure 2 shows a point of sale area in a retail outlet, in particular a checkout in a supermarket, although, the present invention is equally applicable to other point of sale areas in other retail outlets. The checkout comprises an entry conveyor (2), onto which a customer loads products (4), a money till (6), a checkout keyboard (10) and a barcode reader (12). The checkout is operated by a supermarket checkout operator (8). In Figure 2 the operator (8) is passing a product (14) past a scanner, for example a barcode reader (12) for a barcode on the product (14) to be read.
The surveillance system according to the present invention comprises a signal processor (18) for identifying products (4, 14) in moving images generated by a Closed Circuit Television (CCIV) camera (16). The moving images generated by the CCTV camera (16) comprise a series of sequential images.
The CCIV camera (16) is directed towards the checkout, from above, so as to continuously record images of the checkout operator (8) passing products (14) past the barcode reader (12). In particular the images recorded by the CCTV camera (16) include the area of the images shown in Figures IA to 11, hereafter referred to as the surveillance area [Box a, Figure 4].
An image recorded using the CCTV camera (16) is passed to a signal processor (18), which processes the image to identify the product(14). Figure lB shows an image of the surveillance area of the checkout of Figure 2, as recorded by the CCTV camera (16). The surveillance area has a background, that is the checkout without an operator, products or other items arrayed thereon, shown in Figure IA. The signal processor (18) then subtracts the background image of Figure 1A from the recorded image of Figure lB to generate a foreground image illustrated schematically in Figure 1C. The signal processor (18) then removes any noise from the foreground image of Figure 1C so as to generate a clean foreground image illustrated schematically in Figure 1D. The signal processor (18) then removes any shadows from the clean foreground image of Figure 1D, so as to generate a shadowless clean foreground image, shown schematically in Figure 1 E. The signal processor (18) then identifies any areas of skin in the shadowless clean foreground image of Figure 1 E. The areas of the image showing skin are shown schematically in Figure 1 F. The signal processor (18) then subtracts identified areas of skin from the image of Figure 1 E so as to generate an image isolating the product (14), as is shown in Figure 1G [Box b, Figure 4].
The signal processor (18) can also identify any areas of an operator's clothing in the image of Figure 1G. The signal processor (18) can then subtract identified areas of clothing, for example the sleeves, from the image of Figure 1G. From the processed image of Figure IH, the signal processor (18) then designates the area or a group of pixels of the image in which the product (14) is located, as is shown schematically in Figure 1H [Box c, Figure 4]. Each of the signal processing steps described in relation to Figures 1A to II, are known to a person skilled the art of image processing.
Once the group of pixels within the image forming the image of the product (14) is identified by the signal processor, this group of pixels is tracked across the image by the signal processor (18) as the product (14) is moved across the surveillance area. Again, such methods of tracking groups of pixels are known to a person skilled in the art of image processing.
The tracking of the product movement over the checkout provides information about the typical path products pass over a checkout, which may be of value to the owners or managers of the retail outlet. The paths tracked for products may be recorded and collated in order to derive this information. Different checkout operators may use different paths and the paths used by different operators may be compared so as to identify training needs for checkout operators.
The signal processor (18) is configured to identify the passage of the identified group of pixels across boundaries defined in the image. Figure 3 shows an image similar to that of Figure 1 H, showing the identified group of pixels (14') corresponding to the package (14) of Figure 1H. Figure 3 also shows the boundaries as two lines defined in the image. The positions of the lines (22, 24) in the image of Figure 3 correspond to the positioning of the lines in Figure 2. In Figure 2 the dashed line between the conveyor (2) and the checkout operator (8) corresponds to the positioning of an entry boundary (22) in the image of Figure 3. Also, in Figure 2, the dot-dashed line to the right of the bar code reader (12) corresponds to the positioning of an exit boundary (24) in the image of Figure 3.
An accurate way to track a package (14) across a checkout is to track a group of pixels (14') in an image of the checkout sequentially moving across the entry boundary (22) and then the exit boundary (24) [Boxes d and f of Figure 4].
If a product (14) passing over the checkout is scanned by the barcode reader [Box k, Figure 4] (12) the product firstly moves over the entry boundary (22), then it is scanned by the barcode reader and finally it moves over the exit boundary (24). Therefore, the signal processor (18) is configured to identify a no-scan event when: 1. the group of pixels (14') is tracked moving across the entry boundary (22) (shown in dashed lines) [Box d, Figure 4]; and subsequently 2. the group of pixels (14') is tracked moving across the exit boundary (24) (shows in dot-dashed lines) [Box f, Figure 4]; and 3. no bar code scan has occurred in the time period between which the pixels are tracked over the entry boundary and the pixels are tracked over the exit boundary [Box e, Figure 4].
A no-scan event occurs where the product (14) moves across the checkout, Ic. between the entry boundary (22) and the exit boundary (24), but no barcode is scanned [Box g, Figure 4].
The signal processor may use the following steps when identifying a no-scan event: 1. Identify the group of pixels (14') representing the product (14) [Box c, Figure 4]; 2. Track the group of pixels (14') across the entry boundary (22) [Box d, Figure 4); 3. If a bar code scanning event is detected, the surveillance system moves on to the next product which passes the surveillance area [Box c via Boxes h and m, Figure 4]; and 4. If no bar code scanning event is detected before the group of pixels are tracked across the exit boundary (24) the signal processor identifies a no-scan event [Box g, Figure 4].
At step 3 above, if the group of pixels (14') is identified as passing the entry boundary (22) and then a scanning event is detected, the signal processor (18) stops tracking the movement of the group of pixels (14') [Box h, Figure 4].
This saves processing effort as once the scanning event is detected, there is no likelihood of a no-scan event being detected for that group of pixels.
The signal processor (18) includes a clock and so also records the time and date at which each detected no-scan event occurs [Box i, Figure 4].
The signal processor (18) may output data highlighting no-scan events, for example via a user interface (22). The no-scan events may be highlighted as they occur in real time, for example by sounding an alarm [Box j, Figure 4].
Alternatively, no-scan events can be viewed on request at a later time, for example via the user interface (22). Moving images recorded by the CCTV camera (16) just prior to the detection of a no-scan event may be displayed on a display screen of the user interface (22) each time a no-scan event occurs or as requested. Therefore, a no-scan event can be investigated, for example, by checking the CCIV images recorded by the camera (16) just before the time recorded for the alarm.
Claims (24)
1. A surveillance system for monitoring retail outlet transactions in which a scanner scans at least one product during the transaction, comprising: a camera for recording images of a surveillance area where the scanning occurs, so as to generate a series of sequential images; a signal processor: for receiving notification of scans made by such a scanner; for processing the images by tracking the movement of such a product in the images; and for identifying a no-scan event when such a product is tracked across an exit boundary within the images without an associated notification of a scan; wherein the exit boundary is located in each image at a position corresponding to a boundary within the surveillance area which is passed by such a product during or after it undergoes scanning.
2. A surveillance system according to claim 1 wherein the exit boundary is located in each image at a position corresponding to a boundary within the surveillance area which is passed by such a product after it undergoes scanning.
3. A surveillance system according to claim 1 or claim 2 wherein the signal processor comprises means for identifying a no-scan event when such a product is tracked sequentially across an entry boundary and then across the exit boundary without an associated notification of a scan and wherein the entry boundary is located in each image at a position corresponding to a boundary within the surveillance area which is passed by such a product before it undergoes scanning.
4. A surveillance system according to claim 2 wherein the signal processor comprises means for identifying a no-scan event when such a product is tracked sequentially across an entry boundary and then across the exit boundary without an associated notification of a scan in the time period during which the product is between the boundaries.
5. A surveillance system according to any one of the preceding claims wherein the camera is a video camera.
6. A surveillance system according to any one of the preceding claims wherein the signal processor comprises means for locating such a product in the images.
7. A surveillance system according to any one of the preceding claims wherein the signal processor additionally comprises a clock for recording a time at which each no-scan event is identified.
8. A surveillance system according to claim 3 or any one of claims 4 to 7 when dependent on claim 3 wherein the signal processor comprises means for stopping tracking of a product where the product is tracked across the entry boundary on receipt of an associated notification of a scan.
9. A surveillance system according to any one of the preceding claims wherein the signal processor comprises means for locating such a product in each image, comprising: means for removing the background of the surveillance area from the image; means for removing noise and shadows from the image; and means for removing areas showing skin and/or clothing from the image.
10. A surveillance system according to any one of the preceding claims additionally comprising the scanner.
11. A surveillance system according to claim 10 wherein the scanner is a bar code reader.
12. A surveillance system according to any one of the preceding claims wherein the surveillance area is an area of a checkout.
13. A surveillance system according to any one of the preceding claims additionally comprising an alarm, wherein the signal processor comprises means for activating the alarm when a no-scan event is identified.
S
14. A method of surveillance for monitoring retail outlet transactions in which a scanner scans at least one product during the transaction, comprising the steps of: recording images of a surveillance area where the scanning occurs, so as to generate a series of sequential images; receiving notification of scans made by such a scanner; processing the images by tracking the movement of such a product in the images; and identifying a no-scan event when such a product is tracked across an exit boundary within the images without an associated notification of a scan; wherein the exit boundary is located in each image at a position corresponding to a boundary within the surveillance area which is passed by such a product during or after it undergoes scanning.
15. A surveillance method according to claim 14 wherein the exit boundary is located in each image at a position corresponding to a boundary within the surveillance area which is passed by such a product after it undergoes scanning.
16. A surveillance method according to claim 14 or claim 15 comprising the step of identifying a no-scan event when such a product is tracked sequentially across an entry boundary and then across the exit boundary without an associated notification of a scan and wherein the entry boundary is located in each image at a position corresponding to a boundary within the
S
surveillance area which is passed by such a product before it undergoes scanning.
17. A surveillance method according to claim 16 comprising the step of identifying a no-scan event when such a product is tracked sequentially across an entry boundary and then across the exit boundary without an associated notification of a scan in the time period during which the product is between the boundaries.
18. A surveillance method according to any one of claims 14 to 17 comprising the step of locating such a product in the images.
19. A surveillance method according to any one of claims 14 to 18 comprising the step of recording a time at which each no-scan event is identified.
20. A surveillance method according to claim 16 or any one of claims 17 to 19 when dependent on claim 16 comprising the step of stopping tracking of a product where the product is tracked across the entry boundary on receipt of an associated notification of a scan.
21. A surveillance method according to any one of claims 14 to 20 comprising the step of locating such a product in each image, by the steps of: removing the background of the surveillance area from the image; removing noise and shadows from the image; and removing areas showing skin and/or clothing from the image.
22. A surveillance method according to any one of claims 14 to 20 additionally comprising the steps of: scanning at least one product during the transaction.
23. A surveillance method according to any one of claims 14 to 22 wherein the surveillance area is an area of a checkout.
24. A surveillance method according to any one of claims 14 to 22 additionally comprising the step of activating the alarm when a no- scan event is identified.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB0713750A GB2451073A (en) | 2007-07-16 | 2007-07-16 | Checkout surveillance system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| GB0713750A GB2451073A (en) | 2007-07-16 | 2007-07-16 | Checkout surveillance system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| GB0713750D0 GB0713750D0 (en) | 2007-08-22 |
| GB2451073A true GB2451073A (en) | 2009-01-21 |
Family
ID=38461626
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| GB0713750A Withdrawn GB2451073A (en) | 2007-07-16 | 2007-07-16 | Checkout surveillance system |
Country Status (1)
| Country | Link |
|---|---|
| GB (1) | GB2451073A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2589172A (en) * | 2019-09-09 | 2021-05-26 | Zebra Tech Corp | Point-of-sale scanner signaling to a camera |
| EP3839905A1 (en) * | 2019-12-17 | 2021-06-23 | Toshiba TEC Kabushiki Kaisha | Sales management device and control program for sales management device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5609223A (en) * | 1994-05-30 | 1997-03-11 | Kabushiki Kaisha Tec | Checkout system with automatic registration of articles by bar code or physical feature recognition |
| US5679941A (en) * | 1994-05-30 | 1997-10-21 | Kabushiki Kaisha Tec | Checkout device |
| US5747784A (en) * | 1996-10-22 | 1998-05-05 | Ncr Corporation | Method and apparatus for enhancing security in a self-service checkout station |
| US5918211A (en) * | 1996-05-30 | 1999-06-29 | Retail Multimedia Corporation | Method and apparatus for promoting products and influencing consumer purchasing decisions at the point-of-purchase |
| US20030018522A1 (en) * | 2001-07-20 | 2003-01-23 | Psc Scanning, Inc. | Biometric system and method for identifying a customer upon entering a retail establishment |
-
2007
- 2007-07-16 GB GB0713750A patent/GB2451073A/en not_active Withdrawn
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5609223A (en) * | 1994-05-30 | 1997-03-11 | Kabushiki Kaisha Tec | Checkout system with automatic registration of articles by bar code or physical feature recognition |
| US5679941A (en) * | 1994-05-30 | 1997-10-21 | Kabushiki Kaisha Tec | Checkout device |
| US5918211A (en) * | 1996-05-30 | 1999-06-29 | Retail Multimedia Corporation | Method and apparatus for promoting products and influencing consumer purchasing decisions at the point-of-purchase |
| US5747784A (en) * | 1996-10-22 | 1998-05-05 | Ncr Corporation | Method and apparatus for enhancing security in a self-service checkout station |
| US20030018522A1 (en) * | 2001-07-20 | 2003-01-23 | Psc Scanning, Inc. | Biometric system and method for identifying a customer upon entering a retail establishment |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2589172A (en) * | 2019-09-09 | 2021-05-26 | Zebra Tech Corp | Point-of-sale scanner signaling to a camera |
| GB2589172B (en) * | 2019-09-09 | 2022-03-09 | Zebra Tech Corp | Point-of-sale scanner signaling to a camera |
| US11868842B2 (en) | 2019-09-09 | 2024-01-09 | Zebra Technologies Corporation | Point-of-sale scanner signaling to a camera |
| EP3839905A1 (en) * | 2019-12-17 | 2021-06-23 | Toshiba TEC Kabushiki Kaisha | Sales management device and control program for sales management device |
Also Published As
| Publication number | Publication date |
|---|---|
| GB0713750D0 (en) | 2007-08-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11100333B2 (en) | Method and apparatus for detecting suspicious activity using video analysis | |
| US8448858B1 (en) | Method and apparatus for detecting suspicious activity using video analysis from alternative camera viewpoint | |
| JP5054670B2 (en) | Method and apparatus for detecting suspicious behavior using video analysis | |
| US7516888B1 (en) | Method and apparatus for auditing transaction activity in retail and other environments using visual recognition | |
| US8570375B1 (en) | Method and apparatus for random-access review of point of sale transactional video | |
| US20090039165A1 (en) | Methods and Apparatus for a Bar Code Scanner Providing Video Surveillance | |
| US20210390308A1 (en) | Systems and methods for verifying machine-readable label associated with merchandise | |
| JP2018032404A (en) | Fraud investigation in self-checkout terminal | |
| US20050102183A1 (en) | Monitoring system and method based on information prior to the point of sale | |
| CN113366543A (en) | System and method for detecting scanning anomaly of self-checkout terminal | |
| JP2016081364A (en) | Information notification program and information processing device | |
| GB2451073A (en) | Checkout surveillance system | |
| CN110276619B (en) | Information processing method and device and information processing system | |
| CN115546900B (en) | Risk identification method, device, equipment and storage medium | |
| CN115546703B (en) | Risk identification method, device and equipment for self-service cash register and storage medium | |
| EP4266279A1 (en) | Antitheft system for items in automatic checkouts and the like | |
| AU720390B2 (en) | Analysis rule expedited pos system evaluation system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| WAP | Application withdrawn, taken to be withdrawn or refused ** after publication under section 16(1) |