CA2283382C - An apparatus and method for capturing and projecting an image of a cart - Google Patents
An apparatus and method for capturing and projecting an image of a cart Download PDFInfo
- Publication number
- CA2283382C CA2283382C CA 2283382 CA2283382A CA2283382C CA 2283382 C CA2283382 C CA 2283382C CA 2283382 CA2283382 CA 2283382 CA 2283382 A CA2283382 A CA 2283382A CA 2283382 C CA2283382 C CA 2283382C
- Authority
- CA
- Canada
- Prior art keywords
- image
- display
- captured
- images
- items
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19665—Details related to the storage of video surveillance data
- G08B13/19671—Addition of non-video data, i.e. metadata, to video stream
- G08B13/19673—Addition of time stamp, i.e. time metadata, to video stream
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19697—Arrangements wherein non-video detectors generate an alarm themselves
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/188—Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Signal Processing (AREA)
- Closed-Circuit Television Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
An inspection apparatus is located at a check-out desk for acquiring and displaying a still image of portion of a moving shopping cart, the image being immediately available for viewing by the cashier for purposes of inspection. Images are also sequentially stored along with corresponding date/time/desk information to create a record that may be retained for an arbitrary period of time.
The apparatus comprises a sensor for detecting the presence and longitudinal reference position of a shopping cart, an electronic camera, a short-term image storage buffer, a medium-term image store, an image display device, electronic logic to manage the timing of image acquisition and display, and a data communications interface to a computer system.
The apparatus comprises a sensor for detecting the presence and longitudinal reference position of a shopping cart, an electronic camera, a short-term image storage buffer, a medium-term image store, an image display device, electronic logic to manage the timing of image acquisition and display, and a data communications interface to a computer system.
Description
F:\I\ICL.536\Pat-App.013\Patent.Doc AN APPAR,~TUS AND METHOD FOR CAPTURING AND
PF~OJECT'ING AN IMAGE OF A CART
Field of the Invention This invention relates to electronic surveillance systems for loss-prevention and security, and in particular, to an inspection apparatus for capturing and processing an image of a portion of a cart at a checkout location of a retail store.
Background to the Inventiam Shopping carts, as used in supermarkets, for example, often include racks below the main storage basket for carrying additional items. The lower rack is a convenience for the customer, and increases the carrying capacity of the carts. However, due to the obscuring presence of the check-out desk, goods carried on the lower rack tend not to enter the attendant's lines of sight for long, if at all, and consequently may leave the store unnoticed and unpaid for.
A variety of devices exist ~to assist cashiers in detecting the presence of goods in the lower portion of the shopping cart. The most common are mirrors positioned so as to give the attendant an improved view of the lower cart region for at least a short period of time. However, this approach requires the attendant to look at the proper moment when the cart is in the viewing area, which is not always practical.
A number of devices that automatically detect the presence of objects on the lower rack of a cart as it passes a fixed location have been used. United States Patent Nos.
4327819 and 4723118 describe detection means responsive to the weight of objects placed on the lower rack. United States Patent Nos. 3725894.. 5485006, and 4338594 describe a detection approach based on the interruption of optical beams by obje-cts placed on the lower rack. Note that while the properties of object weight or optical ~~pacity rnay be exploited to detect their presence on the lower rack, neither technique can determine the type or number of objects on the lower rack. Consequently, the detection signal must initiate an alarm (audible or visible) which prompts the attendant to perform a visual inspection of the lower rack. These devices, therefore, are an aid to the cashier, as they do not perform inspection automatically.
PF~OJECT'ING AN IMAGE OF A CART
Field of the Invention This invention relates to electronic surveillance systems for loss-prevention and security, and in particular, to an inspection apparatus for capturing and processing an image of a portion of a cart at a checkout location of a retail store.
Background to the Inventiam Shopping carts, as used in supermarkets, for example, often include racks below the main storage basket for carrying additional items. The lower rack is a convenience for the customer, and increases the carrying capacity of the carts. However, due to the obscuring presence of the check-out desk, goods carried on the lower rack tend not to enter the attendant's lines of sight for long, if at all, and consequently may leave the store unnoticed and unpaid for.
A variety of devices exist ~to assist cashiers in detecting the presence of goods in the lower portion of the shopping cart. The most common are mirrors positioned so as to give the attendant an improved view of the lower cart region for at least a short period of time. However, this approach requires the attendant to look at the proper moment when the cart is in the viewing area, which is not always practical.
A number of devices that automatically detect the presence of objects on the lower rack of a cart as it passes a fixed location have been used. United States Patent Nos.
4327819 and 4723118 describe detection means responsive to the weight of objects placed on the lower rack. United States Patent Nos. 3725894.. 5485006, and 4338594 describe a detection approach based on the interruption of optical beams by obje-cts placed on the lower rack. Note that while the properties of object weight or optical ~~pacity rnay be exploited to detect their presence on the lower rack, neither technique can determine the type or number of objects on the lower rack. Consequently, the detection signal must initiate an alarm (audible or visible) which prompts the attendant to perform a visual inspection of the lower rack. These devices, therefore, are an aid to the cashier, as they do not perform inspection automatically.
United States Patent Nos. 5883968 describes a device that employs imaging and automated image analysis to detect the presence and type of goods in the lower rack. In particular, it describes the use of a digital image analysis technique whereby a reference image of an empty cart is compared to a database of images. It also describes the use of colour-discriminating and Identification Code discriminating techniques. Note that, in general, automated image-based techniques for object discrimination and identification are subject to a large number of variables (e.g. lighting colour, lighting direction, objection position, object orientation, shadowing, occlusion) that affect the reliability and accuracy of the identification process. It is often difficult in practice to control these variables. Such systems, therefore, can produce both false-positive and false-negative type errors during detection.
Accordingly, it is an object of one aspect of the present invention to solve many, if not all, of the drawbacks of the prior art systems by providing an improved inspection aid to the cashier through the use of images as opposed to alarms. Moreover, the present invention uses human visual processing as opposed to automated image processing, and is therefore less likely to produce identification errors.
Summary of the Invention In one aspect of the present invention , there is provided a system for monitoring and evaluating the performance of a checkout clerk at a retail store, the system comprising:
a. a detector to detect the presence at a predetermined location of a shopping cart containing items to be purchased;
b. a camera to capture at least one image of a portion of the shopping cart containing the items to be purchased;
c. means to electronically stamp the at least one captured image with a current date and time, and a unique location identifier;
d. storage and retrieval media for storing the at least one captured image, e. means to compare the items to be purchased in the captured image with data from corresponding sales receipts so as to monitor and evaluate the performance of the checkout clerk; and f. means to connect the system with other similar systems over a local area network.
Accordingly, it is an object of one aspect of the present invention to solve many, if not all, of the drawbacks of the prior art systems by providing an improved inspection aid to the cashier through the use of images as opposed to alarms. Moreover, the present invention uses human visual processing as opposed to automated image processing, and is therefore less likely to produce identification errors.
Summary of the Invention In one aspect of the present invention , there is provided a system for monitoring and evaluating the performance of a checkout clerk at a retail store, the system comprising:
a. a detector to detect the presence at a predetermined location of a shopping cart containing items to be purchased;
b. a camera to capture at least one image of a portion of the shopping cart containing the items to be purchased;
c. means to electronically stamp the at least one captured image with a current date and time, and a unique location identifier;
d. storage and retrieval media for storing the at least one captured image, e. means to compare the items to be purchased in the captured image with data from corresponding sales receipts so as to monitor and evaluate the performance of the checkout clerk; and f. means to connect the system with other similar systems over a local area network.
In a second aspect of the present invention, there is provided a method of monitoring and evaluating the performance of a checkout clerk at a retail store, the method comprising:
a. detecting the presence at a predetermined location of a shopping cart containing items to be purchased;
b. capturing at least one image of a portion of the shopping cart containing the items to be purchased;
c. projecting the at least one captured image from a display such that the image can be viewed by the checkout clerk to determine if there are items on the portion of the cart.
d. electronically stamping the at least one captured image with a current date and time, and a unique location identifier; and e. storing the at least one captured image in storage and retrieval media, f. retrieving the stored image and comparing the items to be purchased in the image with data from corresponding sales receipts to monitor and evaluate the performance of the checkout clerk.
More specifically, in a preferred embodiment, the present invention is directed to an apparatus for use in stores such as supermarkets and other retail outlets for aiding cashiers in the inspection of goods contained in the lower portion of shopping carts. In addition, the preferred embodiment provides a sequential record that may be inspected by store managers to audit the performance of cashier's effectiveness with respect to said inspection task.
Brief Description of the Drawings FIG. 1 is a pictorial perspective of one embodiment of the apparatus of the present invention showing the physical arrangement of the sensing, imaging and display elements when used as an aid to supermarket cashiers in examining goods in the lower tray of a shopping cart;
FIG. 2 is a functional system block diagram of one embodiment of the present invention; and FIG. 3 is a state-transition diagram of one embodiment of the present invention showing the state machine that controls the image display.
Description of the Preferred Embodiment One embodiment of the present invention is shown in FIG. l, which shows detection and imaging unit 1 arranged so as to provide an inspection aid to a cashier at a supermarket checkout desk. Included in the detection and imaging unit housing are windows la and lb which provide an optically transparent protection for the optical sensing means and imaging means that are described in more detail later. Also shown is a retro-reflective optical target 3b, which is typically a self adhesive patch of conformable, retro-reflective material that is bonded to a convenient surface of the shopping cart. The spacing of the detection and imaging elements is arranged by design such that an image of the lower portion of each passing shopping cart is captured at approximately the time that the cart passes through the field of view ld. The image is then displayed to the cashier at an appropriate time on the display unit 2.
In FIG. 2 a functional block diagraan of the preferred embodiment is shown.
The description begins following initial power-on and before the detection of an object. For the purposes of this description, the short-term l;~uffer 5 has two slots, corresponding to two images, one of which is for immediate display and t:he second retained for later display. Note that in general, the short-term buffer can be designed to accommodate as many images as are necessary for a specific application.
Imaging element 4 provides a continuous stream of digitized image data to the short-term buffer 5. Typically, this image data stream is produced by means of a standard interlaced format video camera 4a, providing an analog video signal through anti-abasing filter 4b to synchronization signal detection circuitry and analog-to-digital conversion circuitry 4e. The sampling frequency of the analog-to-digital converter must be sufficiently high to capture the chrominance and luminance portions of the siignal as well as the line and frame synchronization pulses contained in the signal. The digitized video signal data is written to the short-term buffer on a continuous basis at a rate matching the standard video frame rate of 60 frames per second (30 even frames plus 30 odd frames). Note however, that only data corresponding to either the even frames or the odd frames, but not both, is retained, for reasons that will now be explained.
A video frame is composed of a series of parallel horizontal lines. In the standard interlaced format, the so-called even and odd frames are acquired 1/60 of a second apart.
Consequently, movement of an object seen by the camera produces an apparent displacement in the image of that object between even and odd fr~unes. This apparent displacement goes unnoticed by human - S -vision when the video disp lay is continuously updated at the video frame rate of thirty fresh even/odd frames per second, as happens in a normal video display mode.
However, if a single even/odd frame pair is replayed repetitively with the aim of creating a so-called "freeze-frame" or still image mode, the aforementioned displacement artifact is highly S noticeable. It has been found that for this special case, eliminating either the even or the odd frame from the pair can produce a more satisfactory displayed image.
When a first object with an attached target 3b passes by sensor 3a, a binary output signal is produced. This signal is received by a logic-processing element 8, which determines that the video frame buffered at the current instant should be retained (that is, not erased by the arnval of subsequent video data) and sent to the display element 7 for viewing by an attendant. The display of the image is typically effected by reading the digitized frame from the buffer to a digital-to-analog converter '1a, then conditioning the signal through a reconstruction filter 7b, and finally connecting to the input of a standard video display monitor 7c.
For reasons previously explained, only the even frame or the odd frame, but not both, is retained. Therefore, to preserve the standardized video signal timing (essential for proper display on a standard monitor) the frame is read tvice every 1/30 second, thereby approximating the form and timing of a normal even/odd pair.
The displayed image remains visible; on the monitor until receipt of an event signal 8c, which causes the logic-processing element to terminate the displayed image. The event signal 8c is typically derived from the detection of an event that coincides with the end of the inspection process. If detection of ~~uch an event is impractical, or cannot be achieved with high consistency, a second display termination signal can be created on the basis of a pre-determined elapsed time using a timer 8a.
If a second object with an attached target 3b now passes by sensor 3a, an output signal is again produced. Again, this signal is recc;ived by the logic-processing element 8.
However, if this signal is received before the current displayed image is terminated by the action of either signal 8c or 8a, then the logic-processing element determines that the video frame buffered at the current instant should be retained, but not displayed at this point.
The receipt of a display-terminating signal, either 8c or 8a, will cause the logic-processing element to replace the currently displayed image by the frame retained in the buffer. If yet another display-terminating signal 8c or 8a is received prior to the detection of another object target 3b, the display clears to a "blank" state.
The data for every image that is displayed is also routed to a medium-term storage unit 6. This buffer has a multiplicity of memory slots 6a accommodating a larger number of images, as would be required to account for say., the activity over a period of a day or a week. Included is a date/time generator 6b for adding date/time information to each image and a general processing element 6c for memory mmlagement and data transfer tasks. The medium-term memory is arranged as a first-in, first-out buffer. Therefore, the finite number of slots always holds the most recent images, older images being automatically discarded.
A communications interface 9 provides the ability to transfer images from the medium-term storage unit to a computer ~t0 where. they can be saved as data files, viewed, and merged with other data. The communications interface 9 can be a direct point-to-point connection, such as an asynchronous serial link or ~:elephonc: modem, or a mufti-point local area network, which would support access to a multiplicity of irr~aging stations via distributed communications ports 9a, 9b, etc.
In the typical arrangement for the preferred embodiment, detection of the presence and position of a shopping cart is by means of an infra-red transmitting optical sensor 3a, responsive only to special retro-reflective target 3b material attached at an appropriate location on every shopping cart. The sensor 3a and th.e target 3b material may be of state-of the-art design, embodying features such as polarizing filter, frequency modulation and background suppression to ensure high detection accuracy.
A typical form of imaging means 4 is the combination of electronic video camera 4a and analog-to-digital converter 7d. Cameras with standardized analog video output signal formats such as PAL or NTSC are readily available. Typically, said camera 4a is position in the check-out aisle so as to achieve a wide-angle, side-on view of the lower region of the shopping cart, though alternate views are possible, An analog-to-digital converter 7a operating at minimum 12 MHz sampling rate is used to digitize the NTSC video frame without sacrificing the quality of the image. An anti-aliasing filter 4b is used to ensure that the analog signal contains only frequency components commensurate with the sampling frequency.
While being digitized, the video signal is written to short-term memory 6a typically static random access memory (SRAM). The timing of the conversion and writing process is arranged such that a single complete video frame is captured. Said timing is facilitated by standardized synchronization signals that are contained within the video signal. The amount of short-term memory capacity required by the design relates to the practical imperative of processing a queue.
For example, a cashier will frequently encounter situations whereby a second shopping cart passes the camera location before the first cart has completed inspection.
This situation requires that both the displayed image and the most recent image are buffered, otherwise, one or the other would be lost.
The digitized video frame is rapidly retrieved for display by first scanning the corresponding set of SRAM memory cell locations at the appropriate rate (i.e. 12 MHz) and streaming the data to a digital-to-analog converter 7a. The converter 7a output is then passed through a reconstruction filter 7b and directed to a compatible monitor 7c (e.g. NTSC). The digitized video frame is re-scanned repeatedly at the standard video rate for the duration that the image remains displayed.
The preferred form of video monitor 7c uses a compact, flat-screen technology such as liquid crystal display (LCD). The display is mounted at a convenient height where the cashier can easily see it. A swivel mount may be employed so that the monitor may be easily adjusted to suit cashiers of different heights.
To display the correct images at the correct times requires logic processing and event-signalling, typically implemented in the form of a state-machine. The term "state machine"
refers to a defined set of logical variables (i.e. variables that can have the value 1 or 0). A state machine is the name conventionally used for a process that produces a specific logical output state depending on the combination of the current internal state and external events. State machines can be implemented in either hardware logic circuits or in software programs executed by microprocessor.
In this preferred embodiment, the state machine manages the various possible display states in response to event signals. The set of display states could include, for example, "display blank", "display buffered image 1" and "display buffered image 2". The set of event signals could include "cart detected", "transaction completed" and "timeout exceeded". Each event signal is derived from a measurable physical event: for example, the detection of an optical target when the cart passes a predefined position; the detection of a voltage change on the cash drawer unlatch solenoid when the cashier totals the sale; and the passing of a pre-defined period if time since a given image appeared on the display. The following examples provide an explanation of the action of the state machine.
Receipt of the "cart detected" event causes the video frame captured during that sampling instant to be displayed if the display state at the time of the vent is "display blank". Otherwise the video frame is buffered, pending clearing of the currently displayed image by either a "transaction completed" event or a "timeout exceeded" event.
Receipt of the "transaction completed" or "timeout exceeded" event causes the display to clear to the "display blank" state if there is no frame waiting in the buffer at the time of the vent.
Otherwise, the display changes to show the buffered image.
Note that the event signal "transaction completed" is the most usual cause of a change in the output display state. However, the "timeout exceeded" event is provided as an alternate signal with a similar effect. The "timeout" function ensures that an image never remains displayed longer than a pre-determined length of time, for example four minutes. An example of when this is used is when a single can moves forward, then briefly backward, across the sensor thereby creating two "cart detected" event signals, and (if the buffer is clear) acquiring two images of the same cart. Since the cashier will only create a single "transaction completed"
event (corresponding to the single cart), the display will be cleared of the first image but not the second. When this occurs, the display state is out-of step with the sequence of passing carts, and remains so indefinitely. To remedy to this condition, an artificial "transaction completed" event is created after a short time-limit. With this provision, the system self corrects as soon as there is a gap between incoming carts that exceeds the time-out limit.
The images that are acquired) and displayed in the manner described in the preceding paragraphs may also be stored along with a corresponding date and time code to create a retrievable record.
This storage function is fa<;ilitated by a microprocessor 6c and time-code generator 6b and medium-term memory 6a. Medium germ memory 6a is preferably of a non-volatile type such as "flash" (electrically prograrrunable) RAM or battery-backed RAM. The memory management may be arranged in the manner of a circular buffer, whereby the oldest data is overwritten by new data once the memory capacity is exceeded. Typically, the memory capacity is designed to accommodate enough images to cover at least one workshift.
Retrieval of images stored in medium-term memory 6a is facilitated by the use of a communications interface f to a computer system 10. Once transferred from the long-term memory to the computer system 10 the images may be viewed, correlated with other data such as sales receipts, and archived to a mass storage system. The communications interface 9 may be any of a number of standaJ-dized types, from simple point-to-point serial-data or parallel-data ports to a mufti-point local area network.
1 S The present invention has been described with reference to its preferred embodiment, however, it will be understood that th.e scope of the present invention is not limited to the preferred embodiment, but its scope is defined by the claims. Further, it will be understaood modifications may be made within the scope of the present invention without departing from the spirit thereof, and the present invention includes all such modifications.
a. detecting the presence at a predetermined location of a shopping cart containing items to be purchased;
b. capturing at least one image of a portion of the shopping cart containing the items to be purchased;
c. projecting the at least one captured image from a display such that the image can be viewed by the checkout clerk to determine if there are items on the portion of the cart.
d. electronically stamping the at least one captured image with a current date and time, and a unique location identifier; and e. storing the at least one captured image in storage and retrieval media, f. retrieving the stored image and comparing the items to be purchased in the image with data from corresponding sales receipts to monitor and evaluate the performance of the checkout clerk.
More specifically, in a preferred embodiment, the present invention is directed to an apparatus for use in stores such as supermarkets and other retail outlets for aiding cashiers in the inspection of goods contained in the lower portion of shopping carts. In addition, the preferred embodiment provides a sequential record that may be inspected by store managers to audit the performance of cashier's effectiveness with respect to said inspection task.
Brief Description of the Drawings FIG. 1 is a pictorial perspective of one embodiment of the apparatus of the present invention showing the physical arrangement of the sensing, imaging and display elements when used as an aid to supermarket cashiers in examining goods in the lower tray of a shopping cart;
FIG. 2 is a functional system block diagram of one embodiment of the present invention; and FIG. 3 is a state-transition diagram of one embodiment of the present invention showing the state machine that controls the image display.
Description of the Preferred Embodiment One embodiment of the present invention is shown in FIG. l, which shows detection and imaging unit 1 arranged so as to provide an inspection aid to a cashier at a supermarket checkout desk. Included in the detection and imaging unit housing are windows la and lb which provide an optically transparent protection for the optical sensing means and imaging means that are described in more detail later. Also shown is a retro-reflective optical target 3b, which is typically a self adhesive patch of conformable, retro-reflective material that is bonded to a convenient surface of the shopping cart. The spacing of the detection and imaging elements is arranged by design such that an image of the lower portion of each passing shopping cart is captured at approximately the time that the cart passes through the field of view ld. The image is then displayed to the cashier at an appropriate time on the display unit 2.
In FIG. 2 a functional block diagraan of the preferred embodiment is shown.
The description begins following initial power-on and before the detection of an object. For the purposes of this description, the short-term l;~uffer 5 has two slots, corresponding to two images, one of which is for immediate display and t:he second retained for later display. Note that in general, the short-term buffer can be designed to accommodate as many images as are necessary for a specific application.
Imaging element 4 provides a continuous stream of digitized image data to the short-term buffer 5. Typically, this image data stream is produced by means of a standard interlaced format video camera 4a, providing an analog video signal through anti-abasing filter 4b to synchronization signal detection circuitry and analog-to-digital conversion circuitry 4e. The sampling frequency of the analog-to-digital converter must be sufficiently high to capture the chrominance and luminance portions of the siignal as well as the line and frame synchronization pulses contained in the signal. The digitized video signal data is written to the short-term buffer on a continuous basis at a rate matching the standard video frame rate of 60 frames per second (30 even frames plus 30 odd frames). Note however, that only data corresponding to either the even frames or the odd frames, but not both, is retained, for reasons that will now be explained.
A video frame is composed of a series of parallel horizontal lines. In the standard interlaced format, the so-called even and odd frames are acquired 1/60 of a second apart.
Consequently, movement of an object seen by the camera produces an apparent displacement in the image of that object between even and odd fr~unes. This apparent displacement goes unnoticed by human - S -vision when the video disp lay is continuously updated at the video frame rate of thirty fresh even/odd frames per second, as happens in a normal video display mode.
However, if a single even/odd frame pair is replayed repetitively with the aim of creating a so-called "freeze-frame" or still image mode, the aforementioned displacement artifact is highly S noticeable. It has been found that for this special case, eliminating either the even or the odd frame from the pair can produce a more satisfactory displayed image.
When a first object with an attached target 3b passes by sensor 3a, a binary output signal is produced. This signal is received by a logic-processing element 8, which determines that the video frame buffered at the current instant should be retained (that is, not erased by the arnval of subsequent video data) and sent to the display element 7 for viewing by an attendant. The display of the image is typically effected by reading the digitized frame from the buffer to a digital-to-analog converter '1a, then conditioning the signal through a reconstruction filter 7b, and finally connecting to the input of a standard video display monitor 7c.
For reasons previously explained, only the even frame or the odd frame, but not both, is retained. Therefore, to preserve the standardized video signal timing (essential for proper display on a standard monitor) the frame is read tvice every 1/30 second, thereby approximating the form and timing of a normal even/odd pair.
The displayed image remains visible; on the monitor until receipt of an event signal 8c, which causes the logic-processing element to terminate the displayed image. The event signal 8c is typically derived from the detection of an event that coincides with the end of the inspection process. If detection of ~~uch an event is impractical, or cannot be achieved with high consistency, a second display termination signal can be created on the basis of a pre-determined elapsed time using a timer 8a.
If a second object with an attached target 3b now passes by sensor 3a, an output signal is again produced. Again, this signal is recc;ived by the logic-processing element 8.
However, if this signal is received before the current displayed image is terminated by the action of either signal 8c or 8a, then the logic-processing element determines that the video frame buffered at the current instant should be retained, but not displayed at this point.
The receipt of a display-terminating signal, either 8c or 8a, will cause the logic-processing element to replace the currently displayed image by the frame retained in the buffer. If yet another display-terminating signal 8c or 8a is received prior to the detection of another object target 3b, the display clears to a "blank" state.
The data for every image that is displayed is also routed to a medium-term storage unit 6. This buffer has a multiplicity of memory slots 6a accommodating a larger number of images, as would be required to account for say., the activity over a period of a day or a week. Included is a date/time generator 6b for adding date/time information to each image and a general processing element 6c for memory mmlagement and data transfer tasks. The medium-term memory is arranged as a first-in, first-out buffer. Therefore, the finite number of slots always holds the most recent images, older images being automatically discarded.
A communications interface 9 provides the ability to transfer images from the medium-term storage unit to a computer ~t0 where. they can be saved as data files, viewed, and merged with other data. The communications interface 9 can be a direct point-to-point connection, such as an asynchronous serial link or ~:elephonc: modem, or a mufti-point local area network, which would support access to a multiplicity of irr~aging stations via distributed communications ports 9a, 9b, etc.
In the typical arrangement for the preferred embodiment, detection of the presence and position of a shopping cart is by means of an infra-red transmitting optical sensor 3a, responsive only to special retro-reflective target 3b material attached at an appropriate location on every shopping cart. The sensor 3a and th.e target 3b material may be of state-of the-art design, embodying features such as polarizing filter, frequency modulation and background suppression to ensure high detection accuracy.
A typical form of imaging means 4 is the combination of electronic video camera 4a and analog-to-digital converter 7d. Cameras with standardized analog video output signal formats such as PAL or NTSC are readily available. Typically, said camera 4a is position in the check-out aisle so as to achieve a wide-angle, side-on view of the lower region of the shopping cart, though alternate views are possible, An analog-to-digital converter 7a operating at minimum 12 MHz sampling rate is used to digitize the NTSC video frame without sacrificing the quality of the image. An anti-aliasing filter 4b is used to ensure that the analog signal contains only frequency components commensurate with the sampling frequency.
While being digitized, the video signal is written to short-term memory 6a typically static random access memory (SRAM). The timing of the conversion and writing process is arranged such that a single complete video frame is captured. Said timing is facilitated by standardized synchronization signals that are contained within the video signal. The amount of short-term memory capacity required by the design relates to the practical imperative of processing a queue.
For example, a cashier will frequently encounter situations whereby a second shopping cart passes the camera location before the first cart has completed inspection.
This situation requires that both the displayed image and the most recent image are buffered, otherwise, one or the other would be lost.
The digitized video frame is rapidly retrieved for display by first scanning the corresponding set of SRAM memory cell locations at the appropriate rate (i.e. 12 MHz) and streaming the data to a digital-to-analog converter 7a. The converter 7a output is then passed through a reconstruction filter 7b and directed to a compatible monitor 7c (e.g. NTSC). The digitized video frame is re-scanned repeatedly at the standard video rate for the duration that the image remains displayed.
The preferred form of video monitor 7c uses a compact, flat-screen technology such as liquid crystal display (LCD). The display is mounted at a convenient height where the cashier can easily see it. A swivel mount may be employed so that the monitor may be easily adjusted to suit cashiers of different heights.
To display the correct images at the correct times requires logic processing and event-signalling, typically implemented in the form of a state-machine. The term "state machine"
refers to a defined set of logical variables (i.e. variables that can have the value 1 or 0). A state machine is the name conventionally used for a process that produces a specific logical output state depending on the combination of the current internal state and external events. State machines can be implemented in either hardware logic circuits or in software programs executed by microprocessor.
In this preferred embodiment, the state machine manages the various possible display states in response to event signals. The set of display states could include, for example, "display blank", "display buffered image 1" and "display buffered image 2". The set of event signals could include "cart detected", "transaction completed" and "timeout exceeded". Each event signal is derived from a measurable physical event: for example, the detection of an optical target when the cart passes a predefined position; the detection of a voltage change on the cash drawer unlatch solenoid when the cashier totals the sale; and the passing of a pre-defined period if time since a given image appeared on the display. The following examples provide an explanation of the action of the state machine.
Receipt of the "cart detected" event causes the video frame captured during that sampling instant to be displayed if the display state at the time of the vent is "display blank". Otherwise the video frame is buffered, pending clearing of the currently displayed image by either a "transaction completed" event or a "timeout exceeded" event.
Receipt of the "transaction completed" or "timeout exceeded" event causes the display to clear to the "display blank" state if there is no frame waiting in the buffer at the time of the vent.
Otherwise, the display changes to show the buffered image.
Note that the event signal "transaction completed" is the most usual cause of a change in the output display state. However, the "timeout exceeded" event is provided as an alternate signal with a similar effect. The "timeout" function ensures that an image never remains displayed longer than a pre-determined length of time, for example four minutes. An example of when this is used is when a single can moves forward, then briefly backward, across the sensor thereby creating two "cart detected" event signals, and (if the buffer is clear) acquiring two images of the same cart. Since the cashier will only create a single "transaction completed"
event (corresponding to the single cart), the display will be cleared of the first image but not the second. When this occurs, the display state is out-of step with the sequence of passing carts, and remains so indefinitely. To remedy to this condition, an artificial "transaction completed" event is created after a short time-limit. With this provision, the system self corrects as soon as there is a gap between incoming carts that exceeds the time-out limit.
The images that are acquired) and displayed in the manner described in the preceding paragraphs may also be stored along with a corresponding date and time code to create a retrievable record.
This storage function is fa<;ilitated by a microprocessor 6c and time-code generator 6b and medium-term memory 6a. Medium germ memory 6a is preferably of a non-volatile type such as "flash" (electrically prograrrunable) RAM or battery-backed RAM. The memory management may be arranged in the manner of a circular buffer, whereby the oldest data is overwritten by new data once the memory capacity is exceeded. Typically, the memory capacity is designed to accommodate enough images to cover at least one workshift.
Retrieval of images stored in medium-term memory 6a is facilitated by the use of a communications interface f to a computer system 10. Once transferred from the long-term memory to the computer system 10 the images may be viewed, correlated with other data such as sales receipts, and archived to a mass storage system. The communications interface 9 may be any of a number of standaJ-dized types, from simple point-to-point serial-data or parallel-data ports to a mufti-point local area network.
1 S The present invention has been described with reference to its preferred embodiment, however, it will be understood that th.e scope of the present invention is not limited to the preferred embodiment, but its scope is defined by the claims. Further, it will be understaood modifications may be made within the scope of the present invention without departing from the spirit thereof, and the present invention includes all such modifications.
Claims (11)
1. A system for monitoring and evaluating the performance of a checkout clerk at a retail store, the system comprising:
a. a detector to detect the presence at a predetermined location of a shopping cart containing items to be purchased;
b. a camera to capture at least one image of a portion of the shopping cart containing the items to be purchased;
c. means to electronically stamp the at least one captured image with a current date and time, and a unique location identifier;
d. storage and retrieval media for storing the at least one captured image, e. means to compare the items to be purchased in the captured image with data from corresponding sales receipts so as to monitor and evaluate the performance of the checkout clerk; and f. means to connect the system with other similar systems over a local area network.
a. a detector to detect the presence at a predetermined location of a shopping cart containing items to be purchased;
b. a camera to capture at least one image of a portion of the shopping cart containing the items to be purchased;
c. means to electronically stamp the at least one captured image with a current date and time, and a unique location identifier;
d. storage and retrieval media for storing the at least one captured image, e. means to compare the items to be purchased in the captured image with data from corresponding sales receipts so as to monitor and evaluate the performance of the checkout clerk; and f. means to connect the system with other similar systems over a local area network.
2. The system of claim 1, wherein the at least one image is captured by the camera in electronic form.
3. The system of claim 2, further comprising a display for projecting the at least one captured image, the display located such that the at least one projected image can be viewed by the checkout clerk at the checkout location.
4. The system of claim 3, wherein the camera captures a plurality of images and the storage and retrieval media provides a short-term buffer for the captured images prior to transmitting one of the captured images to the display means.
5. The system of claim 4, further comprising electronic logic processing means to process and transmit the captured images and to manage timing of the transmission of the captured images to the display means.
6. The system of claim 5, further comprising a data communication interface means to enable a user to retrieve one of the stored images and to transmit the retrieved image to a computer-based display monitor.
7. The system of claim 1, wherein the detector is an infra-red transmitting optical sensor responsive to a retro-reflective material located on the cart.
8. The system of claim 1, wherein the camera comprises an electronic video camera and an analog-to-digital converter.
9. A method of monitoring and evaluating the performance of a checkout clerk at a retail store, the method comprising:
a. detecting the presence at a predetermined location of a shopping cart containing items to be purchased;
b. capturing at least one image of a portion of the shopping cart containing the items to be purchased;
c. projecting the at least one captured image from a display such that the image can be viewed by the checkout clerk to determine if there are items on the portion of the cart.
d. electronically stamping the at least one captured image with a current date and time, and a unique location identifier; and e. storing the at least one captured image in storage and retrieval media, f. retrieving the stored image and comparing the items to be purchased in the image with data from corresponding sales receipts to monitor and evaluate the performance of the checkout clerk.
a. detecting the presence at a predetermined location of a shopping cart containing items to be purchased;
b. capturing at least one image of a portion of the shopping cart containing the items to be purchased;
c. projecting the at least one captured image from a display such that the image can be viewed by the checkout clerk to determine if there are items on the portion of the cart.
d. electronically stamping the at least one captured image with a current date and time, and a unique location identifier; and e. storing the at least one captured image in storage and retrieval media, f. retrieving the stored image and comparing the items to be purchased in the image with data from corresponding sales receipts to monitor and evaluate the performance of the checkout clerk.
10. The method of claim 9, further comprising the step of converting the at least one captured image to digital electronic form.
11. The method of claim 10, wherein a plurality of images are captured and the captured images are stored for a short-term prior to projecting one of the stored images from the display.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA 2283382 CA2283382C (en) | 1999-10-04 | 1999-10-04 | An apparatus and method for capturing and projecting an image of a cart |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA 2283382 CA2283382C (en) | 1999-10-04 | 1999-10-04 | An apparatus and method for capturing and projecting an image of a cart |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2283382A1 CA2283382A1 (en) | 1999-12-12 |
| CA2283382C true CA2283382C (en) | 2001-06-12 |
Family
ID=4164201
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA 2283382 Expired - Fee Related CA2283382C (en) | 1999-10-04 | 1999-10-04 | An apparatus and method for capturing and projecting an image of a cart |
Country Status (1)
| Country | Link |
|---|---|
| CA (1) | CA2283382C (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7219838B2 (en) | 2004-08-10 | 2007-05-22 | Howell Data Systems | System and method for notifying a cashier of the presence of an item in an obscured area of a shopping cart |
| DE202011103715U1 (en) * | 2011-07-26 | 2011-11-24 | Certus Warensicherungs-Systeme Gmbh | Apparatus for visually monitoring the passage of a commercial cash register PC in a shopping mall |
-
1999
- 1999-10-04 CA CA 2283382 patent/CA2283382C/en not_active Expired - Fee Related
Also Published As
| Publication number | Publication date |
|---|---|
| CA2283382A1 (en) | 1999-12-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110866429B (en) | Missing scanning identification method, device, self-service cashing terminal and system | |
| US5751346A (en) | Image retention and information security system | |
| US6317152B1 (en) | Digital video recording system | |
| US20090272801A1 (en) | Deterring checkout fraud | |
| EP3211891A1 (en) | Identification and imaging of terminal-proximate event occurrences | |
| US20030018897A1 (en) | Video identification verification system and method for a self-checkout system | |
| US20030122667A1 (en) | System and method for enhancing security at a self-checkout station | |
| US20070058040A1 (en) | Video surveillance using spatial-temporal motion analysis | |
| US20040111324A1 (en) | Integrated point-of-sale and surveillance system | |
| EP0847032A2 (en) | Enhanced security self-service checkout station | |
| KR100468590B1 (en) | Digital video recoding system and method by POS data | |
| WO2001093435A3 (en) | Distribution system | |
| JP2005115504A (en) | Image retrieval system and image database | |
| CN111310733A (en) | Method, device and equipment for detecting personnel entering and exiting based on monitoring video | |
| US20040130623A1 (en) | Point-of sale (POS) system with built-in digital video recorder (DVR) and data handling method thereof | |
| CA2283382C (en) | An apparatus and method for capturing and projecting an image of a cart | |
| US20140002646A1 (en) | Bottom of the basket surveillance system for shopping carts | |
| JP2005135313A (en) | Commodity management system | |
| CA2476150C (en) | Method and system for overlaying image with text | |
| JP2021009488A (en) | Theft suppression device | |
| EP0921505A2 (en) | Retail security system | |
| US9047742B2 (en) | Visual security for point of sale terminals | |
| WO2019151068A1 (en) | Information processing method, information processing device, and recording medium | |
| KR100794957B1 (en) | Digital video recoder capable of searching a text | |
| WO2003049057A1 (en) | A surveillance system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |