EP4266279A1 - Antitheft system for items in automatic checkouts and the like - Google Patents

Antitheft system for items in automatic checkouts and the like Download PDF

Info

Publication number
EP4266279A1
EP4266279A1 EP23168428.3A EP23168428A EP4266279A1 EP 4266279 A1 EP4266279 A1 EP 4266279A1 EP 23168428 A EP23168428 A EP 23168428A EP 4266279 A1 EP4266279 A1 EP 4266279A1
Authority
EP
European Patent Office
Prior art keywords
items
user
monitoring device
behavior
automatic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP23168428.3A
Other languages
German (de)
French (fr)
Inventor
Christian MAIOLO
Andrea GOLDONI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Intent Tech Srl
Original Assignee
Intent Tech Srl
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intent Tech Srl filed Critical Intent Tech Srl
Publication of EP4266279A1 publication Critical patent/EP4266279A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19602Image analysis to detect motion of the intruder, e.g. by frame subtraction
    • G08B13/19613Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
    • AHUMAN NECESSITIES
    • A47FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47FSPECIAL FURNITURE, FITTINGS, OR ACCESSORIES FOR SHOPS, STOREHOUSES, BARS, RESTAURANTS OR THE LIKE; PAYING COUNTERS
    • A47F9/00Shop, bar, bank or like counters
    • A47F9/02Paying counters
    • A47F9/04Check-out counters, e.g. for self-service stores

Definitions

  • the present invention relates to an antitheft system for items, i.e. products, of various types in automatic checkouts and the like.
  • the system according to the present invention is particularly, although not exclusively, useful and practical in the prevention of theft of items, i.e. products, at the automatic checkouts that are nowadays commonly installed at hypermarkets and supermarkets in the large-scale retail trade.
  • automated checkout means a self-service cash register where the user, i.e. the person buying the items, performs autonomously, i.e. alone, the reading or scanning of the bar codes of these items that he or she wishes to buy, pays the total price, for example with cash or an electronic payment card, and, finally, takes the associated purchase till receipt.
  • an automatic checkout of known type comprises substantially a checkout management device 16 and a checkout counter 18.
  • the buyer user 20 is positioned at the checkout counter 18.
  • the checkout counter 18 comprises an upper surface supported by a supporting structure. On the upper surface of the checkout counter 18, three main regions 18a, 18b and 18c can be identified, which are arranged in succession along a direction of egress of the items purchased by the buyer user 20 at the commercial business or sales point.
  • the three regions 18a, 18b and 18c comprise a first region 18a for setting down the items to be purchased, a second region 18b for registering the items to be purchased, and a third region 18c for bagging the registered, and therefore purchased, items.
  • the first region 18a is commonly constituted by a set-down surface, or surface portion, on which the buyer user 20 places the items to be purchased.
  • the second region 18b where the buyer user 20 is positioned, comprises means for product identification 19, preferably an optical barcode reader, with which the buyer user 20 registers the items to be purchased, preferably by scanning the corresponding bar codes.
  • the third region 18c is commonly constituted by a surface or surface portion for bagging, on which the buyer user 20 collects the items that he has purchased.
  • the checkout management device 16 is functionally connected to the means for product identification 19 comprised in the second region 18b of the checkout counter 18 of the automatic checkout.
  • the checkout management device 16 is configured to register, for example locally, information relating to the items scanned and registered during the purchase operations, in particular every purchase operation performed at the automatic checkout and, as a consequence, every item comprised in every purchase operation.
  • the checkout management device 16 is further configured to check the payment of the total price at the conclusion of every purchase operation.
  • the checkout management device 16 can be configured to record the date and time of the operation, the name and address of the business, the total number of the items, the total price of the items, information relating to the method of payment (cash, electronic payment card etc.), and the like.
  • antitheft systems for items are also known which are provided with electronic devices, in particular sensor devices, for example weight sensors, installed in, or proximate to, the automatic checkouts and the like, which are configured to verify that the items collected by the buyer user 20 in the third region 18c of the checkout counter 18 of the automatic checkout correspond to the items registered by that buyer user 20 in the second region 18b of the checkout counter 18 of the automatic checkout.
  • electronic devices in particular sensor devices, for example weight sensors, installed in, or proximate to, the automatic checkouts and the like, which are configured to verify that the items collected by the buyer user 20 in the third region 18c of the checkout counter 18 of the automatic checkout correspond to the items registered by that buyer user 20 in the second region 18b of the checkout counter 18 of the automatic checkout.
  • the lack of correspondence resulting from the above verification implies that the buyer user 20 has engaged in a fraudulent behavior, i.e. has attempted to steal something, in that he has placed at least one item in the third region 18c of the checkout counter 18 of the automatic checkout without this at least one item being registered in the second region 18b of the checkout counter 18 of the automatic checkout.
  • This lack of correspondence resulting from the above verification can block the operation of the automatic checkout or it can automatically generate an alarm signal, for example visual and/or aural, which enables the security personnel to intervene promptly.
  • the ill-intentioned buyer user 20 can position him- or herself with respect to the video surveillance system so that the video surveillance system cannot clearly view the operations executed at the checkout counter 18 of the automatic checkout, so as to conceal an attempted theft from the security personnel.
  • the ill-intentioned buyer user 20 can interfere with the operation of the sensor devices of the antitheft system so that the antitheft system detects no lack of correspondence between the items collected in the third region 18c of the checkout counter 18 of the automatic checkout and the items registered in the second region 18b of the checkout counter 18 of the automatic checkout.
  • fraudulent behavior of the buyer user 20 means an action to move an article from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18, with that item passing through the second region 18b of the checkout counter 18 without being scanned and consequently without being registered. Therefore, a fraudulent behavior is an attempted theft.
  • correct behavior of the buyer user 20 means an action to move an article from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18, with that item passing through the second region 18b of the checkout counter 18 and being scanned, by way of the means for product identification 19, and consequently being registered.
  • the aim of the present invention is to overcome the limitations of the known art described above, by devising an antitheft system for items in automatic checkouts and the like that makes it possible to obtain better effects than those that can be obtained with conventional solutions and/or similar effects at lower cost and with higher performance levels.
  • an object of the present invention is to conceive of an antitheft system for items in automatic checkouts and the like that makes it possible to perform behavioral analysis of the operations performed by the buyer user at the automatic checkout, by automating the collection of possible fraudulent behaviors, i.e. potential attempted thefts, for a prompt and precise investigation by security personnel.
  • Another object of the present invention is to devise an antitheft system for items in automatic checkouts and the like that makes it possible to support security personnel in their decisions, by reporting in real time possible fraudulent behaviors, i.e. potential attempted thefts, and by providing an immediate or retrospective verification to the security personnel.
  • Another object of the present invention is to conceive of an antitheft system for items in automatic checkouts and the like that makes it possible to prevent, or at least minimize, the possibility of circumvention or deceit by a buyer user.
  • Another object of the present invention is to devise an antitheft system for items in automatic checkouts and the like that makes it possible to provide data to support theft prevention strategies.
  • Another object of the present invention is to provide an antitheft system for items in automatic checkouts and the like that is highly reliable, easily and practically implemented, and economically competitive when compared to the known art.
  • an antitheft system for items in automatic checkouts and the like characterized in that it comprises a monitoring device and at least one vision device, said monitoring device being functionally connected to said vision device, said vision device being associable with a respective checkout counter of an automatic checkout where a user is operating and being orientable toward said checkout counter so as to view both said checkout counter and said user;
  • the antitheft system for items in automatic checkouts and the like comprises substantially a monitoring device 12 and at least one vision device 14.
  • the monitoring device 12 is functionally connected to the vision device 14, and vice versa.
  • the vision device 14 can be a video camera.
  • an automatic checkout comprises substantially a checkout management device 16 and a checkout counter 18.
  • the checkout counter 18 of the automatic checkout comprises a first region 18a, a second region 18b and a third region 18c.
  • the vision device 14 of the system 10 according to the invention is associated or associable with a respective checkout counter 18 of an automatic checkout where a user 20, i.e. the buyer of the items or products, is operating.
  • the vision device 14 is firmly oriented or orientable toward the checkout counter 18 of the automatic checkout, where the buyer user 20 sets down, scans, registers and collects the items during the purchase operation.
  • the vision device 14 views both the buyer user 20 and the checkout counter 18, in particular the first region 18a, the second region 18b and the third region 18c.
  • the vision device 14 also views the items being handled by the buyer user 20.
  • the vision device 14 is firmly oriented looking downward from above, where the checkout counter 18 of the automatic checkout is located. Even more preferably, the longitudinal axis of the vision device 14 is perpendicular to the longitudinal axis of the checkout counter 18 of the automatic checkout.
  • the vision device 14 is configured to acquire a context video that comprises images of the buyer user 20, of the items being handled by the buyer user 20, and of the checkout counter 18, in particular the first region 18a, the second region 18b and the third region 18c.
  • the vision device 14 is configured to acquire the context video with a frame rate higher than eight frames per second, since a frame rate less than or equal to eight frames per second would not be optimal and could limit the efficacy of some of the processing and/or analysis operations described below.
  • the vision device 14 is further configured to send the context video, thus acquired, to the monitoring device 12.
  • the monitoring device 12 of the system 10 comprises substantially an electronic control unit 30, an image processing module 32 and a behavior analysis module 34.
  • the monitoring device 12 further comprises a memory unit 38.
  • the electronic control unit 30 is the main functional element of the monitoring device 12, and for this reason it is functionally connected with the other elements comprised in the monitoring device 12.
  • the electronic control unit 30 of the monitoring device 12 is provided with suitable capacity for processing and for interfacing with the other elements of the monitoring device 12, and it is configured to command, control and coordinate the operation of the elements of the monitoring device 12 with which it is functionally connected.
  • the monitoring device 12 is configured to receive the context video acquired by and originating from the vision device 14.
  • the memory unit 38 of the monitoring device 12 is configured to record the context video acquired by and originating from the vision device 14.
  • the image processing module 32 of the monitoring device 12 is configured to extract information relating to the behavior (meaning movements) of the buyer user 20 from the context video, which was previously acquired by the vision device 14, identifying movements of items from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18.
  • the image processing module 32 is configured as above by drawing on suitable artificial vision techniques.
  • the image processing module 32 is further configured to segment the context video prior to extracting the information relating to the behavior of the buyer user 20.
  • the memory unit 38 of the monitoring device 12 is further configured to record the information relating to the behavior (meaning movements) of the buyer user 20 which is extracted by the image processing module 32.
  • the image processing module 32 of the monitoring device 12 is configured to extract the information relating to the behavior of the buyer user 20 by applying the artificial vision technique known as Optical Flow.
  • Optical Flow estimates the movement in a video, in this case the context video, assigning each pixel of the current frame a movement vector that points toward the position of that pixel in a subsequent frame.
  • Optical Flow is particularly useful for identifying the direction of travel of an item that passes through the second region 18b of the checkout counter 18, the region that comprises the means for product identification 19.
  • the direction of travel is essential for distinguishing the "empty" hand (not holding an item) of the buyer user 20, moving from the third region 18c of the checkout counter 18 to the first region 18a of the checkout counter 18, from the "full" hand (holding an item) of the same buyer user 20, which by contrast moves from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18.
  • the image processing module 32 of the monitoring device 12 is configured to extract the information relating to the behavior of the buyer user 20 by applying the artificial vision technique known as Background Suppression.
  • Background Suppression detects moving objects in a video, in this case the context video, acquired by a static device, in this case the vision device 14.
  • the logic behind this technique is to detect moving objects from differences between the current frame and a reference frame, called the "background image”.
  • the image processing module 32 of the monitoring device 12 is further configured to extract information relating to the items moved by the buyer user 20 from the context video, which was previously acquired by the vision device 14, identifying specific predefined types of items, for example items of high economic worth.
  • the memory unit 38 of the monitoring device 12 is further configured to record information relating to the item moved by the buyer user 20 which is extracted by the image processing module 32 of the monitoring device 12.
  • the behavior analysis module 34 of the monitoring device 12 is configured to identify a type of behavior of the buyer user 20, distinguishing between correct behavior (such as for example shown in Figure 3A ) or fraudulent behavior (such as for example shown in Figure 3B ), on the basis of the information relating to the behavior (meaning movements) of the buyer user 20, previously extracted by the image processing module 32.
  • the behavior analysis module 34 is configured as above by drawing on suitable artificial intelligence techniques (machine learning, deep learning, statistical models and mathematical models).
  • the behavior analysis module 34 implements a Convolutional Neural Network (CNN).
  • the memory unit 38 of the monitoring device 12 is further configured to record the type of behavior, alternatively correct or fraudulent, of the buyer user 20, identified by the behavior analysis module 34 of the monitoring device 12.
  • the monitoring device 12 further comprises a comparison module 36.
  • the monitoring device 12 is functionally connected to the checkout management device 16, and vice versa.
  • the comparison module 36 is configured to interface with the checkout management device 16 of the automatic checkout, and to acquire information relating to the items scanned and registered during the purchase operation.
  • the comparison module 36 is further configured to correlate the information relating to the behavior (meaning movements) of the buyer user 20 with the information relating to the items scanned and registered in the automatic checkout during the purchase operation, identifying any discrepancy between the movements of items from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18 (and/or their number) and the items scanned and registered in the automatic checkout during the purchase operation (and/or their number).
  • the memory unit 38 of the monitoring device 12 is further configured to record a possible discrepancy between the movements of items from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18 (and/or their number) and the items scanned and registered during the purchase operation (and/or their number), identified by the comparison module 36 of the monitoring device 12.
  • the antitheft system 10 for items in automatic checkouts and the like further comprises a user interface device 24, 26 which is operated by security personnel.
  • the monitoring device 12 is functionally connected to the user interface device 24, 26, and vice versa, for example by way of a telematic communication network 22.
  • the user interface device 24, 26 can be a personal computer 24, of the desktop or laptop type, or a mobile device 26, of the smartphone or tablet type.
  • the monitoring device 12 is configured to send an indicator of the type of behavior of the buyer user 20, either correct or fraudulent, identified by the behavior analysis module 34 of the monitoring device 12, to the user interface device 24, 26.
  • the user interface device 24, 26 is configured to graphically display this indicator of the type of behavior of the buyer user 20, either correct (such as for example shown in Figure 4A ) or fraudulent (such as for example shown in Figure 4B ).
  • the monitoring device 12 is configured to send an alarm signal to the user interface device 24, 26 when the behavior analysis module 34 of the monitoring device 12 identifies a fraudulent behavior of the buyer user 20.
  • the user interface device 24, 26 is configured to graphically display this alarm signal.
  • the monitoring device 12 is configured to send an alarm signal to the user interface device 24, 26 when the comparison module 36 of the monitoring device 12 identifies a discrepancy between the movements of items from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18 (and/or their number) and the items scanned and registered in the automatic checkout during the purchase operation (and/or their number).
  • the user interface device 24, 26 is configured to graphically display this alarm signal.
  • the graphic indicators described above, displayed by the user interface device 24, 26, are accompanied by respective acoustic alerts.
  • the user interface device 24, 26 is further configured to display the context video acquired by the vision device 14 and originating from the monitoring device 12. In an embodiment, the user interface device 24, 26 is further configured to graphically display, superimposed on the context video, at least one orientation guide of the vision device 14.
  • the user interface device 24, 26 is further configured to graphically display one or more rapid training buttons which are operated by security personnel and are configured to confirm or review the work of the behavior analysis module 34 of the monitoring device 12, and therefore train said behavior analysis module.
  • the user interface device 24, 26 is further configured to graphically display statistical information of various kinds extracted from a processing of information and data recorded in the memory unit 38 of the monitoring device 12. For example, this statistical information can be displayed by the user interface device 24, 26 in the form of summary tables, detailed tables, diagrams, graphs etc.
  • the present invention fully achieves the set aim and objects.
  • the antitheft system for items in automatic checkouts and the like thus conceived makes it possible to overcome the qualitative limitations of the known art, in that it makes it possible to obtain better effects than those that can be obtained with conventional solutions and/or similar effects at lower cost and with higher performance levels.
  • An advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to perform behavioral analysis of the operations performed by the buyer user at the automatic checkout, by automating the collection of possible fraudulent behaviors, i.e. potential attempted thefts, for a prompt and precise investigation by security personnel.
  • Another advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to support security personnel in their decisions, by reporting in real time possible fraudulent behaviors, i.e. potential attempted thefts, and by providing an immediate or retrospective verification to the security personnel.
  • Another advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to prevent, or at least minimize, the possibility of circumvention or deceit by a buyer user.
  • Another advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to provide data to support theft prevention strategies.
  • the antitheft system for items in automatic checkouts and the like has been conceived in particular for the prevention of theft of items, i.e. products, at the automatic checkouts that are nowadays commonly installed at hypermarkets and supermarkets in the large-scale retail trade, it can also be used, more generally, for the prevention of theft of items, i.e. products, at automatic checkouts installed at a commercial business or sales point of any kind and size.

Abstract

An antitheft system (10) for items in automatic checkouts and the like, which comprises a monitoring device (12) and at least one vision device (14), the monitoring device (12) being functionally connected to the vision device (14), the vision device (12) being associable with a respective checkout counter (18) of an automatic checkout where a user (20) is operating and being orientable toward the checkout counter (18) so as to view both the checkout counter (18) and the user (20);
the vision device (14) is configured to acquire a context video comprising images of the checkout counter (18), of the user (20) and of items handled by the user (20);
the monitoring device (12) comprises:
- an image processing module (32) which is configured to extract information relating to the behavior of the user (20) from the context video, identifying movements of items from a first region (18a) of the checkout counter (18) to a third region (18c) of the checkout counter (18);
- a behavior analysis module (34) which is configured to identify a type of behavior of the user (20), distinguishing between correct behavior or fraudulent behavior, on the basis of the information relating to the behavior of the user (20).

Description

  • The present invention relates to an antitheft system for items, i.e. products, of various types in automatic checkouts and the like. The system according to the present invention is particularly, although not exclusively, useful and practical in the prevention of theft of items, i.e. products, at the automatic checkouts that are nowadays commonly installed at hypermarkets and supermarkets in the large-scale retail trade.
  • Note that the term "automatic checkout" means a self-service cash register where the user, i.e. the person buying the items, performs autonomously, i.e. alone, the reading or scanning of the bar codes of these items that he or she wishes to buy, pays the total price, for example with cash or an electronic payment card, and, finally, takes the associated purchase till receipt.
  • With reference to a portion of Figure 1, an automatic checkout of known type comprises substantially a checkout management device 16 and a checkout counter 18. The buyer user 20 is positioned at the checkout counter 18.
  • The checkout counter 18 comprises an upper surface supported by a supporting structure. On the upper surface of the checkout counter 18, three main regions 18a, 18b and 18c can be identified, which are arranged in succession along a direction of egress of the items purchased by the buyer user 20 at the commercial business or sales point.
  • In particular, the three regions 18a, 18b and 18c comprise a first region 18a for setting down the items to be purchased, a second region 18b for registering the items to be purchased, and a third region 18c for bagging the registered, and therefore purchased, items.
  • The first region 18a is commonly constituted by a set-down surface, or surface portion, on which the buyer user 20 places the items to be purchased. Then, i.e. downstream of the first region 18a, the second region 18b, where the buyer user 20 is positioned, comprises means for product identification 19, preferably an optical barcode reader, with which the buyer user 20 registers the items to be purchased, preferably by scanning the corresponding bar codes. Finally, i.e. downstream of the second region 18b, the third region 18c is commonly constituted by a surface or surface portion for bagging, on which the buyer user 20 collects the items that he has purchased.
  • The checkout management device 16 is functionally connected to the means for product identification 19 comprised in the second region 18b of the checkout counter 18 of the automatic checkout. The checkout management device 16 is configured to register, for example locally, information relating to the items scanned and registered during the purchase operations, in particular every purchase operation performed at the automatic checkout and, as a consequence, every item comprised in every purchase operation. The checkout management device 16 is further configured to check the payment of the total price at the conclusion of every purchase operation.
  • For example, for every purchase operation, in addition to the list of the items and of the corresponding prices, the checkout management device 16 can be configured to record the date and time of the operation, the name and address of the business, the total number of the items, the total price of the items, information relating to the method of payment (cash, electronic payment card etc.), and the like.
  • Nowadays video surveillance systems are known which are installed proximate to automatic checkouts and the like, and from which security personnel can monitor and be vigilant for behaviors (meaning movements) of the buyer user 20 who is operating on the automatic checkout, intervening promptly if a fraudulent behavior, i.e. an attempted theft, is detected.
  • Currently antitheft systems for items are also known which are provided with electronic devices, in particular sensor devices, for example weight sensors, installed in, or proximate to, the automatic checkouts and the like, which are configured to verify that the items collected by the buyer user 20 in the third region 18c of the checkout counter 18 of the automatic checkout correspond to the items registered by that buyer user 20 in the second region 18b of the checkout counter 18 of the automatic checkout.
  • The lack of correspondence resulting from the above verification implies that the buyer user 20 has engaged in a fraudulent behavior, i.e. has attempted to steal something, in that he has placed at least one item in the third region 18c of the checkout counter 18 of the automatic checkout without this at least one item being registered in the second region 18b of the checkout counter 18 of the automatic checkout.
  • This lack of correspondence resulting from the above verification can block the operation of the automatic checkout or it can automatically generate an alarm signal, for example visual and/or aural, which enables the security personnel to intervene promptly.
  • These antitheft systems for items in automatic checkouts and the like of known type are not devoid of drawbacks, among which is the fact that they can be circumvented or deceived by a buyer user 20 who wishes to exhibit a fraudulent behavior.
  • For example, the ill-intentioned buyer user 20 can position him- or herself with respect to the video surveillance system so that the video surveillance system cannot clearly view the operations executed at the checkout counter 18 of the automatic checkout, so as to conceal an attempted theft from the security personnel.
  • Also for example, the ill-intentioned buyer user 20 can interfere with the operation of the sensor devices of the antitheft system so that the antitheft system detects no lack of correspondence between the items collected in the third region 18c of the checkout counter 18 of the automatic checkout and the items registered in the second region 18b of the checkout counter 18 of the automatic checkout.
  • Note that the term "fraudulent behavior" of the buyer user 20 means an action to move an article from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18, with that item passing through the second region 18b of the checkout counter 18 without being scanned and consequently without being registered. Therefore, a fraudulent behavior is an attempted theft.
  • Note that the term "correct behavior" of the buyer user 20 means an action to move an article from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18, with that item passing through the second region 18b of the checkout counter 18 and being scanned, by way of the means for product identification 19, and consequently being registered.
  • The aim of the present invention is to overcome the limitations of the known art described above, by devising an antitheft system for items in automatic checkouts and the like that makes it possible to obtain better effects than those that can be obtained with conventional solutions and/or similar effects at lower cost and with higher performance levels.
  • Within this aim, an object of the present invention is to conceive of an antitheft system for items in automatic checkouts and the like that makes it possible to perform behavioral analysis of the operations performed by the buyer user at the automatic checkout, by automating the collection of possible fraudulent behaviors, i.e. potential attempted thefts, for a prompt and precise investigation by security personnel.
  • Another object of the present invention is to devise an antitheft system for items in automatic checkouts and the like that makes it possible to support security personnel in their decisions, by reporting in real time possible fraudulent behaviors, i.e. potential attempted thefts, and by providing an immediate or retrospective verification to the security personnel.
  • Another object of the present invention is to conceive of an antitheft system for items in automatic checkouts and the like that makes it possible to prevent, or at least minimize, the possibility of circumvention or deceit by a buyer user.
  • Another object of the present invention is to devise an antitheft system for items in automatic checkouts and the like that makes it possible to provide data to support theft prevention strategies.
  • Another object of the present invention is to provide an antitheft system for items in automatic checkouts and the like that is highly reliable, easily and practically implemented, and economically competitive when compared to the known art.
  • This aim and these and other objects which will become more apparent hereinafter are achieved by an antitheft system for items in automatic checkouts and the like, characterized in that it comprises a monitoring device and at least one vision device, said monitoring device being functionally connected to said vision device, said vision device being associable with a respective checkout counter of an automatic checkout where a user is operating and being orientable toward said checkout counter so as to view both said checkout counter and said user;
    • said vision device being configured to acquire a context video comprising images of said checkout counter, of said user and of items handled by said user;
    • said monitoring device comprising:
      • an image processing module which is configured to extract information relating to the behavior of said user from said context video, identifying movements of items from a first region of said checkout counter to a third region of said checkout counter; and
      • a behavior analysis module which is configured to identify a type of behavior of said user, distinguishing between correct behavior or fraudulent behavior, on the basis of said information relating to the behavior of said user.
  • Further characteristics and advantages of the present invention will become apparent from the description of a preferred, but not exclusive, embodiment of the antitheft system for items in automatic checkouts and the like according to the invention, illustrated by way of non-limiting example with the aid of the accompanying drawings wherein:
    • Figure 1 is a block diagram schematically showing an embodiment of the antitheft system for items in automatic checkouts and the like according to the present invention;
    • Figure 2 is a block diagram schematically showing an embodiment of the monitoring device of the antitheft system for items in automatic checkouts and the like according to the present invention;
    • Figures 3A and 3B respectively show an example of correct behavior (meaning movements) of the buyer user and an example of fraudulent behavior (meaning movements) of the buyer user, at an automatic checkout;
    • Figures 4A and 4B respectively show a graphic indicator of correct behavior of the buyer user and a graphic indicator of fraudulent behavior of the buyer user.
  • With reference to Figure 1, the antitheft system for items in automatic checkouts and the like according to the present invention, generally designated by the reference numeral 10, comprises substantially a monitoring device 12 and at least one vision device 14. The monitoring device 12 is functionally connected to the vision device 14, and vice versa. The vision device 14 can be a video camera.
  • As mentioned, an automatic checkout comprises substantially a checkout management device 16 and a checkout counter 18. The checkout counter 18 of the automatic checkout comprises a first region 18a, a second region 18b and a third region 18c.
  • The vision device 14 of the system 10 according to the invention is associated or associable with a respective checkout counter 18 of an automatic checkout where a user 20, i.e. the buyer of the items or products, is operating.
  • The vision device 14 is firmly oriented or orientable toward the checkout counter 18 of the automatic checkout, where the buyer user 20 sets down, scans, registers and collects the items during the purchase operation. In other words, the vision device 14 views both the buyer user 20 and the checkout counter 18, in particular the first region 18a, the second region 18b and the third region 18c. As a consequence, the vision device 14 also views the items being handled by the buyer user 20.
  • Preferably, the vision device 14 is firmly oriented looking downward from above, where the checkout counter 18 of the automatic checkout is located. Even more preferably, the longitudinal axis of the vision device 14 is perpendicular to the longitudinal axis of the checkout counter 18 of the automatic checkout.
  • The vision device 14 is configured to acquire a context video that comprises images of the buyer user 20, of the items being handled by the buyer user 20, and of the checkout counter 18, in particular the first region 18a, the second region 18b and the third region 18c. Advantageously, the vision device 14 is configured to acquire the context video with a frame rate higher than eight frames per second, since a frame rate less than or equal to eight frames per second would not be optimal and could limit the efficacy of some of the processing and/or analysis operations described below.
  • The vision device 14 is further configured to send the context video, thus acquired, to the monitoring device 12.
  • With reference to Figure 2, the monitoring device 12 of the system 10 according to the invention comprises substantially an electronic control unit 30, an image processing module 32 and a behavior analysis module 34. Advantageously, the monitoring device 12 further comprises a memory unit 38.
  • The electronic control unit 30 is the main functional element of the monitoring device 12, and for this reason it is functionally connected with the other elements comprised in the monitoring device 12.
  • The electronic control unit 30 of the monitoring device 12 is provided with suitable capacity for processing and for interfacing with the other elements of the monitoring device 12, and it is configured to command, control and coordinate the operation of the elements of the monitoring device 12 with which it is functionally connected.
  • The monitoring device 12 is configured to receive the context video acquired by and originating from the vision device 14.
  • Preferably, the memory unit 38 of the monitoring device 12 is configured to record the context video acquired by and originating from the vision device 14.
  • The image processing module 32 of the monitoring device 12 is configured to extract information relating to the behavior (meaning movements) of the buyer user 20 from the context video, which was previously acquired by the vision device 14, identifying movements of items from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18. In particular, the image processing module 32 is configured as above by drawing on suitable artificial vision techniques. Preferably, the image processing module 32 is further configured to segment the context video prior to extracting the information relating to the behavior of the buyer user 20.
  • Preferably, the memory unit 38 of the monitoring device 12 is further configured to record the information relating to the behavior (meaning movements) of the buyer user 20 which is extracted by the image processing module 32.
  • In an embodiment, the image processing module 32 of the monitoring device 12 is configured to extract the information relating to the behavior of the buyer user 20 by applying the artificial vision technique known as Optical Flow.
  • Optical Flow estimates the movement in a video, in this case the context video, assigning each pixel of the current frame a movement vector that points toward the position of that pixel in a subsequent frame.
  • For example, Optical Flow is particularly useful for identifying the direction of travel of an item that passes through the second region 18b of the checkout counter 18, the region that comprises the means for product identification 19. For example, the direction of travel is essential for distinguishing the "empty" hand (not holding an item) of the buyer user 20, moving from the third region 18c of the checkout counter 18 to the first region 18a of the checkout counter 18, from the "full" hand (holding an item) of the same buyer user 20, which by contrast moves from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18.
  • In an embodiment, the image processing module 32 of the monitoring device 12 is configured to extract the information relating to the behavior of the buyer user 20 by applying the artificial vision technique known as Background Suppression.
  • Background Suppression detects moving objects in a video, in this case the context video, acquired by a static device, in this case the vision device 14. The logic behind this technique is to detect moving objects from differences between the current frame and a reference frame, called the "background image".
  • For Background Suppression to be effective, some conditions must be met, which are a constant background, in this case the checkout counter 18 of the automatic checkout, and lighting that is substantially uniform over time.
  • Advantageously, the image processing module 32 of the monitoring device 12 is further configured to extract information relating to the items moved by the buyer user 20 from the context video, which was previously acquired by the vision device 14, identifying specific predefined types of items, for example items of high economic worth.
  • Preferably, the memory unit 38 of the monitoring device 12 is further configured to record information relating to the item moved by the buyer user 20 which is extracted by the image processing module 32 of the monitoring device 12.
  • The behavior analysis module 34 of the monitoring device 12 is configured to identify a type of behavior of the buyer user 20, distinguishing between correct behavior (such as for example shown in Figure 3A) or fraudulent behavior (such as for example shown in Figure 3B), on the basis of the information relating to the behavior (meaning movements) of the buyer user 20, previously extracted by the image processing module 32. In particular, the behavior analysis module 34 is configured as above by drawing on suitable artificial intelligence techniques (machine learning, deep learning, statistical models and mathematical models). Preferably, the behavior analysis module 34 implements a Convolutional Neural Network (CNN).
  • Preferably, the memory unit 38 of the monitoring device 12 is further configured to record the type of behavior, alternatively correct or fraudulent, of the buyer user 20, identified by the behavior analysis module 34 of the monitoring device 12.
  • Advantageously, the monitoring device 12 further comprises a comparison module 36. In this case, the monitoring device 12 is functionally connected to the checkout management device 16, and vice versa.
  • The comparison module 36 is configured to interface with the checkout management device 16 of the automatic checkout, and to acquire information relating to the items scanned and registered during the purchase operation. The comparison module 36 is further configured to correlate the information relating to the behavior (meaning movements) of the buyer user 20 with the information relating to the items scanned and registered in the automatic checkout during the purchase operation, identifying any discrepancy between the movements of items from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18 (and/or their number) and the items scanned and registered in the automatic checkout during the purchase operation (and/or their number).
  • Preferably, the memory unit 38 of the monitoring device 12 is further configured to record a possible discrepancy between the movements of items from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18 (and/or their number) and the items scanned and registered during the purchase operation (and/or their number), identified by the comparison module 36 of the monitoring device 12.
  • Advantageously, the antitheft system 10 for items in automatic checkouts and the like according to the present invention further comprises a user interface device 24, 26 which is operated by security personnel. The monitoring device 12 is functionally connected to the user interface device 24, 26, and vice versa, for example by way of a telematic communication network 22. The user interface device 24, 26 can be a personal computer 24, of the desktop or laptop type, or a mobile device 26, of the smartphone or tablet type.
  • Advantageously, the monitoring device 12 is configured to send an indicator of the type of behavior of the buyer user 20, either correct or fraudulent, identified by the behavior analysis module 34 of the monitoring device 12, to the user interface device 24, 26. As a consequence, the user interface device 24, 26 is configured to graphically display this indicator of the type of behavior of the buyer user 20, either correct (such as for example shown in Figure 4A) or fraudulent (such as for example shown in Figure 4B).
  • Advantageously, the monitoring device 12 is configured to send an alarm signal to the user interface device 24, 26 when the behavior analysis module 34 of the monitoring device 12 identifies a fraudulent behavior of the buyer user 20. As a consequence, the user interface device 24, 26 is configured to graphically display this alarm signal.
  • Advantageously, the monitoring device 12 is configured to send an alarm signal to the user interface device 24, 26 when the comparison module 36 of the monitoring device 12 identifies a discrepancy between the movements of items from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18 (and/or their number) and the items scanned and registered in the automatic checkout during the purchase operation (and/or their number). As a consequence, the user interface device 24, 26 is configured to graphically display this alarm signal.
  • Optionally, the graphic indicators described above, displayed by the user interface device 24, 26, are accompanied by respective acoustic alerts.
  • In a preferred embodiment, the user interface device 24, 26 is further configured to display the context video acquired by the vision device 14 and originating from the monitoring device 12. In an embodiment, the user interface device 24, 26 is further configured to graphically display, superimposed on the context video, at least one orientation guide of the vision device 14.
  • In an embodiment, the user interface device 24, 26 is further configured to graphically display one or more rapid training buttons which are operated by security personnel and are configured to confirm or review the work of the behavior analysis module 34 of the monitoring device 12, and therefore train said behavior analysis module.
  • In an embodiment, the user interface device 24, 26 is further configured to graphically display statistical information of various kinds extracted from a processing of information and data recorded in the memory unit 38 of the monitoring device 12. For example, this statistical information can be displayed by the user interface device 24, 26 in the form of summary tables, detailed tables, diagrams, graphs etc.
  • In practice it has been found that the present invention fully achieves the set aim and objects. In particular, it has been seen that the antitheft system for items in automatic checkouts and the like thus conceived makes it possible to overcome the qualitative limitations of the known art, in that it makes it possible to obtain better effects than those that can be obtained with conventional solutions and/or similar effects at lower cost and with higher performance levels.
  • An advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to perform behavioral analysis of the operations performed by the buyer user at the automatic checkout, by automating the collection of possible fraudulent behaviors, i.e. potential attempted thefts, for a prompt and precise investigation by security personnel.
  • Another advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to support security personnel in their decisions, by reporting in real time possible fraudulent behaviors, i.e. potential attempted thefts, and by providing an immediate or retrospective verification to the security personnel.
  • Another advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to prevent, or at least minimize, the possibility of circumvention or deceit by a buyer user.
  • Another advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to provide data to support theft prevention strategies.
  • Although the antitheft system for items in automatic checkouts and the like according to the invention has been conceived in particular for the prevention of theft of items, i.e. products, at the automatic checkouts that are nowadays commonly installed at hypermarkets and supermarkets in the large-scale retail trade, it can also be used, more generally, for the prevention of theft of items, i.e. products, at automatic checkouts installed at a commercial business or sales point of any kind and size.
  • The invention thus conceived is susceptible of numerous modifications and variations, all of which are within the scope of the appended claims. Moreover, all the details may be substituted by other, technically equivalent elements.
  • In practice the materials employed, provided they are compatible with the specific use, and the contingent dimensions and shapes, may be any according to requirements and to the state of the art.
  • In conclusion, the scope of protection of the claims shall not be limited by the explanations or by the preferred embodiments illustrated in the description by way of examples, but rather the claims shall comprise all the patentable characteristics of novelty that reside in the present invention, including all the characteristics that would be considered as equivalent by the person skilled in the art.
  • The disclosures in Italian Patent Application No. 102022000007865 from which this application claims priority are incorporated herein by reference.
  • Where the technical features mentioned in any claim are followed by reference numerals and/or signs, those reference numerals and/or signs have been included for the sole purpose of increasing the intelligibility of the claims and accordingly, such reference numerals and/or signs do not have any limiting effect on the interpretation of each element identified by way of example by such reference numerals and/or signs.

Claims (15)

  1. An antitheft system (10) for items in automatic checkouts and the like, characterized in that it comprises a monitoring device (12) and at least one vision device (14), said monitoring device (12) being functionally connected to said vision device (14), said vision device (12) being associable with a respective checkout counter (18) of an automatic checkout where a user (20) is operating and being orientable toward said checkout counter (18) so as to view both said checkout counter (18) and said user (20);
    said vision device (14) being configured to acquire a context video comprising images of said checkout counter (18), of said user (20) and of items handled by said user (20);
    said monitoring device (12) comprising:
    - an image processing module (32) which is configured to extract information relating to the behavior of said user (20) from said context video, identifying movements of items from a first region (18a) of said checkout counter (18) to a third region (18c) of said checkout counter (18); and
    - a behavior analysis module (34) which is configured to identify a type of behavior of said user (20), distinguishing between correct behavior or fraudulent behavior, on the basis of said information relating to the behavior of said user (20).
  2. The antitheft system (10) for items in automatic checkouts and the like according to claim 1, characterized in that said monitoring device (12) further comprises a memory unit (38) which is configured to record:
    - said context video, acquired by and originating from said vision device (14);
    - said information relating to the behavior of said user (20), extracted from said image processing module (32) of said monitoring device (12); and
    - said type of behavior of said buyer user (20), identified by said behavior analysis module (34) of said monitoring device (12).
  3. The antitheft system (10) for items in automatic checkouts and the like according to claim 1 or 2, characterized in that said monitoring device (12) is functionally connected to a checkout management device (16) of said automatic checkout, and in that said monitoring device (12) further comprises a comparison module (36) which is configured to correlate said information relating to the behavior of said user (20) with information relating to said items scanned and registered in said automatic checkout, identifying any discrepancy between said movements of items from said first region (18) of said checkout counter (18) to said third region (18c) of said checkout counter (18) and said items scanned and registered in said automatic checkout.
  4. The antitheft system (10) for items in automatic checkouts and the like according to claim 3, characterized in that said memory unit (38) of said monitoring device (12) is further configured to register said possible discrepancy, identified by said comparison module (36) of said monitoring device (12), between said movements of items from said first region (18a) of said checkout counter (18) to said third region (18c) of said checkout counter (18) and said items scanned and registered in said automatic checkout.
  5. The antitheft system (10) for items in automatic checkouts and the like according to any one of the preceding claims, characterized in that said image processing module (32) of said monitoring device (12) is configured to extract said information relating to the behavior of said user (20) by applying the artificial vision technique known as Optical Flow.
  6. The antitheft system (10) for items in automatic checkouts and the like according to any one of the preceding claims, characterized in that said image processing module (32) of said monitoring device (12) is configured to extract said information relating to the behavior of said user (20) by applying the artificial vision technique known as Background Suppression.
  7. The antitheft system (10) for items in automatic checkouts and the like according to any one of the preceding claims, characterized in that said image processing module (32) of said monitoring device (12) is further configured to extract information relating to said items moved by said buyer user (20) from the context video, identifying specific predefined types of items.
  8. The antitheft system (10) for items in automatic checkouts and the like according to any one of the preceding claims, characterized in that said vision device (14) is configured to acquire said context video with a frame rate higher than eight frames per second.
  9. The antitheft system (10) for items in automatic checkouts and the like according to any one of the preceding claims, characterized in that it further comprises a user interface device (24, 26), said monitoring device (12) being functionally connected to said user interface device (24, 26).
  10. The antitheft system (10) for items in automatic checkouts and the like according to claim 9, characterized in that said user interface device (24, 26) is configured to graphically display a signal relating to said type of behavior of said buyer user (20), identified by said behavior analysis module (34) of said monitoring device (12).
  11. The antitheft system (10) for items in automatic checkouts and the like according to claim 9 or 10, characterized in that said user interface device (24, 26) is configured to graphically display an alarm signal when said behavior analysis module (34) of said monitoring device (12) identifies a fraudulent behavior of said user (20).
  12. The antitheft system (10) for items in automatic checkouts and the like according to any one of claims 9 to 11, characterized in that said user interface device (24, 26) is configured to graphically display an alarm signal when said comparison module (36) of said monitoring device (12) identifies said possible discrepancy between said movements of items from said first region (18) of said checkout counter (18) to said third region (18c) of said checkout counter (18) and said items scanned and registered in said automatic checkout.
  13. The antitheft system (10) for items in automatic checkouts and the like according to any one of claims 9 to 12, characterized in that said user interface device (24, 26) is configured to display said context video acquired by said vision device (14) and originating from said monitoring device (12).
  14. The antitheft system (10) for items in automatic checkouts and the like according to claim 13, characterized in that said user interface device (24, 26) is further configured to graphically display, superimposed on said context video, at least one orientation guide of said vision device (14).
  15. The antitheft system (10) for items in automatic checkouts and the like according to any one of claims 9 to 14, characterized in that said user interface device (24, 26) is configured to graphically display statistical information of various kinds extracted from a processing of information and data recorded in said memory unit (38) of said monitoring device (12).
EP23168428.3A 2022-04-21 2023-04-18 Antitheft system for items in automatic checkouts and the like Pending EP4266279A1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
IT202200007865 2022-04-21

Publications (1)

Publication Number Publication Date
EP4266279A1 true EP4266279A1 (en) 2023-10-25

Family

ID=82608617

Family Applications (1)

Application Number Title Priority Date Filing Date
EP23168428.3A Pending EP4266279A1 (en) 2022-04-21 2023-04-18 Antitheft system for items in automatic checkouts and the like

Country Status (1)

Country Link
EP (1) EP4266279A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080226129A1 (en) * 2007-03-12 2008-09-18 Malay Kundu Cart Inspection for Suspicious Items
EP2704079A1 (en) * 2012-08-31 2014-03-05 NCR Corporation Checkout security using video surveillance
US9589433B1 (en) * 2013-07-31 2017-03-07 Jeff Thramann Self-checkout anti-theft device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080226129A1 (en) * 2007-03-12 2008-09-18 Malay Kundu Cart Inspection for Suspicious Items
EP2704079A1 (en) * 2012-08-31 2014-03-05 NCR Corporation Checkout security using video surveillance
US9589433B1 (en) * 2013-07-31 2017-03-07 Jeff Thramann Self-checkout anti-theft device

Similar Documents

Publication Publication Date Title
CN104885130B (en) Verification to the fraudulent activities at self checkout terminal
US8876001B2 (en) Methods and apparatus for image recognition in checkout verification
US20220156686A1 (en) Commodity monitoring device, commodity monitoring system, output destination device, commodity monitoring method, display method and program
US20060032914A1 (en) System and method for notifying a cashier of the presence of an item in an obscured area of a shopping cart
CA3053106A1 (en) Systems and methods for shopping in a physical store
CN212229682U (en) Open type self-service sale system based on buyer position positioning
EP1354296A2 (en) Monitoring responses to visual stimuli
US20130015242A1 (en) Tri-Optic Scanner
CN110866429A (en) Missed scanning identification method and device, self-service cash register terminal and system
WO1993017404A1 (en) Security system
US10383461B2 (en) System of control and identification of goods in a shop
US10372998B2 (en) Object recognition for bottom of basket detection
JP2004171240A (en) Illegality monitoring system and program
Connell et al. Retail video analytics: an overview and survey
JP2016081364A (en) Information notification program and information processing device
JP2004171241A (en) Illegality monitoring system and program
JP4159572B2 (en) Abnormality notification device and abnormality notification method
EP4266279A1 (en) Antitheft system for items in automatic checkouts and the like
KR102349793B1 (en) Commodity cost unmanned payment method
JP7200487B2 (en) Settlement system, settlement method and program
JP2016024601A (en) Information processing apparatus, information processing system, information processing method, commodity recommendation method, and program
EP4125019A1 (en) Information processing program, information processing method, and information processing apparatus
EP4125020A1 (en) Information processing program, information processing method, and information processing apparatus
JP2022148093A (en) Shoplifting prevention system using sales registration machine
DE102016120807B4 (en) Monitoring system for a telecommunications-based self-service checkout

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20240318

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR