EP4266279A1 - Antitheft system for items in automatic checkouts and the like - Google Patents
Antitheft system for items in automatic checkouts and the like Download PDFInfo
- Publication number
- EP4266279A1 EP4266279A1 EP23168428.3A EP23168428A EP4266279A1 EP 4266279 A1 EP4266279 A1 EP 4266279A1 EP 23168428 A EP23168428 A EP 23168428A EP 4266279 A1 EP4266279 A1 EP 4266279A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- items
- user
- monitoring device
- behavior
- automatic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012806 monitoring device Methods 0.000 claims abstract description 62
- 238000012545 processing Methods 0.000 claims abstract description 21
- 230000006399 behavior Effects 0.000 claims description 60
- 238000000034 method Methods 0.000 claims description 7
- 230000003287 optical effect Effects 0.000 claims description 5
- 230000001629 suppression Effects 0.000 claims description 4
- 238000007726 management method Methods 0.000 description 8
- 230000002265 prevention Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000012795 verification Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000003542 behavioural effect Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000011835 investigation Methods 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000012011 method of payment Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- ZRHANBBTXQZFSP-UHFFFAOYSA-M potassium;4-amino-3,5,6-trichloropyridine-2-carboxylate Chemical compound [K+].NC1=C(Cl)C(Cl)=NC(C([O-])=O)=C1Cl ZRHANBBTXQZFSP-UHFFFAOYSA-M 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
- G08B13/19613—Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
-
- A—HUMAN NECESSITIES
- A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
- A47F—SPECIAL FURNITURE, FITTINGS, OR ACCESSORIES FOR SHOPS, STOREHOUSES, BARS, RESTAURANTS OR THE LIKE; PAYING COUNTERS
- A47F9/00—Shop, bar, bank or like counters
- A47F9/02—Paying counters
- A47F9/04—Check-out counters, e.g. for self-service stores
Definitions
- the present invention relates to an antitheft system for items, i.e. products, of various types in automatic checkouts and the like.
- the system according to the present invention is particularly, although not exclusively, useful and practical in the prevention of theft of items, i.e. products, at the automatic checkouts that are nowadays commonly installed at hypermarkets and supermarkets in the large-scale retail trade.
- automated checkout means a self-service cash register where the user, i.e. the person buying the items, performs autonomously, i.e. alone, the reading or scanning of the bar codes of these items that he or she wishes to buy, pays the total price, for example with cash or an electronic payment card, and, finally, takes the associated purchase till receipt.
- an automatic checkout of known type comprises substantially a checkout management device 16 and a checkout counter 18.
- the buyer user 20 is positioned at the checkout counter 18.
- the checkout counter 18 comprises an upper surface supported by a supporting structure. On the upper surface of the checkout counter 18, three main regions 18a, 18b and 18c can be identified, which are arranged in succession along a direction of egress of the items purchased by the buyer user 20 at the commercial business or sales point.
- the three regions 18a, 18b and 18c comprise a first region 18a for setting down the items to be purchased, a second region 18b for registering the items to be purchased, and a third region 18c for bagging the registered, and therefore purchased, items.
- the first region 18a is commonly constituted by a set-down surface, or surface portion, on which the buyer user 20 places the items to be purchased.
- the second region 18b where the buyer user 20 is positioned, comprises means for product identification 19, preferably an optical barcode reader, with which the buyer user 20 registers the items to be purchased, preferably by scanning the corresponding bar codes.
- the third region 18c is commonly constituted by a surface or surface portion for bagging, on which the buyer user 20 collects the items that he has purchased.
- the checkout management device 16 is functionally connected to the means for product identification 19 comprised in the second region 18b of the checkout counter 18 of the automatic checkout.
- the checkout management device 16 is configured to register, for example locally, information relating to the items scanned and registered during the purchase operations, in particular every purchase operation performed at the automatic checkout and, as a consequence, every item comprised in every purchase operation.
- the checkout management device 16 is further configured to check the payment of the total price at the conclusion of every purchase operation.
- the checkout management device 16 can be configured to record the date and time of the operation, the name and address of the business, the total number of the items, the total price of the items, information relating to the method of payment (cash, electronic payment card etc.), and the like.
- antitheft systems for items are also known which are provided with electronic devices, in particular sensor devices, for example weight sensors, installed in, or proximate to, the automatic checkouts and the like, which are configured to verify that the items collected by the buyer user 20 in the third region 18c of the checkout counter 18 of the automatic checkout correspond to the items registered by that buyer user 20 in the second region 18b of the checkout counter 18 of the automatic checkout.
- electronic devices in particular sensor devices, for example weight sensors, installed in, or proximate to, the automatic checkouts and the like, which are configured to verify that the items collected by the buyer user 20 in the third region 18c of the checkout counter 18 of the automatic checkout correspond to the items registered by that buyer user 20 in the second region 18b of the checkout counter 18 of the automatic checkout.
- the lack of correspondence resulting from the above verification implies that the buyer user 20 has engaged in a fraudulent behavior, i.e. has attempted to steal something, in that he has placed at least one item in the third region 18c of the checkout counter 18 of the automatic checkout without this at least one item being registered in the second region 18b of the checkout counter 18 of the automatic checkout.
- This lack of correspondence resulting from the above verification can block the operation of the automatic checkout or it can automatically generate an alarm signal, for example visual and/or aural, which enables the security personnel to intervene promptly.
- the ill-intentioned buyer user 20 can position him- or herself with respect to the video surveillance system so that the video surveillance system cannot clearly view the operations executed at the checkout counter 18 of the automatic checkout, so as to conceal an attempted theft from the security personnel.
- the ill-intentioned buyer user 20 can interfere with the operation of the sensor devices of the antitheft system so that the antitheft system detects no lack of correspondence between the items collected in the third region 18c of the checkout counter 18 of the automatic checkout and the items registered in the second region 18b of the checkout counter 18 of the automatic checkout.
- fraudulent behavior of the buyer user 20 means an action to move an article from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18, with that item passing through the second region 18b of the checkout counter 18 without being scanned and consequently without being registered. Therefore, a fraudulent behavior is an attempted theft.
- correct behavior of the buyer user 20 means an action to move an article from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18, with that item passing through the second region 18b of the checkout counter 18 and being scanned, by way of the means for product identification 19, and consequently being registered.
- the aim of the present invention is to overcome the limitations of the known art described above, by devising an antitheft system for items in automatic checkouts and the like that makes it possible to obtain better effects than those that can be obtained with conventional solutions and/or similar effects at lower cost and with higher performance levels.
- an object of the present invention is to conceive of an antitheft system for items in automatic checkouts and the like that makes it possible to perform behavioral analysis of the operations performed by the buyer user at the automatic checkout, by automating the collection of possible fraudulent behaviors, i.e. potential attempted thefts, for a prompt and precise investigation by security personnel.
- Another object of the present invention is to devise an antitheft system for items in automatic checkouts and the like that makes it possible to support security personnel in their decisions, by reporting in real time possible fraudulent behaviors, i.e. potential attempted thefts, and by providing an immediate or retrospective verification to the security personnel.
- Another object of the present invention is to conceive of an antitheft system for items in automatic checkouts and the like that makes it possible to prevent, or at least minimize, the possibility of circumvention or deceit by a buyer user.
- Another object of the present invention is to devise an antitheft system for items in automatic checkouts and the like that makes it possible to provide data to support theft prevention strategies.
- Another object of the present invention is to provide an antitheft system for items in automatic checkouts and the like that is highly reliable, easily and practically implemented, and economically competitive when compared to the known art.
- an antitheft system for items in automatic checkouts and the like characterized in that it comprises a monitoring device and at least one vision device, said monitoring device being functionally connected to said vision device, said vision device being associable with a respective checkout counter of an automatic checkout where a user is operating and being orientable toward said checkout counter so as to view both said checkout counter and said user;
- the antitheft system for items in automatic checkouts and the like comprises substantially a monitoring device 12 and at least one vision device 14.
- the monitoring device 12 is functionally connected to the vision device 14, and vice versa.
- the vision device 14 can be a video camera.
- an automatic checkout comprises substantially a checkout management device 16 and a checkout counter 18.
- the checkout counter 18 of the automatic checkout comprises a first region 18a, a second region 18b and a third region 18c.
- the vision device 14 of the system 10 according to the invention is associated or associable with a respective checkout counter 18 of an automatic checkout where a user 20, i.e. the buyer of the items or products, is operating.
- the vision device 14 is firmly oriented or orientable toward the checkout counter 18 of the automatic checkout, where the buyer user 20 sets down, scans, registers and collects the items during the purchase operation.
- the vision device 14 views both the buyer user 20 and the checkout counter 18, in particular the first region 18a, the second region 18b and the third region 18c.
- the vision device 14 also views the items being handled by the buyer user 20.
- the vision device 14 is firmly oriented looking downward from above, where the checkout counter 18 of the automatic checkout is located. Even more preferably, the longitudinal axis of the vision device 14 is perpendicular to the longitudinal axis of the checkout counter 18 of the automatic checkout.
- the vision device 14 is configured to acquire a context video that comprises images of the buyer user 20, of the items being handled by the buyer user 20, and of the checkout counter 18, in particular the first region 18a, the second region 18b and the third region 18c.
- the vision device 14 is configured to acquire the context video with a frame rate higher than eight frames per second, since a frame rate less than or equal to eight frames per second would not be optimal and could limit the efficacy of some of the processing and/or analysis operations described below.
- the vision device 14 is further configured to send the context video, thus acquired, to the monitoring device 12.
- the monitoring device 12 of the system 10 comprises substantially an electronic control unit 30, an image processing module 32 and a behavior analysis module 34.
- the monitoring device 12 further comprises a memory unit 38.
- the electronic control unit 30 is the main functional element of the monitoring device 12, and for this reason it is functionally connected with the other elements comprised in the monitoring device 12.
- the electronic control unit 30 of the monitoring device 12 is provided with suitable capacity for processing and for interfacing with the other elements of the monitoring device 12, and it is configured to command, control and coordinate the operation of the elements of the monitoring device 12 with which it is functionally connected.
- the monitoring device 12 is configured to receive the context video acquired by and originating from the vision device 14.
- the memory unit 38 of the monitoring device 12 is configured to record the context video acquired by and originating from the vision device 14.
- the image processing module 32 of the monitoring device 12 is configured to extract information relating to the behavior (meaning movements) of the buyer user 20 from the context video, which was previously acquired by the vision device 14, identifying movements of items from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18.
- the image processing module 32 is configured as above by drawing on suitable artificial vision techniques.
- the image processing module 32 is further configured to segment the context video prior to extracting the information relating to the behavior of the buyer user 20.
- the memory unit 38 of the monitoring device 12 is further configured to record the information relating to the behavior (meaning movements) of the buyer user 20 which is extracted by the image processing module 32.
- the image processing module 32 of the monitoring device 12 is configured to extract the information relating to the behavior of the buyer user 20 by applying the artificial vision technique known as Optical Flow.
- Optical Flow estimates the movement in a video, in this case the context video, assigning each pixel of the current frame a movement vector that points toward the position of that pixel in a subsequent frame.
- Optical Flow is particularly useful for identifying the direction of travel of an item that passes through the second region 18b of the checkout counter 18, the region that comprises the means for product identification 19.
- the direction of travel is essential for distinguishing the "empty" hand (not holding an item) of the buyer user 20, moving from the third region 18c of the checkout counter 18 to the first region 18a of the checkout counter 18, from the "full" hand (holding an item) of the same buyer user 20, which by contrast moves from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18.
- the image processing module 32 of the monitoring device 12 is configured to extract the information relating to the behavior of the buyer user 20 by applying the artificial vision technique known as Background Suppression.
- Background Suppression detects moving objects in a video, in this case the context video, acquired by a static device, in this case the vision device 14.
- the logic behind this technique is to detect moving objects from differences between the current frame and a reference frame, called the "background image”.
- the image processing module 32 of the monitoring device 12 is further configured to extract information relating to the items moved by the buyer user 20 from the context video, which was previously acquired by the vision device 14, identifying specific predefined types of items, for example items of high economic worth.
- the memory unit 38 of the monitoring device 12 is further configured to record information relating to the item moved by the buyer user 20 which is extracted by the image processing module 32 of the monitoring device 12.
- the behavior analysis module 34 of the monitoring device 12 is configured to identify a type of behavior of the buyer user 20, distinguishing between correct behavior (such as for example shown in Figure 3A ) or fraudulent behavior (such as for example shown in Figure 3B ), on the basis of the information relating to the behavior (meaning movements) of the buyer user 20, previously extracted by the image processing module 32.
- the behavior analysis module 34 is configured as above by drawing on suitable artificial intelligence techniques (machine learning, deep learning, statistical models and mathematical models).
- the behavior analysis module 34 implements a Convolutional Neural Network (CNN).
- the memory unit 38 of the monitoring device 12 is further configured to record the type of behavior, alternatively correct or fraudulent, of the buyer user 20, identified by the behavior analysis module 34 of the monitoring device 12.
- the monitoring device 12 further comprises a comparison module 36.
- the monitoring device 12 is functionally connected to the checkout management device 16, and vice versa.
- the comparison module 36 is configured to interface with the checkout management device 16 of the automatic checkout, and to acquire information relating to the items scanned and registered during the purchase operation.
- the comparison module 36 is further configured to correlate the information relating to the behavior (meaning movements) of the buyer user 20 with the information relating to the items scanned and registered in the automatic checkout during the purchase operation, identifying any discrepancy between the movements of items from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18 (and/or their number) and the items scanned and registered in the automatic checkout during the purchase operation (and/or their number).
- the memory unit 38 of the monitoring device 12 is further configured to record a possible discrepancy between the movements of items from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18 (and/or their number) and the items scanned and registered during the purchase operation (and/or their number), identified by the comparison module 36 of the monitoring device 12.
- the antitheft system 10 for items in automatic checkouts and the like further comprises a user interface device 24, 26 which is operated by security personnel.
- the monitoring device 12 is functionally connected to the user interface device 24, 26, and vice versa, for example by way of a telematic communication network 22.
- the user interface device 24, 26 can be a personal computer 24, of the desktop or laptop type, or a mobile device 26, of the smartphone or tablet type.
- the monitoring device 12 is configured to send an indicator of the type of behavior of the buyer user 20, either correct or fraudulent, identified by the behavior analysis module 34 of the monitoring device 12, to the user interface device 24, 26.
- the user interface device 24, 26 is configured to graphically display this indicator of the type of behavior of the buyer user 20, either correct (such as for example shown in Figure 4A ) or fraudulent (such as for example shown in Figure 4B ).
- the monitoring device 12 is configured to send an alarm signal to the user interface device 24, 26 when the behavior analysis module 34 of the monitoring device 12 identifies a fraudulent behavior of the buyer user 20.
- the user interface device 24, 26 is configured to graphically display this alarm signal.
- the monitoring device 12 is configured to send an alarm signal to the user interface device 24, 26 when the comparison module 36 of the monitoring device 12 identifies a discrepancy between the movements of items from the first region 18a of the checkout counter 18 to the third region 18c of the checkout counter 18 (and/or their number) and the items scanned and registered in the automatic checkout during the purchase operation (and/or their number).
- the user interface device 24, 26 is configured to graphically display this alarm signal.
- the graphic indicators described above, displayed by the user interface device 24, 26, are accompanied by respective acoustic alerts.
- the user interface device 24, 26 is further configured to display the context video acquired by the vision device 14 and originating from the monitoring device 12. In an embodiment, the user interface device 24, 26 is further configured to graphically display, superimposed on the context video, at least one orientation guide of the vision device 14.
- the user interface device 24, 26 is further configured to graphically display one or more rapid training buttons which are operated by security personnel and are configured to confirm or review the work of the behavior analysis module 34 of the monitoring device 12, and therefore train said behavior analysis module.
- the user interface device 24, 26 is further configured to graphically display statistical information of various kinds extracted from a processing of information and data recorded in the memory unit 38 of the monitoring device 12. For example, this statistical information can be displayed by the user interface device 24, 26 in the form of summary tables, detailed tables, diagrams, graphs etc.
- the present invention fully achieves the set aim and objects.
- the antitheft system for items in automatic checkouts and the like thus conceived makes it possible to overcome the qualitative limitations of the known art, in that it makes it possible to obtain better effects than those that can be obtained with conventional solutions and/or similar effects at lower cost and with higher performance levels.
- An advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to perform behavioral analysis of the operations performed by the buyer user at the automatic checkout, by automating the collection of possible fraudulent behaviors, i.e. potential attempted thefts, for a prompt and precise investigation by security personnel.
- Another advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to support security personnel in their decisions, by reporting in real time possible fraudulent behaviors, i.e. potential attempted thefts, and by providing an immediate or retrospective verification to the security personnel.
- Another advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to prevent, or at least minimize, the possibility of circumvention or deceit by a buyer user.
- Another advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to provide data to support theft prevention strategies.
- the antitheft system for items in automatic checkouts and the like has been conceived in particular for the prevention of theft of items, i.e. products, at the automatic checkouts that are nowadays commonly installed at hypermarkets and supermarkets in the large-scale retail trade, it can also be used, more generally, for the prevention of theft of items, i.e. products, at automatic checkouts installed at a commercial business or sales point of any kind and size.
Abstract
the vision device (14) is configured to acquire a context video comprising images of the checkout counter (18), of the user (20) and of items handled by the user (20);
the monitoring device (12) comprises:
- an image processing module (32) which is configured to extract information relating to the behavior of the user (20) from the context video, identifying movements of items from a first region (18a) of the checkout counter (18) to a third region (18c) of the checkout counter (18);
- a behavior analysis module (34) which is configured to identify a type of behavior of the user (20), distinguishing between correct behavior or fraudulent behavior, on the basis of the information relating to the behavior of the user (20).
Description
- The present invention relates to an antitheft system for items, i.e. products, of various types in automatic checkouts and the like. The system according to the present invention is particularly, although not exclusively, useful and practical in the prevention of theft of items, i.e. products, at the automatic checkouts that are nowadays commonly installed at hypermarkets and supermarkets in the large-scale retail trade.
- Note that the term "automatic checkout" means a self-service cash register where the user, i.e. the person buying the items, performs autonomously, i.e. alone, the reading or scanning of the bar codes of these items that he or she wishes to buy, pays the total price, for example with cash or an electronic payment card, and, finally, takes the associated purchase till receipt.
- With reference to a portion of
Figure 1 , an automatic checkout of known type comprises substantially acheckout management device 16 and acheckout counter 18. Thebuyer user 20 is positioned at thecheckout counter 18. - The
checkout counter 18 comprises an upper surface supported by a supporting structure. On the upper surface of thecheckout counter 18, threemain regions buyer user 20 at the commercial business or sales point. - In particular, the three
regions first region 18a for setting down the items to be purchased, asecond region 18b for registering the items to be purchased, and athird region 18c for bagging the registered, and therefore purchased, items. - The
first region 18a is commonly constituted by a set-down surface, or surface portion, on which thebuyer user 20 places the items to be purchased. Then, i.e. downstream of thefirst region 18a, thesecond region 18b, where thebuyer user 20 is positioned, comprises means forproduct identification 19, preferably an optical barcode reader, with which thebuyer user 20 registers the items to be purchased, preferably by scanning the corresponding bar codes. Finally, i.e. downstream of thesecond region 18b, thethird region 18c is commonly constituted by a surface or surface portion for bagging, on which thebuyer user 20 collects the items that he has purchased. - The
checkout management device 16 is functionally connected to the means forproduct identification 19 comprised in thesecond region 18b of thecheckout counter 18 of the automatic checkout. Thecheckout management device 16 is configured to register, for example locally, information relating to the items scanned and registered during the purchase operations, in particular every purchase operation performed at the automatic checkout and, as a consequence, every item comprised in every purchase operation. Thecheckout management device 16 is further configured to check the payment of the total price at the conclusion of every purchase operation. - For example, for every purchase operation, in addition to the list of the items and of the corresponding prices, the
checkout management device 16 can be configured to record the date and time of the operation, the name and address of the business, the total number of the items, the total price of the items, information relating to the method of payment (cash, electronic payment card etc.), and the like. - Nowadays video surveillance systems are known which are installed proximate to automatic checkouts and the like, and from which security personnel can monitor and be vigilant for behaviors (meaning movements) of the
buyer user 20 who is operating on the automatic checkout, intervening promptly if a fraudulent behavior, i.e. an attempted theft, is detected. - Currently antitheft systems for items are also known which are provided with electronic devices, in particular sensor devices, for example weight sensors, installed in, or proximate to, the automatic checkouts and the like, which are configured to verify that the items collected by the
buyer user 20 in thethird region 18c of thecheckout counter 18 of the automatic checkout correspond to the items registered by thatbuyer user 20 in thesecond region 18b of thecheckout counter 18 of the automatic checkout. - The lack of correspondence resulting from the above verification implies that the
buyer user 20 has engaged in a fraudulent behavior, i.e. has attempted to steal something, in that he has placed at least one item in thethird region 18c of thecheckout counter 18 of the automatic checkout without this at least one item being registered in thesecond region 18b of thecheckout counter 18 of the automatic checkout. - This lack of correspondence resulting from the above verification can block the operation of the automatic checkout or it can automatically generate an alarm signal, for example visual and/or aural, which enables the security personnel to intervene promptly.
- These antitheft systems for items in automatic checkouts and the like of known type are not devoid of drawbacks, among which is the fact that they can be circumvented or deceived by a
buyer user 20 who wishes to exhibit a fraudulent behavior. - For example, the ill-
intentioned buyer user 20 can position him- or herself with respect to the video surveillance system so that the video surveillance system cannot clearly view the operations executed at thecheckout counter 18 of the automatic checkout, so as to conceal an attempted theft from the security personnel. - Also for example, the ill-
intentioned buyer user 20 can interfere with the operation of the sensor devices of the antitheft system so that the antitheft system detects no lack of correspondence between the items collected in thethird region 18c of thecheckout counter 18 of the automatic checkout and the items registered in thesecond region 18b of thecheckout counter 18 of the automatic checkout. - Note that the term "fraudulent behavior" of the
buyer user 20 means an action to move an article from thefirst region 18a of thecheckout counter 18 to thethird region 18c of thecheckout counter 18, with that item passing through thesecond region 18b of thecheckout counter 18 without being scanned and consequently without being registered. Therefore, a fraudulent behavior is an attempted theft. - Note that the term "correct behavior" of the
buyer user 20 means an action to move an article from thefirst region 18a of thecheckout counter 18 to thethird region 18c of thecheckout counter 18, with that item passing through thesecond region 18b of thecheckout counter 18 and being scanned, by way of the means forproduct identification 19, and consequently being registered. - The aim of the present invention is to overcome the limitations of the known art described above, by devising an antitheft system for items in automatic checkouts and the like that makes it possible to obtain better effects than those that can be obtained with conventional solutions and/or similar effects at lower cost and with higher performance levels.
- Within this aim, an object of the present invention is to conceive of an antitheft system for items in automatic checkouts and the like that makes it possible to perform behavioral analysis of the operations performed by the buyer user at the automatic checkout, by automating the collection of possible fraudulent behaviors, i.e. potential attempted thefts, for a prompt and precise investigation by security personnel.
- Another object of the present invention is to devise an antitheft system for items in automatic checkouts and the like that makes it possible to support security personnel in their decisions, by reporting in real time possible fraudulent behaviors, i.e. potential attempted thefts, and by providing an immediate or retrospective verification to the security personnel.
- Another object of the present invention is to conceive of an antitheft system for items in automatic checkouts and the like that makes it possible to prevent, or at least minimize, the possibility of circumvention or deceit by a buyer user.
- Another object of the present invention is to devise an antitheft system for items in automatic checkouts and the like that makes it possible to provide data to support theft prevention strategies.
- Another object of the present invention is to provide an antitheft system for items in automatic checkouts and the like that is highly reliable, easily and practically implemented, and economically competitive when compared to the known art.
- This aim and these and other objects which will become more apparent hereinafter are achieved by an antitheft system for items in automatic checkouts and the like, characterized in that it comprises a monitoring device and at least one vision device, said monitoring device being functionally connected to said vision device, said vision device being associable with a respective checkout counter of an automatic checkout where a user is operating and being orientable toward said checkout counter so as to view both said checkout counter and said user;
- said vision device being configured to acquire a context video comprising images of said checkout counter, of said user and of items handled by said user;
- said monitoring device comprising:
- an image processing module which is configured to extract information relating to the behavior of said user from said context video, identifying movements of items from a first region of said checkout counter to a third region of said checkout counter; and
- a behavior analysis module which is configured to identify a type of behavior of said user, distinguishing between correct behavior or fraudulent behavior, on the basis of said information relating to the behavior of said user.
- Further characteristics and advantages of the present invention will become apparent from the description of a preferred, but not exclusive, embodiment of the antitheft system for items in automatic checkouts and the like according to the invention, illustrated by way of non-limiting example with the aid of the accompanying drawings wherein:
-
Figure 1 is a block diagram schematically showing an embodiment of the antitheft system for items in automatic checkouts and the like according to the present invention; -
Figure 2 is a block diagram schematically showing an embodiment of the monitoring device of the antitheft system for items in automatic checkouts and the like according to the present invention; -
Figures 3A and 3B respectively show an example of correct behavior (meaning movements) of the buyer user and an example of fraudulent behavior (meaning movements) of the buyer user, at an automatic checkout; -
Figures 4A and 4B respectively show a graphic indicator of correct behavior of the buyer user and a graphic indicator of fraudulent behavior of the buyer user. - With reference to
Figure 1 , the antitheft system for items in automatic checkouts and the like according to the present invention, generally designated by thereference numeral 10, comprises substantially amonitoring device 12 and at least onevision device 14. Themonitoring device 12 is functionally connected to thevision device 14, and vice versa. Thevision device 14 can be a video camera. - As mentioned, an automatic checkout comprises substantially a
checkout management device 16 and acheckout counter 18. Thecheckout counter 18 of the automatic checkout comprises afirst region 18a, asecond region 18b and athird region 18c. - The
vision device 14 of thesystem 10 according to the invention is associated or associable with arespective checkout counter 18 of an automatic checkout where auser 20, i.e. the buyer of the items or products, is operating. - The
vision device 14 is firmly oriented or orientable toward thecheckout counter 18 of the automatic checkout, where thebuyer user 20 sets down, scans, registers and collects the items during the purchase operation. In other words, thevision device 14 views both thebuyer user 20 and thecheckout counter 18, in particular thefirst region 18a, thesecond region 18b and thethird region 18c. As a consequence, thevision device 14 also views the items being handled by thebuyer user 20. - Preferably, the
vision device 14 is firmly oriented looking downward from above, where thecheckout counter 18 of the automatic checkout is located. Even more preferably, the longitudinal axis of thevision device 14 is perpendicular to the longitudinal axis of thecheckout counter 18 of the automatic checkout. - The
vision device 14 is configured to acquire a context video that comprises images of thebuyer user 20, of the items being handled by thebuyer user 20, and of thecheckout counter 18, in particular thefirst region 18a, thesecond region 18b and thethird region 18c. Advantageously, thevision device 14 is configured to acquire the context video with a frame rate higher than eight frames per second, since a frame rate less than or equal to eight frames per second would not be optimal and could limit the efficacy of some of the processing and/or analysis operations described below. - The
vision device 14 is further configured to send the context video, thus acquired, to themonitoring device 12. - With reference to
Figure 2 , themonitoring device 12 of thesystem 10 according to the invention comprises substantially anelectronic control unit 30, animage processing module 32 and abehavior analysis module 34. Advantageously, themonitoring device 12 further comprises amemory unit 38. - The
electronic control unit 30 is the main functional element of themonitoring device 12, and for this reason it is functionally connected with the other elements comprised in themonitoring device 12. - The
electronic control unit 30 of themonitoring device 12 is provided with suitable capacity for processing and for interfacing with the other elements of themonitoring device 12, and it is configured to command, control and coordinate the operation of the elements of themonitoring device 12 with which it is functionally connected. - The
monitoring device 12 is configured to receive the context video acquired by and originating from thevision device 14. - Preferably, the
memory unit 38 of themonitoring device 12 is configured to record the context video acquired by and originating from thevision device 14. - The
image processing module 32 of themonitoring device 12 is configured to extract information relating to the behavior (meaning movements) of thebuyer user 20 from the context video, which was previously acquired by thevision device 14, identifying movements of items from thefirst region 18a of thecheckout counter 18 to thethird region 18c of thecheckout counter 18. In particular, theimage processing module 32 is configured as above by drawing on suitable artificial vision techniques. Preferably, theimage processing module 32 is further configured to segment the context video prior to extracting the information relating to the behavior of thebuyer user 20. - Preferably, the
memory unit 38 of themonitoring device 12 is further configured to record the information relating to the behavior (meaning movements) of thebuyer user 20 which is extracted by theimage processing module 32. - In an embodiment, the
image processing module 32 of themonitoring device 12 is configured to extract the information relating to the behavior of thebuyer user 20 by applying the artificial vision technique known as Optical Flow. - Optical Flow estimates the movement in a video, in this case the context video, assigning each pixel of the current frame a movement vector that points toward the position of that pixel in a subsequent frame.
- For example, Optical Flow is particularly useful for identifying the direction of travel of an item that passes through the
second region 18b of thecheckout counter 18, the region that comprises the means forproduct identification 19. For example, the direction of travel is essential for distinguishing the "empty" hand (not holding an item) of thebuyer user 20, moving from thethird region 18c of thecheckout counter 18 to thefirst region 18a of thecheckout counter 18, from the "full" hand (holding an item) of thesame buyer user 20, which by contrast moves from thefirst region 18a of thecheckout counter 18 to thethird region 18c of thecheckout counter 18. - In an embodiment, the
image processing module 32 of themonitoring device 12 is configured to extract the information relating to the behavior of thebuyer user 20 by applying the artificial vision technique known as Background Suppression. - Background Suppression detects moving objects in a video, in this case the context video, acquired by a static device, in this case the
vision device 14. The logic behind this technique is to detect moving objects from differences between the current frame and a reference frame, called the "background image". - For Background Suppression to be effective, some conditions must be met, which are a constant background, in this case the
checkout counter 18 of the automatic checkout, and lighting that is substantially uniform over time. - Advantageously, the
image processing module 32 of themonitoring device 12 is further configured to extract information relating to the items moved by thebuyer user 20 from the context video, which was previously acquired by thevision device 14, identifying specific predefined types of items, for example items of high economic worth. - Preferably, the
memory unit 38 of themonitoring device 12 is further configured to record information relating to the item moved by thebuyer user 20 which is extracted by theimage processing module 32 of themonitoring device 12. - The
behavior analysis module 34 of themonitoring device 12 is configured to identify a type of behavior of thebuyer user 20, distinguishing between correct behavior (such as for example shown inFigure 3A ) or fraudulent behavior (such as for example shown inFigure 3B ), on the basis of the information relating to the behavior (meaning movements) of thebuyer user 20, previously extracted by theimage processing module 32. In particular, thebehavior analysis module 34 is configured as above by drawing on suitable artificial intelligence techniques (machine learning, deep learning, statistical models and mathematical models). Preferably, thebehavior analysis module 34 implements a Convolutional Neural Network (CNN). - Preferably, the
memory unit 38 of themonitoring device 12 is further configured to record the type of behavior, alternatively correct or fraudulent, of thebuyer user 20, identified by thebehavior analysis module 34 of themonitoring device 12. - Advantageously, the
monitoring device 12 further comprises acomparison module 36. In this case, themonitoring device 12 is functionally connected to thecheckout management device 16, and vice versa. - The
comparison module 36 is configured to interface with thecheckout management device 16 of the automatic checkout, and to acquire information relating to the items scanned and registered during the purchase operation. Thecomparison module 36 is further configured to correlate the information relating to the behavior (meaning movements) of thebuyer user 20 with the information relating to the items scanned and registered in the automatic checkout during the purchase operation, identifying any discrepancy between the movements of items from thefirst region 18a of thecheckout counter 18 to thethird region 18c of the checkout counter 18 (and/or their number) and the items scanned and registered in the automatic checkout during the purchase operation (and/or their number). - Preferably, the
memory unit 38 of themonitoring device 12 is further configured to record a possible discrepancy between the movements of items from thefirst region 18a of thecheckout counter 18 to thethird region 18c of the checkout counter 18 (and/or their number) and the items scanned and registered during the purchase operation (and/or their number), identified by thecomparison module 36 of themonitoring device 12. - Advantageously, the
antitheft system 10 for items in automatic checkouts and the like according to the present invention further comprises auser interface device monitoring device 12 is functionally connected to theuser interface device telematic communication network 22. Theuser interface device personal computer 24, of the desktop or laptop type, or amobile device 26, of the smartphone or tablet type. - Advantageously, the
monitoring device 12 is configured to send an indicator of the type of behavior of thebuyer user 20, either correct or fraudulent, identified by thebehavior analysis module 34 of themonitoring device 12, to theuser interface device user interface device buyer user 20, either correct (such as for example shown inFigure 4A ) or fraudulent (such as for example shown inFigure 4B ). - Advantageously, the
monitoring device 12 is configured to send an alarm signal to theuser interface device behavior analysis module 34 of themonitoring device 12 identifies a fraudulent behavior of thebuyer user 20. As a consequence, theuser interface device - Advantageously, the
monitoring device 12 is configured to send an alarm signal to theuser interface device comparison module 36 of themonitoring device 12 identifies a discrepancy between the movements of items from thefirst region 18a of thecheckout counter 18 to thethird region 18c of the checkout counter 18 (and/or their number) and the items scanned and registered in the automatic checkout during the purchase operation (and/or their number). As a consequence, theuser interface device - Optionally, the graphic indicators described above, displayed by the
user interface device - In a preferred embodiment, the
user interface device vision device 14 and originating from themonitoring device 12. In an embodiment, theuser interface device vision device 14. - In an embodiment, the
user interface device behavior analysis module 34 of themonitoring device 12, and therefore train said behavior analysis module. - In an embodiment, the
user interface device memory unit 38 of themonitoring device 12. For example, this statistical information can be displayed by theuser interface device - In practice it has been found that the present invention fully achieves the set aim and objects. In particular, it has been seen that the antitheft system for items in automatic checkouts and the like thus conceived makes it possible to overcome the qualitative limitations of the known art, in that it makes it possible to obtain better effects than those that can be obtained with conventional solutions and/or similar effects at lower cost and with higher performance levels.
- An advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to perform behavioral analysis of the operations performed by the buyer user at the automatic checkout, by automating the collection of possible fraudulent behaviors, i.e. potential attempted thefts, for a prompt and precise investigation by security personnel.
- Another advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to support security personnel in their decisions, by reporting in real time possible fraudulent behaviors, i.e. potential attempted thefts, and by providing an immediate or retrospective verification to the security personnel.
- Another advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to prevent, or at least minimize, the possibility of circumvention or deceit by a buyer user.
- Another advantage of the antitheft system for items in automatic checkouts and the like according to the present invention consists in that it makes it possible to provide data to support theft prevention strategies.
- Although the antitheft system for items in automatic checkouts and the like according to the invention has been conceived in particular for the prevention of theft of items, i.e. products, at the automatic checkouts that are nowadays commonly installed at hypermarkets and supermarkets in the large-scale retail trade, it can also be used, more generally, for the prevention of theft of items, i.e. products, at automatic checkouts installed at a commercial business or sales point of any kind and size.
- The invention thus conceived is susceptible of numerous modifications and variations, all of which are within the scope of the appended claims. Moreover, all the details may be substituted by other, technically equivalent elements.
- In practice the materials employed, provided they are compatible with the specific use, and the contingent dimensions and shapes, may be any according to requirements and to the state of the art.
- In conclusion, the scope of protection of the claims shall not be limited by the explanations or by the preferred embodiments illustrated in the description by way of examples, but rather the claims shall comprise all the patentable characteristics of novelty that reside in the present invention, including all the characteristics that would be considered as equivalent by the person skilled in the art.
- The disclosures in
Italian Patent Application No. 102022000007865 - Where the technical features mentioned in any claim are followed by reference numerals and/or signs, those reference numerals and/or signs have been included for the sole purpose of increasing the intelligibility of the claims and accordingly, such reference numerals and/or signs do not have any limiting effect on the interpretation of each element identified by way of example by such reference numerals and/or signs.
Claims (15)
- An antitheft system (10) for items in automatic checkouts and the like, characterized in that it comprises a monitoring device (12) and at least one vision device (14), said monitoring device (12) being functionally connected to said vision device (14), said vision device (12) being associable with a respective checkout counter (18) of an automatic checkout where a user (20) is operating and being orientable toward said checkout counter (18) so as to view both said checkout counter (18) and said user (20);said vision device (14) being configured to acquire a context video comprising images of said checkout counter (18), of said user (20) and of items handled by said user (20);said monitoring device (12) comprising:- an image processing module (32) which is configured to extract information relating to the behavior of said user (20) from said context video, identifying movements of items from a first region (18a) of said checkout counter (18) to a third region (18c) of said checkout counter (18); and- a behavior analysis module (34) which is configured to identify a type of behavior of said user (20), distinguishing between correct behavior or fraudulent behavior, on the basis of said information relating to the behavior of said user (20).
- The antitheft system (10) for items in automatic checkouts and the like according to claim 1, characterized in that said monitoring device (12) further comprises a memory unit (38) which is configured to record:- said context video, acquired by and originating from said vision device (14);- said information relating to the behavior of said user (20), extracted from said image processing module (32) of said monitoring device (12); and- said type of behavior of said buyer user (20), identified by said behavior analysis module (34) of said monitoring device (12).
- The antitheft system (10) for items in automatic checkouts and the like according to claim 1 or 2, characterized in that said monitoring device (12) is functionally connected to a checkout management device (16) of said automatic checkout, and in that said monitoring device (12) further comprises a comparison module (36) which is configured to correlate said information relating to the behavior of said user (20) with information relating to said items scanned and registered in said automatic checkout, identifying any discrepancy between said movements of items from said first region (18) of said checkout counter (18) to said third region (18c) of said checkout counter (18) and said items scanned and registered in said automatic checkout.
- The antitheft system (10) for items in automatic checkouts and the like according to claim 3, characterized in that said memory unit (38) of said monitoring device (12) is further configured to register said possible discrepancy, identified by said comparison module (36) of said monitoring device (12), between said movements of items from said first region (18a) of said checkout counter (18) to said third region (18c) of said checkout counter (18) and said items scanned and registered in said automatic checkout.
- The antitheft system (10) for items in automatic checkouts and the like according to any one of the preceding claims, characterized in that said image processing module (32) of said monitoring device (12) is configured to extract said information relating to the behavior of said user (20) by applying the artificial vision technique known as Optical Flow.
- The antitheft system (10) for items in automatic checkouts and the like according to any one of the preceding claims, characterized in that said image processing module (32) of said monitoring device (12) is configured to extract said information relating to the behavior of said user (20) by applying the artificial vision technique known as Background Suppression.
- The antitheft system (10) for items in automatic checkouts and the like according to any one of the preceding claims, characterized in that said image processing module (32) of said monitoring device (12) is further configured to extract information relating to said items moved by said buyer user (20) from the context video, identifying specific predefined types of items.
- The antitheft system (10) for items in automatic checkouts and the like according to any one of the preceding claims, characterized in that said vision device (14) is configured to acquire said context video with a frame rate higher than eight frames per second.
- The antitheft system (10) for items in automatic checkouts and the like according to any one of the preceding claims, characterized in that it further comprises a user interface device (24, 26), said monitoring device (12) being functionally connected to said user interface device (24, 26).
- The antitheft system (10) for items in automatic checkouts and the like according to claim 9, characterized in that said user interface device (24, 26) is configured to graphically display a signal relating to said type of behavior of said buyer user (20), identified by said behavior analysis module (34) of said monitoring device (12).
- The antitheft system (10) for items in automatic checkouts and the like according to claim 9 or 10, characterized in that said user interface device (24, 26) is configured to graphically display an alarm signal when said behavior analysis module (34) of said monitoring device (12) identifies a fraudulent behavior of said user (20).
- The antitheft system (10) for items in automatic checkouts and the like according to any one of claims 9 to 11, characterized in that said user interface device (24, 26) is configured to graphically display an alarm signal when said comparison module (36) of said monitoring device (12) identifies said possible discrepancy between said movements of items from said first region (18) of said checkout counter (18) to said third region (18c) of said checkout counter (18) and said items scanned and registered in said automatic checkout.
- The antitheft system (10) for items in automatic checkouts and the like according to any one of claims 9 to 12, characterized in that said user interface device (24, 26) is configured to display said context video acquired by said vision device (14) and originating from said monitoring device (12).
- The antitheft system (10) for items in automatic checkouts and the like according to claim 13, characterized in that said user interface device (24, 26) is further configured to graphically display, superimposed on said context video, at least one orientation guide of said vision device (14).
- The antitheft system (10) for items in automatic checkouts and the like according to any one of claims 9 to 14, characterized in that said user interface device (24, 26) is configured to graphically display statistical information of various kinds extracted from a processing of information and data recorded in said memory unit (38) of said monitoring device (12).
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
IT202200007865 | 2022-04-21 |
Publications (1)
Publication Number | Publication Date |
---|---|
EP4266279A1 true EP4266279A1 (en) | 2023-10-25 |
Family
ID=82608617
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP23168428.3A Pending EP4266279A1 (en) | 2022-04-21 | 2023-04-18 | Antitheft system for items in automatic checkouts and the like |
Country Status (1)
Country | Link |
---|---|
EP (1) | EP4266279A1 (en) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080226129A1 (en) * | 2007-03-12 | 2008-09-18 | Malay Kundu | Cart Inspection for Suspicious Items |
EP2704079A1 (en) * | 2012-08-31 | 2014-03-05 | NCR Corporation | Checkout security using video surveillance |
US9589433B1 (en) * | 2013-07-31 | 2017-03-07 | Jeff Thramann | Self-checkout anti-theft device |
-
2023
- 2023-04-18 EP EP23168428.3A patent/EP4266279A1/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080226129A1 (en) * | 2007-03-12 | 2008-09-18 | Malay Kundu | Cart Inspection for Suspicious Items |
EP2704079A1 (en) * | 2012-08-31 | 2014-03-05 | NCR Corporation | Checkout security using video surveillance |
US9589433B1 (en) * | 2013-07-31 | 2017-03-07 | Jeff Thramann | Self-checkout anti-theft device |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN104885130B (en) | Verification to the fraudulent activities at self checkout terminal | |
US8876001B2 (en) | Methods and apparatus for image recognition in checkout verification | |
US20220156686A1 (en) | Commodity monitoring device, commodity monitoring system, output destination device, commodity monitoring method, display method and program | |
US20060032914A1 (en) | System and method for notifying a cashier of the presence of an item in an obscured area of a shopping cart | |
CA3053106A1 (en) | Systems and methods for shopping in a physical store | |
CN212229682U (en) | Open type self-service sale system based on buyer position positioning | |
EP1354296A2 (en) | Monitoring responses to visual stimuli | |
US20130015242A1 (en) | Tri-Optic Scanner | |
CN110866429A (en) | Missed scanning identification method and device, self-service cash register terminal and system | |
WO1993017404A1 (en) | Security system | |
US10383461B2 (en) | System of control and identification of goods in a shop | |
US10372998B2 (en) | Object recognition for bottom of basket detection | |
JP2004171240A (en) | Illegality monitoring system and program | |
Connell et al. | Retail video analytics: an overview and survey | |
JP2016081364A (en) | Information notification program and information processing device | |
JP2004171241A (en) | Illegality monitoring system and program | |
JP4159572B2 (en) | Abnormality notification device and abnormality notification method | |
EP4266279A1 (en) | Antitheft system for items in automatic checkouts and the like | |
KR102349793B1 (en) | Commodity cost unmanned payment method | |
JP7200487B2 (en) | Settlement system, settlement method and program | |
JP2016024601A (en) | Information processing apparatus, information processing system, information processing method, commodity recommendation method, and program | |
EP4125019A1 (en) | Information processing program, information processing method, and information processing apparatus | |
EP4125020A1 (en) | Information processing program, information processing method, and information processing apparatus | |
JP2022148093A (en) | Shoplifting prevention system using sales registration machine | |
DE102016120807B4 (en) | Monitoring system for a telecommunications-based self-service checkout |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20240318 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR |