EP4070295A1 - Système et procédé de détection de fraude - Google Patents

Système et procédé de détection de fraude

Info

Publication number
EP4070295A1
EP4070295A1 EP20816473.1A EP20816473A EP4070295A1 EP 4070295 A1 EP4070295 A1 EP 4070295A1 EP 20816473 A EP20816473 A EP 20816473A EP 4070295 A1 EP4070295 A1 EP 4070295A1
Authority
EP
European Patent Office
Prior art keywords
article
weight
user
processing unit
item
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP20816473.1A
Other languages
German (de)
English (en)
French (fr)
Inventor
Dylan LETIERCE
Jonathan MALGOGNE
Christophe CHALOIN
Damien MANDRIOLI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Publication of EP4070295A1 publication Critical patent/EP4070295A1/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19617—Surveillance camera constructional details
    • G08B13/1963—Arrangements allowing camera rotation to change view, e.g. pivoting camera, pan-tilt and zoom [PTZ]
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/22—Electrical actuation
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/02—Mechanical actuation
    • G08B13/14—Mechanical actuation by lifting or attempted removal of hand-portable articles
    • G08B13/1472—Mechanical actuation by lifting or attempted removal of hand-portable articles with force or weight detection
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/02—Mechanical actuation
    • G08B13/14—Mechanical actuation by lifting or attempted removal of hand-portable articles
    • G08B13/1481—Mechanical actuation by lifting or attempted removal of hand-portable articles with optical detection

Definitions

  • the present invention relates to the field of fraud detection during the purchase of articles. It finds a particularly advantageous application in the field of large-scale distribution and trolleys, intelligent shopping baskets or payment devices.
  • An object of the present invention is therefore to provide a solution to at least some of these problems.
  • the present invention relates to a method for detecting fraud in the case of the purchase by at least one user of at least one article comprising at least: a.
  • a step of capturing a plurality of data from at least one sensor, and preferably from a plurality of sensors, performed by at least one user terminal comprising at least the following steps: i. Obtaining the identifier of the article by at least one identification device; ii. Determination by at least one optical device of at least one trajectory of the article manually moved by the user in a three-dimensional space, said three-dimensional space comprising at least:
  • An identification zone corresponding to a volume of three-dimensional space in which at least part of the article is placed by the user to achieve the obtaining of the identifier of the article
  • a processing step carried out by the computer processing unit, of the plurality of data comprising at least the following steps: i. Generating at least one behavior of said article from at least the trajectory of the article in three-dimensional space; ii. Comparison of the behavior of said article with a plurality of predetermined behavior models so as to identify a handling anomaly; vs. A step of determining a probability of fraud based on said comparison of behavior.
  • the present invention cleverly uses a plurality of sensors to cross a plurality of data so as to identify a fraud situation.
  • the proposed method makes it possible to identify a manipulation made by the user consisting in adding an article without first identifying and therefore counting it.
  • the item is not identified by the user until it reaches the entrance area and is placed in the container.
  • the present invention makes it possible to determine the behavior of an article so as to identify whether this behavior is consistent or not with a behavior considered to be standard, that is to say not fraudulent.
  • the present invention makes it possible to classify potentially fraudulent behavioral behaviors as long as it deviates beyond a predetermined threshold from one or more standard behavior models.
  • the present invention cleverly uses a plurality of predetermined behavior models comprising one or more standard behavior models.
  • the present invention makes it possible to detect a plurality of frauds during the purchase of an article in a store, for example, using automatic checkout systems, for example, or even so-called intelligent carts.
  • the present invention solves the majority, if not all, of fraud situations.
  • the present invention makes it possible to guide the customer during his purchasing process and to identify fraud or errors without the user automatically receiving notification. Given that the contents of the trolley are checked in quasi-real time, payment without going through a checkout or terminal and without directly checking the entire contents of a trolley is therefore possible thanks to the present invention.
  • the step of capturing a plurality of data comprises at least one measurement, by at least one measuring device, of the weight of the article, and a sending step by the user terminal to the computer processing unit of the measured weight of the article.
  • the processing step comprises, preferably before the step of generating the behavior of the article, at least the following steps: a. Identification in at least one database of the article from the identifier, the database comprising at least the identifier of the article associated with a predetermined weight of the article; b. Obtaining the predetermined weight of the item from the database: i. In the event that the predetermined weight is equal to zero or is not entered, the computer processing unit assigns the measured weight of the item as the predetermined weight associated with said identifier in the database; ii. In the case where the predetermined weight is different is different from zero and entered, the computer processing unit performs a comparison of the predetermined weight and the measured weight so as to identify a weight anomaly if the difference in weight is greater than a threshold predetermined.
  • the determination of a probability of fraud is carried out as a function of said comparison of the predetermined weight with the measured weight, this probability being non-zero if a weight anomaly has been identified.
  • the present invention can reduce, if not prevent, fraud.
  • the present invention also relates to a system for detecting at least one fraud in the case of a purchase of at least one article by a user in a store, comprising at least: a.
  • a user terminal comprising at least: i. An identification device configured to identify the article when a user passes the article in proximity, preferably within one meter, of the identification device; ii.
  • a measuring device configured to measure the weight of the article; iii.
  • An optical device configured to at least determine at least one trajectory of the article manually moved by the user in three-dimensional space;
  • a computer processing unit in communication with at least the user terminal, the computer processing unit being remote or not from the user terminal and being configured for: i. Generating at least one behavior of said article from at least the trajectory of the article in three-dimensional space; ii. Compare the behavior of said article with a plurality of predetermined behavior models so as to identify a handling anomaly;
  • the computer processing unit is further in communication with a database comprising the identifier of the article associated with a predetermined weight of the article.
  • the computer processing unit is further in communication with a data comparison module configured to compare the measured weight with the weight indicated in the database according to the identified item, the comparison module being configured. to identify a weighing anomaly.
  • the optical device is configured to collect a plurality of images in addition to the article, and the computer processing unit is further in communication with a module for analyzing the images collected by said optical device configured to identify a handling anomaly.
  • the computer processing unit is further configured for: a. Compare the predetermined weight of the item obtained from the database with the measured weight so as to identify a weight anomaly if the difference in weight is greater than a predetermined threshold; b. Determine a probability of fraud as a function of said comparison of weight, this probability being non-zero if a weight anomaly has been identified.
  • the computer processing unit is further configured to analyze the plurality of images collected so as to identify a handling anomaly.
  • the present invention also relates to a computer program product comprising instructions, which when carried out by at least one processor, executes at least the steps of the method according to the present invention.
  • FIG. 1 represents a fraud detection system according to an embodiment of the present invention.
  • FIG. 2 represents a diagram of the positioning of the identification device, of the optical device and of their observation zones according to an embodiment of the present invention.
  • FIG. 3 represents a trolley integrating at least part of the system according to one embodiment of the present invention.
  • Figure 4 shows a graphical interface of a portable analysis device according to one embodiment of the present invention.
  • FIG. 5 shows an algorithm for recording data and analyzing said recording according to an embodiment of the present invention.
  • the optical device is configured to allow depth to be taken into account in capturing three-dimensional images.
  • the optical device is configured to allow taking into account a so-called depth spatial dimension extending along an axis orthogonal to the two axes forming the plane of a diopter of the optical device.
  • the trajectory of the article in three-dimensional space comprises at least a plurality of points, each point of said plurality of points comprising at least three spatial coordinates, preferably in an orthonormal three-dimensional space.
  • the optical device is configured to allow the depth to be taken into account in determining said trajectory of the article.
  • the optical device is configured to allow taking into account a so-called depth spatial dimension extending along an axis orthogonal to the two axes forming the plane of a diopter of the optical device in determining said trajectory of the optical device. 'article.
  • the trajectory of the article in three-dimensional space comprises at least a plurality of points, each point of said plurality of points comprising at least three spatial coordinates, possibly each evolving at course of the trajectory, preferably in an orthogonal three-dimensional space.
  • the optical device comprises a stereoscopic optical device, preferably is a stereoscopic optical device.
  • the probability is non-zero if a handling anomaly is identified.
  • the predetermined weight of the article contained in the database comprises a range of weights, preferably a minimum predetermined weight and a maximum predetermined weight.
  • the step of determining the trajectory of the article in space three-dimensional comprises tracking the article in at least one area taken from at least the identification area, the entry area, at least one outer area, at least one inner area corresponding at least to the entry of at least one minus one container, the entry area separating the exterior area from the interior area.
  • the division of the space into several zones allows better tracking of the item and the functionalization of the space.
  • the determination of the trajectory of the article in three-dimensional space comprises at least the passages, and preferably only the passages, of the article from one area of the three-dimensional space to another area of the three-dimensional space. three-dimensional space.
  • the step of determining the trajectory of the article comprises at least determining the trajectory of an object other than the article moving in three-dimensional space, preferably said object being taken from: a hand, an arm, another article, a bag, an accessory worn by the user, a garment worn by the user.
  • the step of generating the behavior of the article includes mentioning any reconciliation beyond a predetermined threshold of said object with the article.
  • the behavior generated from said article comprises at least one chain of events detected by the plurality of sensors, these events being taken from at least: the identification of the article, the passage of a zone of space three-dimensional to another area of three-dimensional space, the measurement of the weight of the article, the approach of the article by another object.
  • the step of capturing the plurality of data includes collecting by the optical device a plurality of images of at least the item and at least one hand of the user carrying the item.
  • the processing step comprises a step of analyzing the plurality of images collected in order to record at least one two-dimensional representation of the article and to identify whether the user's hand is empty or full. .
  • the processing step comprises at least one comparison of an image of the article present in the database and of one or more images of the plurality of images collected so as to identify an anomaly between the image of the article from the database and the collected image (s) of the article.
  • the step of comparing an image of the article comprises at least one step of optical recognition of the article by the computer processing unit, preferably by a trained neural network.
  • the step of collecting a plurality of images comprises at least one step of recording by the optical device a video, advantageously temporally compressed, preferably from the plurality of images collected.
  • the step of recording the video comprises the overlay of a datum collected by at least one sensor at the time of collection of said datum, said sensor being taken from at least: the identification device, the optical device, the measuring device, a sensor spatial orientation, a motion sensor.
  • the step of determining the trajectory of the article comprises at least: a. Collecting a plurality of two-dimensional images, preferably in color; b. The collection of a plurality of three-dimensional images.
  • the collection of the plurality of two-dimensional images is performed by at least one camera and by at least one additional camera, and the collection of the plurality of three-dimensional images is performed by at least one stereoscopic camera.
  • the stereoscopic camera is configured to spatially track the article in three-dimensional space
  • the additional camera is configured to transmit a plurality of two-dimensional images to at least one neural network so as to train said neural network in recognizing the geometric shape of the article
  • the database can also provide corrective data to refine the model generated by the neural network, the spatial position of the article and its geometric shape are then used for tracking of the article by the two-dimensional camera when the article leaves the field of view of the stereoscopic camera.
  • the collaboration of the two cameras allows for better tracking of the item as well as better identification, thus reducing the number of possible frauds.
  • the two-dimensional camera comprises a lens having an angle greater than 100 degrees, preferably called a "wide-angle", and is configured to ensure the tracking of the spatial position of the article outside the field of observation of the article.
  • the stereoscopic camera and to collect images of the geometric shape of the article, the spatial position of the article and its geometric shape are then used for tracking the article by the stereoscopic camera and by the additional two-dimensional camera when the The article comes within the field of observation of the stereoscopic camera.
  • the collaboration of the two cameras allows for better tracking of the item as well as better identification, thus reducing the number of possible frauds.
  • the method comprises, before the step of identifying the article, a step of identifying the user followed by a step of reading a user profile specific to the user from a database. user profile data.
  • the predetermined behavior models comprise at least one standard behavior model comprising at least the following chain of events: a. Identification of the article; b. Item tracking from the identification area to the entry area; vs. Item tracking from the entry area to the interior area; d. Preferably, followed by an empty hand of the user from the inner area to the outer area before or after the measurement of the item's weight.
  • a handling anomaly comprises at least one of the following situations: replacement of the article by another article, addition of another article in a container at the same time as said article, withdrawal of another article from said container when depositing said article in said container, replacement of an identified article by another unidentified article, identification of an article with a fraudulent identifier.
  • the method comprises, if a weight anomaly is detected, the following steps: a. Formulation by message, preferably visual and / or audio, to the user of a request to withdraw the article, this formulation being carried out by a user interface, the user interface being for example the computer processing unit; b. Formulation by message, preferably visual and / or audio, to the user of a request to weigh the article again so as to obtain a new weight, this formulation being carried out by a user interface, the user interface being for example the computer processing unit; vs. Sending by the user terminal to the computer processing unit of the new weight of the item; d. Processing, carried out by the computer processing unit, of the new identifier of the article, of the new weight of the article, and preferably of the images collected, comprising at least the comparison of the predetermined weight with the new measured weight so as to identify a weight abnormality.
  • the method comprises, if an anomaly is detected, the following steps: a. Formulation by message, preferably visual and / or audio, to the user of a request to re-identify the article, this formulation being carried out by a user interface, the user interface being for example the unit of data processing ; b. Sending by the user terminal to the computer processing unit of the new identifier of the article; vs. Formulation by message, preferably visual and / or audio, to the user of a request to weigh the article again so as to obtain a new weight, this formulation being carried out by a user interface, the user interface being for example the computer processing unit; d. Sending by the user terminal to the computer processing unit of the new weight of the article; e. Processing, by the computer processing unit, of the new identifier of the article, of the new weight of the article, and preferably of the images collected, comprising at minus the comparison of the predetermined weight with the new measured weight so as to identify a weight abnormality.
  • the method comprises a continuous step of recording an initial video of a predetermined duration by the optical device, said initial video being erased at the end of said predetermined duration unless an event is detected by at least a sensor taken from at least: the identification device, the measuring device, the optical device, a movement sensor, a spatial orientation sensor.
  • the processing step is performed only when the step of capturing the at least a plurality of data is completed.
  • the method comprises, when the probability of fraud is greater than a predetermined threshold, sends it from the computer processing unit of a plurality of secondary data depending on said plurality of data to at least one management station so that a first supervisor analyzes said plurality of secondary data.
  • This provides a first automated anti-fraud filter, and a second anti-fraud filter involving one or more human operators.
  • said plurality of secondary data is transmitted to at least one portable analysis device, preferably located in the same building as the user terminal, so that a second monitor analyzes said plurality of secondary data and travels to the user.
  • said plurality of secondary data comprises at least one of the following data: the identifier of the article, the weight of the article, an original image of the article, one or more images of the plurality of collected images, a video, preferably temporally compressed.
  • the user terminal is a mobile cart.
  • This provides a smart cart that allows the user to easily pay for purchases at the end of the session, with items being scanned during the shopping session.
  • At least part of the computer processing unit is embedded in the mobile cart.
  • the system comprises at least one management station, preferably delocalized, configured to receive at least a plurality of data from the processing unit. computer so as to be analyzed by at least a first supervisor.
  • the system comprises at least one portable analysis device configured to receive a plurality of data from the management station so as to allow a second supervisor to analyze said plurality of data and to come alongside the manager. 'user.
  • the computer processing unit is in communication with another database comprising at least the history of fraud detected by the user.
  • the user terminal is a fixed terminal, typically intended to be placed in a store, for example near the outlet of the store.
  • the computer processing unit is in communication with at least one classification module comprising at least one neural network trained to detect a fraud situation from data transmitted to the computer processing unit.
  • the user terminal comprises at least one display device configured to display at least the identifier and / or the weight of the article.
  • the system comprises at least one electric battery.
  • a three-dimensional space is meant a space comprising at least three spatial dimensions, at least part of this space being captured by an optical device, preferably stereoscopic, configured to consider these three spatial dimensions, that is, that is to say that it is possible to determine the spatial position of one or more objects present in this three-dimensional space via this optical device.
  • this optical device is configured to take into account, in addition, the depth relative to said optical device, that is to say that it is possible to evaluate the distance from one or more objects present in this space. three-dimensional with respect to said optical device.
  • an object in this three-dimensional space, an object can describe a trajectory and this object therefore comprises three spatial coordinates at each point of this trajectory, because the optical device is able to evaluate the evolution of said object in the three dimensions of space. .
  • This allows an advantageously much more flexible placement of the optical device while preserving the understanding of the actions performed in three-dimensional space.
  • the optical device according to the present invention is not necessarily arranged vertically to the two-dimensional area to be evaluated.
  • the present invention relates to a system, as well as a method for detecting fraud during the purchase of an article by a user in a store, for example.
  • the present invention cleverly enables the detection of fraud during the purchase of an item. Indeed, via a clever method based on an advantageous system, the present invention makes it possible to detect fraud in the case of automatic collection systems, or even automatic payment, also called automatic checkouts or even automatic payment trolleys, for example without limitation. .
  • FIGS 1 to 3 illustrate a fraud detection system according to an embodiment of the present invention.
  • FIG. 1 schematically illustrates such a system 1000.
  • the fraud detection system 1000 comprises at least: a.
  • a user terminal 10 comprising at least: i. An identification device 1100 configured to obtain the identifier of an item 20; ii.
  • a measuring device 1200 configured to measure the weight of said item 20; iii.
  • An optical device 1300 configured to at least detect and spatially track said item 20; iv.
  • a motion sensor and / or a spatial displacement sensor such as a gyroscope for example.
  • a computer processing unit 1400 configured to process a plurality of data and determine a probability of fraud, preferably to determine whether or not there is fraud.
  • the user terminal 10 comprises part or all of the computer processing unit 1400.
  • the user terminal 10 is a mobile cart 10, as illustrated in FIG. 3 for example.
  • the user terminal is a terminal, for example a payment terminal or an automatic cash register.
  • the user terminal 10 can comprise a container 11 intended to receive the article 20 after the user has identified said article 20.
  • at least the identification device 1100, the measuring device 1200 and optical device 1300 are mounted on the same device, preferably mobile, such as for example a carriage 10 as described below in FIG. 3.
  • the identification device 1100 is configured to determine the identifier of the item 20. This determination can take any form. It can for example include the fact of having the identification device 1100 read the barcode of the article 20. This can be a radiofrequency technology of the RFID type or even a visual recognition of the device. article 20, or even a touch interface allowing the user to indicate to the system 1000 the article in question so that the identifier of the article 20 is determined. In the case of visual recognition of section 20, the identification device 1100 may include the optical device 1300 and / or vice versa.
  • the identification device 1100 may include a portable device, for example belonging to the user.
  • the identification device 1100 can use at least one camera of this portable device to identify the article 20.
  • This portable device can for example be a digital tablet or a smart phone.
  • the user presents the article 20 to the identification device 1100 of the barcode reader type, for example, the identifier is obtained by the identification device 1100 and then transmitted to the computer processing unit 1400. Then, the user moves the item 20 into the container 11.
  • the container 11 advantageously comprises the measuring device 1200.
  • the measuring device 1200 is configured to measure the weight of the article 20.
  • the measuring device 1200 comprises a force sensor from which the container 11 configured to receive said article 20 is suspended. here identified.
  • the container 11 can be placed on the force sensor.
  • the measuring device 1200 comprises a scale on which the article 20 is placed in order to measure its weight. Once the weight is measured, this data is transmitted from the measuring device 1200 to the computer processing unit 1400.
  • the optical device 1300 comprises a so-called two-dimensional camera 1310 configured to collect two-dimensional images of a predetermined two-dimensional scene, and preferably a stereoscopic camera also called three-dimensional camera 1320.
  • This stereoscopic camera, or more generally this three-dimensional sensor 1320 is configured to collect three-dimensional images of a predetermined three-dimensional scene.
  • the optical device 1300 is configured to transmit said collected images to the processing unit. computing 1400.
  • the optical device 1300 comprises a camera.
  • the system 1000 can include a plurality of sensors, including the identification device 1100, the measuring device 1200 and the optical device 1300, but also a movement sensor for example, or an accelerometer, or a gyroscope, or any other sensor that can be used to collect one or more data useful for identifying a potential fraud situation.
  • a movement sensor for example, or an accelerometer, or a gyroscope, or any other sensor that can be used to collect one or more data useful for identifying a potential fraud situation.
  • the present invention advantageously takes advantage of the crossing of data collected by a plurality of sensors. This data crossing is advantageously carried out by an artificial intelligence module 1420, preferably comprising at least one network of trained neurons, advantageously automatically.
  • the computer processing unit 1400 is configured to process the data obtained, collected by the identification device 1100, the measuring device 1200, the optical device 1300, and preferably by any other sensor. Indeed, preferably, the computer processing unit 1400 is configured to receive: a. At least one identifier of said article 20 from the identification device 1100; b. At least one measurement of the weight of said item 20 from the measuring device 1200; vs. At least a plurality of images collected by the optical device 1300;
  • the computer processing unit 1400 is in communication with at least one database 1410 comprising for each identifier at least one series of data whose predetermined weight of said item 20, and preferably an image or a graphic representation of said item. article 20.
  • the computer processing unit 1400 may include a weight comparison module for example.
  • the predetermined weight of the article 20 corresponds to an interval weight.
  • the database can include a weight interval and not a precise value. This makes it possible in particular to avoid many situations where the weight would not correspond precisely. Indeed, the articles 20 hardly all have the same weight.
  • this weight range can correspond to the weight of the article 20 at plus or minus 2%, preferably 5%. and advantageously 10%. According to a preferred example, this range has a minimum value and a maximum value, preferably prerecorded or acquired by learning over the operating time of the invention.
  • the predetermined weight recorded in the database 1410 is zero, that is to say it is equal to zero or is not entered.
  • the system 1000 is self-learning, that is, it will feed its database 1410 from the measured weight. For example, the user scans an item 20, the system 1000 identifies the item 20 and accesses the database 1410 of items 20 to compare the weight of said scanned item 20 with that of the database 1410. If the base data returns a zero weight value or if the weight value is not entered in the database 1410, then the system 1000 goes into self-learning mode and replaces this zero or uninformed weight value by the value of the measured weight.
  • the system 1000 captures images of the article 20 so that it can subsequently associate a two-dimensional image of the article 20 with the identifier of the article 20 and the weight of the article. 20. If during the shopping session, the user manipulates said article 20, its weight, identifier and visual recognition will be used to prevent a situation of fraud. Note also that on the first scan, the system 1000 is designed to reason logically, i.e. if the user attempts to have a label of fruits and vegetables on an item other than fruits and vegetables, the visual analysis, described below, can trigger a notification of a potential fraud situation even though the weight is not listed in the database 1410,.
  • this weight can be used as a predetermined weight if before weighing the predetermined weight of said item in the database was zero.
  • this predetermined threshold is less than 10Og, preferably 50g and advantageously 25g.
  • the computer processing unit 1400 is configured to obtain from said database 1410 at least the predetermined weight of said item 20 and to compare this predetermined weight with the measured weight transmitted by the measuring device 1200.
  • the computer processing unit 1400 is configured to process the plurality of collected images.
  • This processing can include the identification and / or the spatial localization of the article 20.
  • this can be used to compare the identifier of the article 20 with the optical identification carried out by the computer processing unit 1400 from the collected plurality of images.
  • the spatial location of the article 20 is used in order to verify that the identified article 20 is indeed the weighed article 20 and that the user has not inverted the article 20. identified with another item 20 of the same weight.
  • the optical device 1300 comprises only a single camera capable of capturing two-dimensional images and three-dimensional images.
  • the optical device 1300 is configured to pick up points in a three-dimensional space, thus allowing depth to be taken into account in capturing the three-dimensional images.
  • optical device 1300 is configured to sense two-dimensional color data.
  • optical device 1300 is configured to track an object, preferably section 20 or one or more hands of a user for example, in a space.
  • This space is compartmentalized into various virtual zones. These virtual zones are defined by the computer processing unit 1400 and are used for the analysis of the images collected, or even the triggering of actions.
  • the analyzed three-dimensional space considered comprises at least four zones: a. A scanning area 1321, located at the level of the identification device, for example in front of a barcode scanner; b. An exterior zone 1322, located above the container 11, preferably above the cart, or outside a drop zone for an automatic scan, for example; vs. An interior zone 1323, located inside the container, preferably in a so-called deposit zone, advantageously in the trolley; d. An entry zone 1234, located between the outer 1322 and inner 1323 zone.
  • the system 1000 also includes at least a portable fraud analysis device 1700.
  • This device 1700 is configured for use by a user called a supervisor, his role being to monitor certain situations of possible fraud. Indeed, cleverly, and as described below, in the event of doubt concerning a fraud situation, a supervisor having a fraud analysis device 1700 receives a plurality of information on it allowing him to '' assess whether or not there is fraud. This analysis step will be described below, in particular its advantageous presentation allowing very high and reliable reactivity on the part of the supervisor.
  • the processing unit 1400 can be in communication with a management station 1600.
  • This management station 1600 allows supervision of a plurality of fraud detection systems 1000.
  • This management station 1600 will also be described more precisely below.
  • FIG. 3 illustrates a fraud detection system 1000 according to a preferred embodiment.
  • a carriage 10 comprises a gripping device 13 and a frame 15 supported by wheels 14 thus making the carriage 10 mobile.
  • the carriage 10 further comprises the identification device 1100, the optical device 1300, the measuring device 1200 and at least one container 11.
  • the cart 10 can include at least one display device 12 making it possible to inform the user if necessary, or even a touch interface service for the management of the user's virtual cart, for example.
  • the computer processing unit 1400 can be embedded in the cart 10 and / or be partially or totally delocalized and be in communication with the elements embedded in the cart 10.
  • the trolley 10 comprises a container 11, preferably suspended from at least one force sensor thus playing the role of measuring device 1200 of the weight of the article 20.
  • the identification device 1100 is a scanner.
  • bar codes Preferably, the carriage 10 comprises the optical device 1300 capable of collecting two-dimensional images, preferably in color, and three-dimensional images.
  • the trolley 10 can include a plurality of sensors such as for example a sensor for spatial position, movement, direction of movement, presence, NFC (Near Field Communication) sensor, RFID sensor ( from the English radio frequency identification), LI-FI sensor (from the English Light Fidelity), bluetooth sensor, or even WI-FI TM type radio communication sensor, etc ...
  • sensors such as for example a sensor for spatial position, movement, direction of movement, presence, NFC (Near Field Communication) sensor, RFID sensor (from the English radio frequency identification), LI-FI sensor (from the English Light Fidelity), bluetooth sensor, or even WI-FI TM type radio communication sensor, etc ...
  • the cart 10 includes one or more bluetooth, WI-FI TM or Lora (Long Range English) type communication modules.
  • the cart 10 includes various sensors linked to artificial intelligence which aims to understand each action performed on the cart 10 by the user and to detect fraudulent actions.
  • This intelligence can for example take the form of a data processing module comprising at least one network of neurons, preferably trained.
  • This neural network can be embedded in the cart 10.
  • the cart 10 comprises an electrical power source 16 to supply the various elements previously indicated for example.
  • the fraud detection system 1000 is at least partly mobile and at least partly on board a trolley 10 as described above.
  • the system 1000 comprises an interface 12 which can be either arranged on the cart 10 itself in the form of a touch interface 12, or be virtualized in the form of a mobile application that the user will have. previously downloaded, for example on their smart phone.
  • the user after selecting the item 20 to purchase, scans it with the identification device 1100.
  • the barcode of the item 20 is scanned by the device. identification 1100.
  • the user has a predetermined time, for example 10 seconds, to place the scanned item 20, that is to say identified, on or in the container. 11.
  • the container 11 is configured to cooperate with the measuring device 1200 so that the weight of the article 20 is measured by the measuring device 1200.
  • the measuring device 1200 is on board the trolley.
  • the user must have the scanned article 20 in the cart 10 in less than 10 seconds, for example without limitation.
  • the measuring device 1200 can be externalized relative to the carriage 10 so that the user, after having scanned the article 20, places the latter on or in the measuring device 1200 so that its weight is measured there, before placing the item 20 in the container 11.
  • the measuring device 1200 determines the weight of item 20.
  • the identifier before the weighing, is transmitted to the computer processing unit 1400. According to another embodiment, the identifier is transmitted to the computer processing unit 1400 after the weighing, and of preferably at the same time as the measured weight.
  • item 20 is added to a virtual cart allowing the system 1000 and the user to track the user's purchases.
  • only one action is possible at a time, i.e. it is not possible to scan, or identify, another article 20 as long as article 20, previously scanned, is not deposited and its weight has not been assessed.
  • the present invention allows the user to cancel his scan in order to potentially scan another article 20.
  • the user cancels the previous scan via the control interface 12, or he waits for the time. predetermined previously indicated, for example the 10 seconds.
  • the present invention also takes into account the situation where the user would wish to remove an article 20 from the cart 10.
  • the user uses the control interface 12 to indicate to the latter that he wishes to withdraw an article 20. of the trolley 10. Then, the user can remove as many articles 20 as he wishes, but must preferably scan them one by one, advantageously waiting each time between each scan for the system 1000 to have detected that the weight of container 11 has varied.
  • the weight variation would be detected by the system 1000, preferably by the measuring device 1200, and would be mentioned to the user, preferably via the interface.
  • the present invention is specially designed to secure the purchase of an article 20 and thus significantly reduce fraud while allowing better cash flow since payment is provided directly through the present invention, directly via the carriage 10 for example, preferably through the display device 12 which can be used command interface 12 and preferably payment.
  • the fraud detection method comprises at least: a. A step of capturing a plurality of data. These data are at least those previously indicated. This capture step is advantageously carried out by the user terminal 10.
  • This capture step comprises at least the following steps: i. Obtaining the identifier of article 20 by the identification device 1100; this step is for example performed by a scan of section 20 using the identification device 1100; The user is invited to scan any article 20 that he wishes to place in the cart 10 for example. ii.
  • An identification zone 1321 corresponding to a volume of three-dimensional space in which at least part of the article 20 is placed by the user to achieve the obtaining of the identifier of the article 20;
  • An entry zone 1324 corresponding to a volume of the three-dimensional space crossed by the article 20 when the user places the article 20 in at least one container 11, preferably associated with the user terminal 10;
  • an interior zone 1323 corresponding to the entrance of a container 11;
  • the external zone 1322 advantageously corresponds to the three-dimensional space surrounding the entrance zone 1324, itself surrounding the inner zone 1323.
  • the determination of the trajectory consists in following article 20 from one area to another area and recording either the entire trajectory or only the sequence of passage from one area to another.
  • any object located in the field of view of the optical device 1300 is tracked in three-dimensional space.
  • the system 1000 is designed to mention this when analyzing the data afterwards. iii.
  • This collection of images lasts until the item 20 is arranged so that the measuring device 1200 can measure its weight; once the article 20 has been scanned, the user has a predetermined time to place the article 20 in the container 11 and thus carry out the weighing thereof; in addition, the scan of item 20 triggers the capture of the plurality of two-dimensional and preferably three-dimensional images; this step of collecting the plurality of images has the function of following the article 20 visually from the scan zone 1321 to its place of deposit in the interior zone 1323; this makes it possible, among other things, to verify that the scanned article 20 is not inverted with another article before being deposited in the container 11 for example.
  • the predetermined weight of the article 20 contained in the database during the first scan of the article 20 may be equal to zero or not be entered; iii.
  • comparing the predetermined weight with the measured weight so as to identify a weight abnormality; preferably, a weight anomaly corresponds to a measured weight different from the predetermined weight found in the database 1410 except for the situation where the predetermined weight is equal to zero or is not entered; otherwise beyond a difference in weight greater than a predetermined threshold, a weight anomaly is considered; this weight anomaly can occur when the user swaps the scanned article 20 with another article whose weight is different, or when it modifies the barcode for example in order to scan an article of a different weight from the real deposited article.
  • the present invention is configured to replace this value by the value of the measured weight, this measured weight value then becoming the value of the predetermined weight during, at least, the following of the user's shopping session.
  • a behavior is not consistent with a standard behavior model from the moment it exhibits a difference with this model of more than 2%, preferably 5% and advantageously 10%;
  • a behavior is not consistent with a standard behavior model from the moment when certain key events of the model are not present in the generated behavior, such key events can for example be the fact that article 20 n 'has not been identified, that article 20 has not been deposited, that article 20 has not crossed the entry zone 1324, etc ...;
  • a behavior is not consistent with a standard behavior model from the moment when certain suspicious events are present in the generated behavior, such suspicious events can for example be the fact that the optical device is temporarily obstructed, or although an object came close to article 20, etc ...
  • a handling anomaly consists, for example, in scanning an article and depositing another of the same weight, or else in scanning an article with a label that does not correspond to said article even if the weight is correct; visual analysis and automated preference is needed for this type of situation, this analysis is provided by the present invention;
  • the computer processing unit 1400 comprises an artificial intelligence module 1420 comprising at least one neural network, advantageously trained in determining handling anomalies;
  • the analysis of the plurality of images collected consists of an analysis of a three-dimensional scene and in particular of the displacement of a plurality of points associated with the article 20 in a three-dimensional space partitioned into different zones; these areas will be described later.
  • the principle of this analysis of the plurality of images is to determine whether the movement of the article 20 in space corresponds to a predetermined model taken from among a plurality of models considered non-fraudulent which will be described below; in the event that the movement of article 20 through these different zones does not correspond to a non-fraudulent model, then there is potentially a situation of fraud.
  • the present invention in addition to considering the movement of the article 20 in this compartmentalized virtual space, the present invention also considers the interactions between the article 20 and any other foreign element; advantageously, if a cloud of points, that is to say a hand or another object approaches and interacts with the cloud of points corresponding to article 20, the suspicion of fraud increases; Preferably, if the foreign element is a hand identified as empty, then the suspicion of fraud can be reduced c. A step of evaluating a probability of fraud, this probability being non-zero if: i. A handling anomaly is identified; and / or ii. Preferably, a weight abnormality is identified.
  • a probability of fraud may correspond to binary data such as, for example, 1 or 0, 1 corresponding to the fact that the fraud is certain and 0 corresponding to the fact that there is no fraud.
  • a probability of fraud can correspond to a percentage of fraud, for example an absence of fraud equals 0% and a certainty of fraud at 100%.
  • a probability of fraud can be a numerical value between 0 and 100 and / or be a binary value equal to 0 or 1.
  • This fraud evaluation step consists of crossing a plurality of data items so as to evaluate a probability of fraud, in particular if an anomaly in weight and / or handling is detected.
  • this data crossing is carried out by an artificial intelligence module 1420 preferably comprising a network of trained neurons, preferably automatically.
  • the present invention provides a hybrid solution in which part of the analysis is performed automatically and another part is carried out through the intervention of supervisors if necessary.
  • the present invention may include at least one portable analysis device 1700 for use by at least one supervisor.
  • the portable analysis device 1700 is configured to receive a plurality of data from the computer processing unit 1400 and / or from a management station 1600 which will be described later.
  • the portable analysis device 1700 is configured to display at least part of this data in a form allowing rapid decision-making, for example in less than 10 seconds, preferably in less than 5 seconds and advantageously in less than 2 seconds, from the supervisor.
  • the aim is to send the most qualitative information to the supervisors, preferably for remote control.
  • the computer processing unit 1400 selects a selection of images from among the plurality of images collected and transmits this selection to the portable analysis device 1700.
  • This selection is advantageously carried out by considering particular instants, for example the moment. scanning, weighing, moving article 20, entering or leaving a zone, etc.
  • the computer processing unit 1400 produces a video, preferably temporally compressed, which it also transmits to the portable analysis device 1700.
  • a temporally compressed video comprises a video of which the number of images per second is greater than 24 for example, or even of a video whose playing time from start to end is less than the duration of the action shown, we also speak of time lapse video or even video accelerated.
  • this video also comprises, preferably on its time line, the notification of the particular instants mentioned above in the form for example of benchmarks. This allows the supervisor to select, if desired, a specific portion of the video relating to a particular event that is noted there. This allows a simple, intuitive and quick way to select an event and access the passage of the video and preferably other data related to this event.
  • the computer processing unit 1400 transmits to the portable analysis device 1700 information related to the scanned item 20 and / or a text explaining the anomaly (s) detected, or even the type of fraud suspected and / or detected.
  • the computer processing unit 1400 transmits these data either directly to the portable analysis device 1700, or via a computer server 1600.
  • This computer server 1600 is advantageously configured to conform the data to be transmitted from such as for example to prioritize them according to various prioritization parameters and / or for example to order them.
  • this computer server is an integral part of a management station 1600.
  • the computer processing unit 1400 transmits said data to at least one management station 1600 via a computer server for example, then an employee, called a super-supervisor for example, is then in charge of analyzing whether or not there is fraud.
  • a validation command is sent to the computer processing unit 1400 validating the user's action.
  • the super-supervisor transmits the considered data to the supervisor's 1700 analysis device.
  • This supervisor is advantageously mobile and can thus contact the user whose action appears to be fraudulent. The supervisor is thus intended to take charge of the situation, on the one hand by analyzing said data and on the other hand by going to the location of the possible fraud.
  • the portable analysis device 1700 may for example comprise a tablet, a computer, a smart phone or even any medium allowing the display of data and preferably comprising an advantageously tactile interface.
  • the data presented on the portable analysis device 1700 is shaped to be easily understood and analyzed.
  • the present invention provides a clear, simple and intuitive presentation of the data allowing the supervisor to decide very quickly, preferably in less than 10 seconds, whether the situation is a fraud situation or not.
  • the computer processing unit 1400 transmits the necessary data so that the super-supervisor located at the management station 1600 can filter out potential situations of fraud. If, according to his analysis, there is no fraud, he sends a validation command to the user so that he can continue with his purchases or payment.
  • a summary of all the “suspicious” actions, that is to say potentially fraudulent, is presented on the management station 1600 of a super-supervisor and / or on the portable analysis device 1700 the supervisor, for example the supervisor located at the exit of the store, so that he can interact with the user during the payment phase, for example.
  • the super-supervisor has all the information necessary to control the action on a graphical interface.
  • This graphical interface is advantageously configured to display the image and the title of the article 20 concerned, a small description of the type of fraud detected, a series of images of the action, such as a comic strip for example in the form of thumbnails, and advantageously a video, preferably accelerated; the objective being that the supervisor and / or the super-supervisor can determine if the action is fraudulent in a very short time, generally in less than 10 seconds, preferably 5 seconds and advantageously in 2 seconds.
  • the interface and / or the data conformation are configured to simplify the work of the supervisor and the super-supervisor.
  • the present invention first uses a first automated filter, represented by the computer processing unit 1400, preferably based on the use of an artificial intelligence comprising at least one neural network, to filter the data. potentially fraudulent situations of others, then a second filter is applied.
  • This second filter comprises the mobile monitors using a portable analysis device 1700.
  • this second filter comprises the super-monitors arranged at the level of management station 1600, therefore the supervisors.
  • Mobile monitors using a portable analyzer 1700 represent a third filter. The combination of these different filters makes the work of each filter easier and faster.
  • the present invention analyzes the possibility of fraud on the basis of an analysis of three-dimensional scenes.
  • the three-dimensional scenes also called a plurality of images
  • These 3D scenes which are preferably dynamic, include one or more pluralities of moving points.
  • a first plurality of points corresponds to item 20 which is then tracked in space.
  • a second plurality of points may correspond to a user's hand or to another item. Any plurality of points which interact, that is to say which approach at a distance less than a predetermined threshold from the first cloud of points, is considered as a potential source of fraud.
  • the displacement of the first plurality of points among the various zones is recorded and compared with a plurality of non-fraudulent displacement models. If ever a chain of actions does not correspond to a chain of actions belonging to a predetermined pattern among the non-fraudulent models, then the probability of fraud increases.
  • Standard model of behavior corresponding to the addition of an article 20 a. Identification of article 20; b. Definition of the geometric shape of the validated article, called “globe” hereafter, in the 1321 scan area; vs. Validated comparison of at least one two-dimensional image of the article 20 contained in the database 1410 with at least one two-dimensional image of the article 20 taken during its identification; d. The validated article 20 leaves the 1321 scan area; e. The validated article 20 passes or not through the external zone 1322; f. The validated article 20 enters the entry zone 1324; g. Validated comparison of the two-dimensional image of the article 20 taken during the identification of the article 20 with the two-dimensional image of the article 20 validated during the passage in the entry zone 1324; h.
  • Article 20 validated enters the interior zone 1323; i. Measuring the increase in the weight of the container 11 accordingly, that is to say a measurement of the starting weight increased by the predetermined weight of the identified item 20, this increase in weight can take place before or after the two-dimensional identification with an empty hand exiting the inner zone 1323 through the entry zone 1324.
  • Standard pattern of behavior corresponding to the identification of an empty hand a. An empty hand enters the outer zone 1322; b. An empty hand enters the 1324 entry area; vs. An empty hand in the 1323 inner zone; d. Variation in weight or not, this variation in weight can occur before or after the following two events; e. An empty hand enters the 1324 entry area; f. An empty hand enters the outer zone 1322;
  • Standard behavior model corresponding to the user taking, to look at it for example, an article 20 already validated and present in the container: a. An empty hand enters the outer zone 1322; b. An empty hand enters the 1324 entry area; vs. An empty hand in the 1323 inner zone; d.
  • the weight decreases, this reduction in weight being able to occur at this moment or else during the 5 following events; e. A full hand enters entry area 1324, becomes the object tracked by optical device 1300; f.
  • the tracked object enters the outer zone 1322; g.
  • the tracked object enters entry area 1324; h.
  • Standard behavior model corresponding to the user posing an article 20, and forgetting to identify it a.
  • a full hand enters the entry area 1324, becomes the tracked object;
  • the measured weight increases, this weight increase being able to occur at this moment or during the 3 following events;
  • vs. An empty hand enters the entry area 1324;
  • An empty hand enters the outer zone 1322;
  • An empty hand enters the entry area 1324;
  • An empty hand enters the inner zone 1323;
  • the weight decreases, as a consequence, that is to say that it returns to the starting weight of the model, this reduction in weight being able to take place between this moment and the end of the model;
  • the tracked object enters the input zone 1324, i. Two-dimensional comparison validated between the two-dimensional image of the object tracked during the first passage through the entry zone with the two-dimensional image of the object tracked during the second passage through the entry zone 1324; j.
  • the tracked object
  • Standard model of behavior corresponding to the withdrawal of an article 20 a. An empty hand enters the outer zone 1322; b. An empty hand enters the 1324 entry area; vs. An empty hand enters the inner zone 1323; d. The weight decreases, this reduction in weight being able to occur between this moment and the end of the model; e. A full hand enters entry area 1324, becomes the tracked object; f. The tracked object enters the outer zone 1322; g. The tracked object falls within the 1321 scan area; h. Identification of an item 20 of the virtual shopping cart selected as an item 20 to be withdrawn by the user; i. Two-dimensional comparison validated between the image of item 20 when it was identified and the image of the tracked object as it passed through the entrance area 1324.
  • the present invention advantageously takes advantage of these standard behavior models. Indeed, instead of seeking to classify a chain of events as fraudulent, it is easier and faster to compare a chain of events with a series of models considered as non-fraudulent. As soon as there is a difference greater than a predetermined threshold between the evaluated behavior and a standard behavior model, fraud is suspected. If this is the In this case, it is the turn of one or more super-supervisors or supervisors to intervene.
  • FIG. 4 illustrates, according to one embodiment of the present invention, an interface of a management station 1600 and / or of a portable analysis device 1700.
  • This interface is advantageously tactile.
  • This interface includes a nifty graphical interface.
  • This graphical interface includes a graphical representation 21 of section 20, as well as optionally a description 22, preferably short and synthetic.
  • This graphical interface includes a simple and synthetic description of the potential type of fraud 23.
  • This graphical interface can include a plurality of images in the form of thumbnails 24 which may for example represent precise and relevant actions of the user taking into account the type of fraud. estimated.
  • This graphical interface preferably comprises a video, advantageously temporally compressed, as described above.
  • the graphical interface comprises at least a first actuator 26 and at least a second actuator 27.
  • the first actuator 26 can for example be configured to allow the supervisor or the super-supervisor to indicate that there is no is no fraud.
  • the second actuator 27 can for example be configured to allow the supervisor or super supervisor to validate that there is a fraud situation.
  • the graphical interface of the management station 1600 may include a third actuator, not illustrated in this figure, configured to transmit the analysis of the data to the mobile supervisor through a portable analysis device 1700 of so that he can go there and validate or not a fraud situation.
  • the user has no action reported as potentially fraudulent by the computer processing unit 1400 and / or no action reported as fraudulent by the supervisor and / or the super-supervisor, then he can pay without any interruption, the purpose being that a user who does not cheat is absolutely not disturbed during his shopping session.
  • a fraud is reported by a supervisor and / or a super supervisor, then: a.
  • the user is notified and awaits the arrival of a supervisor; and / or b.
  • the payment phase is interrupted pending the arrival of a supervisor;
  • a supervisor In any situation, in the event of doubt or validated fraud, a supervisor is responsible for going to the user and checking the item (s) to which the probability of fraud relates. In this way, the supervisor's control is rapid and directly oriented towards one or more items among several others.
  • the present invention also provides a clever way of prioritizing the data and the potential fraud situations to be dealt with.
  • the present invention cleverly crosses several data to assess a probability of fraud, then this data is cleverly conformed and each situation prioritized to allow a fluidity of the user experience and a high reactivity of the supervisors and / or super-supervisors.
  • processing the plurality of data comprises processing a plurality of collected images, which may include two-dimensional images, preferably in color, and three-dimensional images.
  • This processing is advantageously carried out by the computer processing unit 1400 which is preferably on board a mobile element such as the carriage 10 described above.
  • the carriage 10, at least the computer processing unit 1400 must analyze scenes acquired by several sensors; a so-called two-dimensional camera 1310, advantageously wide-angle; a so-called stereoscopic 3D camera 1320; a gyroscope; a measuring device 1200; an identification device 1100; etc ...
  • Analyzing these scenes generally requires a lot of system resources, and therefore computing power, and therefore energy.
  • the system 1000 according to the present invention is cleverly designed to do this type of processing with little power, few system resources, and quickly.
  • this processing can be deported in a computer server in order to reduce the power consumption, but also the system resources used by the carriage 10.
  • the processing must be done directly with the system resources and the energy available in the cart 10.
  • the present invention is designed to limit the cost and energy of an anti-fraud solution.
  • scene analysis is not necessarily a time priority, that is, this analysis does not need to be performed in real time. This, among other things, is where the present invention provides a clever solution.
  • the method of the present invention comprises a step recording of the scenes by all the sensors on a video, in order to analyze them a posteriori.
  • two-dimensional and three-dimensional video recording begins, i.e. two-dimensional and three-dimensional image collection, when there is an object in an area of the previously defined space, for example in the entry zone 1324 or scan zone 1321, or even in the external zone 1322.
  • the data measured or collected by the other sensors are recorded at the precise moment of each event.
  • each event is temporally embedded via, for example, metadata in the video. So, for example, every scan and every consequent change in weight is recorded and noted in the video.
  • the present invention is configured to generate a timeline comprising events that can be taken from: 2D images, 3D images, identification, variation in weight, and more generally any measurement by one of the sensors. This frieze thus makes it possible to chronologically represent the events that have occurred.
  • This enriched timeline thus saves time in the analysis of a potential fraud situation.
  • the recording of this video is defined by the point capture in a given space.
  • the recording when the recording starts, it takes into account the previous X seconds in order to have information linked to the scene before the event that triggered the recording, that is to say that the video recording, otherwise called temporally compressed video, begins with the action that triggered its recording.
  • the system continuously records a predetermined duration, for example 5 seconds, which it deletes as it goes.
  • a predetermined duration for example 5 seconds, which it deletes as it goes.
  • it records 5 seconds of data for example and erases them after 5 seconds unless an event is detected involving the start of recording for subsequent analysis, the images recorded before this event are then taken into account in the generation. time-compressed video.
  • the start of this recording is subject to a change of state of at least one sensor taken from among all the sensors of the system.
  • the sensors of the system are taken from at least: the identification device 1100, the measuring device 1200, the optical device 1300, a movement sensor, a gyroscope, a spatial positioning sensor, an accelerometer, .. .
  • the senor can be a virtual sensor, that is to say a virtual event such as the passage of a point cloud from one spatial area to another spatial area.
  • a virtual event such as the passage of a point cloud from one spatial area to another spatial area.
  • this crossing can be viewed as a change of state, with the analysis of the 3D scene playing the role of a virtual sensor.
  • said recording is carried out, preferably by collecting a plurality of images and data from the various sensors. It will be noted that preferably, all the measurements of each sensor are recorded. For example, when a scan is in progress, an increase in container weight is expected as a result or when the scan is canceled, or when the weight has changed and the system is waiting to return to a stable state, these are examples of events giving rise to the start of data recording.
  • a first recording can be started when the conditions set out above are present, then, if there is an absence of user actions, for example beyond a predetermined time, then the first recording stops. And a second recording starts as soon as the user takes a new action.
  • the final analysis includes the analysis of the first recording and of the second recording even though this analysis is carried out on a timeline comprising one or more time gaps, that is to say one or more periods not recorded because they are exempt. actions.
  • the recording will start, but if the user leaves and does not take any action after 10 seconds for example, the recording will stop, and a new recording will start as soon as an action is detected.
  • the analysis will be done considering the two records, because the analysis is only done when the carriage 10 becomes stable again, it will however have a hole in the data recording.
  • an analysis can relate to several records.
  • the same recording can be used for several different analyzes.
  • the start of the recording can also be initiated by the three-dimensional capture of the crossing of the entry zone 1324 of the carriage 10, for example.
  • a stable state is defined when all the sensors do not detect a measurement variation greater than a predetermined threshold, this threshold possibly being a function of each sensor.
  • An unstable situation is therefore defined as corresponding to the detection of a measurement variation by at least one of the sensors above said predetermined threshold, preferably specific to said sensor. Note that scanning an item is considered an unstable state by the present invention.
  • the tracking of article 20 and / or of the user's hand or hands is triggered following the scanning of said article 20.
  • the tracking of an article 20 can be triggered when the user takes out an article from the container 11, taking into account the detection of the change in weight by the measuring device 1200.
  • the three-dimensional shape of the article 20, also called an object is reconstituted, preferably in two parts, this three-dimensional shape will be called "validated shape".
  • the first part of this validated shape is the end of the shape that we will call the “globe” which represents the article and the hand.
  • the second part of this shape is the arm and potentially part of the user's body.
  • the shape present in the scan zone 1321 becomes the validated shape and the globe is the end of it.
  • the globe must pass from the scan zone 1321 to the external zone 1322, then pass through the entry zone 1324 and disappear into the internal zone 1323. Then the article is supposed to be deposited in the container 11, and therefore a variation of weight must be measured, finally the globe comes out through the entry area.
  • the globe can also pass directly from scan area 1321 to entry area 1324.
  • a two-dimensional analysis of the images of the 2D camera 1310 through a neural network is carried out in order to verify that the globe which comes out of the container after the deposit of the article 20 in the container 11 does indeed correspond to an empty hand. . If the scan detects an empty hand passing through the entry area 1324 and towards the outer area 1322, then there is no fraud. The same situation applies if the scan detects an empty hand after measuring an increase in weight consistent with the item ID 20, then there is no fraud.
  • the probability of fraud can be moderate, or even zero.
  • the measuring device 1200 detects a deposition action, that is to say an increase in the weight of the container 11, while the validated shape is still in the outer zone 1322, we can deduce a strong probability of fraud via the detection of a handling anomaly.
  • the scenario without fraud is the same, but in the other direction, i.e. a hand identified as empty recovers an item 20 whose weight is subtracted from that of the container. 11 and this article 20 is then scanned, the correspondence between the predetermined weight and the less measured weight confirms the absence of fraud, for example. Conversely, if a weight is removed without a subsequent scan or if the weight of the scanned item 20 does not correspond to the weight removed, the probability of fraud increases.
  • the system 1000 will detect a full hand via the two-dimensional analysis, this hand crossing the entry zone 1324, or even interior 1323, and the measuring device 1200 will detect an increase in the weight of the container 11 and its contents.
  • the probability of a weight anomaly that is to say of a fraud, is high.
  • a handling anomaly is detected, and the probability of fraud increases.
  • the measuring device 1200 detects an increase in weight, a deposit action has been taken, and if no scan has been performed, the probability of fraud increases.
  • the tracked shape does not leave the field of observation of the optical device 1300 and re-enters the interior zone 1323 without an unknown shape approaching it, without entering the entry zone 1324 or without the device optics 1300 is not obstructed, there is no handling anomaly and the probability of fraud is low.
  • the present invention provides for a two-dimensional comparison of the article 20 taken out and the article 20 withdrawn.
  • the function of the system 1000 is to find this shape when the shape enters the field of observation of the optical device 1300.
  • the system 1000 comprises a two-dimensional camera 1310 called “wide-angle", that is to say having an optical angle greater than 100 degrees.
  • This 1310 2D camera is configured to additionally provide this tracking function.
  • the optical device comprises an additional 2d camera configured to cooperate with the 3D camera.
  • the additional 2D camera is configured to collect two-dimensional images of the three-dimensional scene.
  • the optical device 1300 comprises a plurality of 3D cameras 1320 and 2D cameras 1310, or even additional 2D cameras.
  • the 2D camera 1310 “knows” its aspect, its geometric shape, and its exit position in order to continue tracking the object on 2D camera 1310.
  • the three-dimensional camera 1320 allows the system 1000 to learn the shape of the tracked article 20 and to track its position in space, this learned shape and this known position are then transmitted to the two-dimensional camera 1310 for monitoring. over a larger area, as soon as the article 20 leaves the surveillance area of the three-dimensional camera 1320.
  • the 2D camera 1310 can communicate its position as well as its aspect in return, so that the camera 3D 1320 can resume monitoring, or even improve learning, for example.
  • an analysis can be made on the 2D camera 1310 to determine whether a full hand or an empty hand has approached the item 20, or the object, being tracked.
  • article or object is used independently to define article 20.
  • the present invention comprises a double check mode.
  • This mode is to be implemented when there is a doubt concerning a fraud.
  • This mode consists of transmitting a request to the user to re-scan an item 20 which is supposed to be in container 11, a few minutes after he has inserted it or during payment.
  • the present invention provides an effective solution to this type of fraud. Indeed, to thwart this type of fraud, the present invention proposes to take pictures towards the article 20 from different angles. During a scan, these photos have a dual purpose: a. Initially, these photos are used to feed a neural network model linked to the identifier of article 20, for example to its barcode. This neural network is then configured to indicate whether item 20 is present on an image taken by optical device 1300 or not. This is done by comparing the photo just taken and all the other photos taken during a scan for the same item 20 for example. b. Secondly, the photos pass through the neural network model and the output of said network is a probability of conformity between the scanned article 20 and article 20 expected in relation to its identifier, in order to assess whether the scanned barcode corresponds to article 20 of the photo.
  • the tag exchange being done most often with fruits and vegetables
  • the neural network is trained to identify a bag of fruits and / or vegetables, and if during a scan of a “fruits and vegetables” barcode, the optical device 1300 does not recognize a bag of this type, then a fraud is suspected.
  • the database can include a score per article corresponding to the fact that it is a cheap article and therefore regularly used to carry out fraud, either by using the label of such an article, or its packaging for example non-limiting. Also, preferably, these inexpensive items have a higher fraud score than luxury items.
  • the luxury items have a higher fraud score than the other items.
  • FIG 5 schematically shows the process of recording data and processing them.
  • This figure illustrates two parts of a fraud detection algorithm according to an embodiment of the present invention.
  • the recording 110 of the data begins 120 when an object is detected by the optical device, preferably by the stereoscopic camera and advantageously when the detected object is located in one of the zones of three-dimensional space. If no object is detected 122, recording remains pending.
  • X seconds are expected 150 and appended 151 to the end of the recording when it is finalized 146.
  • the recording then stops 160.
  • Analysis 210 is pending as long as a recording is in progress, So the system monitors whether an identification is in progress 220: yes 221, no 222, if a weight measurement is in progress 225: yes 223, no 222 .
  • the system prepares 230 to analyze a record.
  • the algorithm finalizes its analysis 270 and returns to its initial state of waiting for a new analysis to be performed.
  • a portion of the system resources allocated to the analysis is redistributed for data collection.
  • the present invention uses few system resources and low energy by separating into two distinct phases, the data collection and the analysis of this collected data.
  • the present invention thus makes it possible to obtain high-quality fraud detection while providing a low-cost technical solution, the solution being optimized for large-scale and inexpensive application.
  • the present invention makes it possible to resolve at least the following fraud situations: a.
  • the user scans an item 20 and places two; b.
  • the user discreetly places an item 20 in container 11 without scanning it; vs.
  • the user scans a € 5 bottle of wine and places one at € 50 of equivalent weight, or even that looks like him morphologically; d.
  • the user swaps an article 20 not scanned with an article 20 already scanned in the container 11; e.
  • the user scans an item 20 with a fruit or vegetable barcode label; f.
  • the user puts a scent in a fruit and vegetable bag and I scan it with a fruit and vegetable barcode label.
  • the present invention therefore uses the fusion of several data coming from several sensors to determine a probability of fraud.
  • the present invention comprises an analysis of its so-called self-learning data, that is to say that the computer processing unit is configured to automatically learn the elements making up a fraud.
  • the system is configured to learn that generally a series of actions, or that certain values of the data collected lead to a situation of fraud.
  • the processing unit receives a plurality of data as input and as output the situation is judged as fraud or not by the supervisors and / or super-supervisors.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Burglar Alarm Systems (AREA)
  • Pinball Game Machines (AREA)
  • Alarm Systems (AREA)
  • Image Analysis (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Closed-Circuit Television Systems (AREA)
EP20816473.1A 2019-12-05 2020-12-03 Système et procédé de détection de fraude Pending EP4070295A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR1913824A FR3104304B1 (fr) 2019-12-05 2019-12-05 Système et procédé de détection de fraude
PCT/EP2020/084359 WO2021110789A1 (fr) 2019-12-05 2020-12-03 Système et procédé de détection de fraude

Publications (1)

Publication Number Publication Date
EP4070295A1 true EP4070295A1 (fr) 2022-10-12

Family

ID=70613871

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20816473.1A Pending EP4070295A1 (fr) 2019-12-05 2020-12-03 Système et procédé de détection de fraude

Country Status (7)

Country Link
US (1) US20230005348A1 (enExample)
EP (1) EP4070295A1 (enExample)
JP (1) JP2023504871A (enExample)
CN (1) CN115004268A (enExample)
CA (1) CA3160743A1 (enExample)
FR (1) FR3104304B1 (enExample)
WO (1) WO2021110789A1 (enExample)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7555852B2 (ja) * 2021-03-03 2024-09-25 東芝テック株式会社 不正行為認識装置及びその制御プログラム、並びに不正行為認識方法
FR3128048B1 (fr) * 2021-10-13 2024-09-20 Mo Ka Borne d’encaissement automatique intelligente
EP4457150A1 (en) 2021-12-29 2024-11-06 R.A Jones & Co. Packaging unit for packaging articles in boxes and method for packaging articles in boxes
CN114898249B (zh) * 2022-04-14 2022-12-13 烟台创迹软件有限公司 用于购物车内商品数量确认的方法、系统及存储介质
US20250078098A1 (en) * 2023-08-30 2025-03-06 Maplebear Inc. Management System for Automatic Determination of Anomaly Behavior for User of a Smart Shopping Cart
JP2025123936A (ja) * 2024-02-13 2025-08-25 東芝テック株式会社 情報処理装置及び情報処理システム
DE102024105309A1 (de) * 2024-02-26 2025-08-28 KBST GmbH System und Verfahren zum Verifizieren eines in einen Einkaufswagen und/oder Einkaufskorbs eingelegten Produkts
CN118278732B (zh) * 2024-03-25 2024-11-29 深圳泰昌同信科技有限公司 一种收银行为的智能监测系统及方法

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050102183A1 (en) * 2003-11-12 2005-05-12 General Electric Company Monitoring system and method based on information prior to the point of sale
US20080031491A1 (en) * 2006-08-03 2008-02-07 Honeywell International Inc. Anomaly detection in a video system
JP2010094332A (ja) * 2008-10-17 2010-04-30 Okamura Corp 商品陳列装置
JP5216726B2 (ja) * 2009-09-03 2013-06-19 東芝テック株式会社 セルフチェックアウト端末装置
CA2940398C (en) * 2014-01-21 2023-12-05 Tyco Fire & Security Gmbh Systems and methods for customer deactivation of security elements
WO2016052229A1 (ja) * 2014-09-29 2016-04-07 日本電気株式会社 情報処理装置、情報処理方法、およびプログラム
EP3248870B1 (en) * 2016-05-25 2019-07-03 AIRBUS HELICOPTERS DEUTSCHLAND GmbH Multi-blade rotor for a rotary wing aircraft
CN105915857A (zh) * 2016-06-13 2016-08-31 南京亿猫信息技术有限公司 超市购物车监控系统及其监控方法
CN106408369B (zh) * 2016-08-26 2021-04-06 西安超嗨网络科技有限公司 一种智能鉴别购物车内商品信息的方法
JP6638075B2 (ja) * 2017-01-04 2020-01-29 西安超▲ハイ▼網絡科技有限公司Ｘｉａｎ Ｃｈａｏｈｉ Ｎｅｔ Ｔｅｃｈｎｏｌｏｇｙ Ｃｏ．， Ｌｔｄ． インテリジェントショッピングカートの商品の識別方法およびシステム
JP6649306B2 (ja) * 2017-03-03 2020-02-19 株式会社東芝 情報処理装置、情報処理方法及びプログラム
US11250376B2 (en) * 2017-08-07 2022-02-15 Standard Cognition, Corp Product correlation analysis using deep learning
CN109934569B (zh) * 2017-12-25 2024-04-12 图灵通诺（北京）科技有限公司 结算方法、装置和系统
JP6330115B1 (ja) * 2018-01-29 2018-05-23 大黒天物産株式会社 商品管理サーバ、自動レジシステム、商品管理プログラムおよび商品管理方法
FR3077404B1 (fr) * 2018-01-31 2022-07-15 Afraite Seugnet Mehdi Methodes et systemes d'assistance a l'achat en un point de vente physique
CN108460933B (zh) * 2018-02-01 2019-03-05 王曼卿 一种基于图像处理的管理系统及方法
JP6573185B1 (ja) * 2018-12-07 2019-09-11 株式会社鈴康 情報処理システム、情報処理方法及びプログラム
CN109829777A (zh) * 2018-12-24 2019-05-31 深圳超嗨网络科技有限公司 一种智能购物系统和购物方法
US11430044B1 (en) * 2019-03-15 2022-08-30 Amazon Technologies, Inc. Identifying items using cascading algorithms

Also Published As

Publication number Publication date
CN115004268A (zh) 2022-09-02
WO2021110789A1 (fr) 2021-06-10
US20230005348A1 (en) 2023-01-05
FR3104304A1 (fr) 2021-06-11
CA3160743A1 (fr) 2021-06-10
FR3104304B1 (fr) 2023-11-03
JP2023504871A (ja) 2023-02-07

Similar Documents

Publication Publication Date Title
WO2021110789A1 (fr) Système et procédé de détection de fraude
CN111263224B (zh) 视频处理方法、装置及电子设备
EP3752990B1 (fr) Dispositif de stockage d'objets, et procédé mettant en oeuvre un tel dispositif
US12093889B2 (en) Method, a device and a system for checkout
EP3940614A1 (fr) Magasin autonome de vente au détail en libre-service
CN110400138A (zh) 防盗刷的移动支付方法和装置
CN111178116A (zh) 无人售货方法、监控摄像机及系统
EP3506166A1 (fr) Prédiction de déplacement et de topologie pour un réseau de caméras
CN111222870A (zh) 结算方法、装置和系统
CN109145127A (zh) 图像处理方法及装置、电子设备和存储介质
CN110689389A (zh) 基于计算机视觉的购物清单自动维护方法及装置、存储介质、终端
CN109934569B (zh) 结算方法、装置和系统
CN112001349A (zh) 数据审核方法、系统及电子设备
CN117671605B (zh) 一种基于大数据的冷柜摄像头角度控制系统及方法
FR3118763A1 (fr) Dispositif et procédé de comptage et d’identification de bouteilles
FR3112467A1 (fr) Magasin autonome de vente au détail en libre-service
EP2329474B1 (fr) Procede et systeme de surveillance de scenes
FR3033648A1 (fr) Procede et dispositif de suivi d'individus dans un lieu equipe de moyens repartis de detection
FR3112468A1 (fr) Magasin autonome de vente au détail en libre-service
FR3112466A1 (fr) Magasin autonome de vente au détail en libre-service
FR2995430A1 (fr) Cabine d'essayage interactive et procede d'interaction avec ladite cabine d'essayage
FR3088457A1 (fr) Procede de detection automatique de l'usurpation de visage
FR2800943A1 (fr) Systeme de suivi de consommateurs notamment dans une enceinte d'hypermarche
CN116542686A (zh) 无人购物引导方法、装置、存储介质及电子设备
EP4494112A1 (fr) Procede mis en ?uvre par ordinateur pour la detection automatique d'objets abandonnes

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20220610

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20240430