WO2021186149A1

WO2021186149A1 - Security system

Info

Publication number: WO2021186149A1
Application number: PCT/GB2021/050614
Authority: WO
Inventors: Joseph Morris
Original assignee: Tj Morris Ltd
Priority date: 2020-03-20
Filing date: 2021-03-11
Publication date: 2021-09-23
Also published as: GB2593209A; GB202004049D0

Abstract

The present invention concerns a security system for retail premises. A plurality of cameras is mounted in the retail premises to image exit routes from the retail premises. Exit monitoring logic is implemented in a computing system for analysing images from the cameras to identify an individual (30) making an irregular exit from the retail premises. Security scoring logic serves to analyse at least one image of the individual identified by the exit monitoring logic and to attribute to that individual a security score.

Description

Security System

The present invention is concerned with security in retail premises.

The term "retail premises" as used herein refers to any form of shop, shopping centre, mall, market, supermarket or other building or enclosed area in which goods are available for purchase. A problem that faces most retailers is shoplifting - removal of goods from the retail premises without payment. The provision of multiple cameras in retail premises which observe customer behaviour is now commonplace and one of the functions of these cameras is detection of shoplifting, but with numerous parallel video feeds which are active for much or all of the working day, there is potentially a large volume of video data. Review of all of this by a human operator may involve excessive labour.

A variety of image processing techniques have been applied in relation to video surveillance data for security and for other purposes concerned with analysis of human behaviour.

For example US2015/0146605A1 (Lipton et al) discloses a video surveillance system in which images are processed to extract video primitives, and using the video primitives event occurrences are extracted using event discriminators. In one example the system is applied to tracking people in a grocery store, but this is done in order to measure the efficacy of a retail display in attracting attention, based on factors such as numbers of people who slow down or stop at the display.

US2015/288928A (McCoy et al) concerns a camera security system in which video analysis is carried out to identify and track objects and to enable the system to respond to certain events. One example given concerns determination when a person moves in the wrong direction through a given area, as when a person enters a secure area of an airport through an exit only hallway.

US2003/107650A (Colmenarez et al) describes a surveillance and security system using a pattern recognition module for recognising when observed behaviour is associated with predefined suspicious behaviours. Some examples are given of approaches to detection of shoplifting, including determining whether an individual is wearing a coat by comparison of a camera image against archive images.

US 2002/070859 (Gutta et al) discloses a security monitoring system using a camera to monitor the path of an individual and a "trajectory analyser" used to trigger an alarm if the trajectory does not match a known trajectory stored in a database. Face data may also be used to identify authorised individuals.

US2010/0208063 (Lee et al) discloses a number of essentially statistical techniques for analysing the trajectories of individuals moving through a space and thus detecting abnormalities of behaviour. So the prior art does teach a range of techniques for monitoring human behaviour based on camera data, and in some cases these techniques are applied to detection of certain suspicious behaviours. But existing systems are subject to an important practical problem due to the quantity of video data provided by a typical in-store camera system. Even using modern computer hardware, this volume of video data is problematic. The cost of sufficient server capacity to process all such video data may be large or even prohibitive. Some means is needed to reduce the volume of video data to be subject to image analysis, and to concentrate such analysis on times and/or locations where there is reason to think that shoplifting may be taking place.

In accordance with a first aspect of the present invention there is a security system for retail premises having a plurality of exit routes for individuals, the security system comprising a plurality of cameras for mounting in the retail premises to image the exit routes and so to provide image data; a plurality of exit sensor arrangements each assignable to an exit route and configured to enable detection of an irregular exit by an individual through the assigned exit route; selection logic configured to respond to detection of an irregular exit by selecting a subset of the image data, the subset of the image data being selected by reference to the time of the irregular exit and by reference to the exit route assigned to the sensor arrangement that detected the irregular exit; and security scoring logic for analysing the selected subset of the image data and attributing to it a security score.

In accordance with a second aspect of the present invention there is a method for identifying possible shoplifters in retail premises having a plurality of exit routes, the method comprising: imaging exit routes from the retail premises using a plurality of cameras to provide image data; detecting an irregular exit through one of the exit routes by an individual; selecting a subset of the image data from the image data by reference to the time of the irregular exit and to the exit route taken by the individual; and analysing the selected subset of the image data and attributing to it a security score.

Specific embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which: Figures 1-5 each represent an image of a supermarket vestibule obtained through an overhead security camera; and

Figures 6-8 each represent a view of a supermarket till arrangement obtained through an overhead security camera.

Typical layout of retail premises

Retail premises often have defined and separate routes for entry and for exit of customers. For example a modern supermarket often has a single main entrance leading to the shopping area in which goods are displayed for purchase. Often the main entrance is arranged to facilitate entry of individuals to the retail premises but to offer at least some degree of impediment to individuals moving in the opposite direction, to exit the retail premises.

Figures 1 to 5 provide overhead views of a vestibule 10 forming the main entrance of a supermarket and having outer doors 12 communicating with the exterior of the building and inner doors 14 leading toward the shopping area. The drawings represent the vestibule 10 as it appears in images taken from an overhead security camera installed in the ceiling of the vestibule. In this example the doors 12, 14 are configured to open automatically as an individual approaches along the entrance path, in the direction of arrow 16, moving from the exterior of the building toward the shopping area. The doors 12, 14 are configured not to open automatically in response to an individual moving in the opposite direction, toward the exterior. The doors may for example be controlled by motion detectors surveying only one side of the door, so that the approach of an individual from the opposite side of the door does not trigger the motion sensor and does not cause the door to open.

Such arrangements do not actually prevent egress of individuals through the main entrance. An individual may for example simply wait until the door 12, 14 is opened as another individual enters, and thus make their exit through it.

A supermarket (or other retail premises) will have at least one designated exit to the exterior. The floor plan of a supermarket is typically designed such that to reach the main exit (or exits) from the shopping area it is necessary to pass through one of a number of payment paths. A payment path, as the term is used herein, is a defined path through which a customer passes with his or her chosen goods in order to make payment. The payment path leads past a payment facility which may be a cash till. Figures 6 to 8 show examples a widely used and familiar form of payment path using cash tills 20. Again the drawings represent the view through an overhead security camera, and in this installation of there is a respective camera above each of the payment paths. Looking at Figure 6, goods 22 are in this example placed on a conveyor 23 prior to being scanned by an operator 19 at the till 20.The operator A constricted payment path indicated by arrow 24 is provided between one till island 26 and its neighbour.

The arrangement offers some degree of impediment to persons exiting the shopping area without making payment, since the payment path is narrow and, whilst the till 20 is manned, is easily observable by the till operator. But shoplifters can make their exit through the payment paths nonetheless.

Often one or more tills 20 will be unmanned at any given time. In this instance the corresponding payment path 24 may be closed by a physical barrier. This typically takes the form of a gate 28 (see Figure 7) that can be opened to lie across the payment path 24 and offer a degree of impediment to an individual making an exit through the payment path 24. But even when closed the gates 28 do not actually prevent individuals from using the payment path as a route for egress. Typically the gate 28 is able to be pushed open against the force of a spring even when the payment path 24 is closed. This is for safety reasons - in an emergency such as a fire customers may need to escape through the gate 28.

Increasingly, retailers are providing means for taking payment which do not rely on manual scanning of goods by human till operators, e.g. because the customer carries out the scanning process, interacting with an automated till arrangement, or because the customer uses an app on a mobile computing device to log chosen goods while moving around the store, so that payment can quickly be made on exit. But in these cases too there is, in the physical floor plan of the retail premises, typically a defined payment path through which the customer must pass in order to visit the payment facility, make payment and reach the main exit. The invention is applicable to retail premises using such payment arrangements.

The layout of retail premises is intended to provide customers with a regular route - they enter through the entrance, select goods in the shopping area, and having done so pass through one of the payment paths, pausing in the payment path to make payment for the goods, and so reach the exit. The great majority of individuals follow this pattern of behaviour. A small minority (perhaps as low as 1%) are observed in practice to follow a different path to the exit, making what will be referred to herein as an "irregular" exit.

Irregular exits

A shoplifter in possession of goods will need to exit the retail premises in order to make a theft. Having picked up goods in the shopping area, the shoplifter will make an irregular exit which may be: by passing through an open payment path without making payment, in order to reach the main exit; by passing through a closed payment path; or by passing through the main entrance in the wrong direction, to reach the exterior of the retail premises.

In-store cameras

The invention makes use of image data received from cameras observing the retail premises. The invention imposes no particular limitation on the form of the cameras used. Present embodiments are implemented using electronic closed circuit in-store video cameras mounted overhead. Such cameras are already commonly installed in existing retail premises.

It is common practice to provide in-store cameras which observe the available exit routes. Often these comprise cameras above each of the payment paths 24 and cameras observing the vestibule 10. The attached drawings represent images obtained using such cameras.

In certain embodiments of the invention one or more of the cameras used may be capable of 3D imaging. That is, it may provide pixel data which includes a measure of distance of the imaged surface from the camera. 3D imaging cameras can provide a 3D digital model of the surfaces in the field of view, which is potentially advantageous in the present context. The invention imposes no particular limitation on the form of 3D camera, which could for example be a "time of flight" camera, or could use structured light at a frequency out of the visible range. The cameras - whether 2D or 3D - will typically be responsive to light frequencies in the optical range but this is not essential to the invention - other imaging technologies, responsive to light frequencies outside the visible spectrum, e.g. in the infra-red range, could in principle be employed.

The data output from the cameras is referred to herein as "image data". Although reference is made below to analysis of "images", it must be understood in every case that the images may be from a stream of images forming a video.

Overview of security system

The mere fact that an individual makes an irregular exit in one of the ways listed above does not in itself imply that he or she is shoplifting. It may for example simply be that the individual has not found the goods they require, and thus leaves the premises without making a purchase.

The security system to be described herein carries out two particular functions in order to identify possible shoplifters:- it provides for detection of irregular exits by individuals; and in response to detection of an irregular exit, the security system selects - from the large body of image data provided by the various in-store security cameras - a smaller subset of the image data. This subset of the image data is analysed to attribute a security score which can be used as a basis for a decision about any further action to be taken.

These aspects of the system will be explained below.

Detection of irregular exits

The security system comprises sensor arrangements for detection of an individual making an irregular exit. The sensor arrangements monitor the possible exit routes, and in particular in the present embodiment they monitor the main entrance(s) and the payment paths. Each sensor arrangement is typically assigned to monitor a specific exit route.

So far as the payment paths are concerned, there are in the present embodiment several ways that an individual making an irregular exit by one of these routes may be identified. Note that the mere passage of an individual through an open payment path in the direction toward the main exit is not in itself irregular - customers are intended to take this route to the exit, making payment as they do so

In relation to the payment paths, the present embodiment uses two particular two particular forms of sensor able to respond to irregular exits:

- sensors are provided on the till gates 28 to detect when an individual passes through the payment path associated with an unmanned till. When the payment path is unmanned, the gate 28 is normally closed. The sensors detect movement of the gate as an individual pushes it open against its springing in order to pass through.

- till operators are provided with an input device for signalling the occurrence of an irregular exit, in particular the passage of an individual through the payment path without payment being made. The input device may comprise a button, for example, or a designated key on a keypad. In this context the "sensor arrangement" is of course sensing the operator's input. Till operators will typically be instructed to use their judgement as to whether an individual making an exit through the payment path 24 is likely to be shoplifting, and may ignore persons who appear simply to be exiting the store legitimately without making a purchase.

In the present embodiment each of the payment paths 24 is provided with a sensor arrangement comprising both these types of sensor. Activation of either of the sensors causes a signal to be sent, whose effect will be explained below. Examples of irregular exits are depicted in Figures 7 and 8. In both drawings an individual 30g/30h is seen to have pushed open the security gate 28, triggering the associated sensor. In another form of irregular exit, the shoplifter passes through the payment path 24 without making payment. In this case a till operative is able to observe the shoplifter's behaviour and actuate the input device to signal the occurrence of the irregular exit.

In the present embodiment of the invention, detection of irregular exits via the payment paths is made without analysis of the image data from the overhead cameras observing the payment paths. This is advantageous in that it reduces the volume of image data to be analysed.

Nonetheless in certain embodiments of the invention the overhead camera image data is utilised in detecting irregular exits. Processing of this data may be used to recognise when an individual passes through a closed payment path 24. The processing involved in this determination may merely involve detection of suitable movement in the field of view, and need not be excessively burdensome. More sophisticated image processing techniques may be used to recognise when an individual moves through an open payment path 24 without pausing to make payment, as an indicator that an irregular exit is taking place. In such embodiments, the camera itself is for this purpose part of the sensor arrangement used to detect irregular exits.

In other embodiments of the invention motion sensors are used to detect irregular exits. The motion sensors may be sensitive to the direction of motion of an individual. So while simple passive infrared sensors may be used in the simplest embodiments of the invention, other more preferable forms of sensor include microwave motion sensors, ultrasonic motion sensors, RADAR, LIDAR etc. Additionally or alternatively, tagging technology may be used to detect irregular exits. For example, radio frequency tags may be attached to store trolleys or baskets so that their path through the store is able to be tracked, enabling the system to determine when a basket or trolley is used in an irregular exit.

In relation to the main entrance of the premises, passage of an individual in the direction toward the exterior of the retail premises constitutes an irregular exit. This may be detected using motion sensors monitoring the main entrance, or using RF tags. So in embodiments of the invention the detection of an irregular exit may be made without need of analysis of image data from cameras monitoring the entrance.

Nonetheless in other embodiments of the invention image data from cameras monitoring the entrance may be analysed to detect irregular exits by this route. The processing involved can be straightforward, involving simple detection of the direction of motion of an individual in the image, and so need not be excessively burdensome in terms of processing capacity. Security Scoring

The security score can be thought of as an estimate of the likelihood that the individual is engaged in shoplifting. It can be used as a basis for a determining whether to take further action in relation to the individual, which could be real time action (such as alerting a security guard to approach the individual) but in present embodiments involves other responses such as use of the resultant data in compiling a report for police, or in excluding the individual from the store on future occasions, or in analysing shoplifter behaviour to improve other security measures.

Security scoring is carried out by a digital computer system which may take any of a variety of different forms. Often the retail premises will have an on-site computer system handling feeds from the in store cameras. That computer system may be programmed to implement the present invention. But there is no reason why for example the present invention should not be implemented in the servers of an internal company network, or in the cloud. Data may be exported from individual retail premises for analysis in a shared computer system. The digital computing system may be implemented in a single computer device but it may be implemented in a set of networked computers. Specific aspects of the system may be divided between different networked computers.

The digital computer system receives outputs of the sensor arrangements and uses these to monitor for irregular exits. The processing load involved in such monitoring depends on the nature of the sensor arrangements used in any particular installation but it may be small.

The digital computer system also receives or has access to feeds of image data from the in-store cameras. This typically comprises image data from multiple in-store cameras active for all or much of the time. The volume of this image data is typically large.

When an irregular exit is detected, the digital computer system makes a selection of a subset of the image data for further analysis. This selection involves (a) selecting image data from one (or in some examples more than one) of the in-store cameras and (b) selecting image data (from the chosen camera feeds) in a certain period of time. The selection is made by reference to (a) the time of detection of the irregular exit and (b) the sensor arrangement which made the detection, and the exit route that the sensor arrangement is assigned to monitor. Thus for example when a security gate sensor assigned to a certain payment path 24 is activated by an individual making an irregular exit, the digital computer system receives and logs the sensor's signal. In response, the digital computing system makes a selection of a subset of the body of image data for analysis. This subset of the image data comprises, in the present example, image data taken from the camera observing the relevant payment path 24. More specifically, it comprises image data from that camera during a period including the irregular exit. The camera in question may for example be an overhead camera or it may be sited to view the path 24 from one side, or cameras of both types may be provided. The particular period chosen may for example take in a certain period of time before and after detection of the irregular exit. The period in question may be flexibly defined - for example the digital computing system may in the process of its image analysis track the progress of an individual making an irregular exit and select for analysis image data over the entire time period that the individual is in the camera's field of view. Images from more than one camera may be relevant to certain exit routes, in which case the selected image data may comprise image data from more than one camera.

But in any of these cases the invention makes it possible to select a subset of the image data which is small in relation to the large total volume of data output by the in-store cameras. Security scoring is carried out using this subset of the overall body of image data. Typically this data subset will include images of an individual making an irregular exit. The data subset is subject to image analysis to determine the security score. This image analysis may in itself be relatively demanding of computer processing capacity, but it can be carried out without an excessive overall processing burden as it is being carried out only on the selected data subset. Typically the image processing involves identifying an individual in the images and analysing that individual's image for features suggestive of shoplifting.

It is not necessarily possible to determine with confidence from the images whether an individual in the chosen images is shoplifting. But the security score is intended to make it possible to focus attention on individuals considered likely to be shoplifting. In the present embodiment a high score suggests a high likelihood that the individual is shoplifting. A lower security score suggests a lower likelihood of this. In some instances the score may for example be a binary indicator, with one value indicating a high risk and the other indicating a low risk.

Factors that may be taken into account in generating the security score include the following:- a. whether the individual is pushing a store trolley, and if so whether that trolley contains goods. The individuals 30c and 30g depicted respectively in Figures 3 and 7 are each pushing a store trolley 32c, 32g containing goods 34c, 34g through an irregular exit route (in Figure 3 it is the main entrance, and in Figure 7 it is a payment path 24). It may well be that these individuals are engaged in shoplifting, and they will each be given a high security score. The individual 30b depicted in Figure 2 pushing an empty store trolley 32b through the entrance, will typically receive a lower score; b. whether the individual is carrying a store shopping basket containing stock. A store basket which contains or which possibly contains stock is a factor that raises the security score. A customer should not normally take a store basket out of the store, so an individual passing through an exit carrying a store basket may be identified as being engaged in the potential theft of the basket. c. whether the individual is carrying bags or other containers (other than a store trolley or basket). If so the appearance of the bags may also be a factor. The presence of bags may increase the security score. If the bags appear to be filled, this may further increase the security score. Figure 4 provides an example of a camera image in which the individual 30d can be seen to be carrying bags 40 which from their profile can be recognised as being filled. d. whether the individual is carrying store goods 36, as is the individual 32e in Figure 5. Clearly this factor contributes to an increased security score. e. whether the individual is using a wheeled shopping trolley (other than a store trolley). This is another factor that increases the security score. f. the individual's garb. Presence of a hat or other headgear may increase the security score, especially if it obscures the face of the wearer. The individual 30c of Figure 3 for example has a cap 42 which prevents his face form being observed form overhead. Shoplifters often wear headgear to make their identification more difficult. Bulky clothing in which stock may be hidden can contribute to increased security score but the system may take account of other factors - time of year and weather factors including temperature and precipitation - since in cold weather or rain the wearing of a coat may not be suggestive of intent to shoplift, whereas in warm fine weather this type of garb may give more reason for concern. g. the time of day and/or the day of the week. These can be a factor in the frequency of shoplifting, so they may be taken in to account in evaluating the security score. h. the route taken to exit the retail premises. Shoplifters are more likely to take an exit route that is easy or quick, e.g. by choosing to pass through payment paths that are close to a main exit, so individuals identified in these areas may receive a higher security score. i. in some instances recognition techniques may be used to identify known shoplifters so that they will be given an increased security score. Suitable recognition techniques including facial recognition, recognition of clothing, gait analysis, recognition of physical features (height, profile etc) and others are known to the skilled person. The security system may - where privacy law permits - incorporate or refer to a database of known suspects for this purpose. j. the form of sensor data used to detect the irregular exit. In particular, alerts provided by an input from a till operative may be given a higher weight than others, since these will reflect a human judgement that shoplifting may be underway.

This list is not exhaustive. Deep learning techniques and/or artificial intelligence techniques are in the present embodiment employed to correlate the different factors to improve the reliability of the security score in determining the risk of shoplifting, the system being trained on real data. Deep learning techniques and/or artificial intelligence techniques may also be employed in the image analysis carried out by the system, with training being carried out in either case using real data from in store sensor arrangements.

Claims

1. A security system for retail premises having a plurality of exit routes for individuals, the security system comprising a plurality of cameras for mounting in the retail premises to image the exit routes and so to provide image data; a plurality of exit sensor arrangements each assignable to an exit route and configured to enable detection of an irregular exit by an individual through the assigned exit route; selection logic configured to respond to detection of an irregular exit by selecting a subset of the image data, the subset of the image data being selected by reference to the time of the irregular exit and by reference to the exit route assigned to the sensor arrangement that detected the irregular exit; and security scoring logic for analysing the selected subset of the image data and attributing to it a security score.

2. A security system as claimed in claim 1 which is for use in retail premises in which at least one of the exit routes comprises a payment path leading past a point of sale device, the payment path being provided with a barrier able to be opened to permit passage through the payment path and to be closed to obstruct passage through the payment path, the security system comprising at least one exit sensor arrangement configured to respond to unauthorised opening of the barrier.

3. A security system as claimed in claim 1 or claim 2 in which at least one exit sensor arrangement is configured to sense a user input indicative of an irregular exit.

4. A security system as claimed in any preceding claim in which at least one of the exit sensor arrangements comprises a camera whose output is supplied to exit monitoring logic configured to detect an irregular exit.

5. A security system as claimed in claim 4 which comprises at least one payment path camera for observing a payment path in the retail premises, the payment path leading past a point of sale device toward an exit from the retail premises, wherein the exit monitoring logic is configured to signal an irregular exit in response to detection of an individual moving through the payment path without making payment.

6. A security system as claimed in claim 4 or claim 5 in which the exit monitoring logic is configured to signal an irregular exit in response to detection of an individual moving through a payment path which is closed.

7. A security system as claimed in any preceding claim in which the security scoring logic is configured to analyse the selected subset of the image data to detect the individual making the irregular exit, and to determine whether that individual is carrying goods.

8. A security system as claimed in any preceding claim in which the security scoring logic is configured to analyse the selected subset of the image data to detect the individual making the irregular exit, and to determine whether that individual has a trolley.

9. A security system as claimed in claim 8 in which, in the event that the identified individual has a trolley, the security scoring logic is configured to analyse the image of the identified individual to determine whether the trolley contains goods.

10. A security system as claimed in any preceding claim in which the security scoring logic is configured to analyse the selected subset of the image data to detect the individual making the irregular exit and to determine whether that individual is carrying a bag or basket.

11. A security system as claimed in claim 10 in which in the event that the individual is determined to be carrying a bag or basket, the security scoring logic is configured to analyse the image of the identified individual to determine whether the bag or basket contains goods.

12. A security system as claimed in any preceding claim in which the security scoring logic is configured to analyse the image of the identified individual to recognise aspects of the individual's garb which are taken into account in determining the security score.

13. A security system as claimed in any preceding claim in which the security score is an assessment of risk that the individual making the irregular exit is shoplifting.

14. Retail premises provided with a security system as claimed in any preceding claim.

15. A method for identifying possible shoplifters in retail premises having a plurality of exit routes, the method comprising: imaging exit routes from the retail premises using a plurality of cameras to provide image data; detecting an irregular exit through one of the exit routes by an individual; selecting a subset of the image data from the image data by reference to the time of the irregular exit and to the exit route taken by the individual; and analysing the selected subset of the image data and attributing to it a security score.