EP4207120A1

EP4207120A1 - Security monitoring system

Info

Publication number: EP4207120A1
Application number: EP21218151.5A
Authority: EP
Inventors: Nicholas J. HAckett; Julien PIEDBOIS
Original assignee: Verisure SARL
Current assignee: Verisure SARL
Priority date: 2021-12-29
Filing date: 2021-12-29
Publication date: 2023-07-05
Also published as: WO2023126347A1

Abstract

There is provided a security monitoring system for premises, the system including:
a Wi-Fi access point of a Wi-Fi network, the Wi-Fi network including a plurality of Wi-Fi stations, the Wi-Fi access point and the plurality of stations being configured to operate a radio-based location sensing arrangement to detect human presence and location based on detecting perturbations of radio signals of the Wi-Fi network, one of the stations of the Wi-Fi network being a motion-triggered video camera, the video camera being configured to transmit a notification to a remote entity in the event that it is triggered,
the Wi-Fi access point being configured to:
respond to the transmission of a notification by the video camera by using the radio-based location sensing arrangement to determine whether and where there is human presence within the premises; and, in the event that human presence is detected,
generate a report, that includes details of where and when human presence has been detected;
and to notify the report to one or more designated recipients.

Description

Field

The present invention relates generally to security monitoring system installations for premises.

Background

Security monitoring systems for monitoring premises, often referred to as alarm systems, typically provide a means for detecting the presence and/or actions of people at the premises and reacting to detected events. Commonly such systems include sensors to detect the opening and closing of doors and windows, movement detectors to monitor spaces (both within and outside buildings) for signs of movement, microphones to detect sounds such as breaking glass, and image sensors to capture still or moving images of monitored zones. Such systems may be self-contained, with alarm indicators such as sirens and flashing lights that may be activated in the event of an alarm condition being detected. Such installations typically include a control unit (which may also be termed a central unit or local management device), generally mains powered, that is coupled to the sensors, detectors, cameras, etc. ("nodes"), and which processes received notifications and determines a response. The local management device or central unit may be linked to the various nodes by wires, but increasingly is instead linked wirelessly, rather than by wires, since this facilitates installation and may also provide some safeguards against sensors/detectors effectively being disabled by disconnecting them from the central unit. Similarly, for ease of installation and to improve security, the nodes of such systems typically include an autonomous power source, such as a battery power supply, rather than being mains powered.
As an alternative to self-contained systems, a security monitoring system may include an installation at a premises, domestic or commercial, that is linked to a remotely located monitoring station where, typically, human operators manage the responses required by different alarm and notification types. These monitoring stations are often referred to as Central Monitoring Station (CMS) because they may be used to monitor a large number of security monitoring systems distributed around the monitoring station, the CMS located rather like a spider in a web. In such centrally monitored systems, the local management device or central unit at the premises installation typically processes notifications received from the nodes in the installation and notifies the Central Monitoring Station of only some of these, depending upon the settings of the system - in particular whether it is fully or only partially armed, and the nature of the detected events. In such a configuration, the central unit at the installation is effectively acting as a gateway between the nodes and the Central Monitoring Station. Again, in such installations the central unit may be linked by wires, or wirelessly, to the various nodes of the installation, and these nodes will typically be battery rather than mains powered.
Despite the ready availability of professionally installed security monitoring systems, many homes remain unprotected by such systems, possibly because people are put off by the expected cost of installing such systems and subscribing to monitoring services.
In recent years many people have installed Wi-Fi enabled video cameras in their homes - from suppliers such as Ring and Arlo, enabling users to check up on the status of their homes remotely. These cameras typically include a PIR motion sensor that activates the camera when someone comes in view of the sensor. Captured video can be stored in the cloud - typically requiring payment of a subscription fee - for viewing at users' leisure, and push alerts can be sent to the user's smartphone (and those of others, if desired) so that the user can receive real-time notification of presence. Typically, these cameras are installed more for simple peace of mind, than with a view to creating a stand-alone security monitoring system. Indeed, it is not uncommon for occupants of premises that already have a professionally installed security monitoring system also to install such cameras so that they can be reassured about the situation at home when they are away. In particular, parents of school-age children often install such cameras so that they can see when their children get home from school - and also so that they can "keep an eye on" their kids in the hours before a parent returns home. Such cameras are known by the term "convenience cameras" because it is convenient to use them for a kind of self-monitored surveillance, without needing to worry about someone in an alarm surveillance centre being able to look in on private life in the home - something about which people feel very sensitive, especially when they are parents of young children. The suppliers of such convenience cameras also offer "monitoring packages" which typically involve installing extra devices such as a control unit and PIR motion sensors to create in effect a burglar alarm installation, and paying a significantly enhanced subscription fee. Many owners of convenience cameras are put off by the cost of the extra equipment that needs to be installed, as well as by the typically hefty annual subscription. Yet many of these people, especially if they are parents of school age children, or part-time carers for elderly parents or relatives would welcome more extensive reassurance about conditions in the homes already provided with a convenience camera.
The present invention enables this more extensive reassurance, and this may be made possible without the need to install in effect a complete burglar alarm system.
Embodiments of the invention are based on the insight that it may be possible to combine a radio-based location sensing arrangement to detect human presence and location based on detecting perturbations of radio signals, for example such a system based on use of Wi-Fi signals, with a convenience camera it is possible to provide enhanced security without the need to install a lot of additional components or pay significant annual fees.
More generally, a non-alarm sensor, which is preferably a convenience camera but could be another source of information such as a microphone, can be combined with such a radio-based location sensing arrangement to provide a system such that in the event of the non-alarm sensor detecting an "event" the radio-based location sensing arrangement is checked to determine whether there is a human presence when/where no human presence is expected, and triggering an alarm event and push/stream video/images (information) to a user or other remote consumer in the event that the radio-based location sensing arrangement detects human presence.
For the purposes of the present application, "burglary" should be understood in the sense that the offence is defined under English law: entering a building or part of a building as a trespasser intent to commit theft, grievous bodily harm, or criminal damage; or having entered as a trespasser, stealing, or inflicting/attempting to inflict grievous bodily harm. There is no requirement for the entry to involve "breaking in", simply entering through an open or unlocked entrance is sufficient. Throughout the specification we may refer to someone intent on committing burglary as a burglar, intruder, or villain, as distinct from those resident at the property (on a temporary or permanent basis) who do not fit within the definition of burglar - who we will refer to a residents. In terms of detecting presence, movement, and location, both of these classes of people fall within the term "occupant".

Summary

In a first aspect there is provided a security monitoring system for premises, the system including: a Wi-Fi access point of a Wi-Fi network, the Wi-Fi network including a plurality of Wi-Fi stations, the Wi-Fi access point and the plurality of stations being configured to operate a radio-based location sensing arrangement to detect human presence and location based on detecting perturbations of radio signals of the Wi-Fi network, one of the stations of the Wi-Fi network being a motion-triggered video camera, the video camera being configured to transmit a notification to a remote entity in the event that it is triggered, the Wi-Fi access point being configured to: respond to the transmission of a notification by the video camera by using the radio-based location sensing arrangement to determine whether and where there is human presence within the premises; and, in the event that human presence is detected, generate a report, that includes details of where and when human presence has been detected; and to notify the report to one or more designated recipients.
Optionally, the Wi-Fi access point is configured to receive a push notification from the remote entity in the event of a notification being transmitted to the remote entity as a result if the video camera having been triggered. Such push notification are typically sent from the camera supplier's / operator's "cloud" servers to designated recipients whose contact details (email address or mobile phone number, for example) have been provided to the camera supplier/operator by the user of the camera - the Wi-Fi access point can have its own email address and be provided with logic to enable it to respond to the content of emails received from the service, and/or have a SIM so that it can receive push notifications directly. All we need is for the Wi-Fi access point to be able to recognise alert messages from the camera operator/supplier and to distinguish these from other types of message.
Optionally, the Wi-Fi access point is configured to recognise notifications sent by the video camera to the remote entity. That is, the Wi-Fi access point can be configured to recognise characteristics of addressing, format or size, for example, of the Wi-Fi signals that the camera transmits when it is reporting an event to the remote entity. Again, the Wi-Fi access point doesn't need to read these messages as long as there is some characteristic that distinguishes these messages from, for example, housekeeping or "checking in" messages.
Optionally, the Wi-Fi access point has access to a database of locations within the premises, and optionally, the Wi-Fi access point is configured to use the database to determine whether a detected human presence is within a location included in the database. A database of locations may be provided to enable the Wi-Fi access to point to label the locations where human presence is detected (main bedroom, office, Juliet's room, etc.), and can also be used to identify particular locations where human presence should always be flagged up - for example the office, the gun room, the master bedroom, etc., and to indicate other areas where human presence should only be flagged up at certain times of day, for example, and yet other areas where human presence may be acceptable at any time or at any time during the day - or for a certain duration, for example on the doorstep of the front door, where temporary presence at any time during the day is probably acceptable, but may be a concern at night or if the presence extends to more than a few minutes.
Optionally, the Wi-Fi access point has access to a calendar or timetable for use in determining whether detected human presence should be notified or not. By having access to a calendar, timetable or schedule of the occupant(s) of the premises - e.g. the household schedule, it is possible for the Wi-Fi access point to adapt its behaviour to account for human presence, its count, and its location.
According to a second aspect, there is provided a method performed by a premises security monitoring system, the system including:

a Wi-Fi access point of a Wi-Fi network, the Wi-Fi network including a plurality of Wi-Fi stations, the Wi-Fi access point and the plurality of stations being configured to operate a radio-based location sensing arrangement to detect human presence and location based on detecting perturbations of radio signals of the Wi-Fi network, one of the stations of the Wi-Fi network being a motion-triggered video camera, the video camera being configured to transmit a notification to a remote entity in the event that it is triggered,
the method comprising:
- the video camera transmitting a notification to the remote entity upon being triggered;
- the Wi-Fi access point, in response to the transmission of the notification by the video camera, determining using the radio-based location sensing arrangement whether and where there is human presence within the premises; and, in the event that human presence is detected,
- generating a report, that includes details of where and when human presence has been detected; and
- notifying the report to one or more designated recipients.

Optionally, the determining whether and where there is human presence within the premises is triggered by the Wi-Fi access point receiving a push notification from the remote entity in the event of a notification being transmitted to the remote entity as a result if the video camera having been triggered.
Optionally, the determining whether and where there is human presence within the premises is triggered by the Wi-Fi access point recognising a notification sent by the video camera to the remote entity.
Optionally, the method, further comprising the Wi-Fi access point accessing a database of locations within the premises and using the database to determine whether a detected human presence is within a location included in the database.
According to a third aspect, there is provided a method of detecting the presence of an intruder in premises, the method comprising:

detecting an event at the premises using a non-alarm sensor, such as a "convenience" camera;
receiving notification of the event and, based on receiving the notification, checking a radio-based location sensing arrangement to detect human presence and location based on detecting perturbations of radio signals to determine whether there is a human presence where no human presence should be, and if such human presence is detected creating an alert.

Brief description of the drawings:

Embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

Figure 1 illustrates schematically a plan of a single floor of premises in which a security monitoring system has been provided using a video camera and a Wi-Fi access point that provides radio-based presence and location sensing; and
Figure 2 illustrates schematically the principles of radio-based presence and location sensing;

Specific description

Figure 1 is a schematic plan of a single storey dwelling 100 which we will use to illustrate aspects of the invention.
The dwelling has a front door 102, the dwelling's main entrance, that is accessed externally by a path 104. A doorbell 105 is provided adjacent the front door, preferably in the form of a video doorbell which should preferably be arranged to provide a view of the whole of the front approach of the house (by means of which the front door can be accessed) including the path 104. The main entrance 102 leads into a hall 106 by means of which all the rooms of the house may be accessed. The dwelling has a rear door 108 that leads out from a kitchen 110 onto a terrace 112. Each of the rooms includes at least one window 114, or in the case of the living room 116 and master bedroom 118 a pair of French Windows 120 that open out onto the terrace 112. The living room's French Windows permit access to the terrace and a rear garden, but are not intended, or used, for regular access to the interior of the premises.
A "convenience" video camera (for example as provided by Ring as "stick up cameras") 130 is located in the hall 106 at a point where it provides a good view of the entrance door 102, of the door of the study 136, as well as of a large expanse of the hall. This camera is Wi-Fi enabled and is part of a Wi-Fi network provided by Wi-Fi access point 128 which is located in the living room. The Wi-Fi access point 128 may connect to the internet via gateway/router 600 which provides a cables broadband connection, or optionally directly to the internet using a PLMN, such as a UMTS, 4G or 5G network using an appropriate internal transceiver and SIM.
The video camera 130 is operatively connected, via the Wi-Fi network to a remote entity 200, typically a cloud server provided by the supplier/operator of the camera (e.g managed by Ring or Arlo, for example) to which events (including video clips) are communicated when the camera is triggered (typically by means of an internal motion sensor such as a PIR). These events may be onward reported through the cloud (the manufacturer's or operator's back end systems) to designated individuals - for example by sending push notifications and/or email notifications.
The front door is also provided with a doorbell 105, and if this is a video doorbell this too may report incidents to the remote entity 200 via the Wi-Fi network provided by access point 128. Likewise, a garden camera 144, if present, may also report incidents to the remote entity 200 via the Wi-Fi network provided by access point 128.
Figure 1 also shows the presence of a plurality of devices that have Wi-Fi functionality, and that are all stations, STA, in the Wi-Fi network provided by Access Point 128. The observant reader will have noticed that this includes features whose presence has not yet been mentioned, and these are elements that may play a role in providing radio-based presence and location sensing (which we will hereafter refer to a Wi-Fi sensing, or WFS, for convenience). The access point 128 functions as a WFS receiver, and various Wi-Fi devices are positioned around the premises both to perform their normal role but also to act as illuminators for WFS. Within the house there are several Wi-Fi extenders 150 which are convenient because they can readily be positioned in any vacant electrical socket, provide good signal strength, and typically have a small form factor. These may also be used outside - if weatherproof or protected from the weather. The external Wi-Fi camera 144, and video doorbell, if present, can also function as outdoor illuminators to improve the reach of the WFS. In the kitchen 110 a smart speaker 152 and a Wi-Fi extender 150 are provided as illuminators, while broadband router or gateway 600 provides illumination from within the dining room. In the study 136 a "smart plug" 154 acts as illuminator, while in the hall the video camera 130 acts as an illuminator. It will be appreciated that this choice of Wi-Fi sources, and their disposition, is given merely to illustrate a suitable approach - the number of illuminators and the positioning required very much depend upon the area of cover required for WFS and on the size and type of construction of the property being monitored. It may not always be necessary to use external Wi-Fi illuminators in order to extend Wi-Fi sensing to beyond the external walls of a building, but by providing some suitably positioned external illuminated, as shown schematically here, it should be possible to extend the range of WFS to achieve the desired results. With the ubiquity and wide penetration of Wi-Fi devices into the lives of a large proportion of the population of the developed world, it may often the case that no, or very few, Wi-Fi devices need to be added to a home in order to provide a satisfactory level of WFS cover - although that may not always be the case with the elderly.
We will now provide a brief introduction to radio-based presence detection, which may for example be based on analysing the signal dynamics and signal statistics of radio signals and/or detecting changes in channel state information (CSI). A radio (or wireless) signal as used herein refers to a signal transmitted from a radio transmitter and received by a radio receiver, wherein the radio transmitter and radio receiver operate according to a standard or protocol. Such standards include, but are not limited to, IEEE 802.11. (which includes the Wi-Fi standards), IEEE 802.15 (which includes Zigbee), Bluetooth SIG, IEEE 802.16, IEEE 802.20, UMTS, GSM 850, GSM 900, GSM 180, GSM 19011, GPM ITU-R 5.13, GPM ITU-R 5.150, ITU-R 5.280, 3GPP 4G (including LTE), 3GPP 5G, 3GPP NR, AND IMT-2000. However, the radio transmitters and receivers providing and using radio signals for WFS may operate in non-telecommunications or Industrial, Scientific and Medical (ISM) spectral regions without departing from the scope of the invention.
Essentially the idea is to use radio signals to probe a zone or zones of interest, and to analyse and extract statistics from these signals, in particular looking at the physical layer and/or data link layer such as MAC address measurements that expose the frequency response of a radio channel (e.g., CSI or RSSI measurements). These measurements are processed to detect anomalies and variations over time, and in particular to detect changes signifying the entrance of a person and/or movement of a person within a monitored zone. The zone(s) to be monitored need to be covered sufficiently by radio signals, but the sources of the radio signals may either already be present before a monitoring system is established - for example from the plurality of Wi-Fi or Bluetooth capable devices that are now dotted around the typical home or office, or the sources may be added specifically to establish a monitoring system. Often some established (i.e., already located or installed) radio devices are supplemented by some extra devices added as part of establishing a radio-based presence detection system. Among the types of devices (pre-installed or specifically added) that may be used as part of such a detection system are Wi-Fi access points, Wi-Fi routers, smart speakers, Wi-Fi repeaters, as well as video cameras and video doorbells, smart bulbs, etc. Because presence (or intrusion) is detected by detecting a change in the properties or character of radio signals compared to some previous reference signal(s), it is preferred to establish what might be termed the monitoring network between radio devices that are essentially static (i.e., that remain in the same position for extended periods) rather than relying on devices that are repeatedly moved - such as smart phones, headphones, laptops, and tablet devices. It is not strictly speaking essential for all the devices whose signals are used by the monitoring system to be part of the same network - for example, signals from Wi-Fi access points of neighbouring premises could be used as part of a monitoring system in different premises. Again, a primary consideration is the stability of the signals from the signal sources that are used. Wi-Fi access points provided by broadband routers are seldom moved and rarely turned off, consequently they can generally be relied upon as a stable signal source - even if they are in properties neighbouring the property containing the zone or zones to be monitored.
The idea is illustrated very schematically in Figure 2, here with an installation 200 including just a single source (or illuminator) 202 and just a single receiver 204, for simplicity, although in practice there will typically be multiple sources (illuminators) and sometimes plural receivers. The installation 200 has been established to monitor a monitored zone 206. In Figure 2A we see that in steady state, and in the absence of a person, radio signals are transmitted from the source 202, spread through the monitored zone 206, and are received by the receiver 204. Of course, in most installations there will be walls, ceilings, floors, and other structures that will tend to reflect, at least in part, signals from the source. Furniture and other objects may block and attenuate the signals, the reflected signals will give rise to multiple paths, and the signals may interfere with each other, and there may be scattering and other behaviours, such as phase shifts, frequency shifts, all leading to complexity in the channels experienced by the radio signals that arrive at the receiver 204. But while the environment is static and unchanging, the receiver will tend to see a consistent pattern of radio signals. And this is true whether or not the source transmits continuously or transmits periodically. But this consistent pattern of received signals is changed by the arrival of an intruder 208, as shown in Figure 2B. From Figure 2B we see that, at the very least, the presence of a person in the monitored zone blocks at least some of the signals from the source, and that affects the pattern of radio signals received by the receiver 204. The changed pattern of signals received by the receiver enables the presence of the intruder to be detected by a presence monitoring algorithm that is supplied with information derived from the received signals. It will be appreciated that the nature and extent of the perturbation of the signals passing from the source 202 to the receiver 204 is likely to change as the intruder 208 enters, passes through, and leaves the monitored area 206, and that this applies also to reflected, refracted, and attenuated signals. These changes may enable the location of a person within the zone, and their speed of movement, to be determined. Indeed, these techniques have been shown even to be capable of detecting gestures, and patterns of human respiration, as well as enabling "people counting".
It will be realised that signals that are received from an illuminator device (or from more than one illuminator device) after having passed through a monitored space (or volume), have in effect been filtered by the environment to which they have been exposed. We can therefore imagine the monitored volume as a filter having a transfer coefficient, and we can see that a received signal is at least in part defined by the properties, or channel response, of the wireless channel through which it propagated. If the environment provided by the monitored volume changes, for example by the addition of a person, then the transfer coefficient of the filter, and the channel response or properties, will also change. The changes in the transfer coefficient, and in the channel response, consequent on the change in the environment of the monitored space, can be detected and quantified by analysing radio signals received by the wireless sensing receiver(s). Both the introduction of an object, e.g. a person, into the monitored space, and movement of that object within the monitored space will change the environment and hence change the effective transfer coefficient and the channel response.
The radio-based sensing system can be trained by establishing a base setting in which the monitored zone is unoccupied, which is then labelled as unoccupied for example using a smartphone app or the like, and then training occupied states by a person entering, standing, and then walking through each of the zones one by one. Presence at different locations in each of the zones may be captured and labelled in the system in the same way. This process may be repeated with two people, and then optionally with more people. In essence this is a supervised machine learning approach, but other approaches to training may be used.
The system may need to be retrained for the base setting if bulky furniture or other large objects (particularly if made of metal) are added to or moved within the monitored space, because these can be expected to change the propagation properties of the relevant zone/space. The data for unoccupied states are preferably retained within a database of "unoccupied" states, even when there are changes to the arrangement of furniture etc. It may not be necessary to retrain for the occupied states if the system can determine a delta function between the previous base state and the new one, because the delta function may also be applicable in occupied states. But if not, it may be sufficient to retrain only a subset of the occupied states previously learnt. The system may also be configured to self-learn to accommodate changes in the characteristics of the zones when unoccupied, and to add newly determined unoccupied state data to the database.
Although the Figure 2 example uses just a single source (illuminator) and a single receiver, as already mentioned generally multiple sources (illuminators) will be used in order to achieve satisfactory coverage of the zone or zones to be monitored. Multiple zones may be monitored by a single receiver through the use of multiple strategically placed sources, but each zone, or some zones of multiples zones may have a dedicate receiver that does not serve other zones. Likewise, a radio signal source (illuminator) may provide illuminating signals for a single monitored zone or for multiple monitored zones. Also, a presence monitoring system (and a security monitoring system including such a presence monitoring system) may use mesh network arrangement, for example a Wi-Fi mesh network, in which multiple devices act as receivers for illuminating signals - either for a single monitored zone or for multiple monitored zones.
In a first aspect there is provided a security monitoring system for premises, the system including: a Wi-Fi access point of a Wi-Fi network, the Wi-Fi network including a plurality of Wi-Fi stations, the Wi-Fi access point and the plurality of stations being configured to operate a radio-based location sensing arrangement to detect human presence and location based on detecting perturbations of radio signals of the Wi-Fi network, one of the stations of the Wi-Fi network being a motion-triggered video camera, the video camera being configured to transmit a notification to a remote entity in the event that it is triggered, the Wi-Fi access point being configured to: respond to the transmission of a notification by the video camera by using the radio-based location sensing arrangement to determine whether and where there is human presence within the premises; and, in the event that human presence is detected, generate a report, that includes details of where and when human presence has been detected; and to notify the report to one or more designated recipients.
Optionally, the Wi-Fi access point is configured to receive a push notification from the remote entity in the event of a notification being transmitted to the remote entity as a result if the video camera having been triggered. Such push notification are typically sent from the camera supplier's / operator's "cloud" servers to designated recipients whose contact details (email address or mobile phone number, for example) have been provided to the camera supplier/operator by the user of the camera - the Wi-Fi access point can have its own email address and be provided with logic to enable it to respond to the content of emails received from the service, and/or have a SIM so that it can receive push notifications directly. All we need is for the Wi-Fi access point to be able to recognise alert messages from the camera operator/supplier and to distinguish these from other types of message.
Optionally, the Wi-Fi access point is configured to recognise notifications sent by the video camera to the remote entity. That is, the Wi-Fi access point can be configured to recognise characteristics of addressing, format or size, for example, of the Wi-Fi signals that the camera transmits when it is reporting an event to the remote entity. Again, the Wi-Fi access point doesn't need to read these messages as long as there is some characteristic that distinguishes these messages from, for example, housekeeping or "checking in" messages.
Optionally, the Wi-Fi access point has access to a database of locations within the premises, and optionally, the Wi-Fi access point is configured to use the database to determine whether a detected human presence is within a location included in the database. A database of locations may be provided to enable the Wi-Fi access to point to label the locations where human presence is detected (main bedroom, office, Juliet's room, etc.), and can also be used to identify particular locations where human presence should always be flagged up - for example the office, the gun room, the master bedroom, etc., and to indicate other areas where human presence should only be flagged up at certain times of day, for example, and yet other areas where human presence may be acceptable at any time or at any time during the day - or for a certain duration, for example on the doorstep of the front door, where temporary presence at any time during the day is probably acceptable, but may be a concern at night or if the presence extends to more than a few minutes.
Optionally, the Wi-Fi access point has access to a calendar or timetable for use in determining whether detected human presence should be notified or not. By having access to a calendar, timetable or schedule of the occupant(s) of the premises - e.g. the household schedule, it is possible for the Wi-Fi access point to adapt its behaviour to account for human presence, its count, and its location.
According to a second aspect, there is provided a method performed by a premises security monitoring system, the system including:

The access point 128 may be further configured to use data from the radio-based location sensing arrangement to perform people counting, and optionally to use determine the presence of one or more intruders based on a detected change in the people count when the system is in the nocturnal armed mode. For example, the techniques and methods described in US2020/0302187A1 , assigned to Origin Wireless, can be used to count occupants and determine their locations in installations, systems and methods according to embodiments of the invention. The access point 128 can, for example use people counting to keep track of the number of people in the house - and this may include being aware of the number of people who have, for example, gone into the rear garden or who are on the terrace.
The access point 128 may also be configured to receive and store a number corresponding to the number of usual residents - something that is particularly useful if the dwelling is the home of someone who lives alone. In such a case, the access point 128 can improve its confidence in determining whether detected human presence (and count) constitutes a reportable event or not. The access point 128 could also be programmed to deal with days or periods when more residents than usual are expected - for example the weekly visit of a cleaner, carer, or heath visitor, or the monthly visits of family or friends. Such information could be entered by the resident or an authorised user or installer using, for example an app on a device or online, or via spoken commands or touch screen input at the access point 128, or entered via a web app or a smartphone app.
A brief explanation will now be given of how Wi-Fi Sensing works, and how Wi-Fi Sensing can be integrated into a security monitoring system, and in particular how WFS can be integrated into a central unit of a security monitoring system.
Wi-Fi Sensing can be performed with any Wi-Fi device and can be used on any available communication path. Each communication path between two devices gives the chance to extract information about the surrounding environment. Wi-Fi sensing is based on an ability to estimate the wireless channel and hence the surrounding environment. Because Wi-Fi networks comprise many devices spread throughout a geographical area, they are well suited to exploiting these devices' transmissions in effect to provide a radar system. Depending on the number of devices, the radar system may be monostatic, bistatic, or multistatic. In monostatic WFS, a single device measures its own transmitted Wi-Fi signals. In bistatic WFS, the receiver and transmitter are two different devices (for instance, an AP and a STA in infrastructure mode). In multistatic WFS, the received signals from multiple Wi-Fi transmitters are used to learn about a shared environment.
At least one Wi-Fi transmitter and one Wi-Fi receiver are required to perform WFS measurements, and these can be located in the same device (to create a kind of monostatic radar) or in different devices. The measurement is always performed by a Wi-Fi Sensing-enabled receiver on the Wi-Fi signal transmitted by a transmitter, and which may or may not originate from a Wi-Fi sensing-capable device. The device that transmits the signal that is used for measurements is called the "illuminator," as its transmissions enable collection of information about the channel - that is, it illuminates the channel.
Different modes of Wi-Fi Sensing measurements are recognised - Passive, Triggered, Invoked, and Pushed, and these depend upon what triggers the illuminator device to transmit a Wi-Fi signal. Preferably the agent improves the usefulness of the standard beacon interval by using optimised timings.
In passive mode, WFS relies on transmissions that are part of regular Wi-Fi communication. The Wi-Fi Sensing receiver(s) rely only on transmissions between itself and the illuminator device(s). Passive transmissions do not introduce overhead, but the Wi-Fi sensing device lacks control over the rate of transmissions, transmission characteristics (bandwidth, number of antennas, use of beamforming), or environmental measurements.
Triggered measurement happen when a Wi-Fi Sensing device is triggered to transmit a Wi-Fi packet for the purpose of WFS measurements, either in response to a received Wi-Fi packet or by the higher layers (for instance, in WFS software).
Invoked measurement involves utilizing a packet transmission that is in response to a packet received from the Wi-Fi Sensing receiver device.
In pushed mode, a transmission is initiated by the illuminator device for measurement. A pushed transmission can be either a unicast or a multicast/broadcast message. Multicast/broadcast messages can be used for measurements by multiple WFS receivers simultaneously if the devices are not in power-save mode. Triggered transmissions introduce overhead because additional over-the-air transmissions are required. Pushed transmissions introduce less overhead compared to invoked transmissions, because the exchange is unidirectional rather than bidirectional. Triggered transmissions allow for a system to control both the rate and occurrence of measurements.
A WFS network is made up of one or more WFS illuminators and one or more WFS receivers. A WFS system is made up of three main components and that are present in Wi-Fi Sensing illuminators and receivers:

first is the Wi-Fi radio, which encompasses the radio technology specified in IEEE 802.11 standards, the interfaces and the APIs connecting the radio to the higher layers;
second is the Wi-Fi Sensing software agent, consisting of a signal processing algorithm and interfaces, the agent interacting with the Wi-Fi environment, and turning radio measurement data into motion or context-aware information; and
thirdly, an application layer operates on the Wi-Fi sensing output and forms the services or features which are ultimately presented to an end user - such as a security monitoring service provided by a security monitoring system that detects presence using WFS.

A WFS system can be built based on existing Wi-Fi standards, hardware, software and infrastructure.
The fundamental component required to enable Wi-Fi sensing on the radio is the interface to enable control and extraction of periodic channel or environmental measurement data. Regardless of device type, operating band or Wi-Fi generation, the core APIs to enable Wi-Fi sensing are similar, as the required data and control are common.
The WFS software Agent can reside on any Wi-Fi device; for example, in the infrastructure mode, the agent may reside on the AP, in which case channel measurements from all the STAs associated with the AP can be collected. The software agent may also be located on a STA. But in the security management system applications this would mean that the STA would either need to be the controller of the security management system (e.g. the CU), or would have to be reporting to the controller of the security management system (e.g. the CU). Generally, we therefore prefer to run the software agent on the CU, and given that the CU is conveniently also an access point, it makes sense for us to run the software agent on the CU acting as AP rather than merely as an STA.
The WFS software Agent uses the WFS radio APIs to interact with the Wi-Fi radio, the APIs enabling extraction of desired channel environment measurement information and providing the ability to assert any related controls to configure WFS features.
The WFS Agent has two main subsystems: Configuration and Control; and a Sensing Algorithm. The Configuration and Control subsystem interact with the radio, using a standard set of APIs. The Configuration and Control subsystem performs tasks including sensing capability identification, pushed illumination coordination, and radio measurement configuration. The sensing algorithm subsystem includes intelligence needed to extract the desired features from the radio measurement data and may differ according to the desired sensing application.
The WFS software Agent is needed on any sensing receiver but is merely optional on an illuminator - only being required if the illuminator also acts as a receiver. If included on an illuminator, only the configuration and control subsystem is needed. By having the agent on the illuminator, additional enhancements are enabled, including sensing capability identification and co-ordinated pushed illumination. If the illuminator is not running an agent, it is still technically able to participate in the sensing network, but only the most basic features that currently exist in Wi-Fi standards will be supported.
The WFS software Agent processes and analyses the channel measurement information and makes sensing decisions, such as detecting motion. This information is then shared with the application layer via the Wi-Fi Sensing agent I/O interface. As well as interfacing with the radio and the application layer, the Wi-Fi Sensing agent also interfaces with the existing Wi-Fi services on the system. This interface is necessary for the agent to provide feedback for sensing optimizations that can be used in radio resource management decisions, such as band steering or AP selection requests.
The application layer of a WFS system creates the sensing service and in effect presents the information to the end user (in our case to the security management system).
The application layer can potentially reside on any networked device: in some embodiments of the present invention, it will reside in the central unit 122 along with the WFS agent, but in other embodiments the application layer may exist in an external server or even in the central monitoring station. We prefer, however, to provide the application layer on the central unit to avoid potential problems with signalling delays (for example due to accidental or deliberate network interruption) between the central unit (or other WFS receiver) and a remotely located entity. The application layer receives input from one or multiple Wi-Fi sensing software agents. It combines the information and delivers it to the security management system which may then in turn provide it to the CMS and/or to a cloud service by means of which push notifications may be sent to a registered user device such as a smartphone - allowing users to receive real-time notifications and the ability to view historic data.
A typical Wi-Fi home network follows one of two common deployment scenarios. The first consists of a single AP that serves as the internet gateway for all the devices in the house. The second consists of multiple APs forming an ESS and extending coverage throughout the home. Depending on the use case, the Wi-Fi Sensing receiver may be the AP and/or other devices in the network. Not all the devices in a home deployment need to be Wi-Fi Sensing capable.
Wi-Fi Sensing can be deployed in all types of Wi-Fi networks and topologies, operating in different frequency bands (2.4, 5, 6, and 60 GHz) and different bandwidths. The sensing resolution and performance depends on the use case requirements. In general, it is enhanced with the increase in the number of participating devices and higher bandwidths. Applications that require lower resolutions and longer range, such as home monitoring, can be deployed using Wi-Fi networks operating in 2.4GHz and 5GHz. Applications that require higher resolutions and lower range, such as gesture recognition, require 60GHz Wi-Fi networks.
In multi-AP and/or multi-band deployments, there may be an advantage to having a Wi-Fi sensing device connected to a specific AP or operating in a specific frequency band. Radio resource management (RRM) events, such as AP and/or band steering, should be conducted in coordination with the Wi-Fi Sensing agent/operation.
The devices involved with Wi-Fi Sensing will depend upon the deployment environment and the specific use case. The sensing measurements also need to be processed by the device with enough computation power. The coordination of sensing, including participating devices, is a role particularly suited to an AP. Typically the central unit of a security monitoring system will have ample processing power, as well as being able to function as an AP, to handle this task efficiently and speedily.
The nature of Wi-Fi networks is such that it should be possible able to add additional Wi-Fi sensing capable devices to the network to enhance accuracy, coverage and/or localization. These additional devices do not necessarily need to be Wi-Fi Sensing capable or dedicated Wi-Fi sensing devices to participate; however, optionally they may also identify their Wi-Fi sensing capabilities and supported features to the AP. Internet of Things (IoT) devices for home deployment can typically also be used as part of a WFS installation supporting a WFS-enabled security monitoring system: example include Wi-Fi controllable plugs and sockets, light bulbs, thermostats, smart speakers, and video door bells. However, even when a device connects to the AP and reports that it is Wi-Fi sensing capable, the Wi-Fi Sensing agent may elect not to make use of that device.
WFS for a security monitoring system may be run over a dedicated Wi-Fi network, the premises having at least one other Wi-Fi network for other purposes. But for reasons of simplicity and economy it may often be preferred to operate a single Wi-Fi network to serve all a household's (or small business's) needs including WFS for a security monitoring service. If a single-network solution is adopted, performance degradation due to airtime usage and sensing overhead must be minimized and hence Wi-Fi transactions required for conducting sensing measurements and sensing management and processing must be optimized for efficiency.
For each Wi-Fi Sensing application, at least one network device executes the sensing software, or Wi-Fi Sensing Agent. The Wi-Fi Sensing agent is typically placed on the AP, but it can be placed on any STA (although, as previously mentioned, we prefer to run the Wi-Fi Sensing agent on the AP). Following authentication and association of a device with the Wi-Fi network, the Wi-Fi Sensing agent should discover the device and its sensing capabilities. Depending on the capabilities of the device, its role in the Wi-Fi sensing network would be determined. If the new device is another Wi-Fi Sensing-capable AP, then coordination among the agents is required.
The WFS agent needs to have a mechanism to determine which devices are capable and needs to participate in the sensing for each application on a device-specific basis.
A WFS agent also needs to be capable of configuring the radio for measurements and triggering transmissions on a periodic basis for sensing measurements, and to enable/disable measurements or adjust configuration parameters for Wi-Fi sensing-capable devices. Optionally, the Wi-Fi Sensing agent is also able to request specific radio resource management operations, such as AP or band steering. The WFS agent is also preferably able to detect and process specific sensing events and communicate the relevant information to the application layer (e.g., the security monitoring system) for specific handling and user presentation.
One of the parameters that impacts the quality of the received signal in a wireless network is the amount of interference present. Interference can be caused by other Wi-Fi devices operating in the same band, which causes cochannel interference, or in an adjacent channel, which causes adjacent channel interference. It can also be caused by non-W-Fi devices, which can be other communication systems or unintentional transmissions that create electromagnetic noise in the band. Interference can impact Wi-Fi Sensing performance in two ways. Firstly, it may interfere with the sensing transmissions and thereby reduce the number of measurements made in a given time interval. As such, it introduces jitter in time instants during which the measurements are made. Secondly channel-state measurements may capture the impact of transient interference, such as for a non-Wi-Fi device, as opposed to motion in the environment.
Wireless systems deploy various techniques to avoid or reduce the impact of interference, and these techniques also help to improve WFS performance. These techniques aim at maximizing the reuse of spectrum, while minimizing the overlap of spectrum used by nearby networks: for example, Dynamic Channel Allocation (DCA); Auto Channel Selection (ACS); optimized RF planning; (e.g., non-overlapping channels and use of reduced channel width when applicable), and power control.
As already mentioned, increasing the number of illuminators may result in a higher sensing performance: with more transmitters that are located sufficiently apart from one another, motion in a larger area can be detected; when motion is detected using transmissions on one or more transmitters, information is provided that can be used to determine localization of the motion; and sensing accuracy is improved with a higher number of measurements taken across a larger number of transmitters in most scenarios.
The IEEE 802.11a preamble is useful for Wi-Fi Sensing. The preamble contains a short training field (STF), a guard interval and a long training field (LTF). The STF is used for signal detection, automatic gain control (AGC), coarse frequency adjustment and timing synchronization. The LTF is used for fine frequency adjustment and channel estimation. Since only 52 subcarriers are present, the channel estimation will consist of 52 frequency points. Newer OFDM PHY versions (HT/VHT/HE) maintain the IEEE 802.11a preamble for backward compatibility and refer to it as the legacy preamble. The legacy preamble spans a 20MHz bandwidth and consists of a legacy STF (L-STF) and legacy LTF (L-LTF). As more recently defined OFDM PHY versions (HT/VHT/HE) introduce wider channel bandwidths (up to 160MHz) for backward compatibility, the legacy preamble is duplicated on each 20MHz channel. This allows the receiver to compute 52, 104, 208 or 416 valid L-LTF frequency points, which represent the channel estimation between the two devices. Also potentially useful for Wi-Fi Sensing are the MIMO training fields present in HT, VHT and HE LTFs. The MIMO fields are modulated using the full bandwidth (20MHz to 160MHz) and are traditionally used by the receiver to estimate the mapping between the constellation outputs and the receive chains. Since these fields span the full bandwidth, they provide more frequency points. For example, a 20MHz L-LTF contains 52 subcarriers, while a 20MHz HT/VHT-LTF contains 56 subcarriers. The latest introduction of the HE PHY has the potential to enhance Wi-Fi Sensing. In addition to enabling operation in the 6GHz spectrum, the HE PHY has increased the number of subcarriers per 20MHz bandwidth by 4x, which effectively allows for better object resolution.
The IEEE 802.11ad amendment defines a Directional-Multi-Gigabit (DMG) PHY for operation in the 60GHz band. While there are three different modulation schemes (Control, Single-Carrier and OFDM) defined, Control and the Single Carrier PHY are the primary PHY used in 802.11ad (and is also part of the subsequent 802.11ay amendment). Regardless of the modulation scheme, every packet starts with a preamble that consists of a short training field (STF) and a channel estimation field (CEF). The STF is used for timing estimation and AGC adjustment. CEF is used for channel estimation. Similar to the OFDM-based PHYs, the necessary channel estimation for Wi-Fi Sensing is available following successful reception and processing of the preamble of a packet and can be provided to the higher layers. The wide channel bandwidth available in 802.11ad/ay can significantly improve the performance of Wi-Fi Sensing in terms of the resolution; however, the limited communication range in 60GHz band restricts the sensing range and coverage. As such, in many situations the central unit of a security monitoring system may relay instead on frequency bands with longer range, sufficient to cover the majority of households. However, for smaller-scale installations the use of the 60GHz band may be attractive and therefore embodiments of the invention may use this band for WFS.
When it comes to identifying peer devices in a WFS installation, the MAC layer mechanisms may be used to obtain information about the connected devices and the roles they play in Wi-Fi sensing. The MAC layer also initiates and drives transmissions required for channel estimation among the devices in the Wi-Fi Sensing network.
Various aspects of peer identification arise with Wi-Fi Sensing. The first is identifying the devices and the channel estimation mapped to the physical environment between any two devices. Typically, an STA is identified by a 48-bit MAC address. A MAC address is sufficient identification for STAs associated with a Wi-Fi network; however, if the association is lost during the lifetime of the application, then randomized MAC addresses may be used. In this case, a different or more involved mechanism would be required to identify each STA. This identification must match the corresponding channel estimate measurement obtained from the PHY. The second is identifying the device network role and its connection type, such as whether it is an AP or an STA, or whether it is part of a mesh or a P2P connection. This information is used by the Wi-Fi Sensing agent to decide the best method for conducting measurements.
The third aspect is the identification of WFS device capabilities, such as sensing capabilities, supported measurement rate, and the availability and willingness of the device to participate in sensing measurements. This information is required from all devices in the network for the Wi-Fi Sensing agent to select devices participating in the sensing measurements.
As already noted, there are different types of transmissions that can be used for illumination of the Wi-Fi channel and obtaining measurements between two devices. Passive transmissions rely on existing Wi-Fi traffic and do not introduce any new MAC layer requirements. Triggered transmissions, however, rely on additional transmissions. Depending on whether existing packet exchange procedures are used for triggered transmissions or new exchanges are defined, the requirements on the MAC layer will be different. An example of one existing packet exchange that can be used for triggering invoked transmissions is null data packet (NDP) and ACK exchange. NDP transmission by the Wi-Fi Sensing receiver can be used to invoke a Wi-Fi Sensing transmitter to respond with an ACK, which may then be used to compute a channel estimation. The disadvantage of using ACK packets for channel estimation, in 2.4/5GHz bands, is that the ACKs are only transmitted in legacy mode. Another example of how an invoked measurement can be triggered is by use of the implicit unidirectional beamforming procedure, first defined in the IEEE 802.11n standard. In this procedure, an STA requests beamforming training by sending a MAC frame with the training request (TRQ) bit set to 1. This triggers the receiving device to send an NDP announcement, followed by an NDP to illuminate the channel. The benefit of this invoked measurement is that it is not limited to the legacy preamble for channel measurements and uses the MIMO training fields, as well.
In pushed measurements, a transmission is triggered by the illuminator to be received by one or multiple Wi-Fi Sensing receivers. Beacon frames are an example of using existing MAC packet exchanges for pushed measurements.
Also as already noted, to support different use cases, either the AP or STA may take the role of sensing receiver; additionally, there may be multiple sensing receivers required to support the application. Moreover, there may be multiple illuminators involved in the measurements. MAC layer coordination is used to coordinate the sensing transmissions among the illuminators and the sensing receivers in an efficient way. MAC layer scheduling may also be used to enable periodic measurements on which some use cases rely. Coordination and scheduling at the MAC layer should enable different options for conducting sensing measurements among multiple illuminators and sensing receivers, with minimal added overhead, while accounting for the power save state of the devices.
To interact with the MAC and PHY, the WFS agent has an interface to pass the WFS control information to the radio and extract the measurement data. The interface should be PHY agnostic and is, therefore, defined in a generic manner and extendable to cover different radio driver implementations, including drivers from different chipset vendors. The interface definition should allow for potential additional features or capabilities provided by a specific PHY or a chipset, as well as a path for growing the technology. Definition of a standard interface/API enables radio firmware and driver developers to ensure compliance and enables reuse of components or common codes, which may be placed into a library. Most Wi-Fi drivers are based on either the wireless-extensions framework or the more recent and actively developed cfg80211 / nl80211 framework. As the system integration components are largely provided, these frameworks enable Wi-Fi driver developers to focus on the hardware aspects of the driver. These frameworks also offer significant potential as a location for defining a WFS API. The WFS interface should provide the WFS agent with STA identification and enable the WFS agent to track the physical device in the network (i.e., the AP to which it is connected), as well as the device's capability and availability to participate in the measurements.
The WFS agent requires control of the STAs that will participate in the sensing measurements, as well as what measurement type (passive vs triggered) will be performed. The WFS interface should provide such control, either on a global system scale or on a per STA basis so that the WFS agent can conduct WFS measurements in the most efficient manner.
Based on the specific WFS application or use case, different measurement rates may be required. The measurement rate is typically decided by the WFS agent, and the interface should support its control. However, to provide the lowest jitter and best efficiency possible, it is best to rely on the MAC layer for scheduling. WFS applications may have different measurement parameter requirements (bandwidth, antenna configuration, etc.). The configuration of measurement parameters allows the application to obtain only the data it requires to maintain efficiency. The measurement parameters should be configurable independently for each STA.
The WFS interface should be flexible enough for the radio to specify whether the data payload is in time-domain or frequency-domain, the numerical format, etc. By having this knowledge, the Wi-Fi Sensing agent can correctly interpret the data.

Claims

A security monitoring system for premises, the system including:
a Wi-Fi access point of a Wi-Fi network, the Wi-Fi network including a plurality of Wi-Fi stations, the Wi-Fi access point and the plurality of stations being configured to operate a radio-based location sensing arrangement to detect human presence and location based on detecting perturbations of radio signals of the Wi-Fi network, one of the stations of the Wi-Fi network being a motion-triggered video camera, the video camera being configured to transmit a notification to a remote entity in the event that it is triggered,

the Wi-Fi access point being configured to:
respond to the transmission of a notification by the video camera by using the radio-based location sensing arrangement to determine whether and where there is human presence within the premises; and, in the event that human presence is detected,

generate a report, that includes details of where and when human presence has been detected;

and to notify the report to one or more designated recipients.
The security monitoring system of claim 1, wherein the Wi-Fi access point is configured to receive a push notification from the remote entity in the event of a notification being transmitted to the remote entity as a result if the video camera having been triggered.
The security monitoring system of claim 1, wherein the Wi-Fi access point is configured to recognise notifications sent by the video camera to the remote entity.
The security monitoring system of any one of the preceding claims, wherein the Wi-Fi access point has access to a database of locations within the premises.
The security monitoring system of claim 4, wherein the Wi-Fi access point is configured to use the database to determine whether a detected human presence is within a location included in the database.
The security monitoring system of any one of the preceding claims, wherein the Wi-Fi access point has access to a calendar or timetable for use in determining whether detected human presence should be notified or not.
A method performed by a premises security monitoring system, the system including:
a Wi-Fi access point of a Wi-Fi network, the Wi-Fi network including a plurality of Wi-Fi stations, the Wi-Fi access point and the plurality of stations being configured to operate a radio-based location sensing arrangement to detect human presence and location based on detecting perturbations of radio signals of the Wi-Fi network, one of the stations of the Wi-Fi network being a motion-triggered video camera, the video camera being configured to transmit a notification to a remote entity in the event that it is triggered,

the method comprising:
the video camera transmitting a notification to the remote entity upon being triggered;

the Wi-Fi access point, in response to the transmission of the notification by the video camera, determining using the radio-based location sensing arrangement whether and where there is human presence within the premises; and, in the event that human presence is detected,

generating a report, that includes details of where and when human presence has been detected; and

notifying the report to one or more designated recipients.
The method of claim 7, wherein the determining whether and where there is human presence within the premises is triggered by the Wi-Fi access point receiving a push notification from the remote entity in the event of a notification being transmitted to the remote entity as a result if the video camera having been triggered.
The method of claim 7, wherein the determining whether and where there is human presence within the premises is triggered by the Wi-Fi access point recognising a notification sent by the video camera to the remote entity.
The method of any one of claim 7 to 9, further comprising the Wi-Fi access point accessing a database of locations within the premises and using the database to determine whether a detected human presence is within a location included in the database.
A method of detecting the presence of an intruder in premises, the method comprising:
detecting an event at the premises using a non-alarm sensor, such as a "convenience" camera;

receiving notification of the event and, based on receiving the notification, checking a radio-based location sensing arrangement to detect human presence and location based on detecting perturbations of radio signals to determine whether there is a human presence where no human presence should be, and if such human presence is detected creating an alert.