CN111788496A

CN111788496A - Systems and methods for occupancy sensing using multiple modalities

Info

Publication number: CN111788496A
Application number: CN201980016764.0A
Authority: CN
Inventors: A·莫蒂; R·库玛; 张玉婷; C·卡博塔耶夫; E·B·沈
Original assignee: Signify Holding BV
Current assignee: Signify Holding BV
Priority date: 2018-03-02
Filing date: 2019-02-25
Publication date: 2020-10-16
Also published as: JP2021513171A; US11860268B2; EP3759516A1; WO2019166354A1; US20210041523A1; JP6921332B2; EP3759516B1

Abstract

A system and method for determining the number of occupants at a location using multiple modalities. The method includes collecting a first data set with a motion sensor of a lighting system. The second data set is collected using a transceiver of the RF subsystem. First and second estimates are computed from the second data set using first and second algorithms. The first estimate and the second estimate are fused to create a fused occupant estimate. Training the first algorithm, the second algorithm, or both the first algorithm and the second algorithm by: the second occupant estimate and/or the second data set are input to recalibrate parameters of the first algorithm, and/or the first occupant estimate and/or the first data set are input to recalibrate parameters during training of the second algorithm. The building control system may operate in response to the fused occupant estimates.

Description

Systems and methods for occupancy sensing using multiple modalities

Technical Field

The present disclosure relates generally to a system and method for determining a number of occupants in a location, which may be particularly useful for a control system operating the location in response to the determined number of occupants.

Background

Automated building functions (e.g., heating, ventilation and air conditioning, or HVAC systems, lighting systems, etc.) can be used to optimize occupant comfort and minimize energy usage, and thus cost, of maintaining the building. For example, a Passive Infrared (PIR) sensor is a cost-effective solution, which is typically deployed in a building to control one or more systems in the building. PIR sensors are commonly used for: automatically controlling when to turn the lighting fixture on and/or off depending on whether the PIR sensor detects motion. While these sensors can detect whether a location is occupied, they rely on a line of sight between the occupant and the sensor and are not suitable for determining how many occupants are at the location.

The use of Radio Frequency (RF) waves has also been utilized to detect motion. In one system, RF waves have been used to detect and triangulate the number of cell phones (or other transceivers) in a location, which can be used as a simulation of the number of occupants at the location. However, these methods require the individual to carry a cell phone or other device, which is limiting for producing an accurate occupant count. In another approach, RF signals reflected from humans are detected and analyzed to measure the occupancy of a space. However, these RF systems may suffer from multipath reflections and close range problems, where occupants farther away are hidden by reflections from occupants closer.

Accordingly, there is a continuing need in the art for a system that: the system improves the ability of the system to accurately determine the number of occupants in a location, particularly in systems where the control system controls the location in response to the determined number of occupants.

Disclosure of Invention

The present disclosure relates to an inventive system and method for determining the number of occupants in a location, and more particularly for operating a control system of the location in response to the determined number of occupants.

The disclosed system may include both: a motion detector subsystem comprising one or more motion sensors, such as a lighting system with one or more embedded PIR sensors; and a Radio Frequency (RF) subsystem including one or more RF transceivers, such as a network router. Data collected by the RF transceiver is used to generate a first occupant estimate using a first algorithm, and data collected by the motion sensor is used to generate a second occupant estimate using a second algorithm. The estimates produced by these two sensor modalities are fused to produce an accurate occupant count at a location. The first and second algorithms may be trained by using data and/or estimates related to each subsystem as input to algorithms associated with other subsystems, thereby further improving their respective accuracy over time. Accurate occupant estimation can be used to operate the control system of the location, such as to provide better or more efficient lighting, temperature, ventilation and space optimization, thereby maximizing the energy efficiency and occupant comfort of the building.

In general, in one aspect, there is provided a method for determining a number of occupants at a location using a plurality of modalities, the method comprising: collecting a first data set from one or more motion sensors embedded in a lighting system in the location; calculating a first occupant estimate from the first data set using a first algorithm associated with a lighting system; collecting a second data set from one or more Radio Frequency (RF) transceivers of an RF subsystem in the location; calculating a second occupant estimate from the second data set using a second algorithm associated with the RF subsystem; fusing the first occupant estimate and the second occupant estimate to create a fused occupant estimate corresponding to a number of occupants at the location; training the first algorithm, the second algorithm, or both the first algorithm and the second algorithm by performing at least one of: (i) input the second occupant estimate, the second data set, or both to recalibrate parameters of a first algorithm, and (ii) input the first occupant estimate, the first data set, or both to recalibrate parameters of a second algorithm.

According to an embodiment, the method further comprises: operating the building control system in the location in response to the fused occupant estimates. According to an embodiment, a building control system includes a security system, a Heating Ventilation and Air Conditioning (HVAC) system, a sound masking system, a lighting system, or a combination comprising at least one of the foregoing. According to an embodiment, the one or more motion sensors include a Passive Infrared (PIR) sensor and the one or more RF transceivers include a Wi-Fi enabled router. According to an embodiment, collecting the second data set comprises: the method includes transmitting RF waves with at least one RF transceiver and receiving reflections of the RF waves with the at least one RF transceiver.

According to an embodiment, prior to collecting the first or second data set, the training further comprises: inputting data representing a physical layout of the location; inputting data representing coordinates of each of the one or more RF transceivers; inputting data representing coordinates of each of the one or more motion sensors, or a combination comprising at least one of the foregoing. According to an embodiment, the second data set includes data representing RF reflections from distant occupants reflected by RF reflections of closer occupants [ CPA _ ma1], and the training includes: simultaneously or synchronously comparing the coordinates of each of the first data set and one or more motion sensors with the second data set to locate the location of the remote occupant. According to an embodiment, the first algorithm comprises a function fitted to a plurality of data points describing the number of motion sensors triggered with respect to a true occupant count in the location, and the training comprises: synchronously comparing the first data set with the second data set to form one or more new data points, wherein a true occupant count is set as the second occupant estimate; and recalculating the function after including the one or more new data points in the plurality of data points.

According to an embodiment, training the first algorithm further comprises: establishing a proxy model and simulating how many of the one or more motion sensors are triggered in response to different real occupancy conditions; and determining a function that maps the number of triggered sensors to a true occupancy.

According to an embodiment, the fusing comprises calculating the fused occupant estimate according to the following equation:

where N is the fused occupant estimate, N_MDIs the first occupant estimate, N_RFIs the second occupant estimate, V_MDIs a first variance associated with the lighting system, and V_RFIs a second party associated with the RF subsystemAnd (4) poor.

In general, in one aspect, a controller for operating a building control system is provided. The controller includes: a communication module configured to receive a first data set from a lighting system having one or more motion sensors and a second data set from a Radio Frequency (RF) subsystem having one or more RF transceivers; a memory having stored therein a first algorithm associated with the lighting system and a second algorithm associated with the RF subsystem; a processor configured to calculate a first occupant estimate from the first data set and a second occupant estimate from the second data set using a first algorithm, and train the first algorithm, the second algorithm, or both the first and second algorithms by performing at least one of: (i) input the second occupant estimate, the second data set, or both, to recalibrate parameters of a first algorithm, and (ii) input the first occupant estimate, the first data set, or both, to recalibrate parameters of a second algorithm; and a fusion module configured to create fused occupant estimates by fusing the first occupant estimate and the second occupant estimate; wherein the controller is configured to control operation of the building control system in response to the fused occupant estimation.

In general, in one aspect, a system for determining a number of occupants at a location is provided. The system comprises: a lighting system comprising one or more motion sensors, the lighting system configured to collect a first data set using the one or more motion sensors; a Radio Frequency (RF) subsystem comprising one or more transceivers, the RF subsystem configured to collect a second data set utilizing the one or more transceivers; a controller configured to determine a first occupant estimate from a first data set using a first algorithm associated with a lighting system and a second occupant estimate from a second data set using a second algorithm associated with an RF subsystem, wherein the controller is configured to train the first algorithm by inputting the second occupant estimate, the second data set, or both, to recalibrate parameters of the first algorithm, train the second algorithm by inputting the first occupant estimate, the first data set, or both, to recalibrate parameters of the second algorithm, or a combination comprising at least one of the foregoing; and a fusion module configured to create fused occupant estimates by fusing the first occupant estimate and the second occupant estimate.

According to an embodiment, the system further comprises a building control system configured to operate in response to the fused occupant estimation. According to an embodiment, a building control system includes a security system, a Heating Ventilation and Air Conditioning (HVAC) system, a sound masking system, a lighting system, or a combination comprising at least one of the foregoing. According to an embodiment, the one or more motion sensors are passive infrared sensors and the RF subsystem includes at least one network router including a transceiver.

It should be appreciated that all combinations of the foregoing concepts and additional concepts discussed in greater detail below (provided such concepts are not mutually inconsistent) are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein.

Drawings

In the drawings, like reference characters generally refer to the same parts throughout the different views. Moreover, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention.

FIG. 1 schematically illustrates a system for determining the number of occupants in a location and operating a control system of the location in response to the number of occupants, according to one embodiment disclosed herein.

Fig. 2 schematically illustrates a light facility with a PIR sensor that may be used to form a LOS subsystem according to one embodiment disclosed herein.

FIG. 3 is a block diagram illustrating additional components of the system of FIG. 1 in accordance with one embodiment of the present Chinese disclosure.

FIG. 4 is a block diagram illustrating a means for using a proxy model to create a mapping function for estimating occupancy of a location according to one embodiment disclosed herein.

Fig. 5 is a graph illustrating how a mapping function for the graph of fig. 4 may be created according to one embodiment disclosed herein.

FIG. 6 is a graph that may be used as part of an error analysis in accordance with one embodiment disclosed herein.

FIG. 7 is a flow chart illustrating a method for operating a system, such as the systems of FIGS. 1 and 3, and for determining the number of occupants in a location and operating a control system of the location in response to the number of occupants, according to one embodiment disclosed herein.

Fig. 8 is a graph illustrating how data from the RF subsystem may be used to create new data points for recalculating the function used by the motion detection subsystem according to one embodiment disclosed herein.

Detailed Description

The present disclosure describes various embodiments of systems and methods for controlling the operation of a control system for a location. More generally, applicants have recognized and appreciated that it would be beneficial to control operation of a control system in a location based on the number of occupants in the location. It is a particular object of certain embodiments of the present disclosure to accurately determine the number of occupants in a location to increase the efficiency and/or effectiveness of a control system for that location.

In view of the foregoing, various embodiments and implementations relate to a system and method for determining a number of occupants in a location and operating a control system of the location in response to the number of occupants. The disclosed system may include both: a motion detector subsystem comprising one or more motion sensors, such as a lighting system with one or more embedded PIR sensors; and a Radio Frequency (RF) subsystem including one or more RF transceivers, such as a network router. Data collected by the RF transceiver is used to generate a first occupant estimate using a first algorithm, and data collected by the motion sensor is used to generate a second occupant estimate using a second algorithm. The estimates produced by these two sensor modalities are fused to produce an accurate occupant count at a location. The first and second algorithms may be trained by using data and/or estimates related to each subsystem as input to algorithms associated with other subsystems, thereby further improving their respective accuracy over time. Accurate occupant estimation can be used to operate the control system of the location, such as to provide better or more efficient lighting, temperature, ventilation and space optimization, thereby maximizing the energy efficiency and occupant comfort of the building.

Referring to FIG. 1, in one embodiment, a system 100 is provided to determine a number of occupants (e.g., a number of people) within a location 102 using multiple modalities. Operation of certain functions or features of the location may be controlled in response to the determined number of occupants. The system 100 includes a Radio Frequency (RF) subsystem, referred to herein by reference numeral 104, and a motion detector subsystem, referred to herein by reference numeral 106. As will be described in more detail below, the RF subsystem 104 and the motion detector subsystem 106 are used together to determine the number of occupants in the location 102. The term "occupant" may be used interchangeably herein with "individual" and these terms are intended to refer primarily to a person, but it should be appreciated that these terms may instead refer, in some embodiments, to animals, insects, etc., or even non-biological entities that move into, out of, and/or around the environment (e.g., due to wind, water currents, etc.).

In fig. 1, location 102 is illustrated as an office space having desks, workstations, conference rooms, and the like, but it should be appreciated that any other indoor or outdoor area may be monitored. The system 100 may include a control system 105, or more specifically, if the location 102 is a building (e.g., an office space), the control system 105 may be referred to as a building control system. For example, the control system 105 may be or include a Heating Ventilation and Air Conditioning (HVAC) system, a sound masking system, a lighting system, a security system, or any other system or function useful to the location 102.

The RF subsystem 104 includes one or more transceivers capable of transmitting and receiving Radio Frequency (RF) waves. A transceiver refers to any device or combination of devices (e.g., separate transmitter and receiver) capable of transmitting and receiving RF waves. In fig. 1, the locations of four such transceivers are indicated by reference characters A, B, C and D. In one embodiment, the transceiver of the RF subsystem 104 is or includes a Wi-Fi enabled router. It should be appreciated that other radio frequency based communication or signal generation and reception systems may be implemented via any combination of related hardware and/or software known or developed in the art.

It should be appreciated that any RF-based detection technique may be used for the subsystem 104. For example, RF waves have been used in the art to identify the motion of an individual based on a transceiver (such as a smartphone) held by the individual. It has also been found that RF waves can be used to track people throughout a location based on reflections of the RF waves transmitted and subsequently received by the transceiver, as discussed in more detail below. Advantageously, RF transceivers in the form of network Wi-Fi routers are ubiquitous in many buildings and are therefore well suited to forming RF subsystems 104 in many common environments.

The motion detector subsystem 106 in fig. 1 includes sixty-five motion sensors, which are labeled with numbers 1 through 65 in this figure. As used herein, "motion sensor" refers to any device or technology that detects an object or object motion within a direct line of sight or field of view of the sensor. It should be appreciated that motion may be determined based on various parameters indicative of motion detected by the sensors. For example, many common motion detectors detect motion based on a sensed thermal difference between a moving object and the surrounding environment. In one embodiment, the motion sensor comprises a Passive Infrared (PIR) sensor, although other motion sensors may be used, such as a camera or other sensor capable of receiving visible light signals.

Lighting systems are ubiquitous infrastructures in buildings and office spaces. So-called "intelligent" lighting systems are characterized by: one or more luminaires equipped with Light Emitting Diodes (LEDs) or other controllable light sources, which may be connected to each other and/or to other network devices via an ethernet or wireless network. The luminaire also has PIR or other sensors for controlling the operation of the lamps in an energy efficient manner (e.g., a sensor that enables the lamps to be automatically turned on/off depending on whether there is detected movement). The connectivity enables individual luminaires to work together to maximize energy efficiency, and enables remote monitoring and predictive maintenance of the system. Advantageously, existing lighting systems of this type with embedded PIRs or other motion sensors may be used to form the motion detector subsystem 106. Other existing systems with motion sensors (such as security systems, etc.) may alternatively or additionally be utilized, or the motion sensors may be deployed exclusively for purposes of forming the motion detector subsystem 106.

Fig. 2 shows an example of a motion sensor enabled device in the form of a ceiling mounted light fixture (or luminaire) 106a with an embedded PIR sensor that enables the light fixture 106a to be turned on/off depending on detected motion. The lighting system may comprise one or more light fixtures 106 a. The PIR sensor has a field of view 112, which is generally conical or pyramidal in shape, having a height H from the PIR sensor. The light fixtures 106a and/or other motion sensors used by the subsystem 106 may additionally or alternatively include the ability to distinguish between different types of movement. For example, a motion sensor may be able to distinguish between "primary" movement (e.g., whole body movement) and "secondary" movement (e.g., movement of only a body extremity), such as via a relative detected size and/or detected speed of movement of a moving object.

Additionally, the light facility 106a or other motion sensors of the motion detector subsystem 106 may be capable of identifying a plurality of different physical regions or zones, such as a first zone 114 bounded by X1 and Y1 and a second zone 116 bounded by X2 and Y2 in fig. 2 (e.g., identified by using a plurality of sensors as is known in the art). The zones may be arranged in any pattern, such as a grid, concentric circles, and the like. In this manner, each motion sensor may define one or more separate regions of location 102. The separate regions may be combined to create a more general region corresponding to a larger area of the location 102. For example, referring back to fig. 1, the location 102 is generally separated into four distinct regions represented by dashed lines, although it should be appreciated that the location 102 may be any other number of regions. In this manner, the motion sensors can be used not only to determine the total number of occupants, but also to determine the relative position or orientation of the occupants. Additionally, the control system 105 may use this information to enable, disable, or alter the functionality of its components only in certain areas (e.g., reduce the temperature in one zone while maintaining the temperature in all other zones).

The system 100 may also include a controller 110, the controller 110 having a processor 107, a memory 108, and/or a communication module 109. The controller 110 may be utilized to store data collected by the subsystems 104 and 106 (e.g., stored in the memory 108) and/or to calculate occupancy based on the collected data (e.g., utilizing the processor 107). In one embodiment, the controller 110 is also used to control components of the control system 105 (e.g., an HVAC system). Alternatively, the control system 105 may include a separate controller similar to the controller 110 that is in communication with the controller 110. As should be appreciated in view of the above description, elements of the various systems and subsystems may be shared (e.g., the control system 105 may control operation of Wi-Fi enabled routers forming the subsystem 104, or control operation of a lighting system including PIRs or other sensors forming the subsystem 106). Controller 110 may be part of any of subsystems 104 and/or 106, control system 105, or separate from but in communication with these systems and subsystems. It should be appreciated that multiple controllers may be used in place of the single controller 110, for example, the subsystem 104 and the subsystem 106 may have separate controllers in communication with each other. The transceivers of the subsystem 104, the sensors of the motion detector subsystem 106, the components of the control system 105, and the controller 110 may communicate with or among each other via any wired or wireless communication technology (e.g., bluetooth, Wi-Fi, Zigbee, ethernet, etc.).

The processor 107 may include any suitable form of device, mechanism, or module configured to execute software instructions, such as a microcontroller, multiple microcontrollers, circuitry, a single processor, or multiple processors. Memory 108 may include any suitable form(s), including non-volatile memory or volatile memory. Volatile memory can include Random Access Memory (RAM). The non-volatile memory may include Read Only Memory (ROM), flash memory, Hard Disk Drive (HDD), Solid State Drive (SSD), or other data storage medium. The memory 108 may be used by the processor 107 for temporary storage of data during its operation. Data and software, such as data collected by

subsystems

104 and 106, as well as algorithms, operating systems, firmware, or other data or applications discussed below, may be installed or stored in memory 108. The communication module 109 may be or include any transmitter, receiver, antenna, radio or other communication device, mechanism or technology, and software configured to enable operation thereof.

Fig. 3 includes a block diagram from which further aspects of the operation and structure of system 100 may be appreciated. To determine the number of occupants, the system 100 may include a first algorithm 118 (or "RF algorithm 118") and a second algorithm 120 (or "motion detector algorithm 120"), the first algorithm 118 being constructed and/or trained to estimate occupancy of the location 102 based on a first set of data (or "RF data") measured by the RF subsystem 104 (e.g., data corresponding to reflected RF waves), the second algorithm 120 being constructed and/or trained to estimate occupancy of the location based on a second set of data ("motion data") measured by the motion detector subsystem 106 (e.g., data corresponding to motion detected in the field of view of each motion sensor). In one embodiment, the first and/or

second algorithms

118 and 120 are or employ machine learning algorithms. It should be appreciated that any number of machine learning systems, architectures, and/or techniques may be utilized, such as artificial neural networks, deep learning engines, and so forth.

To build and/or train the RF algorithm 118, the layout of the location 102 (e.g., data describing the physical layout of the location 102, such as the boundaries of different regions, the location of each table or workstation, etc.) may be provided as input to the RF algorithm 118. Additionally, the RF algorithm 118 may receive as input the location or coordinates of each Transceiver (TX) of the RF subsystem 104. Similarly, the motion detector algorithm 120 may receive as input the layout of the locations 102, and the locations or coordinates of the sensors of the motion detector subsystem 106. The location coordinates may be provided according to any reference coordinate system. For example, if a motion sensor is embedded as part of the luminaire (e.g., as discussed with respect to light fixture 106 a), this information may be determined from a commissioning database of the lighting system. The location of other noteworthy features (such as a table, a particular area, etc.) may also be set using the same coordinate system.

In operation, the algorithm 118 may be utilized to calculate a first occupant estimate 122 (or RF-based estimate 122) based on RF data measured by the RF subsystem 104 and to estimate a second occupant estimate 124 (or motion-based estimate 124) based on motion data measured by the motion detector subsystem 106. As discussed in more detail below, the

estimates

122 and 124 may be used to help train the motion detector algorithm 120 by providing an RF-based estimate 122 and to help train the RF algorithm 118 by providing a motion-based estimate 124, thereby helping to enhance the performance of the

algorithms

118 and 120. Additionally, as also discussed in more detail below, the

estimates

122 and 124 may be fused or combined at a fusion module 126 to produce a final fused occupancy count or estimate. In one embodiment, the controller 110 includes a fusion module 126, and the fusion module 126 may be implemented via, for example, software installed in the memory 108 of the controller 110. If the reinforcement is performed as part of the fusion process, the controller 110 may be configured to perform the reinforcement, for example, via the fusion module 126.

The fused occupant estimates may be sent to a control system of the location (e.g., control system 105 of location 102) to enable, disable, and/or otherwise modify the functionality or operation of components of the control system (e.g., increase or decrease temperature in response to a changing number of occupants, turn on/off ventilation fans, change the intensity of sound masking systems, etc.). As noted above, the

estimates

122 and 124, and thus the fused estimates, may associate occupants with different coordinates or zones to enable the control system 105 to separately and/or differently control operations in each zone.

It is noted that in both training and operation, the inputs to the RF and

motion detector algorithms

118 and 120 may additionally be developed by data from the location 102 and/or

subsystems

104 and 106, depending on the particular configuration of the system 100. In one embodiment, the RF subsystem 104 is, includes, or is arranged using the architecture and/or principles of a WiTrack System developed by the Massachusetts institute of technology. In this embodiment, the RF subsystem 104 will operate by transmitting RF signals and capturing their reflections from the human body. As generally described below, occupant estimates will be generated based on data received from reflected RF waves.

In one non-limiting example, the RF algorithm 118 may retrieve data received by the RF subsystem 104 by processing signals from a transceiver (e.g., a receiver antenna) to track occupant movement. First, time of flight (TOF) can be measured as: the time it takes for a signal to travel from the transceiver (e.g., transmit antenna) of the RF subsystem 104 to the reflected body, and then back to the transceiver (e.g., receive antenna) of the RF subsystem 104. An initial measurement of TOF may be obtained using a Frequency Modulated Carrier Wave (FMCW) transmission technique. The estimate can be cleaned to eliminate sudden jumps due to noise and multipath effects. Once the TOF is determined, as perceived from each transceiver (e.g., receiving antenna), the geometric placement of the transceivers (e.g., based on the coordinate inputs noted above) can be utilized to locate the moving body in three dimensions. Additionally, this type of system can be used to detect falls by monitoring rapid changes in the height of an individual or subject, as well as the final height after the change. These systems can also be used to distinguish between secondary movements, such as distinguishing between arm movements and body movements as a whole.

The algorithm 120 may be similarly constructed and used according to the particular needs of the algorithm 120, for example to include simulations or field experiments that enable the algorithm 120 to correlate sensed motion detection data of the motion detector subsystem 106 into occupant estimates. In one particular non-limiting example, it may be assumed that the occupancy of an area may be measured based on the number of people using the space, such as via the number of desks occupied in an open office space. In this example, let X = X₁，…，x_NIndicating "N" motion sensors in that location (i.e., subsystem 106), and Y = Y₁，…，y_MIndicating "M" occupied tables (i.e., the estimated number of occupants). The motion sensor may be configured to detect or measure motion, e.g., output a1 if there is motion, and otherwise output a 0. In some embodiments, additional information may be determined, such as the relative size or speed of the moving object. The number of sensors (N) may be large and therefore function approximation may not be easy. Thus, to perform the dimensionality reduction, the sum of the triggered sensors B may be_sumIs determined as B_sum=

. Furthermore, can be represented by A_sum（t）=

To give a total table occupancy A_sum。

One of the key requirements of supervised learning algorithms (e.g., training of algorithm 120) is: tagged data (i.e., data related to examples on which the algorithm is based that are considered to be real, known, or foundational facts, or data that is known to be machine learned or not used) is accessed. As defined above, this requires measuring a large amount of data for { X, Y }. This may be done via actual experimentation, or may be done by building a model that simulates the behavior in that location while being computationally tractable. This type of model may be referred to as a proxy model.

FIG. 4 illustrates a block diagram that describes how the proxy model 128 may be used to create the algorithm 120, according to one embodiment. The proxy model 128 may be used in an "offline" or learning phase to create a mapping function (g) that defines the algorithm 120 or is used by the algorithm 120 in an "online" or operational phase. As noted above, data relating to the physical layout of the location 102, as well as the motion sensors of the subsystem 106 and the physical layout of the table in the location 102, may be set according to the same reference frame or global coordinate system and provided to the model. In this manner, the coordinate data may be viewed as a bipartite graph, where the motion sensors and tables are two disjoint sets with edges between the sensors and tables, for each table within the sensing area (e.g., field of view 112) of each sensor. In building the model 128, it may be assumed that if movement is detected in the field of view of the motion sensor, it will transition to a sensor identification occupancy state, e.g., the sensor output will be 1.

Given the proxy model 128, the data may be simulated by any desired method. In one embodiment, the Monte Carlo analysis is performed by: randomly simulating a desk occupancy in location 102 (at a given A)_sumKnown value of (B), and then using the proxy model 128 to determine the number of triggered sensors (B)_sumAs defined above). After a sufficiently large amount of data has been collected, it can be determined which sensor (B) is to be triggered_sum) Mapping to occupant count (A)_sum) Function (g) of (c). An example is illustrated in FIG. 5, where each point represents B_sumA value of (A), B_sumIs based on the actual occupancy under different conditions (e.g., occupied tables in different zones)Use case (A)_sum) Is calculated from different given values of (A), wherein the function (g) is given by B_sumWith the actual occupancy (A)_sum) The associated best approximation. It should be appreciated that instead of the proxy model 128, the function (g) may be generated by performing an actual experiment in the location by altering the true occupancy (A)_sum) And measuring the number of triggered sensors (B)_sum) To proceed with.

Another consideration is: it may be necessary to convert actual or real-life motion/detection data from the motion sensors of the motion detector subsystem 106 to conform to the proxy model 128. That is, since the proxy model 128 does not account for people moving around the location, and also does not account for both primary and secondary movements, the real-life scenario in this example may tend to overestimate the number of occupants due to increased sensor activity. For this purpose, it is possible to use a plurality of levels of information provided by the motion sensor, which distinguishes between primary and secondary movements, as indicated above with respect to fig. 2. Thus, it may be set or assumed that the secondary movements are related to the person working at his desk, and thus used to record B for use with the proxy model 128_sumSimilar values, while assuming that the primary movement corresponds to a person moving briefly throughout the location 102 and is therefore not recorded. In this example, a pre-processing unit 130 is included, and the pre-processing unit 130 is configured to: the motion data is evaluated to identify data related to both secondary and primary movements, and only the data related to the secondary movements is passed to a mapping function (g) to determine the occupant number. Of course, in other embodiments, it may be desirable to count both secondary and primary movements, or to record only the primary movements regardless of the secondary movements, or to process the motion data in some other manner to have consistency between the proxy model and the data measured by the motion detector subsystem 106 when in actual operation.

As indicated above, the fusion module 126 may fuse the RF-based estimation 122 and the motion-based estimation according to any data or information fusion techniqueAnd 124. In one embodiment, let N_RFAnd N_MDOccupant estimates 122 and 124, given by the RF subsystem 104 and the motion detector subsystem 106, respectively, are represented. The variances of the two systems can be respectively V_RFAnd V_MDTo indicate. The two occupant estimates may then be fused by the fusion module 126 to obtain a final occupant count N by the equation:

if desired, computational errors may also be analyzed by determining the probability of an error occurring each time the system 100 makes an occupancy determination. For example, the probability of a maximum of "k" errors occurring in a year is given by the following equation:

where N is the total number of estimates reported in a year and p_failIs the probability of underestimating the occupant by some amount. For example, if it is assumed that the system 100 reports occupancy hourly during an eight hour period of operation during each workday, p_fail= 1-0.99146 indicates that the occupant is underestimated current probability of more than 10%. This analysis provides the minimum improvement required to achieve a 95% probability of occurring up to four times that underestimates the occupant by more than 10% of accidents. FIG. 6 shows how the probability of up to four failures occurring in a year is taken as p_failThe function of (c) varies for different reporting frequencies. This graph (and similar graphs for other reporting frequencies) may further be used to train

algorithms

118 and 120 corresponding to

subsystems

104 and 106, respectively.

FIG. 7 illustrates a method 150 for operating a system (e.g., system 100) configured to estimate occupancy of a location and control features or functions of the location according to one embodiment disclosed herein. The method 150 begins at

steps

152 and 154, where a first data set (i.e., RF data) is collected by an RF subsystem (e.g., RF subsystem 104) and a second data set (i.e., motion data) is collected by one or more motion sensors (e.g., motion sensors of motion detector subsystem 106) in

steps

152 and 154. At step 156, a first occupant estimate (e.g., RF-based estimate 122) is derived from the RF data (e.g., via RF algorithm 118), while at step 158, a second occupant estimate (e.g., motion-based estimate 124) is derived from the motion data (e.g., via motion detector algorithm 120).

The method may then proceed to the reinforcement stage 160 by proceeding from

steps

156 and 158 to

steps

162 and 164, respectively, if desired. At step 162, the RF data is used as input to a training motion detector algorithm, and at step 164, the motion data is used as input to a training RF algorithm. For example, RF data including the RF-based estimate 122 may be input to the motion detector algorithm 120 as an example of "tagged" or known information, while motion data including the motion-based estimate 124 may be input to the RF algorithm 118 as an example of "tagged" or known information. Additional examples are provided below in which data associated with each algorithm and/or estimating parameters for recalibrating another algorithm. In this manner, each different subsystem is used to train or enhance algorithms associated with other subsystems, and the unique advantages of each subsystem can enhance the ability of these algorithms to most accurately estimate occupancy. In one embodiment, only one of the algorithms (e.g., step 162 or step 164) is trained during the reinforcement phase 160. The reinforcement phase 160 may be performed for each iteration of the method 150, or the reinforcement phase 160 may be performed periodically over time. Since the RF data and estimates are used to train the motion detector algorithm and the motion-based estimates and motion data are used to train the RF algorithm, step 162 returns to step 154 and step 164 returns to step 152.

If the enhancement stage 160 is not used,

steps

156 and 158 instead proceed to step 166, where the RF estimate and the motion-based estimate are fused (e.g., via the fusion module 126 as discussed above) in step 166. Finally, the method 150 includes a step 168 in which a control system of the location (e.g., the control system 105 of the location 102) is controlled in response to the fused estimate generated in the step 166. The method 150 may be repeated as many times as desired to enable the control system 105 to operate proactively and in time in response to the number of occupants in the location 102.

In view of the above description and fig. 8, one embodiment of the reinforcement phase 160 of the method 150 may be appreciated. In this embodiment, the outputs of the two

subsystems

104 and 106 are fused to improve the accuracy of the overall occupant count. As noted above, the motion detector algorithm 120 and the motion-based estimation 124 may be based on a relationship between the true occupancy in a location and the total number of times the motion sensor of the subsystem 106 is triggered. For example, the relationship may be determined by real-life experimentation, or using monte carlo or other simulations in an offline training phase, as discussed above and shown in fig. 5. This analysis results in a set of (occupancy, trigger) pairs, which are plotted as points in both fig. 5 and fig. 8. A non-linear function is fitted to these observations to define the relationship between occupancy and the number of sensor triggers. For example, let f (x, θ) indicate a non-linear function, where x indicates the number of sensor activations and θ indicates one or more adjustable parameters of the function f. This function f may then be used by the motion detector algorithm 120 and/or include the motion detector algorithm 120.

In this example, the adjustable parameter θ may be improved using feedback provided by the RF subsystem 104. That is, during training of the algorithm 120, the algorithm 120 corresponding to the motion detector subsystem 106 may be enhanced by inputting data collected by the transceiver of the RF subsystem 104, and/or inputting the RF-based estimate 122. That is, the data from the two

subsystems

104 and 106 may be synchronized, for example using time stamps, to obtain a total number of motion sensor triggers that correspond not only to the motion based estimation 124 but also to the RF based estimation 122. This produces additional observation pairs (occupancy, triggers) of this type, where the occupancy value is not provided from simulation/experimentation, but from the data of the RF subsystem 104 (e.g., from the RF-based estimation 122). These data points are indicated as Xs in the example fig. 8. The new set of data points can be used to recalibrate the parameter θ to recalculate the function f, which in turn redefines the algorithm 120. Further, these observations are weighted differently than the results of the simulation/experiment to reflect confidence in the estimates provided by the RF subsystem 104 (e.g., estimates 122 generated via data of the RF subsystem 104 may be weighted more or less heavily than the simulation/experiment). Recalibration may be performed periodically or may be triggered due to an event, such as a detected high crowd entry or a planned event.

As another embodiment of the enhancement stage 160, the data collected by the motion detector subsystem 106 and/or the motion-based estimation 124 may alternatively or additionally be used to improve the accuracy of the RF-based estimation 122. For example, the RF algorithm 118 may employ a process such as sequential silhouette elimination (SSC) to overcome the near-far problem described above. Again, the near-far problem arises when reflections off closer occupants are more powerful than reflections off occupants further away, thereby obscuring the signals from occupants further away and hindering the ability of the RF subsystem 106 to detect or track those occupants. SSC typically requires mapping the location of the nearest occupant that will have generated TOF measurements and then removing this effect to recover the location of the other occupants. To this end, known coordinates of the motion sensors (and/or fields of view of the motion sensors) of subsystem 106, e.g., via commissioning information (which defines the location of the motion sensors, office layout, desk, etc.), may be used to verify the location of the detected occupant. For example, the known coordinates of the triggered sensors may be viewed simultaneously by the RF subsystem 104 (or a timestamp for synchronizing the motion data and comparing it to the RF data). In this way, the parameters of the algorithm 118 are recalibrated, thereby improving the accuracy of mapping TOF to different locations. This enhances the overall accuracy of the RF-based estimate 122 by improving the ability of the RF algorithm 118 to overcome the near-far problem in determining its estimate.

Although several inventive embodiments have been described and illustrated herein, those of ordinary skill in the art will readily envision a variety of other means and/or structures for performing the function and/or obtaining the results and/or one or more of the advantages described herein, and each of such variations and/or modifications is deemed to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and/or configurations will depend upon the specific application or applications for which the teachings of the present invention is/are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific embodiments of the invention described herein. It is, therefore, to be understood that the foregoing embodiments are presented by way of example only and that, within the scope of the appended claims and equivalents thereto, inventive embodiments may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and/or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and/or methods, where such features, systems, articles, materials, kits, and/or methods are not mutually inconsistent, is included within the scope of the present disclosure.

The phrase "and/or" as used herein in the specification and in the claims should be understood to mean "one or two" of the elements so joined, i.e., elements that appear in combination in some cases and in separate cases in other cases. Multiple elements listed with "and/or" should be interpreted in the same manner, i.e., "one or more" of the elements so joined. In addition to the elements specifically identified by the "and/or" clause, other elements may optionally be present, whether related or unrelated to those elements specifically identified. As used herein in the specification and in the claims, "or" should be understood to have the same meaning as "and/or" as defined above.

It will also be understood that, unless explicitly indicated to the contrary, in any methods claimed herein that include more than one step or action, the order of the steps or actions of the method is not necessarily limited to the order in which the steps or actions of the method are recited.

Claims

1. A method for determining a number of occupants at a location (102) using multiple modalities, comprising:

collecting a first data set from one or more motion sensors (106 a) embedded in a lighting system (106) in the location;

calculating a first occupant estimate (124) from the first data set using a first algorithm (120) associated with the lighting system;

collecting a second data set from one or more Radio Frequency (RF) transceivers of a RF subsystem (104) in the location;

calculating a second occupant estimate (122) from the second data set using a second algorithm (118) associated with the RF subsystem;

fusing the first occupant estimate and the second occupant estimate to create a fused occupant estimate corresponding to a number of occupants at the location; and

training the first algorithm, the second algorithm, or both the first algorithm and the second algorithm by performing at least one of: (i) input the second occupant estimate, the second data set, or both to recalibrate parameters of the first algorithm, and (ii) input the first occupant estimate, the first data set, or both to recalibrate parameters of the second algorithm.

2. The method of claim 1, further comprising: operating a building control system (105) in the location in response to the fused occupant estimation.

3. The method of claim 2, wherein the building control system comprises a security system, a Heating Ventilation and Air Conditioning (HVAC) system, a sound masking system, the lighting system, or a combination comprising at least one of the foregoing.

4. The method of claim 1, wherein the one or more motion sensors comprise Passive Infrared (PIR) sensors and the one or more RF transceivers comprise Wi-Fi enabled routers.

5. The method of claim 1, wherein collecting the second data set comprises: emitting an RF wave with the at least one RF transceiver, and receiving a reflection of the RF wave with the at least one RF transceiver.

6. The method of claim 1, wherein prior to collecting the first or second data set, the training further comprises: inputting data representing a physical layout of the location; inputting data representing coordinates of each of the one or more RF transceivers; inputting data representing coordinates of each of the one or more motion sensors, or a combination comprising at least one of the foregoing.

7. The method of claim 6, wherein the second data set includes data representing RF reflections from remote occupants confused by RF reflections of closer occupants, and the training includes: simultaneously or synchronously comparing the coordinates of the first data set and each of the one or more motion sensors with the second data set to locate the location of the remote occupant.

8. The method of claim 6, wherein the first algorithm comprises a function (f) fitted to a plurality of data points describing the number (X) of motion sensors triggered relative to a true occupant count (Y) in the location, and the training comprises: synchronously comparing the first data set with the second data set to form one or more new data points, wherein the real occupant count is set to the second occupant estimate; and recalculating the function after including the one or more new data points in the plurality of data points.

9. The method of claim 1, wherein training the first algorithm further comprises: establishing a proxy model (128) and simulating how many of the one or more motion sensors are triggered (Bsum) in response to different real occupancy conditions (Asum); and determining a function (f) mapping the number of triggered sensors to the real occupancy.

10. The method of claim 1, wherein the fusing comprises calculating the fused occupant estimate according to the equation:

wherein N is the fused occupant estimate, N_MDIs the first occupant estimate, N_RFIs the second occupant estimate, V_MDIs a first variance associated with the lighting system, and V_RFIs a second variance associated with the RF subsystem.

11. A controller (110) for operating a building control system (105), comprising:

a communication module (109) configured to receive a first data set from a lighting system (106) having one or more motion sensors (106 a) and a second data set from an Radio Frequency (RF) subsystem (104) having one or more RF transceivers;

a memory (108) having stored therein a first algorithm (120) associated with the lighting system and a second algorithm (118) associated with the RF subsystem;

a processor (106) configured to calculate a first occupant estimate (124) from the first data set and a second occupant estimate (122) from the second data set using the first algorithm, and train the first algorithm, the second algorithm, or both the first and second algorithms by performing at least one of: (i) input the second occupant estimate, the second data set, or both, to recalibrate parameters of the first algorithm, and (ii) input the first occupant estimate, the first data set, or both, to recalibrate parameters of the second algorithm; and

a fusion module (126) configured to create fused occupant estimates by fusing the first occupant estimate and the second occupant estimate;

wherein the controller is configured to control operation of the building control system in response to the fused occupant estimation.

12. A system (100) for determining a number of occupants at a location (102), comprising:

a lighting system (106) comprising one or more motion sensors (106 a), the lighting system configured to collect a first data set with the one or more motion sensors;

a Radio Frequency (RF) subsystem (104) including one or more transceivers, the RF subsystem configured to collect a second data set with the one or more transceivers;

a controller (110) configured to determine a first occupant estimate (124) from the first data set using a first algorithm (120) associated with the lighting system and to determine a second occupant estimate (122) from the second data set using a second algorithm (118) associated with the RF subsystem,

wherein the controller is configured to train the first algorithm by inputting the second occupant estimate, the second data set, or both to recalibrate parameters of the first algorithm, train the second algorithm by inputting the first occupant estimate, the first data set, or both to recalibrate parameters of the second algorithm, or a combination comprising at least one of the foregoing; and

a fusion module (126) configured to create fused occupant estimates by fusing the first occupant estimate and the second occupant estimate.

13. The system of claim 12, further comprising: a building control system (105) configured to operate in response to the fused occupant estimation.

14. The system of claim 13, wherein the building control system comprises a security system, a Heating Ventilation and Air Conditioning (HVAC) system, a sound masking system, the lighting system, or a combination comprising at least one of the foregoing.

15. The system of claim 12, wherein the one or more motion sensors are passive infrared sensors and the RF subsystem comprises at least one network router including the transceiver.