CN116965055A - Radio frequency sensing system - Google Patents

Abstract

A Radio Frequency (RF) sensing system (10) is provided, and the RF sensing system (10) includes an RF sensing node (200), the RF sensing node (200) having an RF node transceiver (210), and a node controller (220) having an RF sensing controller (223), the RF sensing controller (223) sensing RF signals from other RF nodes (200). The controller (100) includes a mode controller (120) having a calibration mode controller (122). In the calibration mode, the calibration mode controller (122) analyzes metadata from the RF sensing nodes (200) to select a number of RF sensing nodes (200) from the RF sensing nodes to perform RF sensing based on the analyzed metadata. The calibration mode controller (122) outputs instructions to the user device (300) to instruct the user to perform an activity or movement to enable calibration of the RF sensing node (200) by analyzing RF signals detected by the RF sensing node (200) during the user's activity or movement.

Description

Radio frequency sensing system

Technical Field

The invention relates to a radio frequency sensing system and a radio frequency sensing method. The invention also relates to a computer program product for calibrating a radio frequency sensing system.

Background

The intelligent and connected light modules communicate with each other, for example by means of exchanging wireless signals. In addition to their primary function of providing illumination, light modules may perform secondary functions that extend beyond the normal function of providing light. For example, wireless signals exchanged in the Radio Frequency (RF) range may be used in sensing functions, which include a variety of mechanisms for collecting motion information. Such sensing functionality can generally be easily integrated into a given hardware architecture, as hardware components are generally not required other than those already included in the optical module that allow wireless communication in any way. Thus, the sensing function can generally be implemented on a purely software basis, simply by changing the way in which wireless signals are transmitted and received using given hardware, and the way in which the signals are processed, to infer information about the presence and/or motion. This allows cost-effective implementation of sensing in already installed intelligent and connected optical modules.

Implementing sensing functions on wirelessly connected optical modules typically results in a distributed sensor, i.e. a sensing network, wherein the optical modules are densely distributed over common professional or residential spaces such as offices, living rooms, bedrooms, etc. The sensitivity and accuracy of a network generally increases with increasing network density, i.e. for example with the number of optical modules located in a given space. This is because the reliability of signals transmitted and received by different optical modules and the information derived therefrom generally increases as the distance between the optical modules decreases. On the other hand, this also increases the complexity of the overall system.

US2020/0096345 discloses a cognitive fingerprint for an indoor location sensor network. During the training phase, the user is instructed to walk around the region of interest and annotate the position sensor readings with X, Y positions using the interface.

Disclosure of Invention

It is an object of the present invention to provide an RF sensing system, an RF sensing method and a computer program product, which allow a user to calibrate the radio frequency sensing system conveniently and time-effectively.

An RF sensing system is provided. The RF sensing system may be implemented as an illumination system. It may be used for motion or activity sensing. The RF sensing system includes a plurality of RF sensing nodes disposed in a sensing region. The plurality of RF sensing nodes includes a first plurality of sensing nodes. Each first RF sensing node can include an RF node transceiver, a node controller having an RF sensing controller for sensing RF signals from other RF nodes, for determining signal quality parameters, or for performing a combination thereof (which is configured to sense RF signals and determine signal quality parameters). The RF sensing system further comprises a controller having a transceiver and a mode controller having an operational mode controller and a calibration mode controller. In the calibration mode, the calibration mode controller is configured to analyze metadata from the plurality of sensing nodes to select or pre-select a number of sensing nodes from the plurality of sensing nodes for performing RF motion sensing in or for the sensing region. The calibration mode controller outputs instructions to (the user interface of) the user device to instruct the user to perform an activity or movement in the sensing area, thereby enabling calibration of the RF sensing node by analyzing RF signals detected by the RF sensing node or by a preselected RF sensing node during the user's activity or movement in the sensing area.

According to one example, the metadata includes at least one of naming information of the RF sensing node, type of the RF sensing node, location of the RF sensing node, and relative location of the RF sensing node. The calibration mode controller may analyze the metadata of the RF nodes and perform preselection from among the plurality of RF nodes based on the analysis. Thus, the metadata is used to improve the calibration process.

According to one example, the calibration mode controller outputs instructions to (the user interface of) the user device based on the analyzed metadata to instruct the user to perform an activity or movement in the sensing area. The instructions may be visual and/or audible instructions. The detected RF signal may be used to calibrate the RF sensing system and in particular to detect specific activities or movements.

According to one example, the calibration mode controller extracts context information from the analyzed metadata and uses the context information in the preselection of the RF sensing node and the instructions output to the user device.

According to one example, the calibration mode controller verifies consistency and rationality of the extracted context information.

According to one example, the calibration mode controller generates augmented reality assistance instructions for a user to perform an activity or movement during a calibration process.

According to one example, the calibration mode controller is configured to generate instructions or augmented reality assistance instructions for a user to perform movements or activities with the user device to determine metadata of the RF node during a calibration process.

According to one example, the calibration mode controller generates instructions or augmented reality assistance instructions for a user to perform an action, activity or movement with the user device to optically determine metadata of the RF node during a calibration process.

According to one example, the calibration mode controller performs a pre-selection of the RF sensing node taking into account the processing power, internal metrics or parameters of the RF sensing node.

The RF node may have a main function different from motion sensing. The primary function may be to act as a lighting device. A secondary function of the RF node is to perform motion sensing. The RF nodes will be placed in the sensing region in the appropriate locations to perform their primary function (e.g., illumination). In other words, the location of the respective RF node may be selected to satisfy its primary function. It should be noted that the location of the RF node may not be optimal for the secondary function of motion sensing.

According to one example, calibration is performed to achieve improved motion or activity sensing by means of RF nodes located within the sensing area or by means of RF nodes not directly located within the sensing area. Alternatively, automatic setup of the RF motion-sensing system may be performed. Here, the controller may detect which RF sensing nodes are in the sensing or detection region of the controller. The detection is performed by means of RF signals or RF messages between the controller and the RF node. In a next step, the settings may be improved by analyzing the metadata of the RF nodes. The metadata may include the name, location, and/or type of the node. In other words, the metadata may include information about the corresponding RF node. The metadata may thus include the name of the corresponding RF node. Such names may be given to the RF node during the setting of the main functions of the RF node. Metadata for the respective RF node may be stored in the RF node or may be transmitted to the controller. Here, the metadata may be stored temporarily or permanently. The controller may analyze metadata of the respective RF nodes and may select some RF nodes from the plurality of RF nodes for motion sensing. As one example, all RF nodes may be used for their primary function of providing illumination, while only a subset of the RF nodes are used for the secondary function of RF-based motion sensing. The secondary functions of the unselected RF nodes may be disabled or the controller may ignore any sense signals or sense messages from those unselected RF nodes. Alternatively, the controller may deactivate secondary functions of the unselected RF nodes so that they do not perform sensing operations.

In addition, the controller may initiate a calibration mode during which the controller may forward information and instructions to the user's user device to perform an activity or movement (e.g., walk-through) through the sensing region. During the traversal, the controller or the RF sensing node itself detects the sensed RF signal and uses the detected RF signal to improve calibration of the sensing function.

Based on the RF signals detected during the calibration mode, the controller may further select an RF node from the plurality of RF nodes and ignore secondary functions of unselected RF nodes in order to further improve calibration and subsequent motion sensing. In other words, the selection of the RF sensing node may be performed in a one-step process or a two-step process.

Thus, the calibration process of the RF sensing system is improved, while those RF nodes that do not contribute to RF sensing may be deactivated from sensing, or their RF sensing signals may be ignored by the controller. Thus, only those RF sensing nodes that actually contribute positively to motion sensing in the sensing region may be used, while others may be ignored. Thus, the complexity of the RF sensing system may be reduced while improving the accuracy of motion sensing.

The user device may be any device having a user interface, such as a smart phone, tablet, smart glasses or notebook.

Particularly in case the RF nodes are implemented as intelligent lamps, they are usually provided at existing lamp sockets or in the form of lamps. The lamps are then arranged to fulfill their main function, namely illumination of the room. It should be noted that the selected location and type of smart light may not correspond to the optimal location for RF-based motion sensing. During the calibration process, it must be determined which available RF nodes are suitable for RF sensing. Those non-selected RF nodes will then be discarded, i.e. their signals will not be processed during motion sensing, or alternatively they will not continue to forward their RF-based sensing signals.

For example, for a cost effective RF motion sensing system, it may also be possible to define or set a maximum number of RF nodes for motion sensing. The maximum number of RF nodes in RF-based motion sensing may be determined by the available RF bandwidth or by the available processing power of a controller in the RF motion sensing system.

The metadata used during the calibration process may include information about the environment of the sensing region, information about the RF nodes, information about the activities performed by the user in the sensing region, and the type of sensing desired.

RF sensing may include motion sensing, activity sensing, people counting, and location detection. Motion sensing may include detecting motion, such as a person moving in a sensing area, a person performing an activity, fall detection, breath detection, gesture detection.

The transceiver may be a dedicated transmitter and a dedicated receiver such that the transceiver may transmit and receive signals simultaneously. Alternatively, the transceiver may be implemented as a combination of a transmitter and a receiver, where the devices may transmit or receive signals simultaneously.

The user device may include a user interface and a speaker. The user interface may be a display of a smart device, such as a smart phone. The display may be capable of displaying augmented reality information and images. The user device may comprise or may be connected to a virtual reality device, such as virtual reality glasses or virtual reality headphones. The user device may include at least one camera and an optional LiDAR sensor to enable panoramic scanning of the sensing area with the RF nodes to determine metadata of the sensing area and metadata of the RF nodes in the sensing area. The metadata may include information about the luminaire types of the RF nodes, their respective locations in the room, and the locations of the RF nodes relative to objects in the sensing area.

It shall be understood that the RF sensing system of claim 1, the RF sensing method of claim 10 and the computer program product of claim 14 have similar and/or identical preferred embodiments, in particular as defined in the dependent claims.

It is to be understood that the preferred embodiments of the invention may also be any combination of the dependent claims or the above embodiments with the corresponding independent claims.

These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter.

Drawings

In the following figures:

FIG. 1 shows a schematic representation of an RF sensing system, and

FIG. 2 shows a block diagram of an RF sensing system.

Detailed Description

Fig. 1 shows a schematic representation of an RF sensing system. The RF sensing system 10 comprises a controller 100 and a number of RF sensing nodes 200 within a sensing region 20. The sensing region 20 may be an indoor region or a garden. The RF sensing system has a main function that may be related to, for example, lighting. The RF node 200 may have a light unit for providing illumination. As a secondary function, the RF sensing system is adapted to sense motion in the sensing region 20. The user may have a user device 300 (with a user interface 310) capable of RF communication with the controller 100.

FIG. 2 shows a block diagram of the RF sensing system as shown in FIG. 1. The controller 100 includes a transceiver 110 configured for RF-based communications. The transceiver 110 may comprise a first, a second and a third sub-transceiver, e.g. for ZigBee based communication 111, wiFi based communication 112, or LTE or 5G based communication 113. The controller 100 includes a mode controller 120. The mode controller 120 may have an operation mode controller 121 and a calibration mode controller 122. The operation mode controller 121 is used for a normal operation mode (motion sensing). The calibration mode controller 122 is used to perform calibration of the system.

The controller 100 may also include a node metadata analyzer 130 and an optional memory 140. Memory 140 may be used to store metadata and parameters for RF nodes in the RF motion-sensing system. Metadata and parameters of RF nodes in the RF motion-sensing system can also be stored elsewhere in the system or in a remote location.

The plurality of RF nodes 200 may include a plurality of first RF nodes and optionally at least one second RF node, which may be different from the first RF nodes 200. For example, the second node may have only a transmitter rather than a transceiver. Each first RF node 200 may be considered a network device within an RF network system (RF motion sensing system) and may be capable of performing secondary functions (such as motion sensing) in addition to primary functions (e.g., lighting). Accordingly, the first RF node 200 may include a node transceiver 210. The node transceiver 210 may have sub-transceivers such as a ZigBee-based transceiver 211, a WiFi-based transceiver 212, an LTE-based or 5G-based transceiver 213. The node transceiver 210 may communicate with other node transceivers and/or transceivers of a controller. In addition, transceiver 110 may include multiple antennas or a single antenna 114. Each first RF node may also include a node controller 220. Node controller 220 may include a node processor 221, a node memory 222, and an RF sense controller 223. The RF sense controller 223 may be configured to sense RF signals from other RF nodes, determine signal quality parameters, or perform a combination thereof (i.e., sense RF signals and determine signal quality parameters (e.g., received signal strength indicator, RSSI, etc.)). Further, the first RF node 200 may include a primary function 230. Such a primary function 230 may be a smart light for providing illumination. The first RF node 200 is capable of performing primary and secondary functions, namely RF-based (motion) sensing. RF-based (motion) sensing is performed using elements of the RF node that are also required for the primary function or communication to achieve the primary function (e.g., intelligent lighting). In particular, the secondary function of RF-based motion sensing is performed by detecting RF signals or RF messages that are communicated between RF nodes or with a controller of the sensing system 10. Thus, the secondary functions of the sensing system may be implemented without additional hardware in the RF node. In fact, to perform the secondary function of motion sensing within the sensing region 20 (multiple RF nodes 200 are disposed within the sensing region 20), only the RF signals and messages exchanged within the motion sensing system need to be analyzed.

The user may have a user device 300 with a user interface 310 and optional speaker and may move in the sensing region 20. Instructions from the controller to the user may be output as audio signals via a speaker or via the user interface 310.

RF sensing may include motion sensing, activity sensing, people counting, and location detection. Motion sensing may include detecting motion, such as a person moving in a sensing area, a person performing an activity, fall detection, breath detection, gesture detection.

The transceiver may be a dedicated transmitter and a dedicated receiver such that the transceiver may transmit and receive signals simultaneously. Alternatively, the transceiver may be implemented as a combination of a transmitter and a receiver, where the devices may transmit or receive signals simultaneously.

In the calibration mode, and under the control of the calibration mode controller 122, calibration is performed so as to consider the conditions in the sensing region. These conditions may include the relative position of RF nodes 200 in sensing region 20 and relative positions with respect to each other. Furthermore, environmental effects such as effects on transmission characteristics (e.g., channel characteristics of RF links between RF nodes 200) may be considered. Furthermore, during the calibration process, the effect of user movements within the sensing area and the effect of these movements on the sensing area of the RF node 200 should also be considered. Thus, during the calibration process, the context information of the RF sensing system in the sensing region may be part of the input. This may be performed in order to improve the sensitivity of the RF motion sensing system. Additional context information may be used to select some RF nodes from among multiple RF nodes in the RF motion-sensing system.

The metadata of the RF sensing node 200 can be part of context information that can include information about the environment of the sensing region, information about the RF node, information about activities performed by the user in the sensing region, and the type of sensing desired.

The calibration mode may be selected by a user via the user device 300 or may be initiated by the controller 100. In the calibration mode, the calibration mode controller 122 controls the calibration process. During the calibration process, the controller 100 may output instructions to the user interface 310 of the user device via the transceiver 110, which the user follows to perform the calibration process, for example by walking around the sensing region 20.

According to a first example, the calibration mode controller 122 analyzes metadata of the RF sensing node 200. In particular, the calibration mode controller 122 may analyze the name or identifier of the RF node 200 that has been assigned by the user during the setup process. According to a first example, the RF node 200 may be implemented as a smart lamp with a lighting unit 230. Thus, the calibration mode controller 122 analyzes the names or identifications of the intelligent lights (RF nodes) in the sensing region 20. In a first example, the sensing region 20 may have the following smart lights:

Parlor left stand

Parlor right stand

Parlor ceiling 1

Parlor ceiling 2

Parlor ceiling 3

The calibration mode controller 122 then analyzes the identification or names of the different lamps to determine their importance to the RF sensing process. It has been noted that the names or identifications of the different lamps may comprise context information about the location of the lamps and the type of the lamps. As in the first example, it can be concluded that the living room left and right standing lights are most likely to be far enough apart from each other for efficient RF motion sensing. On the other hand, the living room ceiling 1, 2, and 3 lights may be considered as three lights, which may be in close proximity to each other and may be associated with three lights within the same light fixture housing. Therefore, it is expected that the RF signals between the living room ceiling 1 (RF node), the living room ceiling 2 (RF node), and the living room ceiling 3 (RF node) may not have the same information as the RF signals between the living room left-hand lamp and the living room right-hand lamp. In other words, the RF signal between the three ceiling lights may not be important for the calibration procedure and possible subsequent RF motion sensing. Thus, at least two of the ceiling lights may be discarded for RF sensing purposes. Thus, according to a first example, an RF signal exchange between the living room left and right standing lights, and an RF signal exchange between one of the living room left and ceiling lights, and an RF signal exchange between one of the living room right and ceiling lights may be selected during the calibration procedure. During the calibration process, only one of the three ceiling lights may be selected, while the other two may be discarded in a subsequent motion sensing operation. Thus, the first preselection of available RF nodes may be performed based on contextual information contained in the names or identifications of the lamps within the sensing region. The preselection may be performed before or during the user's traversal through the sensing region, where the user follows instructions received by the controller 100. Subsequent additional selections may be performed from the RF node 200 during or after the calibration process.

For example, the controller may forward instructions to the user interface 310, such as: and walking among the living room left standing lamp, the living room right standing lamp and the living room ceiling lamp.

By selecting only one of the three ceiling lamps, the complexity of the subsequent RF motion sensing can be reduced, since according to the first embodiment, only three of the five lamps available are selected, while the other two (two of the three ceiling lamps) are discarded (only for the secondary purpose of motion sensing). Furthermore, by reducing the total number of selected RF nodes for subsequent RF motion sensing, the calibration process may be improved and the coverage may be improved such that the final allocation of selected lamps may be made faster, such that the overall calibration process is faster, which should improve customer acceptance.

Furthermore, the detection of the relative position of the RF nodes in the sensing region may be performed during a calibration procedure or during a pre-selection of the respective RF nodes. This may be achieved by determining the signal strength between the RF nodes. Furthermore, for 60GHz WiFi applications, the spatial relationship between RF nodes (lighting units) may be determined. The 60GHz WiFi environment may determine: from the first lamp, the other two lamps in the room are at an angle of 180 ° to each other. Thus, it can be concluded that: the lamps are located on a straight line between the other two lamps. Calibration mode controller 122 may also take this into account.

In a second example, metadata of RF node 200 (e.g., a name or identification of the corresponding RF node) may be used and analyzed to extract the context information. Here, information about the sensing room or the sensing region may be extracted. Thus, a naming analysis is performed to extract the context information. For example, if four lights are sensed in a room, and the following names are given:

sofa left

Sofa right

Television left side

Television right

The naming information may be analyzed and information of the environment sensing area may be extracted. For example, it is possible to extract: in the sensing area there is a sofa and a television, and two lights are arranged on both sides of the sofa, and two lights are arranged on both sides of the television. If two lights are arranged on both sides of the television, it is expected that the user will not walk between the two lights because the television will be out of the way. Similarly, it is contemplated that if two lights are disposed on both sides of the sofa, it may be difficult for a user to walk between the two lights. Thus, preselection may be performed from available RF nodes based on such context information. For example, the RF signal between the television left and right lamps may not contain valuable information and thus may be discarded or not used during calibration and subsequent RF motion sensing operations.

In addition, the extracted context information may also be used during the user's calibration traversal. It may not be meaningful to instruct the user to walk between the left and right television lights. Thus, the instructions sent by the calibration mode controller 122 to traverse the calibration may be affected by the name or identification of the RF node.

In a second example, the calibration mode controller 122 may issue instructions to the user via the user interface 310 of the user device 300 to cause the user to sit on the sofa in different positions (e.g., left, right, and middle) and walk away from the sofa toward the television and/or between the sofa and the television.

The calibration mode controller 122 may thus be used for naming analysis to identify a preferred RF node 200 for RF-based motion sensing. Based on the naming analysis, a preselection of available RF nodes 200 may be performed for calibration and/or subsequent motion sensing.

In a third example, the available metadata of the RF motion-sensing node 200 can include the name or identification of the RF node and the type of RF node, such as the type of lighting unit (candles or bulbs). In a third example, the lamps in the sensing region may be:

restaurant ceiling 1-candle lamp type

Restaurant ceiling 2-candle lamp type

Restaurant ceiling 3-candle lamp type

Restaurant wall 1-lamp type

Restaurant wall 2-lamp type

Restaurant wall 3-lamp type

Thus, the RF nodes (lamps) 200 in the sensing region 20 may be three candles and three light fixtures. This information can now be analyzed to extract context information, in particular about the spatial relationship between the different lamps. In addition, naming analyses can be performed to determine whether naming information can be meaningful and not misinterpreted. The reason behind this is that, typically during initial setup of the lighting appliances (lights) in the sensing area, the user is lazy to refer to the respective lighting appliance with the proper name. Instead, the user may use the same name, but distinguish the names by ending numbers, such as restaurant ceiling 1, restaurant ceiling 2, restaurant ceiling 3. Therefore, it must be determined whether the names associated with the different RF nodes (lighting units) contain the actual context information. In a third example, if only the names of the RF nodes (lamps) are analyzed, it can be determined that there are three lamps (restaurant ceiling 1, restaurant ceiling 2, restaurant ceiling 3) arranged close to each other, and that there is a second set of lamps, namely restaurant wall 1, restaurant wall 2, restaurant wall 3, so that the system can conclude: there are only two groups of lights in the sensing room, with each group of lights being arranged close to each other.

In order to determine whether names or information related to different RF nodes (lamps) contain valuable context information, other tests may be required to determine the spatial relationship between the different RF nodes (lamps). In a third example, it should be noted that the general lighting application knowledge is: wall lamp luminaires are known to be single lamp luminaires and are unlikely to be grouped, whereas ceiling candles may be part of a single luminaire, such as candela. Thus, in a third example, it may be determined that three restaurant ceiling lights are arranged very close to each other such that only one of the three restaurant ceiling lights should be selected as an active RF sensing node, with three restaurant wall lamps being expected to be arranged at a distance from each other such that they can all be among the preselected RF nodes for the calibration process and possible subsequent RF-based motion sensing. In addition, the RF signals exchanged between the different RF nodes may also be analyzed to determine, for example, their signal strength and extract spatial information from these parameters.

In a fourth example, in an office environment, the following lamps may appear under the following names:

office wall

Office table

Office stand

Office cylinder

Office stripe

From the name analysis, it may be determined that contextual information about the expected location of the RF node (lamp) may be extracted. Based on this expected location information, a preselection of available RF nodes (lamps) may be performed in order to improve the RF sensing data. In a fourth example, it can be concluded that wall lamps, desk lamps and floor lamps are surrounded by free air or a less RF attenuating ambient environment, so that good RF signals can be expected. On the other hand, down lamps and strip lamps may be expected to be embedded in ceilings or surrounded by furniture, such that their RF signals may be attenuated more than the RF signals of other lamps. Thus, the down light and the strip light may be one of the non-selected lights in the calibration process and subsequent RF motion sensing. In a fourth example, the calibration mode controller 122 may issue instructions to the user via the user interface 310 to walk around office wall lamps, office desk lamps, and office floor lamps. During the traversal, the RF signals of the three lamps may be analyzed to determine whether the RF signals have sufficient strength or quality to perform RF motion sensing with the signals. Alternatively, RF signals from two other lamps (i.e., office down and office strip lamps) may also be checked to determine if they contain valuable information. Based on the results, the lamps may be selected or unselected lamps.

In a fifth example (which may be based on any of the previous examples), an augmented reality AR may be used on the user interface during the traversal in the calibration process. Thus, the AR assistance instructions may be generated by the controller 100 and given to the user via the user interface 310. In a fifth embodiment, the name or identification of the RF node (lamp) in sensing region 2 may be used to improve the navigation instructions for the user during the calibration process. Naming analysis can be used to determine the type of lamp and the possible locations of that type. This information may be used by the AR-assisted penetration because the intended light and intended location may be displayed on the user interface 310 of the user device 300. Additionally, arrows or other symbols may be displayed to assist the user in navigating the sensing region 220 under guidance during the calibration process. For example, as in the fourth example, when the camera of the user device detects the environment and an arrow or other symbol is projected onto the display and over the video of the surrounding environment, the user may be guided to walk around office wall lights, office desk lights, and office floor lights. This is expected to greatly improve the acceptability of the pass through calibration

On the user device 300 (e.g., smart phone, tablet, notebook, etc.), there may be a configuration or calibration app. The app may communicate with the controller 100 to receive the pass-through instructions to enable calibration of the RF motion-sensing system. The app may activate a camera of the user device and may superimpose AR information on a video or photo image generated by the camera. The superimposed information may relate to symbols for guiding the user through during calibration. Furthermore, augmented reality may be used to activate lights in a sensing area and thereby locate the position of the lights in that area on the camera image.

As mentioned in the first, second, third or fourth example, the name and identification of the RF node (lamp) may be analyzed by extracting context information. For example, a room may have only a single luminaire. In this case, the app or controller 100 on the user device may locate the ceiling light 1 and the restaurant wall lamp 1 on the camera image and the superimposed AR image. If multiple lamps of one type are located in a single room, the app can infer a schematic of the naming and most likely location (extracted from name analysis). Additionally or alternatively, the app or controller may turn on a light of one of the RF nodes so that the user may identify the particular light and, for example, direct the camera of the user device to the light. Here, the sensor of the user device may be used to determine the direction or position of the selected and activated light. Thus, the identification of the location of the respective lamp may be improved. Optionally, the camera of the user device may be used to determine the distance between the camera and the activated light (RF node). This may be performed, for example, by using a camera or a LiDAR sensor in the user device.

The AR-enabled app on the user device may instruct the user (based on instructions from the controller) to walk to the center of the room and align the camera with the corresponding lights (RF nodes), with the user device detecting the direction and distance of each light (RF node). Based on all of these measurements, the app or controller may calculate a 3D model of the sensing region and the RF nodes within that region. Thus, the 3D model may be used to further pre-select some of the current RF nodes for calibration and/or subsequent RF-based motion sensing.

In a fifth example, an image captured by a camera of a user device may be used to determine an object (such as a couch or a television) in a sensing region. Further, as described in the second example, the detected object may be set in relation to the current RF node (e.g., left tv, right tv, left sofa, right sofa). During the calibration phase, this information may be entered into instructions for the user to walk through. In particular, if the camera of the user device has detected certain objects, the traversal may be adjusted so that the user is not instructed to walk against objects in the sensing area (e.g., televisions or sofas). This allows the user to walk through more safely during the calibration phase, as injuries can be avoided.

In addition, based on an analysis of images taken by the camera of the user device, not only objects in the path of travel of the user but also images in the vicinity of the RF node 200 can be determined. In particular, objects that may have a negative impact on the RF broadcast or on the RF broadcast signal may be identified. Such an object may be a metal object or the like. Furthermore, the information may be a preselected portion of those RF nodes that should be used during RF motion sensing.

Further, the AR-enabled app on the user device may display on the user interface the travel path that it wishes the user to take when calibrating the RF motion-sensing system. The AR app may also provide real-time feedback on the AR image on the user interface, particularly feedback on RF sensing coverage. Furthermore, a visual overview of the areas in the room that have been covered by the pass-through test may be indicated. Further, the AR-enabled app may display information about RF sensing detection performance (weak sensing signals and strong responses on areas in the room (e.g., areas where traversal has been successfully performed)).

The user device may include a user interface and a speaker. The user interface may be a display of a smart device, such as a smart phone. The display may be capable of displaying augmented reality information and images. The user device may comprise or may be connected to a virtual reality device, such as virtual reality glasses or virtual reality headphones. The user device may include at least one camera and an optional LiDAR sensor to enable panoramic scanning of the sensing area with the RF nodes to determine metadata of the sensing area and metadata of the RF nodes in the sensing area. Alternatively, the user device or smart device may include a LiDAR sensor without a camera. Panoramic scanning may also be performed by only LiDAR sensors without using cameras. The metadata may include information about the luminaire types of the RF nodes, their respective locations in the room, and the locations of the RF nodes relative to objects in the sensing area.

In a sixth example, for example in a calibration mode, the sensing region may be analyzed. Here, for example, the type of room in which the sensing area exists should be determined. This may be performed by asking the user to identify the room via the user interface 310. Additionally or alternatively, naming analysis of RF nodes in the sensing region may be used to determine the type of room. For example, as in example 1, based on the name analysis, it may be determined that the sensing region exists in the living room. In a third example, based on the naming analysis, it may be determined that the sensing region is in a restaurant. In a fourth example, it may be determined that the sensing area is an office based on naming analysis. Based on the results of the room determination, the controller 100 or calibration mode controller 122 may forward instructions to the user interface 310 of the user to perform the traversal. Here, the travel path may be different depending on the room actually determined. For example, the travel path in a bedroom will be different from the travel path in a living room, as there is typically a different type of furniture in a bedroom than in a living room. For example, if the sensing area is in a bedroom, the controller 100 or calibration mode controller 122 would forward instructions to the user via the user interface to walk around the bed and take some time to lie in the bed, as these are two typical activities in the bedroom.

If the sensing area is in the hallway, the controller 100 may forward instructions to the user interface to instruct the user to enter and leave the hallway through as many connected rooms as possible. Here, the corridor has the function of connecting different rooms. Thus, the user is instructed to perform typical activities in the hallway.

If the sensing area is in the kitchen, the controller 100 will forward instructions to the user interface 310 to instruct the user to open and close a cabinet or sit at a dining table. This is advantageous because it may allow for more accurate RF motion sensing.

Thus, according to a sixth example, the user is instructed via the user interface to perform an activity typical for the room in which he is located.

Alternatively, the user may be required to re-perform a specific activity or take more time to perform an activity.

According to one example, the calibration process may be extended to a longer period of time to improve the calibration. Thus, for example, the user's daily activities may be detected by performing RF-based motion detection within a few days. Thereafter, a calibration mode may be initiated. Thus, the calibration may also be performed not at the beginning of the RF-based motion process, but after a period of time to further improve the calibration.

In a seventh example, if the type of room in which the sensing region is located has been determined, the controller 100 may forward instructions to the user via the user interface 310 to instruct the user to perform typical and meaningful activities in the room, thereby achieving improved calibration. If the sensing area is in a restaurant, the controller may forward instructions to a user interface of the user device to instruct the user to perform typical activities in the restaurant. For example, the user is required to sit on the most opposing chair, such as on both ends of a table. At the same time, the RF signals detected by the RF nodes may be analyzed to determine an optimal subset of RF nodes.

If the sensing area is in a bathroom, the controller instructs the user via the user interface 310 to perform typical activities such as sitting on a toilet and standing in a shower, standing in front of a sink, etc. At the same time, the detected radio frequency signal is analyzed.

If the sensing area is in the living room, the user is required to perform typical activities.

If the RF motion detection system is to be used as an intruder detection, the controller instructs the user via the user interface 310 to walk along the intruder's possible entry path (e.g., windows and doors to the garden).

In an eighth example, the controller 100 may analyze internal metrics or parameters of the RF node 200. These parameters may include CPU power, free memory, terminal radio load (streaming messages, routing to other nodes, etc.). This information may be used during preselection from available RF nodes. The idea behind including this information in preselection is that not all RF nodes may have the same processing power. Preferably, those RF nodes with sufficient CPU power, free memory and still available radio bandwidth should be selected for calibration and/or RF-based motion sensing. Thus, the internal matrix (matrix) or parameters of the RF node may be further entered during pre-selection of the available nodes, during calibration and/or during RF motion sensing. This may be particularly advantageous if the number of RF nodes present in the room exceeds the number of RF nodes required to perform efficient RF-based motion sensing. Thus, information about the internal matrix and parameters of the RF node may be used during preselection or selection of the RF node. For example, if two or three nodes with similar RF sensing sensitivity are available, one of the nodes with more available CPU power, more free memory, or reduced radio load may be selected.

Thus, for RF nodes with less free processing resources or with more critical links (e.g. in ZigBee network topologies), such RF nodes may be removed from active RF motion sensing. Thus, according to an eighth example, additional information may be performed that is useful during preselection or selection of those RF sensing nodes that are subsequently actually used in the RF motion-sensing system.

Optionally, after the preselection of available RF nodes has been performed, the controller 100 may instruct the user via the user interface 310 to further travel. This subsequent pass may be used to verify whether the preselection has no negative effect on the sensitivity of the RF motion sensing system. However, if it is determined that the sensitivity is insufficient, the preselection or selection of the RF node may be performed again.

In a ninth example, to further improve RF motion sensing sensitivity, the controller 100 or calibration mode controller 122 may determine in which room the sensing region is present. This may be performed, for example, by means of naming analysis as explained above. The controller 100 may then determine activities in the sensing region that may have a negative impact on the RF signals and thus on the detection of RF signals by the different RF nodes. The controller then forwards instructions to the user interface to instruct the user to perform these specific activities, thereby detecting the amount of negative impact of these activities on the RF-based motion sensing.

As an example, if the sensing area is in a bathroom, the user may be required to flush the toilet, open a shower, open a sink tap in the bathroom to determine their impact on RF-based sensing. As a further example, if the sensing area is in the kitchen, the user is required to turn on different electrical or electronic devices in the kitchen, such as ovens, refrigerators, microwave ovens, etc. In addition, the user may be required to open and close the doors of ovens, refrigerators and microwave ovens to determine the negative impact of these activities on the RF signals received by the different RF nodes.

The results of these calibration steps may affect the overall results of RF node selection and RF-based motion sensing. In addition, this information can be used to pre-select from available RF nodes.

In the tenth embodiment, the controller 100 may instruct the user to perform an activity that is not among the normal activities but should be detected via the user interface 310. An example of such abnormal activity is a person falling or if a person has been sleeping, for example in a living room, and has not moved significantly around the last hour.

As an example, the controller 100 may instruct the user via the user interface 310 to simulate a fall, for example by sitting down or lying down (e.g., in a particular area that is more likely to fall).

As an example, fall detection modes may be available in the controller, which may be activated during normal RF-based motion sensing. During fall detection, the system can detect any unusual activity, or no activity in the room but just activity. The controller may issue a warning if a fall has been detected. The user device carried by the user during the calibration process may have a sensor for detecting motion. Data from the user device may be shared with the controller to align RF sensing with motion sensing of the user device, for example to improve calibration of fall detection.

According to a further example, which may be based on any of the previous examples, the metadata of the RF node, e.g. with the light unit, may comprise information about the hierarchical structure within the main function of the RF node system, e.g. the lighting system. Here, the sensing region may be automatically selected based on those devices belonging to a certain hierarchical structure (e.g. lighting system). For example, in such lighting systems, a user may group lights into rooms. When the sensing function is enabled, the user may indicate that they want sensing enabled in a bedroom (as opposed to a living room). The system will automatically know which lamps are lamps in the bedroom and are thus at least the lamps that are the early candidates for performing RF sensing. Thus, a plurality of RF nodes in question may be determined. As described above, preselection of RF nodes for RF sensing and calibration may be performed.

According to further examples, a sensing area, for example in an apartment, may consist of a living room and an open kitchen. The user may wish to perform RF sensing only in the living room. However, RF sensing may also utilize light from the kitchen (although RF sensing is not in progress in the kitchen). Similarly, in a first bedroom, the number of available RF nodes may not be sufficient to perform effective RF sensing, and then the system may infer from the naming information that the room (e.g., named "child bedroom" by the user in the configuration app) is adjacent to "master bedroom" and thus attempt to include light from "child bedroom" in the RF sensing of "master bedroom". Thus, if no RF nodes arranged directly in the sensing area are able to provide additional or indirect information for RF sensing, they may also be included in the selected RF nodes for RF sensing.

According to one example, the augmented reality app runs on the user device and may also be used to provide not only instructions to the user, but also feedback to him. For example, when performing a pass through of a room, the AR app may also show the quality of the sensor coverage, e.g. covering a first color (e.g. green), for marking the subspace as having good sensor coverage (as determined by the pass through portion that has been performed). Points with suboptimal sensing coverage may be marked with a second color (e.g., red) in the AR app, and areas requiring additional second passes may be marked as squares or with other markings.

An RF motion sensing system may be considered an RF sensing network comprising a plurality of network devices adapted to transmit and receive radio frequency signals, i.e. wireless signals consisting of electromagnetic radiation in the radio frequency range. Preferably, the RF sensing network comprises at least three of such network devices, wherein each network device is adapted to transmit RF signals to at least one, preferably a plurality, and most preferably all of the other network devices, and to receive radio frequency signals transmitted by one of the network devices. A sensing network may be understood as a system comprising at least three network devices. It is therefore also understood as a non-local sensor. The network device may be any device having network device communication capabilities. The network device communication capability may receive and transmit wireless signals, particularly radio frequency signals and/or wired signals. For example, the network device communication capability may include a network device transceiver for receiving and transmitting radio frequency signals, or a transmitter for transmitting radio frequency signals and a receiver for receiving radio frequency signals. In particular, the network device may be any intelligent device, i.e. a device comprising communication capabilities for receiving and transmitting wireless signals, in particular RF signals, but otherwise implementing the functionality of a corresponding legacy device. In particular, such a smart device may be a smart home device, in which case the corresponding legacy functionality would be that of a legacy home device (such as a lamp or a home appliance). In a preferred embodiment, the network device is a smart optical module, a smart plug or a smart switch.

For example, wireless signals exchanged in the RF range may be used in a sensing function that includes a variety of mechanisms for collecting information about the presence and/or movement of humans (particularly about the number of people present, the people falling, the people making a particular gesture, or the people breathing) as a result of the effects of their body on the exchanged wireless signals. Such sensing functionality can generally be easily integrated into a given hardware architecture, as hardware components are generally not required other than those already included in the optical module that allow wireless communication in any way. Thus, the sensing function can generally be implemented on a purely software basis, simply by changing the way in which wireless signals are transmitted and received using given hardware, and the way in which the signals are processed, to infer information about the presence and/or motion. This allows cost-effective implementation of sensing in already installed light modules.

According to one example, criteria and mechanisms are provided for determining the best way for a user to perform a walk through test in space to select and/or calibrate a node in an RF sensing system. In addition, the RF sensing calibration may be simplified with augmented reality AR.

Note that not all nodes with RF sensing capabilities in space are suitable for sensing due to relative sensing performance or excessive resource occupancy (foltprint). Thus, determining the best sensing node, especially for advanced sensing like breath detection, may only be possible if the user is involved, since only the user may provide proper context information about the space himself or the sensing system may be dynamically triggered with his body during calibration in order to identify the most affected node pairs.

In one example, an augmented reality AR assistance application on a smart device is used to identify appropriate lights for RF sensing and visualize the requested travel path in the building space to a user performing RF sensing calibration.

Thus, a user may achieve an immediate, first time, correct deployment of RF sensing with acceptable performance without requiring many iterations or long periods of (offline) RF sensing training prior to activating the system. In addition, for more advanced RF sensing applications (such as respiratory and heart rate detection), guided calibration of the system may be required due to the high sensitivity to the environment surrounding the detection area.

Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

A single unit or device may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

The processes performed by one or several units or devices, such as the processing of detection signals for motion detection, may be performed by any other number of units or devices. These processes, particularly the control of the motion sensing system according to the motion sensing method performed by the RF motion sensing system, may be implemented as program code means of a computer program and/or as dedicated hardware.

A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet, or other wired or wireless telecommunication systems.

Any reference signs in the claims shall not be construed as limiting the scope.

Claims (14)

1. A Radio Frequency (RF) sensing system (10) comprising

A plurality of RF sensing nodes (200), wherein the plurality of sensing nodes (200) comprises a plurality of first sensing nodes (200), wherein each first RF sensing node (200) comprises an RF node transceiver (210), and a node controller (220) having an RF sensing controller (223), the RF sensing controller (223) being configured to sense RF signals from other RF nodes (200) or to determine signal quality parameters or to perform a combination thereof,

a controller (100) comprising a transceiver (110), a mode controller (120), the mode controller (120) having an operating mode controller (121) and a calibration mode controller (122),

wherein in a calibration mode, the calibration mode controller (122) is configured to analyze metadata from the plurality of RF sensing nodes (200) to select a number of RF sensing nodes (200) from the plurality of RF sensing nodes for performing RF sensing in a sensing region (20) based on the analyzed metadata,

wherein the calibration mode controller (122) is configured to output instructions to a user device (300) to instruct a user to perform an activity or movement in the sensing area (20) to enable calibration of the RF sensing node (200) by analyzing RF signals detected by the RF sensing node (200) during the user's activity or movement in the sensing area (20).

2. The radio frequency sensing system (10) of claim 1, wherein

The metadata includes at least one of naming information of the RF sensing node (200), a type of the RF sensing node (200), a location of the RF sensing node (200), and a relative location of the RF sensing node (200),

wherein the calibration mode controller (122) is configured to analyze metadata of the RF nodes (200) and perform a pre-selection from a plurality of RF nodes (200) based on the analysis.

3. The radio frequency sensing system (10) of claim 2, wherein

The calibration mode controller (122) is configured to output instructions to the user device (300) to instruct a user to perform an activity or movement in the sensing area (20) based on the analyzed metadata.

4. A radio frequency sensing system (10) according to claim 2 or 3, wherein

The calibration mode controller (122) is configured to extract context information from the analyzed metadata and to use the context information in a pre-selection of an RF sensing node (200) and for instructions output to the user equipment (300).

5. The radio frequency sensing system (10) of claim 4, wherein

The calibration mode controller (122) is configured to verify consistency and rationality of the extracted context information.

6. The radio frequency sensing system (10) according to one of claims 1 to 5, wherein

The calibration mode controller (122) is configured to generate augmented reality assistance instructions for a user to perform an activity or movement during a calibration process.

7. The radio frequency sensing system (10) according to one of claims 1 to 6, wherein

The calibration mode controller (122) is configured to generate instructions or augmented reality assistance instructions for a user to perform movements or activities with the user device (300) to determine metadata of the RF node (200) during a calibration procedure.

8. The radio frequency sensing system (10) according to one of claims 1 to 7, wherein

The calibration mode controller (122) is configured to generate instructions or augmented reality assistance instructions for a user to perform actions, activities or movements with the user device (300) to optically determine metadata of the RF node (200) during a calibration procedure.

9. The radio frequency sensing system (10) according to one of claims 1 to 8, wherein

The calibration mode controller (122) is configured to perform a preselection of the RF sensing node (200) taking into account processing capabilities, internal metrics or parameters of the RF sensing node (200).

10. A radio frequency sensing method in an RF sensing system, the RF sensing system comprising: a plurality of RF sensing nodes (200), wherein the plurality of sensing nodes (200) comprises a plurality of first sensing nodes (200), wherein each first RF sensing node (200) comprises an RF node transceiver (210), a node controller (220) having an RF sensing controller (223), the RF sensing controller (223) being configured to sense RF signals from other RF nodes (200) or to determine signal quality parameters or to perform a combination thereof; and a controller (100) comprising a transceiver (110), a mode controller (120), the mode controller (120) having an operation mode controller (121) and a calibration mode controller (122); the radio frequency sensing method comprises the following steps:

In a calibration mode, analyzing metadata from the plurality of RF sensing nodes (200) to select a number of RF sensing nodes (200) from the plurality of RF sensing nodes for performing RF sensing in a sensing region (20) based on the analyzed metadata, and

instructions are output to the user device (300) to instruct a user to perform an activity or movement in the sensing area (20) to enable calibration of the RF sensing node (200) by analyzing RF signals detected by the RF sensing node (200) during movement of the user in the sensing area (20).

11. The radio frequency sensing method of claim 10, wherein the metadata comprises at least one of naming information of the RF sensing node (200), type of the RF sensing node (200), location of the RF sensing node (200), relative location of the RF sensing node (200),

wherein the calibration mode controller (122) is configured to analyze metadata of the RF nodes (200) and perform a pre-selection from a plurality of RF nodes (200) based on the analysis.

12. The radio frequency sensing method according to claim 10 or 11, further comprising the step of outputting instructions to a user interface (310) of a user device (300) based on the analyzed metadata to instruct a user to perform an activity or movement in the sensing area (20).

13. The radio frequency sensing method according to any of claims 10 to 12, further comprising the steps of

Extracting context information from the analyzed metadata, and

the context information is used in a pre-selection of the RF sensing node (200) and for instructions output to the user interface (310).

14. A computer program product for calibrating a radio frequency sensing system according to claim 1, the computer program product comprising program code means for causing a radio frequency sensing system (100) according to claim 1 to perform a radio frequency sensing method according to claim 10.