Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04W—WIRELESS COMMUNICATION NETWORKS
  • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
  • H04W4/70—Services for machine-to-machine communication [M2M] or machine type communication [MTC]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N20/00—Machine learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/004—Artificial life, i.e. computing arrangements simulating life
  • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L67/00—Network arrangements or protocols for supporting network services or applications
  • H04L67/01—Protocols
  • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04W—WIRELESS COMMUNICATION NETWORKS
  • H04W52/00—Power management, e.g. Transmission Power Control [TPC] or power classes
  • H04W52/02—Power saving arrangements
• • H—ELECTRICITY
  • H01—ELECTRIC ELEMENTS
  • H01Q—ANTENNAS, i.e. RADIO AERIALS
  • H01Q1/00—Details of, or arrangements associated with, antennas
  • H01Q1/36—Structural form of radiating elements, e.g. cone, spiral, umbrella; Particular materials used therewith
  • H01Q1/38—Structural form of radiating elements, e.g. cone, spiral, umbrella; Particular materials used therewith formed by a conductive layer on an insulating support
• • H—ELECTRICITY
  • H01—ELECTRIC ELEMENTS
  • H01Q—ANTENNAS, i.e. RADIO AERIALS
  • H01Q21/00—Antenna arrays or systems
  • H01Q21/28—Combinations of substantially independent non-interacting antenna units or systems

Definitions

• the present disclosure relates to methods for managing, and for facilitating management of, sensor entities on a radio stripe.
• the methods are performed by a management node, a Lightweight Machine to Machine (LwM2M) server node, and a LwM2M client node.
• the present disclosure also relates to a management node, a LwM2M server node, and a LwM2M client node, and to a computer program product configured, when run on a computer, to carry out methods for managing, and for facilitating management of, sensor entities on a radio stripe.
• LwM2M Lightweight Machine to Machine
• Massive MIMO is one example of Multi-user MIMO (MU-MIMO), which is a set of multiple-input and multiple-output technologies for wireless communication.
• D-maMIMO Distributed Massive MIMO
• WO 2018103897 A9 discloses an antenna arrangement for use in D-maMIMO in which a flexible elongated body is provided with a plurality of antenna devices along its length, and comprises a data bus and power supply line for transmitting data to and from the plurality of antenna devices.
• This antenna arrangement is referred to as a Radio Stripe and can be used to distribute an antenna system in challenging (for example dense) outdoor and indoor areas including factories, stadiums, etc.
• the radio stripe deployment may integrate additional sensors such as temperature sensors, microphones/speakers, vibration sensors, etc., and may provide additional features such as fire alarms, burglar alarms, earthquake warning, indoor positioning, and climate monitoring and control, as discussed in WO 2018103897 A9.
• Radio stripes may be particularly suited to the provision of connectivity in Ultra Low Latency and other sensitive use cases.
• the availability of power supply to radio stripe deployments may be limited or constrained, owing to the specifics of a given deployment.
• ineffective or inefficient use of sensors mounted on the radio stripe is undesirable.
• a method for using Reinforcement Learning to manage sensor entities on a radio stripe comprising obtaining a representation of a current state of the radio stripe, and, for sensor entities on the radio stripe, using the current state of the radio stripe and a selection policy to select an action to be carried out on the sensor entity, and causing the action to be carried out on the sensor entity.
• the method further comprises obtaining an updated representation of a current state of the radio stripe and a value of a reward function that measures impact of the selected actions on performance of the task, and updating the selection policy using at least the obtained reward function value.
• a sensor entity comprises at least one sensor device mounted on the radio stripe, and each sensor device of the sensor entity is exposed as a Lightweight Machine to Machine (LwM2M) object instance.
• LwM2M Lightweight Machine to Machine
• a method for facilitating management of sensor entities on a radio stripe using Reinforcement Learning comprising registering a plurality of LwM2M object instances and associated resources, wherein the LwM2M object instances represent sensor devices mounted on the radio stripe and are hosted at a LwM2M client node associated with the radio stripe.
• a sensor entity comprises at least one sensor device mounted on the radio stripe, and the sensor entities are managed by a management node using a method according to the preceding aspect of the present invention.
• a method for facilitating management of sensor entities on a radio stripe using Reinforcement Learning comprising exposing, to a LwM2M server node, a plurality of LwM2M object instances and associated resources hosted at a LwM2M client node, wherein the LwM2M object instances represent sensor devices mounted on the radio stripe.
• a sensor entity comprises at least one sensor device mounted on the radio stripe, and the sensor entities are managed by a management node using a method according to a preceding aspect of the present invention.
• a computer program product comprising a computer readable non-transitory medium, the computer readable medium having computer readable code embodied therein, the computer readable code being configured such that, on execution by a suitable computer or processor, the computer or processor is caused to perform a method according to any one of the aspects or examples of the present disclosure.
• a management node for using Reinforcement Learning to manage sensor entities on a radio stripe, wherein the sensor entities are operable to perform a task.
• the management node comprises processing circuitry configured to cause the management node to obtain a representation of a current state of the radio stripe, and for sensor entities on the radio stripe, use the current state of the radio stripe and a selection policy to select an action to be carried out on the sensor entity, and cause the action to be carried out on the sensor entity.
• the processing circuitry is further configured to cause the management node to obtain an updated representation of a current state of the radio stripe and a value of a reward function that measures impact of the selected actions on performance of the task, and update the selection policy using at least the obtained reward function value.
• a sensor entity comprises at least one sensor device mounted on the radio stripe, and each sensor device of the sensor entity is exposed as a LwM2M object instance.
• a LwM2M server node for facilitating management of sensor entities on a radio stripe using Reinforcement Learning, wherein the sensor entities are operable to perform a task.
• the LwM2M server node comprises processing circuitry configured to cause the LwM2M server node to register a plurality of LwM2M object instances and associated resources, wherein the LwM2M object instances represent sensor devices mounted on the radio stripe and are hosted at a LwM2M client node associated with the radio stripe.
• a sensor entity comprises at least one sensor device mounted on the radio stripe, and wherein the sensor entities are managed by a management node using a method according to a preceding aspect of the present disclosure.
• a LwM2M client node for facilitating management of sensor entities on a radio stripe using Reinforcement Learning, wherein the sensor entities are operable to perform a task and the LwM2M client node is associated with the radio stripe.
• the LwM2M client node comprises processing circuitry configured to cause the LwM2M client node to expose, to a LwM2M server node, a plurality of LwM2M object instances and associated resources hosted at a LwM2M client node, wherein the LwM2M object instances represent sensor devices mounted on the radio stripe.
• a sensor entity comprises at least one sensor device mounted on the radio stripe, and wherein the sensor entities are management by a management node using a method according to a preceding aspect of the present disclosure.
• aspects of the present disclosure thus provide methods and nodes that enable the management of sensor devices on a radio stripe.
• the sensor devices are exposed as LwM2M object instances, and managed a logical construct of sensor entities.
• a single sensor entity on the radio stripe may comprise one or more sensor devices, and a Reinforcement Learning process is used to select actions for execution on the sensor entities in light of performance by the entities of a task.
• An action selected for execution on a sensor entity may be executed on each of the sensor devices comprised within the logical sensor entity.
• Figure 1 illustrates the main components in a Reinforcement Learning system
• Figure 2 is a flow chart illustrating process steps in a method for using Reinforcement Learning to manage sensor entities on a radio stripe
• Figures 3a and 3b show flow charts illustrating process steps in another example of a method for using Reinforcement Learning to manage sensor entities on a radio stripe
• Figure 4 is a flow chart illustrating process steps in a method for facilitating management of sensor entities on a radio stripe using Reinforcement Learning
• Figure 5 is a flow chart illustrating process steps in another method for facilitating management of sensor entities on a radio stripe using Reinforcement Learning
• Figure 6 is a block diagram illustrating functional modules in an example management node
• FIG. 7 is a block diagram illustrating functional modules in an example LwM2M server node
• Figure 8 is a block diagram illustrating functional modules in an example LwM2M client node
• Figure 9 is a process flow illustrating an overview of implementation of the methods of Figures 3a, 3b, 4 and 5;
• Figure 10 illustrates a training phase of the process flow of Figure 9
• Figure 11 illustrates a prediction phase of the process flow of Figure 9.
• FIGS 12 to 15 illustrate an example use case for the methods of the present disclosure.
• Examples of the present disclosure propose methods and nodes that enable the exposure and control of sensors on a radio stripe, and, according to the nature of the actions selected, may individually control the energy consumption of sensors on the radio stripe.
• the usage of power is often critical for sensors, which are frequently constrained devices in that they have limited access to at least one of power supply, processing, communication resource, etc.
• sensors can obtain power from the main power bus on the radio stripe, but this itself may also have a limited power supply. Ensuring that sensors are managed to maximize their energy efficiency on the radio stripe is therefore highly beneficial.
• a LwM2M client node may be running on a modified APU of the radio stripe, and this may expose the sensor devices on the radio stripe as LwM2M object instances.
• the sensor devices may then be reconfigured by a LwM2M server node with which the LwM2M client node is registered.
• the reconfiguration parameters (for example control of sensor state or information exposure) are generated by a management node using a Reinforcement Learning method (such as Q-learning for example), which takes as input information from the radio stripe’s power and data bus, and the local LwM2M server.
• the management node may seek to select actions that maximize energy efficiency while ensuring the sensor devices are adequately performing the task that is required of them.
• Example methods according to the present disclosure make use of Reinforcement Learning (RL), and of the LwM2M management protocol. There now follows a brief discussion of each of these concepts.
• RL Reinforcement Learning
• Reinforcement Learning is a type of Machine Learning (ML), in which an agent learns to solve a problem by interacting with an environment following a policy, and obtaining a reward for every action it takes.
• Figure 1 illustrates the main components in a RL system including: an agent, an environment, an action and a reward.
• Q-learning is a form of Reinforcement Learning that does not require a model, but involves learning a policy that an agent can use to take actions based on the current input characteristics.
• Rewards are calculated in successive passes to reach a future state. The rewards are saved to a repository called a Q-table which is used as feedback input to the agent.
• OMA Open Mobile Alliance
• LwM2M Lightweight Machine to Machine protocol
• OMA Lightweight Device Management
• LwM2M Lightweight Machine to Machine protocol
• the “Internet of Things” (loT) refers to devices enabled for communication network connectivity, so that these devices may be remotely managed, and data collected or required by the devices may be exchanged between individual devices and between devices and application servers.
• Such devices examples of which may include sensors and actuators, are often, although not necessarily, subject to severe limitations on processing power, storage capacity, energy supply, device complexity and/or network connectivity, imposed by their operating environment or situation, and may consequently be referred to as constrained devices.
• the Constrained nature of loT devices has prompted the design and implementation of new protocols and mechanisms.
• the Constrained Application Protocol as defined in RFC 7252, is one example of a protocol designed for loT applications in constrained nodes and constrained networks.
• LwM2M is designed to run on top of CoAP, and LwM2M is therefore compatible with any constrained device which supports CoAP.
• LwM2M defines three components:
• LwM2M Client contains several LwM2M objects with resources.
• a LwM2M Management Server can execute commands on the resources to manage the client, including reading, deleting or updating resources.
• LwM2M Clients are generally run in constrained devices.
• LwM2M Management Server manages LwM2M Clients by sending management commands to them.
• LwM2M Bootstrap Server is used to manage the initial configuration parameters of LwM2M Clients during bootstrapping of a device.
• LwM2M defines several interfaces, including:
• LwM2M Bootstrap Server sets the initial configuration on a LwM2M Client when the client device bootstraps.
• LwM2M Client registers to one or more LwM2M Management Servers when bootstrapping is completed.
• LwM2M Management Server can send management commands to LwM2M Clients to perform several management actions on LwM2M resources of the client.
• An access control object of the client determines the set of actions the server can perform.
• LwM2M Clients can initiate communication to a LwM2M Management Server and report information in the form of notifications.
• a constrained device is configured during bootstrap for a specific environment and/or domain before being deployed to use that domain’s LwM2M Management Server.
• a LwM2M Bootstrap Server updates client security information with the assigned LwM2M Management Server address and credentials for the LwM2M Client. In this manner, the assigned LwM2M Management Server is given management rights on the client.
• FIG. 2 is a flow chart illustrating process steps in a method 200 for using Reinforcement Learning (RL) to manage sensor entities on a radio stripe, wherein the sensor entities are operable to perform a task.
• a radio stripe comprises a flexible elongated body with a plurality of antenna devices along its length.
• the body of a radio stripe is flexible in that, under normal operating conditions for the radio stripe, the body may be deformed or bent under the influence of a force, and may return to its original shape when the force is removed. It may be assumed that the normal operating conditions for the radio stripe include normal atmospheric pressure, and an atmospheric temperature range that may reasonably be expected in an outdoor deployment.
• the degree of flexibility of the body may vary from one end of such a temperature range to the other, but that an ability of the body to deform, substantially elastically, is displayed across substantially the entire temperature range that may be expected in an outdoor deployment.
• the body may be flexible in one or more degrees of freedom.
• the body may be operable to be wound upon a spool for storage, and to follow corners and other contours of a building or other infrastructure installation on which the radio stripe may be deployed.
• the body of the radio stripe may be flexible in at least two degrees of freedom, for example both along and perpendicular to its longitudinal axis.
• the plurality of antenna devices may be mounted along the length of the body with sufficient spacing between them to allow for bending of the body between the antenna devices, and so to maintain an overall degree of flexibility of the radio stripe such that it may be wound and deployed as discussed above.
• the body of a radio stripe is elongated in that its length substantially exceeds its width. The factor by which the length of the body exceeds its width may vary from around 50 to upwards of 1000, such that the width of the body is practically negligible with respect to its length.
• a radio stripe for the purposes of the present disclosure also comprises a data bus and power supply line for transmitting data to and from the plurality of antenna devices.
• the radio stripe also comprises at least one sensor device mounted on the flexible body, meaning the sensor device or devices are physically connected to the flexible body and are connected to the data and power bus of the radio stripe.
• the radio stripe may in one embodiment comprise an adhesive tape for ease of installation.
• the radio stripe may in another embodiment have the shape of a cable.
• the method is be performed by a management node, which may comprise a physical or virtual node, and may be implemented in a computer system, computing device or server apparatus and/or in a virtualized environment, for example in a cloud, edge cloud or fog deployment.
• a virtual node may include a piece of software or computer program, a code fragment operable to implement a computer program, a virtualised function, or any other logical entity.
• the management node may for example be implemented in a core network of a communication network. In other examples, the management node may be implemented in a Radio Access node, which itself may comprise a physical node and/or a virtualized network function that is operable to exchange wireless signals.
• a Radio Access node may comprise a base station node such as a NodeB, eNodeB, gNodeB, or any future implementation of this functionality.
• the management node may encompass multiple logical entities, as discussed in greater detail below, and may for example comprise a Virtualised Network Function (VNF).
• VNF Virtualised Network Function
• the management node may be implemented in a device, which may be a wireless device, a wired device, a constrained device, etc.
• the management node may in some examples be implemented in a cloud or edge cloud location near to the LwM2M server node.
• the management node uses information from and controls a LwM2M client node (for example running on a modified APU on the radio stripe) via a LwM2M server node.
• the method 200 comprises obtaining a representation of a current state of the radio stripe in step 210, and then performing steps 220 and 230 for sensor entities on the radio stripe, as illustrated at 260.
• the method 200 comprises using the current state of the radio stripe and a selection policy to select an action to be carried out on the sensor entity, and in step 230, the method 200 comprises causing the action to be carried out on the sensor entity.
• the method further comprises, in step 240, obtaining an updated representation of a current state of the radio stripe and a value of a reward function that measures impact of the selected actions on performance of the task.
• the method 200 comprises updating the selection policy using at least the obtained reward function value.
• a sensor entity comprises at least one sensor device mounted on the radio stripe, and each sensor device of the sensor entity is exposed as a Lightweight Machine to Machine, LwM2M, object instance.
• a sensor entity in the method 200 comprises a logical entity that encompasses one or more physical sensor devices. It will be appreciated that several sensor devices may operate in combination, for example sensing temperature and humidity at a specific location on the radio stripe. It may be advantageous in some circumstances to manage the two sensors together, such that they always have the same operational state. This may also facilitate the Reinforcement Learning, reducing the state action space for the RL by reducing the number of entities within the environment and consequently the size of the state representation and the number of possible actions.
• a mapping may therefore be envisaged between physical sensor devices and logical sensor entities. In some examples the mapping may be one-to-one, with each sensor entity comprising a single sensor device. In other examples, the mapping could be one-to-many, so that a single logical sensor entity corresponds to several sensors that are grouped under the same operational state, and so considered as a single entity for the RL.
• the method 200 specifies that each sensor device of a sensor entity is exposed as a LwM2M object instance.
• Object types may include temperature sensors, humidity sensors, pressure sensors, etc. Each individual sensor device may therefore be exposed as a specific instance of an object type.
• Object instances are exposed by a LwM2M client node at which the object instances are hosted. As discussed above, a LwM2M client node may be instantiated on an APU of the radio stripe, or may be running in a device that is otherwise in communication with or connected to the radio stripe.
• Each object instance is associated with one or more resources, which are sources of information about the object and/or means of control of the object.
• associated resources may include current sensed value, maximum and minimum sensed values, etc., as well as resources associated with the state of the device. Thus, some resources may enable control of the device, to power it on or off or change other aspects of its operational state. These resources can be observed and controlled via a LwM2M server node with which the LwM2M client node is registered.
• FIGS 3a and 3b show flow charts illustrating another example of a method 300 for using Reinforcement Learning to manage sensor entities on a radio stripe, wherein the sensor entities are operable to perform a task.
• the method 300 is performed by a management node, which may comprise a physical or virtual node, and may be implemented in a computer system, computing device or server apparatus and/or in a virtualized environment, for example in a cloud, edge cloud or fog deployment.
• a sensor entity comprises at least one sensor device mounted on the radio stripe, and each sensor device of the sensor entity is exposed as a LwM2M object instance.
• the method 300 illustrates examples of how the steps of the method 200 may be implemented and supplemented to provide the above discussed and additional functionality.
• the management node obtains a representation of a current state of the radio stripe.
• the representation of a current state of the radio stripe may comprise at least one of a value of a parameter characterizing operation of a power bus of the radio stripe, a value of a parameter characterizing operation of a data bus of the radio stripe, a performance requirement for the task, sensor entities present on the radio stripe, and/or operational states of sensor entities present on the radio stripe.
• obtaining a representation of a current state of the radio stripe may comprise obtaining an indication of sensor entities present on the radio stripe and operational states of sensor entities present on the radio stripe from a LwM2M server node with which sensor devices of the sensor entities are registered.
• Obtaining the state representation may further comprise obtaining metrics for the data and power buses from an APU on the radio stripe.
• obtaining a representation of a current state of the radio stripe may further comprise obtaining from the LwM2M server node an indication of LwM2M sensor device object instances present on the radio stripe and operational states of the object instances, and mapping the obtained indications to indications of sensor entities present on the radio stripe and operational states of the sensor entities.
• the LwM2M server node is only running the LwM2M management of objects and resources as exposed by the LwM2M client, and the management node consequently performs any mapping that may be appropriate between individual devices as exposed on the radio stripe and the sensor entities that the management node uses for performing RL.
• management node retains control over the granularity of the management that it performs, i.e., over whether to consider each sensor device individually in the RL, or to group multiple sensor devices under a single sensor entity for RL, so as to reduce the state action space for the RL.
• the management node then performs steps 320 and 330 for sensor entities on the radio stripe.
• the management node uses the current state of the radio stripe and a selection policy to select an action to be carried out on the sensor entity.
• an action may comprise placing a sensor entity in at least one of a candidate set of operational states, and the candidate set of operational states may comprises: powered on, powered off, exposure enabled, and exposure disabled.
• Each of these states represents a different level of energy saving, although not all states may be available for all sensor devices.
• a powered off state may enable the device to be completely switched off, and so making no requirements on the power supplied by the power bus of the radio stripe.
• Disabling exposure of a device will not cause the device to be powered off, but will nonetheless result in energy savings if the device is not visible to the LwM2M server node, and so there is no associated communication cost owing to the transfer of information and management commands relating to the device.
• Additional operational states may also be envisaged in the candidate set.
• “energy mode” states such as ⁇ high-performance, mid-performance etc., ⁇ may be included in the candidate set. It will be appreciated that increasing the number of operational states in the candidate set will necessarily increase the state action space for the RL, and consequently the additional management flexibility offered by extra operational states may be balanced against the computational complexity of the RL process for a larger state action space.
• placing a sensor entity in an operational state comprises placing all of the sensor devices comprised within that sensor entity into the relevant operational state.
• using the current state of the radio stripe and a selection policy to select an action to be carried out on the sensor entity may comprise selecting an action that is predicted to result in a maximum value of the reward function that measures impact of the action on performance of the task.
• using a selection policy to select an action for the current state of the radio stripe may comprise identifying in a Q table the action associated with the highest predicted reward function value, as shown at step 320c.
• the selection policy may comprise random selection.
• the management node causes the action to be carried out on the sensor entity. As illustrated at 330a, this may comprise sending an instruction to a LwM2M server with which the individual sensor devices of the sensor entities are registered. The LwM2M server may then send an appropriate instruction to the relevant LwM2M client, which instruction may be carried over CoAP. In the event of anything other than one-to- one mapping between server entities and server devices as discussed above, the management node may map the selected instructions for server entities to instructions for individual server devices before sending the instruction to the LwM2M server node.
• the management node then obtains an updated representation of a current state of the radio stripe and a value of a reward function that measures impact of the selected actions on performance of the task in step 340.
• the reward function may comprise a function of at least one of a performance measure for the task, and/or a measure of energy expenditure of the radio stripe.
• the performance measure of the task may include a measure of the number of observers requiring values from the sensor entities.
• the reward function may be such that maximizing a value of the reward function comprises minimizing energy expenditure of the radio stripe without causing the performance measure for the task to fall below a minimum threshold.
• the management node updates the selection policy using at least the obtained reward function value.
• this may comprise updating the selection policy to improve prediction accuracy. This may comprise for example, improving performance of a function that predicts reward values (minimizing difference between actual and predicted reward), or improving performance of a stochastic model that provides probabilities that any given available action will result in the highest reward.
• using the selection policy comprises, respectively, selecting the action associated with the highest predicted reward, or selecting the action associated with the highest probability of resulting in the highest reward.
• updating the selection policy using at least the obtained reward function value may comprise updating the Q table with the obtained value of the reward function.
• using the selection policy comprises selecting the action that is associated in the Q-table with the highest predicted reward function value
• the methods 100, 200 may be complemented by a method 400 performed by a LwM2M server node, and a method 500 performed by a LwM2M client node.
• FIG. 4 is a flow chart illustrating process steps in a method 400 for facilitating management of sensor entities on a radio stripe using Reinforcement Learning, wherein the sensor entities are operable to perform a task.
• the sensor entities are managed by a management node using examples of the method 200 and/or 300.
• the method is performed by a LwM2M server node, which may comprise a physical or virtual node, and may be implemented in a computer system, computing device or server apparatus and/or in a virtualized environment, for example in a cloud, edge cloud or fog deployment.
• the LwM2M server node is running a LwM2M management server, as defined in LwM2M.
• a sensor entity comprises at least one sensor device mounted on the radio stripe.
• a LwM2M server node carrying out the method 400 first registers a plurality of LwM2M object instances and associated resources, wherein the LwM2M object instances represent sensor devices mounted on the radio stripe and are hosted at a LwM2M client node associated with the radio stripe.
• the LwM2M client node may previously have bootstrapped with the LwM2M server node.
• the LwM2M server node may provide to the management node an indication of sensor entities present on the radio stripe and operational states of sensor entities present on the radio stripe in step 420. This may comprise providing to the management node an indication of LwM2M sensor device object instances present on the radio stripe and operational states of the object instances at step 420a.
• the operational states indicated to the management server may comprise at least one of: powered on, powered off, exposure enabled and/or exposure disabled, and may additionally comprise energy mode states or other states encompassed within the candidate set discussed above with respect to the method 300.
• the LwM2M server node may obtain the operational states by reading values of the appropriate resources from the LwM2M client node.
• the LwM2M server node may receive from the management node an instruction to carry out an action on at least one LwM2M sensor device object instance registered with the LwM2M server node.
• the action may comprise placing a sensor device into at least one of a candidate set of operational states, and wherein the candidate set of operational states comprises: powered on, powered off, exposure enabled and/or exposure disabled.
• the candidate set may contain other operational states, as discussed above.
• the LwM2M server node may additionally provide an update indication to the management node, following execution of the action, for example to confirm it has taken place.
• the LwM2M server node may send a message to the LwM2M client node hosting the LwM2M server device object instances, the message updating a value of a resource of the LwM2M server device object instance to execute the action.
• the LwM2M sever node may then inform a registered observer of a LwM2M sensor device object instance registered with the LwM2M server node of an operational state of the LwM2M sensor device object instance, in step 450.
• the server node may therefore inform consumers of data from the relevant sensors when an action is taken to change the operational state of the sensor.
• FIG. 5 is a flow chart illustrating process steps in a method 500 for facilitating management of sensor entities on a radio stripe using Reinforcement Learning, wherein the sensor entities are operable to perform a task.
• the sensor entities are managed by a management node using examples of the method 200 and/or 300.
• the method is performed by a LwM2M client node associated with the radio stripe.
• the LwM2M client node may comprise a physical or virtual node, and may be implemented in a computer system, computing device or server apparatus and/or in a virtualized environment, for example in a cloud, edge cloud or fog deployment.
• the LwM2M client node may for example be instantiated on the radio stripe’s APU or on a separate (relatively small) device attached to the radio stripe and sharing the data and power bus.
• the LwM2M client node is running a LwM2M client, as defined in LwM2M.
• a sensor entity comprises at least one sensor device mounted on the radio stripe.
• a LwM2M client node carrying out the method 500 first exposes, to a LwM2M server node, a plurality of LwM2M object instances and associated resources hosted at a LwM2M client node, wherein the LwM2M object instances represent sensor devices mounted on the radio stripe.
• the LwM2M client node may also receive a message from the LwM2M server node, the message updating a value of a resource of a LwM2M sensor device object instance exposed to the LwM2M server node.
• the LwM2M client node may then cause the sensor device represented by the LwM2M sensor device object instance to enter an operational state in accordance with the updated value of the resource in step 530.
• the operational state may comprise at least one of: powered on, powered off, exposure enabled and/or exposure disabled, or other operational states, as discussed above.
• the methods 200 and 300 may be performed by a management node, and the present disclosure provides a management node that is adapted to perform any or all of the steps of the above discussed methods.
• the management node may comprise a physical node such as a computing device, server etc., or may comprise a virtual node.
• a virtual node may comprise any logical entity, such as a Virtualized Network Function (VNF) which may itself be running in a cloud, edge cloud or fog deployment.
• VNF Virtualized Network Function
• the management node may be operable to be instantiated in a cloud or edge cloud based deployment.
• FIG 6 is a block diagram illustrating an example management node 600 which may implement the method 200 and/or 300, as illustrated in Figures 2 and 3a and 3b, according to examples of the present disclosure, for example on receipt of suitable instructions from a computer program 650.
• the management node 600 comprises a processor or processing circuitry 602, and may comprise a memory 604 and interfaces 606.
• the processing circuitry 602 is operable to perform some or all of the steps of the method 200 and/or 300 as discussed above with reference to Figures 2 and 3a and 3b.
• the memory 604 may contain instructions executable by the processing circuitry 602 such that the management node 600 is operable to perform some or all of the steps of the method 200 and/or 300, as illustrated in Figures 2 and 3a and 3b.
• the instructions may also include instructions for executing one or more telecommunications and/or data communications protocols.
• the instructions may be stored in the form of the computer program 650.
• the processor or processing circuitry 602 may include one or more microprocessors or microcontrollers, as well as other digital hardware, which may include digital signal processors (DSPs), special-purpose digital logic, etc.
• the processor or processing circuitry 602 may be implemented by any type of integrated circuit, such as an Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) etc.
• the memory 604 may include one or several types of memory suitable for the processor, such as read-only memory (ROM), random-access memory, cache memory, flash memory devices, optical storage devices, solid state disk, hard disk drive, etc.
• the management node 600 may further comprise one or more interfaces suitable for communication with a LwM2M server node and/or other communication network nodes.
• the method 400 may be performed by a LwM2M server node, and the present disclosure provides a LwM2M server node that is adapted to perform any or all of the steps of the above discussed methods.
• the LwM2M server node may comprise a physical node such as a computing device, server etc., or may comprise a virtual node.
• a virtual node may comprise any logical entity, such as a Virtualized Network Function (VNF) which may itself be running in a cloud, edge cloud or fog deployment.
• VNF Virtualized Network Function
• the LwM2M server node may be operable to be instantiated in a cloud based deployment.
• FIG 7 is a block diagram illustrating an example LwM2M server node 700 which may implement the method 400, as illustrated in Figure 4, according to examples of the present disclosure, for example on receipt of suitable instructions from a computer program 750.
• the LwM2M server node 700 comprises a processor or processing circuitry 702, and may comprise a memory 704 and interfaces 706.
• the processing circuitry 702 is operable to perform some or all of the steps of the method 400 as discussed above with reference to Figure 4.
• the memory 704 may contain instructions executable by the processing circuitry 702 such that the LwM2M server node 700 is operable to perform some or all of the steps of the method 400, as illustrated in Figure 4.
• the instructions may also include instructions for executing one or more telecommunications and/or data communications protocols.
• the instructions may be stored in the form of the computer program 750.
• the processor or processing circuitry 702 may include one or more microprocessors or microcontrollers, as well as other digital hardware, which may include digital signal processors (DSPs), special-purpose digital logic, etc.
• the processor or processing circuitry 702 may be implemented by any type of integrated circuit, such as an Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) etc.
• the memory 704 may include one or several types of memory suitable for the processor, such as read-only memory (ROM), random-access memory, cache memory, flash memory devices, optical storage devices, solid state disk, hard disk drive, etc.
• the LwM2M server node 700 may further comprise one or more interfaces suitable for communication with a management node, a LwM2M client node and/or other communication network nodes.
• the method 500 may be performed by a LwM2M client node, and the present disclosure provides a LwM2M client node that is adapted to perform any or all of the steps of the above discussed methods.
• the LwM2M client node may comprise a physical node such as a computing device, server etc., or may comprise a virtual node.
• a virtual node may comprise any logical entity, such as a Virtualized Network Function (VNF) which may itself be running in a cloud, edge cloud or fog deployment.
• VNF Virtualized Network Function
• the LwM2M client node may be operable to be instantiated in a cloud based deployment or in a device, such as an APU or other device mounted on the radio stripe.
• FIG 8 is a block diagram illustrating an example LwM2M client node 800 which may implement the method 500, as illustrated in Figure 5, according to examples of the present disclosure, for example on receipt of suitable instructions from a computer program 850.
• the LwM2M client node 800 comprises a processor or processing circuitry 802, and may comprise a memory 804 and interfaces 806.
• the processing circuitry 802 is operable to perform some or all of the steps of the method 500 as discussed above with reference to Figure.
• the memory 804 may contain instructions executable by the processing circuitry 802 such that the LwM2M client node 800 is operable to perform some or all of the steps of the method 500, as illustrated in Figure 5.
• the instructions may also include instructions for executing one or more telecommunications and/or data communications protocols.
• the instructions may be stored in the form of the computer program 850.
• the processor or processing circuitry 802 may include one or more microprocessors or microcontrollers, as well as other digital hardware, which may include digital signal processors (DSPs), special-purpose digital logic, etc.
• the processor or processing circuitry 802 may be implemented by any type of integrated circuit, such as an Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) etc.
• the memory 804 may include one or several types of memory suitable for the processor, such as read-only memory (ROM), random-access memory, cache memory, flash memory devices, optical storage devices, solid state disk, hard disk drive, etc.
• the LwM2M client node 800 may further comprise one or more interfaces suitable for communication with a LwM2M server node and/or other communication network nodes.
• FIGS 2 to 5 discussed above provide an overview of methods which may be performed according to different examples of the present disclosure. These methods may be performed by a management node, LwM2M server node and LwM2M client node respectively, as illustrated in Figures 6 to 8. There now follows a detailed discussion of how different process steps illustrated in Figures 2 to 5 and discussed above may be implemented. The functionality and implementation detail described below is discussed with reference to the modules of Figures 6 to 8 performing examples of the methods 200, 300, 400 and/or 500, substantially as described above.
• Figures 9 to 11 illustrate example implementation of the methods disclosed herein.
• Figure 9 is a process flow illustrating an overview of implementation of the methods 300, 400 and 500.
• the example implementation assumes a radio stripe with one or more sensors attached, and at least one APU which is modified and able to run a small footprint LwM2M Client (“APU with LwM2M client” in Figure 9).
• APU with LwM2M client is modified and able to run a small footprint LwM2M Client
• Another option would be to have a small device attached to the radio stripe sharing data and power bus and running a LwM2M client (for example an NXP Kinetis KL02 with an 8MHz 32-bit processor, 4KB of RAM and 32KB of internal storage).
• Figure 9 also illustrates a LwM2M server, a management node, labelled as “online learning agent”, and data consumers of the radio stripe sensor data.
• the implementation process flow of Figure 9 comprises the following steps:
• Step 1 APU with LwM2M client can discover and access the neighboring sensors on the same Radio Stripe Power Bus and Data Bus (1)
• Step 2 The APU with LwM2M client goes through LwM2M bootstrap procedure (configuration of keys, LwM2M server access) and registration procedure (initial object discovery and configuration) (Steps 410, 510 of methods 400, 500).
• the server discovers LwM2M available objects, instances, resources and attributes for configured sensors (2).
• Step 3 The available sensors objects are observed by the LwM2M server (3, 4). (5) The available sensors’ values can be read by LwM2M sensors' data consumers.
• Step 4 (8) A Reinforcement Learning (RL) Model (e.g., Q-Learning) is trained using inputs from the local actors (discussed in greater detail with reference to Figure 10), including: - (6) information about Radio Stripe Power Bus load (i.e. mW or percent) and Radio Stripe Data Bus load (i.e. kbps or percent) - (7) information from LwM2M server: exposed objects parameters (on/off, observe interval, etc.), connected observers, etc. (Steps 310, 420 of method 300, 400)
• RL Reinforcement Learning
• Step 5 (9) The result/output of the RL is sent to the LwM2M server (e.g., turning ON/OFF exposed sensor objects, enable/disable exposed sensor objects, modify observability interval, etc.). This can be done via a REST API (discussed in greater detail with reference to Figure 11) (steps 330, 330a, 430 of method 300, 400).
• LwM2M server e.g., turning ON/OFF exposed sensor objects, enable/disable exposed sensor objects, modify observability interval, etc.
• Step 6 (10) Based on the received input, the LwM2M server reconfigures the APU with LwM2M client (e.g., turning ON/OFF exposed sensor objects, enable/disable exposed sensor objects, modify observability interval, etc.) by sending the proper LwM2M commands (steps 440, 520 of method 400, 500).
• LwM2M client e.g., turning ON/OFF exposed sensor objects, enable/disable exposed sensor objects, modify observability interval, etc.
• LwM2M functionality enabling and disabling of LwM2M objects can be done using the Create, Read, Update and Delete (CRUD) LwM2M operations (sent from LwM2M Server to LwM2M Client). For example, turning ON/OFF can be done by adding a resource 5850 to the LwM2M Sensor Object and interacting via the LwM2M Write- Attributes operation.
• the resource identifier 5850 is defined with read and write capabilities for type Boolean.
• LwM2M Object 3306 (Actuation) or Object 3311 (Light control) allow for actuation (turning ON or OFF) by writing to resource 5850.
• Note2 energy efficiency: As noted above, there is a difference in saved energy between disabling object exposure (with Delete) and turning a sensor OFF with the help of LwM2M functionality (e.g., using Resource 5850 if supported). In the former, the energy saving comes from the reduction in communication as the sensor value cannot be observed (but the sensor will continue to operate on the stripe). In the latter, there is an additional reduction in consumed energy as the sensor will be turned off.
• Step 7 (11) Update information on sensors’ data availability can be sent/signaled to the LwM2M sensors' data consumers (step 450 of method 400).
• the training phase of the example implementation is illustrated in Figure 10.
• the management node online learning agent
• the management node will take inputs including, but not limited to, metrics from the radio stripes’ shared data (1) and power bus (2), along with available sensors (5). Additional inputs from possible Service Level Agreement (SLA) requirements can also be considered (3).
• Sensors are registered in the remote LwM2M server that is responsible for handling the operations of the sensors by manipulating the objects provided by the LwM2M client (5).
• the discovery of sensors (4) can be part of the registration or discovery LwM2M procedures as described previously.
• the example implementation uses a Q-learning method, which is a form of Reinforcement Learning that is model-less and seeks a selection policy that maximizes a reward (6).
• the reward can be defined to prioritize for example some combination of energy optimization and meeting SLA requirements. All the states of the radio stipe and sensors are saved in a Q-table in the form of [state, action] entries (7).
• the training phase involves learning a selection policy that tries to reach a state associated with a maximal reward. The agent will learn from the initial parameters representing the current state but will keep updating as a better selection policy for selection of actions is learnt.
• the reward can be a function that considers the available power_bus and data_bus capacity to serve an optimal number of third-party clients which observe the sensor data objects exposed by the LwM2M client.
• the available power_bus and data_bus capacity can be a function that considers the available power_bus and data_bus capacity to serve an optimal number of third-party clients which observe the sensor data objects exposed by the LwM2M client.
• the reward parameter can be defined as a goal to achieve only energy optimization, or looking to meet SLA needs of a use case, or both.
• the prediction phase of the example implementation is illustrated in Figure 11.
• all the inputs (1 ,2,3,5) mentioned in the Training phase are used together with the Q-table based knowledge base/repository to find the corresponding state and take an optimal decision which will be sent to the LwM2M server for implementation (9).
• the LwM2M server will use the LwM2M functionality client to manage the LwM2M client’s controlled sensors on the radio stripe.
• the action space of the management node may vary according to the number of sensor entities being managed.
• one management node i.e. one RL agent
• one LwM2M client which runs for example on a modified APU on the radio stripe.
• the client runs on a device which exposes a number of sensors (Num_sensors)
• S states and A actions there will be a Q-table of size S x A.
• temperature sensors are pre-installed (onboarded) on the radio stripe, which is deployed along a shelf area with raw materials or finished products. There may be a need to monitor the temperature and humidity to maintain the integrity of the materials or product. Depending on which material or item is to be monitored, the right set of temperature or humidity sensors can be switched on/off for a level/pallet/shelf/section etc. Not all sensors on the stripe may be switched on and be used if they are not needed for the particular scenario, so saving power and energy consumption.
• FIGs 12 and 13 illustrate an inventory area in an example factory for the present use case.
• the area is covered for connectivity using a radio stripe on which several temperature sensors are mounted.
• the SLA requirement for this scenario is that the area marked as shown in Figure 13 needs to be monitored with a certain level of accuracy.
• This SLA requirement along with metrics from the radio stripe itself, i.e. , from the data and the power bus and the LwM2M objects, is provided as input to the management node.
• the management node uses these parameters to learn (via Q- learning) a selection policy that meets a reward parameter, in this case to meet the SLA requirements, as illustrated in Figure 14.
• the prediction phase recommends a set of LwM2M objects that meet the SLA requirements as depicted in Figure 15.
• the management node therefore selects the optimal set of sensors to be used in order to minimize overall energy consumption of the radio stripe while fulfilling SLA requirements.
• radio stripes may offer significant advantages in connectivity
• sensor devices mounted on radio stripes may offer additional functionality for pollution monitoring, vibration monitoring for earthquake early warning or other purposes, air quality monitoring for management policies, etc.
• Examples of the present invention may gain be employed to ensure energy efficiency in the management of such sensors, by turning off, or disabling exposure, of sensors in areas that do not require monitoring at any given time.
• examples of the present disclosure may be envisaged for use in the logistics domain, with radio stripes providing connectivity together with monitoring to ensure compliance with requirements for temperature or other environmental conditions.
• Example methods of the present disclosure can ensure that such monitoring is provided in an energy efficient matter.
• Examples of the present disclosure thus provide methods and nodes that use a Reinforcement Learning solution to control individual sensors attached to a radio stripe via a LwM2M client (running for example on a modified radio stripe APU). These methods enable fine grained control of sensor devices on a radio stripe, in order to balance energy efficiency with performance of whatever task the sensors are required to undertake. Energy demands and consumption are critical when deploying a large number of sensors on radio stripes that provide ad-hoc connectivity. Power on and exposure of only those sensors that are needed for a given use case and operational scenario can provide energy savings, as well as supporting optimal usage of power available to the radio stripe for servicing radio requirements, thus supporting high throughput performance of the radio stripe itself.
• the methods of the present disclosure may be implemented in hardware, or as software modules running on one or more processors. The methods may also be carried out according to the instructions of a computer program, and the present disclosure also provides a computer readable medium having stored thereon a program for carrying out any of the methods described herein.
• a computer program embodying the disclosure may be stored on a computer readable medium, or it could, for example, be in the form of a signal such as a downloadable data signal provided from an Internet website, or it could be in any other form.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• Computing Systems (AREA)
• Software Systems (AREA)
• Computer Networks & Wireless Communication (AREA)
• Signal Processing (AREA)
• Medical Informatics (AREA)
• General Engineering & Computer Science (AREA)
• Artificial Intelligence (AREA)
• Health & Medical Sciences (AREA)
• General Health & Medical Sciences (AREA)
• Mathematical Physics (AREA)
• Data Mining & Analysis (AREA)
• Evolutionary Computation (AREA)
• General Physics & Mathematics (AREA)
• Biomedical Technology (AREA)
• Molecular Biology (AREA)
• Computational Linguistics (AREA)
• Biophysics (AREA)
• Life Sciences & Earth Sciences (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Mobile Radio Communication Systems (AREA)

Applications Claiming Priority (1)

Publications (1)

Family

ID=91486189

Family Applications (1)

Country Status (2)

Family Cites Families (8)

• 2022
  • 2022-12-13 EP EP22968672.0A patent/EP4635209A1/de active Pending
  • 2022-12-13 WO PCT/SE2022/051172 patent/WO2024128943A1/en not_active Ceased

Also Published As

Similar Documents

Legal Events