US20240007383A1 - Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode - Google Patents
Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode Download PDFInfo
- Publication number
- US20240007383A1 US20240007383A1 US18/467,393 US202318467393A US2024007383A1 US 20240007383 A1 US20240007383 A1 US 20240007383A1 US 202318467393 A US202318467393 A US 202318467393A US 2024007383 A1 US2024007383 A1 US 2024007383A1
- Authority
- US
- United States
- Prior art keywords
- network
- heartbeat
- devices
- data
- communication device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000004891 communication Methods 0.000 claims abstract description 295
- 238000012790 confirmation Methods 0.000 claims abstract description 214
- 230000015654 memory Effects 0.000 claims description 38
- 238000012545 processing Methods 0.000 claims description 38
- 230000004044 response Effects 0.000 claims description 17
- 239000003795 chemical substances by application Substances 0.000 description 36
- 238000004590 computer program Methods 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 230000002776 aggregation Effects 0.000 description 4
- 238000004220 aggregation Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 230000007423 decrease Effects 0.000 description 4
- 230000036541 health Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001143 conditioned effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000010183 spectrum analysis Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
- H04L41/0813—Configuration setting characterised by the conditions triggering a change of settings
- H04L41/082—Configuration setting characterised by the conditions triggering a change of settings the condition being updates or upgrades of network functionality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/085—Retrieval of network configuration; Tracking network configuration history
- H04L41/0859—Retrieval of network configuration; Tracking network configuration history by keeping history of different configuration generations or by rolling back to previous configuration versions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0896—Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0805—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
- H04L43/0817—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
- H04L41/0663—Performing the actions predefined by failover planning, e.g. switching to standby network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L61/00—Network arrangements, protocols or services for addressing or naming
- H04L61/45—Network directories; Name-to-address mapping
- H04L61/4505—Network directories; Name-to-address mapping using standardised directories; using standardised directory access protocols
- H04L61/4511—Network directories; Name-to-address mapping using standardised directories; using standardised directory access protocols using domain name system [DNS]
Definitions
- IoT internet of things
- the devices tend to need a heartbeat (also referred to as a keep alive signal) back from the cloud-environment to ensure that the device may continue to do the task or operation that was ongoing.
- a heartbeat also referred to as a keep alive signal
- the cloud service becomes a large part of the total system reliability.
- Some ways to increase reliability of cloud services include duplicating the entire system in more than one location, effectively providing N+1 redundancy. However, this technique is difficult to manage, and full consistency between the two systems can be complex and costly
- the heartbeat confirmation data is data indicating that the device was ok to continue operations with the primary cloud.
- the heartbeat confirmation data is stored in a table in the secondary network, allowing a second system to provide maximum separation without having a fully rewritten software service. This allows for the maintaining of consistent operation of an IoT device or other device even during a fault condition of the primary network without having to have full consistent copies of state data at all times across multiple cloud infrastructures, as well as having a smaller, independent service to ensure reliability across portions of the cloud networks.
- Serving heartbeats may be replicated with significantly simpler states to the point of potentially a static web page handler. However, any suitable operation may be employed to serve heartbeats.
- the secondary cloud system being updated by the primary cloud system with device heartbeat confirmation data makes maintaining the consistent state significantly simpler.
- One aspect of the disclosure provides a method for providing operational reliability of a communication device adapted to communicate with a first network.
- the method includes receiving, at data processing hardware of a secondary network, a heartbeat request from the communication device requesting permission for the communication device to perform a device operation.
- the method further includes obtaining, by the data processing hardware, device state data associated with the communication device.
- the method also includes determining, by the data processing hardware, whether the communication device is permitted to perform the device operation based on the device state data associated with the communication device.
- the method also includes transmitting, by the data processing hardware, a heartbeat confirmation signal to the communication device.
- the heartbeat confirmation signal permits the communication device to perform the device operation.
- Implementations of the disclosure may include one or more of the following optional features.
- Obtaining the device state data may include receiving the device state data associated with the communication device from the network device of the first network.
- the method when the device state data is received before receiving the heartbeat request from the communication device and when the network fault condition is not present, the method includes storing the device state data in a backup datastore, retrieving the device state data from the backup datastore using a device identifier identifying the communication device, and associating the communication device with the device state data stored in the backup datastore.
- determining whether the communication device is permitted to perform the device operation includes determining whether the device state data includes heartbeat confirmation data.
- the heartbeat confirmation data may indicate permission by the network device of the first network for the communication device to perform the operation.
- the method may include determining that the communication device is permitted to perform the device operation.
- the method may include determining that the communication device is not permitted to receive the heartbeat confirmation signal.
- the method includes preventing transmission of the heartbeat confirmation signal to the communication device when the device state data does not include heartbeat confirmation data, thereby preventing the communication device from performing the device operation.
- determining that the communication device is permitted to perform the device operation includes determining whether the device state data includes invalidation data that invalidates the communication device from the ability to perform the device operation.
- the method includes determining that the communication device is permitted to perform the device operation.
- the method may include rescinding permission for the communication device to perform the device operation and withholding transmission of the heartbeat confirmation signal to the communication device, thereby preventing the communication device from performing the operation.
- the invalidation data may indicate that the communication device requires a software update, the communication device is out of compliance with contractual agreements, and/or a resource necessary for the communication device to perform the device operation is unavailable to the communication device.
- the communication device may be configured to send the heartbeat request for permission to perform a device operation to the network device of the first network and the network device of the first network may be configured to send the heartbeat confirmation signal to the communication device.
- the network device may also be configured to update the device state data associated with the communication device to include the heartbeat confirmation signal sent to the communication device.
- the network fault condition is present when the communication device does not receive a heartbeat confirmation signal from the network device of the first network within a threshold period of time after sending a heartbeat request to the network device.
- the communication device may include an Internet of Things (IoT) device, a temperature sensor, a wireless base station, a mobile device, and/or a smart appliance.
- IoT Internet of Things
- One aspect of the disclosure provides a system for reliable operation of network-connected devices in a cloud-degraded mode.
- One aspect of the disclosure includes data processing hardware of a backup network and memory hardware in communication with the data processing hardware.
- the memory hardware may store instructions that when executed on the data processing hardware cause the data processing hardware to perform operations.
- the operations include receiving a heartbeat request from the communication device.
- the heartbeat request requests permission for the communication device to perform a device operation.
- the operations further include obtaining device state data associated with the communication device and determining whether the communication device is permitted to perform the device operation based on the device state data associated with the communication device.
- the operations include transmitting a heartbeat confirmation signal to the communication device.
- the heartbeat confirmation signal permits the communication device to perform the device operation.
- Implementations of the disclosure may include one or more of the following optional features.
- Obtaining the device state data may include receiving the device state data associated with the communication device from the network device of the first network.
- the operations include storing the device state data in a backup datastore, retrieving the device state data from the backup datastore using a device identifier identifying the communication device, and associating the communication device with the device state data stored in the backup datastore.
- determining whether the communication device is permitted to perform the device operation includes determining whether the device state data includes heartbeat confirmation data.
- the heartbeat confirmation data may indicate permission by the network device of the first network for the communication device to perform the device operation.
- the network device of the first network is configured to push device state data associated with the communication device to the data processing hardware prior to the network fault condition. Additionally or alternatively, the network device of the first network may be configured to send the device state data associated with the communication device in response to a pull operation by the data processing hardware.
- the device state data may include a device identifier identifying the communication device, invalidation data that invalidates the communication device from the ability to perform the device operation, operational reliability information associated with the communication device, a payment status, an ability of the communication device to update organizational metadata, and/or an aggregation calculation of overall environment health.
- the communication device may be configured to send the heartbeat request to the network device of the first network and the network device of the first network may be configured to send the heartbeat confirmation signal to the communication device.
- the network device may also be configured to update the device state data associated with the communication device to include the heartbeat confirmation signal sent to the communication device.
- the network fault condition is present when the communication device does not receive a heartbeat confirmation signal from the network device of the first network within a threshold period of time after sending a heartbeat request to the network device.
- the communication device may include an Internet of Things (IoT) device, a temperature sensor, a wireless base station, a mobile device, and/or a smart appliance.
- IoT Internet of Things
- FIG. 1 is a schematic view of an example network system including a primary network and a secondary network.
- FIG. 2 is a schematic view of an example network device employed in the primary network of the network system of FIG. 1 .
- FIG. 3 is a schematic view of an example backup device employed in the secondary network of the network system of FIG. 1 .
- FIG. 4 is a flowchart of an example arrangement of operations for a method of providing operational reliability for a communication device adapted to communicate with a primary network.
- FIG. 5 is a flowchart of an example arrangement of operations for a method of providing operational reliability for a communication device adapted to communicate with a primary network.
- FIG. 6 is a flowchart of an example arrangement of operations for a method of providing operational reliability for a communication device adapted to communicate with a primary network.
- FIG. 7 is a schematic view of an exemplary computing device.
- FIG. 1 illustrates one example of a network system 100 that employs a primary network 102 and a secondary network 104 .
- the primary network 102 is a cloud network infrastructure that is operated by a first provider and the secondary network 104 is a different cloud network infrastructure that is operated by a different service provider.
- any suitable network configurations may be employed.
- Various communication devices 106 , 106 a e register and communicate with the primary network 102 through techniques including through wireless or wired interfaces. While examples of five different types of communication devices 106 are shown, any number of communication devices 106 may register and communicate with the primary network 102 at any given time.
- some communication devices 106 a , 106 b , 106 e may include internet of things (IoT) devices or user equipment (UE) devices such as a temperature sensor or smart appliance in a building, or a mobile computing device (e.g., smartphone, tablet, wearable, etc.), Other communication devices 106 c , 106 d may include wireless base stations.
- each of the communication devices 106 communicate with the primary network 102 via internet protocol (IP)-based communication protocols or any other suitable protocols.
- IP internet protocol
- the one or more communication devices 106 can communicate with the primary network 102 and secondary network 104 through a domain name system (DNS) as part of an Internet-based cloud network system.
- the primary network 102 includes a primary network device 200 .
- the network device 200 may be one or more servers such as web servers that communicate with one or more of the communication devices 106 .
- the network device 200 includes data processing hardware 210 and memory hardware 220 in communication with the data processing hardware 210 .
- the network device 200 executes a software application or service, such as a Web Registration Service 202 .
- Communication devices 106 may register with the Web Registration Service 202 to receive a heartbeat confirmation signal 126 (also referred to as a heartbeat confirmation or keep alive signal) to permit the communication devices 106 to continue to perform tasks or operations that were ongoing.
- a heartbeat confirmation signal 126 also referred to as a heartbeat confirmation or keep alive signal
- All of the communication devices 106 may be of the same type, such as all IoT type devices, or may be various combinations of differing types of devices such as smartphones and wireless base stations that the smartphones may wish to employ for communication purposes.
- An exemplary primary network 102 includes a citizen's broadband radio services (CBRS) system; however, the primary network 102 may be any suitable system.
- CBRS citizen's broadband radio services
- a communication device 106 such as an IoT device 106 a sends a heartbeat request 128 to the network device 200 (e.g., data processing hardware 210 ) in the primary network 102 requesting that the network device 200 send back a heartbeat confirmation signal 126 .
- the heartbeat confirmation signal 126 may include heartbeat confirmation data 118 .
- the IoT device 106 a receives the heartbeat confirmation signal 126 , the IoT device 106 a continues executing a current task or operation with the primary network 102 since no network fault condition 130 is present.
- the communication of heartbeat requests 128 and heartbeat confirmation signals 126 between a communication device and the network device 200 can occur at least once every 20 seconds, 240 seconds, or any suitable period.
- the server 200 records device state data 110 indicating heartbeat confirmation data 118 issued to each particular communication device 106 and stores the device state data 110 together with a device identifier (ID) 114 associated with the particular communication device 106 in a device state datastore 212 in communication with the network device 200 .
- the device state datastore 212 resides on the memory hardware 220 of the network device 200 .
- the device state data 110 stored in the device state datastore 212 includes other information, including information related to executing the current task or operation with the primary network 102 .
- Device state data 110 may include, without limitation, data representing payment status, an ability to update organizational metadata, aggregation calculations of overall environment health, and/or any other suitable data.
- the IoT device 106 a As long as the IoT device 106 a receives a corresponding heartbeat confirmation signal 126 responsive to issuing a heartbeat request 128 , the IoT device 106 a knows that it can continue its operation with the primary network 102 . However, the IoT device 106 a will recognize that a network fault condition 130 occurred with the primary network 102 , e.g., when a heartbeat confirmation signal 126 is not received from the primary network 102 . In some examples, communication devices 106 recognize a network fault condition 130 is present when no heartbeat confirmation signal 126 is received within a predetermined period of time after issuing/sending a heartbeat request 128 .
- the primary network 102 is a cloud network of a first provider.
- the cloud network 102 (e.g. primary network 102 ( FIG. 1 )) may include a Primary Network Device 200 .
- the Primary Network Device may execute a Web Registration Service 202 .
- the communication device registers with the Web Registration Service 202 to perform operations with the primary network 102 .
- the device state data 110 may be state data that is stored while the communication device 106 is performing operations with the primary cloud network.
- the IoT device 106 a may send the heartbeat request 128 to the primary network device 200 of the primary network 102 and receive a corresponding heartbeat confirmation signal 126 indicating that the IoT device 106 a is able to communicate with the primary network 102 .
- SAS spectrum analysis system
- the network device 200 of the primary network 102 may store the heartbeat confirmation data 118 as device state data 110 linked to the IoT device 106 a via the corresponding device ID 114 in the device state datastore 212 , whereby the device state data 110 indicates that an operating instance of the IoT device 106 a in the primary network 102 is proper. That is, that the IoT device 106 a is properly registered with the Web Registration Service 202 .
- the secondary network 104 includes a backup network device 300 .
- the backup network device 300 may be referred to as a fallback network device 300 , backup device 300 , or a mirror network device 300 .
- the network device 300 may be one or more servers such as web servers that communicate with one or more of the communication devices 106 .
- the backup network device 300 includes data processing hardware 310 and memory hardware 320 in communication with the data processing hardware 310 .
- the memory hardware 320 may store instructions that, when executed on the data processing hardware 310 , cause the data processing hardware 310 to perform operations such as those described with respect to flowcharts 400 , 500 , and 600 ( FIGS. 4 - 6 ).
- the backup network device 300 of the secondary network 104 stores, or mirrors, a subset of the device state data 110 that is stored in the datastore 212 of the primary network 102 by the primary network device 200 .
- the subset of state data 110 obtained from the primary network 102 is backed up by the backup network device 300 and is shown in FIG. 1 as backed-up device state data 110 .
- the network device 300 serves as a backup, or fallback, device for a subset of the device state data 110 , e.g., during a network fault condition 130 associated with the primary network 102 as perceived by the communication device 106 .
- the subset of state data 110 is provided by the primary network 102 through any suitable mechanism including being pushed to the secondary network 104 or pulled by the secondary network 104 as part of a backup operation associated with a cloud instance of an application running in the cloud.
- a software application executing on the backup device 300 may perform the backup operation.
- the secondary network 104 is a smaller cloud network than that of the primary network 102 thereby reducing cost and operational complexity of a backup system.
- the backup device 300 in the secondary network 104 only backs up a subset of the state data 110 that is kept by the primary network 102 for each of the devices that are in communication with the primary network 102 , thereby reducing storage cost and operational complexity.
- the secondary network 104 may not allow communication devices 106 to newly register. I.e., in some implementations, the backup device 300 does not execute a Web Registration Service.
- the network system 100 also includes an invalidation agent 120 (e.g. heartbeat invalidation agent 120 ( FIG. 1 )) that may be any suitable device that is authorized to invalidate a communication device 106 with respect to the primary network 102 and the secondary network 104 .
- the invalidation agent 120 may provide corresponding invalidation data 124 , 124 a to the primary network device 200 associated with the primary network 102 and/or corresponding invalidation data 124 , 124 b to the backup device 300 associated with the secondary network 104 .
- a software service provider, system administrator, or other entity may employ the invalidation agent 120 to identify particular communication devices 106 that should not receive heartbeat confirmation signals 126 from the network 102 , 104 when requested (e.g., via heartbeat requests 128 ) from those particular communication devices 106 . More specifically, the invalidation data 124 invalidates a particular communication device 106 from the ability to perform tasks or operations by instructing the primary network device 200 and/or the backup device 300 to not transmit heartbeat confirmation signals 126 to the particular communication device 106 . The invalidation agent 120 may determine that a resource necessary for the communication device 106 to perform the device operation is unavailable to the communication device 106 .
- a frequency or certain frequencies used by one or more base stations (e.g., 106 c ) in a CBRS system may be determined by the system and/or the invalidation agent 120 to be in use by a governmental entity and are unavailable for use by non-governmental devices.
- Communication device 106 b can be prevented from attempting to access those frequencies.
- an Internet of Things (IoT) device 106 a may be prevented from using the primary network 102 when a software update is required, when the owner of the communication device is out of compliance with contractual agreements made in connection with software usage, or other criteria.
- IoT Internet of Things
- the backup device 300 includes memory such as a backup datastore 318 that stores the backed-up subset of device state data 110 , including heartbeat confirmation data 118 .
- the heartbeat confirmation data 118 includes a copy of the valid heartbeat confirmation data 118 that is obtained from the primary network 102 .
- the invalidation data 124 a provided to the primary network device 200 is propagated to the backup device 300 and stored in the backup datastore 318 .
- the subset of state data 110 backed up by the backup network device 300 may include data representing the mapping of data between the communication device and the invalidation agent 120 .
- the invalidation data 124 b provided to the backup network device 300 is stored in the backup datastore 318 .
- only keep alive signal data 118 (heartbeat confirmation data) is backed up by the secondary network 104 . This can allow for a small use of memory such as in a table format.
- the backup device 300 of the secondary network 104 backs up only operational reliability information of a particular communication device that is communicating with the primary network 102 .
- only the keep alive signal data also referred to as the heartbeat confirmation data 118 , is backed up from the primary network 102 .
- the heartbeat confirmation data 118 is associated with an ongoing communication or task performed by a device with the primary network 102 , and may indicate that primary network device 200 of the primary network 102 permits the communication device 106 to continue performing the task or operation in the presence of a network fault condition 130 preventing communication between the communication device 106 and the primary network device 200 .
- the invalidation data 124 b is provided directly to the backup device 300 during network fault conditions 130 associated with the primary network 102 .
- the operations described herein provide a type of If This Then That—level of functionality for the heartbeat deauthorization to still propagate, rescinding permission for the communication device to continue performing tasks or operations during network fault conditions 130 associated with the primary network 102 .
- the invalidation agent 120 may transmit an invalidation message 122 to the backup device 300 when the invalidation agent 120 detects a network fault condition 130 associated with the primary network 102 .
- the network system 100 and methods 400 , 500 , 600 offer a very simple technique for configuring and handling heartbeat-signal protocols.
- the system in one example, is composed of a primary service and a heartbeat-only backup service.
- the primary service is run on cloud infrastructure of the primary network 102 whereas the heartbeat-only service is run on an entirely separate cloud infrastructure, namely the secondary network 104 , with the ability to confirm operability by the secondary network 104 to the device 106 in response to a network fault condition 130 associated with the primary network 102 .
- the network device 200 may execute a Web Registration Service 202 , and communication devices 106 may register with the Web Registration Service 202 to receive a heartbeat signal 126 (also referred to as a heartbeat confirmation signal, a keep alive signal, or simply a heartbeat) to permit the communication devices 106 to initiate or to continue to perform tasks or operations.
- a communication device 106 may send the heartbeat request 128 to the network device 200 and receive a corresponding heartbeat confirmation signal 126 .
- communication devices 106 continuously receive heartbeat signals 126 (e.g., at regular intervals) and continue to perform tasks or operations responsive to receiving the heartbeat signals 126 .
- the network device 200 records device state data 110 together with a device identifier (ID) 114 associated with the particular communication device 106 in a device state datastore 212 .
- ID may uniquely identify the communication device 106 .
- the device state data 110 may include any information related to operations or tasks performed by the communication device 106 , such as, without limitation, data representing payment status, an ability to update organizational metadata, aggregation calculations of overall environment health, and/or any other suitable data.
- the device ID 114 may be used as an index, e.g., to look up or identify the device state data 110 stored in the datastore 212 and associated with the particular communication device 106 .
- the device state data 110 associated with a particular communication device 106 may be updated over time using the device ID 114 as an index.
- the network device 200 may update the device state data 110 as the communication device 106 continues to perform tasks or operations responsive to receiving heartbeat confirmation signals 126 from the network device 200 .
- the device state data 110 includes heartbeat confirmation data 118 associated with heartbeat confirmation signals 126 sent to the communication device 106 by the network device 200 to permit the communication device 106 to perform tasks or operations.
- the network device 200 updates the heartbeat confirmation data 118 each time the network device 200 sends a heartbeat signal 126 to the communication device 106 .
- the heartbeat confirmation data 118 may provide a log of timestamped heartbeat requests 128 received from the communication device 106 and timestamped heartbeat signals 126 sent to the communication device 106 .
- the device state data 110 may reflect the most recent time the network device 200 permitted the communication device 106 to initiate or to continue to perform tasks or operations.
- the network device 200 receives an invalidation message 122 from an invalidation agent 120 .
- the invalidation message 122 may contain heartbeat invalidation data 124 , 124 a indicating that the communication device 106 should not receive any heartbeat confirmation signals 126 from the primary network device 200 .
- an Internet-of-Things (IoT) device 106 a may be prevented from using the primary network 102 when the IoT device 106 a requires a software update to address a potential security vulnerability, or when the owner of the IoT device 106 a fails to comply with contractual agreements, such as obtaining or maintain valid software licenses.
- the invalidation message 122 may contain heartbeat invalidation data 124 , 124 a including an indication of these criteria or other criteria that a communication device 106 should not receive any heartbeat confirmation signals 126 from the primary network device 200 as possible.
- the invalidation data 124 a may contain the device identifier 114 (e.g., device ID 114 ( FIG. 1 )) identifying the particular communication device 106 that should not receive heartbeat confirmation signals 126 containing heartbeat confirmation data 118 indicative of permission by the primary network device 200 (e.g., by the Web Registration Service 202 ) for the particular communication device 106 to perform tasks or operations. Accordingly, in response to receiving invalidation data 124 a contained in an invalidation message 122 from the invalidation agent 120 (e.g. heartbeat invalidation agent 120 ( FIG. 1 )), the network device 200 may decline to transmit a heartbeat confirmation signal 126 to the communication device 106 identified by the invalidation data 124 a .
- the device identifier 114 e.g., device ID 114 ( FIG. 1 )
- the network device 200 may query the datastore 212 using the associated device ID 114 to determine whether the device state data 110 for the communication device 106 includes invalidation data 124 .
- the network device 200 may decline to transmit a corresponding heartbeat confirmation signal 126 to the communication device 106 , thereby preventing the communication device 106 from performing the task or operation.
- the network device 200 stops updating the heartbeat confirmation data 118 in the device state data 110 .
- the device state data 110 includes the received invalidation data 124 a .
- the network device 200 may be configured to update the device state data 110 with invalidation data 124 a received from the invalidation agent 120 indicating that the particular communication device 106 should no longer receive heartbeat confirmation signals 126 from the network device 200 .
- the backup device 300 may execute a backup controller 302 and a confirmation controller 304 .
- the confirmation controller 304 may be referred to as a backup confirmation controller 304 or a heartbeat confirmation controller 304 .
- a network fault condition 130 FIG. 1
- the communication device 106 may be unable to connect to the primary network 102 or may fail to receive a heartbeat confirmation 126 within a threshold period of time after sending a heartbeat request 128 .
- the network fault condition 130 associated with the primary network 102 may be caused by a power outage that affects the primary network 102 or a damaged communication cable or fiber, or any other failure that prevents communication between the communication device 106 and the primary network 102 .
- the network fault condition 130 is due to a failure of the ability of the communication device 106 to communicate with the primary network 102 even though the primary network 102 is otherwise operational. Other sources of the network fault condition 130 may apply as well.
- the communication device 106 fails to receive a heartbeat confirmation 126 within a threshold period of time, the communication device 106 attempts to re-register with the Web Registration Service 202 .
- the secondary network 104 may provide a fallback for servicing heartbeat requests 128 from the communication device 106 and issuing heartbeat confirmation signals 126 to the communication device 106 during network fault conditions 130 when the communication device 106 is unable to receive heartbeat confirmation signals 126 from the primary network device 200 of the primary network 102 .
- the communication device 106 sends a heartbeat request 128 to the backup heartbeat confirmation controller 304 of the backup device 300 during the network fault condition 130 and receives a corresponding heartbeat confirmation signal 126 (also referred to as a keep alive signal) to permit the communication device 106 to continue to perform tasks or operations despite the inability of the communication device 106 to communicate with the primary network device 200 of the primary network 102 .
- the backup heartbeat confirmation controller 304 may first query the backup controller 302 to determine whether the backed-up device state data 110 associated with the communication device 106 indicates that the communication device 106 is permitted to receive heartbeat confirmation signals 126 .
- the device heartbeat backup controller 302 receives at least a subset of device state data 110 from the primary network device 200 of the primary network 102 .
- the subset of state data 110 may include heartbeat confirmation data 118 , the device ID 114 associated with the communication device 106 , and any invalidation data 124 a the primary network registration device 200 (e.g. primary network device 200 ( FIG. 1 )) received from the heartbeat invalidation agent 120 .
- the received state data 110 (e.g., device state data 110 ( FIG. 1 )) includes all the state data 110 associated with the communication device 106 that is stored in the device state datastore 212 of the primary network 102 as described previously.
- the received state data 110 only includes the heartbeat confirmation data 118 .
- Other subsets of the state data 110 may be received by the backup controller 302 .
- the primary network device 200 pushes the device state data 110 to the backup controller 302 while the primary network device 200 is sending heartbeat confirmation signals 126 to the communication device 106 .
- the primary network device 200 may push the device state data 110 associated with the communication device 106 to the backup controller 302 at periodic intervals, when the state data 110 changes, or under any other suitable circumstances.
- the backup controller 302 of the backup device 300 pulls the device state data 110 from the primary network 102 during a pull operation.
- the invalidation agent 120 transmits invalidation messages 122 to the backup device 300 of the secondary network 104 only in the presence of a network fault condition 130 .
- the invalidation agent 120 may transmit an invalidation message 122 containing invalidation data 124 b to the backup controller 302 when the invalidation agent 120 detects, or otherwise learns of, a network fault condition 130 preventing communication between a particular communication device 106 and the primary network device 200 of the primary network 102 .
- the backup controller 302 updates the device state data 110 stored in the backup datastore 318 to include the invalidation data 124 b associated with the communication device 106 , e.g., indexed by the device ID 114 .
- the heartbeat invalidation agent 120 may use invalidation data 124 associated with the communication device 106 to invalidate any heartbeat confirmation data 118 associated with the communication device 106 during a network fault condition 130 associated with the primary network, thereby preventing the backup controller 302 of the backup device 300 from sending any heartbeat confirmation signals.
- the backup heartbeat confirmation controller 304 determines whether to send a heartbeat confirmation 126 to the communication device 106 to permit the communication device 106 to initiate or to continue to perform tasks or operations.
- the heartbeat confirmation controller 304 may determine whether to send a heartbeat confirmation signal 126 to the communication device 106 based on the backed-up device state data 110 stored in the backup datastore 318 .
- the backup confirmation controller 304 may query the backup controller 302 to retrieve the backed up device state data 110 from the backup datastore 318 using the associated device ID 114 .
- the backup heartbeat confirmation controller 304 provides the heartbeat request 128 and associated device ID 114 to the backup controller 302 .
- the backup controller 302 may retrieve the backed-up device state data 110 , e.g. using the associated device ID 114 and provide the backed-up device state data 110 to the backup heartbeat confirmation controller 304 .
- the backup heartbeat confirmation controller 304 retrieves the backed-up device state data 110 directly from the backup datastore 318 .
- the backup heartbeat confirmation controller 304 may use other techniques to retrieve the backed-up device state data 110 .
- the backup controller 302 may determine that the retrieved backed-up device state data 110 includes heartbeat confirmation data 118 without any invalidation data 124 and provides a signal to the backup confirmation controller 304 indicating that communication device 106 should receive the heartbeat confirmation signal 126 .
- the backup controller 302 provides the retrieved backed-up device state data 110 to the backup heartbeat confirmation controller 304 and the backup heartbeat confirmation controller 304 determines whether or not to send a heartbeat confirmation signal 126 to the communication device 106 based on the backed-up device state data 110 . For instance, if the device state data 110 includes heartbeat confirmation data 118 and no invalidation data 124 is present, the backup confirmation controller 304 may determine to send the heartbeat confirmation signal 126 to the communication device 106 . As described above, the heartbeat confirmation data 118 may reflect the most recent time the network device 200 permitted the communication device 106 to initiate or to continue to perform tasks or operations.
- the backup heartbeat confirmation controller 304 may determine to send a heartbeat confirmation 126 to the communication device 106 when the heartbeat confirmation data 118 in the backup datastore 318 indicates that the network device 200 previously permitted the communication device to initiate or to continue to perform tasks or operations.
- the backup heartbeat confirmation controller 304 may determine not to send a heartbeat confirmation 126 to the communication device 106 when the backed-up device state data 110 does not include heartbeat confirmation data 118 associated with the communication device 106 .
- Other techniques for determining whether to send a heartbeat confirmation 126 to the communication device 106 based on the heartbeat confirmation data 118 are possible.
- the backup heartbeat confirmation controller 304 sends a heartbeat confirmation signal 126 to the communication device 106
- the backup heartbeat confirmation controller 304 updates the heartbeat confirmation data 118 associated with the communication device 106 in the backup datastore 318 .
- the backed-up device state data 110 may include invalidation data 124 received indirectly by the backup controller 302 via the primary network device 200 of the primary network 102 as part of device state data 110 received by the backup controller 302 during a push or pull operation or received directly from the invalidation agent 120 and stored in the backup datastore 318 .
- the backup controller 302 may simply append the invalidation data 124 to the backed-up device state data 110 being sent to the backup confirmation controller 304 .
- the backup confirmation controller 304 may decline to transmit a heartbeat confirmation signal 126 to the communication device 106 identified by the invalidation data 124 (e.g., heartbeat invalidation data 124 ( FIG. 1 )). For instance, when the backup confirmation controller 304 receives a heartbeat request 128 from a communication device 106 for performing a task or operation, the backup confirmation controller 304 (or the backup controller 302 ) may query the backup datastore 318 using the device ID 114 to determine whether the device state data 110 associated with the communication device 106 includes invalidation data 124 .
- a heartbeat confirmation signal 126 e.g., heartbeat invalidation data 124 ( FIG. 1 )
- the backup confirmation controller 304 may rescind permission for the communication device 106 to perform the device operation and withhold, or decline to transmit, a corresponding heartbeat confirmation signal 126 to the communication device 106 , thereby preventing the communication device 106 from performing the task or operation.
- FIG. 4 is a flowchart of an example arrangement of operations for a method 400 of providing operational reliability for a communication device 106 adapted to communicate with a primary network 102 .
- the method 400 includes receiving, at data processing hardware 310 of a second network 104 (e.g., backup network 104 or secondary network 104 ( FIG. 1 )), from the communication device 106 , a heartbeat request 128 requesting permission for the communication device 106 to perform a device operation.
- a second network 104 e.g., backup network 104 or secondary network 104 ( FIG. 1 )
- the communication device 106 is configured to send the heartbeat request 128 to the network device 200 of the first network 102 and receive a corresponding heartbeat confirmation signal 126 from the network device 200 when the network fault condition 130 is not present.
- the network fault condition 130 is present when the communication device 106 does not receive a heartbeat confirmation signal 126 from the network device 200 of the first network 102 within a threshold period of time after sending a heartbeat request 128 to the network device 200 .
- the method 400 includes obtaining, by the data processing hardware 310 , device state data 110 associated with the communication device 106 .
- the data processing hardware 310 may receive the device state data 110 directly or indirectly from the network device 200 of the first network 102 .
- the network device 200 may periodically push the device state data 110 to the data processing hardware 310 when communication between the communication device 106 and the network device 200 is not degraded (e.g., the fault condition 130 is not present), and the data processing hardware 310 may store the device state data 110 in a backup data store 318 .
- the data processing hardware 300 may access the backup data store 318 during the network fault condition 130 to retrieve device state data 110 stored therein for a particular communication device 106 .
- the data processing hardware 310 may perform a pull operation to obtain the device state data 110 from the network device 200 .
- the data processing hardware 310 may perform the pull operation by querying the network device 200 to retrieve the device state data 110 for the particular communication device 106 .
- the query may include a device identifier 114 identifying the communication device 106 so that the network device 200 can retrieve the associated device state data 110 from a datastore 212 .
- the method 400 includes determining, by the data processing hardware 310 , whether the communication device 106 is permitted to perform the device operation based on the device state data 110 associated with the communication device 106 . For example, when the device state data 110 includes heartbeat confirmation data 118 indicative of permission by the network device 200 of the first network 102 for the communication device 106 to perform the device operation, the method 400 may determine that the communication device 106 is permitted to perform the device operation. However, when the device state data 110 does not include heartbeat confirmation data 118 , the method 400 may determine that the communication device 106 is not permitted to perform the device operation. Furthermore, when the device state data 110 includes invalidation data 124 despite having heartbeat confirmation data 118 , method 400 may determine that the communication device 106 is not permitted to perform the device operation.
- the method 400 includes, when the communication device 106 is permitted to perform the device operation, transmitting, by the data processing hardware 310 , a heartbeat confirmation signal 126 to the communication device 106 .
- the heartbeat confirmation signal 126 permits the communication device 106 to perform the device operation.
- the data processing hardware 310 may withhold transmission of the heartbeat confirmation signal 126 to the communication device 106 , thereby preventing the communication device 106 from performing the device operation.
- the device heartbeat backup controller 302 backs up a subset of device state data 110 from the primary network 102 that includes heartbeat confirmation data 118 that was stored as device state data 110 , for one or more communication devices 106 , 106 a - 106 e that uses the primary network 102 .
- the backup occurs by storing the subset of state data 110 in the datastore 318 on a per-device identifier 114 basis.
- datastore records may include a device identifier 114 for each of the communication devices 106 a - 106 e along with corresponding heartbeat confirmation data 118 that the primary network 102 stored as device state data 110 .
- a network fault condition 130 may occur, such as an inability of IoT device 106 a to connect to the primary network 102 as indicated by an IP address failure resulting from a DNS attempt to connect to the primary network 102 by the device that makes a heartbeat request 128 to the primary network 102 .
- the network fault condition 130 is indicated by the device, such as communication device making an Internet protocol address request through a DNS to the secondary network 104 because the communication device was unable to receive a proper response when it attempted to connect to the primary network 102 via a DNS attempt once connected to the secondary network 104 (the backup device) 300 .
- the IoT device 106 a issues a heartbeat request 128 to the secondary network 104 .
- the backup heartbeat confirmation controller 304 provides a heartbeat confirmation 126 to the communication device 106 based on the backed up heartbeat confirmation data 118 that is associated with the heartbeat request 128 from the device.
- an invalidation agent 120 can use the invalidation agent 120 to designate a particular communication device using the device identifier 114 to indicate that the communication device is not to receive a heartbeat confirmation 126 from a network (e.g., primary network 102 or secondary network 104 ) in response to a heartbeat confirmation request 128 from the communication device 106 .
- a network e.g., primary network 102 or secondary network 104
- the invalidation agent 120 determines, for example, that a particular frequency or set of frequencies are not available from base station 106 c for use by communication device 106 b .
- Communication device 106 b can be prevented from attempting to access certain frequencies.
- a frequency or certain frequencies used by one or more base stations (e.g., 106 c ) in a CBRS system may be determined by the system and/or the invalidation agent 120 to be in use by a governmental entity or devices and are therefore unavailable for use by non-governmental devices.
- the frequencies are considered available for public use by user equipment and the heartbeat confirmation 126 may then be sent from the secondary network 104 to a respective IoT device 106 a .
- an Internet of Things (IoT) device may be prevented from using the primary network 102 when a software update is required, when the owner of the communication device is out of compliance with contractual agreements made in connection with software usage, or based on any other desired criteria.
- Digital policy certificates or other mechanisms may be employed in determining when to send invalidation data 124 a and/or 124 b to a respective network.
- the operation will be described in connection with using multiple end-user communication devices 106 .
- the system may be employed with any suitable number of end-user communication devices 106 as desired.
- 5 and 6 is done in the context of the subset of state data 110 being only heartbeat confirmation data 118 associated with each of a plurality of devices that use the primary network 102 , as opposed to other examples where the subset of state data includes more than heartbeat confirmation data 118 .
- the method 500 includes the backup device 300 receiving, from the primary network device 200 , only heartbeat confirmation data 118 (and corresponding device ID 114 ) associated with a plurality of devices that use the primary network 102 .
- the heartbeat confirmation data 118 that is backed up is a subset of the device state data 110 that the server 200 maintains for each respective device.
- the method includes backing up only the heartbeat confirmation data 118 for the plurality of devices that use the primary network 102 . This is done by the device heartbeat backup controller 302 backing up only the heartbeat confirmation data 118 from the primary network 102 .
- the method includes, in response to a network fault condition 130 and in response to a heartbeat request 128 from at least one of the plurality of devices, providing a heartbeat confirmation 126 based on the heartbeat confirmation data 118 that was backed up, to at least one of the plurality of devices. This is done by the backup heartbeat confirmation controller 304 in this example.
- invalidation data 124 b for the secondary network 104 such as through an invalidation message 122 from the invalidating agent 120 , and in response to the network fault condition 130 , and in response to the heartbeat request 128 from the at least one device, denying a heartbeat confirmation 126 for the requesting device when the invalidation data 124 b indicates to invalidate the communication device 106 .
- the invalidation message 122 can come at any time. If the invalidation message 122 is sent after the heartbeat confirmation 126 was already sent, a subsequent heartbeat request 128 will be denied because the invalidation data 124 b will be used to indicate that the invalidation agent 120 no longer wishes the communication device to have a heartbeat confirmation 126 .
- no heartbeat confirmation 126 will be sent if upon an initial heartbeat request 128 from the device after a network fault condition 130 occurs, the invalidation data 124 b indicates that the communication device is invalid, the backup heartbeat confirmation controller 304 will not issue a heartbeat confirmation 126 .
- the invalidation data 124 is stored as part of a data record on a per-device basis so that the backed-up device state data includes a device identifier 114 for each device of interest, the corresponding backed up heartbeat confirmation data 118 from the primary network 102 and any invalidation data 124 b that has been received for the particular communication device from invalidation agent 120 .
- the invalidation data 124 b in one example includes the device ID 114 plus the data indicating whether to invalidate a communication device 106 .
- the backup device 300 denies a heartbeat confirmation 126 for the communication device making the heartbeat request 128 based on an invalidating message 122 from the invalidation agent 120 .
- the method 600 includes the device registering or connecting with the primary network 102 via a DNS address.
- the server 200 e.g., primary network device 200 ( FIG. 1 )
- the server 200 also provides the heartbeat confirmation data 118 to the secondary network 104 as part of a backup operation.
- the backup network device 300 will respond to the communication device that is requesting a heartbeat from the secondary network 104 with heartbeat confirmation 126 based on the heartbeat confirmation data 118 that was backed up.
- the secondary network 104 and backup device 300 allows the overall system to keep consistent views of the devices in the primary network 102 without having to have full constant copies of the data at all times as well as having smaller, independent software services to ensure reliability.
- a static page handler exists that stores only the most important aspects of operation, namely serving heartbeat confirmation signals 126 to the devices.
- the secondary network 104 only needs to be updated by the primary network 102 with heartbeat confirmation data 118 and makes maintaining a consistent state of the devices significantly simpler than prior methods.
- the device ID 114 and corresponding heartbeat confirmation data 118 may be the same data. For example when a table is used, the mere existence of the device ID 114 in the table could serve as the heartbeat confirmation data 118 for the particular communication device.
- FIG. 7 is schematic view of an example computing device 700 that may be used to implement the systems (e.g., the network device 200 and the backup network device 300 ) and methods (e.g., the methods 400 , 500 , 600 ) described in this document.
- the computing device 700 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, mobile computing devices, wearable computing devices (e.g., headsets and/or watches), servers, blade servers, mainframes, and other appropriate computers.
- the components shown here, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed in this document.
- the computing device 700 includes a processor 710 (also referred to as processing hardware or data processing hardware), memory 720 (also referred to as memory hardware), a storage device 730 , a high-speed interface/controller 740 connecting to the memory 720 and high-speed expansion ports 750 , and a low speed interface/controller 760 connecting to a low speed bus 770 and a storage device 730 .
- a processor 710 also referred to as processing hardware or data processing hardware
- memory 720 also referred to as memory hardware
- storage device 730 a high-speed interface/controller 740 connecting to the memory 720 and high-speed expansion ports 750
- a low speed interface/controller 760 connecting to a low speed bus 770 and a storage device 730 .
- Each of the components 710 , 720 , 730 , 740 , 750 , and 760 are interconnected using various busses, and may be mounted on a common motherboard or in other manners as appropriate.
- the processor 710 can process instructions for execution within the computing device 700 , including instructions stored in the memory 720 or on the storage device 730 to display graphical information for a graphical user interface (GUI) on an external input/output device, such as display 780 coupled to high speed interface 740 .
- GUI graphical user interface
- multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory.
- multiple computing devices 700 may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system).
- the memory 720 stores information non-transitorily within the computing device 700 .
- the memory 720 may be a computer-readable medium, a volatile memory unit(s), or non-volatile memory unit(s).
- the non-transitory memory 720 may be physical devices used to store programs (e.g., sequences of instructions) or data (e.g., program state information) on a temporary or permanent basis for use by the computing device 700 .
- non-volatile memory examples include, but are not limited to, flash memory and read-only memory (ROM)/programmable read-only memory (PROM)/erasable programmable read-only memory (EPROM)/electronically erasable programmable read-only memory (EEPROM) (e.g., typically used for firmware, such as boot programs).
- volatile memory examples include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM) as well as disks or tapes.
- the storage device 730 is capable of providing mass storage for the computing device 700 .
- the storage device 730 is a computer-readable medium.
- the storage device 730 may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid-state memory device, or an array of devices, including devices in a storage area network or other configurations.
- a computer program product is tangibly embodied in an information carrier.
- the computer program product contains instructions that, when executed, perform one or more methods, such as those described above.
- the information carrier is a computer- or machine-readable medium, such as the memory 720 , the storage device 730 , or memory on processor 710 .
- the high-speed controller 740 manages bandwidth-intensive operations for the computing device 700 , while the low speed controller 760 manages lower bandwidth-intensive operations. Such allocation of duties is exemplary only.
- the high-speed controller 740 is coupled to the memory 720 , the display 780 (e.g., through a graphics processor or accelerator), and to the high-speed expansion ports 750 , which may accept various expansion cards (not shown).
- the low-speed controller 760 is coupled to the storage device 730 and a low-speed expansion port 790 .
- the low-speed expansion port 790 which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), may be coupled to one or more input/output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter.
- input/output devices such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter.
- the computing device 700 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a standard server 700 a or multiple times in a group of such servers 700 a , as a laptop computer 700 b , or as part of a rack server system 700 c.
- implementations of the systems and techniques described herein can be realized in digital electronic and/or optical circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof.
- ASICs application specific integrated circuits
- These various implementations can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.
- the processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output.
- the processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).
- processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer.
- a processor will receive instructions and data from a read only memory or a random access memory or both.
- the essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data.
- a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks.
- mass storage devices for storing data
- a computer need not have such devices.
- Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks.
- the processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
- one or more aspects of the disclosure can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touch screen for displaying information to the user and optionally a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer.
- a display device e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touch screen for displaying information to the user and optionally a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer.
- Other kinds of devices can be used to provide interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Cardiology (AREA)
- General Health & Medical Sciences (AREA)
- Environmental & Geological Engineering (AREA)
- Computer And Data Communications (AREA)
- Mobile Radio Communication Systems (AREA)
- Debugging And Monitoring (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
In the presence of a network fault condition preventing communication between a communication device and a network device of a first network, a method includes receiving a heartbeat request from the communication device requesting permission for the communication device to perform a device operation. The method further includes obtaining device state data associated with the communication device and determining whether the communication device is permitted to perform the device operation based on the device state data associated with the communication device. When the communication device is permitted to perform the device operation, the method also includes transmitting a heartbeat confirmation signal to the communication device. The heartbeat confirmation signal permits the communication device to perform the device operation.
Description
- This U.S. patent application is a continuation of, and claims priority under 35 U.S.C. § 120 from, U.S. patent application Ser. No. 17/938,565, filed on Oct. 6, 2022, which is continuation of U.S. patent application Ser. No. 17/278,649, now U.S. Pat. No. 11,496,383, filed on Mar. 22, 2021, which is a national stage application of, and claims priority under 35 U.S.C. § 371 from PCT/US2019/054038, filed on Oct. 1, 2019, which claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Application 62/743,040, filed on Oct. 9, 2018. The disclosures of these prior applications are considered part of the disclosure of this application and are hereby incorporated by reference in their entireties.
- This disclosure relates to reliable operation of network-connected devices in a cloud-degraded mode.
- There are cloud services that require very high degrees of operational reliability to support internet of things (IoT) devices and other devices. The devices tend to need a heartbeat (also referred to as a keep alive signal) back from the cloud-environment to ensure that the device may continue to do the task or operation that was ongoing. At the point that the devices become critical, an issue arises that the cloud service becomes a large part of the total system reliability.
- Some ways to increase reliability of cloud services include duplicating the entire system in more than one location, effectively providing N+1 redundancy. However, this technique is difficult to manage, and full consistency between the two systems can be complex and costly
- A primary network, such as a primary cloud network includes a network device such as one or more servers, that stores device state data for various devices that are in communication with the primary network. For example, the devices include internet of things (IoT) devices, wireless base stations, end-user devices (wireless mobile and wired devices) or any other devices. A backup network device, such as one or more servers, in a secondary network such as a secondary cloud network, in one example, is a smaller system and backs up a subset of the state data that is kept by the primary network. In some implementations, the backup device in the secondary network keeps a copy of valid heartbeat confirmation data (keep alive signal data) that is obtained from the primary network. In one example, only keep alive signal data is backed up by the secondary network. However, any suitable subset of device state data may also be backed up as long as it includes the backup of the keep alive data, also referred to as heartbeat confirmation data.
- In some implementations, a system is composed of a primary service provided by one or more servers in the primary cloud infrastructure and a heartbeat-only system (HBO) that is maintained through a backup device in a secondary cloud infrastructure. The secondary network is used during a fault condition that occurs in connection with the primary network. The system, method, and apparatus in some implementations also allow an invalidation agent to invalidate heartbeat confirmation data before, during, or after the fault condition.
- The heartbeat confirmation data is data indicating that the device was ok to continue operations with the primary cloud. In one example, the heartbeat confirmation data is stored in a table in the secondary network, allowing a second system to provide maximum separation without having a fully rewritten software service. This allows for the maintaining of consistent operation of an IoT device or other device even during a fault condition of the primary network without having to have full consistent copies of state data at all times across multiple cloud infrastructures, as well as having a smaller, independent service to ensure reliability across portions of the cloud networks. Serving heartbeats may be replicated with significantly simpler states to the point of potentially a static web page handler. However, any suitable operation may be employed to serve heartbeats. The secondary cloud system being updated by the primary cloud system with device heartbeat confirmation data makes maintaining the consistent state significantly simpler.
- Aspects of the present disclosure provide for reliable operation of network-connected devices in a cloud-degraded mode. One aspect of the disclosure provides a method for providing operational reliability of a communication device adapted to communicate with a first network. In the presence of a network fault condition preventing communication between the communication device and a network device of the first network, the method includes receiving, at data processing hardware of a secondary network, a heartbeat request from the communication device requesting permission for the communication device to perform a device operation. The method further includes obtaining, by the data processing hardware, device state data associated with the communication device. The method also includes determining, by the data processing hardware, whether the communication device is permitted to perform the device operation based on the device state data associated with the communication device. When the communication device is permitted to perform the device operation, the method also includes transmitting, by the data processing hardware, a heartbeat confirmation signal to the communication device. The heartbeat confirmation signal permits the communication device to perform the device operation.
- Implementations of the disclosure may include one or more of the following optional features. Obtaining the device state data may include receiving the device state data associated with the communication device from the network device of the first network. In some implementations, when the device state data is received before receiving the heartbeat request from the communication device and when the network fault condition is not present, the method includes storing the device state data in a backup datastore, retrieving the device state data from the backup datastore using a device identifier identifying the communication device, and associating the communication device with the device state data stored in the backup datastore. In some examples, determining whether the communication device is permitted to perform the device operation includes determining whether the device state data includes heartbeat confirmation data. The heartbeat confirmation data may indicate permission by the network device of the first network for the communication device to perform the operation. When the device state data includes heartbeat confirmation data, the method may include determining that the communication device is permitted to perform the device operation. When the device state data does not include heartbeat confirmation data, the method may include determining that the communication device is not permitted to receive the heartbeat confirmation signal. In some examples, the method includes preventing transmission of the heartbeat confirmation signal to the communication device when the device state data does not include heartbeat confirmation data, thereby preventing the communication device from performing the device operation.
- In some implementations, determining that the communication device is permitted to perform the device operation includes determining whether the device state data includes invalidation data that invalidates the communication device from the ability to perform the device operation. In these implementations, when the device state data includes heartbeat confirmation data and does not include invalidation data, the method includes determining that the communication device is permitted to perform the device operation. However, when the device state data includes invalidation data as well as heartbeat confirmation data, the method may include rescinding permission for the communication device to perform the device operation and withholding transmission of the heartbeat confirmation signal to the communication device, thereby preventing the communication device from performing the operation. The invalidation data may indicate that the communication device requires a software update, the communication device is out of compliance with contractual agreements, and/or a resource necessary for the communication device to perform the device operation is unavailable to the communication device.
- In some examples, the network device of the first network is configured to push device state data associated with the communication device to the data processing hardware prior to the network fault condition. Additionally or alternatively, the network device of the first network may be configured to send the device state data associated with the communication device in response to a pull operation by the data processing hardware. The device state data may include a device identifier identifying the communication device, invalidation data that invalidates the communication device from the ability to perform the device operation, operational reliability information associated with the communication device, a payment status, an ability of the communication device to update organizational metadata, and/or an aggregation calculation of overall environment health.
- When the network fault condition is not present, the communication device may be configured to send the heartbeat request for permission to perform a device operation to the network device of the first network and the network device of the first network may be configured to send the heartbeat confirmation signal to the communication device. The network device may also be configured to update the device state data associated with the communication device to include the heartbeat confirmation signal sent to the communication device. In some implementations, the network fault condition is present when the communication device does not receive a heartbeat confirmation signal from the network device of the first network within a threshold period of time after sending a heartbeat request to the network device. The communication device may include an Internet of Things (IoT) device, a temperature sensor, a wireless base station, a mobile device, and/or a smart appliance.
- Another aspect of the disclosure provides a system for reliable operation of network-connected devices in a cloud-degraded mode. One aspect of the disclosure includes data processing hardware of a backup network and memory hardware in communication with the data processing hardware. The memory hardware may store instructions that when executed on the data processing hardware cause the data processing hardware to perform operations. In the presence of a network fault condition preventing communication between the communication device and a network device of the first network, the operations include receiving a heartbeat request from the communication device. The heartbeat request requests permission for the communication device to perform a device operation. The operations further include obtaining device state data associated with the communication device and determining whether the communication device is permitted to perform the device operation based on the device state data associated with the communication device. When the communication device is permitted to perform the device operation, the operations include transmitting a heartbeat confirmation signal to the communication device. The heartbeat confirmation signal permits the communication device to perform the device operation.
- Implementations of the disclosure may include one or more of the following optional features. Obtaining the device state data may include receiving the device state data associated with the communication device from the network device of the first network. In some implementations, when the device state data is received before receiving the heartbeat request from the communication device and when the network fault condition is not present, the operations include storing the device state data in a backup datastore, retrieving the device state data from the backup datastore using a device identifier identifying the communication device, and associating the communication device with the device state data stored in the backup datastore. In some examples, determining whether the communication device is permitted to perform the device operation includes determining whether the device state data includes heartbeat confirmation data. The heartbeat confirmation data may indicate permission by the network device of the first network for the communication device to perform the device operation. When the device state data includes heartbeat confirmation data, the operations may include determining that the communication device is permitted to perform the device operation. When the device state data does not include heartbeat confirmation data, the operations may include determining that the communication device is not permitted to receive the heartbeat confirmation signal. The operations may include preventing transmission of the heartbeat confirmation signal to the communication device when the device state data does not include heartbeat confirmation data, thereby preventing the communication device from performing the device operation.
- In some implementations, determining the communication device is permitted to perform the device operation includes determining whether the device state data includes invalidation data that invalidates the communication device from the ability to perform the device operation. In these implementations, when the device state data includes heartbeat confirmation data and does not include invalidation data, the operations include determining that the communication device is permitted to perform the device operation. However, when the device state data includes invalidation data as well as heartbeat confirmation data, the operations may include rescinding permission for the communication device to perform the device operation and withholding transmission of the heartbeat confirmation signal to the communication device, thereby preventing the communication device from performing the operation. The invalidation data may indicate that the communication device requires a software update, the communication device is out of compliance with contractual agreements, and/or a resource necessary for the communication device to perform the device operation is unavailable to the communication device.
- In some examples, the network device of the first network is configured to push device state data associated with the communication device to the data processing hardware prior to the network fault condition. Additionally or alternatively, the network device of the first network may be configured to send the device state data associated with the communication device in response to a pull operation by the data processing hardware. The device state data may include a device identifier identifying the communication device, invalidation data that invalidates the communication device from the ability to perform the device operation, operational reliability information associated with the communication device, a payment status, an ability of the communication device to update organizational metadata, and/or an aggregation calculation of overall environment health.
- When the network fault condition is not present, the communication device may be configured to send the heartbeat request to the network device of the first network and the network device of the first network may be configured to send the heartbeat confirmation signal to the communication device. The network device may also be configured to update the device state data associated with the communication device to include the heartbeat confirmation signal sent to the communication device. In some implementations, the network fault condition is present when the communication device does not receive a heartbeat confirmation signal from the network device of the first network within a threshold period of time after sending a heartbeat request to the network device. The communication device may include an Internet of Things (IoT) device, a temperature sensor, a wireless base station, a mobile device, and/or a smart appliance.
-
FIG. 1 is a schematic view of an example network system including a primary network and a secondary network. -
FIG. 2 is a schematic view of an example network device employed in the primary network of the network system ofFIG. 1 . -
FIG. 3 is a schematic view of an example backup device employed in the secondary network of the network system ofFIG. 1 . -
FIG. 4 is a flowchart of an example arrangement of operations for a method of providing operational reliability for a communication device adapted to communicate with a primary network. -
FIG. 5 is a flowchart of an example arrangement of operations for a method of providing operational reliability for a communication device adapted to communicate with a primary network. -
FIG. 6 is a flowchart of an example arrangement of operations for a method of providing operational reliability for a communication device adapted to communicate with a primary network. -
FIG. 7 is a schematic view of an exemplary computing device. - Like reference symbols in the various drawings indicate like elements.
-
FIG. 1 illustrates one example of anetwork system 100 that employs aprimary network 102 and asecondary network 104. In one example, theprimary network 102 is a cloud network infrastructure that is operated by a first provider and thesecondary network 104 is a different cloud network infrastructure that is operated by a different service provider. However, any suitable network configurations may be employed.Various communication devices primary network 102 through techniques including through wireless or wired interfaces. While examples of five different types ofcommunication devices 106 are shown, any number ofcommunication devices 106 may register and communicate with theprimary network 102 at any given time. For instance, somecommunication devices Other communication devices communication devices 106 communicate with theprimary network 102 via internet protocol (IP)-based communication protocols or any other suitable protocols. In one example, the one ormore communication devices 106 can communicate with theprimary network 102 andsecondary network 104 through a domain name system (DNS) as part of an Internet-based cloud network system. Theprimary network 102 includes aprimary network device 200. The network device 200 (e.g.primary network device 200FIG. 1 )) may be one or more servers such as web servers that communicate with one or more of thecommunication devices 106. As such, thenetwork device 200 includes data processing hardware 210 andmemory hardware 220 in communication with the data processing hardware 210. In some examples, thenetwork device 200 executes a software application or service, such as aWeb Registration Service 202.Communication devices 106 may register with theWeb Registration Service 202 to receive a heartbeat confirmation signal 126 (also referred to as a heartbeat confirmation or keep alive signal) to permit thecommunication devices 106 to continue to perform tasks or operations that were ongoing. All of thecommunication devices 106 may be of the same type, such as all IoT type devices, or may be various combinations of differing types of devices such as smartphones and wireless base stations that the smartphones may wish to employ for communication purposes. An exemplaryprimary network 102 includes a citizen's broadband radio services (CBRS) system; however, theprimary network 102 may be any suitable system. - A
communication device 106 such as anIoT device 106 a sends aheartbeat request 128 to the network device 200 (e.g., data processing hardware 210) in theprimary network 102 requesting that thenetwork device 200 send back aheartbeat confirmation signal 126. Theheartbeat confirmation signal 126 may includeheartbeat confirmation data 118. When theIoT device 106 a receives theheartbeat confirmation signal 126, theIoT device 106 a continues executing a current task or operation with theprimary network 102 since nonetwork fault condition 130 is present. The communication ofheartbeat requests 128 and heartbeat confirmation signals 126 between a communication device and thenetwork device 200 can occur at least once every 20 seconds, 240 seconds, or any suitable period. In some implementations, theserver 200 recordsdevice state data 110 indicatingheartbeat confirmation data 118 issued to eachparticular communication device 106 and stores thedevice state data 110 together with a device identifier (ID) 114 associated with theparticular communication device 106 in a device state datastore 212 in communication with thenetwork device 200. In some examples, the device state datastore 212 resides on thememory hardware 220 of thenetwork device 200. Thedevice state data 110 stored in the device state datastore 212 includes other information, including information related to executing the current task or operation with theprimary network 102.Device state data 110 may include, without limitation, data representing payment status, an ability to update organizational metadata, aggregation calculations of overall environment health, and/or any other suitable data. As long as theIoT device 106 a receives a correspondingheartbeat confirmation signal 126 responsive to issuing aheartbeat request 128, theIoT device 106 a knows that it can continue its operation with theprimary network 102. However, theIoT device 106 a will recognize that anetwork fault condition 130 occurred with theprimary network 102, e.g., when aheartbeat confirmation signal 126 is not received from theprimary network 102. In some examples,communication devices 106 recognize anetwork fault condition 130 is present when noheartbeat confirmation signal 126 is received within a predetermined period of time after issuing/sending aheartbeat request 128. - In some examples, the
primary network 102 is a cloud network of a first provider. The cloud network 102 (e.g. primary network 102 (FIG. 1 )) may include aPrimary Network Device 200. The Primary Network Device may execute aWeb Registration Service 202. In some examples, the communication device registers with theWeb Registration Service 202 to perform operations with theprimary network 102. Thedevice state data 110 may be state data that is stored while thecommunication device 106 is performing operations with the primary cloud network. For example, when theIoT device 106 a is a mobile device communicating in a CBRS system and theprimary network 102 corresponds to a spectrum analysis system (SAS) of the CBRS system, theIoT device 106 a may send theheartbeat request 128 to theprimary network device 200 of theprimary network 102 and receive a correspondingheartbeat confirmation signal 126 indicating that theIoT device 106 a is able to communicate with theprimary network 102. As such, thenetwork device 200 of theprimary network 102 may store theheartbeat confirmation data 118 asdevice state data 110 linked to theIoT device 106 a via thecorresponding device ID 114 in the device state datastore 212, whereby thedevice state data 110 indicates that an operating instance of theIoT device 106 a in theprimary network 102 is proper. That is, that theIoT device 106 a is properly registered with theWeb Registration Service 202. - In some examples, the
secondary network 104 includes abackup network device 300. Thebackup network device 300 may be referred to as afallback network device 300,backup device 300, or amirror network device 300. Thenetwork device 300 may be one or more servers such as web servers that communicate with one or more of thecommunication devices 106. As such, thebackup network device 300 includesdata processing hardware 310 andmemory hardware 320 in communication with thedata processing hardware 310. Thememory hardware 320 may store instructions that, when executed on thedata processing hardware 310, cause thedata processing hardware 310 to perform operations such as those described with respect toflowcharts FIGS. 4-6 ). In this example, thebackup network device 300 of thesecondary network 104 stores, or mirrors, a subset of thedevice state data 110 that is stored in thedatastore 212 of theprimary network 102 by theprimary network device 200. The subset ofstate data 110 obtained from theprimary network 102 is backed up by thebackup network device 300 and is shown inFIG. 1 as backed-updevice state data 110. Hence, thenetwork device 300 serves as a backup, or fallback, device for a subset of thedevice state data 110, e.g., during anetwork fault condition 130 associated with theprimary network 102 as perceived by thecommunication device 106. The subset ofstate data 110 is provided by theprimary network 102 through any suitable mechanism including being pushed to thesecondary network 104 or pulled by thesecondary network 104 as part of a backup operation associated with a cloud instance of an application running in the cloud. A software application executing on thebackup device 300 may perform the backup operation. In this example, thesecondary network 104 is a smaller cloud network than that of theprimary network 102 thereby reducing cost and operational complexity of a backup system. In addition, thebackup device 300 in thesecondary network 104 only backs up a subset of thestate data 110 that is kept by theprimary network 102 for each of the devices that are in communication with theprimary network 102, thereby reducing storage cost and operational complexity. For example, thesecondary network 104 may not allowcommunication devices 106 to newly register. I.e., in some implementations, thebackup device 300 does not execute a Web Registration Service. - The
network system 100 also includes an invalidation agent 120 (e.g. heartbeat invalidation agent 120 (FIG. 1 )) that may be any suitable device that is authorized to invalidate acommunication device 106 with respect to theprimary network 102 and thesecondary network 104. Theinvalidation agent 120 may providecorresponding invalidation data primary network device 200 associated with theprimary network 102 and/orcorresponding invalidation data backup device 300 associated with thesecondary network 104. A software service provider, system administrator, or other entity may employ theinvalidation agent 120 to identifyparticular communication devices 106 that should not receive heartbeat confirmation signals 126 from thenetwork particular communication devices 106. More specifically, theinvalidation data 124 invalidates aparticular communication device 106 from the ability to perform tasks or operations by instructing theprimary network device 200 and/or thebackup device 300 to not transmit heartbeat confirmation signals 126 to theparticular communication device 106. Theinvalidation agent 120 may determine that a resource necessary for thecommunication device 106 to perform the device operation is unavailable to thecommunication device 106. This may be done in a CBRS system, for example, if theinvalidation agent 120 determines that a particular frequency or set of frequencies are not available frombase station 106 c for use bycommunication device 106 b. For example, a frequency or certain frequencies used by one or more base stations (e.g., 106 c) in a CBRS system may be determined by the system and/or theinvalidation agent 120 to be in use by a governmental entity and are unavailable for use by non-governmental devices.Communication device 106 b can be prevented from attempting to access those frequencies. In other systems, an Internet of Things (IoT)device 106 a may be prevented from using theprimary network 102 when a software update is required, when the owner of the communication device is out of compliance with contractual agreements made in connection with software usage, or other criteria. - The
backup device 300 includes memory such as abackup datastore 318 that stores the backed-up subset ofdevice state data 110, includingheartbeat confirmation data 118. Theheartbeat confirmation data 118 includes a copy of the validheartbeat confirmation data 118 that is obtained from theprimary network 102. In some implementations, theinvalidation data 124 a provided to theprimary network device 200 is propagated to thebackup device 300 and stored in thebackup datastore 318. For instance, the subset ofstate data 110 backed up by thebackup network device 300 may include data representing the mapping of data between the communication device and theinvalidation agent 120. In some implementations, theinvalidation data 124 b provided to thebackup network device 300 is stored in thebackup datastore 318. In another example, only keep alive signal data 118 (heartbeat confirmation data) is backed up by thesecondary network 104. This can allow for a small use of memory such as in a table format. - As such, the
backup device 300 of thesecondary network 104 backs up only operational reliability information of a particular communication device that is communicating with theprimary network 102. In one example, only the keep alive signal data, also referred to as theheartbeat confirmation data 118, is backed up from theprimary network 102. Theheartbeat confirmation data 118 is associated with an ongoing communication or task performed by a device with theprimary network 102, and may indicate thatprimary network device 200 of theprimary network 102 permits thecommunication device 106 to continue performing the task or operation in the presence of anetwork fault condition 130 preventing communication between thecommunication device 106 and theprimary network device 200. In some implementations, theinvalidation data 124 b is provided directly to thebackup device 300 duringnetwork fault conditions 130 associated with theprimary network 102. The operations described herein provide a type of If This Then That—level of functionality for the heartbeat deauthorization to still propagate, rescinding permission for the communication device to continue performing tasks or operations duringnetwork fault conditions 130 associated with theprimary network 102. For instance, theinvalidation agent 120 may transmit aninvalidation message 122 to thebackup device 300 when theinvalidation agent 120 detects anetwork fault condition 130 associated with theprimary network 102. Thenetwork system 100 andmethods primary network 102 whereas the heartbeat-only service is run on an entirely separate cloud infrastructure, namely thesecondary network 104, with the ability to confirm operability by thesecondary network 104 to thedevice 106 in response to anetwork fault condition 130 associated with theprimary network 102. - Referring to
FIG. 2 , thenetwork device 200 may execute aWeb Registration Service 202, andcommunication devices 106 may register with theWeb Registration Service 202 to receive a heartbeat signal 126 (also referred to as a heartbeat confirmation signal, a keep alive signal, or simply a heartbeat) to permit thecommunication devices 106 to initiate or to continue to perform tasks or operations. In the example shown, acommunication device 106 may send theheartbeat request 128 to thenetwork device 200 and receive a correspondingheartbeat confirmation signal 126. In some implementations,communication devices 106 continuously receive heartbeat signals 126 (e.g., at regular intervals) and continue to perform tasks or operations responsive to receiving the heartbeat signals 126. In some examples, for aparticular communication device 106 communicating with the network device 200 (e.g., via communication ofheartbeat requests 128 and corresponding heartbeat confirmation signals 126), thenetwork device 200 recordsdevice state data 110 together with a device identifier (ID) 114 associated with theparticular communication device 106 in a device state datastore 212. Thedevice ID 114 may uniquely identify thecommunication device 106. Thedevice state data 110 may include any information related to operations or tasks performed by thecommunication device 106, such as, without limitation, data representing payment status, an ability to update organizational metadata, aggregation calculations of overall environment health, and/or any other suitable data. Thedevice ID 114 may be used as an index, e.g., to look up or identify thedevice state data 110 stored in thedatastore 212 and associated with theparticular communication device 106. Thedevice state data 110 associated with aparticular communication device 106 may be updated over time using thedevice ID 114 as an index. For example, thenetwork device 200 may update thedevice state data 110 as thecommunication device 106 continues to perform tasks or operations responsive to receiving heartbeat confirmation signals 126 from thenetwork device 200. In some examples, thedevice state data 110 includesheartbeat confirmation data 118 associated with heartbeat confirmation signals 126 sent to thecommunication device 106 by thenetwork device 200 to permit thecommunication device 106 to perform tasks or operations. - In some implementations, the
network device 200 updates theheartbeat confirmation data 118 each time thenetwork device 200 sends aheartbeat signal 126 to thecommunication device 106. As such, theheartbeat confirmation data 118 may provide a log of timestampedheartbeat requests 128 received from thecommunication device 106 and timestamped heartbeat signals 126 sent to thecommunication device 106. In the example shown, thedevice state data 110 may reflect the most recent time thenetwork device 200 permitted thecommunication device 106 to initiate or to continue to perform tasks or operations. In some examples, thenetwork device 200 receives aninvalidation message 122 from aninvalidation agent 120. Theinvalidation message 122 may contain heartbeat invalidationdata communication device 106 should not receive any heartbeat confirmation signals 126 from theprimary network device 200. For example, an Internet-of-Things (IoT)device 106 a may be prevented from using theprimary network 102 when theIoT device 106 a requires a software update to address a potential security vulnerability, or when the owner of theIoT device 106 a fails to comply with contractual agreements, such as obtaining or maintain valid software licenses. Theinvalidation message 122 may contain heartbeat invalidationdata communication device 106 should not receive any heartbeat confirmation signals 126 from theprimary network device 200 as possible. Theinvalidation data 124 a may contain the device identifier 114 (e.g., device ID 114 (FIG. 1 )) identifying theparticular communication device 106 that should not receive heartbeat confirmation signals 126 containingheartbeat confirmation data 118 indicative of permission by the primary network device 200 (e.g., by the Web Registration Service 202) for theparticular communication device 106 to perform tasks or operations. Accordingly, in response to receivinginvalidation data 124 a contained in aninvalidation message 122 from the invalidation agent 120 (e.g. heartbeat invalidation agent 120 (FIG. 1 )), thenetwork device 200 may decline to transmit aheartbeat confirmation signal 126 to thecommunication device 106 identified by theinvalidation data 124 a. For instance, when thenetwork device 200 receives aheartbeat request 128 from acommunication device 106 for performing a task or operation, thenetwork device 200 may query thedatastore 212 using the associateddevice ID 114 to determine whether thedevice state data 110 for thecommunication device 106 includesinvalidation data 124. Wheninvalidation data 124 exists, thenetwork device 200 may decline to transmit a correspondingheartbeat confirmation signal 126 to thecommunication device 106, thereby preventing thecommunication device 106 from performing the task or operation. In some examples, when thenetwork device 200 ceases the transmission of heartbeat confirmation signals 126 to thecommunication device 106, thenetwork device 200 stops updating theheartbeat confirmation data 118 in thedevice state data 110. In some examples, thedevice state data 110 includes the receivedinvalidation data 124 a. As such, thenetwork device 200 may be configured to update thedevice state data 110 withinvalidation data 124 a received from theinvalidation agent 120 indicating that theparticular communication device 106 should no longer receive heartbeat confirmation signals 126 from thenetwork device 200. - Referring to
FIG. 3 , thebackup device 300 may execute abackup controller 302 and aconfirmation controller 304. Theconfirmation controller 304 may be referred to as abackup confirmation controller 304 or aheartbeat confirmation controller 304. In some examples, a network fault condition 130 (FIG. 1 ) associated with theprimary network 102, as perceived by thecommunication device 106, may occur. For example, thecommunication device 106 may be unable to connect to theprimary network 102 or may fail to receive aheartbeat confirmation 126 within a threshold period of time after sending aheartbeat request 128. Thenetwork fault condition 130 associated with theprimary network 102 may be caused by a power outage that affects theprimary network 102 or a damaged communication cable or fiber, or any other failure that prevents communication between thecommunication device 106 and theprimary network 102. In some examples, thenetwork fault condition 130 is due to a failure of the ability of thecommunication device 106 to communicate with theprimary network 102 even though theprimary network 102 is otherwise operational. Other sources of thenetwork fault condition 130 may apply as well. In some examples, when thecommunication device 106 fails to receive aheartbeat confirmation 126 within a threshold period of time, thecommunication device 106 attempts to re-register with theWeb Registration Service 202. In these examples, anetwork fault condition 130 may be established when thecommunication device 106 is unable to re-register with theWeb Registration Service 202 within a connection timeout period. Since thecommunication device 106 may not receive heartbeat confirmation signals 126 from theprimary network 102 during thenetwork fault condition 130, thecommunication device 106 will cease performing tasks or operations conditioned upon receiving periodic heartbeat confirmation signals 126. In some examples, during anetwork fault condition 130 associated with the primary network, thecommunication device 106 is configured to initiate communication with thesecondary network 104 in lieu of theprimary network 102. That is, thesecondary network 104 may provide a fallback for servicingheartbeat requests 128 from thecommunication device 106 and issuing heartbeat confirmation signals 126 to thecommunication device 106 duringnetwork fault conditions 130 when thecommunication device 106 is unable to receive heartbeat confirmation signals 126 from theprimary network device 200 of theprimary network 102. In the example shown, thecommunication device 106 sends aheartbeat request 128 to the backupheartbeat confirmation controller 304 of thebackup device 300 during thenetwork fault condition 130 and receives a corresponding heartbeat confirmation signal 126 (also referred to as a keep alive signal) to permit thecommunication device 106 to continue to perform tasks or operations despite the inability of thecommunication device 106 to communicate with theprimary network device 200 of theprimary network 102. Here, before sending theheartbeat confirmation signal 126 to thecommunication device 106, the backupheartbeat confirmation controller 304 may first query thebackup controller 302 to determine whether the backed-updevice state data 110 associated with thecommunication device 106 indicates that thecommunication device 106 is permitted to receive heartbeat confirmation signals 126. - In the example shown, the device
heartbeat backup controller 302 receives at least a subset ofdevice state data 110 from theprimary network device 200 of theprimary network 102. The subset ofstate data 110 may includeheartbeat confirmation data 118, thedevice ID 114 associated with thecommunication device 106, and anyinvalidation data 124 a the primary network registration device 200 (e.g. primary network device 200 (FIG. 1 )) received from theheartbeat invalidation agent 120. In some examples, the received state data 110 (e.g., device state data 110 (FIG. 1 )) includes all thestate data 110 associated with thecommunication device 106 that is stored in the device state datastore 212 of theprimary network 102 as described previously. In some examples, the receivedstate data 110 only includes theheartbeat confirmation data 118. Other subsets of thestate data 110 may be received by thebackup controller 302. In some implementations, theprimary network device 200 pushes thedevice state data 110 to thebackup controller 302 while theprimary network device 200 is sending heartbeat confirmation signals 126 to thecommunication device 106. In these implementations, theprimary network device 200 may push thedevice state data 110 associated with thecommunication device 106 to thebackup controller 302 at periodic intervals, when thestate data 110 changes, or under any other suitable circumstances. In additional implementations, thebackup controller 302 of thebackup device 300 pulls thedevice state data 110 from theprimary network 102 during a pull operation. For example, thebackup controller 302 may pull thedevice state data 110 from theprimary network device 200 as part of a scheduled backup operation and/or responsive to thebackup confirmation controller 304 receiving aheartbeat request 128 from the communication device. For instance, thebackup controller 302 may pull a subset ofstate data 110 for one ormore communication devices 106 by querying theprimary network device 200 with the associated one ormore device IDs 114. In some examples, thebackup controller 302 backs up thedevice state data 110 received from theprimary network device 200 in abackup datastore 318. As with the device state datastore 212 ofFIG. 2 , thebackup datastore 318 may index the receiveddevice state data 110 stored therein using thedevice ID 114 identifying thecommunication device 106 associated with thedevice state data 110. In one example, thebackup device 300 includes a look-up table, indexed by thedevice ID 114. Other techniques for storing, indexing, looking-up, and retrieving the receiveddevice state data 110 from thebackup datastore 318 are possible as well. - As described above with reference to
FIG. 2 , the receiveddevice state data 110 may reflect the most recent time thenetwork device 200 permitted thecommunication device 106 to initiate or to continue to perform tasks or operations. Accordingly, the receiveddevice state data 110 stored in thebackup datastore 318 may reflect the most recent time thenetwork device 200 permitted thecommunication device 106 to initiate or to continue to perform tasks or operations. Here, thedevice state data 110 received by thebackup controller 302 may also includeheartbeat invalidation data primary network device 200 of theprimary network 102 in aninvalidation message 122 from theinvalidation agent 120. That is, the deviceheartbeat backup controller 302 may receiveinvalidation data primary network 102, rather than directly from theheartbeat invalidation agent 120. Accordingly, the receiveddevice state data 110 stored in thebackup datastore 318 may reflect the indication that thecommunication device 106 should not receive any heartbeat confirmation signals 126. In some examples, thebackup controller 302 receives aninvalidation message 122 directly from aninvalidation agent 120. Theinvalidation message 122 may contain heartbeat invalidationdata invalidation data FIG. 1 )) indicating that thecommunication device 106 should not receive any heartbeat confirmation signals 126 from thebackup confirmation controller 304 of thesecondary network 104 in response to thecommunication device 106 sending aheartbeat request 128 to thebackup confirmation controller 304. Theinvalidation data 124 b may contain thedevice ID 114 identifying theparticular communication device 106 that should not receive heartbeat confirmation signals 126 containingheartbeat confirmation data 118 that would otherwise permit theparticular communication device 106 to perform tasks or operations. Theinvalidation agent 120 may providecorresponding invalidation data primary network device 200 associated with theprimary network 102 and/orcorresponding invalidation data backup controller 302 associated with thesecondary network 104. In some examples, theinvalidation agent 120 transmitsinvalidation messages 122 to thebackup device 300 of thesecondary network 104 only in the presence of anetwork fault condition 130. For instance, theinvalidation agent 120 may transmit aninvalidation message 122 containinginvalidation data 124 b to thebackup controller 302 when theinvalidation agent 120 detects, or otherwise learns of, anetwork fault condition 130 preventing communication between aparticular communication device 106 and theprimary network device 200 of theprimary network 102. In some examples, responsive to receiving aninvalidation message 122 associated with aparticular communication device 106, thebackup controller 302 updates thedevice state data 110 stored in the backup datastore 318 to include theinvalidation data 124 b associated with thecommunication device 106, e.g., indexed by thedevice ID 114. In the example shown, theheartbeat invalidation agent 120 may useinvalidation data 124 associated with thecommunication device 106 to invalidate anyheartbeat confirmation data 118 associated with thecommunication device 106 during anetwork fault condition 130 associated with the primary network, thereby preventing thebackup controller 302 of thebackup device 300 from sending any heartbeat confirmation signals. - In some examples, responsive to receiving a
heartbeat request 128 from acommunication device 106, the backupheartbeat confirmation controller 304 determines whether to send aheartbeat confirmation 126 to thecommunication device 106 to permit thecommunication device 106 to initiate or to continue to perform tasks or operations. Theheartbeat confirmation controller 304 may determine whether to send aheartbeat confirmation signal 126 to thecommunication device 106 based on the backed-updevice state data 110 stored in thebackup datastore 318. Here, thebackup confirmation controller 304 may query thebackup controller 302 to retrieve the backed updevice state data 110 from thebackup datastore 318 using the associateddevice ID 114. In some examples, the backupheartbeat confirmation controller 304 provides theheartbeat request 128 and associateddevice ID 114 to thebackup controller 302. Thebackup controller 302 may retrieve the backed-updevice state data 110, e.g. using the associateddevice ID 114 and provide the backed-updevice state data 110 to the backupheartbeat confirmation controller 304. In some examples, the backupheartbeat confirmation controller 304 retrieves the backed-updevice state data 110 directly from thebackup datastore 318. The backupheartbeat confirmation controller 304 may use other techniques to retrieve the backed-updevice state data 110. In the alternative, thebackup controller 302 may determine that the retrieved backed-updevice state data 110 includesheartbeat confirmation data 118 without anyinvalidation data 124 and provides a signal to thebackup confirmation controller 304 indicating thatcommunication device 106 should receive theheartbeat confirmation signal 126. In some examples, thebackup controller 302 provides the retrieved backed-updevice state data 110 to the backupheartbeat confirmation controller 304 and the backupheartbeat confirmation controller 304 determines whether or not to send aheartbeat confirmation signal 126 to thecommunication device 106 based on the backed-updevice state data 110. For instance, if thedevice state data 110 includesheartbeat confirmation data 118 and noinvalidation data 124 is present, thebackup confirmation controller 304 may determine to send theheartbeat confirmation signal 126 to thecommunication device 106. As described above, theheartbeat confirmation data 118 may reflect the most recent time thenetwork device 200 permitted thecommunication device 106 to initiate or to continue to perform tasks or operations. In some examples, the backupheartbeat confirmation controller 304 may determine to send aheartbeat confirmation 126 to thecommunication device 106 when theheartbeat confirmation data 118 in thebackup datastore 318 indicates that thenetwork device 200 previously permitted the communication device to initiate or to continue to perform tasks or operations. The backupheartbeat confirmation controller 304 may determine not to send aheartbeat confirmation 126 to thecommunication device 106 when the backed-updevice state data 110 does not includeheartbeat confirmation data 118 associated with thecommunication device 106. Other techniques for determining whether to send aheartbeat confirmation 126 to thecommunication device 106 based on theheartbeat confirmation data 118 are possible. In some examples, when the backupheartbeat confirmation controller 304 sends aheartbeat confirmation signal 126 to thecommunication device 106, the backupheartbeat confirmation controller 304 updates theheartbeat confirmation data 118 associated with thecommunication device 106 in thebackup datastore 318. - The backed-up
device state data 110 may includeinvalidation data 124 received indirectly by thebackup controller 302 via theprimary network device 200 of theprimary network 102 as part ofdevice state data 110 received by thebackup controller 302 during a push or pull operation or received directly from theinvalidation agent 120 and stored in thebackup datastore 318. Similarly, if theheartbeat invalidation agent 120 providesinvalidation data 124 b to thecontroller 302 while thecontroller 302 is performing a backed-updevice state data 110 retrieval operation, thebackup controller 302 may simply append theinvalidation data 124 to the backed-updevice state data 110 being sent to thebackup confirmation controller 304. In some examples, based on theinvalidation data 124, thebackup confirmation controller 304 may decline to transmit aheartbeat confirmation signal 126 to thecommunication device 106 identified by the invalidation data 124 (e.g., heartbeat invalidation data 124 (FIG. 1 )). For instance, when thebackup confirmation controller 304 receives aheartbeat request 128 from acommunication device 106 for performing a task or operation, the backup confirmation controller 304 (or the backup controller 302) may query thebackup datastore 318 using thedevice ID 114 to determine whether thedevice state data 110 associated with thecommunication device 106 includesinvalidation data 124. Wheninvalidation data 124 exists, thebackup confirmation controller 304 may rescind permission for thecommunication device 106 to perform the device operation and withhold, or decline to transmit, a correspondingheartbeat confirmation signal 126 to thecommunication device 106, thereby preventing thecommunication device 106 from performing the task or operation. -
FIG. 4 is a flowchart of an example arrangement of operations for amethod 400 of providing operational reliability for acommunication device 106 adapted to communicate with aprimary network 102. Atoperation 402, in the presence of anetwork fault condition 130 preventing communication between acommunication device 106 and a network device 200 (e.g., primary network device 200 (FIG. 1 )) of a first network 102 (e.g., primary network 102 (FIG. 1 )), themethod 400 includes receiving, atdata processing hardware 310 of a second network 104 (e.g.,backup network 104 or secondary network 104 (FIG. 1 )), from thecommunication device 106, aheartbeat request 128 requesting permission for thecommunication device 106 to perform a device operation. In some implementations, thecommunication device 106 is configured to send theheartbeat request 128 to thenetwork device 200 of thefirst network 102 and receive a correspondingheartbeat confirmation signal 126 from thenetwork device 200 when thenetwork fault condition 130 is not present. In some scenarios, thenetwork fault condition 130 is present when thecommunication device 106 does not receive aheartbeat confirmation signal 126 from thenetwork device 200 of thefirst network 102 within a threshold period of time after sending aheartbeat request 128 to thenetwork device 200. - At
operation 404, themethod 400 includes obtaining, by thedata processing hardware 310,device state data 110 associated with thecommunication device 106. For example, thedata processing hardware 310 may receive thedevice state data 110 directly or indirectly from thenetwork device 200 of thefirst network 102. For instance, thenetwork device 200 may periodically push thedevice state data 110 to thedata processing hardware 310 when communication between thecommunication device 106 and thenetwork device 200 is not degraded (e.g., thefault condition 130 is not present), and thedata processing hardware 310 may store thedevice state data 110 in abackup data store 318. Thus, thedata processing hardware 300 may access thebackup data store 318 during thenetwork fault condition 130 to retrievedevice state data 110 stored therein for aparticular communication device 106. Additionally or alternatively, thedata processing hardware 310 may perform a pull operation to obtain thedevice state data 110 from thenetwork device 200. Here, thedata processing hardware 310 may perform the pull operation by querying thenetwork device 200 to retrieve thedevice state data 110 for theparticular communication device 106. The query may include adevice identifier 114 identifying thecommunication device 106 so that thenetwork device 200 can retrieve the associateddevice state data 110 from adatastore 212. - At
operation 406, themethod 400 includes determining, by thedata processing hardware 310, whether thecommunication device 106 is permitted to perform the device operation based on thedevice state data 110 associated with thecommunication device 106. For example, when thedevice state data 110 includesheartbeat confirmation data 118 indicative of permission by thenetwork device 200 of thefirst network 102 for thecommunication device 106 to perform the device operation, themethod 400 may determine that thecommunication device 106 is permitted to perform the device operation. However, when thedevice state data 110 does not includeheartbeat confirmation data 118, themethod 400 may determine that thecommunication device 106 is not permitted to perform the device operation. Furthermore, when thedevice state data 110 includesinvalidation data 124 despite havingheartbeat confirmation data 118,method 400 may determine that thecommunication device 106 is not permitted to perform the device operation. - At
operation 408, themethod 400 includes, when thecommunication device 106 is permitted to perform the device operation, transmitting, by thedata processing hardware 310, aheartbeat confirmation signal 126 to thecommunication device 106. Here, theheartbeat confirmation signal 126 permits thecommunication device 106 to perform the device operation. Alternatively, in scenarios when thecommunication device 106 is not permitted to perform the device operation (e.g., noheartbeat confirmation data 118 and/ordevice state data 110 includes invalidation data 124), thedata processing hardware 310 may withhold transmission of theheartbeat confirmation signal 126 to thecommunication device 106, thereby preventing thecommunication device 106 from performing the device operation. - Referring to
FIGS. 5 and 6 , one example of the operation of abackup device 300 will be described. In some implementations, the deviceheartbeat backup controller 302 backs up a subset ofdevice state data 110 from theprimary network 102 that includesheartbeat confirmation data 118 that was stored asdevice state data 110, for one ormore communication devices primary network 102. The backup occurs by storing the subset ofstate data 110 in thedatastore 318 on a per-device identifier 114 basis. For example, datastore records may include adevice identifier 114 for each of thecommunication devices 106 a-106 e along with correspondingheartbeat confirmation data 118 that theprimary network 102 stored asdevice state data 110. Anetwork fault condition 130 may occur, such as an inability ofIoT device 106 a to connect to theprimary network 102 as indicated by an IP address failure resulting from a DNS attempt to connect to theprimary network 102 by the device that makes aheartbeat request 128 to theprimary network 102. - In one example, the
network fault condition 130 is indicated by the device, such as communication device making an Internet protocol address request through a DNS to thesecondary network 104 because the communication device was unable to receive a proper response when it attempted to connect to theprimary network 102 via a DNS attempt once connected to the secondary network 104 (the backup device) 300. TheIoT device 106 a issues aheartbeat request 128 to thesecondary network 104. In response to the heartbeat request from the device, the backupheartbeat confirmation controller 304 provides aheartbeat confirmation 126 to thecommunication device 106 based on the backed upheartbeat confirmation data 118 that is associated with theheartbeat request 128 from the device. In one example, the backupheartbeat confirmation controller 304 uses thedevice ID 114 submitted by theIoT device 106 a as part of theheartbeat request 128 to look updevice state data 110 in thebackup datastore 318. Theheartbeat confirmation controller 304 may determine whether the receiveddevice ID 114 matches adevice ID 114 that is stored as part of the subset ofdevice state data 110. If thedevice ID 114 was backed up, the corresponding backed-upheartbeat confirmation data 118 is used to return aheartbeat confirmation 126 to thecommunication device 106. The communication device then can continue its task such as continuing to execute an in-memory program that does not require theprimary network 102 to be operating. - With continued reference to
FIGS. 5 and 6 , where aninvalidation agent 120 is used in thesystem 100, a software service provider, system administrator or other entity can use theinvalidation agent 120 to designate a particular communication device using thedevice identifier 114 to indicate that the communication device is not to receive aheartbeat confirmation 126 from a network (e.g.,primary network 102 or secondary network 104) in response to aheartbeat confirmation request 128 from thecommunication device 106. This may be done, for example, if theinvalidation agent 120 determines, for example, that a particular frequency or set of frequencies are not available frombase station 106 c for use bycommunication device 106 b.Communication device 106 b can be prevented from attempting to access certain frequencies. This may be suitable in a CBRS type system. For example, a frequency or certain frequencies used by one or more base stations (e.g., 106 c) in a CBRS system may be determined by the system and/or theinvalidation agent 120 to be in use by a governmental entity or devices and are therefore unavailable for use by non-governmental devices. When particular frequencies are not being used by the base station, for example, the frequencies are considered available for public use by user equipment and theheartbeat confirmation 126 may then be sent from thesecondary network 104 to arespective IoT device 106 a. In other systems, an Internet of Things (IoT) device may be prevented from using theprimary network 102 when a software update is required, when the owner of the communication device is out of compliance with contractual agreements made in connection with software usage, or based on any other desired criteria. Digital policy certificates or other mechanisms may be employed in determining when to sendinvalidation data 124 a and/or 124 b to a respective network. In addition, the operation will be described in connection with using multiple end-user communication devices 106. However, the system may be employed with any suitable number of end-user communication devices 106 as desired. In addition, the description with respect toFIGS. 5 and 6 is done in the context of the subset ofstate data 110 being onlyheartbeat confirmation data 118 associated with each of a plurality of devices that use theprimary network 102, as opposed to other examples where the subset of state data includes more thanheartbeat confirmation data 118. - As shown in
block 502, themethod 500 includes thebackup device 300 receiving, from theprimary network device 200, only heartbeat confirmation data 118 (and corresponding device ID 114) associated with a plurality of devices that use theprimary network 102. Theheartbeat confirmation data 118 that is backed up is a subset of thedevice state data 110 that theserver 200 maintains for each respective device. As shown inblock 504, the method includes backing up only theheartbeat confirmation data 118 for the plurality of devices that use theprimary network 102. This is done by the deviceheartbeat backup controller 302 backing up only theheartbeat confirmation data 118 from theprimary network 102. As shown inblock 506, the method includes, in response to anetwork fault condition 130 and in response to aheartbeat request 128 from at least one of the plurality of devices, providing aheartbeat confirmation 126 based on theheartbeat confirmation data 118 that was backed up, to at least one of the plurality of devices. This is done by the backupheartbeat confirmation controller 304 in this example. - As shown in
block 508, whereinvalidation data 124 b for thesecondary network 104, such as through aninvalidation message 122 from the invalidatingagent 120, and in response to thenetwork fault condition 130, and in response to theheartbeat request 128 from the at least one device, denying aheartbeat confirmation 126 for the requesting device when theinvalidation data 124 b indicates to invalidate thecommunication device 106. Stated another way, theinvalidation message 122 can come at any time. If theinvalidation message 122 is sent after theheartbeat confirmation 126 was already sent, asubsequent heartbeat request 128 will be denied because theinvalidation data 124 b will be used to indicate that theinvalidation agent 120 no longer wishes the communication device to have aheartbeat confirmation 126. - In another example, no
heartbeat confirmation 126 will be sent if upon aninitial heartbeat request 128 from the device after anetwork fault condition 130 occurs, theinvalidation data 124 b indicates that the communication device is invalid, the backupheartbeat confirmation controller 304 will not issue aheartbeat confirmation 126. In one example, theinvalidation data 124 is stored as part of a data record on a per-device basis so that the backed-up device state data includes adevice identifier 114 for each device of interest, the corresponding backed upheartbeat confirmation data 118 from theprimary network 102 and anyinvalidation data 124 b that has been received for the particular communication device frominvalidation agent 120. As such, theinvalidation data 124 b in one example includes thedevice ID 114 plus the data indicating whether to invalidate acommunication device 106. As such, thebackup device 300 denies aheartbeat confirmation 126 for the communication device making theheartbeat request 128 based on aninvalidating message 122 from theinvalidation agent 120. - Referring to
FIG. 6 , from a system perspective, as shown inblock 602, themethod 600 includes the device registering or connecting with theprimary network 102 via a DNS address. As shown inblock 604, if the connection is successful, the method continues to block 606 where the server 200 (e.g., primary network device 200 (FIG. 1 )) updates a heartbeat okay (heartbeat confirmation data) 118 table stored asdevice state data 110. This includes, for example, placing data in a memory table, database record or other format to indicate that aheartbeat confirmation 126 has been sent to the device that had requested a heartbeat. Theserver 200 also provides theheartbeat confirmation data 118 to thesecondary network 104 as part of a backup operation.Heartbeat confirmation data 118 will continually be provided by theprimary network 102 as long as the primary network is operating properly. However, if there is anetwork fault condition 130 associated with theprimary network 102 as perceived by thecommunication device 106, such asIoT device 106 a, for example, as shown inblock 608, the communication device will then attempt connection to thesecondary network 104 via the DNS. As shown inblock 610, thebackup device 300 and in this example, the backupheartbeat confirmation controller 304 checks the backed up state data in thebackup datastore 318 based on thedevice ID 114 to determine if the backed up data indicates that the heartbeat most recently stored by theprimary network 102 indicates that aheartbeat confirmation 126 was provided by theprimary network 102 by the device before thenetwork fault condition 130. As shown inblock 612, thebackup network device 300 also checks the database record associated with the device to see if theinvalidation agent 120 had sent aninvalidation message 122 for that device. If yes, then as shown inblock 614, thesecondary network 104 will deny aheartbeat confirmation 126 to the requestingcommunication device 106. However, if theinvalidation agent 120 has not indicated to invalidate thecommunication device 106, as shown inblock 616, thebackup network device 300 will respond to the communication device that is requesting a heartbeat from thesecondary network 104 withheartbeat confirmation 126 based on theheartbeat confirmation data 118 that was backed up. - Among other advantages, employing a second and smaller cloud system that keeps a copy of the valid heartbeat information, such as in a table, allows the second system to have maximum separation without having a fully rewritten software service. The
secondary network 104 andbackup device 300 allows the overall system to keep consistent views of the devices in theprimary network 102 without having to have full constant copies of the data at all times as well as having smaller, independent software services to ensure reliability. In one example, a static page handler exists that stores only the most important aspects of operation, namely serving heartbeat confirmation signals 126 to the devices. Thesecondary network 104 only needs to be updated by theprimary network 102 withheartbeat confirmation data 118 and makes maintaining a consistent state of the devices significantly simpler than prior methods. As used herein, thedevice ID 114 and correspondingheartbeat confirmation data 118 may be the same data. For example when a table is used, the mere existence of thedevice ID 114 in the table could serve as theheartbeat confirmation data 118 for the particular communication device. -
FIG. 7 is schematic view of anexample computing device 700 that may be used to implement the systems (e.g., thenetwork device 200 and the backup network device 300) and methods (e.g., themethods computing device 700 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, mobile computing devices, wearable computing devices (e.g., headsets and/or watches), servers, blade servers, mainframes, and other appropriate computers. The components shown here, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed in this document. - The
computing device 700 includes a processor 710 (also referred to as processing hardware or data processing hardware), memory 720 (also referred to as memory hardware), astorage device 730, a high-speed interface/controller 740 connecting to thememory 720 and high-speed expansion ports 750, and a low speed interface/controller 760 connecting to alow speed bus 770 and astorage device 730. Each of thecomponents processor 710 can process instructions for execution within thecomputing device 700, including instructions stored in thememory 720 or on thestorage device 730 to display graphical information for a graphical user interface (GUI) on an external input/output device, such asdisplay 780 coupled tohigh speed interface 740. In other implementations, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also,multiple computing devices 700 may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system). - The
memory 720 stores information non-transitorily within thecomputing device 700. Thememory 720 may be a computer-readable medium, a volatile memory unit(s), or non-volatile memory unit(s). Thenon-transitory memory 720 may be physical devices used to store programs (e.g., sequences of instructions) or data (e.g., program state information) on a temporary or permanent basis for use by thecomputing device 700. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM)/programmable read-only memory (PROM)/erasable programmable read-only memory (EPROM)/electronically erasable programmable read-only memory (EEPROM) (e.g., typically used for firmware, such as boot programs). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM) as well as disks or tapes. - The
storage device 730 is capable of providing mass storage for thecomputing device 700. In some implementations, thestorage device 730 is a computer-readable medium. In various different implementations, thestorage device 730 may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid-state memory device, or an array of devices, including devices in a storage area network or other configurations. In additional implementations, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as thememory 720, thestorage device 730, or memory onprocessor 710. - The high-
speed controller 740 manages bandwidth-intensive operations for thecomputing device 700, while thelow speed controller 760 manages lower bandwidth-intensive operations. Such allocation of duties is exemplary only. In some implementations, the high-speed controller 740 is coupled to thememory 720, the display 780 (e.g., through a graphics processor or accelerator), and to the high-speed expansion ports 750, which may accept various expansion cards (not shown). In some implementations, the low-speed controller 760 is coupled to thestorage device 730 and a low-speed expansion port 790. The low-speed expansion port 790, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), may be coupled to one or more input/output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter. - The
computing device 700 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as astandard server 700 a or multiple times in a group ofsuch servers 700 a, as alaptop computer 700 b, or as part of arack server system 700 c. - Various implementations of the systems and techniques described herein can be realized in digital electronic and/or optical circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.
- These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, non-transitory computer readable medium, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and/or data to a programmable processor.
- The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
- To provide for interaction with a user, one or more aspects of the disclosure can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touch screen for displaying information to the user and optionally a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received from the web browser.
- A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the disclosure. Accordingly, other implementations are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results.
Claims (20)
1. A computer-implemented method executed by data processing hardware of a communication device that causes the data processing hardware to perform operations comprising:
receiving, from a primary network device, heartbeat confirmation data associated with a plurality of devices that use a primary network;
storing the heartbeat confirmation data as backup for the plurality of devices that use the primary network;
receiving, from an invalidating agent, a device invalidation message invalidating at least one of the plurality of devices;
detecting a network fault condition associated with the primary network;
receiving a heartbeat request from the at least one of the plurality of devices during the network fault condition; and
based on receiving the device invalidation message from the invalidating agent, denying the heartbeat request from the at least one of the plurality of devices.
2. The computer-implemented method of claim 1 , wherein storing the heartbeat confirmation data comprises storing the heartbeat confirmation data at a backup network device.
3. The computer-implemented method of claim 2 , wherein the backup network device receives the heartbeat confirmation data from the primary network device in response to a pull request.
4. The computer-implemented method of claim 1 , wherein detecting the network fault condition associated with the primary network comprises detecting a power outage at the primary network or a communication failure between the primary network and the at least one of the plurality of devices.
5. The computer-implemented method of claim 1 , wherein detecting the network fault condition associated with the primary network comprises:
determining that the at least one of the plurality of devices failed to provide a first heartbeat confirmation signal within a threshold period of time; and
based on determining that the at least one of the plurality of devices failed to provide the first heartbeat confirmation signal within the threshold period of time, detecting the network fault condition.
6. The computer-implemented method of claim 5 , wherein the operations further comprise:
receiving a second heartbeat confirmation signal from another device of the plurality of devices; and
in response to receiving the second heartbeat confirmation signal, performing a device operation.
7. The computer-implemented method of claim 5 , wherein the operations further comprise registering the at least one of the plurality of devices to receive the first heartbeat confirmation signal from the at least one of the plurality of devices.
8. The computer-implemented method of claim 7 , wherein the operations further comprise, based on registering the at least one of the plurality of devices to receive the first heartbeat confirmation signal, storing a device identifier associated with the at least one of the plurality of devices.
9. The computer-implemented method of claim 1 , wherein the primary network device is associated with a cloud network infrastructure.
10. The computer-implemented method of claim 1 , wherein the operations further comprise storing device state data associated with the plurality of devices that use the primary network.
11. A system comprising:
data processing hardware of a communication device; and
memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
receiving, from a primary network device, heartbeat confirmation data associated with a plurality of devices that use a primary network;
storing the heartbeat confirmation data as backup for the plurality of devices that use the primary network;
receiving, from an invalidating agent, a device invalidation message invalidating at least one of the plurality of devices;
detecting a network fault condition associated with the primary network;
receiving a heartbeat request from the at least one of the plurality of devices during the network fault condition; and
based on receiving the device invalidation message from the invalidating agent, denying the heartbeat request from the at least one of the plurality of devices.
12. The system of claim 11 , wherein storing the heartbeat confirmation data comprises storing the heartbeat confirmation data at a backup network device.
13. The system of claim 12 , wherein the backup network device receives the heartbeat confirmation data from the primary network device in response to a pull request.
14. The system of claim 11 , wherein detecting the network fault condition associated with the primary network comprises detecting a power outage at the primary network or a communication failure between the primary network and the at least one of the plurality of devices.
15. The system of claim 11 , wherein detecting the network fault condition associated with the primary network comprises:
determining that the at least one of the plurality of devices failed to provide a first heartbeat confirmation signal within a threshold period of time; and
based on determining that the at least one of the plurality of devices failed to provide the first heartbeat confirmation signal within the threshold period of time, detecting the network fault condition.
16. The system of claim 15 , wherein the operations further comprise:
receiving a second heartbeat confirmation signal from another device of the plurality of devices; and
in response to receiving the second heartbeat confirmation signal, performing a device operation.
17. The computer-implemented method of claim 15 , wherein the operations further comprise registering the at least one of the plurality of devices to receive the first heartbeat confirmation signal from the at least one of the plurality of devices.
18. The system of claim 17 , wherein the operations further comprise, based on registering the at least one of the plurality of devices to receive the first heartbeat confirmation signal, storing a device identifier associated with the at least one of the plurality of devices.
19. The system of claim 11 , wherein the primary network device is associated with a cloud network infrastructure.
20. The system of claim 11 , wherein the operations further comprise storing device state data associated with the plurality of devices that use the primary network.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18/467,393 US20240007383A1 (en) | 2018-10-09 | 2023-09-14 | Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode |
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201862743040P | 2018-10-09 | 2018-10-09 | |
PCT/US2019/054038 WO2020076557A1 (en) | 2018-10-09 | 2019-10-01 | Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode |
US202117278649A | 2021-03-22 | 2021-03-22 | |
US17/938,565 US11784905B2 (en) | 2018-10-09 | 2022-10-06 | Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode |
US18/467,393 US20240007383A1 (en) | 2018-10-09 | 2023-09-14 | Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/938,565 Continuation US11784905B2 (en) | 2018-10-09 | 2022-10-06 | Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode |
Publications (1)
Publication Number | Publication Date |
---|---|
US20240007383A1 true US20240007383A1 (en) | 2024-01-04 |
Family
ID=68387396
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/278,649 Active US11496383B2 (en) | 2018-10-09 | 2019-10-01 | Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode |
US17/938,565 Active US11784905B2 (en) | 2018-10-09 | 2022-10-06 | Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode |
US18/467,393 Pending US20240007383A1 (en) | 2018-10-09 | 2023-09-14 | Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/278,649 Active US11496383B2 (en) | 2018-10-09 | 2019-10-01 | Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode |
US17/938,565 Active US11784905B2 (en) | 2018-10-09 | 2022-10-06 | Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode |
Country Status (6)
Country | Link |
---|---|
US (3) | US11496383B2 (en) |
EP (1) | EP3864799A1 (en) |
JP (2) | JP7250121B2 (en) |
KR (1) | KR102567900B1 (en) |
CN (1) | CN112805964B (en) |
WO (1) | WO2020076557A1 (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP7283103B2 (en) * | 2019-02-13 | 2023-05-30 | 日本電信電話株式会社 | Gateway, communication system and communication method |
JP7271986B2 (en) * | 2019-02-13 | 2023-05-12 | 日本電信電話株式会社 | Gateway, communication system and communication method |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20030048503A (en) * | 2001-12-12 | 2003-06-25 | 주식회사 엘지이아이 | Communication system and method for data synchronization of duplexing server |
US7634570B2 (en) * | 2003-03-12 | 2009-12-15 | Microsoft Corporation | Managing state information across communication sessions between a client and a server via a stateless protocol |
JP2005085102A (en) * | 2003-09-10 | 2005-03-31 | Canon Inc | Guarantee system |
US20070041327A1 (en) * | 2005-08-16 | 2007-02-22 | Cisco Technology, Inc. | Multicast heartbeat signaling |
JP2009258917A (en) * | 2008-04-15 | 2009-11-05 | Mitsubishi Electric Corp | Proxy server, authentication server, and communication system |
US8095828B1 (en) * | 2009-08-31 | 2012-01-10 | Symantec Corporation | Using a data storage system for cluster I/O failure determination |
WO2012061678A1 (en) * | 2010-11-05 | 2012-05-10 | Interdigital Patent Holdings, Inc. | Device validation, distress indication, and remediation |
KR101407597B1 (en) * | 2011-05-16 | 2014-06-13 | 에스케이텔레콤 주식회사 | System and method for providing push service |
US20140079865A1 (en) | 2012-09-14 | 2014-03-20 | Cp Kelco Aps | Process for Preparing a Stabilized Protein Suspension |
US20140095925A1 (en) * | 2012-10-01 | 2014-04-03 | Jason Wilson | Client for controlling automatic failover from a primary to a standby server |
US10061664B2 (en) * | 2015-01-15 | 2018-08-28 | Cisco Technology, Inc. | High availability and failover |
CN107124324B (en) * | 2016-02-25 | 2020-09-01 | 阿里巴巴集团控股有限公司 | Heartbeat protocol method and equipment based on lease |
US10652097B2 (en) * | 2018-03-30 | 2020-05-12 | Hewlett Packard Enterprise Development Lp | Virtual network probing |
-
2019
- 2019-10-01 JP JP2021519613A patent/JP7250121B2/en active Active
- 2019-10-01 WO PCT/US2019/054038 patent/WO2020076557A1/en unknown
- 2019-10-01 CN CN201980066740.6A patent/CN112805964B/en active Active
- 2019-10-01 KR KR1020217008784A patent/KR102567900B1/en active IP Right Grant
- 2019-10-01 US US17/278,649 patent/US11496383B2/en active Active
- 2019-10-01 EP EP19795380.5A patent/EP3864799A1/en active Pending
-
2022
- 2022-10-06 US US17/938,565 patent/US11784905B2/en active Active
-
2023
- 2023-03-20 JP JP2023044054A patent/JP2023078322A/en active Pending
- 2023-09-14 US US18/467,393 patent/US20240007383A1/en active Pending
Also Published As
Publication number | Publication date |
---|---|
CN112805964B (en) | 2024-03-22 |
JP2023078322A (en) | 2023-06-06 |
US11784905B2 (en) | 2023-10-10 |
KR102567900B1 (en) | 2023-08-18 |
WO2020076557A1 (en) | 2020-04-16 |
CN112805964A (en) | 2021-05-14 |
US20220006718A1 (en) | 2022-01-06 |
US20230030237A1 (en) | 2023-02-02 |
JP7250121B2 (en) | 2023-03-31 |
KR20210044281A (en) | 2021-04-22 |
US11496383B2 (en) | 2022-11-08 |
KR20230124099A (en) | 2023-08-24 |
EP3864799A1 (en) | 2021-08-18 |
JP2022504548A (en) | 2022-01-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11784905B2 (en) | Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode | |
US9262324B2 (en) | Efficient distributed cache consistency | |
US10831776B2 (en) | On-demand file synchronization | |
US9634966B2 (en) | Integrated two-way communications between database client users and administrators | |
US10963353B2 (en) | Systems and methods for cross-regional back up of distributed databases on a cloud service | |
US11397632B2 (en) | Safely recovering workloads within a finite timeframe from unhealthy cluster nodes | |
EP3648405A1 (en) | System and method to create a highly available quorum for clustered solutions | |
US9130994B1 (en) | Techniques for avoiding dynamic domain name system (DNS) collisions | |
CN111031126B (en) | Cluster cache sharing method, system, equipment and storage medium | |
US20170195409A1 (en) | Intelligent mapping for an enterprise grid | |
CN109445966B (en) | Event processing method, device, medium and computing equipment | |
EP3629178B1 (en) | System and method for providing backup services to high availability applications | |
CN111859410B (en) | System and method for restricting recovery access | |
KR102665749B1 (en) | Method and apparatus for ensuring continued device operational reliability in cloud-degraded mode | |
US10938919B1 (en) | Registering client devices with backup servers using domain name service records | |
US20240028478A1 (en) | Clustered asset backup in non-federated way | |
CN112965763B (en) | Service processing system, method, device and storage medium | |
US20200028769A1 (en) | Efficient heartbeat with remote servers by nas cluster nodes |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: GOOGLE LLC, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HEATH, TALIVER;HARRISON, KATE;HSUAN, YI;AND OTHERS;REEL/FRAME:064908/0346 Effective date: 20210322 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |