WO2018170773A1 - Procédé de gestion de données de passerelle de l'internet des objets et passerelle de l'internet des objets - Google Patents

Procédé de gestion de données de passerelle de l'internet des objets et passerelle de l'internet des objets Download PDF

Info

Publication number
WO2018170773A1
WO2018170773A1 PCT/CN2017/077628 CN2017077628W WO2018170773A1 WO 2018170773 A1 WO2018170773 A1 WO 2018170773A1 CN 2017077628 W CN2017077628 W CN 2017077628W WO 2018170773 A1 WO2018170773 A1 WO 2018170773A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
internet
column family
internet protocol
converting
Prior art date
Application number
PCT/CN2017/077628
Other languages
English (en)
Chinese (zh)
Inventor
刘强
李成功
黄龑
Original Assignee
深圳中兴力维技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳中兴力维技术有限公司 filed Critical 深圳中兴力维技术有限公司
Priority to PCT/CN2017/077628 priority Critical patent/WO2018170773A1/fr
Publication of WO2018170773A1 publication Critical patent/WO2018170773A1/fr

Links

Definitions

  • the present invention relates to the field of Internet of Things technologies, and in particular, to an Internet of Things gateway data management method and an Internet of Things gateway.
  • the IoT gateway is the transportation center between the Internet of Things and the big data platform.
  • the big data platform connects to many sensing devices through the IoT gateway, resulting in continuous and large amounts of information.
  • the big data platform performs the approach calculation and trend judgment by receiving a large amount of collected data.
  • the main object of the present invention is to provide an Internet of Things gateway data management method and an Internet of Things gateway, which are intended to improve data transmission and processing efficiency.
  • the present invention provides an Internet of Things gateway data management method, and the data management method includes the following steps:
  • the sensing device accessing the Internet of Things gateway parses the data transmitted by the heterogeneous protocol to obtain the data information of the sensing device;
  • the step of converting the fragment data into internet protocol data comprises:
  • the step of converting the column family data into internet protocol data comprises:
  • the step of converting the package into internet protocol data comprises:
  • the method before the step of converting the compressed data into internet protocol data according to a preset network rule, the method further includes:
  • the compressed data is backed up and stored.
  • the present invention also provides an Internet of Things gateway, where the Internet of Things gateway includes:
  • a parsing module configured to parse data sent by the heterogeneous protocol for the sensing device accessing the Internet of Things gateway to obtain data information of the sensing device;
  • a first data processing module configured to perform data fragmentation on the data information according to a preset fragmentation rule, and obtain score piece data;
  • the data information includes attributes of the data;
  • a second data processing module configured to package the fragment data into Internet Protocol data
  • a data sending module configured to send the internet protocol data.
  • the second data processing module is specifically configured to:
  • the step of converting, by the second data processing module, the column family data block into internet protocol data comprises: [0032] performing block processing on the block data of the column family data according to a preset rule, and obtaining partition column family data in which the block data has a logical partition;
  • the step of converting the packet of the second data processing module into Internet protocol data comprises: [0035] cleaning data according to a preset cleaning rule, and compressing the cleaned data to obtain compressed data;
  • the Internet of Things gateway further includes:
  • a backup module configured to perform backup storage of the compressed data before the step of converting, by the second data processing module, the compressed data into Internet protocol data according to a preset network rule.
  • the IoT gateway data management method and the Internet of Things gateway provided by the present invention convert the real data stream into batch data by performing the fragmentation operation on the data information, which can increase each processing in the subsequent processing process.
  • the amount of data reduces the number of processing, thereby improving the efficiency of data transmission and processing, and facilitating the rapid storage and data analysis of the big data platform that subsequently receives data.
  • FIG. 1 is a flowchart of a first embodiment of a method for managing an Internet of Things gateway data according to the present invention
  • FIG. 2 is a flowchart of a second embodiment of a method for managing an Internet of Things gateway data according to the present invention
  • FIG. 3 is a flowchart of a third embodiment of a method for managing an Internet of Things gateway data according to the present invention.
  • FIG. 4 is a flowchart of a fourth embodiment of an Internet of Things gateway data management method according to the present invention.
  • FIG. 5 is a flowchart of a fifth embodiment of an Internet of Things gateway data management method according to the present invention.
  • FIG. 6 is a schematic diagram showing a data hierarchical structure of the data processing method of the Internet of Things gateway shown in FIG. 5; 7 is a schematic diagram of a logical architecture of an existing big data platform;
  • FIG. 9 is a schematic block diagram of an embodiment of an Internet of Things gateway according to the present invention.
  • a first embodiment of a network switching method according to the present invention includes the following steps:
  • Step S100 The sensing device accessing the Internet of Things gateway performs data parsing through the heterogeneous protocol to obtain data information of the sensing device.
  • there are multiple sensing devices connected to the IoT gateway and usually include multiple types.
  • Each type of device will transmit data to the IoT gateway through a heterogeneous protocol.
  • the data transmitted through the heterogeneous protocol includes at least two parts, one of which is related data obtained by the sensing device during the monitoring process, and the other is the type of protocol used by the sensing device to transmit data.
  • the Internet of Things gateway can parse the related data according to the protocol type in the data information.
  • Step S101 performing data fragmentation on the data information according to a preset fragmentation rule in the IoT gateway, and obtaining fragment data;
  • the data information includes attributes of the data, for example: a sensing device The type of protocol used and other attributes.
  • the fixed inter-frame interval may be a slice unit, or the size of the buffer data may be a slice unit.
  • Step S102 Packetizing the fragment data into Internet Protocol data.
  • the Internet Protocol data is used to interconnect the front-end aware device with the network.
  • Step S103 sending the internet protocol data.
  • the real data stream is converted into batch data, which can increase the amount of data processed in each subsequent processing process, and reduce the number of processing times. Therefore, the effect of improving data transmission and processing efficiency is achieved, which facilitates rapid storage and data analysis of a big data platform that subsequently receives data.
  • a second embodiment of the network switching method of the present invention is based on the first embodiment.
  • Step S101 is described in detail. details as follows:
  • Step S200 is the same as step S100 in the first embodiment, and details are not described herein again.
  • Step S201 is the same as step S101 in the first embodiment, and details are not described herein again.
  • Step S202 Set a column family of data according to the data attribute of the fragment data.
  • Step S203 the fragment data is divided into blocks according to the column family, and column family data corresponding to the fragment data is obtained.
  • data segmentation is performed according to a fixed inter-turn interval 30s; data is included in each slice data: temperature, humidity, current, voltage, position, and device ID; then this embodiment will be established
  • the column family of data including temperature, humidity, current, voltage, position, and device ID; further put together the data of the same column family of the same piece of data, that is, the columnar storage logic of the sliced data is realized.
  • Step S204 The column family data is packaged and converted into Internet Protocol data.
  • Step S205 is the same as step S103 in the first embodiment, and details are not described herein again.
  • a third embodiment of the network switching method of the present invention is based on the second embodiment.
  • Step S204 is described in detail. details as follows:
  • Step S300 is the same as step S200 in the second embodiment, and details are not described herein again.
  • Step S301 is the same as step S201 in the second embodiment, and details are not described herein again.
  • Step S302 is the same as step S202 in the second embodiment, and details are not described herein again.
  • Step S303 is the same as step S203 in the second embodiment, and details are not described herein again.
  • Step S304 the block data of the column family data is partitioned according to a preset rule, and the partition column family data whose block data has a logical partition is obtained.
  • the method of partitioning the column family data can hash the column family data by using hash Hash or user designation. Please refer to Figure 6.
  • the storage mode of the voltage column family is exemplified, and the voltage column family is divided into N partitions.
  • Step S305 packetizing the partition column family data into Internet Protocol data.
  • Step S306 is the same as step S205 in the second embodiment, and details are not described herein again.
  • the block data can be hashed into different partitions, so that the block data can be distributed and stored, and the data tilt problem can be solved, and the data reading capability can be improved.
  • distributed storage is the basis of distributed computing, and the logical architecture diagram of the platform is shown in Figure 7. If the column family data is not partitioned, it will be concentrated on a certain disk in the storage process, which will form a read/write bottleneck during the calculation process, which will affect the distributed computing performance. By partitioning the column family data, the column group data can be allocated to multiple disks in a partitioned manner, and multiple disk 1/0 can be used to improve data reading capability, which is beneficial to the back-end big data platform to perform distributed computing on data. .
  • a fourth embodiment of the network switching method of the present invention is provided.
  • the present embodiment is based on the third embodiment, and step S305 is described in detail. details as follows:
  • Step S400 is the same as step S300 in the third embodiment, and details are not described herein again.
  • Step S401 is the same as step S301 in the third embodiment:
  • Step S402 is the same as step S302 in the third embodiment:
  • Step S403 is the same as step S303 in the third embodiment:
  • Step S404 is the same as step S304 in the third embodiment:
  • Step S405 Clean the partition column family data according to a preset cleanup rule, and compress the cleaned data to obtain compressed data.
  • the cleaning rule may be to perform data cleaning on the duplicate and abnormal data under the same device in the partition information;
  • the compression method may be compressed by a compression algorithm such as GZIP, LZO, and Snappy. It can also be encrypted in compression or encrypted after compression, such as MD5, RSA and other encryption methods.
  • Step S406 Convert the compressed data into Internet Protocol data according to a preset network rule.
  • Step S407 is the same as step S306 in the third embodiment, and details are not described herein again.
  • a fifth embodiment of the network switching method of the present invention is based on the fourth embodiment, and a new step is added. details as follows:
  • Step S500 is the same as step S400 in the third embodiment, and details are not described herein again.
  • Step S501 is the same as step S401 in the third embodiment, and details are not described herein again.
  • Step S502 is the same as step S402 in the third embodiment, and details are not described herein again.
  • Step S503 is the same as step S403 in the third embodiment, and details are not described herein again.
  • Step S504 is the same as step S404 in the third embodiment, and details are not described herein again.
  • Step S505 is the same as step S405 in the third embodiment, and details are not described herein again.
  • Step S506 The compressed data is backed up and stored.
  • Step S507 is the same as step S406 in the third embodiment, and details are not described herein again.
  • Step S508 is the same as step S407 in the third embodiment, and details are not described herein again.
  • the compressed data is backed up, so that data retransmission, search, and the like are performed in the event of a network failure.
  • the maximum backup period of data can be calculated and confirmed according to the amount of data processed by the IoT gateway every day and the capacity of the storage module. Users can set it themselves, but it cannot exceed the maximum backup period.
  • the recommended backup period is 1 to 3 months. .
  • the present invention also provides an Internet of Things gateway.
  • the sensing device 1, the sensing device 2, the sensing device 3, and the sensing device 4 of the IoT gateway are accessed.
  • Each sensing device is used to detect a battery pack that is used to form a battery matrix.
  • the Internet of Things Gateway connects to the network, with clients and big data platforms connected to the other ends of the network.
  • the Internet of Things gateway includes:
  • the parsing module 1000 is configured to obtain, by the sensing device accessing the Internet of Things gateway, the data sent by the heterogeneous protocol, and obtain the data information of the sensing device; wherein, the sensing device accessing the Internet of Things gateway has Multiple, and usually include multiple types. Each type of device will transmit data to the IoT gateway through heterogeneous protocols.
  • the data transmitted through the heterogeneous protocol includes at least two parts, one of which is related data obtained by the sensing device during the monitoring process, and the other of which is the type of protocol used by the sensing device to transmit data.
  • the Internet of Things gateway can parse the related data according to the protocol type in the data information.
  • the first data processing module 1100 is configured to perform data fragmentation on the data information according to a preset fragmentation rule to obtain fragmentation data; the data information includes attributes of the data, for example: used by the sensing device Protocol type and other attributes.
  • the fixed inter-frame interval may be a slice unit, or the size of the cache data may be a slice unit.
  • the second data processing module 1200 is configured to package the fragment data into Internet Protocol data.
  • the data sending module 1300 is configured to send the internet protocol data.
  • the Internet Protocol data is used to implement the interconnection between the front-end sensing device and the network.
  • the real data stream is converted into batch data, which can increase the amount of data processed in each subsequent processing process, and reduce the number of processing times. Therefore, the effect of improving data transmission and processing efficiency is achieved, which facilitates rapid storage and data analysis of a big data platform that subsequently receives data.
  • the second data processing module 1200 is specifically configured to:
  • the column family of data is set according to the data attribute of the slice data.
  • the fragment data is divided according to the column family, and column family data corresponding to the fragment data is obtained.
  • data segmentation is performed according to a fixed inter-turn interval 30s; data is included in each slice data: temperature, humidity, current, voltage, position, and device ID; then this embodiment will be established
  • the column family of data including temperature, humidity, current, voltage, position, and device ID; further put together the data of the same column family of the same piece of data, that is, the columnar storage logic of the sliced data is realized.
  • the step of converting, by the second data processing module 1200, the column family data block into the internet protocol data comprises:
  • the storage mode of the voltage family is exemplified, and the voltage column family is divided into N partitions.
  • the block data can be hashed into different partitions, so that the block data can be distributed and stored, and the data tilt problem can be solved, and the data reading capability can be improved.
  • distributed storage is the basis of distributed computing, and the logical architecture diagram of its platform is as Figure 7 shows. If the column family data is not partitioned, it will be concentrated on a certain disk in the storage process, which will form a read-write bottleneck during the calculation process, which affects the distributed computing performance. By partitioning the column family data, the column group data can be allocated to multiple disks in a partitioned manner, and multiple disk I/Os can be used to improve data reading capability, which is beneficial to the back-end big data platform to perform distributed computing on data. .
  • the step of converting the packet of the second data processing module 1200 into internet protocol data comprises:
  • the data is cleaned according to the preset cleaning rule, and the cleaned data is compressed to obtain compressed data;
  • the cleaning rule may be data cleaning of duplicate and abnormal data under the same device in the partition information;
  • Compression can be performed using compression algorithms such as GZIP, LZO, and Snappy. It can also be encrypted in compression or encrypted after compression, such as MD5, RSA and other encryption methods.
  • the Internet of Things gateway further includes:
  • the backup module 1400 is configured to perform backup storage on the compressed data before the step of converting the compressed data into Internet protocol data according to the preset network rule by the second data processing module 1200.
  • the compressed data is backed up, so as to perform data retransmission, search, and the like in the event of a network failure.
  • the maximum backup period of data can be calculated and confirmed according to the amount of data processed by the IoT gateway every day and the capacity of the storage module. Users can set it themselves, but it cannot exceed the maximum backup period.
  • the recommended backup period is 1 to 3 months. .
  • the technical solution of the present invention which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk,
  • a storage medium such as ROM/RAM, disk
  • the optical disc includes a number of instructions for causing a terminal device (which may be a mobile terminal, a computer, a server, an air conditioner, or a network device, etc.) to perform the methods described in various embodiments of the present invention.
  • the IoT gateway data management method and the Internet of Things gateway provided by the present invention convert the real data stream into batch data by performing the fragmentation operation on the data information, which can increase each processing in the subsequent processing process.
  • the amount of data reduces the number of processing, thereby improving the efficiency of data transmission and processing, and facilitating the rapid storage and data analysis of the big data platform that subsequently receives data.

Landscapes

  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

L'invention concerne un procédé de gestion de données de passerelle de l'Internet des objets et une passerelle de l'Internet des objets. Le procédé de gestion de données de passerelle de l'Internet des objets comprend les étapes suivantes : l'analyse de données qui sont transmises au moyen d'un protocole hétérogène par un dispositif de détection accédant à la passerelle de l'Internet des objets pour obtenir des informations de type données concernant le dispositif de détection ; la réalisation d'une fragmentation de données sur les informations de type données selon une règle de fragmentation prédéfinie dans la passerelle de l'Internet des objets, pour obtenir des données fragmentées, les informations de type données comprenant l'attribut des données ; la mise en paquets des données fragmentées et leur conversion en données de protocole Internet ; et l'envoi des données de protocole Internet. La présente invention a pour effet d'améliorer la transmission de données et l'efficacité de traitement.
PCT/CN2017/077628 2017-03-22 2017-03-22 Procédé de gestion de données de passerelle de l'internet des objets et passerelle de l'internet des objets WO2018170773A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/077628 WO2018170773A1 (fr) 2017-03-22 2017-03-22 Procédé de gestion de données de passerelle de l'internet des objets et passerelle de l'internet des objets

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/077628 WO2018170773A1 (fr) 2017-03-22 2017-03-22 Procédé de gestion de données de passerelle de l'internet des objets et passerelle de l'internet des objets

Publications (1)

Publication Number Publication Date
WO2018170773A1 true WO2018170773A1 (fr) 2018-09-27

Family

ID=63584863

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/077628 WO2018170773A1 (fr) 2017-03-22 2017-03-22 Procédé de gestion de données de passerelle de l'internet des objets et passerelle de l'internet des objets

Country Status (1)

Country Link
WO (1) WO2018170773A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11997165B2 (en) 2023-03-10 2024-05-28 Riot Secure Ab System and method for optimizing data communication of internet of things (IoT) devices

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101795480A (zh) * 2009-11-10 2010-08-04 王晓喃 无线传感器网络与IPv6网络全IP通信的路由实现系统
CN101958835A (zh) * 2010-09-28 2011-01-26 深圳中兴力维技术有限公司 一种数据传输方法以及逻辑网络适配器
CN104507076A (zh) * 2014-11-19 2015-04-08 哈尔滨工程大学 一种基于Contiki系统和6lbr系统的实用物联网应用平台
US20150347555A1 (en) * 2014-05-31 2015-12-03 Linkedin Corporation Waterwheel sharding
CN105393251A (zh) * 2013-06-12 2016-03-09 甲骨文国际公司 数据库内的分片队列

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101795480A (zh) * 2009-11-10 2010-08-04 王晓喃 无线传感器网络与IPv6网络全IP通信的路由实现系统
CN101958835A (zh) * 2010-09-28 2011-01-26 深圳中兴力维技术有限公司 一种数据传输方法以及逻辑网络适配器
CN105393251A (zh) * 2013-06-12 2016-03-09 甲骨文国际公司 数据库内的分片队列
US20150347555A1 (en) * 2014-05-31 2015-12-03 Linkedin Corporation Waterwheel sharding
CN104507076A (zh) * 2014-11-19 2015-04-08 哈尔滨工程大学 一种基于Contiki系统和6lbr系统的实用物联网应用平台

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11997165B2 (en) 2023-03-10 2024-05-28 Riot Secure Ab System and method for optimizing data communication of internet of things (IoT) devices

Similar Documents

Publication Publication Date Title
JP6106901B2 (ja) 分散ファイルストレージシステムにおけるデータ処理の方法およびデバイス
US9501366B2 (en) Dispersed storage network with parameter search and methods for use therewith
US8447898B2 (en) Task offload to a peripheral device
US10860245B2 (en) Method and apparatus for optimizing data storage based on application
US8601210B2 (en) Cache memory allocation process based on TCPIP network and/or storage area network array parameters
US20120209967A1 (en) Scalable distributed storage and delivery
JP5554817B2 (ja) データ送信装置及び方法
US20070101023A1 (en) Multiple task offload to a peripheral device
WO2013178155A1 (fr) Procédé et appareil de stockage pour système de fichiers répartis
CN104714905A (zh) 用于执行从第一适配器到第二适配器的失效转移操作的方法和系统
CN114201421B (zh) 一种数据流处理方法、存储控制节点及可读存储介质
WO2014000430A1 (fr) Procédé et appareil de réalisation de services battement de cœur en grappe à haute disponibilité
US10521317B1 (en) Compressing data to be replicated utilizing a compression method selected based on network behavior
WO2017054540A1 (fr) Procédé et dispositif de traitement de données, serveur et contrôleur
US11762557B2 (en) System and method for data compaction and encryption of anonymized datasets
CN102843284B (zh) iSCSI存储节点、架构以及读取、写入方法
CN107422989A (zh) 一种Server SAN系统多副本读取方法及存储架构
WO2018170773A1 (fr) Procédé de gestion de données de passerelle de l'internet des objets et passerelle de l'internet des objets
US8676928B1 (en) Method and system for writing network data
US20180067820A1 (en) Method and system for using pre-existing connections in a dispersed storage network
CN102006285B (zh) 一种用于网络安全设备的报文处理方法及装置
WO2022078347A1 (fr) Procédé et appareil de planification de tâches, dispositif électronique et support de stockage
WO2013086867A1 (fr) Procédé et équipement de traitement d'informations
CN115617760A (zh) 一种大批量文件快速上传的方法
TW200525966A (en) Method, apparatus, system, and article of manufacture for processing control data by an offload adapter

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17902420

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17902420

Country of ref document: EP

Kind code of ref document: A1