CN111556141A - Intelligent gateway data acquisition system and method based on Json data sheet - Google Patents
Intelligent gateway data acquisition system and method based on Json data sheet Download PDFInfo
- Publication number
- CN111556141A CN111556141A CN202010340790.3A CN202010340790A CN111556141A CN 111556141 A CN111556141 A CN 111556141A CN 202010340790 A CN202010340790 A CN 202010340790A CN 111556141 A CN111556141 A CN 111556141A
- Authority
- CN
- China
- Prior art keywords
- data
- instruction
- sensor
- acquisition
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000004364 calculation method Methods 0.000 claims abstract description 42
- 238000013500 data storage Methods 0.000 claims abstract description 41
- 230000005540 biological transmission Effects 0.000 claims abstract description 16
- 238000004891 communication Methods 0.000 claims description 29
- 230000000737 periodic effect Effects 0.000 claims description 22
- 238000012544 monitoring process Methods 0.000 claims description 11
- 238000004806 packaging method and process Methods 0.000 claims description 10
- 238000013461 design Methods 0.000 claims description 6
- 238000005538 encapsulation Methods 0.000 claims description 6
- 238000005259 measurement Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 238000013075 data extraction Methods 0.000 claims description 5
- 238000012423 maintenance Methods 0.000 claims description 4
- 238000013480 data collection Methods 0.000 claims description 3
- 230000010354 integration Effects 0.000 abstract 1
- 238000004458 analytical method Methods 0.000 description 4
- 238000013499 data model Methods 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- QVFWZNCVPCJQOP-UHFFFAOYSA-N chloralodol Chemical compound CC(O)(C)CC(C)OC(O)C(Cl)(Cl)Cl QVFWZNCVPCJQOP-UHFFFAOYSA-N 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005111 flow chemistry technique Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 239000007858 starting material Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- G—PHYSICS
- G08—SIGNALLING
- G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
- G08C17/00—Arrangements for transmitting signals characterised by the use of a wireless electrical link
- G08C17/02—Arrangements for transmitting signals characterised by the use of a wireless electrical link using a radio link
-
- G—PHYSICS
- G08—SIGNALLING
- G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
- G08C19/00—Electric signal transmission systems
-
- G—PHYSICS
- G08—SIGNALLING
- G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
- G08C23/00—Non-electrical signal transmission systems, e.g. optical systems
- G08C23/06—Non-electrical signal transmission systems, e.g. optical systems through light guides, e.g. optical fibres
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/565—Conversion or adaptation of application format or content
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Abstract
The invention provides an intelligent gateway data acquisition system based on a Json data sheet.A task control module is used for controlling the automatic acquisition work of a data integration channel and realizing multi-channel parallel data acquisition; the instruction control module utilizes a sensor instruction rule set to perform dynamic access, equipment control and data acquisition on the sensing equipment; the data acquisition module utilizes a data acquisition rule set to customize and arrange the acquisition modes of the multiple sensors and realize automatic data acquisition; the data calculation module carries out parametric calculation and quality calibration on the sensing data by using the data calculation rule set; the data storage module packages and gathers source data of the sensing equipment into a Json data sequence by using a data storage rule set and stores the Json data sequence in a local database; the data transmission module converts the Json data sequence into a Json data packet and an XML file packet by using a data storage rule set and uploads the Json data packet and the XML file packet to the big data platform; the invention also provides a data acquisition method using the intelligent gateway data acquisition system based on the Json data sheet.
Description
Technical Field
The invention relates to the technical field of data information acquisition, in particular to an intelligent gateway data acquisition system and method based on a Json data sheet.
Background
In the construction of urban major projects, sensor monitoring technology based on the internet of things is increasingly widely used. Such as: various micro-miniature pull crack sensors, vibrating wire sensors, etc. In current engineering practice, when data acquisition is performed by using a sensor monitoring technology, data acquisition and preprocessing are required to be performed by means of specially customized professional acquisition equipment and a customized system. In the mode, when large-scale real-time intelligent monitoring work is carried out on a super large project, the number of input acquisition equipment and professional systems is huge; and the systems of various sensor manufacturers are incompatible, the instruction protocol is not universal, the setting and changing mode is complex when the network is accessed again after the sensors are replaced, the acquisition mode is single, and the efficiency is low.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides an intelligent gateway data acquisition system and method based on a Json data sheet, and aims to solve the technical problems that when an internet of things sensor is used for data acquisition, the set change mode is complex, the acquisition mode is single, and the efficiency is low when the sensor is replaced and the network is accessed again in the prior art.
The technical scheme adopted by the invention is as follows:
in a first aspect, the invention provides an intelligent gateway data acquisition system based on a Json data sheet, which comprises a task control module, an instruction control module, a data acquisition module, a data calculation module, a data storage module and a data transmission module;
the task control module is used for constructing an independent data acquisition task according to the equipment communication channel of the intelligent gateway and configuring and managing task execution on each data communication channel;
the command control module is used for carrying out standardized encapsulation on a control command of the sensor according to a sensor command rule set configured by design and then coding and sending the control command; the device is also used for carrying out self-adaptive decoding on the data collected and returned by the sensor and extracting corresponding data information;
the data acquisition module comprises a data initial learning module and a periodic acquisition module; the data initial learning module is used for acquiring initial configuration data of the sensor, and the periodic acquisition module is used for periodically acquiring data of the data acquisition channel according to data acquisition rule configuration;
the data calculation module performs parametric calculation and data quality calibration on the acquired sensor data according to the data calculation rule set;
the data storage module is used for converting the acquired sensor data into a Json data sequence according to a data storage rule set and then storing the Json data sequence in a local database;
and the data transmission module converts the Json data sequence stored in the data storage module into a Json data packet and an XML file packet respectively according to the data storage rule set and uploads the Json data packet and the XML file packet to the big data platform.
In a second aspect, the present invention further provides a data acquisition method for an intelligent gateway data acquisition system based on a Json data slice, which is characterized by comprising the following steps:
s1, establishing a data acquisition task through a task control module, and configuring and scheduling task parameters of accessed multi-type sensing equipment according to a communication channel;
s2, carrying out standardized packaging on a sensor control instruction according to a designed and configured sensor instruction rule set through an instruction control module, then coding and sending the sensor control instruction, and acquiring initial data of a sensor;
s3, using a periodic acquisition module in the data acquisition module to perform periodic data acquisition on the data acquisition channel;
s4, carrying out self-adaptive decoding on the data acquired and returned by the sensor through the instruction control module;
s5, carrying out data parameterization calculation and data quality calibration through a data calculation module;
and S6, converting the data subjected to quality calibration into a Json data sequence through a data storage module according to a data storage rule set, storing the Json data sequence in a local database, and uploading the Json data sequence to a big data platform through a data transmission module.
Further, the task control module specifically works according to the following steps:
managing a data acquisition task according to an equipment communication channel of the intelligent gateway;
configuring task parameters of data acquisition of a current communication channel, wherein the parameters comprise an acquisition method, initial time, sampling interval, a cooperative channel and a cooperative mode;
configuring a sensor of a current communication channel, and setting a sampling limit difference of the sensor;
and maintaining the data acquisition channel according to the acquisition task configuration, wherein the maintenance comprises starting, stopping and resetting.
Further, the instruction control module specifically works according to the following steps:
preloading a sensor instruction rule set file, and organizing the rule set file in a Json format;
searching matched sensor types and loading corresponding instruction sets according to task scheduling parameters of the data acquisition module;
capturing corresponding standard instruction set configuration according to the currently executed instruction name;
extracting coding information according to the currently matched instruction configuration, carrying out instruction encapsulation, and sending the coding information to a sensor under a corresponding communication channel;
monitoring data return information of a current communication channel, and stopping data reception when a data string meeting a stop identifier is received;
according to the decoding and data extraction rule configured by the current instruction, self-decoding of the data string is carried out; loading corresponding calculation rules and calculation models according to the matched sensor types, and performing regularized calculation on data by using a data calculation module;
and matching the storage identifiers of the data blocks by combining the data storage rule set according to the data identifier information configured by the current instruction, and packaging into a Json data sheet.
Further, the set of sensor instruction rules is formed as follows:
the single equipment control instruction comprises an instruction name, a start character, an instruction code, an instruction decode, an instruction sort, an equipment address, an instruction state, a data extraction, a data bit identification and a calculation model;
the control and access of the sensor are formed by one to a plurality of standard instructions, and the drive control and data acquisition of the sensor are realized.
Further, the data initial learning module specifically works according to the following steps:
under the selected communication channel, adding a new sensor node, and loading a sensor instruction rule set corresponding to the node;
calling a control instruction to check the working state of the sensing equipment, and calling a measurement instruction to acquire initial configuration data and initial observation data of the sensor;
storing the obtained initial configuration and observation data in a local database;
further, the periodic acquisition module specifically works according to the following steps:
starting a task timer, loading parameter configuration, sensor configuration and acquisition rule configuration of a matching channel, and executing data acquisition work of a current period; the data acquisition working mode comprises polling data acquisition and grouped data acquisition;
and after the data acquisition of the current period is finished, the data quality is checked and corrected by the data calculation module, and after the data is qualified, the local data is stored and uploaded to the big data platform.
Furthermore, the periodic acquisition module arranges the acquisition rules of the sensors and cooperatively controls data acquisition; the cooperative control data acquisition comprises the cooperative control of the sensors across the channels and the cooperative monitoring of the multi-type sensing equipment under the same channel.
Further, the data storage rule set comprises a data name, a data tag, a data type, a data index and a data association; the data storage rule set is used for changing configuration of different types of sensors during data storage.
According to the technical scheme, the beneficial technical effects of the invention are as follows:
1. the method comprises the steps of designing standard instruction rule sets of different types of sensors, realizing flow processing of instructions in the processes of packaging, sending, encoding, decoding and storing, realizing customized data acquisition and analysis work of a single sensing device through serial configuration of an instruction execution process, and finally realizing simultaneous online data acquisition of the different types of sensors by taking a gateway channel as a working unit, thereby effectively improving the data acquisition efficiency.
2. The method comprises the steps of designing a sensor instruction rule set, wherein different types of sensors correspond to a plurality of instruction sequence sets, the sets corresponding to each type of sensor are different, and the customized design of an instruction system is carried out on different engineering projects, so that the situation that the existing working parameters are not changed is ensured, the situation that the different types of sensors are accessed in a dynamic capacity expansion mode under the situation that the intelligent gateway setting is not adjusted is realized by changing the sets corresponding to the plurality of instruction sequences of the sensors.
3. By polling data acquisition, grouping data acquisition and regularized acquisition configuration, the data acquisition mode is diversified.
4. Through the arrangement of the acquisition rules of a plurality of sensing devices, the cooperative control data acquisition is realized, the data acquisition time can be saved, errors are reduced, and the data acquisition efficiency is further improved.
5. The data storage rule set is used for changing configuration of different types of sensors during data storage. The sensor instruction rule set, the data storage rule set and the data calculation rule are matched for use, so that the sensor access capability of the intelligent gateway is greatly expanded. The processes of data acquisition, storage, calculation, transmission and the like are decoupled through the design of a rule set, dynamic matching loading is carried out when needed, and the compatibility of gateway equipment is improved.
Drawings
In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings that are needed in the detailed description of the invention or the prior art will be briefly described below. Throughout the drawings, like elements or portions are generally identified by like reference numerals. In the drawings, elements or portions are not necessarily drawn to scale.
FIG. 1 is a flow chart of the system operation of the present invention.
FIG. 2 is a system architecture diagram of the present invention.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and therefore are only examples, and the protection scope of the present invention is not limited thereby.
It is to be noted that, unless otherwise specified, technical or scientific terms used herein shall have the ordinary meaning as understood by those skilled in the art to which the invention pertains.
Example 1
As shown in fig. 1, the invention provides an intelligent gateway data acquisition system based on a Json data sheet, which comprises a task control module, an instruction control module, a data acquisition module, a data calculation module, a data storage module and a data transmission module;
the task control module is used for constructing an independent data acquisition task according to the equipment communication channel of the intelligent gateway and configuring and managing task execution on each data communication channel;
the command control module is used for carrying out standardized encapsulation on a control command of the sensor according to a sensor command rule set configured by design and then coding and sending the control command; the device is also used for carrying out self-adaptive decoding on the data collected and returned by the sensor and extracting corresponding data information;
the data acquisition module comprises a data initial learning module and a periodic acquisition module; the data initial learning module is used for acquiring initial configuration data of the sensor, and the periodic acquisition module is used for periodically acquiring data of the data acquisition channel according to data acquisition rule configuration;
the data calculation module performs parametric calculation and data quality calibration on the acquired sensor data according to the data calculation rule set;
the data storage module is used for converting the acquired sensor data into a Json data sequence according to a data storage rule set and then storing the Json data sequence in a local database;
and the data transmission module converts the Json data sequence stored in the data storage module into a Json data packet and an XML file packet respectively according to the data storage rule set and uploads the Json data packet and the XML file packet to the big data platform.
The working principle of example 1 is explained in detail below:
the technical scheme of the embodiment is a software system loaded and operated on an intelligent gateway. In this embodiment, the data acquisition of the sensor of the internet of things is implemented by the intelligent gateway. The intelligent gateway carries out networking connection on various types of sensors and the remote server in a wired or wireless mode; and then according to the communication protocols of different sensors, software connection is carried out by using interfaces allowed by the protocols, so that control, data acquisition and data transmission of various sensors are realized.
For controlling the sensor to perform data acquisition, the following steps are performed in this embodiment:
1. the task control module establishes a data acquisition task, and performs task execution configuration and task scheduling on the data acquisition task
After the software system of this embodiment is loaded on the intelligent gateway, the task control module starts to work first. The task control module is used for newly adding a data acquisition task according to an equipment communication channel of the intelligent gateway; then configuring data acquisition task parameters, wherein the parameters comprise an acquisition method, initial time, sampling interval, a cooperative channel and a cooperative mode; and then configuring the sensor of the current communication channel, and setting the sampling limit difference of the sensor.
In this embodiment, the gateway communication channel types include RS232, RS485, Wifi, bluetooth, LoRa, network cable, optical fiber, and the like. And in the process of controlling the sensor to collect data, the task control module also carries out maintenance according to the collection task parameters, and the maintenance comprises starting, stopping and resetting of the data collection task.
2. Through the instruction control module, the sensor measurement control instruction is subjected to standardized packaging according to the sensor instruction rule set of design configuration and then is coded and sent, and initial configuration data of the sensor is acquired
Firstly, a sensor instruction rule set file is preloaded through a sensor instruction control module; under the condition of selecting a gateway communication channel, searching matched sensor types according to task scheduling parameters of a data acquisition module, adding sensor nodes, and loading a sensor instruction rule set of the nodes. The sensor instruction rule set includes a plurality of instruction sequences including instruction names, starters, instruction encodings, instruction decodings, instruction classifications, device addresses, instruction states, data extractions, data bit identifications, computational models, and the like. The same type of sensor corresponds to one or more of a plurality of instruction sequences and executes dynamic instruction logic, including feedback instructions and non-feedback instructions. Different types of sensors correspond to a plurality of sets of command sequences, and the sets corresponding to each type of sensor are different. Under the condition that other working parameters are not changed, the dynamic switching of the sensors can be realized by changing the set of a plurality of instruction sequences corresponding to the sensors without adjusting the setting of the intelligent gateway system. In this way, by changing the set of the plurality of instruction sequences, dynamic access to different types of sensors can be realized without adjusting the setting of the intelligent gateway. The integrated application of the instruction rule set comprises ASCII, hexadecimal and other different types of instruction encoding and decoding, and the adaptability is stronger. The acquisition control of different types of sensors can be realized by changing the sensor instruction rule set.
Capturing corresponding standard instruction set configuration according to the currently executed instruction name; and then, carrying out instruction encapsulation according to the instruction coding information and sending the instruction to the sensor under the corresponding gateway communication channel. According to the rules, when the novel sensing equipment is accessed to the gateway, data acquisition can be carried out by editing and configuring the instruction rule set file under the condition that the working parameters of the existing intelligent gateway are not changed.
And then, calling a measurement instruction below the instruction set by using a data initial learning module in the data acquisition module to acquire initial data of the sensor, and storing the acquired initial configuration data in a local database. And simultaneously, testing other instructions of the current type of sensor, and checking the working state of the sensor.
In this embodiment, the acquired initial data includes initial angle data, initial coordinate data, initial frequency data, initial deformation data, initial environment data, and the like.
3. Using a periodic acquisition module in the data acquisition module to perform periodic data acquisition on the data acquisition channel
Firstly, according to a data acquisition task, starting a task timer and executing periodic data acquisition; then loading the parameter configuration, the sensor configuration and the acquisition rule configuration of the gateway communication channel; and performing current regularized data acquisition work, including polling data acquisition and grouping data acquisition. After the sensor is initially configured, the data acquisition module can alternately perform initial configuration data acquisition work and periodic acquisition work.
For polling data acquisition, the acquisition sequence numbers of all sensors under a channel are taken as the basis to carry out sequential acquisition. And calling a sensor instruction rule module in the acquisition process to drive the sensor to acquire data.
For grouped data acquisition, a group is a batch by taking monitoring point groups as reference. And the current intra-group acquisition carries out data acquisition by taking the serial number of the monitoring point as a basis.
And for the configured data acquisition, dynamically observing the sensor according to the acquisition sequence of the configuration file.
In the process of data acquisition, the periodic acquisition module monitors data return information of a gateway communication channel, and stops data reception when receiving a data string meeting the stop identifier.
In this embodiment, the collected data includes angle data, coordinate data, frequency data, deformation data, environment data, and the like.
4. Self-adaptive decoding is carried out on data collected and returned by a sensor through an instruction control module
And using a sensor instruction control module to perform self-adaptive decoding on the data acquired and returned by the sensor according to the decoding configuration of the instruction.
5. Data parameterization calculation and data quality calibration through data calculation module
And (4) performing data quality checking after the acquisition is finished, storing the data into a database after the data is qualified, and performing data repairing and testing if the data is not qualified. And performing parametric calculation of data according to a pre-configuration data calculation rule of the matched sensor by using a regular expression and a formula expression through a data calculation module. Data quality checks include duplicate value checks, missing value checks, outlier checks, etc. on the data.
6. Converting the data subjected to quality calibration into a Json data sequence through a data storage module according to a data storage rule set, storing the Json data sequence in a local database, and uploading the Json data sequence to a big data platform through a data transmission module
Firstly, matching a storage identifier of a data string with a rule set through a data storage module according to the storage rule configuration of an instruction; preloading a data model rule set, and creating and updating a Sqlite database table; matching a corresponding data column set according to the name of the data model; carrying out fusion matching on the shared and exchanged Json data source and the data column to generate an SQL operation statement of the Sqlite database; then, inquiring, editing, deleting and the like are carried out to obtain a Json data sheet; and finally storing the Json data sheet in a local database. In this embodiment, the local database is loaded in the intelligent gateway.
The storage definition of one data item comprises a data name, a data identifier, a data type, a data index, a data association and the like, one type of sensing data comprises one or more data items, and the storage rule set of one intelligent gateway acquisition system comprises one or more storage definitions of the sensing data. After the data acquisition module receives the original data of the sensor, transcoding the data sequence through the data storage module, and storing the data sequence to a local database system in real time. After the network online interface is configured, the data transmission module can perform data packaging and encapsulation through a data sequence, and then upload the data to a big data platform, and can also serve as a standardized data sharing exchange interface to provide services. Data storage for different types of sensors is achieved by storing changes to the rule set.
Json is a data exchange format. The data is stored and represented by adopting a text format completely independent of a programming language, the hierarchy is simple and clear, the reading and the writing of programmers are easy, the machine analysis and the generation are easy, and the network transmission efficiency can be effectively improved. In this embodiment, the sensor instruction rule set, the data acquisition rule set, the data calculation rule set, and the data storage rule set are all organized in the form of a Json data sheet.
Similarly, when the Json data sheet is uploaded to a big data platform, a data model rule file is loaded through a data transmission module; and then, a database storage module is utilized to inquire corresponding database table results, the database table results are respectively converted into a Json data packet and an XML file packet according to a data model, and the Json data packet and the XML file packet are reported to a big data platform.
According to the technical scheme in the embodiment 1, the steps of packaging, sending, coding, decoding and storing the instructions are standardized by designing rule sets such as instructions, calculation and storage of different sensors, customized data acquisition and analysis work of a single sensing device is realized by executing sequential coding of the instructions, and finally, the simultaneous online data acquisition of different sensors is realized by taking a gateway channel as a working unit; by polling data acquisition, multi-test-loop data acquisition and configuration data acquisition, the data acquisition mode is diversified, and the data acquisition efficiency is effectively improved. Through the matching use of the rule sets, the sensor access capability of the intelligent gateway is greatly expanded.
Example 2
The invention also provides a data acquisition method of the intelligent gateway data acquisition system based on the Json data sheet, which is characterized by comprising the following steps:
s1, establishing a data acquisition task through a task control module, and configuring and scheduling task parameters of accessed multi-type sensing equipment according to a communication channel;
s2, carrying out standardized packaging on the sensor control instruction through an instruction module, then coding and sending the sensor control instruction, and acquiring initial data of the sensor;
s3, using a periodic acquisition module in the data acquisition module to perform periodic data acquisition on the data acquisition channel;
s4, carrying out self-adaptive decoding on data acquired and returned by the sensor through a sensor instruction module;
s5, carrying out data parameterization calculation and data quality calibration through a data calculation module;
and S6, converting the data subjected to quality calibration into a Json data sequence through a data storage module according to a data storage rule set, storing the Json data sequence in a local database, and uploading the Json data sequence to a big data platform through a data transmission module.
Example 3
In the actual process of data acquisition, according to the needs of actual conditions, the situation that a plurality of sensors are required to acquire data simultaneously sometimes occurs. In order to solve the technical problem, the following technical scheme is adopted for further optimization on the basis of the embodiment 1:
arranging the acquisition rules of a plurality of sensors through a periodic acquisition module to realize cooperative control data acquisition; the cooperative control data acquisition comprises the cooperative control of the sensors across the channels and the cooperative monitoring of the multi-type sensing equipment under the same channel.
The working principle of example 3 is explained in detail below:
1. sensor coordinated control across channels
For example, in the working process of the measurement robot, a protection door switch and an environmental sensor of the protection device need to be controlled to acquire environmental parameters, in this mode, a measurement robot channel needs to be used as a main channel, a protection door device needs to be used as a second channel, and an environmental sensor needs to be used as a third channel.
2. Multi-type sensing device cooperative monitoring
And in the acquisition process, when the data of a certain sensing device is found to be abnormal and is out of the range of the early warning control value, immediately triggering the cooperative encrypted data observation of other sensing devices in the same group, carrying out quality analysis and result study and judgment on the data of the current encrypted observation group, and evaluating the accuracy of the current sensing data exceeding the early warning threshold value.
Through the arrangement of the acquisition rules of a plurality of sensing devices, the cooperative control data acquisition is realized, the data acquisition time can be saved, errors are reduced, and the data acquisition efficiency is further improved.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.
Claims (9)
1. The utility model provides an intelligent gateway data acquisition system based on Json data piece for the sensor carries out data acquisition, its characterized in that: the data acquisition system comprises a task control module, an instruction control module, a data acquisition module, a data calculation module, a data storage module and a data transmission module;
the task control module is used for establishing a data acquisition task according to a communication channel accessed by the equipment of the intelligent gateway and configuring and managing task execution on the communication channel;
the instruction control module is used for carrying out standardized packaging and encoding sending on the control instruction of the sensor according to a sensor instruction rule set configured by design; the device is also used for carrying out self-adaptive decoding on the data collected and returned by the sensor and extracting corresponding data information;
the data acquisition module comprises a data initial learning module and a periodic acquisition module; the data initial learning module is used for acquiring initial data of the sensor, and the periodic acquisition module is used for periodically acquiring data of the data acquisition channel according to data acquisition rule configuration;
the data calculation module performs data parameterization calculation and data quality calibration on the acquired sensor data according to the data calculation rule set;
the data storage module converts the acquired sensor raw data into a Json data sequence according to a data storage rule set and then stores the Json data sequence in a local database;
and the data transmission module converts the Json data sequence stored in the data storage module into a Json data packet and an XML file packet respectively according to a data storage rule set and uploads the Json data packet and the XML file packet to a big data platform.
2. A data collection method using the Json data slice-based intelligent gateway data collection system of claim 1, comprising the steps of:
s1, establishing a data acquisition task through a task control module, and configuring and scheduling task parameters of accessed multi-type sensing equipment according to a communication channel;
s2, carrying out standardized packaging on a sensor control instruction according to a designed and configured sensor instruction rule set through an instruction control module, then coding and sending the sensor control instruction, and acquiring initial data of a sensor;
s3, using a periodic acquisition module in the data acquisition module to perform periodic data acquisition on the data acquisition channel;
s4, carrying out self-adaptive decoding on the data acquired and returned by the sensor through the instruction control module;
s5, carrying out data parameterization calculation and data quality calibration through a data calculation module;
and S6, converting the data subjected to quality calibration into a Json data sequence through a data storage module according to a data storage rule set, storing the Json data sequence in a local database, and uploading the Json data sequence to a big data platform through a data transmission module.
3. A data acquisition method as claimed in claim 2, wherein the task control module is specifically adapted to operate in accordance with the following steps:
managing a data acquisition task according to the equipment communication channel of the intelligent gateway;
configuring task parameters for data acquisition of a current communication channel, wherein the parameters comprise an acquisition method, initial time, sampling interval, a cooperative channel and a cooperative mode;
configuring a sensor of a current communication channel, and setting a sampling limit difference of the sensor;
and maintaining the data acquisition channel according to the acquisition task configuration, wherein the maintenance comprises starting, stopping and resetting.
4. A data acquisition method according to claim 2, wherein the command control module is specifically adapted to operate in the following steps:
preloading a sensor instruction rule set file, wherein the rule set file is organized in a Json format;
searching matched sensor types and loading corresponding instruction sets according to task scheduling parameters of the data acquisition module;
capturing corresponding standard instruction set configuration according to the currently executed instruction name;
extracting coding information according to the currently matched instruction configuration, carrying out instruction encapsulation, and sending the coding information to a sensor under a corresponding communication channel;
monitoring data return information of a current communication channel, and stopping data reception when a data string meeting a stop identifier is received;
according to the decoding and data extraction rule configured by the current instruction, self-decoding of the data string is carried out; loading corresponding calculation rules and calculation models according to the matched sensor types, and performing regularized calculation on data by using a data calculation module;
and matching the storage identifiers of the data blocks by combining the data storage rule set according to the data identifier information configured by the current instruction, and packaging into a Json data sheet.
5. A data acquisition method according to claim 4, characterized in that: the set of sensor instruction rules consists of:
the device control instruction comprises an instruction name, an initial character, an instruction code, an instruction decode, an instruction classification, a device address, an instruction state, a data extraction, a data bit identification and a calculation model;
the control and access of the sensor are formed by one or more equipment control instructions, and the drive control and data acquisition of the sensor are realized.
6. The data acquisition method according to claim 2, wherein the data initial learning module specifically operates according to the following steps:
under the selected communication channel, adding a new sensor node, and loading a sensor instruction rule set corresponding to the node;
calling a control instruction to check the working state of the sensing equipment, and calling a measurement instruction to acquire initial configuration data and initial observation data of the sensor;
storing the obtained initial configuration and initial observation data in a local database.
7. The data acquisition method according to claim 2, wherein the periodic acquisition module specifically operates according to the following steps:
starting a task timer, loading parameter configuration, sensor configuration and acquisition rule configuration of a matching channel, and executing data acquisition work of a current period; the data acquisition working mode comprises polling data acquisition and grouped data acquisition;
and after the data acquisition of the current period is finished, the data quality is checked and corrected by the data calculation module, and after the data is qualified, the local data is stored and uploaded to the big data platform.
8. A data acquisition method according to claim 2, characterized in that: the periodic acquisition module arranges acquisition rules of a plurality of sensors and cooperatively controls data acquisition; the cooperative acquisition control data comprises sensor cooperative control of cross-channel and cooperative monitoring of multiple types of sensing equipment under the same channel.
9. A data acquisition method according to claim 2, characterized in that: the data storage rule set comprises a data name, a data tag, a data type, a data index and a data association; the data storage rule set is used for changing configuration of different types of sensors during data storage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010340790.3A CN111556141B (en) | 2020-04-26 | 2020-04-26 | Intelligent gateway data acquisition system and method based on Json data sheet |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010340790.3A CN111556141B (en) | 2020-04-26 | 2020-04-26 | Intelligent gateway data acquisition system and method based on Json data sheet |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111556141A true CN111556141A (en) | 2020-08-18 |
| CN111556141B CN111556141B (en) | 2023-04-28 |
Family
ID=72004031
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010340790.3A Active CN111556141B (en) | 2020-04-26 | 2020-04-26 | Intelligent gateway data acquisition system and method based on Json data sheet |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111556141B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112083671A (en) * | 2020-08-20 | 2020-12-15 | 英众世纪(苏州)智能科技有限公司 | A thing networking intelligent gateway for wisdom electric power |
| CN112187953A (en) * | 2020-10-13 | 2021-01-05 | 南开大学 | JSON-based gene ontology mapping system and method |
| CN113938530A (en) * | 2021-09-09 | 2022-01-14 | 中国联合网络通信集团有限公司 | Data transmission method and device |
| CN114257392A (en) * | 2020-09-24 | 2022-03-29 | 深圳富桂精密工业有限公司 | Equipment management method, equipment management system and gateway |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143606A (en) * | 2011-03-07 | 2011-08-03 | 中国农业大学 | Wireless sensor network gateway for farmland information acquisition |
| CN202551364U (en) * | 2012-03-11 | 2012-11-21 | 上海宜云物联科技有限公司 | Multi-protocol gateway for wireless sensor network |
| CN104104674A (en) * | 2014-07-01 | 2014-10-15 | 广东艾科技术股份有限公司 | Multiprotocol distributed data acquisition control system of equipment sensor and method thereof |
| WO2015001041A1 (en) * | 2013-07-05 | 2015-01-08 | Lyse Smart As | A gateway system for facilitating the interoperability between different service domains and a multitude of communication protocols within and outside a home network |
| US20170180913A1 (en) * | 2015-12-18 | 2017-06-22 | Archimedes Controls Corp. | Intelligent multi-channel wireless data acquisition gateway |
| CN106899638A (en) * | 2015-12-21 | 2017-06-27 | 中国科学院信息工程研究所 | A kind of fusions networking relationships system for solving Heterogeneous data and method |
| US20180034715A1 (en) * | 2016-07-29 | 2018-02-01 | Splunk Inc. | Analytics for Edge Devices |
| US20190207650A1 (en) * | 2016-06-22 | 2019-07-04 | Soraa, Inc. | Intelligent modules for intelligent networks |
| CN110098953A (en) * | 2019-03-29 | 2019-08-06 | 南京邮电大学 | The things-internet gateway hierarchical control framework of rule-based engine |
| CN110189070A (en) * | 2019-04-15 | 2019-08-30 | 中国电子科技集团公司电子科学研究院 | Intelligent logistics Internet of things system |
| CN110716706A (en) * | 2019-10-30 | 2020-01-21 | 华北水利水电大学 | Intelligent human-computer interaction instruction conversion method and system |
-
2020
- 2020-04-26 CN CN202010340790.3A patent/CN111556141B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102143606A (en) * | 2011-03-07 | 2011-08-03 | 中国农业大学 | Wireless sensor network gateway for farmland information acquisition |
| CN202551364U (en) * | 2012-03-11 | 2012-11-21 | 上海宜云物联科技有限公司 | Multi-protocol gateway for wireless sensor network |
| WO2015001041A1 (en) * | 2013-07-05 | 2015-01-08 | Lyse Smart As | A gateway system for facilitating the interoperability between different service domains and a multitude of communication protocols within and outside a home network |
| CN104104674A (en) * | 2014-07-01 | 2014-10-15 | 广东艾科技术股份有限公司 | Multiprotocol distributed data acquisition control system of equipment sensor and method thereof |
| US20170180913A1 (en) * | 2015-12-18 | 2017-06-22 | Archimedes Controls Corp. | Intelligent multi-channel wireless data acquisition gateway |
| CN106899638A (en) * | 2015-12-21 | 2017-06-27 | 中国科学院信息工程研究所 | A kind of fusions networking relationships system for solving Heterogeneous data and method |
| US20190207650A1 (en) * | 2016-06-22 | 2019-07-04 | Soraa, Inc. | Intelligent modules for intelligent networks |
| US20180034715A1 (en) * | 2016-07-29 | 2018-02-01 | Splunk Inc. | Analytics for Edge Devices |
| CN110098953A (en) * | 2019-03-29 | 2019-08-06 | 南京邮电大学 | The things-internet gateway hierarchical control framework of rule-based engine |
| CN110189070A (en) * | 2019-04-15 | 2019-08-30 | 中国电子科技集团公司电子科学研究院 | Intelligent logistics Internet of things system |
| CN110716706A (en) * | 2019-10-30 | 2020-01-21 | 华北水利水电大学 | Intelligent human-computer interaction instruction conversion method and system |
Non-Patent Citations (5)
| Title |
|---|
| 上海卓岚: "JSON数据采集网关", 《百度HTTP://WWW.ZLMCU.COM/DOCUMENT/JSONDATA.HTML》 * |
| 刘勇: "《国之重器出版工程 装甲车辆协同作战辅助决策技术》", 31 March 2020 * |
| 孟祥英: "无线传感器网络事件监测系统关键技术的研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 李喆: "基于SDN的IoT设备智能管理系统的研究" * |
| 玄利永: "多传感器网络协同控制研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112083671A (en) * | 2020-08-20 | 2020-12-15 | 英众世纪(苏州)智能科技有限公司 | A thing networking intelligent gateway for wisdom electric power |
| CN114257392A (en) * | 2020-09-24 | 2022-03-29 | 深圳富桂精密工业有限公司 | Equipment management method, equipment management system and gateway |
| CN112187953A (en) * | 2020-10-13 | 2021-01-05 | 南开大学 | JSON-based gene ontology mapping system and method |
| CN112187953B (en) * | 2020-10-13 | 2022-05-03 | 南开大学 | JSON-based gene ontology mapping system and method |
| CN113938530A (en) * | 2021-09-09 | 2022-01-14 | 中国联合网络通信集团有限公司 | Data transmission method and device |
| CN113938530B (en) * | 2021-09-09 | 2023-06-23 | 中国联合网络通信集团有限公司 | Data transmission method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111556141B (en) | 2023-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111556141A (en) | Intelligent gateway data acquisition system and method based on Json data sheet | |
| CN108040005B (en) | Digit control machine tool intelligent monitoring gateway based on thing networking | |
| CN107390636B (en) | Operation monitoring method, maintenance method and system for numerical control equipment | |
| US7908360B2 (en) | Correlation of non-times series events in industrial systems | |
| US20090089325A1 (en) | Targeted resource allocation | |
| CN102262401A (en) | Industrial production line monitoring system | |
| CN102882724B (en) | Field bus equipment monitoring and management method | |
| CN108009081B (en) | Engineering design tool cooperation device and engineering design tool cooperation method | |
| CN112615935B (en) | Terminal equipment networking reference system and interaction method thereof | |
| Hammerstingl et al. | Unified Plug&Produce architecture for automatic integration of field devices in industrial environments | |
| US20090089226A1 (en) | Visualization of non-time series events | |
| Shu et al. | Novel modbus adaptation method for IoT gateway | |
| CN102880146B (en) | Profibus device monitoring and managing system | |
| CN107113199A (en) | Analytical equipment for analyzing and handling communication sequence | |
| Mónica et al. | Agent-based communication to map and exchange shop floor data between MES and material flow simulation based on the open standard CMSD | |
| CN116016579A (en) | Data acquisition system of automatic equipment | |
| Alkan et al. | Assessing complexity of component-based control architectures used in modular automation systems | |
| EP2091211A1 (en) | Method for generic conversion between server data and client data | |
| CN109991924A (en) | A kind of means of communication and system for digital control system Terminal Server Client real time monitoring | |
| KR20230032675A (en) | System for collecting data using computerized numerical control mother machine | |
| CN116032021B (en) | New energy station integrated monitoring method, system, equipment and storage medium | |
| CN111290336A (en) | Numerical control machine tool control method and control system based on Android system | |
| WO2008155596A1 (en) | Standardized protocol independent configuration tool for intelligent electronic devices | |
| CN113791593B (en) | Measuring point derivation method and system for distributed control system, terminal device and readable storage medium | |
| CN117194501B (en) | DCS trend measurement point jump logic configuration method, system, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240318 Address after: No. 6 Qingzhu East Road, Dazhulin Street, Yubei District, Chongqing, 400000 Patentee after: Chongqing Institute of Surveying and Mapping Science and Technology (Chongqing Map Compilation Center) Country or region after: China Address before: 401121 No. 6, Qingzhu East Road, Yubei District, Chongqing Patentee before: CHONGQING SURVEY INSTITUTE Country or region before: China |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240320 Address after: No. 6 Qingzhu East Road, Dazhulin Street, Yubei District, Chongqing, 400000 Patentee after: Chongqing Institute of Surveying and Mapping Science and Technology (Chongqing Map Compilation Center) Country or region after: China Address before: 401121 No. 6, Qingzhu East Road, Yubei District, Chongqing Patentee before: CHONGQING SURVEY INSTITUTE Country or region before: China |
|
| TR01 | Transfer of patent right |