CN111580984B - Data processing method, system, storage medium and terminal - Google Patents

Data processing method, system, storage medium and terminal Download PDF

Info

Publication number
CN111580984B
CN111580984B CN202010245595.2A CN202010245595A CN111580984B CN 111580984 B CN111580984 B CN 111580984B CN 202010245595 A CN202010245595 A CN 202010245595A CN 111580984 B CN111580984 B CN 111580984B
Authority
CN
China
Prior art keywords
data
data processing
gateway
processing method
cloud
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010245595.2A
Other languages
Chinese (zh)
Other versions
CN111580984A (en
Inventor
沈玉龙
请求不公布姓名
张涛
王良民
曹春杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xidian University
Original Assignee
Xidian University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xidian University filed Critical Xidian University
Priority to CN202010245595.2A priority Critical patent/CN111580984B/en
Publication of CN111580984A publication Critical patent/CN111580984A/en
Application granted granted Critical
Publication of CN111580984B publication Critical patent/CN111580984B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/546Message passing systems or structures, e.g. queues
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16YINFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y10/00Economic sectors
    • G16Y10/60Healthcare; Welfare
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/02Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/12Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/18Multiprotocol handlers, e.g. single devices capable of handling multiple protocols
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/54Indexing scheme relating to G06F9/54
    • G06F2209/547Messaging middleware

Abstract

The invention belongs to the technical field of data processing of the Internet of things, and discloses a data processing method, a system, a storage medium, a computer program and a terminal, wherein the data processing method comprises the following steps: uploading data to corresponding intelligent gateways by various heterogeneous nodes; the middleware positioned at the gateway pre-processes the uploaded data and feeds back a part of data with high time delay requirements in real time after being processed by the miniature data center; the data stream is uploaded to the cloud for backup or large data calculation which cannot be calculated at the gateway is performed while mist calculation is performed. The cloud computing platform solves the problems that the traditional cloud computing platform is high in time delay and cannot feed back data in real time and the middleware related to medical equipment is placed in the cloud, so that information is delayed and even patient safety is endangered. The invention aims to shorten the processing time delay of data generated by heterogeneous nodes, and also plays a role in high performance of cloud processing while timely feedback is ensured.

Description

Data processing method, system, storage medium and terminal
Technical Field
The invention belongs to the technical field of data processing of the Internet of things, and particularly relates to a data processing method, a system, a storage medium, a computer program and a terminal.
Background
With the development of the internet of things, various end devices are connected into the internet of things, the uploaded data is huge in quantity, more troublesome is the diversification of data types, and in order to solve the problem that the uploading data formats of heterogeneous nodes are not uniform, a middleware system is generally adopted in the prior art to achieve data preprocessing and application interoperation. However, the defect in the prior art is that most middleware systems are placed in the cloud, and have higher time delay in the data transmission processing stage, so that the real-time monitoring of various physiological indexes of human bodies by portable medical equipment cannot be met. The reason for generating the defect is mainly that all data are uniformly delivered to a cloud processor without distinguishing the data according to timeliness requirements. Only the feasibility of data processing is guaranteed, but the timeliness is ignored.
The difficulty in solving the problems and the defects is mainly that the traditional edge node has limited computing capacity, and if middleware is deployed at the gateway of the traditional internet of things, the expected timely feedback effect cannot be achieved, and even the basic data processing task cannot be completed.
The significance of solving the above problems and defects is mainly divided into two points: 1. data with high real-time requirements are separated from the whole data stream, and corresponding operations are executed for different data. 2. Depending on the existing equipment, such as: MDC and FSG, make up for technical defects with hardware updates to update traditional designs to accommodate more specific application scenarios.
Disclosure of Invention
Aiming at the problems existing in the prior art, the invention provides a data processing method, a system, a storage medium, a computer program and a terminal.
The invention is realized in that a data processing method comprises the following steps:
step one, uploading data to corresponding intelligent gateways by various heterogeneous nodes;
and secondly, preprocessing the uploaded data by the middleware positioned at the gateway, namely, marking different data by various heterogeneous nodes according to the aging priority order, uploading the data to the corresponding intelligent gateway, setting a threshold value in advance by the middleware positioned at the gateway to judge the data with the mark, and processing a part of data with high time delay requirements by a miniature data center. The specific solution is that multi-protocol binding is realized at a unified protocol interface to be used as a southbound interface using protocols such as WIFI, bluetooth, serial ports and the like, the unified API provides access to the data of the Internet of things by using proxy API call, and the middleware sends the data back to the medical equipment through the southbound interface by using a post method by using an HTTP protocol based on a request response model, so that data operation is completed.
And thirdly, uploading the data stream to the cloud for backup or executing the traditional big data calculation which cannot be realized at the gateway while carrying out fog calculation. The advantages of the conventional solution are retained.
Further, a heterogeneous network formed by bottom equipment of the data processing method generates a large number of data streams and uploads the data streams to an intelligent gateway FSG based on fog calculation.
Further, the middleware at the gateway of the data processing method preprocesses the data stream and delivers the data stream to the micro data center, and the micro data center processes and timely reverses some data with low time delay requirements.
Further, the bottom layer equipment of the data processing method responds to the data fed back at the gateway according to the preset threshold value.
Furthermore, the intelligent gateway and the miniature data center of the data processing method also upload the data to the cloud while returning the data.
It is a further object of the present invention to provide a storage medium for receiving user input, the stored computer program causing an electronic device to perform the steps of any one of the claims comprising:
step one, uploading data to corresponding intelligent gateways by various heterogeneous nodes;
secondly, the middleware at the gateway preprocesses the uploaded data and feeds back a part of data with high time delay requirements in real time after being processed by the miniature data center;
and thirdly, uploading the data stream to the cloud for backup or executing big data calculation which cannot be calculated at the gateway while performing fog calculation.
It is a further object of the present invention to provide a computer program product stored on a computer readable medium, comprising a computer readable program for providing a user input interface for implementing the data processing method when executed on an electronic device.
Another object of the present invention is to provide a data processing system for implementing the data processing method, the data processing system comprising:
the data uploading module is used for uploading data to the corresponding intelligent gateway by various heterogeneous nodes;
the real-time feedback module is used for realizing that the middleware positioned at the gateway preprocesses the uploaded data and feeds back a part of data with high time delay requirements in real time after being processed by the miniature data center;
and the big data calculation module is used for realizing that the data stream is uploaded to the cloud for backup or executing big data calculation which cannot be calculated at the gateway while carrying out fog calculation.
Another object of the present invention is to provide a data processing apparatus on which the data processing system is mounted, the data processing apparatus including: the system comprises bottom layer equipment, an intelligent gateway, a miniature data center and a cloud;
the bottom layer equipment is connected with the miniature data center, and the intelligent gateway and the miniature data center are connected with the cloud.
Another object of the present invention is to provide a terminal, which mounts the data processing system.
By combining all the technical schemes, the invention has the advantages and positive effects that: the invention is mainly a functional method of combining a Micro Data Center (MDC) with an intelligent gateway Fog Smart Gateway (FSG). The invention uses equipment (1) miniature data center: miniature data centers (also referred to as modular or miniature modular data centers) are small, stand-alone versions of traditional data centers. The independence of the miniature data center itself, as well as its robustness, flexibility and portability, makes miniature data centers an ideal choice for deployment based on fog computing centers. Due to the small size, the miniature data center has lower deployment cost than the traditional large data center. Given that miniature data centers are prefabricated, their size is much smaller than traditional data centers, which also tends to be more cost effective. The deployment time is faster. (2) And (3) an intelligent gateway: the gateway of the Internet of things with the ultra-strong edge computing capability performs nearby processing through the lowering of the data processing authority. The problem that remote communication transmission is not smooth is not needed, and the gateway has huge functional advantages related to the common Internet of things gateway. Fog calculations are calculations at the edges of the network, equivalent to falling cloud calculations closer to the underlying device. However, unlike cloud computing, which requires data to be transmitted to a cloud-located data center for processing during data transmission, cloud computing uses data processing at a network edge such as a gateway. So that the data transmission is real-time.
The cloud computing platform solves the problems that the traditional cloud computing platform is high in time delay and cannot feed back data in real time and the middleware related to medical equipment is placed in the cloud, so that information is delayed and even patient safety is endangered. The invention aims to shorten the processing time delay of data generated by heterogeneous nodes, and also plays a role in high performance of cloud processing while timely feedback is ensured.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following description will briefly explain the drawings needed in the embodiments of the present application, and it is obvious that the drawings described below are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present invention.
FIG. 2 is a schematic diagram of a data processing system according to an embodiment of the present invention;
in the figure: 1. a data uploading module; 2. a real-time feedback module; 3. and a big data calculation module.
Fig. 3 is a flowchart of an implementation of a data processing method according to an embodiment of the present invention.
FIG. 4 is a schematic diagram of a data processing system provided by an embodiment of the present invention.
Fig. 5 is a schematic diagram of experimental demonstration data of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the following examples in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The new improvement scheme is as follows: an AMQP proxy may be added to support AMQP messaging between medical devices to enable interoperability between HTTP and AMQP.
In view of the problems existing in the prior art, the present invention provides a data processing method, a system, a storage medium, a computer program, and a terminal, and the present invention is described in detail below with reference to the accompanying drawings.
As shown in fig. 1, the data processing method provided by the invention comprises the following steps:
s101: uploading data to corresponding intelligent gateways by various heterogeneous nodes;
s102: the middleware positioned at the gateway pre-processes the uploaded data and feeds back a part of data with high time delay requirements in real time after being processed by the miniature data center;
s103: the data stream is uploaded to the cloud for backup or large data calculation which cannot be calculated at the gateway is performed while mist calculation is performed.
As shown in fig. 2, the data processing system provided by the present invention includes:
and the data uploading module 1 is used for uploading data to the corresponding intelligent gateway by various heterogeneous nodes.
And the real-time feedback module 2 is used for preprocessing the uploaded data by the middleware positioned at the gateway and feeding back a part of data with high time delay requirements in real time after being processed by the miniature data center.
And the big data calculation module 3 is used for realizing that the data stream is uploaded to the cloud for backup or executing big data calculation which cannot be calculated at the gateway while carrying out fog calculation.
The technical scheme of the invention is further described below with reference to the accompanying drawings.
As shown in fig. 3, the data processing method provided by the invention comprises the following steps:
step one, heterogeneous network composed of bottom layer equipment generates a large amount of data flow and uploads the data flow to an intelligent gateway FSG based on fog calculation;
and step two, the middleware positioned at the gateway preprocesses the data stream and delivers the data stream to the miniature data center. The miniature data center processes and timely reverses some data with low time delay requirements;
responding the data fed back by the gateway according to the preset threshold value by the bottom layer equipment;
and step four, the cloud end (namely the intelligent gateway and the miniature data center) also uploads the data to the cloud end while returning the data, because the computing capacity at the gateway is limited and the computing capacity of the cloud computing is required to be supplemented.
As shown in fig. 5, the round trip delay of the middleware deployed at the edge is significantly lower than that of the middleware deployed at the cloud. The scheme can effectively improve the data processing speed.
It should be noted that the embodiments of the present invention can be realized in hardware, software, or a combination of software and hardware. The hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or special purpose design hardware. Those of ordinary skill in the art will appreciate that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, such as provided on a carrier medium such as a magnetic disk, CD or DVD-ROM, a programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The device of the present invention and its modules may be implemented by hardware circuitry, such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., as well as software executed by various types of processors, or by a combination of the above hardware circuitry and software, such as firmware.
The foregoing is merely illustrative of specific embodiments of the present invention, and the scope of the invention is not limited thereto, but any modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present invention will be apparent to those skilled in the art within the scope of the present invention.

Claims (8)

1. A data processing method, characterized in that the data processing method comprises:
step one, uploading data to corresponding intelligent gateways by various heterogeneous nodes;
secondly, the middleware at the gateway preprocesses the uploaded data, the middleware at the gateway sets a threshold value in advance to judge the data with the mark, and a part of data with high time delay requirement is processed by a miniature data center; the method is characterized in that multi-protocol binding is realized at a unified protocol interface to be used as a southbound interface using WIFI, bluetooth and serial ports, the unified API provides access to the data of the Internet of things by using proxy API call, and the middleware sends the data back to the medical equipment through the southbound interface by using a post method by using HTTP protocol based on a request response model to complete data operation;
thirdly, uploading the data stream to a cloud for backup or executing traditional big data calculation which cannot be realized at a gateway while performing fog calculation;
heterogeneous network composed of bottom layer equipment of the data processing method generates a large amount of data flow and uploads the data flow to an intelligent gateway FSG based on fog calculation.
2. The data processing method according to claim 1, wherein middleware located at a gateway of the data processing method pre-processes data streams and delivers the data streams to a micro data center, and the micro data center processes and timely feeds back some data with low latency requirements.
3. The data processing method of claim 1, wherein an underlying device of the data processing method responds to data fed back at the gateway according to a threshold set in advance.
4. The data processing method of claim 1, wherein the intelligent gateway and the mini-data center of the data processing method also upload data to the cloud while returning the data.
5. A storage medium receiving a user input program, the stored computer program causing an electronic device to perform the data processing method of any one of claims 1 to 4, comprising the steps of:
step one, uploading data to corresponding intelligent gateways by various heterogeneous nodes;
secondly, the middleware at the gateway preprocesses the uploaded data and feeds back a part of data with high time delay requirements in real time after being processed by the miniature data center;
and thirdly, uploading the data stream to the cloud for backup or executing big data calculation which cannot be calculated at the gateway while performing fog calculation.
6. A data processing system for implementing the data processing method of any one of claims 1 to 4, the data processing system comprising:
the data uploading module is used for uploading data to the corresponding intelligent gateway by various heterogeneous nodes;
the real-time feedback module is used for realizing that the middleware positioned at the gateway preprocesses the uploaded data and feeds back a part of data with high time delay requirements in real time after being processed by the miniature data center;
and the big data calculation module is used for realizing that the data stream is uploaded to the cloud for backup or executing big data calculation which cannot be calculated at the gateway while carrying out fog calculation.
7. A data processing apparatus incorporating the data processing system of claim 6, the data processing apparatus comprising: the system comprises bottom layer equipment, an intelligent gateway, a miniature data center and a cloud;
the bottom layer equipment is connected with the miniature data center, and the intelligent gateway and the miniature data center are connected with the cloud.
8. A terminal carrying the data processing system of claim 6.
CN202010245595.2A 2020-03-31 2020-03-31 Data processing method, system, storage medium and terminal Active CN111580984B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010245595.2A CN111580984B (en) 2020-03-31 2020-03-31 Data processing method, system, storage medium and terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010245595.2A CN111580984B (en) 2020-03-31 2020-03-31 Data processing method, system, storage medium and terminal

Publications (2)

Publication Number Publication Date
CN111580984A CN111580984A (en) 2020-08-25
CN111580984B true CN111580984B (en) 2024-02-02

Family

ID=72122490

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010245595.2A Active CN111580984B (en) 2020-03-31 2020-03-31 Data processing method, system, storage medium and terminal

Country Status (1)

Country Link
CN (1) CN111580984B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112118303A (en) * 2020-09-08 2020-12-22 湖南建工集团有限公司 Edge computing Internet of things middleware for multiple communication protocols

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106851746A (en) * 2016-12-26 2017-06-13 上海交通大学 The method for realizing software definition QoS configurations in radio sensing network is calculated based on mist
CN106911762A (en) * 2017-01-16 2017-06-30 西安电子科技大学 A kind of framework calculated based on mist in SDN and its processing method
DE102016215742A1 (en) * 2016-08-23 2018-03-01 Robert Bosch Gmbh Gateway and method for connecting a data source system to an IT system
CN108540538A (en) * 2018-03-20 2018-09-14 西安电子科技大学 A kind of cloud and mist conjugate working application structure system based on SDN
CN109862011A (en) * 2019-02-01 2019-06-07 华南理工大学 A kind of environment of internet of things real-time monitoring system calculated based on mist

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102016215742A1 (en) * 2016-08-23 2018-03-01 Robert Bosch Gmbh Gateway and method for connecting a data source system to an IT system
CN106851746A (en) * 2016-12-26 2017-06-13 上海交通大学 The method for realizing software definition QoS configurations in radio sensing network is calculated based on mist
CN106911762A (en) * 2017-01-16 2017-06-30 西安电子科技大学 A kind of framework calculated based on mist in SDN and its processing method
CN108540538A (en) * 2018-03-20 2018-09-14 西安电子科技大学 A kind of cloud and mist conjugate working application structure system based on SDN
CN109862011A (en) * 2019-02-01 2019-06-07 华南理工大学 A kind of environment of internet of things real-time monitoring system calculated based on mist

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
李树磊 ; 樊毅 ; 杜剑波 ; .基于混合云雾计算的物联网架构.邮电设计技术.2018,(02),全文. *
段粘粘 ; .雾计算在高校信息系统中的应用.电脑知识与技术.2018,(02),全文. *

Also Published As

Publication number Publication date
CN111580984A (en) 2020-08-25

Similar Documents

Publication Publication Date Title
US20200192687A1 (en) Techniques for checkpointing/delivery between primary and secondary virtual machines
WO2021259013A1 (en) Data processing method and apparatus, electronic device, and computer-readable medium
WO2021151312A1 (en) Method for determining inter-service dependency, and related apparatus
CN104993947A (en) Method for upgrading firmware in batches
CN110209399A (en) FPGA service system, data processing method and storage medium
CN112527647B (en) NS-3-based Raft consensus algorithm test system
US20190320022A1 (en) Quality of service knobs for visual data storage
CN116541227B (en) Fault diagnosis method and device, storage medium, electronic device and BMC chip
CN112306567B (en) Cluster management system and container management and control method
CN111580984B (en) Data processing method, system, storage medium and terminal
CN103713940A (en) Method for reconfigurable distributed real-time simulation based on RTX-HLA reflection storage card
CN116243995A (en) Communication method, communication device, computer readable storage medium, and electronic apparatus
CN110658751B (en) Implementation method of EtherCAT field bus control system
CN105607940A (en) Method for transmitting information to UEFI BIOS from BDK in ARM platform
US10348428B2 (en) Techniques for synchronized execution of a command at network fabric nodes
CN108228888A (en) A kind of management method of assets information, system, device and readable storage medium storing program for executing
CN104850516A (en) DDR frequency conversion design method and device
WO2017091963A1 (en) Information processing method and apparatus
CN111371871A (en) Block chain link point equipment and block chain network system
CN112905668B (en) Database derivative method, device and medium based on distributed data stream processing engine
CN110109718B (en) Application program interface calling method and device
CN115542772B (en) Simulation task control method, device and simulation system
CN116521603A (en) Method for realizing MCTP protocol based on FPGA
CN108804362A (en) Serial ports bulk data transmission method, device and storage medium
CN115150444A (en) Control method and device for mechanical arm and server

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant