CN105471958A - Data processing system and method of internet of things - Google Patents
Data processing system and method of internet of things Download PDFInfo
- Publication number
- CN105471958A CN105471958A CN201510481516.7A CN201510481516A CN105471958A CN 105471958 A CN105471958 A CN 105471958A CN 201510481516 A CN201510481516 A CN 201510481516A CN 105471958 A CN105471958 A CN 105471958A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- time period
- monitoring
- internet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 25
- 238000000034 method Methods 0.000 title claims abstract description 19
- 238000012544 monitoring process Methods 0.000 claims abstract description 71
- 238000013144 data compression Methods 0.000 claims description 13
- 238000013500 data storage Methods 0.000 claims description 9
- 238000013480 data collection Methods 0.000 abstract description 8
- 230000005540 biological transmission Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 10
- 230000003993 interaction Effects 0.000 description 7
- 230000000903 blocking effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000003672 processing method Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a data processing system and method of an internet of things. The system comprises a data collection module used for collecting data of a data collection terminal and transmitting the collected data to the data monitoring terminal in real time; and a data monitoring module connected with the data collection module and used for storing the data collected by the data collection module to a hardware storage device, compressing the data and sending the compressed data to a data center at intervals. According to the invention, the data of the data collection terminal is transmitted to the data monitoring terminal in real time, the data of the monitoring terminal is simultaneously and temporarily stored to the internet of things hardware storage device and compressed, and the compressed data is sent to the data center for storage at certain intervals, so that the network congestion is effectively relieved.
Description
Technical Field
The invention belongs to the technical field of computer networks, and particularly relates to a system and a method for processing data of an Internet of things.
Background
In cloud computing storage, the storage of internet of things information is very common. Currently, the internet of things information includes three types, namely monitoring video data information, real-time temperature and humidity data information and signal early warning information. The three kinds of information are all real-time information, the data volume is huge, and information congestion is easily caused in the process of storage and transmission. For example, when storing monitoring video data information, suppose that a camera is installed at each monitoring point in an area, and 100 cameras are installed in the area altogether, and then the 100 cameras need to send data to the data center in real time, then the information interaction data of the data center will be huge at this moment, and huge information interaction will occupy a large amount of bandwidth, thereby causing congestion of the monitoring video data information.
Disclosure of Invention
The invention aims to provide a system and a method for processing data of the Internet of things, which use the data compression and data temporary storage technology of a terminal to disperse the bandwidth time occupied by data transmission on the Internet, realize the purpose of synchronously transmitting big data information to a data center in real time and simultaneously preventing the network from being blocked, and overcome the defect of data information congestion caused by transmitting the big data to the data center in real time in the data processing of the Internet of things in the prior art.
According to an aspect of the present invention, there is provided a system for processing data of an internet of things, the system including:
and the data monitoring module is used for monitoring the received acquired data in real time, storing the data, compressing the data in the time period every other time period and uploading the compressed data.
In the foregoing scheme, the system further includes:
the data acquisition module is used for acquiring data and transmitting the acquired data to the data monitoring module in real time; and the number of the first and second groups,
and the data center is used for receiving, storing and processing the compressed and uploaded data.
In the above scheme, the data acquisition module transmits the acquired data to the data monitoring module in real time by sending the virtual signal.
In the above scheme, the data monitoring module includes: the real-time monitoring sub-module, the data storage sub-module, the data compression sub-module and the data sending sub-module; wherein,
the real-time monitoring module is used for monitoring the data in real time;
the data storage submodule is used for storing the data;
the data compression submodule is used for compressing the data in the time period every other time period;
and the data sending submodule is used for sending the compressed data to a data center.
In the above scheme, the system further includes a time period setting module, connected to the data monitoring module, and configured to set an interval time period for the data monitoring module to send compressed data to the data center.
According to another aspect of the present invention, there is also provided a method for processing data of the internet of things, the method including:
and step S2, storing the received collected data, compressing the data in the time period every other time period, and uploading the compressed data.
In the above scheme, the method further includes:
and step S1, collecting data, and transmitting and monitoring the collected data in real time.
In the above scheme, the transmitting the collected data in real time includes:
and transmitting the acquired data in real time by adopting a mode of sending a virtual signal.
In the foregoing scheme, the step S2 includes:
step S21, setting a time period;
step S22, storing the data in the time period;
and step S23, compressing the stored data in the time period and uploading the compressed data.
In the foregoing scheme, the step S21 includes: the time period is set to 10S.
According to the data processing system and method of the Internet of things, the data of the data acquisition terminal is transmitted to the data monitoring terminal in real time, meanwhile, the data of the monitoring terminal is temporarily stored in the hardware storage equipment of the Internet of things and compressed, and the compressed data is sent to the data center at regular intervals for storage, so that network congestion is effectively relieved. The invention uses the data compression and data temporary storage technology of the terminal to disperse the bandwidth time occupied by data transmission to the Internet, thereby realizing the purpose of synchronously transmitting the big data information to the data center in real time and simultaneously not blocking the network.
Drawings
FIG. 1 shows a schematic diagram of data processing of the Internet of things of the prior art;
FIG. 2 shows a schematic diagram of the data processing of the Internet of things of the present invention;
FIG. 3 is a schematic diagram of a data processing system of the Internet of things according to a preferred embodiment of the invention;
FIG. 4 is a schematic diagram of a data monitoring module according to a preferred embodiment of the present invention;
FIG. 5 is a schematic diagram of a data processing system of the Internet of things according to another preferred embodiment of the invention;
fig. 6 shows a flow chart of a data processing method of the internet of things according to a preferred embodiment of the present invention;
fig. 7 shows a sub-flowchart of step S2 in fig. 6.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings in conjunction with the following detailed description. It should be understood that the description is intended to be exemplary only, and is not intended to limit the scope of the present invention. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present invention.
Fig. 1 shows a schematic diagram of data processing of the internet of things in the prior art.
As shown in fig. 1, in the data storage of the internet of things, information exchanged between the data acquisition terminal and the data monitoring terminal is often real-time information, the data volume is huge, and information congestion is easily caused in the storage and transmission process. If the data at the data monitoring terminal is stored in the data center in real time, the information interaction data of the data center is huge, and the huge information interaction occupies a large amount of bandwidth, so that the data information of the data monitoring terminal is blocked.
Fig. 2 shows a schematic diagram of the data processing of the internet of things of the present invention.
As shown in fig. 2, in the invention, while data collected by the data collection module is transmitted to the data monitoring module in real time, the collected data is temporarily stored in the internet of things hardware storage device, the internet of things data stored in the internet of things hardware storage device in the time period is compressed every other time period, and the compressed data is sent to the data center for storage, so that the bandwidth time occupied by the data sending to the internet is dispersed, and the network congestion is effectively relieved.
Fig. 3 shows a schematic structural diagram of a data processing system of the internet of things in the preferred embodiment of the invention.
As shown in fig. 3, the data processing system of the internet of things according to the preferred embodiment of the present invention includes: the system comprises a data acquisition module 1 and a data monitoring module 2.
And the data acquisition module 1 is used for acquiring data and transmitting the acquired data to the data monitoring module 2 in real time. In cloud computing storage, the storage of internet of things information is very common. Currently, the internet of things information includes three types, namely monitoring video data information, real-time temperature and humidity data information and signal early warning information. The three kinds of information are all real-time information, the data volume is huge, and information congestion is easily caused in the process of storage and transmission. For example, when storing monitoring video data information, assuming that a camera is installed at each monitoring point in an area, and 100 cameras are installed in the area, the 100 cameras (equivalent to data acquisition modules) need to simultaneously transmit data to the data monitoring module in real time, that is, information acquired by the cameras needs to be transmitted to the data monitoring module in real time, so as to ensure normal operation between the monitoring module and the monitoring camera, specifically, the cameras transmit the acquired data to the data monitoring module in real time by sending virtual signals, that is, videos are transmitted to a video monitoring screen of the monitoring module in virtual signals.
And the data monitoring module 2 is connected with the data acquisition module 1 and used for monitoring the received data in real time, storing the data, compressing the data in the time period at intervals of a time period and sending the compressed data to the data center 3. If the data collected by the data collection module 1 is sent to the data center 3 for storage in real time, the information interaction data of the data center 3 is huge, the huge information interaction occupies a large amount of bandwidth, and therefore congestion of data information is caused, in order to relieve the congestion of the data information, the data monitoring module 2 stores the data collected by the data collection module 1 into hardware storage equipment and compresses the data, and then sends the compressed data to the data center 3 in a mode of sending at intervals.
And the data center 3 is used for receiving, storing and processing the compressed data.
Fig. 4 shows a schematic structural diagram of a data monitoring module according to a preferred embodiment of the present invention.
As shown in fig. 4, the structure of the data monitoring module 2 shown in fig. 3 includes: a real-time monitoring sub-module 20, a data storage sub-module 21, a data compression sub-module 22 and a data transmission sub-module 23.
And the real-time monitoring sub-module 20 is connected with the data acquisition module 1 and is used for monitoring the received data in real time.
And the data storage submodule 21 is connected with the data acquisition module 1 and is used for storing the data acquired by the data acquisition module 1 into hardware storage equipment.
And the data compression submodule 22 is connected with the data storage submodule 21 and is used for compressing the data stored in the hardware storage device. When the data compression submodule 22 detects data stored in the hardware storage device by the data storage submodule 21, the data compression submodule 22 compresses the data to reduce the storage space occupied by the data and improve the transmission speed of the data.
And the data sending submodule 23 is connected with the data compression submodule 22 and is used for sending the compressed data to a data center at intervals. The time period here may be 5S to 20S, preferably 10S. The data sending submodule 23 will forward the compressed data processed by the data compression submodule 22 to the data center for storage according to a certain period of time, so that the bandwidth time occupied by the data sending to the internet can be dispersed, and the problem of network blocking is solved. Specifically, the data sending submodule 23 forwards the compressed data within 10 seconds to the data center every 10 seconds.
In the preferred embodiment of the invention, the real-time interaction of information between the data acquisition terminal and the data monitoring terminal is ensured through the real-time processing of the data acquisition module, the acquired data is temporarily stored, compressed and retransmitted through the data monitoring module, and the bandwidth time occupied by data transmission on the internet is dispersed, so that the purpose of synchronously transmitting the large data information to the data center in real time and simultaneously preventing the network from being blocked by the large data real-time transmission is realized, and the network is prevented from being blocked by the large data real-time transmission.
Fig. 5 is a schematic structural diagram of an internet of things data processing system according to another preferred embodiment of the present invention.
As shown in fig. 5, in another preferred embodiment of the present invention, a time period setting module 4 is further included, configured to set an interval time period for the data monitoring module 2 to send compressed data to the data center 3. The interval time period of data forwarding can be set by the time period setting unit 4, so as to optimize the network to the maximum extent. Preferably, the time period setting unit 4 sets the interval time period to 10S.
Fig. 6 shows a flow chart of a data processing method of the internet of things according to a preferred embodiment of the present invention; fig. 7 shows a sub-flowchart of step S2 in fig. 6.
As shown in fig. 6, the data processing method of the internet of things according to the preferred embodiment of the present invention includes:
and step S1, collecting data, and transmitting and monitoring the collected data in real time.
In cloud computing storage, the storage of internet of things information is very common. Currently, the internet of things information includes three types, namely monitoring video data information, real-time temperature and humidity data information and signal early warning information. The three kinds of information are all real-time information, the data volume is huge, and information congestion is easily caused in the process of storage and transmission. For example, when storing monitoring video data information, assuming that a camera is installed at each monitoring point in an area, and a total of 100 cameras are installed in the area, the 100 cameras need to transmit data to the data acquisition terminal in real time. In this step, the data collected at the data collecting terminal is transmitted to the data monitoring terminal in real time by adopting a mode of sending a virtual signal, that is, the video is sent to the video monitoring screen of the monitoring terminal by the virtual signal, so as to ensure the normal operation between the monitoring terminal and the monitoring camera.
And step S2, storing the collected data in a hardware storage device, compressing the data, and sending the compressed data to a data center at intervals.
In order to alleviate the congestion of the data information, in the step, the acquired data is stored in the hardware storage device and compressed, and then the compressed data is sent to the data center in a mode of sending at intervals.
Specifically, as shown in fig. 7, the flow of step S2 includes:
and step S21, storing the data collected by the data collecting terminal into a hardware storage device.
And step S22, compressing the data stored in the hardware storage device. The data stored in the hardware storage device is compressed to reduce the storage space occupied by the data and improve the transmission speed of the data.
And step S23, sending the compressed data to a data center at intervals. Specifically, according to a certain time period cycle, preferably 10S, that is, the data compressed in the hardware storage device is forwarded to the data center for storage every 10 seconds, so that the bandwidth time occupied by data transmission on the internet can be dispersed, the problem of network blocking is solved, and the operation speed of the current network during real-time transmission of large data is optimized.
As described above, according to the system and method for processing data of the internet of things of the present invention, while data of the data acquisition terminal is transmitted to the data monitoring terminal in real time, the data of the monitoring terminal is temporarily stored in the hardware storage device of the internet of things and compressed, and the compressed data is transmitted to the data center at regular intervals for storage, thereby effectively relieving network congestion. The invention uses the data compression and data temporary storage technology of the terminal to disperse the bandwidth time occupied by data transmission to the Internet, thereby realizing the purpose of synchronously transmitting the big data information to the data center in real time and simultaneously not blocking the network.
It is to be understood that the above-described embodiments of the present invention are merely illustrative of or explaining the principles of the invention and are not to be construed as limiting the invention. Therefore, any modification, equivalent replacement, improvement and the like made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. Further, it is intended that the appended claims cover all such variations and modifications as fall within the scope and boundaries of the appended claims or the equivalents of such scope and boundaries.
Claims (10)
1. A system for processing data of the Internet of things, which is characterized by comprising:
and the data monitoring module (2) is used for monitoring the received acquired data in real time, storing the data, compressing the data in the time period every other time period and uploading the compressed data.
2. The system of claim 1, further comprising:
the data acquisition module (1) is used for acquiring data and transmitting the acquired data to the data monitoring module (2) in real time; and the number of the first and second groups,
and the data center (3) is used for receiving, storing and processing the compressed and uploaded data.
3. The system according to claim 2, characterized in that the data acquisition module (1) transmits the acquired data to the data monitoring module (2) in real time by means of sending virtual signals.
4. The system according to claim 1, characterized in that the data monitoring module (2) comprises: the real-time monitoring sub-module (20), the data storage sub-module (21), the data compression sub-module (22) and the data sending sub-module (23); wherein,
the real-time monitoring module (20) is used for monitoring the data in real time;
the data storage submodule (21) is used for storing the data;
the data compression submodule (22) is used for compressing the data in the time period every other time period;
and the data sending submodule (23) is used for sending the compressed data to a data center.
5. The system according to claim 1, further comprising a time period setting module (4) connected to the data monitoring module (2) for setting an interval time period for the data monitoring module (2) to transmit the compressed data to the data center.
6. A method for processing data of an Internet of things is characterized by comprising the following steps:
and step S2, storing the received collected data, compressing the data in the time period every other time period, and uploading the compressed data.
7. The method of claim 6, further comprising:
and step S1, collecting data, and transmitting and monitoring the collected data in real time.
8. The method of claim 7, wherein the transmitting the collected data in real-time comprises:
and transmitting the acquired data in real time by adopting a mode of sending a virtual signal.
9. The method according to claim 6, wherein the step S2 includes:
step S21, setting a time period;
step S22, storing the data in the time period;
and step S23, compressing the stored data in the time period and uploading the compressed data.
10. The method according to claim 9, wherein the step S21 includes:
the time period is set to 10S.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510481516.7A CN105471958A (en) | 2015-08-03 | 2015-08-03 | Data processing system and method of internet of things |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510481516.7A CN105471958A (en) | 2015-08-03 | 2015-08-03 | Data processing system and method of internet of things |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105471958A true CN105471958A (en) | 2016-04-06 |
Family
ID=55609222
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510481516.7A Pending CN105471958A (en) | 2015-08-03 | 2015-08-03 | Data processing system and method of internet of things |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105471958A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107360216A (en) * | 2017-06-21 | 2017-11-17 | 深圳市盛路物联通讯技术有限公司 | A kind of Internet of Things data method for controlling reporting and system |
| CN109068183A (en) * | 2018-08-08 | 2018-12-21 | 湖南快乐阳光互动娱乐传媒有限公司 | Live broadcast bullet screen distribution method and device |
| CN110786021A (en) * | 2017-06-23 | 2020-02-11 | 代傲表计简易股份公司 | Method and system for collecting sensor data |
| CN111006976A (en) * | 2018-10-08 | 2020-04-14 | 华北电力大学（保定） | Haze monitoring system and monitoring method |
| CN113342273A (en) * | 2021-06-09 | 2021-09-03 | 北京九贺科技有限公司 | Cache-based big data storage method and system |
| CN113423080A (en) * | 2021-06-18 | 2021-09-21 | 北京金和网络股份有限公司 | Internet of things gateway data uploading method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102158691A (en) * | 2011-05-31 | 2011-08-17 | 中国电子科技集团公司第三十研究所 | System and method for monitoring by combining radio frequency identification with video shooting |
| CN102413311A (en) * | 2010-09-20 | 2012-04-11 | 北京中星微电子有限公司 | Mobile monitoring device, system and monitoring method thereof |
| CN103701630A (en) * | 2013-12-03 | 2014-04-02 | 力合科技（湖南）股份有限公司 | Data processing method and device for data monitoring |
| CN104168148A (en) * | 2014-05-04 | 2014-11-26 | 杭州立地信息技术有限公司 | Multi-interface data collecting device and method |
-
2015
- 2015-08-03 CN CN201510481516.7A patent/CN105471958A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102413311A (en) * | 2010-09-20 | 2012-04-11 | 北京中星微电子有限公司 | Mobile monitoring device, system and monitoring method thereof |
| CN102158691A (en) * | 2011-05-31 | 2011-08-17 | 中国电子科技集团公司第三十研究所 | System and method for monitoring by combining radio frequency identification with video shooting |
| CN103701630A (en) * | 2013-12-03 | 2014-04-02 | 力合科技（湖南）股份有限公司 | Data processing method and device for data monitoring |
| CN104168148A (en) * | 2014-05-04 | 2014-11-26 | 杭州立地信息技术有限公司 | Multi-interface data collecting device and method |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107360216A (en) * | 2017-06-21 | 2017-11-17 | 深圳市盛路物联通讯技术有限公司 | A kind of Internet of Things data method for controlling reporting and system |
| CN110786021A (en) * | 2017-06-23 | 2020-02-11 | 代傲表计简易股份公司 | Method and system for collecting sensor data |
| US11399223B2 (en) | 2017-06-23 | 2022-07-26 | Diehl Metering S.A.S. | Method and system for collecting data supplied by sensors |
| CN110786021B (en) * | 2017-06-23 | 2022-08-12 | 代傲表计简易股份公司 | Method and system for collecting sensor data |
| CN109068183A (en) * | 2018-08-08 | 2018-12-21 | 湖南快乐阳光互动娱乐传媒有限公司 | Live broadcast bullet screen distribution method and device |
| CN111006976A (en) * | 2018-10-08 | 2020-04-14 | 华北电力大学（保定） | Haze monitoring system and monitoring method |
| CN113342273A (en) * | 2021-06-09 | 2021-09-03 | 北京九贺科技有限公司 | Cache-based big data storage method and system |
| CN113342273B (en) * | 2021-06-09 | 2022-09-09 | 重庆云微软件有限公司 | Cache-based big data storage method and system |
| CN113423080A (en) * | 2021-06-18 | 2021-09-21 | 北京金和网络股份有限公司 | Internet of things gateway data uploading method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105471958A (en) | Data processing system and method of internet of things | |
| CN106331613B (en) | Communication method and system based on unmanned aerial vehicle | |
| CN103647954A (en) | Mobile video monitoring system based on 3G channel and intelligent terminal | |
| CN204887278U (en) | Unmanned aerial vehicle is in air from network deployment image transmission system | |
| CN108769576B (en) | Intelligent video processing method and system | |
| CN104811665A (en) | System and method for realizing field real-time monitoring of electric power by using intelligent wearable device | |
| CN102821271A (en) | 3G remote video transmission system | |
| CN202940916U (en) | City video monitoring system | |
| CN104639393A (en) | Passive optical network flow whole-network acquisition device and passive optical network flow whole-network acquisition method | |
| CN103701630A (en) | Data processing method and device for data monitoring | |
| CN105072377A (en) | Law enforcement record system and control method thereof | |
| CN201789586U (en) | Wireless video conference equipment based on different types of 3G network | |
| CN106331603A (en) | Video monitoring method, apparatus, system and server | |
| CN203482319U (en) | Video monitoring and analysis system for power station workshops | |
| CN102300081A (en) | Audio and video monitoring equipment, system and method thereof | |
| CN202190350U (en) | Remote digital video monitoring system | |
| US8629907B2 (en) | Data acquisition system and transmission control device | |
| CN102625074A (en) | Method for transmitting intelligent analyzed data by aid of HD-SDI (high-definition serial digital interface) camera | |
| CN105471850A (en) | Video uploading method and video uploading device | |
| CN104093006A (en) | System and method for carrying out remote image capturing in double-current mode | |
| KR20140146429A (en) | CCTV image aquisition and management system and method using smart devices | |
| CN104349128A (en) | Video and audio coding equipment and system and storage extraction method thereof | |
| CN105006084A (en) | Cell phone video and infrared linking alarm system | |
| CN105262824A (en) | Automobile monitoring method, device and system | |
| CN105338290A (en) | Code stream synthetic method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination |