CN104506352B - A kind of method and system of Internet of Things data pretreatment - Google Patents
A kind of method and system of Internet of Things data pretreatment Download PDFInfo
- Publication number
- CN104506352B CN104506352B CN201410808353.4A CN201410808353A CN104506352B CN 104506352 B CN104506352 B CN 104506352B CN 201410808353 A CN201410808353 A CN 201410808353A CN 104506352 B CN104506352 B CN 104506352B
- Authority
- CN
- China
- Prior art keywords
- data
- internet
- event
- things
- illegal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Abstract
The present invention relates to a kind of method and system of Internet of Things data pretreatment, which includes：User right correction verification module, to accessing validity checking, and according to competence profile, assigns access subnet or the corresponding authority of user, refuses illegal access request, ensure the access security of Internet of Things server；Data and event filtering module, according to data filtering configuration file, only retain satisfactory sensing data and event notice, abandon junk data；And data and event validity check module, according to validity configuration file, pretreatment operation as defined in data validity and work is checked item by item；After data prediction, legal data are saved in database.The method of the present invention is simple, can reduce the error rate of system.
Description
Technical field
The present invention relates to a kind of method and system of Internet of Things data pretreatment.
Background technology
Internet of Things is that the equipment of a small range is formed self-organizing network by specific procotol, and the center of Internet of Things is
One aggregation node, the node generally have the concatenation ability with exterior public network (Internet etc.).Internet of Things mainly solves orphan
Standby networking issue is erected, is networked independent of global network, the general proprietary network for forming self-organizing.Internet of Things is if desired
Any wide-area deployment, the mode generally taken are that small range is networked using Internet of Things, then the aggregation node of each Internet of Things subnet
Network with central server, ultimately form the network of a trans-regional deployment.
Internet of Things is fragile network, and Internet of Things subnet is likely to occur failure, and Internet of Things sub-network data is likely to occur transmission
Abnormal, subnet monitoring is difficult, and network structure may change at any time.Central server will handle this data, it is necessary to a set of
Complete data prediction mechanism, to reduce the probability of system fault.
The content of the invention
The object of the present invention is to provide a kind of method of Internet of Things data pretreatment, the error of Internet of things system can be reduced
Rate.
The present invention is realized using following scheme：A kind of method of Internet of Things data pretreatment, it is characterised in that including following
Step：
Step S11：To accessing validity checking, and according to competence profile, assign access subnet or user is corresponding
Authority, refuses illegal access request, ensures the access security of Internet of Things server；
Step S12：Using Internet of Things data filtering rule, sensing data illegal in sensing data is filtered；Using Internet of Things
Net event filtering rule, filters out illegal event message；Using user right and the business rule of correspondence, illegal user is filtered
Inquiry request；
Step S13：According to Internet of Things data and event type, the form of verification data or event, for cannot positive general knowledge
Other form, notifies system manager and writing system Log, data abandon or back-up processing；It can be identified for form, but
It is the organizational form of data and the difference of Internet of Things server requirement, it is necessary to do the format adaptation of an Internet of Things data and event
Work；The Internet of Things valid data after verification and event are finally submitted to database.
In an embodiment of the present invention, the different data source includes the sensing data of Internet of Things subnet, for thing
The event notice and booking reader's inquiry request of networking subnet；Sensing data for Internet of Things subnet is, it is necessary to check sub-network
Whether mandate accesses, and with the authority for uploading sensing data, assigns and authorize network sensing data upload permissions；For Internet of Things
, it is necessary to check whether sub-network authorizes access, and have event notice reporting authority, imparting authorizes the event notice of net net
Network event notifications report authority；For booking reader's inquiry request, it is necessary to which subscriber authentication, rightness of business are verified, only
Receive the inquiry request in the mandate business of validated user, refuse unauthorized service inquiry request.
In an embodiment of the present invention, the illegal sensing data includes data transfer distortion, data formatting error
With value data distortion.
It is a further object of the present invention to provide a kind of Internet of Things data pretreatment system, it is characterised in that including：User weighs
Correction verification module is limited, subnet or the corresponding authority of user are accessed to accessing validity checking, and according to competence profile, imparting,
Refuse illegal access request, ensure the access security of Internet of Things server；Data and event filtering module, according to data mistake
Configuration file is filtered, only retains satisfactory sensing data and event notice, abandons junk data；And data and event it is effective
Property check module, according to validity configuration file, check data validity item by item and pretreatment operation as defined in making；Data are located in advance
After reason, legal data are saved in database.
Present inventive concept is simple, and the compounding practice of three modules can be achieved with to access data from verification to wrong data
Processing, submission of valid data etc., can farthest reduce the error rate of system.
Brief description of the drawings
Fig. 1 is the general frame of data prediction of the present invention.
Fig. 2 is user right correction verification module block diagram of the present invention.
Fig. 3 is data and event filtering module frame chart.
Fig. 4 is data and event validity correction verification module block diagram.
Embodiment
It is understandable to enable the above objects, features and advantages of the present invention to become apparent, below in conjunction with the accompanying drawings to the present invention
Embodiment be described in detail.
Detail is elaborated in the following description in order to fully understand the present invention.But the present invention can with it is a variety of not
Other manner described here is same as to implement, those skilled in the art can do class in the case of without prejudice to intension of the present invention
Like popularization.Therefore the limitation of the invention from following public embodiment.
Internet of Things data pre-processing engine can be deployed in Internet of Things data central server, may be deployed in Internet of Things net
In artis (correspondence with foreign country node).The object of Internet of Things data pretreatment includes three kinds：The sensing data of Internet of Things subnet；Thing
The event notice of subnet of networking and the inquiry operation (query event) of booking reader.It is pre- for above-mentioned three kinds of data types, data
Processing engine is pre-processed by following processes：
Step S11：To accessing validity checking, and according to competence profile, assign access subnet or user is corresponding
Authority, refuses illegal access request, ensures the access security of Internet of Things server；
Step S12：Using Internet of Things data filtering rule, sensing data illegal in sensing data is filtered；Using Internet of Things
Net event filtering rule, filters out illegal event message；Using user right and the business rule of correspondence, illegal user is filtered
Inquiry request；
Step S13：According to Internet of Things data and event type, the form of verification data or event, for cannot positive general knowledge
Other form, notifies system manager and writing system Log, data abandon or back-up processing；It can be identified for form, but
It is the organizational form of data and the difference of Internet of Things server requirement, it is necessary to do the format adaptation of an Internet of Things data and event
Work；The Internet of Things valid data after verification and event are finally submitted to database.
Specifically, referring to Fig. 1, the present embodiment provides a kind of Internet of Things data pretreatment system, which includes：(1)
User right correction verification module, effect user validation and rightness of business, refuse illegal request of data；(2) data and event
Filtering module, according to data filtering configuration file, only retains satisfactory sensing data and event notice, abandons rubbish number
According to；(3) data and event validity check module, according to validity configuration file, check data validity item by item and provide
Pretreatment operation (value range standardization etc.).After data prediction, legal data are saved in database.
User right correction verification module major function is access validity checking, and according to competence profile, assigns access
Subnet or the corresponding authority of user, refuse illegal access request, ensure the access security of Internet of Things server.For difference
Data source, authorization check takes different strategies.Whether sensing data for Internet of Things subnet is, it is necessary to check sub-network
Access is authorized, and with the authority for uploading sensing data, assigns and authorizes network sensing data upload permissions；For Internet of Things net
For the event notice of net, it is necessary to check whether sub-network authorizes access, and have event notice reporting authority, imparting authorizes network
Event notice reporting authority；For booking reader's inquiry request, it is necessary to which subscriber authentication, rightness of business verification, only receive
Inquiry request in the mandate business of validated user, refuses unauthorized service inquiry request.The block diagram of user right correction verification module
As shown in Figure 2.
Data and event filtering module, for tentatively filtering invalid data (data transfer distortion, data type mistake, number
According to format error, event type mistake, event format mistake, value data distortion, false data), reduce at follow-up data
Difficulty and error probability are managed, improves the stability of Internet of Things server.Using Internet of Things data filtering rule, sensing data is filtered
In illegal sensing data (data transfer distortion, data formatting error, value data distortion etc.)；Using Internet of Things event filtering
Rule, filters out illegal event message；Using user right and the business rule of correspondence, illegal user's inquiry request is filtered.
Internet of Things sensing data data and event filtering module frame chart are as shown in Figure 3.
Internet of Things authorization check module and data filtering module have been able to prevent most unauthorized access, but can not hinder
Only in Lawful access distortion data.The data of distortion or incomplete data can cause data storage and data to analyze mistake.
Internet of Things data and the effect of event validity correction verification module are exactly the distortion data corrected in Lawful access.Comprise the following steps that：
According to Internet of Things data and event type, the form of verification data or event, for the form that cannot normally identify, notifies system
Administrator and writing system Log, data abandon or back-up processing.It can be identified for form, but the organizational form of data
Difference with Internet of Things server requirement is, it is necessary to which the format adaptation for doing an Internet of Things data and event works.Data validity
The final step of verification is to be submitted to external module.Data and event validity correction verification module, as shown in Figure 4.
Although the present invention is disclosed as above with preferred embodiment, it is not for limiting the present invention, any this area
Technical staff without departing from the spirit and scope of the present invention, may be by the methods and technical content of the disclosure above to this hair
Bright technical solution makes possible variation and modification, therefore, every content without departing from technical solution of the present invention, according to the present invention
Any simple modifications, equivalents, and modifications made to above example of technical spirit, belong to technical solution of the present invention
Protection domain.The foregoing is merely presently preferred embodiments of the present invention, all impartial changes done according to scope of the present invention patent
Change and modify, should all belong to the covering scope of the present invention.
Claims (2)
- A kind of 1. method of Internet of Things data pretreatment, it is characterised in that comprise the following steps：Step S11：Subnet or the corresponding authority of user are accessed to accessing validity checking, and according to competence profile, imparting, Refuse illegal access request, ensure the access security of Internet of Things server；Internet of Things data includes the sensing data of Internet of Things subnet, is inquired about for the event notice of Internet of Things subnet and booking reader Request；Sensing data for Internet of Things subnet is, it is necessary to check whether sub-network authorizes access, and have and upload sensing data Authority, assign authorize network sensing data upload permissions；Notified for the event of Internet of Things subnet, it is necessary to check that sub-network is It is no to authorize access, and there is event notice reporting authority, assign and authorize network event notifications to report authority；For booking reader Inquiry request is, it is necessary to subscriber authentication, and rightness of business verification, the inquiry only received in the mandate business of validated user please Ask, refuse unauthorized service inquiry request；Step S12：Using Internet of Things data filtering rule, sensing data illegal in sensing data is filtered；Using Internet of Things thing Part filtering rule, filters out illegal event message；Using user right and the business rule of correspondence, illegal user's inquiry is filtered Request；The illegal sensing data includes data transfer distortion, data formatting error and value data distortion；It is described illegal Event message include event type mistake, event format mistake；Step S13：For the distortion data in Lawful access, according to Internet of Things data and event type, verification data or event Form, for the form that cannot normally identify, notify system manager and writing system Log, data abandon or backup at Reason；It can be identified for form, but the difference that the organizational form of data is required with Internet of Things server is, it is necessary to do an Internet of Things The format adaptation of network data and event work i.e. correction distortion data；Finally the Internet of Things valid data after verification and event are carried It is sent to database.
- A kind of 2. Internet of Things data pretreatment system, it is characterised in that including：User right correction verification module, to accessing validity checking, and according to competence profile, assigns access subnet or user's phase The authority answered, refuses illegal access request, ensures the access security of Internet of Things server；Data and event filtering module, according to data filtering configuration file, only retain satisfactory sensing data and event are led to Know, abandon junk data；The data and event filtering module use Internet of Things data filtering rule, filter non-in sensing data The sensing data of method；Using Internet of Things event filtering rule, illegal event message is filtered out；Using user right and business pair Should be regular, filter illegal user's inquiry request；Data and event validity check module, according to validity configuration file, check item by item as defined in data validity and work Pretreatment operation；After data prediction, legal data are saved in database；For the distortion data in Lawful access, The data and event validity check module according to Internet of Things data and event type, the form of verification data or event, right In the form that cannot normally identify, system manager and writing system Log are notified, data abandon or back-up processing；For lattice Formula can identify, but the difference of the organizational form of data and Internet of Things server requirement, it is necessary to do an Internet of Things data and The format adaptation work of event i.e. correction distortion data；The Internet of Things valid data after verification and event are finally submitted to data Storehouse；Internet of Things data includes the sensing data of Internet of Things subnet, is inquired about for the event notice of Internet of Things subnet and booking reader Request；Sensing data for Internet of Things subnet is, it is necessary to check whether sub-network authorizes access, and have and upload sensing data Authority, assign authorize network sensing data upload permissions；Notified for the event of Internet of Things subnet, it is necessary to check that sub-network is It is no to authorize access, and there is event notice reporting authority, assign and authorize network event notifications to report authority；For booking reader Inquiry request is, it is necessary to subscriber authentication, and rightness of business verification, the inquiry only received in the mandate business of validated user please Ask, refuse unauthorized service inquiry request；The illegal sensing data includes data transfer distortion, data formatting error and value data distortion；It is described illegal Event message include event type mistake, event format mistake.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410808353.4A CN104506352B (en) | 2014-12-24 | 2014-12-24 | A kind of method and system of Internet of Things data pretreatment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410808353.4A CN104506352B (en) | 2014-12-24 | 2014-12-24 | A kind of method and system of Internet of Things data pretreatment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104506352A CN104506352A (en) | 2015-04-08 |
| CN104506352B true CN104506352B (en) | 2018-04-20 |
Family
ID=52948067
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410808353.4A Expired - Fee Related CN104506352B (en) | 2014-12-24 | 2014-12-24 | A kind of method and system of Internet of Things data pretreatment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104506352B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106454836B (en) * | 2015-08-06 | 2021-12-31 | 中兴通讯股份有限公司 | Method and device for enhancing use safety of equipment certificate |
| CN110800261B (en) * | 2017-06-27 | 2022-01-11 | 亚马逊科技公司 | Model and filter deployment across IOT networks |
| US11350360B2 (en) | 2017-06-27 | 2022-05-31 | Amazon Technologies, Inc. | Generating adaptive models for IoT networks |
| US10554382B2 (en) | 2017-06-27 | 2020-02-04 | Amazon Technologies, Inc. | Secure models for IoT devices |
| CN107360240A (en) * | 2017-07-20 | 2017-11-17 | 广东小天才科技有限公司 | A kind of data-updating method and system |
| CN109224999A (en) * | 2018-09-29 | 2019-01-18 | 湖北航鹏化学动力科技有限责任公司 | Vibration mixing apparatus control method and system based on Internet of Things |
| CN110113421A (en) * | 2019-05-08 | 2019-08-09 | 西南民族大学 | A kind of big data information processing system based on Internet of Things |
| CN111142493A (en) * | 2020-01-10 | 2020-05-12 | 汉威科技集团股份有限公司 | Heterogeneous data acquisition device and method for industrial safety supervision |
| CN112565233A (en) * | 2020-11-30 | 2021-03-26 | 上海瑞眼科技有限公司 | Data verification method and system for fire-fighting Internet of things access platform |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102195991A (en) * | 2011-06-28 | 2011-09-21 | 辽宁国兴科技有限公司 | Terminal security management and authentication method and system |
| CN103078947A (en) * | 2013-01-11 | 2013-05-01 | 江苏物联网研究发展中心 | Mass data exchange and transaction architecture system of internet of things |
| CN103139058A (en) * | 2013-01-28 | 2013-06-05 | 公安部第一研究所 | Internet of things security access gateway |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013181980A1 (en) * | 2012-06-06 | 2013-12-12 | 深圳光启创新技术有限公司 | Handshake synchronization method and system based on visible light communication |
-
2014
- 2014-12-24 CN CN201410808353.4A patent/CN104506352B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102195991A (en) * | 2011-06-28 | 2011-09-21 | 辽宁国兴科技有限公司 | Terminal security management and authentication method and system |
| CN103078947A (en) * | 2013-01-11 | 2013-05-01 | 江苏物联网研究发展中心 | Mass data exchange and transaction architecture system of internet of things |
| CN103139058A (en) * | 2013-01-28 | 2013-06-05 | 公安部第一研究所 | Internet of things security access gateway |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104506352A (en) | 2015-04-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104506352B (en) | A kind of method and system of Internet of Things data pretreatment | |
| US10367844B2 (en) | Systems and methods of network security and threat management | |
| US11949704B2 (en) | Attribute-based policies for integrity monitoring and network intrusion detection | |
| US8941465B2 (en) | System and method for secure entry using door tokens | |
| US10102700B2 (en) | System and method for entry access control using radio frequency communication | |
| US20130214901A1 (en) | System, station and method for mustering | |
| EP2866411A1 (en) | Method and system for detecting unauthorized access to and use of network resources with targeted analytics | |
| CN107077472A (en) | Distributed processing system(DPS) | |
| KR20190136011A (en) | Core network access provider | |
| CN103929376A (en) | Terminal admission control method based on switch port management | |
| US20120297461A1 (en) | System and method for reducing cyber crime in industrial control systems | |
| US20190052676A1 (en) | Methods and systems for providing security to iot devices operating in an environment | |
| CN103413083A (en) | Security defending system for single host | |
| CN101079749A (en) | Data consistency detection method | |
| CN110708336B (en) | Video terminal authentication method and device, electronic equipment and storage medium | |
| CN107070951A (en) | A kind of intranet security guard system and method | |
| US11102055B2 (en) | Network self-diagnosis control device based on block chain | |
| CN113630409B (en) | Abnormal flow identification method based on DNS analysis flow and IP flow fusion analysis | |
| CN106209799A (en) | A kind of method, system and dynamic firewall realizing dynamic network protection | |
| CN104468818A (en) | Service processing system and method for internet of things | |
| CN104618176A (en) | Website security detection method and device | |
| CN114500088A (en) | Internet of things equipment access authentication method and system based on block chain and edge calculation | |
| CN109428752B (en) | Verification method and device | |
| KR101070522B1 (en) | System and method for monitoring and blocking of spoofing attack | |
| CN108809891B (en) | Server intrusion detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20180420 Termination date: 20201224 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |