LU502742B1 - Big data processing and analysis system for internet of things - Google Patents

Big data processing and analysis system for internet of things Download PDF

Info

Publication number
LU502742B1
LU502742B1 LU502742A LU502742A LU502742B1 LU 502742 B1 LU502742 B1 LU 502742B1 LU 502742 A LU502742 A LU 502742A LU 502742 A LU502742 A LU 502742A LU 502742 B1 LU502742 B1 LU 502742B1
Authority
LU
Luxembourg
Prior art keywords
data
internet
things
priority
difference
Prior art date
Application number
LU502742A
Other languages
German (de)
Inventor
Litao Zhang
Original Assignee
Univ Zhengzhou Aeronautics
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Univ Zhengzhou Aeronautics filed Critical Univ Zhengzhou Aeronautics
Priority to LU502742A priority Critical patent/LU502742B1/en
Application granted granted Critical
Publication of LU502742B1 publication Critical patent/LU502742B1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/04—Processing captured monitoring data, e.g. for logfile generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/40—Data acquisition and logging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y20/00—Information sensed or collected by the things
    • G16Y20/10—Information sensed or collected by the things relating to the environment, e.g. temperature; relating to location
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/10—Detection; Monitoring
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/70—Services for machine-to-machine communication [M2M] or machine type communication [MTC]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models

Abstract

The invention provides a big data processing and analysis system for Internet of Things, including Internet of Things data acquisition module, Internet of Things data transmission module and Internet of Things data analysis module, wherein the Internet of Things data transmission module includes Internet of Things wireless data transmission module and Internet of Things ad hoc network data transmission module.

Description

DESCRIPTION LU502742
BIG DATA PROCESSING AND ANALYSIS SYSTEM FOR INTERNET OF THINGS
TECHNICAL FIELD
The invention relates to the technical field of the Internet of Things, in particular to a big data processing and analysis system for Internet of Things.
BACKGROUND
At present, the Internet of Things industry has already penetrated into a wide range of fields, such as industrial production, smart homes, space development, ocean exploration, environmental protection, resource investigation, medical diagnosis, bioengineering, and even cultural relics protection, etc. The Internet of Things industry has entered a period of rapid development.
The data analysis systems in the common Internet of Things collect and analyze data directly, and the amount of data processed by the Internet of Things presents a massive feature, so it is difficult to analyze and process abnormal data in time. Therefore, how to efficiently process these data, obtain useful information from them, and then provide intelligent decision-making is the key problem faced by the Internet of Things.
Therefore, the present invention provides a new solution to solve this problem.
SUMMARY
In view of the above situation, in order to overcome the defects of the prior art, the purpose of the present invention is to provide a big data processing and analysis system for Internet of
Things, in order to effectively solve the problem that the amount of data processed by the existing data analysis system in the Internet of Things presents massive characteristics, and it is difficult to analyze and process abnormal data in time.
The technical scheme is as follows: a big data processing and analysis system for Internet of
Things, including Internet of Things data acquisition module, Internet of Things data transmission module and Internet of Things data analysis module, wherein the Internet of Things data transmission module includes Internet of Things wireless data transmission module and Internet of Things ad hoc network data transmission module, characterized in that the Internet of Things data acquisition module is connected to a data preprocessor through real-time data acquired by a sensing device, the data preprocessor extracts the data information in the Internet of Things history database module and the received real-time data for feature-level fusion processing and analysis,
and constructs a causal prediction model to obtain difference data, Internet of Things data analysi$)502742 module adopts preemptive priority scheduling algorithm to obtain real-time data transmitted by
Internet of Things wireless data transmission module and difference data transmitted by Internet of Things ad hoc network data transmission module;
The data information in the data preprocessor extract Internet of Things history database module and the received real-time data are processed and analyzed by feature level fusion, and a causal prediction model is constructed to obtain difference data, data collected by four sensing devices, namely, temperature and humidity sensor, light sensor, smoke sensor and infrared sensor, in the intelligent home of the Internet of Things, the specific steps are as follows:
S1, a data preprocessor preprocesses data information in a history database module of the
Internet of Things, and extracts temperature and humidity data A, illumination data B, smoke data
C and access control data D from the database;
S2, the data preprocessor receives the real-time data collected by the sensing device, including temperature and humidity data a, illumination data b, smoke data c and access control data d;
S3, the data preprocessor performs feature-level fusion processing on the data in steps S1 and
S2, and performs pairwise data fusion on eight data signals according to the purposes of the data, and the fused data are data temperature and humidity data (A, a), illumination data (B, b), smoke data (C, c) and access control data (D, d); and
S4, the data preprocessor calculates the absolute values of data deviation values A-a, B-b, C- c and D-d difference according to the data signals after feature level fusion processing, and defines them as temperature and humidity data deviation value L, illumination data deviation value M, smoke data deviation value N and access control data deviation value U respectively;
SS, the data preprocessor constructs a causal prediction model according to the data deviation values L, M, N, U and the set parameters, and when it is determined that the data has a trend of difference, it is transmitted to the data analysis module of the Internet of Things through the data transmission module of the Internet of Things ad hoc network.
Optionally, the Internet of Things data analysis module adopts the preemptive priority scheduling algorithm to obtain the real-time data transmitted by the Internet of Things wireless data transmission module and the difference data transmitted by the Internet of Things ad hb&/502742 network data transmission module, and the specific steps are as follows: step 1, setting the priority of the task at the first level according to the remaining time of the received data, and the shorter the remaining time, the higher the priority; step 2, on the basis of step 1, setting the priority in two levels according to the data type, respectively setting the priority of real-time data as LY and the priority of difference data as HY, and further setting the priority of data deviation values L, M, N and U in difference data; step 3, if the remaining time of the data with the highest priority in the current priority queue is longer than the running time of the data, the data is called for analysis, otherwise, it will not be analyzed; and step 4, when receiving the difference data, the interrupt preemption mode is adopted, and the difference data is analyzed and processed immediately without waiting for the current execution data to end. During its execution, if another process with higher priority appears, the process scheduler will immediately stop the execution of the current process and redistribute it to the newly arrived process as the process with the highest priority.
The method is ingenious in concept, the data information in the data preprocessor extract
Internet of Things history database module and the received real-time data of the Internet of Things data acquisition module are processed and analyzed by feature level fusion, and a causal prediction model is constructed to obtain difference data. Internet of Things data analysis module adopts preemptive priority scheduling algorithm to obtain real-time data transmitted by Internet of Things wireless data transmission module and difference data transmitted by Internet of Things ad hoc network data transmission module, so as to improve the reliability and stability of data, improve data transmission rate and reduce transmission delay, and make the difference data, that is, abnormal data, be analyzed and processed in time.
BRIEF DESCRIPTION OF THE FIGURES
Fig. 1 is an overall module diagram of a big data processing and analysis system for Internet of Things according to the present invention.
Fig. 2 is a data preprocessing flow chart of a big data processing and analysis system for
Internet of Things.
DESCRIPTION OF THE INVENTION LU502742
The foregoing and other technical contents, features and effects of the present invention will be clearly presented in the following detailed description of the embodiment with reference to Figs. 1 to 2. The structural contents mentioned in the following examples all refer to the drawings of the specification.
In order to verify the feasibility of this method and the effect of actual use, the following examples are given to verify this method.
Embodiment 1 Big data processing and analysis system for Internet of Things
The Internet of Things data acquisition module is connected to a data preprocessor through real-time data acquired by a sensing device, the data preprocessor extracts the data information in the Internet of Things history database module and the received real-time data for feature-level fusion processing and analysis, and constructs a causal prediction model to obtain difference data,
Internet of Things data analysis module adopts preemptive priority scheduling algorithm to obtain real-time data transmitted by Internet of Things wireless data transmission module and difference data transmitted by Internet of Things ad hoc network data transmission module; in this way, the reliability and stability of data can be improved, the data transmission rate can be improved, the transmission delay can be reduced, and the difference data, that is, the abnormal data, can be analyzed and processed in time.
The data information in the data preprocessor extract Internet of Things history database module and the received real-time data are processed and analyzed by feature level fusion, and a causal prediction model is constructed to obtain difference data; taking the data collected by the following four sensing devices, such as temperature and humidity sensor, light sensor, smoke sensor and infrared sensor, as examples, the detailed steps are as follows:
S1, a data preprocessor preprocesses data information in a history database module of the
Internet of Things, and extracts temperature and humidity data A, illumination data B, smoke data
C and access control data D from the database;
S2, the data preprocessor receives the real-time data collected by the sensing device, including temperature and humidity data a, illumination data b, smoke data c and access control data d;
S3, the data preprocessor performs feature-level fusion processing on the data in steps S1 addJ502742
S2, and performs pairwise data fusion on eight data signals according to the purposes of the data, and the fused data are data temperature and humidity data (A, a), illumination data (B, b), smoke data (C, c) and access control data (D, d);
S4, the data preprocessor calculates the absolute values of data deviation values A-a, B-b, C- c and D-d difference according to the data signals after feature level fusion processing, and defines them as temperature and humidity data deviation value L, illumination data deviation value M, smoke data deviation value N and access control data deviation value U respectively;
SS, the data preprocessor constructs a causal prediction model according to the data deviation values L, M, N, U and the set parameters (By setting the weight of the data deviation value, the weighted average method is used for analysis, and whether it is abnormal or not is preliminarily analyzed. If it is abnormal, it is predicted by several groups of data values in unit time, which is the prior art and will not be described in detail here.). When it is determined that the data has a trend of difference, it is transmitted to the data analysis module of the Internet of Things through the data transmission module of the Internet of Things ad hoc network.
Embodiment 2
Based on the embodiment 1, the Internet of Things data analysis module adopts the preemptive priority scheduling algorithm to obtain the real-time data transmitted by the Internet of Things wireless data transmission module and the difference data transmitted by the Internet of
Things ad hoc network data transmission module, and the specific steps are as follows: step 1, setting the priority of the task at the first level according to the remaining time of the received data, and the shorter the remaining time, the higher the priority, that is, using the first-in- first-out data structure; step 2, on the basis of step 1, setting the priority in two levels according to the data type, respectively setting the priority of real-time data as LY, the priority of difference data as HY, the priority of temperature and humidity data deviation value L, illumination data deviation value M, smoke data deviation value N and access control data deviation value U in the difference data; step 3, if the remaining time of the data with the highest priority in the current priority queue is longer than the running time of the data, the data is called for analysis, otherwise, it will not be analyzed; and step 4, when receiving the difference data, the interrupt preemption mode is adopted, and th&J502742 difference data is analyzed and processed immediately without waiting for the current execution data to end. During its execution, if another process with higher priority appears, the process scheduler will immediately stop the execution of the current process (the original process with the highest priority) and redistribute it to the newly arrived process as the process with the highest priority.
When the device is used, the data acquisition module of the Internet of Things is connected to the data preprocessor through the real-time data acquired by the sensing device, and the data preprocessor extracts the data information (temperature and humidity data A, illumination data B, smoke data C, access control data D) in the Internet of Things history database module and the received real-time data (temperature and humidity data a, illumination data b, smoke data c, access control data d); making pairwise data fusion of eight data signals according to the purpose of data.
The fused data are data temperature and humidity data (A, a), light data (B, b), smoke data (C, c) and access control data (D, d), and calculate the absolute values of data deviation values A-a, B-b,
C-c and D-d respectively. They are defined as temperature and humidity data deviation value L, illumination data deviation value M, smoke data deviation value N, and entrance guard data deviation value U, respectively. The data preprocessor constructs a causal relationship prediction model according to the data deviation values L, M, N, and U and the set parameters. When it is judged that there is a trend of difference in data, the data is transmitted to the data analysis module of the Internet of Things through the data transmission module of the Internet of Things ad hoc network, and the feature level fusion processing analysis is carried out, and the causal prediction model is constructed to obtain the difference data. Internet of Things data analysis module adopts preemptive priority scheduling algorithm to obtain real-time data transmitted by Internet of Things wireless data transmission module and difference data transmitted by Internet of Things ad hoc network data transmission module, so as to improve data reliability, stability and data transmission rate.
The above is a further detailed description of the present invention combined with specific embodiments, and it cannot be considered that the specific embodiments of the present invention are only limited to this; For those skilled in the related technical fields to which the present invention belongs, on the premise of the technical scheme of the present invention, the expansion and the replacement of operation methods and data should all fall within the protection scope bË502742 the present invention.

Claims (2)

CLAIMS LU502742
1. À big data processing and analysis system for Internet of Things, including an Internet of Things data acquisition module, an Internet of Things data transmission module and an Internet of Things data analysis module, wherein the Internet of Things data transmission module includes an Internet of Things wireless data transmission module and an Internet of Things ad hoc network data transmission module, characterized in that the Internet of Things data acquisition module is connected to a data preprocessor through real-time data acquired by a sensing device, the data preprocessor extracts the data information in a history database module of the Internet of Things and the received real-time data for feature-level fusion processing and analysis, and constructs a causal prediction model to obtain difference data; the Internet of Things data analysis module adopts a preemptive priority scheduling algorithm to obtain real-time data transmitted by Internet of Things wireless data transmission module and difference data transmitted by Internet of Things ad hoc network data transmission module; the data preprocessor extracts the data information in a history database module of the Internet of Things and the received real-time data for feature-level fusion processing and analysis, and constructs a causal prediction model to obtain difference data, data collected by four sensing devices, namely, temperature and humidity sensor, light sensor, smoke sensor and infrared sensor, in the intelligent home of the Internet of Things, the steps are as follows: S1, the data preprocessor preprocesses data information in a history database module of the Internet of Things, and extracts temperature and humidity data A, illumination data B, smoke data C and access control data D from the database; S2, the data preprocessor receives the real-time data collected by the sensing device, including temperature and humidity data a, illumination data b, smoke data c and access control data d; S3, the data preprocessor performs feature-level fusion processing on the data in S1 and S2, and performs pairwise data fusion on eight data signals according to the purposes of the data, and the fused data are temperature and humidity data (A, a), illumination data (B, b), smoke data (C, c) and access control data (D, d); S4, the data preprocessor calculates the absolute values of data deviation values A-a, B-b, C- c and D-d difference according to the data signals after feature level fusion processing, and defines them as temperature and humidity data deviation value L, illumination data deviation value MU502742 smoke data deviation value N and access control data deviation value U respectively; SS, the data preprocessor constructs a causal prediction model according to the data deviation values L, M, N, U and the set parameters, and when it is determined that the data has a trend of difference, it is transmitted to the data analysis module of the Internet of Things through the Internet of Things ad hoc network data transmission module.
2. The big data processing and analysis system for Internet of Things according to claim 1, characterized in that the Internet of Things data analysis module adopts the preemptive priority scheduling algorithm to obtain the real-time data transmitted by the Internet of Things wireless data transmission module and the difference data transmitted by the Internet of Things ad hoc network data transmission module, and the steps are as follows: step 1, setting the priority of the task at the first level according to the remaining time of the received data, wherein the shorter the remaining time, the higher the priority; step 2, on the basis of step 1, setting the priority in two levels according to the data type, setting the priority of real-time data as LY and the priority of difference data as HY, and further setting the priority of data deviation values L, M, N and U in difference data; step 3, if the remaining time of the data with the highest priority in the current priority queue is longer than the running time of the data, the data are called for analysis, otherwise, they not be analyzed; and step 4, when receiving the difference data, the interrupt preemptive mode 1s adopted, and the difference data is analyzed and processed immediately without waiting for the current execution data to end; during its execution, if another process with higher priority appears, the process scheduler immediately stops the execution of the current process and redistribute to the newly arrived process as the process with the highest priority.
LU502742A 2022-09-01 2022-09-01 Big data processing and analysis system for internet of things LU502742B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
LU502742A LU502742B1 (en) 2022-09-01 2022-09-01 Big data processing and analysis system for internet of things

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
LU502742A LU502742B1 (en) 2022-09-01 2022-09-01 Big data processing and analysis system for internet of things

Publications (1)

Publication Number Publication Date
LU502742B1 true LU502742B1 (en) 2023-03-01

Family

ID=85330685

Family Applications (1)

Application Number Title Priority Date Filing Date
LU502742A LU502742B1 (en) 2022-09-01 2022-09-01 Big data processing and analysis system for internet of things

Country Status (1)

Country Link
LU (1) LU502742B1 (en)

Similar Documents

Publication Publication Date Title
Bhuiyan et al. A deep learning based assistive system to classify COVID-19 face mask for human safety with YOLOv3
KR100976443B1 (en) Home-network error prediction system and home-network fault estimation method
CN108259607B (en) The processing analysis system of internet of things oriented big data
CN109905489B (en) Multi-sensor data relevance processing method and system based on data fusion algorithm
US20180224809A1 (en) Event analyzing device, event analyzing system, event analyzing method, and non-transitory computer readable storage medium
CN109255523A (en) Analysis indexes computing platform based on KKS coding rule and big data framework
CN111143438A (en) Workshop field data real-time monitoring and anomaly detection method based on stream processing
CN110287827B (en) Bridge strain data outlier identification method based on data correlation
CN116489152B (en) Linkage control method and device for Internet of things equipment, electronic equipment and medium
CN107944005B (en) Data display method and device
LU502742B1 (en) Big data processing and analysis system for internet of things
CN108540304B (en) Remote acquisition monitoring system and method based on industrial big data
CN116980958B (en) Radio equipment electric fault monitoring method and system based on data identification
CN106774054A (en) GIS device analysis system and method based on the identification of complicated unstructured data
Li et al. AIoT platform design based on front and rear end separation architecture for smart agricultural
CN113111585A (en) Intelligent cabinet fault prediction method and system and intelligent cabinet
CN113092083A (en) Machine pump fault diagnosis method and device based on fractal dimension and neural network
CN116522096A (en) Three-dimensional digital twin content intelligent manufacturing method based on motion capture
US11947328B2 (en) Control device, control program, and control system
CN110874506A (en) Low-temperature equipment fault prediction method
CN101571931B (en) Inference method facing to indefinite context of general fit calculation
CN114611587A (en) Knowledge and data hybrid driven industrial data alignment method
CN114429148A (en) Power equipment state detection method based on multi-source data fusion
CN103678136B (en) Data race false positive reduction method based on control flow
CN106961477A (en) A kind of collecting method and system

Legal Events

Date Code Title Description
FG Patent granted

Effective date: 20230301