WO2020087848A1 - 一种基于Lora和Capsule的实时行为识别系统及其工作方法 - Google Patents

一种基于Lora和Capsule的实时行为识别系统及其工作方法 Download PDF

Info

Publication number
WO2020087848A1
WO2020087848A1 PCT/CN2019/079387 CN2019079387W WO2020087848A1 WO 2020087848 A1 WO2020087848 A1 WO 2020087848A1 CN 2019079387 W CN2019079387 W CN 2019079387W WO 2020087848 A1 WO2020087848 A1 WO 2020087848A1
Authority
WO
WIPO (PCT)
Prior art keywords
behavior information
unit
behavior
information
module
Prior art date
Application number
PCT/CN2019/079387
Other languages
English (en)
French (fr)
Inventor
许宏吉
石磊鑫
陈敏
张贝贝
李梦荷
Original Assignee
山东大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 山东大学 filed Critical 山东大学
Priority to AU2019371325A priority Critical patent/AU2019371325B2/en
Publication of WO2020087848A1 publication Critical patent/WO2020087848A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Definitions

  • the invention relates to a real-time behavior recognition system based on Lora and Capsule and its working method, belonging to the technical field of artificial intelligence and pattern recognition.
  • the behavior recognition system is a system that obtains human behavior information and implements behavior through a reasonable model.
  • advanced technologies such as the Internet of Things, artificial intelligence, big data and cloud computing, more and more researchers have begun to pay attention to the research on the direction of behavior recognition.
  • Behavior recognition has become a hot research direction in the field of artificial intelligence and pattern recognition research, and the development of wearable devices provides a good opportunity for human behavior recognition.
  • human behavior recognition technology has been initially applied in the fields of games, human motion analysis, smart homes, human-computer interaction, and medical diagnosis and monitoring.
  • the behavior information required by the behavior recognition system mainly comes from the following two aspects:
  • the mainstream behavior recognition technology on the market is mainly to recognize behavior information based on vision.
  • both behavior information acquisition methods can achieve real-time recognition of human behavior through corresponding algorithms
  • the method based on visual behavior information acquisition has certain drawbacks.
  • many behavior information cannot be obtained.
  • the current information transmission technology in the behavior recognition system is mainly Bluetooth, ZigBee, WiFi, 3G, 4G.
  • the mainstream transmission technology cannot achieve both transmission distance and power consumption. Therefore, to realize real-time behavior recognition, a transmission technology with low power consumption and long transmission distance is required.
  • Loa is a radio modem technology released by Semtech. This technology has four categories: Lora-WAN protocol, Lora private protocol, CLASS protocol, and data transparent transmission. Compared with other low-power wide-area Internet of Things technologies, this technology has great power consumption, self-organizing network, etc. Advantage.
  • the model of the system for behavior recognition mainly uses algorithms such as machine learning and deep learning.
  • mainstream machine learning mainly includes K nearest neighbor algorithm (KNN), support vector machine (SVM), random forest, neural network, etc .
  • mainstream deep learning algorithms mainly include deep neural network (DNN) and convolutional neural network (CNN) , Recurrent (recursive) neural network (RNN), deep belief network (DBN), etc.
  • Mainstream algorithms are based on behavior information characteristics for behavior recognition, so input behavior information as long as there are certain characteristics will be divided into a certain type of behavior.
  • behavior information 1100 and 0011 respectively stand up and sit down
  • 1010 and 0101 represent fight behavior
  • mainstream algorithms may judge the behavior information as long as there are two 1 and two 0 characteristics of the behavior, so if you enter the behavior information
  • the characteristics of 0101, 1100, 0011 or 1010 will be considered as fighting behavior, but in fact 1100 or 0011 represents the standing up or sitting down behavior.
  • the existing mainstream algorithms only consider whether the behavior information contains certain features, but do not consider the spatial characteristics of these features, so it will cause misjudgment to a certain extent and reduce the accuracy of behavior recognition.
  • Geoffrey Hinton put forward the concept of capsule network in 2017.
  • the capsule network can not only identify whether the behavior information has certain characteristics, but also can identify the spatial relationship between each feature.
  • the capsule network algorithm model adopted by the invention has a relatively high improvement in the accuracy rate of behavior recognition.
  • the present invention proposes a real-time behavior recognition system based on Lora and Capsule.
  • the invention also discloses the working method of the above system.
  • the intelligent hardware is selected and designed through the device quality (QoD) parameters, and the behavior information is collected through the intelligent device. Then, the behavior information is transmitted through the Lora node. At this time, Lora The base station will receive the behavior information. Since the behavior information at this time is the original behavior information, the system needs to perform uncertainty detection on the original behavior information, and process the incomplete or inconsistent information in the behavior information through context prediction filling, zero-filling, deletion, etc. to improve The credibility of behavioral information. Next, standardize and intercept time-based behavior information. The standardization is to improve the accuracy and generalization ability of the model. The behavior information is intercepted through the sliding window mechanism to normalize the model input.
  • QoD device quality
  • the system will adjust the preset threshold and model parameters according to the quality of user experience (QoE) and quality of service (QoS) to improve the stability, accuracy, and applicability of the system.
  • QoE quality of user experience
  • QoS quality of service
  • the present invention provides a feasible solution for real-time behavior recognition based on sensors, and makes up for defects caused by behavior recognition based on video.
  • Multi-source behavior information lays a foundation for improving the accuracy of behavior recognition.
  • the application of deep learning model based on Capsule network in behavior recognition also greatly improves the accuracy of behavior recognition.
  • a real-time behavior recognition system based on Lora and Capsule includes a behavior information physical layer, a behavior information access layer, a behavior information platform layer, and a behavior information application layer connected in sequence;
  • the physical layer of behavior information is used to: perceive, collect, store, and transmit user behavior information from the environment, and the behavior information includes: acceleration, angular velocity, and heart rate;
  • the behavior information access layer is used for networking transmission of the collected behavior information through the low-power wide-area Internet of Things;
  • the behavior information platform layer is used to: conduct uncertainty detection, standardization and time series-based interception of behavior information in sequence, train the labeled behavior information collection under the built network architecture model, and continuously optimize the loss value Find the best model at the same time; Uncertainty detection means: processing the incomplete or inconsistent information in the behavior information through context prediction filling, zero-filling, and deleting methods to improve the credibility of the behavior information; standardization is a numerical type The data is normalized to improve the accuracy and generalization ability of the model; time series-based interception refers to the interception of behavior information through the sliding window mechanism to ensure the normalization of the model input and improve the accuracy of the model;
  • the behavior information application layer is used to adjust the stability and adaptability of the entire real-time behavior recognition system.
  • the present invention proposes a relatively optimized system in four aspects of transmission technology, information processing, behavior recognition and behavior application, which makes up for the shortcomings of poor real-time behavior recognition on the market and the inability to use it in specific areas.
  • the aspect has also been further improved, so that the system has stability.
  • the behavior information physical layer is a behavior information collection module, and the behavior information collection module includes a sensor module and several intelligent hardware modules; the sensor module includes several different types of sensors, and the intelligent hardware module Several different types of sensors are connected respectively, and the intelligent hardware module is used to control the sensors to perceive different types of user behavior information and store the sensed behavior information.
  • the behavior information collection module selects sub-modules and designs smart devices through the QoD parameters, application scenarios, and user requirements of each sub-module.
  • QoD parameters mainly include the sampling frequency, service life, and accuracy of the sensor module.
  • the intelligent hardware module adopts Lora to transmit behavior information.
  • Transmission technology refers to a technology that makes full use of the transmission capabilities of different channels to form a complete transmission system, so that information can be reliably transmitted. With the progress of society and the development of wireless technology, the convenience of wireless transmission is further amplified under the premise that the packet loss rate is not high.
  • the mainstream wireless technologies are mainly WiFi, Bluetooth, ZigBee, 3G, 4G, etc.
  • Each wireless technology is in an indispensable situation in terms of transmission distance and power consumption, but in order to achieve real-time behavior recognition, the transmission of behavior information requires a Wireless technology with low power consumption and long transmission distance.
  • Low power consumption local area network (LPWAN) is the main technology to solve the current situation, so the present invention uses Lora to transmit behavior information.
  • LPWAN low power consumption local area network
  • the behavior information access layer is a behavior information transmission module, and the behavior information transmission module includes a behavior information transmission module and a behavior information reception module; the behavior information transmission module is connected to the intelligent hardware module for Yu sends behavior information to the behavior information receiving module.
  • the information sending module is a Lora node
  • the behavior information receiving module is a Lora base station.
  • the behavior information platform layer is a behavior information preprocessing module
  • the behavior information preprocessing module includes a behavior information detection module, a behavior information uncertainty elimination module, a behavior information processing module, and a network architecture that are sequentially connected Module
  • the behavior information detection module includes an inconsistency detection / quantization unit and an incompleteness detection / quantization unit;
  • the behavior information uncertainty elimination module includes an inconsistency elimination unit and an incompleteness elimination unit;
  • the behavior information processing module includes a behavior information standardization unit and a behavior information sliding window unit connected in sequence;
  • the network architecture module includes a convolution layer unit, a Capsule layer one unit, a Capsule layer two unit, and a fully connected layer unit connected in sequence;
  • the behavior information receiving module that is, the gateway is connected to the behavior information detection module
  • the behavior information received by the behavior information receiving module that is, the original behavior information is input to the behavior information detection module, and the original behavior information is uncertain by the inconsistency detection / quantization unit and the incompleteness detection / quantization unit Detection, the inconsistency detection / quantization unit detects whether different types of behavior information at the same time have objections, and the incompleteness detection / quantization unit detects whether the perceived behavior information at the same time is missing;
  • the incompleteness elimination unit and the inconsistency elimination unit are used to eliminate the uncertainty, and the incompleteness elimination unit passes the loss of the perceived behavioral information at the same time.
  • the deletion method, the zero-filling method, and the context prediction filling method are used for processing.
  • the inconsistency elimination unit processes inconsistent information through voting, hardware QoD optimal principles, DS (Dempster-Shafer) evidence theory, and fuzzy sets.
  • the behavior information standardization unit Enters the behavior information standardization unit; if it is found that there is no uncertainty in the behavior information, directly enter the behavior information standardization unit; through the behavior information standardization unit and the behavior information sliding window unit for processing, the behavior information
  • the standardization unit is processed by a normalization and normalization method to improve recognition accuracy and applicability;
  • the behavior information sliding window unit performs time series-based interception of behavior information by adjusting the size of the sliding window and the sliding manner of the sliding window;
  • the convolutional layer unit extracts features from the behavior information and converts the characteristic scalar to vector
  • the Capsule layer is a unit It is used to convert the input behavior information into behavior information with spatial characteristics
  • the Capsule layer two units process the behavior information through a dynamic routing protocol
  • the fully connected layer unit transforms the behavior information characteristics into ordered one-dimensional characteristics
  • the main work of the network architecture module is to make recognition based on behavioral information.
  • the proposition of machine learning can truly appreciate the power of artificial intelligence.
  • the proposition of deep learning has a considerable aspect in the recognition rate progress.
  • the focus of machine learning models and deep learning models is whether the input information contains some feature values.
  • the network architecture based on Capsule adopted in the present invention not only pays attention to the characteristics of the behavior information, but also adds the spatial relationship of the characteristics of the behavior information to improve the accuracy of behavior recognition.
  • the behavior preprocessing module mainly improves the credibility of behavior information by preprocessing the behavior information. Compared with some systems that directly perform behavior recognition on the original behavior information, the present invention will greatly improve the stability and accuracy after the behavior information preprocessing.
  • the invention mainly performs uncertainty analysis on the original information, and performs corresponding processing on the category and degree of the uncertainty of the behavior information.
  • the present invention provides a normalization method and a normalization method. The behavior information is intercepted based on time series by adjusting the size of the sliding window and the sliding mode of the sliding window.
  • the behavior information application layer includes a behavior information threshold setting module and a behavior application layer adjustment module
  • the behavior application layer adjustment module includes a behavior recognition unit, a user feedback unit, and an error correction unit connected in sequence
  • the behavior information threshold setting module is used to adjust the threshold in the behavior information uncertainty elimination module, so as to determine whether there is uncertainty in the monitoring data, and adjust the uncertainty processing module to select the data processing method;
  • the behavior recognition unit is used to do the current behavior Real-time identification;
  • the user feedback unit adjusts the preset threshold and the parameters of the network architecture module according to different scenarios and user needs, which improves the applicability of the system to a certain extent;
  • the error correction unit continuously adjusts the network architecture module, Keep the network architecture module in an optimal state all the time.
  • the error correction unit adjusts parameters when the recognition error rate is high, and the user feedback unit adjusts system parameters to adapt to different scenarios.
  • Step S01 Sensors perceive behavior information
  • the QoD parameters of the sensor include: sampling frequency, service life, and accuracy. For example: for users who require key monitoring, high sampling frequency can be used. , High-precision sensors for behavior information perception, for ordinary users can use ordinary sampling frequency, common accuracy sensors for behavior information perception; sensors sense different types of user behavior information;
  • Step S02 Design the intelligent hardware module
  • Step S03 Sending behavior information
  • Step S04 Reception of behavior information
  • the Lora base station is used to receive the behavior information; the behavior information sending device uses a Lora node, and the corresponding behavior information is received using the Lora base station.
  • Step S05 Uncertainty detection of behavior information
  • step S05 is executed.
  • step S06 the original behavior information refers to different types of behavior information of the user perceived by the sensor in step S01;
  • Step S06 Elimination of uncertainty in behavior information
  • the incompleteness elimination unit selects different methods to process the behavior information through the threshold of the uncertainty detection of the behavior information.
  • the accuracy of the behavior information is 85% -90%
  • the context prediction filling method is used to process the behavior information.
  • the correct rate of behavior information is 90% -95%
  • the method of supplementing 0 is used for the behavior information
  • the correct rate of behavior information is 95% -100%
  • the method of deletion is used for the behavior information;
  • the inconsistency elimination unit processes inconsistent information, and the processing methods include voting and election, hardware QoD optimal principles, D-S (Dempster-Shafer) evidence theory, and fuzzy sets; improving the credibility of the original behavior information;
  • Step S07 Processing of behavior information
  • Standardize behavior information with high credibility through behavior information standardization unit uses different standardization methods for different types of data, including: one-hot coding (one-hot coding) for data of categorical characteristics ) Standardization; for data with numeric features, normalization is used for standardization; for data with ordered features, ordered numeric encoding is used for standardization; standardization allows the system to have good scalability.
  • the user preset parameters include: the size of the sliding window and the sliding mode of the window, the behavior information after the standardized processing is subjected to sliding window processing through the behavior information sliding window unit, so that the behavior information becomes the input network architecture Information block of the module;
  • Step S08 Behavior information network architecture
  • the parameters set by the user mainly include: the status of the input data, size, volume
  • a series of parameters such as the size and number of multi-layer cores are used to train the labeled behavior information through several iterations.
  • the model parameters and the dynamic routing protocol in the unit of the Capsule layer are continuously optimized by reducing the loss function, and finally recognized High rate network architecture model;
  • Step S09 Recognition of behavior information
  • Step S10 Error detection
  • step S11 If an error is found, perform step S11; otherwise, perform step S12;
  • the error correction unit adjusts the behavior information threshold range and the corresponding parameters of the behavior information processing module; the behavior information threshold range includes the threshold range of uncertainty detection, and the corresponding parameters of the behavior information processing module include the sliding window of the behavior information sliding window unit Size and sliding mode of the window; when there are many recognition errors, the threshold range of behavior information is appropriately increased, and the size of the sliding window and the sliding mode of the window are reduced.
  • Step S12 User feedback detection
  • step S13 is executed.
  • Step S13 User feedback
  • the user feedback unit makes feedback adjustments to the behavior information threshold range and the corresponding parameters of the behavior information processing module.
  • step S08 in the step S08,
  • the network architecture module includes a convolution layer unit, a Capsule layer one unit, a Capsule layer two unit, and a fully connected layer unit connected in sequence;
  • Batch_Size refers to the number of behavior information that runs in the network architecture module at a time
  • Window_Size refers to the length of each input network architecture module
  • X i refers to each piece of information after behavior information undergoes uncertainty, standardization, and sliding window processing based on time series
  • W ij refers to the weight parameter of the convolutional layer unit, and the initial value defaults to generation truncation Normally distributed random numbers
  • b j refers to the offset parameter of the convolutional layer unit, the initial value defaults to 0.0;
  • n the number of convolution kernels
  • Y j is the output of the convolution layer
  • the output information size is: It needs to ensure that the result of the fraction in the previous formula is a positive integer.
  • the output at this time is a vector behavior information, which meets the input requirements of the Capsule network;
  • W jl refers to the weight parameter of a unit in the Capsule layer, and the initial value defaults to generating a random number with truncated normal distribution
  • b l refers to the offset parameter of a unit in the Capsule layer, the initial value is 0.0 by default;
  • the squsah () function is a new nonlinear function, similar to the previous common nonlinear functions such as tanh (), relu (), etc.
  • the squsah () function is a nonlinear process for vector information; other nonlinear functions are mainly Processing of scalar information;
  • the size of the information output after one unit in the Capsule layer is:
  • b ik refers to the dynamic routing weights of the i-th neuron in the first unit of the Capsule layer and the k-th neuron in the second unit of the Capsule layer;
  • b ij refers to the dynamic routing weights of the i-th neuron in the first unit of the Capsule layer and the j-th neuron in the second unit of the Capsule layer;
  • S j refers to the behavior information characteristics output by the second unit of the Capsule layer after the dynamic routing protocol.
  • the size of the information output by the second unit of the Capsule layer is: Batch_Size ⁇ Num_Output ⁇ Vec_Lenv ⁇ 1;
  • the size of the information output after passing through the fully connected layer unit is:
  • Real-time human behavior recognition has high requirements for transmission media and accuracy.
  • the present invention achieves low power consumption and long-distance transmission of behavior information; at the same time, it also has certain advantages in the accuracy of behavior recognition.
  • the user feedback unit (QoE) and error correction unit (QoS) are used to adjust the parameters in the system to improve the system's adaptability and provide users with personalized and intelligent services.
  • the adjustable parameters include: the threshold of the inconsistency detection / quantization unit and the threshold of the incompleteness detection / quantization unit, the standardization method in the behavior information standardization unit and the sliding window size and sliding method in the behavior information sliding window unit, Parameters such as the number of iterations, the learning rate, and the number of training iterations in the network architecture module.
  • the behavior recognition system After adding behavior information based on sensors to a single information source based on visual behavior information, the behavior recognition system will be more perfect; at the same time, compared with mainstream model algorithms for behavior recognition, the present invention has further improved accuracy. It also has good stability in real-time.
  • FIG. 1 is a schematic diagram of the main module structure framework and connection relationship of the real-time behavior recognition system based on Lora and Capsule of the present invention.
  • FIG. 2 is a schematic diagram of the module composition and connection relationship of the real-time behavior recognition system based on Lora and Capsule of the present invention.
  • FIG. 3 is a schematic diagram of the workflow of the real-time behavior recognition system based on Lora and Capsule of the present invention.
  • FIG. 4 is a schematic diagram of behavior recognition of a real-time behavior recognition system based on Lora and Capsule of the present invention.
  • FIG. 5 is a schematic diagram of the working principle of the unit of the Capsule layer and the unit of the Capsule layer in the behavior recognition of the real-time behavior recognition system based on Lora and Capsule of the present invention.
  • a real-time behavior recognition system based on Lora and Capsule includes a behavior information physical layer, a behavior information access layer, a behavior information platform layer, and a behavior information application layer connected in sequence;
  • the physical layer of behavioral information is used to: perceive, collect, store, and transmit user's behavioral information from the environment.
  • the behavioral information includes: acceleration, angular velocity, and heart rate;
  • the behavior information access layer is used to: transmit the collected behavior information through Lora technology in the low-power wide-area Internet of Things;
  • the behavior information platform layer is used to: conduct uncertainty detection, standardization and time series-based interception of behavior information in sequence, train the labeled behavior information collection under the built network architecture model, while continuously optimizing the loss value Find the best model;
  • Uncertainty detection refers to: processing incomplete or inconsistent information in the behavior information through context prediction filling, zero-filling, and deletion methods to improve the credibility of the behavior information; standardization is performed on numeric data Normalization processing to improve the accuracy and generalization ability of the model;
  • Time series-based interception refers to the interception of behavior information through the sliding window mechanism to ensure the normalization of the model input and improve the accuracy of the model;
  • the behavior information application layer is used to: adjust the stability and adaptability of the entire real-time behavior recognition system.
  • the present invention proposes a relatively optimized system in four aspects of transmission technology, information processing, behavior recognition and behavior application, which makes up for the shortcomings of poor real-time behavior recognition on the market and the inability to use it in specific areas.
  • the accuracy rate of behavior recognition The aspect has also been further improved, so that the system has stability.
  • the physical layer of behavior information is the behavior information collection module.
  • the behavior information collection module includes a sensor module and several intelligent hardware modules; the sensor module includes several different types of sensors, and the intelligent hardware modules are connected to several different types of sensors, respectively, and the intelligent hardware modules are used for control
  • the sensor perceives different types of user behavior information, and stores the sensed behavior information.
  • the behavior information collection module selects sub-modules and designs smart devices through the QoD parameters, application scenarios, and user requirements of each sub-module.
  • QoD parameters mainly include the sampling frequency, service life, and accuracy of the sensor module.
  • the intelligent hardware module adopts Lora to transmit behavior information.
  • Transmission technology refers to a technology that makes full use of the transmission capabilities of different channels to form a complete transmission system, so that information can be reliably transmitted. With the progress of society and the development of wireless technology, the convenience of wireless transmission is further amplified under the premise that the packet loss rate is not high.
  • the mainstream wireless technologies are mainly WiFi, Bluetooth, ZigBee, 3G, 4G, etc.
  • Each wireless technology is in an indispensable situation in terms of transmission distance and power consumption, but in order to achieve real-time behavior recognition, the transmission of behavior information requires a Wireless technology with low power consumption and long transmission distance.
  • Low power consumption local area network (LPWAN) is the main technology to solve the current situation, so the present invention uses Lora to transmit behavior information.
  • LPWAN low power consumption local area network
  • the behavior information access layer is a behavior information transmission module.
  • the behavior information transmission module includes a behavior information transmission module and a behavior information reception module; the behavior information transmission module is connected to the intelligent hardware module, and is used to send behavior information to the behavior information reception module.
  • the information sending module is a Lora node
  • the behavior information receiving module is a Lora base station.
  • the present invention selects Lora nodes and Lora base stations as transmission media.
  • the behavior information platform layer is the behavior information preprocessing module.
  • the behavior information preprocessing module includes a behavior information detection module, a behavior information uncertainty elimination module, a behavior information processing module, and a network architecture module that are connected in sequence;
  • Behavior information detection module includes inconsistency detection / quantization unit and incompleteness detection / quantization unit; behavior information uncertainty elimination module includes inconsistency elimination unit and incompleteness elimination unit; behavior information processing module includes behavior information standardization connected in sequence Unit, behavior information sliding window unit; network architecture module includes convolution layer unit, Capsule layer one unit, Capsule layer two unit, fully connected layer unit connected in sequence;
  • the behavior information receiving module is the gateway connection behavior information detection module
  • the behavior information received by the behavior information receiving module is the original behavior information input to the behavior information detection module, and the inconsistency detection / quantization unit and the incompleteness detection / quantization unit are used to detect the uncertainty of the original behavior information.
  • the quantization unit detects whether there is any objection to different types of behavior information at the same time, and the incompleteness detection / quantization unit detects whether the perceived behavior information at the same time is missing;
  • the incompleteness elimination unit and the inconsistency elimination unit are used to eliminate the uncertainty.
  • Method 0, context prediction filling method, inconsistency elimination unit processes inconsistent information through voting, hardware QoD optimal principle, DS (Dempster-Shafer) evidence theory, fuzzy set, and enters the behavior information standardization unit;
  • QoC index refers to the index used to describe the quality of behavior information, including completeness, credibility, and update; if there is no uncertainty in behavior information, directly enter the behavior information standardization unit; through the behavior information standardization unit and behavior
  • the information sliding window unit is processed, and the behavior information standardization unit is processed through normalization and normalization methods to improve recognition accuracy and applicability; the behavior information sliding window unit performs behavior information adjustment by adjusting the size of the sliding window and the sliding method of the sliding window Interception based on time series;
  • Input the processed behavior information into the trained network architecture model, and realize the behavior recognition through the network architecture model; the convolutional layer unit extracts features from the behavior information and converts the characteristic scalar to vector.
  • a unit in the Capsule layer is used to Convert the input behavior information into behavior information with spatial characteristics; the second unit of the Capsule layer processes the behavior information through a dynamic routing protocol; the fully connected layer unit converts the behavior information features into ordered one-dimensional features, and finally passes the Softmax classifier Calculate all the features and recognize the current behavior.
  • the main work of the network architecture module is to make recognition based on behavioral information.
  • the proposition of machine learning can truly appreciate the power of artificial intelligence.
  • the proposition of deep learning has a considerable aspect in the recognition rate progress.
  • the focus of machine learning models and deep learning models is whether the input information contains some feature values.
  • the network architecture based on Capsule adopted in the present invention not only pays attention to the characteristics of the behavior information, but also adds the spatial relationship of the characteristics of the behavior information to improve the accuracy of behavior recognition.
  • the behavior preprocessing module mainly improves the credibility of behavior information by preprocessing the behavior information. Compared with some systems that directly perform behavior recognition on the original behavior information, the present invention will greatly improve the stability and accuracy after the behavior information preprocessing.
  • the invention mainly performs uncertainty analysis on the original information, and performs corresponding processing on the category and degree of the uncertainty of the behavior information.
  • the present invention provides a normalization method and a normalization method. The behavior information is intercepted based on time series by adjusting the size of the sliding window and the sliding mode of the sliding window.
  • the behavior information application layer includes a behavior information threshold setting module and a behavior application layer adjustment module.
  • the behavior application layer adjustment module includes a behavior recognition unit, a user feedback unit, and an error correction unit connected in sequence;
  • the behavior information threshold setting module is used to adjust the threshold in the behavior information uncertainty elimination module, so as to determine whether there is uncertainty in the monitoring data, and adjust the uncertainty processing module to select the data processing method;
  • the behavior recognition unit is used to do the current behavior Real-time identification;
  • the user feedback unit adjusts the preset threshold and the parameters of the network architecture module according to different scenarios and user needs, which improves the applicability of the system to a certain extent;
  • the error correction unit continuously adjusts the network architecture module, Keep the network architecture module in an optimal state all the time.
  • the behavior recognition unit mainly conducts behavior recognition based on the behavior information and a reasonable model; the QoS index value refers to the quality of service, generates corresponding adjustment information according to the service quality, and then feeds back to the behavior information pre-processing module; obtains the QoE of the user for the entire application service
  • the index value and feedback information are generated and transmitted to the behavior information preprocessing module; the QoE index value is a user rating index used to express the user's satisfaction with the application service, and the main function is to adjust the preset QoC index value.
  • Step S01 Obtain QoD parameters
  • the main QoD parameters include: sensor accuracy, sampling interval and bracelet material.
  • the accuracy of the sensor is 0.94, 0.80, 0.88, and the sampling interval is 0.02s-1s.
  • the material is mainly composed of rubber, alloy and other materials.
  • Step S02 Design collection equipment
  • the design of behavioral information collection equipment can use materials with high sampling frequency, high recognition accuracy, and non-destructive materials.
  • the design can be considered to use materials with low sampling frequency, common identification and slightly lower cost for design.
  • Step S03 Sending behavior information
  • the Lora node is used to send the behavior information
  • Step S04 Reception of behavior information
  • the Lora base station is used to receive the behavior information; the behavior information sending device uses a Lora node, and the corresponding behavior information is received using the Lora base station.
  • Step S05 Uncertainty detection of behavior information
  • the incompleteness detection / quantization unit sets the threshold to 0.85, that is, 85% of the behavior information received per second is missing, and the original behavior information is considered to be incomplete;
  • the inconsistency detection / quantization unit sets the threshold to 0.8, that is, when the similarity of the behavior information received per second is lower than 0.8, it is determined that there is inconsistency in the original behavior information of the group.
  • step S07 Perform inconsistency detection / quantization and incompleteness detection / quantization on the original behavior information according to the set behavior information threshold range to analyze the original behavior information and find that the original behavior information has inconsistencies, incompleteness and other uncertainties, perform the steps S06, otherwise execute step S07.
  • Step S06 Elimination of uncertainty in behavior information
  • the system can delete the incomplete behavior information, or fill in 0 with incomplete information, or fill in the incomplete behavior information according to the prediction of context information.
  • the system defaults to choose according to the context Information prediction will fill in incomplete behavior information;
  • the system can modify the inconsistent information according to the voting and election principles, or modify the inconsistent information according to the QoD optimal principle of information acquisition hardware, or use the DS evidence theory to find the inconsistent information. If the credibility is changed, the system chooses to modify the inconsistent information according to the voting and election principles by default.
  • Step S07 Processing of behavior information
  • the standardization unit of behavior information mainly standardizes the same type of behavior information.
  • the standardization that can be used in the system is mainly the normalization method or the normalization method.
  • the default standardization of the system is the normalization method;
  • the behavior information sliding window unit mainly intercepts behavior information based on time series.
  • the system mainly provides two types of parameters: the size of the sliding window and the sliding mode.
  • the size of the sliding window is 40, 60, 80, 100, and the sliding mode is mainly based on For half-time series sliding and sliding based on all time series, the default sliding window size of the system is 80, and the sliding mode is based on half time series sliding.
  • Step S08 Behavior information network architecture
  • the loss function is continuously optimized to optimize the module parameters and the dynamic routing protocol in the Capsule layer unit, and finally the module with a higher recognition rate is obtained.
  • the training set can select the behavior information of all people, or can select the behavior information of a certain individual for the training module, and then conduct behavior recognition for a certain individual. Because the program requires huge behavior information and needs large resources to support, it is recommended to use it only for some heavyweight criminals.
  • the system selects the criminal behavior information database of the entire prison area as the training module by default.
  • the specific implementation process of the module used in this example is as follows:
  • X i refers to each piece of information after behavior information undergoes uncertainty, standardization, and sliding window processing based on time series
  • W ij refers to the weight parameter of the convolutional layer unit, and the initial value defaults to generation truncation Normally distributed random numbers
  • b j refers to the offset parameter of the convolutional layer unit, the initial value defaults to 0.0;
  • n the number of convolution kernels
  • Y j is the output of the convolution layer
  • the size of the output information is: 5 ⁇ 1 ⁇ 40 ⁇ 256; it needs to ensure that the result of the score in the previous formula is a positive integer.
  • the output at this time is a vector behavior information, which meets the input requirements of the Capsule network;
  • W jl refers to the weight parameter of a unit in the Capsule layer, and the initial value defaults to generating a random number with truncated normal distribution
  • b l refers to the offset parameter of a unit in the Capsule layer, the initial value is 0.0 by default;
  • the squsah () function is a new nonlinear function, similar to the previous common nonlinear functions such as tanh (), relu (), etc.
  • the squsah () function is a nonlinear process for vector information; other nonlinear functions are mainly Processing of scalar information;
  • the size of the information output after one unit in the Capsule layer is: 5 ⁇ 320 ⁇ 8 ⁇ 1;
  • b ik refers to the dynamic routing weights of the i-th neuron in the first unit of the Capsule layer and the k-th neuron in the second unit of the Capsule layer;
  • b ij refers to the dynamic routing weights of the i-th neuron in the first unit of the Capsule layer and the j-th neuron in the second unit of the Capsule layer;
  • S j refers to the behavior information characteristics output by the second unit of the Capsule layer after the dynamic routing protocol.
  • the size of the information output after processing by the two units of the Capsule layer is: 5 ⁇ 12 ⁇ 16 ⁇ 1;
  • the size of the information output after passing through the fully connected layer unit is: 5 ⁇ 192 ⁇ 1;
  • the system adjustable parameters mainly include dynamic routing iterations, learning rate, and training iterations.
  • the dynamic routing iterations are set to 1-10; the learning rate is set to 0.1, 0.01, and 0.001; and the training iterations are set to 1 —50.
  • the system default parameters are 5, 0.01, and 40 in order.
  • Step S09 Recognition of behavior information
  • Step S10 Error detection
  • step S11 If an error is found, perform step S11; otherwise, perform step S12;
  • the error correction unit adjusts the behavior information threshold range and the corresponding parameters of the behavior information processing module;
  • the behavior information threshold range includes the threshold range of uncertainty detection, and the corresponding parameters of the behavior information processing module include the sliding window of the behavior information sliding window unit Size and window sliding mode; when there are many recognition errors, the threshold range of behavior information is appropriately increased, and the size of the sliding window and the sliding interval of the window are reduced;
  • Step S12 User feedback detection
  • step S13 is executed.
  • Step S13 User feedback
  • Parameters that can be adjusted include: the threshold of the inconsistency detection / quantization unit and the threshold of the incompleteness detection / quantization unit in the original behavior information detection module, the standardization method and behavior information slip in the behavior information standardization unit in the behavior information processing module The size and manner of sliding windows in the window unit, the number of iterations, the learning rate, and the number of training iterations in the network architecture module.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Telephonic Communication Services (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

本发明涉及一种基于Lora和Capsule的实时行为识别系统及其工作方法，该系统包括行为信息物理层、行为信息接入层、行为信息平台层、行为信息应用层。本发明将行为信息接入层的传输采用Lora节点、Lora基站，实现远距离、低功耗的行为信息传输；对行为信息不确定性进行了不一致和不完备性方面的处理，提高行为信息的可信度；采用Capsule自动获取有用特征以及特征之间的空间关系来进行行为识别，在精度方面有了很大的提升。

Description

一种基于Lora和Capsule的实时行为识别系统及其工作方法 技术领域
本发明涉及一种基于Lora和Capsule的实时行为识别系统及其工作方法，属于人工智能与模式识别的技术领域。
背景技术
行为识别系统是通过获取人的行为信息，经过合理的模型实现行为的系统。随着物联网、人工智能、大数据和云计算等先进技术的发展和成熟，越来越多的学者开始关注行为识别方向的研究。行为识别已经成为人工智能与模式识别研究领域中一个炙手可热的研究方向，加之可穿戴设备的发展为人体行为识别提供了良好的契机。如今，人体行为识别技术已经在游戏、人体运动分析、智能家居、人机交互以及医疗诊断和监护等领域得到了初步应用。
行为识别系统所需要的行为信息主要来自以下两个方面：
1、基于视觉的行为信息——通过摄像设备采集视觉行为信息。
2、基于传感器的行为信息——通过智能硬件采集体征行为信息。
目前，市面上主流的行为识别技术主要是对基于视觉的行为信息进行识别。虽然两种行为信息的获取方式都可以通过相应的算法实现人体行为的实时识别，但基于视觉行为信息获取的方式存在一定的弊端，在摄像设备的盲区或环境比较昏暗的场景中很多行为信息无法获取。例如：在卫生间内进行打架斗殴时，管理者无法了解卫生间内发生的情况，所以通过单一的视觉行为信息进行行为识别存在很多的缺陷。
另外，目前行为识别系统中信息的传输技术主要以蓝牙、ZigBee、WiFi、3G、4G为主。主流的传输技术在传输距离和功耗方面不可兼得，因此，要想实现实时行为识别，需要一种低功耗、传输距离远的传输技术。Loa是由Semtech公司发布的无线电调制解调技术。该技术具有Lora-WAN协议、Lora私有协议、CLASS协议、数据透传四个类别，相较于其他低功耗广域物联网技术，该技术在功耗、自组网等方面存在很大的优势。
系统进行行为识别的模型主要采用机器学习、深度学习等算法。目前，主流的机器学习主要有K近邻算法(KNN)、支持向量机(SVM)、随机森林、神经网络等；主流的深度学习算法主要有深度神经网络(DNN)、卷积神经网络(CNN)、循环(递归)神经网络(RNN)、深度信念网络(DBN)等。主流算法都是根据行为信息特征进行行为识别，所以输入行为信息只要存在一定的特征就会被分成某一类行为。例如：行为信息1100和0011分别表示起立和坐下，1010和0101表示打架行为，主流的算法可能将只要有两个1和两个0特征的行为信息判断是打架行为，因此如果输入行为信息的特征为0101、1100、0011或1010将都会被认为是打架行为，但实际上1100或0011表示的是起立或坐下行为。这主要是由于现有的主流算法只考虑行为信息是否包含某些特征，而不考虑这些特征的空间特性，所以在一定程度上会造成误判，降低行为识别的准确率。Geoffrey Hinton于2017年提出胶囊网络的概念，胶囊网络不仅可以标识行为信息是否具有某些特征，同时还可以标识各个特征之间的空间关系。本发明采用的胶囊网络算法模型在行为识别的准确率方面有了较高的提升。
发明内容
针对目前行为信息获取的单一性、传输方式的特殊性和预测模型的误判性，本发明提出了一种基于Lora和Capsule的实时行为识别系统。
本发明还公开了上述系统的工作方法。
发明概述：
基于Lora和Capsule的实时行为识别系统，首先，通过设备质量(QoD)参数对智能硬件进行筛选和设计，通过智能设备对行为信息进行采集，然后，将行为信息通过Lora节点进行传输，此时Lora基站将接收到行为信息。由于此时的行为信息为原始行为信息，系统需要对原始行为信息进行不确定性检测，将行为信息中具备不完备或不一致的信息通过上下文预测填充、补0、删除等方法进行处理，从而提高行为信息的可信度。接下来，对经过不确定性处理的行为信息进行标准化和基于时间序列的截取，标准化是为了提高模型的准确率和泛化能力，通过滑动窗口机制进行行为信息的截取是为了模型输入的归一化和提高模型的准确率。再然后，是在搭建的网络架构 模型下将带有标签的行为信息集合进行训练，在不断优化损失函数的同时找到最佳模型。将实时采集的行为信息传入模型，实现行为的实时识别。最后，系统会根据用户体验质量(QoE)和服务质量(QoS)对预设的阈值和模型参数进行调整，提高系统的稳定性、准确率和适用性。
本发明为基于传感器的实时行为识别提供了一种可行的方案，弥补了基于视频行为识别时所造成的缺陷，多源行为信息为提高行为识别的准确率奠定了基础。基于Capsule网络的深度学习模型在行为识别方面的应用也大大提高了行为识别的准确率。
本发明的技术方案为：
一种基于Lora和Capsule的实时行为识别系统，包括依次连接的行为信息物理层、行为信息接入层、行为信息平台层、行为信息应用层；
所述行为信息物理层用于：从环境中感知、采集、存储、传输用户的行为信息，行为信息包括：加速度、角速度、心率；
所述行为信息接入层用于：将采集的行为信息通过低功耗广域物联网进行组网传输；
所述行为信息平台层用于：对行为信息依次进行不确定性检测、标准化和基于时间序列的截取、在搭建的网络架构模型下将带有标签的行为信息集合进行训练，在不断优化损失值的同时找到最佳模型；不确定性检测是指：将行为信息中不完备或不一致的信息通过上下文预测填充、补0、删除方法进行处理，提高行为信息的可信度；标准化是对数值型数据进行归一化处理，从而提高模型的准确率和泛化能力；基于时间序列的截取是指通过滑动窗口机制进行行为信息的截取，以保证模型输入的归一化，提高模型的准确率；
所述行为信息应用层用于：调节整个实时行为识别系统稳定性和自适应性。
本发明在传输技术、信息处理、行为识别和行为应用等四个方面提出了一个相对优化的系统，弥补了目前市面上行为识别实时性差、无法在特定区域使用的不足，在行为识别的准确率方面也有了进一步的提升，使系统具备了稳定性。
根据本发明优选的，所述行为信息物理层即行为信息采集模块，所述行为信息采集模块包括传感器模块和若干个智能硬件模块；所述传感器模块包括若干不同类型的传感器，所述智能硬件模块分别连接若干不同类型的传感器，所述智能硬件模块用于控制传感器感知用户不同类型的行为信息，并把感知到的行为信息进行存储。
行为信息采集模块中通过各个子模块的QoD参数、应用场景和用户需求来进行子模块的选取和智能设备的设计，QoD参数主要包括传感器模块的采样频率、使用寿命、精度等。
智能硬件模块采用了Lora进行行为信息的传输。传输技术是指充分利用不同信道的传输能力构成一个完整的传输系统，使信息得以可靠传输的技术。随着社会的进步和无线技术的发展，在丢包率要求不高的前提下，无线传输的便利性被进一步的放大。目前主流的无线技术主要有WiFi、蓝牙、ZigBee、3G、4G等，各无线技术在传输距离和功耗方面都处于不可兼得的态势，但是为了实现实时行为识别，行为信息的传输需要一种功耗低、传输距离远的无线技术。低功耗局域网(LPWAN)是解决当前形势的主要技术，因此本发明采用了Lora进行行为信息的传输。
根据本发明优选的，所述行为信息接入层即行为信息传输模块，所述行为信息传输模块包括行为信息发送模块和行为信息接收模块；所述行为信息发送模块连接所述智能硬件模块，用于将行为信息发送至所述行为信息接收模块。
进一步优选的，所述信息发送模块为Lora节点，所述行为信息接收模块为Lora基站。
根据本发明优选的，所述行为信息平台层即行为信息预处理模块，所述行为信息预处理模块包括依次连接的行为信息检测模块、行为信息不确定性消除模块、行为信息处理模块、网络架构模块；
所述行为信息检测模块包括不一致性检测/量化单元和不完备性检测/量化单元；
所述行为信息不确定性消除模块包括不一致性消除单元和不完整性消除单元；
所述行为信息处理模块包括依次连接的行为信息标准化单元、行为信息滑窗单元；
所述网络架构模块包括依次连接的卷积层单元、Capsule层一单元、Capsule层二单元、全连接层单元；
所述行为信息接收模块即网关连接所述行为信息检测模块；
所述行为信息接收模块接收到的行为信息即原始行为信息输入到所述行为信息检测模块，通过所述不一致性 检测/量化单元和所述不完备性检测/量化单元对原始行为信息进行不确定性的检测，所述不一致性检测/量化单元检测同一时刻不同类型的行为信息是否存在异议，所述不完备性检测/量化单元检测同一时刻感知的行为信息是否存在丢失；
如果发现行为信息具有不确定性，则通过所述不完整性消除单元和所述不一致性消除单元进行不确定性的消除，所述不完整性消除单元对同一时刻感知行为信息存在的丢失情况通过删除法、补0法、上下文预测填充法进行处理，所述不一致性消除单元对不一致信息通过投票选举、硬件的QoD最优原则、D-S(Dempster-Shafer)证据论、模糊集的方式进行处理，进入所述行为信息标准化单元；如果发现行为信息不存在不确定性，则直接进入所述行为信息标准化单元；通过所述行为信息标准化单元和所述行为信息滑窗单元进行处理，所述行为信息标准化单元通过规范化、归一化方法进行处理，提高识别准确率和适用性；所述行为信息滑窗单元通过调节滑动窗口的大小和滑动窗口的滑动方式对行为信息进行基于时间序列的截取；
将处理完毕的行为信息输入到训练好的网络架构模型中，通过网络架构模型实现行为识别；所述卷积层单元对行为信息提取特征，进行特征标量到矢量的转换，所述Capsule层一单元用于将输入的行为信息转换成具有空间特性的行为信息；所述Capsule层二单元通过动态路由协议对行为信息进行处理；所述全连接层单元将行为信息特征转变为有序的一维特征，最后通过Softmax分类器将所有特征进行运算，识别出当前的行为。
网络架构模块主要工作是根据行为信息来做出识别，在人工智能、模式识别领域中，机器学习的提出可以切实的体会到人工智能的强大，深度学习的提出在识别率方面有了一个长足的进步。但是机器学习模型和深度学习模型关注的重点都是输入信息中是否包含了一些特征值。本发明中采用的基于Capsule的网络架构不仅对行为信息所具有的特征进行了关注，还加入了行为信息特征的空间关系，提高了行为识别的准确率。
行为预处理模块主要是通过对行为信息的预处理提高行为信息的可信度。相比较一些系统直接对原始行为信息进行行为识别而言，本发明在经过行为信息预处理之后会在稳定性和准确率等方面有很大的提升。本发明主要对原始信息进行不确定性分析，通过对行为信息不确定性的类别以及程度进行相应的处理。在信息标准化方面，本发明提供了规范化方法、归一化方法。通过调节滑动窗口的大小和滑动窗口的滑动方式对行为信息进行基于时间序列的截取。
根据本发明优选的，所述行为信息应用层包括行为信息阈值设置模块、行为应用层调整模块，所述行为应用层调整模块包括依次连接的行为识别单元、用户反馈单元、错误修正单元；
行为信息阈值设置模块用于调节行为信息不确定性消除模块中的阈值，从而对监测数据是否存在不确定性，并调整不确定性处理模块选择数据处理方式；行为识别单元用于对当前行为做实时识别；所述用户反馈单元根据不同的场景和用户需求对预设的阈值和网络架构模块的参数进行调整，在一定程度内提高了系统的适用性；错误修正单元不断的调整网络架构模块，让网络架构模块一直处于最优状态。
通过所述的错误修正单元对识别错误率较高时进行参数调整，所述的用户反馈单元用户通过调节系统参数以适用于不同的场景。
上述基于Lora和Capsule的实时行为识别系统及其工作方法，包括步骤如下：
步骤S01：传感器感知行为信息
根据行为识别所需要的原始行为信息、传感器的QoD参数筛选不同厂家、不同类型的传感器，传感器的QoD参数包括：采样频率、使用寿命、精度，例如：对于需要重点监护的用户可以采用采样频率高、精度高的传感器进行行为信息的感知，对于一般用户可以采用采样频率普通、精度普通传感器进行行为信息的感知；传感器感知用户不同类型的行为信息；
步骤S02：设计智能硬件模块
根据方案要求选择合适的智能硬件模块，通过智能硬件模块控制各个传感器，并采集行为识别系统所需的行为信息；
步骤S03：行为信息的发送
根据实时行为识别的要求，可以排除现在主流的一些无线传输方式，例如3G、4G、ZigBee、蓝牙等。低功耗广域网是一种较为适合的传输方式，由于NB-IoT即将投入民用，考虑到私密性，本系统采用Lora节点进行行为信息的发送；
步骤S04：行为信息的接收
采用Lora基站接收行为信息；行为信息的发送设备采用Lora节点，与之相对应的行为信息的接收采用Lora基站。
步骤S05：行为信息的不确定性检测
设定行为信息阈值范围，例如设定行为信息的正确率不低于85％，当行为信息的正确率低于85％时，则认为信息为不确定性行为信息，通过不一致性检测/量化单元、不完备性检测/量化单元依次对原始行为信息进行不一致性检测/量化、不完备性检测/量化，得到检测结果，当原始行为信息存在不一致、不完备时，执行步骤S05，否则，执行步骤S06；原始行为信息是指步骤S01传感器感知的用户不同类型的行为信息；
步骤S06：行为信息的不确定性消除
不完备性消除单元通过行为信息不确定性检测的阈值，选用不同的方法对行为信息进行处理，当行为信息的正确率为85％-90％时，针对行为信息采用上下文预测填充法进行处理，当行为信息的正确率为90％-95％时，针对行为信息采用补0法进行处理，当行为信息的正确率为95％-100％时，针对行为信息采用删除法进行处理；
不一致性消除单元对不一致信息进行处理，处理方法包括投票选举、硬件的QoD最优原则、D-S(Dempster-Shafer)证据论、模糊集；提高原始行为信息的可信度；
步骤S07：行为信息的处理
通过行为信息标准化单元对可信度较高的行为信息进行标准化；行为信息的标准化针对不同类型的数据使用不同的标准化方式，包括：针对类别型特征的数据，采用独热编码(one-hot编码)标准化；针对数值型特征的数据，采用归一化处理标准化；针对有序型特征的数据，采用有序型数值编码标准化；标准化可以让系统具备很好的扩展性。
参照用户预设的参数，用户预设的参数包括：滑动窗口的大小和窗口的滑动方式，通过行为信息滑窗单元对标准化处理后的行为信息进行滑动窗口处理，使行为信息变为输入网络架构模块的信息块；
步骤S08：行为信息网络架构
通过卷积层单元、Capsule层一单元，Capsule层二单元、全连接层单元构建一个四层的网络架构模型，参照用户设置的参数，用户设置的参数主要包括：输入数据的现状、大小、卷积层核大小、个数等一系列参数，通过若干次迭代对具有标签的行为信息进行训练，训练过程中通过降低损失函数来不断优化模型参数以及Capsule层单元中的动态路由协议，最终得到识别率高的网络架构模型；
步骤S09：行为信息的识别
将实时采集的行为信息输入到已经训练好的网络架构模型中进行当前行为的实时识别；
步骤S10：错误检测
判断当前行为识别是否发生错误，如发现存在错误，则执行步骤S11，否则执行步骤S12；
步骤S11：错误修正
错误修正单元对行为信息阈值范围、行为信息处理模块的相应参数进行调整；行为信息阈值范围包括不确定性检测的阈值范围，行为信息处理模块的相应参数包括行为信息滑窗单元中的滑动窗口的大小和窗口的滑动方式；当识别错误较多时，适当的提高行为信息的阈值范围，同时将滑动窗口的大小和窗口的滑动方式变小。
步骤S12：用户反馈检测
判断系统是否存在用户反馈信息，如存在反馈信息，则执行步骤S13。
步骤S13：用户反馈
用户反馈单元对对行为信息阈值范围、行为信息处理模块的相应参数进行反馈调整。
根据本发明优选的，所述步骤S08，
所述网络架构模块包括依次连接的卷积层单元、Capsule层一单元、Capsule层二单元、全连接层单元；
设置卷积层单元中卷积核个数为N 1，每个卷积核大小为1×Nuclear_Size 1，步长为L 1；
设置Capsule层一单元中卷积核个数为N 2，每个卷积核大小为1×Nuclear_Size 2，步长为L 2；
设置Capsule层二单元的输出长度为Num_Output维行为信息，每个维度采用Vec_Lenv个行为信息特征；
设置全连接层单元中输出长度为Output_Length；
包括步骤如下：
(1)输入一个Batch_Size×1×Window_Size×3大小的行为信息，Batch_Size是指一次在网络架构模块中运行的行为信息的个数，Window_Size是指每次输入网络架构模块的长度；
(2)当Batch_Size×1×Window_Size×3大小的行为信息经过卷积层单元之后，通过式(Ⅰ)将输入的行为信息由标量转换成矢量：
Figure PCTCN2019079387-appb-000001
式(Ⅰ)中，X i是指行为信息经过不确定性、标准化、基于时间序列的滑动窗口处理之后的每个信息；W ij是指卷积层单元的权重参数，初始值默认为生成截断正态分布的随机数；
b j是指卷积层单元的偏置参数，初始值默认为0.0；
n表示卷积核的个数；
Y j是表示卷积层输出；
输出信息大小为：
Figure PCTCN2019079387-appb-000002
其中需要保证前面公式中分数的结果为正整数。此时的输出结果是一个矢量行为信息，满足了Capsule网络的输入要求；
(3)令
Figure PCTCN2019079387-appb-000003
把以上M组卷积层封装在Capsule网络中，将矢量的行为信息Y j输入到Capsule层一单元，通过式(Ⅱ)将输入的行为信息转换成具有空间特性的行为信息；
Figure PCTCN2019079387-appb-000004
式(Ⅱ)中，W jl是指Capsule层一单元的权重参数，初始值默认为生成截断正态分布的随机数；
b l是指Capsule层一单元的偏置参数，初始值默认为0.0；
squsah()函数是一种新的非线性函数，类似于之前常见的tanh()、relu()等非线性函数，squsah()函 数是面向矢量信息的非线性处理；而其他非线性函数主要是针对标量信息的处理；
Figure PCTCN2019079387-appb-000005
是指Capsule网络输出的矢量行为信息特征；
经过Capsule层一单元后输出的信息大小为：
Figure PCTCN2019079387-appb-000006
(4)将具有空间特性的行为信息输入到Capsule层二单元，通过动态路由协议即式(Ⅲ)、(Ⅳ)将行为信息进行处理；
Figure PCTCN2019079387-appb-000007
Figure PCTCN2019079387-appb-000008
式(Ⅲ)、(Ⅳ)中，
b ik是指Capsule层一单元中第i个神经元和Capsule层二单元中第k个神经元的动态路由权重；
b ij是指Capsule层一单元中第i个神经元和Capsule层二单元中第j个神经元的动态路由权重；
Figure PCTCN2019079387-appb-000009
是指每个Capsule层的输出；
S j是指Capsule层二单元经过动态路由协议之后输出的行为信息特征。
Figure PCTCN2019079387-appb-000010
是指出网络架构的矢量输出；
Capsule层二单元处理后输出的信息大小为：Batch_Size×Num_Output×Vec_Lenv×1；
(5)通过全连接层单元将行为信息由矢量转换成标量；
经过全连接层单元后输出的信息大小为：
Batch_Size×Output_Length×1；
(6)加入Softmax分类器，通过Softmax分类器进行行为信息的分类识别；通过将信息大小为Batch_Size×Output_Length×1的行为信息特征经过分类器进行各个行为概率的求解，找出对应的各类概率数值最大的行为，网络架构模块最终的识别结果即概率数值最大的行为。
本发明的有益效果为：
1、实用性：
实时人体行为识别对于传输介质以及准确率有较高的要求，本发明很好的实现了行为信息的低功耗、远距离传输；同时在行为识别的准确率方面也具备了一定的优势。
2、自适应性：
针对不同的应用场景，通过用户反馈单元(QoE)和错误修正单元(QoS)对系统中的参数进行调整，提高系统的自适应性，为用户提供个性化、智能化的服务。其中可调节的参数包括：不一致性检测/量化单元的阈值 和不完备性检测/量化单元的阈值，行为信息标准化单元中的标准化方式和行为信息滑窗单元中的滑动窗口大小和滑动的方式，网络架构模块中的迭代次数、学习速率和训练迭代次数等参数。
3、高可靠性：
基于视觉行为信息单一信息源在加入基于传感器的行为信息之后，行为识别系统会更加的完善；同时与主流的模型算法进行行为识别相较而言，本发明在准确率方面有了进一步的提升。在实时性方面也有很好的稳定性。
附图说明
图1是本发明基于Lora和Capsule的实时行为识别系统主模块结构框架及连接关系示意图。
图2是本发明基于Lora和Capsule的实时行为识别系统实现的模块组成及连接关系示意图。
图3是本发明基于Lora和Capsule的实时行为识别系统工作流程示意图。
图4是本发明基于Lora和Capsule的实时行为识别系统的行为识别示意图。
图5是本发明基于Lora和Capsule的实时行为识别系统的行为识别中Capsule层一单元、Capsule层二单元工作原理示意图。
具体实施方式
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合实施例和说明书附图1-5对本发明的技术方案进行清楚、完整的描述，显然，此处所描述的具体实施例仅用以解释本发明，并不用于限定本发明。
实施例1
一种基于Lora和Capsule的实时行为识别系统，如图1所示，包括依次连接的行为信息物理层、行为信息接入层、行为信息平台层、行为信息应用层；
行为信息物理层用于：从环境中感知、采集、存储、传输用户的行为信息，行为信息包括：加速度、角速度、心率；
行为信息接入层用于：将采集的行为信息通过低功耗广域物联网中的Lora技术进行组网传输；
行为信息平台层用于：对行为信息依次进行不确定性检测、标准化和基于时间序列的截取、在搭建的网络架构模型下将带有标签的行为信息集合进行训练，在不断优化损失值的同时找到最佳模型；不确定性检测是指：将行为信息中不完备或不一致的信息通过上下文预测填充、补0、删除方法进行处理，提高行为信息的可信度；标准化是对数值型数据进行归一化处理，从而提高模型的准确率和泛化能力；基于时间序列的截取是指通过滑动窗口机制进行行为信息的截取，以保证模型输入的归一化，提高模型的准确率；
行为信息应用层用于：调节整个实时行为识别系统稳定性和自适应性。
本发明在传输技术、信息处理、行为识别和行为应用等四个方面提出了一个相对优化的系统，弥补了目前市面上行为识别实时性差、无法在特定区域使用的不足，在行为识别的准确率方面也有了进一步的提升，使系统具备了稳定性。
实施例2
根据实施例1所述的一种基于Lora和Capsule的实时行为识别系统，如图2所示，其区别在于，
行为信息物理层即行为信息采集模块，行为信息采集模块包括传感器模块和若干个智能硬件模块；传感器模块包括若干不同类型的传感器，智能硬件模块分别连接若干不同类型的传感器，智能硬件模块用于控制传感器感知用户不同类型的行为信息，并把感知到的行为信息进行存储。
行为信息采集模块中通过各个子模块的QoD参数、应用场景和用户需求来进行子模块的选取和智能设备的设计，QoD参数主要包括传感器模块的采样频率、使用寿命、精度等。
智能硬件模块采用了Lora进行行为信息的传输。传输技术是指充分利用不同信道的传输能力构成一个完整的传输系统，使信息得以可靠传输的技术。随着社会的进步和无线技术的发展，在丢包率要求不高的前提下，无线传输的便利性被进一步的放大。目前主流的无线技术主要有WiFi、蓝牙、ZigBee、3G、4G等，各无线技术在传输距离和功耗方面都处于不可兼得的态势，但是为了实现实时行为识别，行为信息的传输需要一种功耗低、传输距离远的无线技术。低功耗局域网(LPWAN)是解决当前形势的主要技术，因此本发明采用了Lora进行行为信息的传输。
行为信息接入层即行为信息传输模块，行为信息传输模块包括行为信息发送模块和行为信息接收模块；行为 信息发送模块连接所述智能硬件模块，用于将行为信息发送至行为信息接收模块。
信息发送模块为Lora节点，所述行为信息接收模块为Lora基站。鉴于实时性和场景的需求，本发明选取Lora节点和Lora基站作为传输介质。
行为信息平台层即行为信息预处理模块，行为信息预处理模块包括依次连接的行为信息检测模块、行为信息不确定性消除模块、行为信息处理模块、网络架构模块；
行为信息检测模块包括不一致性检测/量化单元和不完备性检测/量化单元；行为信息不确定性消除模块包括不一致性消除单元和不完整性消除单元；行为信息处理模块包括依次连接的行为信息标准化单元、行为信息滑窗单元；网络架构模块包括依次连接的卷积层单元、Capsule层一单元、Capsule层二单元、全连接层单元；
行为信息接收模块即网关连接行为信息检测模块；
行为信息接收模块接收到的行为信息即原始行为信息输入到行为信息检测模块，通过不一致性检测/量化单元和不完备性检测/量化单元对原始行为信息进行不确定性的检测，不一致性检测/量化单元检测同一时刻不同类型的行为信息是否存在异议，不完备性检测/量化单元检测同一时刻感知的行为信息是否存在丢失；
如果发现行为信息具有不确定性，则通过不完整性消除单元和所述不一致性消除单元进行不确定性的消除，不完整性消除单元对同一时刻感知行为信息存在的丢失情况通过删除法、补0法、上下文预测填充法进行处理，不一致性消除单元对不一致信息通过投票选举、硬件的QoD最优原则、D-S(Dempster-Shafer)证据论、模糊集的方式进行处理，进入行为信息标准化单元；QoC指标是指用于描述行为信息质量的指标，包括完整性、可信度和更新度等；如果发现行为信息不存在不确定性，则直接进入行为信息标准化单元；通过行为信息标准化单元和行为信息滑窗单元进行处理，行为信息标准化单元通过规范化、归一化方法进行处理，提高识别准确率和适用性；行为信息滑窗单元通过调节滑动窗口的大小和滑动窗口的滑动方式对行为信息进行基于时间序列的截取；
将处理完毕的行为信息输入到训练好的网络架构模型中，通过网络架构模型实现行为识别；所述卷积层单元对行为信息提取特征，进行特征标量到矢量的转换，Capsule层一单元用于将输入的行为信息转换成具有空间特性的行为信息；Capsule层二单元通过动态路由协议对行为信息进行处理；全连接层单元将行为信息特征转变为有序的一维特征，最后通过Softmax分类器将所有特征进行运算，识别出当前的行为。
网络架构模块主要工作是根据行为信息来做出识别，在人工智能、模式识别领域中，机器学习的提出可以切实的体会到人工智能的强大，深度学习的提出在识别率方面有了一个长足的进步。但是机器学习模型和深度学习模型关注的重点都是输入信息中是否包含了一些特征值。本发明中采用的基于Capsule的网络架构不仅对行为信息所具有的特征进行了关注，还加入了行为信息特征的空间关系，提高了行为识别的准确率。
行为预处理模块主要是通过对行为信息的预处理提高行为信息的可信度。相比较一些系统直接对原始行为信息进行行为识别而言，本发明在经过行为信息预处理之后会在稳定性和准确率等方面有很大的提升。本发明主要对原始信息进行不确定性分析，通过对行为信息不确定性的类别以及程度进行相应的处理。在信息标准化方面，本发明提供了规范化方法、归一化方法。通过调节滑动窗口的大小和滑动窗口的滑动方式对行为信息进行基于时间序列的截取。
行为信息应用层包括行为信息阈值设置模块、行为应用层调整模块，行为应用层调整模块包括依次连接的行为识别单元、用户反馈单元、错误修正单元；
行为信息阈值设置模块用于调节行为信息不确定性消除模块中的阈值，从而对监测数据是否存在不确定性，并调整不确定性处理模块选择数据处理方式；行为识别单元用于对当前行为做实时识别；所述用户反馈单元根据不同的场景和用户需求对预设的阈值和网络架构模块的参数进行调整，在一定程度内提高了系统的适用性；错误修正单元不断的调整网络架构模块，让网络架构模块一直处于最优状态。行为识别单元主要根据行为信息和合理的模型来进行行为识别；QoS指标值是指服务质量，根据服务质量产生相应的调整信息，然后反馈至行为信息预处理模块；获取用户对整个应用服务的QoE指标值，并产生反馈信息，传输到行为信息预处理模块；QoE指标值是用于表达用户对应用服务满意程度的用户评分指标，主要作用为调整预设的QoC指标值。
实施例3
实施例2所述基于Lora和Capsule的实时行为识别系统及其工作方法，如图3所示，以打架斗殴行为识别为例，监狱中的罪犯可能在心理和生理方面都和常人稍有区别，在对待问题角度可能会有偏激的行为。为了防止 过激行为造成严重的影响和危害，系统通过加速度传感器S1、角速度传感器S2、心率传感器S3来获取罪犯在一天中的行为信息，经过信息预处理之后提高信息的可信度，然后通过训练好的模型进行实时行为识别。监狱管理者可以根据不同的场景和不同的罪犯来设置不同的参数进行实时行为识别。具体步骤包括：
步骤S01：获取QoD参数
主要的QoD参数包括：传感器的精度、采样间隔和手环的材质。传感器的精度分别为0.94、0.80、0.88，采样间隔分别为0.02s—1s，材质主要由橡胶、合金等材质构成。
步骤S02：设计采集设备
根据传感器QoD参数、罪犯的需求以及关押等级来进行智能硬件的设计，对于重点监管对象，行为信息采集设备的设计可以采用采样频率高的、识别精度高的、不易破坏的材质，针对表现良好的、轻度监管对象，设计可以考虑采用采样频率较低的、识别普通的、造价稍低的材质来进行设计。
步骤S03：行为信息的发送
根据保密性以及活动范围的大小，采用Lora节点进行行为信息的发送；
步骤S04：行为信息的接收
采用Lora基站接收行为信息；行为信息的发送设备采用Lora节点，与之相对应的行为信息的接收采用Lora基站。
步骤S05：行为信息的不确定性检测
不完备性检测/量化单元将阈值设置为0.85，即每秒接收到的行为信息中有85％的信息缺失则认为原始行为信息存在不完备；
不一致性检测/量化单元将阈值设置为0.8，即每秒接收到的行为信息相似度低于0.8时则判定该组原始行为信息存在不一致。
根据设定的行为信息阈值范围对原始行为信息进行不一致性检测/量化、不完备性检测/量化来对原始行为信息进行分析，发现原始行为信息存在不一致、不完备等不确定性时，执行步骤S06，否则执行步骤S07。
步骤S06：行为信息的不确定性消除
若发现原始行为信息存在不完备性时，系统中可以将不完备的行为信息删除、或将不完备的信息进行补0、或根据上下文信息预测将不完备行为信息进行填充，系统默认选择根据上下文信息预测将不完备行为信息进行填充；
若发现原始行为信息存在不一致性，系统可以将不一致性信息按照投票选举原则进行修改、或将不一致信息根据信息获取硬件的QoD最优原则来进行修改、或将不一致信息使用D-S证据论的方法求出可信度进行修改，系统默认选择将不一致性信息按照投票选举原则进行修改。
经过行为信息的不确定性消除之后，大大的提高了原始行为信息的可信度和为后续行为信息的处理和行为的识别提供了可靠性。
步骤S07：行为信息的处理
行为信息的标准化单元主要对同一类行为信息进行标准化，系统中可使用的标准化主要是规范化方法、或归一化方法，系统默认的标准化为归一化方法；
行为信息滑窗单元主要对行为信息进行基于时间序列的截取，系统主要提供了滑动窗口的大小和滑动的方式两类参数，滑动窗口的大小有40、60、80、100，滑动方式主要有基于一半时间序列的滑动和基于全部时间序列的滑动，系统默认的滑动窗口的大小为80，滑动方式为基于一半时间序列的滑动。
步骤S08：行为信息网络架构
通过卷积层单元、Capsule层一单元，Capsule层二单元、全连接层单元构建一个四层的网络架构模块，参照用户预设的一些参数，通过N次迭代来对具有标签的行为信息进行训练，训练过程中不断的优化损失函数来优化模块参数以及Capsule层单元中的动态路由协议，最终得出识别率较高的模块。其中训练集可以选取所有人的行为信息，也可以选取某一个人的行为信息进行训练模块，然后对某一个人进行行为识别。由于该方案需要庞大的行为信息和需要较大的资源支持，建议只针对部分重量级罪犯进行使用，系统默认选取整个监区罪犯的行为信息数据库进行训练模块。如图4所示，本实例中所采用模块的具体实现流程如下所示：
设置卷积层单元中卷积核个数为256，每个卷积核大小为1×41，步长为1；
设置Capsule层一单元中卷积核个数为32，每个卷积核大小为1×21，步长为2；
设置Capsule层二单元的输出长度为8维行为信息，每个维度采用16个行为信息特征；
设置全连接层单元中输出的长度为6；
包括步骤如下：
(1)输入一个5×1×80×3大小的行为信息；
(2)当5×1×80×3大小的行为信息经过卷积层单元之后，通过式(Ⅰ)将输入的行为信息由标量转换成矢量：
Figure PCTCN2019079387-appb-000011
式(Ⅰ)中，X i是指行为信息经过不确定性、标准化、基于时间序列的滑动窗口处理之后的每个信息；W ij是指卷积层单元的权重参数，初始值默认为生成截断正态分布的随机数；
b j是指卷积层单元的偏置参数，初始值默认为0.0；
n表示卷积核的个数；
Y j是表示卷积层输出；
输出信息大小为：5×1×40×256；其中需要保证前面公式中分数的结果为正整数。此时的输出结果是一个矢量行为信息，满足了Capsule网络的输入要求；
(3)如图5所示，把以上8组卷积层封装在Capsule中，将卷积层单元输出的结果输入到Capsule层一单元，通过式(Ⅱ)将输入的行为信息转换成具有空间特性的行为信息；
Figure PCTCN2019079387-appb-000012
式(Ⅱ)中，W jl是指Capsule层一单元的权重参数，初始值默认为生成截断正态分布的随机数；
b l是指Capsule层一单元的偏置参数，初始值默认为0.0；
squsah()函数是一种新的非线性函数，类似于之前常见的tanh()、relu()等非线性函数，squsah()函数是面向矢量信息的非线性处理；而其他非线性函数主要是针对标量信息的处理；
Figure PCTCN2019079387-appb-000013
是指Capsule网络输出的矢量行为信息特征；
经过Capsule层一单元后输出的信息大小为：5×320×8×1；
(4)将Capsule层一单元输出结果作为Capsule层二单元的输入信息，通过动态路由协议即式(Ⅲ)、(Ⅳ)将行为信息进行处理；
Figure PCTCN2019079387-appb-000014
Figure PCTCN2019079387-appb-000015
式(Ⅲ)、(Ⅳ)中，
b ik是指Capsule层一单元中第i个神经元和Capsule层二单元中第k个神经元的动态路由权重；
b ij是指Capsule层一单元中第i个神经元和Capsule层二单元中第j个神经元的动态路由权重；
Figure PCTCN2019079387-appb-000016
是指每个Capsule层的输出；
S j是指Capsule层二单元经过动态路由协议之后输出的行为信息特征。
Figure PCTCN2019079387-appb-000017
是指出网络架构的矢量输出；
Capsule层二单元处理后输出的信息大小为：5×12×16×1；
(5)通过全连接层单元将行为信息由矢量转换成标量；
经过全连接层单元后输出的信息大小为：5×192×1；
(6)加入Softmax分类器，通过Softmax分类器进行行为信息的分类识别；通过将信息大小为5×192×1的行为信息特征经过分类器进行各个行为概率的求解，找出对应的各类概率数值最大的行为，网络架构模块最终的识别结果即概率数值最大的行为。
系统可调节参数主要有动态路由迭代次数、学习速率和训练迭代次数等参数，动态路由迭代次数的设置为1—10；学习速率的设置为0.1、0.01、0.001；和训练迭代次数的设置为1—50。系统默认参数依次为5、0.01、40。
步骤S09：行为信息的识别
将实时采集的行为信息输入到已经训练好的网络架构模型中进行当前行为的实时识别；
步骤S10：错误检测
判断当前行为识别是否发生错误，如发现存在错误，则执行步骤S11，否则执行步骤S12；
步骤S11：错误修正
错误修正单元对行为信息阈值范围、行为信息处理模块的相应参数进行调整；行为信息阈值范围包括不确定性检测的阈值范围，行为信息处理模块的相应参数包括行为信息滑窗单元中的滑动窗口的大小和窗口的滑动方式；当识别错误较多时，适当的提高行为信息的阈值范围，同时将滑动窗口的大小和窗口的滑动间隔变小；
步骤S12：用户反馈检测
判断系统是否存在用户反馈信息，如存在反馈信息，则执行步骤S13。
步骤S13：用户反馈
根据不同用户所处不同环境来进行一个反馈信息，调整用户预设置的一些参数来调整行为信息处理模块和网络架构模块。可以调节的参数包括：原始行为信息检测模块中的不一致性检测/量化单元的阈值和不完备性检测/量化单元的阈值，行为信息处理模块中的行为信息标准化单元中的标准化方式和行为信息滑窗单元中的滑动窗口大小和滑动的方式，网络架构模块中的迭代次数、学习速率和训练迭代次数等。

Claims (8)

  1. 一种基于Lora和Capsule的实时行为识别系统，其特征在于，包括依次连接的行为信息物理层、行为信息接入层、行为信息平台层、行为信息应用层；
    所述行为信息物理层用于：从环境中感知、采集、存储、传输用户的行为信息，行为信息包括：加速度、角速度、心率；
    所述行为信息接入层用于：将采集的行为信息通过低功耗广域物联网进行组网传输；
    所述行为信息平台层用于：对行为信息依次进行不确定性检测、标准化和基于时间序列的截取、在搭建的网络架构模型下将带有标签的行为信息集合进行训练，在不断优化损失值的同时找到最佳模型；不确定性检测是指：将行为信息中不完备或不一致的信息通过上下文预测填充、补0、删除方法进行处理，提高行为信息的可信度；标准化是对数值型数据进行归一化处理；基于时间序列的截取是指通过滑动窗口机制进行行为信息的截取；
    所述行为信息应用层用于：调节整个实时行为识别系统稳定性和自适应性。
  2. 根据权利要求1所述的一种基于Lora和Capsule的实时行为识别系统，其特征在于，所述行为信息物理层即行为信息采集模块，所述行为信息采集模块包括传感器模块和若干个智能硬件模块；所述传感器模块包括若干不同类型的传感器，所述智能硬件模块分别连接若干不同类型的传感器，所述智能硬件模块用于控制传感器感知用户不同类型的行为信息，并把感知到的行为信息进行存储。
  3. 根据权利要求2所述的一种基于Lora和Capsule的实时行为识别系统，其特征在于，所述行为信息接入层即行为信息传输模块，所述行为信息传输模块包括行为信息发送模块和行为信息接收模块；所述行为信息发送模块连接所述智能硬件模块，用于将行为信息发送至所述行为信息接收模块。
  4. 根据权利要求3所述的一种基于Lora和Capsule的实时行为识别系统，其特征在于，所述信息发送模块为Lora节点，所述行为信息接收模块为Lora基站。
  5. 根据权利要求3所述的一种基于Lora和Capsule的实时行为识别系统，其特征在于，所述行为信息平台层即行为信息预处理模块，所述行为信息预处理模块包括依次连接的行为信息检测模块、行为信息不确定性消除模块、行为信息处理模块、网络架构模块；
    所述行为信息检测模块包括不一致性检测/量化单元和不完备性检测/量化单元；
    所述行为信息不确定性消除模块包括不一致性消除单元和不完整性消除单元；
    所述行为信息处理模块包括依次连接的行为信息标准化单元、行为信息滑窗单元；
    所述网络架构模块包括依次连接的卷积层单元、Capsule层一单元、Capsule层二单元、全连接 层单元；
    所述行为信息接收模块即网关连接所述行为信息检测模块；
    所述行为信息接收模块接收到的行为信息即原始行为信息输入到所述行为信息检测模块，通过所述不一致性检测/量化单元和所述不完备性检测/量化单元对原始行为信息进行不确定性的检测，所述不一致性检测/量化单元检测同一时刻不同类型的行为信息是否存在异议，所述不完备性检测/量化单元检测同一时刻感知的行为信息是否存在丢失；
    如果发现行为信息具有不确定性，则通过所述不完整性消除单元和所述不一致性消除单元进行不确定性的消除，所述不完整性消除单元对同一时刻感知行为信息存在的丢失情况通过删除法、补0法、上下文预测填充法进行处理，所述不一致性消除单元对不一致信息通过投票选举、硬件的QoD最优原则、D-S证据论、模糊集的方式进行处理，进入所述行为信息标准化单元；如果发现行为信息不存在不确定性，则直接进入所述行为信息标准化单元；通过所述行为信息标准化单元和所述行为信息滑窗单元进行处理，所述行为信息标准化单元通过规范化、归一化方法进行处理，提高识别准确率和适用性；所述行为信息滑窗单元通过调节滑动窗口的大小和滑动窗口的滑动方式对行为信息进行基于时间序列的截取；
    将处理完毕的行为信息输入到训练好的网络架构模型中，通过网络架构模型实现行为识别；所述卷积层单元对行为信息提取特征，进行特征标量到矢量的转换，所述Capsule层一单元用于将输入的行为信息转换成具有空间特性的行为信息；所述Capsule层二单元通过动态路由协议对行为信息进行处理；所述全连接层单元将行为信息特征转变为有序的一维特征，最后通过Softmax分类器将所有特征进行运算，识别出当前的行为。
  6. 根据权利要求5所述的一种基于Lora和Capsule的实时行为识别系统，其特征在于，
    所述行为信息应用层包括行为信息阈值设置模块、行为应用层调整模块，所述行为应用层调整模块包括依次连接的行为识别单元、用户反馈单元、错误修正单元；
    行为信息阈值设置模块用于调节行为信息不确定性消除模块中的阈值，从而对监测数据是否存在不确定性，并调整不确定性处理模块选择数据处理方式；行为识别单元用于对当前行为做实时识别；所述用户反馈单元根据不同的场景和用户需求对预设的阈值和网络架构模块的参数进行调整，错误修正单元不断的调整网络架构模块，让网络架构模块一直处于最优状态。
  7. 权利要求6所述的基于Lora和Capsule的实时行为识别系统的工作方法，其特征在于，包括步骤如下：
    步骤S01：传感器感知行为信息
    根据行为识别所需要的原始行为信息、传感器的QoD参数筛选不同厂家、不同类型的传感器， 传感器的QoD参数包括：采样频率、使用寿命、精度，传感器感知用户不同类型的行为信息；
    步骤S02：设计智能硬件模块
    根据方案要求选择合适的智能硬件模块，通过智能硬件模块控制各个传感器，并采集行为识别系统所需的行为信息；
    步骤S03：行为信息的发送
    采用Lora节点进行行为信息的发送；
    步骤S04：行为信息的接收
    采用Lora基站接收行为信息；
    步骤S05：行为信息的不确定性检测
    设定行为信息阈值范围，当原始行为信息存在不一致、不完备时，执行步骤S05，否则，执行步骤S06；原始行为信息是指步骤S01传感器感知的用户不同类型的行为信息；
    步骤S06：行为信息的不确定性消除
    不完备性消除单元通过行为信息不确定性检测的阈值，选用不同的方法对行为信息进行处理，当行为信息的正确率为85％-90％时，针对行为信息采用上下文预测填充法进行处理，当行为信息的正确率为90％-95％时，针对行为信息采用补0法进行处理，当行为信息的正确率为95％-100％时，针对行为信息采用删除法进行处理；
    不一致性消除单元对不一致信息进行处理，处理方法包括投票选举、硬件的QoD最优原则、D-S证据论、模糊集；提高原始行为信息的可信度；
    步骤S07：行为信息的处理
    通过行为信息标准化单元对可信度较高的行为信息进行标准化；行为信息的标准化针对不同类型的数据使用不同的标准化方式，包括：针对类别型特征的数据，采用独热编码标准化；针对数值型特征的数据，采用归一化处理标准化；针对有序型特征的数据，采用有序型数值编码标准化；
    参照用户预设的参数，用户预设的参数包括：滑动窗口的大小和窗口的滑动方式，通过行为信息滑窗单元对标准化处理后的行为信息进行滑动窗口处理，使行为信息变为输入网络架构模块的信息块；
    步骤S08：行为信息网络架构
    通过卷积层单元、Capsule层一单元，Capsule层二单元、全连接层单元构建一个四层的网络架构模型，参照用户设置的参数，通过若干次迭代对具有标签的行为信息进行训练，训练过程中通过降低损失函数来不断优化模型参数以及Capsule层单元中的动态路由协议，最终得到识别率高的网络架构模型；
    步骤S09：行为信息的识别
    将实时采集的行为信息输入到已经训练好的网络架构模型中进行当前行为的实时识别；
    步骤S10：错误检测
    判断当前行为识别是否发生错误，如发现存在错误，则执行步骤S11，否则执行步骤S12；
    步骤S11：错误修正
    错误修正单元对行为信息阈值范围、行为信息处理模块的相应参数进行调整；行为信息阈值范围包括不确定性检测的阈值范围，行为信息处理模块的相应参数包括行为信息滑窗单元中的滑动窗口的大小和窗口的滑动方式；
    步骤S12：用户反馈检测
    判断系统是否存在用户反馈信息，如存在反馈信息，则执行步骤S13；
    步骤S13：用户反馈
    用户反馈单元对对行为信息阈值范围、行为信息处理模块的相应参数进行反馈调整。
  8. 根据权利要求7所述的基于Lora和Capsule的实时行为识别系统的工作方法，其特征在于，所述步骤S08，所述网络架构模块包括依次连接的卷积层单元、Capsule层一单元、Capsule层二单元、全连接层单元；
    设置卷积层单元中卷积核个数为N 1，每个卷积核大小为1×Nuclear_Size 1，步长为L 1；
    设置Capsule层一单元中卷积核个数为N 2，每个卷积核大小为1×Nuclear_Size 2，步长为L 2；
    设置Capsule层二单元的输出长度为Num_Output维行为信息，每个维度采用Vec_Lenv个行为信息特征；
    设置全连接层单元中输出长度为Output_Length；
    包括步骤如下：
    (1)输入一个Batch_Size×1×Window_Size×3大小的行为信息，Batch_Size是指一次在网络架构模块中运行的行为信息的个数，Window_Size是指每次输入网络架构模块的长度；
    (2)当Batch_Size×1×Window_Size×3大小的行为信息经过卷积层单元之后，通过式(Ⅰ)将输入的行为信息由标量转换成矢量：
    Figure PCTCN2019079387-appb-100001
    式(Ⅰ)中，X i是指行为信息经过不确定性、标准化、基于时间序列的滑动窗口处理之后的每个信息；W ij是指卷积层单元的权重参数，初始值默认为生成截断正态分布的随机数；
    b j是指卷积层单元的偏置参数，初始值默认为0.0；
    n表示卷积核的个数；
    Y j是表示卷积层输出；
    输出信息大小为：
    Figure PCTCN2019079387-appb-100002
    (3)令
    Figure PCTCN2019079387-appb-100003
    把以上M组卷积层封装在Capsule网络中，将矢量的行为信息Y j输入到Capsule层一单元，通过式(Ⅱ)将输入的行为信息转换成具有空间特性的行为信息；
    Figure PCTCN2019079387-appb-100004
    式(Ⅱ)中，W jl是指Capsule层一单元的权重参数，初始值默认为生成截断正态分布的随机数；
    b l是指Capsule层一单元的偏置参数，初始值默认为0.0；
    squsah()函数是一种新的非线性函数；
    Figure PCTCN2019079387-appb-100005
    是指Capsule网络输出的矢量行为信息特征；
    经过Capsule层一单元后输出的信息大小为：
    Figure PCTCN2019079387-appb-100006
    (4)将具有空间特性的行为信息输入到Capsule层二单元，通过动态路由协议即式(Ⅲ)、(Ⅳ)将行为信息进行处理；
    Figure PCTCN2019079387-appb-100007
    Figure PCTCN2019079387-appb-100008
    式(Ⅲ)、(Ⅳ)中，
    b ik是指Capsule层一单元中第i个神经元和Capsule层二单元中第k个神经元的动态路由权重；
    b ij是指Capsule层一单元中第i个神经元和Capsule层二单元中第j个神经元的动态路由权重；
    Figure PCTCN2019079387-appb-100009
    是指每个Capsule层的输出；
    S j是指Capsule层二单元经过动态路由协议之后输出的行为信息特征；
    Figure PCTCN2019079387-appb-100010
    是指出网络架构的矢量输出；
    Capsule层二单元处理后输出的信息大小为：Batch_Size×Num_Output×Vec_Lenv×1；
    (5)通过全连接层单元将行为信息由矢量转换成标量；
    经过全连接层单元后输出的信息大小为：
    Batch_Size×Output_Length×1；
    (6)加入Softmax分类器，通过Softmax分类器进行行为信息的分类识别；通过将信息大小为Batch_Size×Output_Length×1的行为信息特征经过分类器进行各个行为概率的求解，找出对应的各类概率数值最大的行为，网络架构模块最终的识别结果即概率数值最大的行为。
PCT/CN2019/079387 2018-11-01 2019-03-23 一种基于Lora和Capsule的实时行为识别系统及其工作方法 WO2020087848A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2019371325A AU2019371325B2 (en) 2018-11-01 2019-03-23 Real-time activity recognition system and working method based on lora and capsule

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811294526.X 2018-11-01
CN201811294526.XA CN109447162B (zh) 2018-11-01 2018-11-01 一种基于Lora和Capsule的实时行为识别系统及其工作方法

Publications (1)

Publication Number Publication Date
WO2020087848A1 true WO2020087848A1 (zh) 2020-05-07

Family

ID=65549883

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/079387 WO2020087848A1 (zh) 2018-11-01 2019-03-23 一种基于Lora和Capsule的实时行为识别系统及其工作方法

Country Status (3)

Country Link
CN (1) CN109447162B (zh)
AU (1) AU2019371325B2 (zh)
WO (1) WO2020087848A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111695342A (zh) * 2020-06-12 2020-09-22 复旦大学 基于语境信息的文本内容校正方法
CN111931882A (zh) * 2020-07-20 2020-11-13 五邑大学 货物自动结账方法、系统及存储介质
CN112327189A (zh) * 2020-10-14 2021-02-05 北方工业大学 一种基于knn算法的储能电池健康状态综合判断方法
CN113971399A (zh) * 2020-07-23 2022-01-25 北京金山数字娱乐科技有限公司 识别模型的训练方法及装置、文本识别方法及装置
CN116204844A (zh) * 2023-04-28 2023-06-02 西南石油大学 一种基于不确定性的电气设备异常数据清洗方法
CN116661330A (zh) * 2023-07-31 2023-08-29 深圳市云图数字科技有限公司 一种智能家居系统的协同运行方法及智能家居系统

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109447162B (zh) * 2018-11-01 2021-09-24 山东大学 一种基于Lora和Capsule的实时行为识别系统及其工作方法
CN110852382B (zh) * 2019-11-12 2023-04-18 山东大学 一种基于时空多特征提取的行为识别系统及其工作方法
CN111639680B (zh) * 2020-05-09 2022-08-09 西北工业大学 一种基于专家反馈机制的身份识别方法
CN111881793B (zh) * 2020-07-20 2024-03-01 东北大学 一种基于胶囊网络的非侵入式负荷监测方法与系统

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040098367A1 (en) * 2002-08-06 2004-05-20 Whitehead Institute For Biomedical Research Across platform and multiple dataset molecular classification
US20100131335A1 (en) * 2008-11-25 2010-05-27 Roh Dong-Hyun User interest mining method based on user behavior sensed in mobile device
CN102348101A (zh) * 2010-07-30 2012-02-08 深圳市先进智能技术研究所 一种考场智能监控系统和方法
CN104200113A (zh) * 2014-09-10 2014-12-10 山东农业大学 基于高斯过程的物联网数据不确定性度量、预测与野值剔除方法
CN104268577A (zh) * 2014-06-27 2015-01-07 大连理工大学 一种基于惯性传感器的人体行为识别方法
CN104598880A (zh) * 2015-03-06 2015-05-06 中山大学 一种基于模糊支持向量机的行为识别方法
CN105139029A (zh) * 2015-08-14 2015-12-09 哈尔滨华夏矿安科技有限公司 一种监狱服刑人员的行为识别方法及装置
CN109447162A (zh) * 2018-11-01 2019-03-08 山东大学 一种基于Lora和Capsule的实时行为识别系统及其工作方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11074495B2 (en) * 2013-02-28 2021-07-27 Z Advanced Computing, Inc. (Zac) System and method for extremely efficient image and pattern recognition and artificial intelligence platform
CN104615983B (zh) * 2015-01-28 2018-07-31 中国科学院自动化研究所 基于递归神经网络和人体骨架运动序列的行为识别方法
CN107092894A (zh) * 2017-04-28 2017-08-25 孙恩泽 一种基于lstm模型的运动行为识别方法
CN108345846A (zh) * 2018-01-29 2018-07-31 华东师范大学 一种基于卷积神经网络的人体行为识别方法及识别系统

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040098367A1 (en) * 2002-08-06 2004-05-20 Whitehead Institute For Biomedical Research Across platform and multiple dataset molecular classification
US20100131335A1 (en) * 2008-11-25 2010-05-27 Roh Dong-Hyun User interest mining method based on user behavior sensed in mobile device
CN102348101A (zh) * 2010-07-30 2012-02-08 深圳市先进智能技术研究所 一种考场智能监控系统和方法
CN104268577A (zh) * 2014-06-27 2015-01-07 大连理工大学 一种基于惯性传感器的人体行为识别方法
CN104200113A (zh) * 2014-09-10 2014-12-10 山东农业大学 基于高斯过程的物联网数据不确定性度量、预测与野值剔除方法
CN104598880A (zh) * 2015-03-06 2015-05-06 中山大学 一种基于模糊支持向量机的行为识别方法
CN105139029A (zh) * 2015-08-14 2015-12-09 哈尔滨华夏矿安科技有限公司 一种监狱服刑人员的行为识别方法及装置
CN109447162A (zh) * 2018-11-01 2019-03-08 山东大学 一种基于Lora和Capsule的实时行为识别系统及其工作方法

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111695342A (zh) * 2020-06-12 2020-09-22 复旦大学 基于语境信息的文本内容校正方法
CN111695342B (zh) * 2020-06-12 2023-04-25 复旦大学 基于语境信息的文本内容校正方法
CN111931882A (zh) * 2020-07-20 2020-11-13 五邑大学 货物自动结账方法、系统及存储介质
CN111931882B (zh) * 2020-07-20 2023-07-21 五邑大学 货物自动结账方法、系统及存储介质
CN113971399A (zh) * 2020-07-23 2022-01-25 北京金山数字娱乐科技有限公司 识别模型的训练方法及装置、文本识别方法及装置
CN112327189A (zh) * 2020-10-14 2021-02-05 北方工业大学 一种基于knn算法的储能电池健康状态综合判断方法
CN112327189B (zh) * 2020-10-14 2023-06-09 北方工业大学 一种基于knn算法的储能电池健康状态综合判断方法
CN116204844A (zh) * 2023-04-28 2023-06-02 西南石油大学 一种基于不确定性的电气设备异常数据清洗方法
CN116204844B (zh) * 2023-04-28 2023-07-04 西南石油大学 一种基于不确定性的电气设备异常数据清洗方法
CN116661330A (zh) * 2023-07-31 2023-08-29 深圳市云图数字科技有限公司 一种智能家居系统的协同运行方法及智能家居系统
CN116661330B (zh) * 2023-07-31 2023-12-01 深圳市云图数字科技有限公司 一种智能家居系统的协同运行方法及智能家居系统

Also Published As

Publication number Publication date
AU2019371325A1 (en) 2020-10-15
CN109447162A (zh) 2019-03-08
AU2019371325B2 (en) 2022-01-27
CN109447162B (zh) 2021-09-24

Similar Documents

Publication Publication Date Title
WO2020087848A1 (zh) 一种基于Lora和Capsule的实时行为识别系统及其工作方法
CN109698836B (zh) 一种基于深度学习的无线局域网入侵检测方法和系统
CN110414305A (zh) 人工智能卷积神经网络人脸识别系统
CN109362066B (zh) 一种基于低功耗广域物联网和胶囊网络的实时行为识别系统及其工作方法
US12008480B2 (en) Context information uncertainty elimination system and working method based on QoX adaptive management
CN109101602A (zh) 图像检索模型训练方法、图像检索方法、设备及存储介质
CN109377616A (zh) 一种基于二维人脸识别的门禁控制系统
CN110705500B (zh) 基于深度学习的人员工作图像的注意力检测方法及系统
WO2018120962A1 (zh) 一种基于可靠性管理的不确定性消除情景感知系统及其工作方法
CN104954210B (zh) 配电通信网中的不同业务类型与无线通信方式的匹配方法
CN112001347B (zh) 一种基于人体骨架形态与检测目标的动作识别方法
CN110222592B (zh) 一种基于互补时序行为提案生成的时序行为检测网络模型的构建方法
CN108563977A (zh) 一种高速公路出入口的行人预警方法及系统
US20230051411A1 (en) Image Processing Method and Apparatus, Computer Device, Storage Medium, and Program Product
US20230215006A1 (en) Method and system for generating a dynamic addictive neural circuits based on weakly supervised contrastive learning
CN113191479A (zh) 联合学习的方法、系统、节点及存储介质
CN116363748A (zh) 基于红外-可见光图像融合的电网现场作业一体管控方法
CN102360457A (zh) 一种自主完善网络结构的贝叶斯网络及本体联合推理的方法
CN113642403A (zh) 基于边缘计算的人群异动智能安全检测系统
Kumar et al. A survey on event detection approaches for sensor based IoT
CN113627326B (zh) 一种基于可穿戴设备和人体骨架的行为识别方法
CN114359578A (zh) 病虫害识别智能终端应用方法及系统
CN102867214A (zh) 区域范围内人数统计管理方法
CN108875030B (zh) 一种基于层次化综合质量指标QoX的上下文不确定性消除系统及其工作方法
CN113015120B (zh) 一种基于神经网络的治污监测系统和方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19877785

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019371325

Country of ref document: AU

Date of ref document: 20190323

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19877785

Country of ref document: EP

Kind code of ref document: A1