CN114758408B - Confusion behavior conflict management system and method based on multiple classifiers - Google Patents

Confusion behavior conflict management system and method based on multiple classifiers Download PDF

Info

Publication number
CN114758408B
CN114758408B CN202011589114.6A CN202011589114A CN114758408B CN 114758408 B CN114758408 B CN 114758408B CN 202011589114 A CN202011589114 A CN 202011589114A CN 114758408 B CN114758408 B CN 114758408B
Authority
CN
China
Prior art keywords
behavior
confusion
conflict
classifiers
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011589114.6A
Other languages
Chinese (zh)
Other versions
CN114758408A (en
Inventor
许宏吉
刘强
元辉
李娟�
孙晓杰
樊士迪
赵文杰
徐杰
周双
王猛猛
王宇豪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University
Original Assignee
Shandong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University filed Critical Shandong University
Priority to CN202011589114.6A priority Critical patent/CN114758408B/en
Publication of CN114758408A publication Critical patent/CN114758408A/en
Application granted granted Critical
Publication of CN114758408B publication Critical patent/CN114758408B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Probability & Statistics with Applications (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The invention provides a system and a method for confusing behavior conflict management based on multiple classifiers, which are used for collecting physiological information and motion information of a measured object; transmitting and storing physiological information and movement information; preprocessing physiological information and motion information to obtain input data of a plurality of classifiers, and inputting the preprocessed data into the plurality of classifiers for behavior recognition; acquiring classification results of a plurality of classifiers, determining whether a confusion behavior and the type of the confusion behavior exist among the classification results, and if so, triggering a conflict detection and elimination task, wherein the conflict detection task carries out label conversion by acquiring the probability of real labels and behavior information to determine whether conflicts exist or not, and the elimination task eliminates the conflicts among different classifiers; the recognition accuracy of the confounding behavior after the conflict is eliminated is calculated. The invention can improve the accuracy of behavior recognition.

Description

Confusion behavior conflict management system and method based on multiple classifiers
Technical Field
The invention belongs to the technical field of pattern recognition and data fusion, and particularly relates to a system and a method for confusion behavior conflict management based on multiple classifiers.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
The human body behavior recognition (Human Activity Recognition, HAR) system is a system for realizing behavior recognition by acquiring various behavior information of a human body and applying a reasonable model. In the last decade, the concept and technology of the internet of things have been rapidly developed by integrating traditional networks, wearable device sensors and networked objects. Meanwhile, the technology for identifying human behaviors is advanced to a new height due to the gradual maturity of main stream technologies such as big data, cloud computing and the like. Nowadays, human behavior recognition technology has been widely applied in the fields of smart home, electronic health, smart medical diagnosis, geriatric care and the like. At present, the behavior recognition model mainly adopts algorithms such as machine learning, deep learning and the like.
The development of human behavior recognition technology has greatly progressed so far, and certain breakthroughs are made in a feature acquisition layer and a complex behavior recognition layer. However, the behavior recognition algorithm is still immature, and no algorithm suitable for all behavior classifications exists at present. On the one hand, a single classifier cannot realize the recognition effect of high recognition rate on human behaviors with low complexity, so that the single classifier often can generate false recognition on certain characteristics and similar behaviors, for example: identifying "standing" as "sitting", identifying "going upstairs" as "going downstairs", etc., these confounding behaviors may cause a decrease in identification accuracy; on the other hand, there is also a large difference between different kinds of classifiers. Thus, experts in the field of pattern recognition have gradually developed deep research into multi-classifier fusion algorithms. The comprehensive use of a single classifier leads the fused multi-classifier to have obvious advantages and suppression defects, and has obvious improvement in classification precision and operation speed.
At present, a human behavior recognition system lacks an effective judging mode for the confusion behavior, and the human behavior recognition technology combines a multi-classifier fusion model and an algorithm, but the conflict generated during fusion of the multi-classifier aiming at the confusion behavior can directly influence the recognition accuracy of the confusion behavior. Therefore, how to eliminate conflicts between multiple classifications by a reasonable fusion algorithm is a challenge to improve the accuracy of recognition of confounding behaviors.
Disclosure of Invention
In order to solve the problems, the invention provides a system and a method for confusing behavior conflict management based on multiple classifiers, which can solve the problem of low recognition accuracy caused by conflict of confusing behaviors among the multiple classifiers and improve the accuracy of behavior recognition.
According to some embodiments, the present invention employs the following technical solutions:
a multi-classifier based confusion behavior conflict management system, comprising:
the behavior information acquisition module comprises a plurality of sensors distributed at different positions of the measured object and is used for acquiring physiological information and motion information of the measured object in real time;
The behavior information transmission module is configured to transmit the information acquired by the behavior information acquisition module to the behavior information processing module;
The behavior information processing module is configured to preprocess the physiological information and the motion information to obtain input data of a plurality of classifiers and input the preprocessed data into the plurality of classifiers for behavior recognition;
The confusion behavior judging module is configured to acquire classification results of the plurality of classifiers, determine whether confusion behaviors exist among the classification results and the types of the confusion behaviors, and if so, send signals to the conflict management module;
The conflict management module is configured to trigger a conflict detection task and a elimination task according to the received signals, wherein the conflict detection task carries out tag conversion by acquiring the probability of the real tag and the behavior information to determine whether the conflict exists, the elimination task eliminates the conflict among different classifiers, and the recognition accuracy of the confusion behavior after the conflict elimination is calculated.
As an alternative embodiment, the physiological information includes heart rate, blood oxygen, and skin resistance values, and the movement information includes triaxial acceleration, triaxial angular velocity, and triaxial magnetic induction information.
The triaxial acceleration comprises X-axis acceleration, Y-axis acceleration and Z-axis acceleration; the triaxial angular velocity includes an X-axis angular velocity, a Y-axis angular velocity, and a Z-axis angular velocity; the triaxial magnetic induction information comprises vectors of a geomagnetic field in a position and a resultant magnetic field generated by the circuit board device in X, Y, Z triaxial.
As an alternative implementation manner, the behavior information transmission module transmits the physiological information and the motion information of the tested object to the cloud server, and the behavior information processing module acquires the physiological information and the behavior information from the cloud server.
As an optional implementation manner, the behavior information transmission module includes a behavior information transmission mode selection unit and a behavior information wireless transmission unit that are sequentially connected, where:
the behavior information transmission mode selection unit is used for selecting a proper wireless transmission mode for transmission;
The behavior information wireless transmission unit is used for transmitting the behavior information according to the selected wireless transmission mode.
As an alternative embodiment, the behavior information processing module includes a behavior information preprocessing unit and a multi-classifier classification unit, where:
the behavior information preprocessing unit performs preprocessing operation according to the physiological information and the motion information to obtain input data adaptive to the multi-classifier;
the multi-classifier classifying unit classifies and identifies the preprocessed data in parallel and obtains the identification result of each classifier.
The confusion behavior judging module comprises a threshold triggering unit and a behavior information output unit, wherein:
The threshold triggering unit is configured to calculate the probability of false recognition of behaviors under each classifier, and if the probability exceeds a preset threshold, the serious confusion exists among different behaviors and the behavior information output unit is triggered;
The behavior information output unit is configured to output specific confused behavior types and probabilities of different classifiers on various behaviors to be identified.
As a further limitation, the preset threshold is adaptively updated according to the recognition results of different classifier combinations and the confusion matrix.
As an alternative embodiment, the conflict management module includes a conflict detection unit and a conflict resolution unit connected in sequence, where:
the conflict detection unit is used for detecting whether conflicts exist among multiple classifiers aiming at confusion behaviors.
The conflict elimination unit is used for eliminating conflicts among multiple classifiers aiming at confusion behaviors and obtaining a final fusion result.
As a further limitation, the collision detection unit includes a true tag and behavior information probability acquisition subunit, a tag conversion subunit, and a multi-classifier collision detection subunit connected in sequence, wherein:
The true tag and behavior information probability acquisition subunit is used for acquiring Softmax layer output probabilities of different classifiers;
The tag conversion subunit is used for determining the output probability of the Softmax layer to be obtained according to the row index value of the confusion behavior in the real tag, and converting the judging result of the behavior types output by the three classification models into different tag types as the identification framework of the follow-up D-S evidence theory;
the multi-classifier conflict detection subunit is configured to detect whether there is a conflict between the multi-classifiers for the confounding behavior.
As a further limitation, the conflict elimination unit comprises an entropy value calculation subunit, an entropy value and credibility mapping subunit, a credibility normalization subunit and a D-S evidence theory fusion judgment subunit which are sequentially connected, wherein the behavior information probability acquisition subunit is connected with the entropy value calculation subunit, and the multi-classifier conflict detection subunit is connected with the D-S evidence theory fusion judgment subunit;
the entropy value calculating subunit is used for calculating entropy values of different classifiers aiming at the confusion behavior probability;
the entropy value and credibility mapping subunit obtains credibility of focal elements of different classifiers under the identification frame through a mapping relation established between the entropy value and uncertain focal elements;
the credibility normalization subunit is used for normalizing the credibility of the coke elements under different classifiers, so that the basic condition of the D-S evidence theory is satisfied;
the D-S evidence theory fusion judgment subunit is used for eliminating conflicts among different classifiers according to the evidence combination rules and obtaining a final fusion judgment result.
A confusing behavior conflict management method based on multiple classifiers includes the following steps:
collecting physiological information and motion information of a measured object;
Transmitting and storing physiological information and movement information;
Preprocessing physiological information and motion information to obtain input data of a plurality of classifiers, and inputting the preprocessed data into the plurality of classifiers for behavior recognition;
acquiring classification results of a plurality of classifiers, determining whether a confusion behavior and the type of the confusion behavior exist among the classification results, and if so, triggering a conflict detection and elimination task, wherein the conflict detection task carries out label conversion by acquiring the probability of real labels and behavior information to determine whether conflicts exist or not, and the elimination task eliminates the conflicts among different classifiers;
the recognition accuracy of the confounding behavior after the conflict is eliminated is calculated.
As an alternative embodiment, the specific process of determining whether the confusion behavior and the type of the confusion behavior exist among the classification results comprises the following steps:
And analyzing the confusion matrixes obtained by the different classifiers, judging whether the confusion behaviors exist or not according to a preset threshold value, outputting the types of the confusion behaviors, and judging that the confusion behaviors exist if the types of the confusion behaviors exceed the preset threshold value.
As an alternative embodiment, the conflict detection and elimination task procedure includes:
Acquiring a real label of a behavior to be identified, determining a column index of a mixed behavior according to the real label, and acquiring probability vectors of various behaviors output by different classifiers according to the index;
Calculating column indexes corresponding to the maximum probability values according to probability vectors of different types of behaviors in different classifiers, and carrying out corresponding marking;
According to the marking sequences of different classifiers, determining whether conflict detection exists between the different classifiers, and if not, selecting the marking sequence of any classifier as an output result;
If the entropy exists, acquiring entropy values of uncertain focal elements by different classifiers, selecting a proper mapping relation according to the acquired entropy values to obtain initial credibility distribution of uncertain focal elements, carrying out normalization processing on the initial credibility of each focal element, carrying out fusion judgment on the normalized result, and outputting a final judgment result.
Compared with the prior art, the invention has the beneficial effects that:
The invention can actively select the behavior information acquisition mode so as to adapt to the acquisition requirements of different scenes;
The invention can judge the occurrence and specific type of the confusion behavior in real time according to the classification results of different classifiers and the confusion matrix, and detect and eliminate the conflict among multiple classifiers in real time according to the D-S evidence theory, thereby improving the recognition accuracy of the confusion behavior.
The single classifier aims at the problem that the recognition accuracy is improved in terms of algorithm time consumption and space complexity, and is not beneficial to the improvement of the overall performance of the system. According to the method, the conflict existing between different classifiers aiming at the confusion behavior is eliminated according to the D-S evidence theory, the recognition accuracy of the confusion behavior is improved, and the energy consumption of a system is reduced.
In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.
FIG. 1 is a diagram showing the module composition and connection relationship of a multi-classifier-based confusion behavior conflict management system according to the present invention.
FIG. 2 is a block diagram of a system for confusing behavior conflict management based on multiple classifiers in accordance with the present invention.
FIG. 3 is a flowchart illustrating a system for confusing behavior conflict management based on multiple classifiers in accordance with the present invention.
FIG. 4 is a diagram of a multi-classifier based confusion matrix of the multi-classifier based confusion behavior conflict management system of the present invention in processing WISDM datasets.
The specific embodiment is as follows:
The invention will be further described with reference to the drawings and examples.
It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present invention. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
Example 1
A multi-classifier based confusion behavior conflict management system, as shown in fig. 1, comprising: the system comprises a behavior information acquisition module, a behavior information transmission module, a behavior information processing module, a confusion behavior judgment module and a conflict management module.
The system comprises a behavior information acquisition module, a behavior information transmission module, a behavior information processing module, a confusion behavior judgment module and a conflict management module which are sequentially connected.
The behavior information acquisition module is used for acquiring physiological information and behavior information of a human body in real time, wherein the physiological information comprises heart rate, blood oxygen and skin resistance value. The motion information includes triaxial acceleration, triaxial angular velocity, and triaxial magnetic induction information. The three-axis acceleration comprises X-axis acceleration, Y-axis acceleration and Z-axis acceleration; the triaxial angular velocity includes an X-axis angular velocity, a Y-axis angular velocity, and a Z-axis angular velocity; the triaxial magnetic induction information comprises vectors of a geomagnetic field in a position and a resultant magnetic field generated by the circuit board device in X, Y, Z triaxial. The behavior information transmission module is used for transmitting the acquired physiological information and motion information of the human body to the cloud server. The behavior information processing module is used for acquiring human physiological information and behavior information of the cloud server, preprocessing the information to obtain input data of the plurality of classifiers, and inputting the preprocessed data into the plurality of classifiers for behavior recognition. The confusion behavior judging module is used for judging whether the confusion behaviors exist or not and the types of the confusion behaviors in real time according to the classification results of the multi-classifier. The conflict management module is used for triggering conflict detection and elimination and obtaining the recognition accuracy of the confusion behavior after the conflict elimination.
As shown in fig. 2, the behavior information acquisition module includes an information type selection unit, a physiological information sensing unit, and a motion information sensing unit.
The information type selection unit is respectively connected with the physiological information sensing unit and the motion information sensing unit. The information type selection unit is used for triggering the physiological information sensing unit or the movement information sensing unit.
The physiological information sensing unit is used for acquiring physiological information such as heart rate, blood oxygen, skin resistance value and the like of a human body.
The motion information sensing unit is used for acquiring nine-axis motion information of a human body, including three-axis acceleration, three-axis angular velocity and three-axis magnetic induction information.
According to the invention, the behavior information transmission module comprises a behavior information transmission mode selection unit and a behavior information wireless transmission unit which are connected in sequence.
The behavior information transmission mode selection unit is used for selecting different types of transmission modes such as LoRa, wiFi, 4G/5G, but is not limited to the wireless transmission modes.
The behavior information wireless transmission unit is used for transmitting the behavior information according to the selected wireless transmission mode.
According to the invention, the behavior information processing module comprises a behavior information preprocessing unit and a multi-classifier classifying unit which are connected in sequence.
The behavior information preprocessing unit performs preprocessing operations such as sliding window segmentation and the like according to human physiological information and motion information acquired from the cloud server end so as to obtain input data of the multi-classifier.
The multi-classifier classifying unit classifies and identifies the preprocessed data in parallel and obtains the identification result of each classifier.
According to the invention, the confusion behavior judging module comprises a threshold triggering unit and a behavior information output unit which are connected in sequence.
The threshold triggering unit is used for calculating the probability of false recognition of behaviors under each classifier, and if the probability exceeds a threshold preset by the system, the system considers that serious confusion exists among different behaviors and triggers the behavior information output unit. For example, the system selects three different classifiers Conv2D, googLeNe, resNet to perform behavior recognition on the input data, the three classifiers output three different confusion matrices, and the system analyzes according to the three different confusion matrices. Taking the confusion matrix of Conv2D output as an example, the recognition accuracy of the six behaviors of "going downstairs", "jogging", "sitting", "standing", "going upstairs" and "walking" are respectively 85.9%, 96.0%, 97.8%, 98.0%, 87.7% and 98.7%, the recognition accuracy of the four behaviors of "going downstairs" and "going upstairs" is lower, and the recognition accuracy of the four behaviors of "jogging", "sitting", "standing" and "walking" is higher, and although the four behaviors may be confused to some extent, the confusion degree is lower than that of the two behaviors of "downstairs" and "going upstairs". Meanwhile, the probability of recognizing "going downstairs" as "going upstairs" is 12.6%, and the probability of recognizing "going upstairs" as "going downstairs" is 7.6%. Assuming that the preset threshold value of the system is 15%, the sum of the probabilities of the two behaviors of going upstairs and going downstairs is 20.2%, and the probability exceeds the preset threshold value of the system, the threshold value triggering unit is triggered, and the system determines that the two behaviors are mixed in real time. The threshold is not fixed since different classifier combinations will output different confusion matrices. For each preprocessed data, the system adaptively updates the threshold according to the recognition results of different classifier combinations and the confusion matrix.
The behavior information output unit is used for outputting specific confused behavior types and probabilities of different classifiers for various behaviors output through a Softmax function.
According to the present invention, the collision management module preferably includes a collision detection unit and a collision elimination unit connected in sequence.
The conflict detection unit is used for detecting whether conflicts exist among the multiple classifiers aiming at the confusion behaviors.
The conflict elimination unit is used for eliminating conflicts among multiple classifiers aiming at the confusion behaviors and obtaining a final fusion result.
According to the invention, the conflict detection unit comprises a true tag and behavior information probability acquisition subunit, a tag conversion subunit and a multi-classifier conflict detection subunit which are connected in sequence.
The true tag and behavior information probability acquisition subunit is used for acquiring Softmax layer output probabilities of different classifiers.
The tag conversion subunit is used for determining the output probability of the Softmax layer to be obtained according to the row index value of the confusion behavior in the real tag, and converting the judging result of the behavior types output by the three classification models into three tag types of 0, 1 and 2, and the three tag types are used as the recognition framework of the follow-up D-S evidence theory. Where "0" means "upstairs", "1" means "downstairs", "2" means that other behavior types are identified that are neither "upstairs" nor "downstairs".
The multi-classifier conflict detection subunit is configured to detect whether a conflict exists between the multi-classifiers for the confounding behavior.
According to the invention, the conflict elimination unit comprises an entropy value calculation subunit, an entropy value and credibility mapping subunit, a credibility normalization subunit and a D-S evidence theory fusion judgment subunit which are connected in sequence. Meanwhile, the behavior information probability acquisition subunit is connected with the entropy value calculation subunit.
The entropy value calculating subunit is used for calculating entropy values of different classifiers aiming at the confusion behavior probabilities.
The entropy value and credibility mapping subunit obtains the credibility of the focal elements of different classifiers under the identification framework through the mapping relation established between the entropy value and the uncertain focal elements.
The credibility normalization subunit is used for normalizing the credibility of the coke elements under different classifiers, so that the basic condition of the D-S evidence theory is satisfied.
The D-S evidence theory fusion judgment subunit is used for eliminating conflict among different classifiers according to the evidence combination rule and obtaining a final fusion judgment result.
Example 2
In this embodiment, a working method of the multi-classifier-based confusion behavior conflict management system is provided, as shown in fig. 3, in which the system actively selects a motion information sensing unit to obtain triaxial acceleration of a human body, and three kinds of classifiers Conv2D, googLeNet, resNet are used to perform human behavior recognition on the preprocessed data in parallel.
Step S01: behavior information acquisition mode selection
The behavior information collection mode set by the system is a motion information collection mode. According to the requirements of the behavior information acquisition modes required by behavior identification, an accelerometer, an angular velocity meter and a magnetometer are selected as sensors of motion information, and the motion information is acquired.
Step S02: behavior information wireless transmission mode selection
The wireless transmission modes of the behavior information set by the system comprise LoRa transmission, wiFi transmission, 4G/5G transmission and the like, but are not limited to the wireless transmission modes. Assuming that a wireless transmission mode of behavior information set by the system is LoRa transmission, the cloud server stores the transmitted behavior information.
Step S03: wireless transmission of behavioral information
And the system transmits the behavior information according to the selected wireless transmission mode.
Step S04: cloud server storage
And the cloud server stores the transmitted behavior information in real time.
Step S05: behavior information acquisition
According to the system requirements, the motion information can be acquired at the cloud server side for behavior recognition or analyzed by acquiring the motion information.
Step S06: behavior information preprocessing
Human physiological information and motion information acquired from the cloud server side are subjected to preprocessing operations such as sliding window segmentation. Each data block after segmentation represents a complete behavior. For example, behavior recognition is based on only three-axis acceleration, where the data may be represented as X ε R n×3, where n represents the total amount of X-axis (Y-axis or Z-axis) data and 3 represents three-axis data using acceleration. The system can preset the size of the sliding window and the sliding mode of the window, if the sampling frequency of the sensor is f, the size selectable range of the sliding window is generally 0.5f-10f, and the sliding mode of the window defaults to a window overlapping rate of 50%. Under such parameter settings, the data may be represented as X i∈Rnwindow X3,i ε [1, N ], where N represents the total amount of data blocks after segmentation and N window represents the size of the sliding window.
Step S07: multi-classifier class identification
The preprocessed data are input into different classifiers, such as Conv2D model, googLeNet model and ResNet model, in parallel, so as to obtain classification results and confusion matrixes under different classifiers.
Step S08: real-time determination of confounding behavior
The system analyzes the confusion matrixes obtained by different classifiers, judges whether the confusion behaviors exist or not according to a preset threshold value, and outputs the types of the confusion behaviors. For example, the Conv2D model has recognition accuracy rates of 85.9%, 96.0%, 97.8%, 98.0%, 87.7%, 98.7% for the six behaviors of "downstairs", "jogging", "sitting", "standing", "upstairs" and "walking", respectively, and the recognition accuracy rates of "downstairs" and "upstairs" are low and are highly likely to be confused. Meanwhile, the probability of recognizing "going downstairs" as "going upstairs" is 12.6%, and the probability of recognizing "going upstairs" as "going downstairs" is 7.6%. Assuming that the preset threshold value of the system is 15%, the sum of the probabilities of the two behaviors identifying errors is 20.2%, and the probability exceeds the preset threshold value of the system, then the system judges that the upstairs and downstairs are the behaviors with confusion in real time.
Step S09: true tag and behavior information probability acquisition
The system first obtains the real tag and determines the column index for the confounding behavior from the real tag. For example, there are six behaviors to be identified for WISDM datasets, respectively, "downstairs", "jogging", "sitting", "standing", "upstairs", "walking". The real label is an m x 6 dimensional matrix, where m represents the number of complete behavior cycles and 6 represents six classes of behavior. And outputting a final recognition result of the classification model by adopting a one-hot coding mode for each row of the matrix. For example, [ 100000 ] is a line of acquisition from a real tag that represents that the identified behavior is "going downstairs". According to step S08, the confusing actions are "downstairs" and "upstairs" with column indexes of 1 and 5 in the real tag, respectively, so that "downstairs" is encoded as 1 and "upstairs" is encoded as 0. Meanwhile, the system performs exclusive-or operation on the codes of the two columns, and if the exclusive-or result is 1, the corresponding row index is reserved; otherwise, it is discarded. For example, "downstairs" is encoded as 1, and "upstairs" is encoded as 0, and the exclusive or result of the two is 1, at this time, the system reserves the corresponding row index, and obtains probability vectors of six types of behaviors output by different classifier Softmax layers according to the reserved row index. For example, the retained row index is 15, and the output probability of the Conv2D model corresponding thereto for six classes of behaviors through a Softmax function is [0.9749 0.0001 0.0000 0.0000 0.0249 0.0001], the output probability of the GoogLeNet model for six classes of behaviors through a Softmax function is [0.3767 0.0000 0.0000 0.0000 0.6233 0.0000], and the output probability of the ResNet model for six classes of behaviors through a Softmax function is [0.9245 0.0001 0.0000 0.0000 0.0753 0.0000]. The system then completes the acquisition of the probability of behavior information.
Step S10: tag conversion
The system calculates a column index corresponding to the maximum probability value of each row according to probability vectors of different types of behaviors in different classifiers, if the column index is 5, the column index is marked as '0', if the column index is 1, the column index is marked as '1', and other cases are marked as '2', at the moment, the frame Θ= {0,1,2}. For example, the output probability of the Conv2D model for six classes of behavior via the Softmax function is [0.3850 0.0043 0.0001 0.0001 0.5810 0.0295], the output probability of the GoogLeNet model for six classes of behavior via the Softmax function is [0.1442 0.0620 0.0001 0.0001 0.3756 0.4180], and the output probability of the ResNet model for six classes of behavior via the Softmax function is [0.6788 0.0131 0.0001 0.0000 0.2615 0.0465]. The Conv2D model has a column index of 5 for the maximum of six classes of behavior probability vectors, and is therefore labeled "0"; the GoogeLeNet model has a column index of 6 for the maximum of the six classes of behavior probability vectors, and is therefore labeled "2"; the ResNet model has a column index of 1 for the maximum of the six classes of behavior probability vectors and is therefore labeled "1". Thus, each classifier is subjected to a tag conversion process to obtain a tag sequence containing "0", "1" and "2".
Step S11: multi-classifier collision detection
The system detects conflict of the marking sequences of different classifiers, if the sum of each column is 3 or 0, the conflict of the confusion behaviors among different classifiers is not shown, otherwise, the conflict of the confusion behaviors among different classifiers is shown.
Step S12: entropy calculation
Since 2 Θ = {0,1,2, {0,1}, {0,2}, {1,0,2}, phi }, the focal elements {0,1}, {0,2}, {1,2}, {0,1,2} have the ability to directly express "uncertainty", the source at this time is {0}, {1}, {2}, and the source entropy in the information theory can quantify the "uncertainty" degree of the source. Thus, in the D-S evidence theory, the entropy values of the computational focal elements {0,1}, {0,2}, {1,2}, {0,1,2} may specifically represent the degree of "uncertainty". For example, the probability of the focal element {1} is p ({ 1 }), the probability of the focal element {0} is p ({ 0 }), and the probability of the focal element {2} is 1-p ({ 1 }) -p ({ 0 }), so the calculation process of the entropy values of the focal elements {0,1,2} is as follows:
H({0,1,2})=-p({0})·log3[p({0})]-p({1})·log3[p({1})]-p({2})·log3[p({2})](1)
Similarly, entropy values of other focal elements can be calculated according to the above formula. Thus, the system can obtain entropy values for the uncertain focal elements for the different classifiers.
Step S13: entropy and confidence map
The system selects a proper mapping relation according to the acquired entropy value to obtain the initial credibility allocation (Basic Probability Assignment, BPA) of the uncertain focal element. For example, an exponential mapping relationship
f(H)=H·e1-H(H∈[0,1]) (2)
The value range between [0,1] is satisfied, f (H) approaches 0 when the entropy value H approaches 0, f (H) approaches 1 when the entropy value H approaches 1, and the derivative of f (H) is 0 at this time. For example, the BPA calculation formula for the focal element {0,1,2} is as follows:
m({0,1,2})=H({0,1,2})·e1-H({0,1,2}) (3)
thus, through the mapping relation, the system can obtain BPA of uncertain focal elements of different classifiers.
Step S14: confidence normalization
In order to meet the basic initial conditions of the D-S evidence theory, the system needs to normalize BPA for each focal element, e.g., BPA normalized for focal element {0,1,2} as follows:
Therefore, the system can obtain BPA normalized by each focal element of different classifiers according to the above process.
Therefore, the system can obtain BPA normalized by each focal element of different classifiers according to the above process.
Step S15: D-S evidence theory fusion judgment
If no conflict exists among the multiple classifiers, the system can select a marking sequence of any classifier as an output result; if the conflict exists among the multiple classifiers, the system carries out fusion judgment on the BPA normalized by each focal element according to the acquired different classifiers and outputs a final judgment result.
Step S16: calculation of recognition accuracy of confusion behavior after eliminating multi-classifier conflict
The system actively calculates the recognition accuracy of the confusion behavior after eliminating the conflict among multiple classifiers aiming at the confusion behavior. For example, the Conv2D model has an identification accuracy of 85.90% for "downstairs" behaviors and 87.70% for "upstairs" behaviors; the recognition accuracy of GoogLeNet model for going downstairs is 87.35%, and the recognition accuracy for going upstairs is 87.02%; the ResNet model has the identification accuracy rate of 80.37% for downstairs and 85.10% for upstairs. After the conflict among different classification models is eliminated through the D-S evidence theory, the identification accuracy of the downstairs behavior is 88.41%, the identification accuracy of the upstairs behavior is 92.89%, and the identification accuracy of the two kinds of confusion behaviors is obviously improved compared with that of a single classification model.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
While the foregoing description of the embodiments of the present invention has been presented in conjunction with the drawings, it should be understood that it is not intended to limit the scope of the invention, but rather, it is intended to cover all modifications or variations within the scope of the invention as defined by the claims of the present invention.

Claims (7)

1. A confusion behavior conflict management system based on multiple classifiers is characterized in that: comprising the following steps:
the behavior information acquisition module comprises a plurality of sensors distributed at different positions of the measured object and is used for acquiring physiological information and motion information of the measured object in real time;
The behavior information transmission module is configured to transmit the information acquired by the behavior information acquisition module to the behavior information processing module;
The behavior information processing module is configured to preprocess the physiological information and the motion information to obtain input data of a plurality of classifiers and input the preprocessed data into the plurality of classifiers for behavior recognition;
the confusion behavior judging module is configured to acquire classification results of the plurality of classifiers, determine whether confusion behaviors and types of the confusion behaviors exist among the classification results of each classifier, and if so, send signals to the conflict management module;
The confusion behavior judging module comprises a threshold triggering unit and a behavior information output unit, wherein: the threshold triggering unit is configured to calculate the probability of false recognition of behaviors under each classifier, and if the probability exceeds a preset threshold, the behaviors are considered to be mixed and the behavior information output unit is triggered; the behavior information output unit is configured to output specific confused behavior types and probabilities of various types of behaviors, which are predetermined by different classifiers, of behaviors to be identified;
the preset threshold value is adaptively updated according to the recognition results of different classifier combinations;
The conflict management module is configured to trigger a conflict detection task and a conflict elimination task according to the received signals, the conflict detection task performs label conversion by acquiring the probability of the real labels and the behavior information to determine whether the conflict exists, the conflict elimination task eliminates the conflict among different classifiers, and the recognition accuracy of the confusion behavior after the conflict elimination is calculated;
The conflict management module comprises a conflict detection unit and a conflict elimination unit which are sequentially connected, wherein: the conflict detection unit is used for detecting whether conflicts exist among the multiple classifiers aiming at the confusion behavior; the conflict elimination unit is used for eliminating conflicts among multiple classifiers aiming at the confusion behaviors and obtaining a final fusion result;
The conflict elimination unit comprises an entropy value calculation subunit, an entropy value and credibility mapping subunit, a credibility normalization subunit and a D-S evidence theory fusion judgment subunit which are sequentially connected, wherein the behavior information probability acquisition subunit is connected with the entropy value calculation subunit, and the multi-classifier conflict detection subunit is connected with the D-S evidence theory fusion judgment subunit;
the entropy value calculating subunit is used for calculating entropy values of different classifiers aiming at the confusion behavior probability;
the entropy value and credibility mapping subunit obtains credibility of focal elements of different classifiers under the identification frame through a mapping relation established between the entropy value and uncertain focal elements;
the credibility normalization subunit is used for normalizing the credibility of the coke elements under different classifiers, so that the basic condition of the D-S evidence theory is satisfied;
the D-S evidence theory fusion judgment subunit is used for eliminating conflicts among different classifiers according to the evidence combination rules and obtaining a final fusion judgment result.
2. The multi-classifier based confusion behavior conflict management system of claim 1, wherein: the physiological information includes heart rate, blood oxygen and skin resistance values, and the movement information includes triaxial acceleration, triaxial angular velocity and triaxial magnetic induction information.
3. The multi-classifier based confusion behavior conflict management system of claim 1, wherein: the behavior information transmission module transmits the physiological information and the motion information of the measured object to the cloud server, and the behavior information processing module acquires the physiological information and the behavior information from the cloud server.
4. The multi-classifier based confusion behavior conflict management system of claim 1, wherein: the conflict detection unit comprises a true tag and behavior information probability acquisition subunit, a tag conversion subunit and a multi-classifier conflict detection subunit which are sequentially connected, wherein:
The true tag and behavior information probability acquisition subunit is used for acquiring Softmax layer output probabilities of different classifiers;
The tag conversion subunit is used for determining the output probability of the Softmax layer to be obtained according to the row index value of the confusion behavior in the real tag, and converting the judging result of the behavior types output by the three classification models into different tag types as the identification framework of the follow-up D-S evidence theory;
the multi-classifier conflict detection subunit is configured to detect whether there is a conflict between the multi-classifiers for the confounding behavior.
5. The multi-classifier-based confusion behavior conflict management method applied to the multi-classifier-based confusion behavior conflict management system as claimed in claim 1, wherein the method is characterized in that: the method comprises the following steps:
collecting physiological information and motion information of a measured object;
Transmitting and storing physiological information and movement information;
Preprocessing physiological information and motion information to obtain input data of a plurality of classifiers, and inputting the preprocessed data into the plurality of classifiers for behavior recognition;
acquiring classification results of a plurality of classifiers, determining whether a confusion behavior and the type of the confusion behavior exist among the classification results, and if so, triggering a conflict detection and elimination task, wherein the conflict detection task carries out label conversion by acquiring the probability of real labels and behavior information to determine whether conflicts exist or not, and the elimination task eliminates the conflicts among different classifiers;
the recognition accuracy of the confounding behavior after the conflict is eliminated is calculated.
6. The method for managing confusion behavior collision based on multiple classifiers according to claim 5, wherein: the specific process for determining whether the confusion behaviors exist among the classification results comprises the following steps:
Analyzing the confusion matrixes obtained by different classifiers, judging whether the confusion behaviors exist or not according to a preset threshold value, outputting the types of the confusion behaviors, calculating the probability of misidentification of the behaviors under each classifier, and judging that the confusion behaviors exist if the probability exceeds the preset threshold value.
7. The method for managing confusion behavior collision based on multiple classifiers according to claim 5, wherein: the conflict detection and elimination task process includes:
Acquiring a real label of a behavior to be identified, determining a column index of a mixed behavior according to the real label, and acquiring probability vectors of various behaviors output by different classifiers according to the index;
Calculating column indexes corresponding to the maximum probability values according to probability vectors of different types of behaviors in different classifiers, and carrying out corresponding marking;
According to the marking sequences of different classifiers, determining whether conflict detection exists between the different classifiers, and if not, selecting the marking sequence of any classifier as an output result;
If the entropy exists, entropy values of the uncertain focal elements of different classifiers are obtained, the mapping relation is selected according to the obtained entropy values to obtain initial credibility distribution of the uncertain focal elements, normalization processing is carried out on the initial credibility of each focal element, fusion judgment is carried out on the normalized results, and a final judgment result is output.
CN202011589114.6A 2020-12-28 2020-12-28 Confusion behavior conflict management system and method based on multiple classifiers Active CN114758408B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011589114.6A CN114758408B (en) 2020-12-28 2020-12-28 Confusion behavior conflict management system and method based on multiple classifiers

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011589114.6A CN114758408B (en) 2020-12-28 2020-12-28 Confusion behavior conflict management system and method based on multiple classifiers

Publications (2)

Publication Number Publication Date
CN114758408A CN114758408A (en) 2022-07-15
CN114758408B true CN114758408B (en) 2024-05-07

Family

ID=82324547

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011589114.6A Active CN114758408B (en) 2020-12-28 2020-12-28 Confusion behavior conflict management system and method based on multiple classifiers

Country Status (1)

Country Link
CN (1) CN114758408B (en)

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101639864A (en) * 2009-08-18 2010-02-03 东南大学 Multi-level hierarchical DSmT rapid approximate reasoning fusion method
US8032467B1 (en) * 2008-05-31 2011-10-04 Hrl Laboratories, Llc Valuation-based learning system
CN103093238A (en) * 2013-01-15 2013-05-08 江苏大学 Visual dictionary construction method based on Dempster-Shafer (D-S) evidence theory
CN106056163A (en) * 2016-06-08 2016-10-26 重庆邮电大学 Multi-sensor information fusion object identification method
CN107368854A (en) * 2017-07-20 2017-11-21 华北电力大学(保定) A kind of circuit breaker failure diagnostic method based on improvement evidence theory
CN109540520A (en) * 2018-11-29 2019-03-29 中国船舶重工集团海装风电股份有限公司 A kind of rolling bearing fault fusion diagnosis method based on improvement D-S evidence theory
CN110441060A (en) * 2019-08-07 2019-11-12 贵州电网有限责任公司 One kind being based on mechanical failure diagnostic method combined of multi-sensor information
CN110826626A (en) * 2019-11-06 2020-02-21 西安交通大学 Confusion matrix-based multi-classifier difference measurement method and system
CN110852382A (en) * 2019-11-12 2020-02-28 山东大学 Behavior recognition system based on space-time multi-feature extraction and working method thereof
CN111027487A (en) * 2019-12-11 2020-04-17 山东大学 Behavior recognition system, method, medium, and apparatus based on multi-convolution kernel residual network
CN111709487A (en) * 2020-06-22 2020-09-25 中国科学院空天信息创新研究院 Underwater multi-source acoustic image substrate classification method and system based on decision-level fusion

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040236611A1 (en) * 2003-04-30 2004-11-25 Ge Financial Assurance Holdings, Inc. System and process for a neural network classification for insurance underwriting suitable for use by an automated system
US20050286772A1 (en) * 2004-06-24 2005-12-29 Lockheed Martin Corporation Multiple classifier system with voting arbitration

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8032467B1 (en) * 2008-05-31 2011-10-04 Hrl Laboratories, Llc Valuation-based learning system
CN101639864A (en) * 2009-08-18 2010-02-03 东南大学 Multi-level hierarchical DSmT rapid approximate reasoning fusion method
CN103093238A (en) * 2013-01-15 2013-05-08 江苏大学 Visual dictionary construction method based on Dempster-Shafer (D-S) evidence theory
CN106056163A (en) * 2016-06-08 2016-10-26 重庆邮电大学 Multi-sensor information fusion object identification method
CN107368854A (en) * 2017-07-20 2017-11-21 华北电力大学(保定) A kind of circuit breaker failure diagnostic method based on improvement evidence theory
CN109540520A (en) * 2018-11-29 2019-03-29 中国船舶重工集团海装风电股份有限公司 A kind of rolling bearing fault fusion diagnosis method based on improvement D-S evidence theory
CN110441060A (en) * 2019-08-07 2019-11-12 贵州电网有限责任公司 One kind being based on mechanical failure diagnostic method combined of multi-sensor information
CN110826626A (en) * 2019-11-06 2020-02-21 西安交通大学 Confusion matrix-based multi-classifier difference measurement method and system
CN110852382A (en) * 2019-11-12 2020-02-28 山东大学 Behavior recognition system based on space-time multi-feature extraction and working method thereof
CN111027487A (en) * 2019-12-11 2020-04-17 山东大学 Behavior recognition system, method, medium, and apparatus based on multi-convolution kernel residual network
CN111709487A (en) * 2020-06-22 2020-09-25 中国科学院空天信息创新研究院 Underwater multi-source acoustic image substrate classification method and system based on decision-level fusion

Non-Patent Citations (8)

* Cited by examiner, † Cited by third party
Title
A new definition of entropy of belief functions in the Dempster-Shafer theory;Radim Jirousek et al.;International Journal of Approximate Reasoning;20171010;49-63 *
D-S证据理论与信息熵结合的新算法;康健;李一兵;谢红;林云;;弹箭与制导学报;20110215(第01期);197-200 *
Uncertain Information Fusion for Gearbox Fault Diagnosis Based on BP Neural Network and DS Evidence Theory;Jie Chen et al.;2016 12th World Congress on Intelligent Control and Automation (WCICA);20160615;1372-1376 *
Uncertainty measure based on Tsallis entropy in evidence theory;Xiaozhuan Gao et al.;International Journal of Intelligent Systems;20191231;3105-3120 *
冲突可衡量多源信息融合技术研究;陈超;陈性元;汪永伟;王义功;;计算机科学;20130915(第09期);111-115 *
基于D-S理论的冲突证据融合算法;李嘉仪;中国优秀硕士学位论文全文数据库 信息科技辑;20200315(第03期);第2-5章 *
多源数据融合系统框架与证据理论算法研究及其在智慧医疗中的应用;杜保臻;中国优秀硕士学位论文全文数据库 信息科技辑;20190915(第09期);第一章-第四章 *
用户无关的多分类器融合行为识别模型;贺炎;王科;王忠民;;西安邮电大学学报;20160910(第05期);50-54 *

Also Published As

Publication number Publication date
CN114758408A (en) 2022-07-15

Similar Documents

Publication Publication Date Title
CN111027487B (en) Behavior recognition system, method, medium and equipment based on multi-convolution kernel residual error network
Gupta et al. A survey on human activity recognition and classification
JP4590018B1 (en) Feature value conversion apparatus and feature value conversion method
Wang et al. Human activity recognition with user-free accelerometers in the sensor networks
US11630506B2 (en) Methods and apparatus for machine learning-based movement recognition
Huynh-The et al. Physical activity recognition with statistical-deep fusion model using multiple sensory data for smart health
Ahmed et al. An approach to classify human activities in real-time from smartphone sensor data
CN110084286A (en) A kind of human motion recognition method of sensor-based ECOC technology
CN114818788A (en) Tracking target state identification method and device based on millimeter wave perception
CN115205891A (en) Personnel behavior recognition model training method, behavior recognition method and device
CN111191632B (en) Gesture recognition method and system based on infrared reflective glove
CN110598599A (en) Method and device for detecting abnormal gait of human body based on Gabor atomic decomposition
CN117643461B (en) Heart rate intelligent monitoring system and method based on artificial intelligence
Georgakopoulos et al. Change detection and convolution neural networks for fall recognition
CN114758408B (en) Confusion behavior conflict management system and method based on multiple classifiers
Alqarni Error-less data fusion for posture detection using smart healthcare systems and wearable sensors for patient monitoring
Indrasiri et al. Image Encoded Time Series Classification of Small Datasets: An Innovative Architecture Using Deep Learning Ensembles
Uslu et al. A segmentation scheme for knowledge discovery in human activity spotting
Yan et al. Topological nonlinear analysis of dynamical systems in wearable sensor-based human physical activity inference
CN111861275B (en) Household work mode identification method and device
İsmail et al. Human activity recognition based on smartphone sensor data using cnn
Liu et al. Mctn: A multi-channel temporal network for wearable fall prediction
Giuberti et al. Simple and robust BSN-based activity classification: Winning the first BSN contest
KR102316556B1 (en) Wearable device based hand gesture recognizer using adaptive dynamic time warping window and method using thereof
Wang et al. Realization of wearable sensors-based human activity recognition with an augmented feature group

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant