CN106815603B - Indoor activity detection and identification method and system based on multiple sensor networks - Google Patents
Indoor activity detection and identification method and system based on multiple sensor networks Download PDFInfo
- Publication number
- CN106815603B CN106815603B CN201710017636.0A CN201710017636A CN106815603B CN 106815603 B CN106815603 B CN 106815603B CN 201710017636 A CN201710017636 A CN 201710017636A CN 106815603 B CN106815603 B CN 106815603B
- Authority
- CN
- China
- Prior art keywords
- trigger
- identification
- pheromone
- activity
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 230000000694 effects Effects 0.000 title claims abstract description 149
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000001514 detection method Methods 0.000 title claims abstract description 27
- 239000003016 pheromone Substances 0.000 claims abstract description 113
- 239000011159 matrix material Substances 0.000 claims abstract description 73
- 238000012549 training Methods 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 16
- 230000008859 change Effects 0.000 claims abstract description 15
- 230000000875 corresponding effect Effects 0.000 claims description 66
- 230000001960 triggered effect Effects 0.000 claims description 19
- 230000037081 physical activity Effects 0.000 claims description 10
- 238000002790 cross-validation Methods 0.000 claims description 8
- 238000012163 sequencing technique Methods 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 9
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Abstract
The invention relates to an indoor activity detection and identification method and system based on a multi-sensor network, wherein the method mainly comprises the steps of processing KNN nearest neighbor algorithm training before activity type identification through position information and trigger information of all trigger sensors in a target indoor environment to obtain an identification library, processing according to the trigger information, a heterogeneous pheromone residual rate mask and a single-frame picture change in practical application to obtain a corresponding pheromone picture matrix, and finally realizing identification of human activity types based on the identification library. Therefore, the invention greatly improves the accuracy and reliability of the activity recognition result by arranging a plurality of trigger sensors in the indoor environment and recording the trigger information and the position information of each trigger sensor.
Description
Technical Field
The invention relates to the field of environment-assisted life, in particular to an indoor activity detection and identification method and system based on a multi-sensor network.
Background
In recent years, in an indoor environment, people have higher and higher requirements on location-based service quality, so as to drive intelligent decisions by ascertaining the real-time activity location of the human and identifying the activity type of the human, and provide the human with services required by the human, such as turning on or off indoor facilities of lights, air conditioners and the like. Therefore, to improve the service indoors to humans, human activity must be detected. However, contact sensors and non-contact sensors are widely applied to indoor environments nowadays, and intelligent building and intelligent home environments based on multiple sensing networks have many successful exploration precedents.
In a home environment, user activity recognition is mainly divided into two categories: the method comprises an activity identification mode based on a trigger type sensing network and an activity identification mode based on an indoor positioning system. The activity recognition mode based on the trigger type sensing network is mainly characterized in that a sensor with a unique identifier is deployed in living activity places and articles such as beds, tableware and sofas and used for representing various daily work and rest activities of users. The indoor positioning system is mainly used for carrying out indoor positioning with higher precision on a user through wireless sensing means such as infrared, WIFI, video and radio frequency, and the judgment on the user activity is realized through matching position information with each indoor functional area.
However, for an activity recognition mode based on a trigger type sensor network, the existing technical scheme is generally an analysis scheme directly based on a trigger sequence, and mainly uses the fact that a certain sensor is triggered or certain sensors are triggered according to a specific sequence as a basis for the occurrence of related activities. For the activity recognition mode based on the indoor positioning system, since indoor areas such as dining tables, sofas, beds and the like can correspond to various activities, and the areas are called as activity ambiguity areas, the activity type of the target cannot be accurately judged only by positioning information. Therefore, the current identification methods for the user activities cannot accurately identify and judge the types of the target activities, and the reliability of the activity identification results of the targets is low.
Disclosure of Invention
In order to solve the defects and shortcomings of the prior art, the invention provides an indoor activity detection and identification method and system based on a multi-sensor network.
An indoor activity detection and identification method based on a multi-sensor network comprises the following steps:
recording the position information of all the trigger sensors in the target indoor environment;
recording trigger information of all trigger sensors;
generating a corresponding two-dimensional plane graph according to the position information of all the trigger sensors;
sequentially reading trigger information according to the trigger time sequence, and making single-frame picture change delta s (t) corresponding to trigger time t on the two-dimensional plane graph according to the position information of the trigger sensor corresponding to the currently read trigger information;
generating a sub-pheromone matrix s (t) of a trigger time t according to the change delta s (t) of a single-frame picture and a Gaussian convolution kernel h (t),
generating a heterogeneous pheromone residual rate mask rho (t) based on Euclidean distance according to the position information of the trigger sensor corresponding to the currently read trigger information;
generating an pheromone graph matrix S (t) at a trigger time t according to the sub-pheromone matrix S (t) and the heterogeneous pheromone residual rate mask rho (t), wherein S (t) rho (t) S (t-1) + S (t);
training a KNN nearest neighbor algorithm by a ten-fold cross-validation method to obtain a KNN identification sample and a corresponding activity type identification library;
and inputting the pheromone graph matrix S (t) into the KNN identification sample, calculating to obtain corresponding identification data, and combining the activity type identification library to realize identification of the current activity type to obtain an identification result.
Therefore, the indoor activity detection and identification method based on the multi-sensor network greatly improves the accuracy and reliability of the activity identification result by arranging the plurality of trigger sensors in the indoor environment and recording the trigger information and the position information of each trigger sensor.
Further, the recording of the trigger information is performed in units of one day; after the position information is recorded, generating a trigger recording table of all trigger sensors in one day by combining the position information and the trigger information; and storing a plurality of trigger record tables generated by the multi-day record into the same database. The KNN algorithm training and the knowledge of the human activity rules are facilitated to provide a reliable basis through the limitation.
Further, the position information includes coordinate information (x, y) of each trigger sensor; and the trigger information comprises a unique label ID of each trigger type sensor, trigger time t and a trigger value ON/OFF. By limiting the above, the position information and the trigger information tend to be simplified, thereby reducing the complexity of data operation, reducing the redundancy of data, and improving the utilization rate of the recorded data.
Further, in the step of generating a heterogeneous pheromone residual rate mask ρ (t) based on the euclidean distance according to the position information of the trigger sensor corresponding to the currently read trigger information, the step of generating the heterogeneous pheromone residual rate mask ρ (t) includes:
establishing a blank matrix rho (M multiplied by N) with the same scale as the sub-pheromone matrix s (t);
for each grid element (M, N) in the blank matrix ρ (M × N), the position coordinates (x, y) of the currently triggered trigger sensor are used as reference points, and the geometric center coordinates (x) of the grid to be calculated are used as reference pointsm,yn) Calculating the residual rate rho of heterogeneous pheromones in the gridmnWherein M is [0, M-1 ]],n∈[0,N-1](ii) a The heterogeneous pheromone residual rate ρmnAbout the Euclidean distance
And obeys normal distribution rho to AxN (0, sigma)2) Function values of the distribution, i.e.
Wherein sigma2Is a normal distribution variance, and A is a function amplitude compensation parameter;
according to the residue rate rho of the isomeric pheromonesmnThe calculation method of (2) is obtained based on that triggered at the trigger time tThe heterogeneous pheromone residual rate mask ρ (t) for sensor position coordinates (x, y).
Through the steps, the farther the distance from the current trigger information is, the lower the pheromone residual rate is, so that the identifiability of the current activity type is favorably improved, and the misjudgment rate is further reduced.
Further, the step generates an information element map matrix S (t) of the trigger time t according to the sub information element matrix S (t) and the heterogeneous information element residual rate mask rho (t), wherein in S (t) () rho (t) × S (t-1) + S (t), the information residual rate mask rho (t) of the trigger sensor triggered according to the trigger time t is multiplied by the element of the position corresponding to the information element map matrix of the previous trigger time t-1, and then the sub information element matrix S corresponding to the current trigger time t is superposedij(t) generating an pheromone map matrix S at a trigger time tij(t)，Sij(t)＝ρij(t)×Sij(t-1)+sij(t) of (d). By limiting the current activity, the residual pheromone of the historical activity can be volatilized quickly, and the current activity pheromone is kept to the maximum extent, so that the identification accuracy of the current activity is further improved.
Further, the step of training the KNN nearest neighbor algorithm through a ten-fold cross-validation method to obtain a KNN recognition sample and a corresponding activity type recognition library specifically comprises the following steps:
recording activity information which is used for representing human body activity and triggered by a trigger type sensor in 220 days, and averagely dividing the activity information of 220 days into 10 groups of activity records, wherein each group of activity records has 22 days of activity information;
sequentially selecting one group from the 10 groups of activity records as an identification model test set according to the time sequence, and using the other nine groups as model training sets to generate 10 groups of training libraries;
sequentially training the KNN nearest neighbor algorithm through 10 groups of training libraries, and after 10 times of training, obtaining 10 primary recognition libraries in total;
and performing arithmetic mean processing on the 10 preliminary recognition libraries to obtain a standard recognition library representing the training results of the 10 groups of training libraries, wherein the standard recognition library comprises the KNN recognition sample and the corresponding activity type recognition library.
Through the limitation, the KNN algorithm is favorably trained most appropriately, so that a more accurate identification sample and an activity type identification library which are attached to the actual activity of the human body are obtained, and the misjudgment rate is further reduced.
Further, the step of inputting the pheromone map matrix s (t) into the KNN recognition sample, calculating to obtain corresponding recognition data, and combining the activity type recognition library to realize recognition of the current activity type to obtain a recognition result specifically includes the following steps:
inputting an pheromone map matrix S (t) into the KNN identification samples;
acquiring k judgment samples which are closest to an input pheromone map matrix S (t) in a feature space from a KNN identification sample through a KNN nearest neighbor algorithm;
searching and acquiring k activity types which are respectively in one-to-one correspondence with the k judgment samples in the activity type identification library;
and counting the number of the same activity types in the k activity types, and selecting the activity type with the largest number as the human body activity type corresponding to the current trigger time t to realize the acquisition and output of the identification result.
Through the limitation, the activity type with the largest occurrence frequency is used for representing the human body activity type, so that the misjudgment rate is further reduced.
Further, the step of obtaining k judgment samples closest to the input pheromone map matrix s (t) in the feature space from the KNN identification samples by the KNN nearest neighbor algorithm specifically includes the following steps:
randomly obtaining k judgment samples from the KNN identification samples to form a judgment queue with the length of k;
respectively calculating the distances between the k judgment samples in the judgment queue and the input pheromone map matrix S (t) in the characteristic space, sequencing the calculated distances from small to large in sequence, and recording the maximum distance;
traversing other judgment samples except the k judgment samples in the KNN identification sample, sequentially calculating the distances between the other judgment samples except the k judgment samples and the input pheromone map matrix S (t) in a feature space, and replacing the judgment sample corresponding to the maximum distance with the current judgment sample if the current distance is less than the maximum distance to form a new judgment queue;
sequencing the new judgment queues again in sequence according to the distance from small to large, and recording the maximum distance again until all samples are traversed;
and acquiring k judgment samples which are closest to the input information element map matrix S (t) in the feature space from the KNN identification samples.
The acquisition of k judgment samples is realized through the limitation, and the judgment basis is further refined, so that the misjudgment rate is further reduced.
Correspondingly, the present invention further provides a multi-sensor network-based indoor activity detection and identification system corresponding to the multi-sensor network indoor activity detection and identification method, comprising:
a trigger module comprising a plurality of trigger sensors for placement in a target indoor environment;
the recording module is used for recording the position information and the trigger information of all the trigger sensors;
and the processing module is used for identifying the human body activity type at the current trigger moment according to the position information and the trigger information of all the trigger sensors to obtain an identification result.
The processing module comprises:
the graph generation submodule is used for generating a corresponding two-dimensional plane graph according to the position information of all the trigger sensors recorded by the recording module;
and the central processing submodule is used for sequentially reading the trigger information according to the trigger time sequence recorded by the recording module, making single-frame picture change delta s (t) corresponding to the trigger time t on the two-dimensional plane graph according to the position information of the trigger sensor corresponding to the currently read trigger information, and generating sub-pheromone moment of the trigger time t according to the single-frame picture change delta s (t) and the Gaussian convolution kernel h (t)The arrays s (t),
generating a heterogeneous pheromone residual rate mask rho (t) based on Euclidean distance according to the position information of the trigger sensor corresponding to the currently read trigger information, generating an pheromone graph matrix S (t) of the trigger time t according to a sub-pheromone matrix S (t) and the heterogeneous pheromone residual rate mask rho (t), and training a KNN nearest neighbor algorithm by a cross-folding cross-validation method to obtain a KNN identification sample and a corresponding activity type identification library, inputting the pheromone graph matrix S (t) into the KNN identification sample, calculating to obtain corresponding identification data, and combining the activity type identification library to realize the identification of the current activity type to obtain an identification result.
Therefore, compared with the prior art, the invention has the beneficial effects that: from the aspect of data expression, the invention provides an expression method for converting an unordered and unreadable sensor trigger sequence into an ordered and highly readable pheromone graph. From the point of view of a classification method, the method classifies the input samples by the limited surrounding adjacent samples through the KNN nearest neighbor algorithm, so that the method is particularly suitable for multi-class classification and the condition that various domains are crossed and overlapped more, and has good applicability to household indoor activity judgment.
For a better understanding and practice, the invention is described in detail below with reference to the accompanying drawings.
Drawings
FIG. 1 is a block diagram of the steps of the indoor activity detection and identification method based on the multi-sensor network according to the present invention;
fig. 2 is a block diagram illustrating a specific method step of step S6 in the multi-sensor network-based indoor activity detection and identification method according to the present invention;
fig. 3 is a block diagram illustrating a specific method step of step S8 in the indoor activity detection and identification method based on multiple sensor networks according to the present invention;
fig. 4 is a block diagram illustrating a specific method step of step S9 in the indoor activity detection and identification method based on multiple sensor networks according to the present invention;
FIG. 5 is a schematic diagram illustrating a process of detecting and recognizing human activity types within a certain period of time according to the present invention;
FIG. 6 is a schematic diagram of a sensor deployed indoors;
FIG. 7 is a diagram illustrating an example of a heterogeneous information residual rate mask;
FIG. 8 is a graph showing a variation of the residual ratio with respect to the Euclidean distance;
FIG. 9 is a graph showing the relationship between total pheromone amount and indoor activity margin;
FIG. 10 is a diagram of an example of single frame pheromone map generation;
fig. 11 is a schematic diagram of an example of the KNN algorithm principle.
Detailed Description
Referring to fig. 1, the present invention provides an indoor activity detection and identification method based on a multi-sensor network, including the following steps:
s1: recording the position information of all the trigger sensors in the target indoor environment; in this embodiment, to reduce redundant data and improve processing efficiency, the position information includes coordinate information (x, y) of each trigger sensor, and preferably, the position information includes only coordinate information (x, y) of each trigger sensor;
s2: recording trigger information of all trigger sensors; in this embodiment, to reduce redundant data and improve processing efficiency, the trigger information includes a unique identifier ID of each trigger sensor, a trigger time t, and a trigger value ON/OFF;
further, as a more preferable technical solution, providing a reliable basis for training the KNN algorithm and learning the rule of human activities, the recording of the trigger information in step S2 is performed in units of one day; after the position information is recorded, generating a trigger recording table of all trigger sensors in one day by combining the position information and the trigger information; storing a plurality of trigger record tables generated by multi-day recording into the same database;
s3: generating a corresponding two-dimensional plane graph according to the position information of all the trigger sensors; in this embodiment, the generation means of the two-dimensional plane graph is the prior art, but in this embodiment, the two-dimensional plane graph is generated by combining the position information of all the trigger sensors with the indoor layout in the target indoor environment, and the two-dimensional plane graph has the coordinate information of the trigger sensors and the corresponding indoor environment information;
s4: and sequentially reading the trigger information according to the trigger time sequence, and making a single-frame picture change delta s (t) corresponding to the trigger time t on the two-dimensional plane graph according to the position information of the trigger sensor corresponding to the currently read trigger information. In this embodiment, the specific method for obtaining Δ s (t) includes: the area is L multiplied by W (m)2) Is gridded into an M × N matrix Δ s, the area of each cell is
Addressing on a plain (i.e. all 0 elements) pheromone matrix corresponding to the trigger position coordinates (x, y):
and performing an auto-increment operation s on elements of the corresponding coordinates in the matrixij＝sij+1；
S5: generating a sub-pheromone matrix s (t) of a trigger time t according to the change delta s (t) of a single-frame picture and a Gaussian convolution kernel h (t),
s6: generating a heterogeneous pheromone residual rate mask rho (t) based on Euclidean distance according to the position information of the trigger sensor corresponding to the currently read trigger information; in this embodiment, in order to further reduce the false positive rate, preferably, referring to fig. 2, the step of generating the heterogeneous pheromone residual rate mask ρ (t) specifically includes:
s61: establishing a blank matrix rho (M multiplied by N) with the same scale as the sub-pheromone matrix s (t); in this embodiment, the established blank matrix ρ (M × N) and the subsequent pheromone map matrix s (t) are also of equal scale;
s62: for each grid element (M, N) in the blank matrix ρ (M × N), the position coordinates (x, y) of the currently triggered trigger sensor are used as reference points, and the geometric center coordinates (x) of the grid to be calculated are used as reference pointsm,yn) Calculating the residual rate rho of heterogeneous pheromones in the gridmnWherein M is [0, M-1 ]],n∈[0,N-1](ii) a The heterogeneous pheromone residual rate ρmnAbout the Euclidean distance
And obeys normal distribution rho to AxN (0, sigma)2) Function values of the distribution, i.e.
Wherein sigma2In order to ensure that the pheromone in the region which is far away from the current starting point and is beyond the length and width of one room volatilizes fast, according to the property that the function integral in the range of normal function (mu-2 sigma, mu +2 sigma) reaches over 95% of the integral value of the full-real number domain, the sigma is generally set to be half of the length or width of a single room, and the value range is generally [2,4 ]]In this embodiment, σ is preferably 2.5. And the peak value of the function amplitude value is not more than 1 because the function amplitude value is endowed with the practical meaning of the residual rate, and the peak value of the residue rate of the heterogeneous pheromone is ensured to be kept at [0.7,0.9 ] because the standard normal distribution peak value is 0.4]Thus, A has a value in the range of [1.75,2.25 ]]In this embodiment, a is preferably 2.03;
s63: according to the residue rate rho of the isomeric pheromonesmnThe calculation method of (1) obtains a heterogeneous pheromone residual rate mask ρ (t) based on the triggered sensor position coordinates (x, y) at the trigger time t;
s7: generating an pheromone graph matrix S (t) at a trigger time t according to the sub-pheromone matrix S (t) and the heterogeneous pheromone residual rate mask rho (t), wherein S (t) rho (t) S (t-1) + S (t); furthermore, the method accelerates the volatilization of residual pheromone of historical activities and simultaneously retains the current activity pheromone to the maximum extent so as to further improve the current activityPreferably, in the step S7, the residual information rate mask ρ (t) of the triggered trigger sensor at the trigger time t is multiplied by the element at the position corresponding to the pheromone map matrix at the previous trigger time t-1, and then the sub-pheromone matrix S corresponding to the current trigger time t is superimposedij(t) generating an pheromone map matrix S at a trigger time tij(t)，Sij(t)＝ρij(t)×Sij(t-1)+sij(t)。
S8: training a KNN nearest neighbor algorithm by a ten-fold cross-validation method to obtain a KNN identification sample and a corresponding activity type identification library;
in this embodiment, in order to perform the most suitable training on the KNN algorithm, so as to obtain a more accurate recognition sample and an activity type recognition library which are fit to the actual activity of the human body, so as to further reduce the false positive rate, preferably, referring to fig. 3, the step S8 includes the following steps:
s81: recording activity information which is used for representing human body activity and triggered by a trigger type sensor in 220 days, and averagely dividing the activity information of 220 days into 10 groups of activity records, wherein each group of activity records has 22 days of activity information;
s82: sequentially selecting one group from the 10 groups of activity records as an identification model test set according to the time sequence, and using the other nine groups as model training sets to generate 10 groups of training libraries;
s83: sequentially training the KNN nearest neighbor algorithm through 10 groups of training libraries, and after 10 times of training, obtaining 10 primary recognition libraries in total;
s84: carrying out arithmetic mean processing on the 10 preliminary recognition libraries to obtain a standard recognition library representing the training results of the 10 groups of training libraries, wherein the standard recognition library comprises the KNN recognition sample and the corresponding activity type recognition library;
s9: inputting the pheromone graph matrix S (t) into the KNN identification sample, calculating to obtain corresponding identification data, and combining the activity type identification library to realize identification of the current activity type to obtain an identification result;
in this embodiment, to further reduce the false positive rate, referring to fig. 4, the step S9 specifically includes the following steps:
s91: inputting an pheromone map matrix S (t) into the KNN identification samples;
s92: acquiring k judgment samples which are closest to an input pheromone map matrix S (t) in a feature space from a KNN identification sample through a KNN nearest neighbor algorithm;
s93: searching and acquiring k activity types which are respectively in one-to-one correspondence with the k judgment samples in the activity type identification library;
s94: and counting the number of the same activity types in the k activity types, and selecting the activity type with the largest number as the human body activity type corresponding to the current trigger time t to realize the acquisition and output of the identification result.
In order to further improve the accuracy of the identification, as a more preferable technical solution, the step S92 includes the following steps:
s921: randomly obtaining k judgment samples from the KNN identification samples to form a judgment queue with the length of k;
s922: respectively calculating the distances between the k judgment samples in the judgment queue and the input pheromone map matrix S (t) in the characteristic space, sequencing the calculated distances from small to large in sequence, and recording the maximum distance;
s923: traversing other judgment samples except the k judgment samples in the KNN identification sample, sequentially calculating the distances between the other judgment samples except the k judgment samples and the input pheromone map matrix S (t) in a feature space, and replacing the judgment sample corresponding to the maximum distance with the current judgment sample if the current distance is less than the maximum distance to form a new judgment queue;
s924: sequencing the new judgment queues again in sequence according to the distance from small to large, and recording the maximum distance again until all samples are traversed;
s925: and acquiring k judgment samples which are closest to the input information element map matrix S (t) in the feature space from the KNN identification samples.
The indoor activity detection and identification method based on the multi-sensor network of the invention is further described with reference to the accompanying drawings 5-11:
fig. 5 is a schematic diagram illustrating a process of detecting and identifying human activity types within a certain period of time according to the present invention, in which a plurality of sensors with unique labels, which are triggered in a contact manner and a non-contact manner, are disposed in advance at various locations in a home, including a bed, a desk, a sofa, a door knob, etc., where specific indoor activities can be represented; the corresponding sensor is triggered and a trigger record is generated when the user performs a specific indoor activity such as sleeping, reading, leisure, toileting, dining.
As shown in fig. 6, fig. 6 illustrates the sensor deployment in the room, and a total of 30 sensors including temperature, closing spring and binarization sensor are deployed in different areas respectively, and independently or cooperatively detect the indoor activity.
Referring to fig. 5 and fig. 6, fig. 5 cuts a segment of the sensor trigger records generated when the user moves in the study, and it can be seen that when the user moves in the study, only the sensors deployed in the study are densely triggered, and the content of each record is trigger time (accurate to microsecond level), sensor number, and trigger value (for the binary sensor, the trigger value is changed between ON and OFF).
Because the position of the sensor is not changed after deployment, the sensor can be used for generating an information element graph reflecting the position information of the user after adding the corresponding position information of the sensor. In this embodiment, the indoor activity area is uniformly divided into M × N pixels, and the distribution of the activity pixels newly added at time t is represented as x (t) { xij(t) | i ═ 1, …, M; j is 1, …, N }. When the triggered sensor is located at pixel i, xiAnd (t) taking a nonzero value, otherwise, taking zero. the sub-pheromone s (t) at time t can be expressed as
Where h (t) is a Gaussian convolution kernel that determines the degree of diffusion of the pheromone units.
Then generating the position coordinate (i, j) according to the position coordinate (i, j) of the non-zero value in x (t)Heterogeneous pheromone residual rate mask p (t), fig. 7 is an example of a mask when a user moves within a study. The heterogeneous pheromone residual rate mask rho (t) generation method is to establish a mask matrix rho with the same scale as the pheromone graph matrix, and a coordinate point (x, y) is taken as a datum point when a current trigger coordinate point (or called a centre of a point) (x, y) is taken as a datum pointi,yj) Residual ratio of (p)ijAbout the Euclidean distance
Obeys ρ to A × N (0, σ) as shown in FIG. 82) Function of the distribution, where the parameter σ2The variance is the parameter A, the amplitude compensation is the set constant, and the residual rate rho can be seenijInversely proportional to the distance d. When the residual rate is closer to 1, the volatilization speed of the pheromone in the area is low, otherwise, when the residual rate is closer to 0, the volatilization speed of the pheromone in the area is high, the mask has the main function of storing the pheromone accumulated in the current activity to the maximum extent, and meanwhile, the volatilization of the residual pheromone in the historical activity is accelerated so as to highlight the image characteristic of the current activity and reduce the misjudgment rate of a subsequent activity recognition algorithm. As shown in fig. 9, which is a relationship between the total amount of pheromones in the pheromone map and two edges of indoor activities in a continuous time, the indoor activities of the users change (mainly, the staying positions of the users shift): in the figure active state 1 represents the previous activity, active state 2 represents the transition state and active state 3 represents the performance of another activity. At this time, the residual pheromone of the activity 1 is rapidly volatilized under the action of the heterogeneous mask in the activity state 2 and the activity state 3, so that the total pheromone amount is rapidly reduced, and meanwhile, after a short time of accumulation, the pheromone of the new activity is rapidly accumulated, so that the total pheromone amount generates an 'activity edge effect', which indicates that the characteristics of the pheromone graph on the activity edge are rapidly changed, the pheromone graph can rapidly respond to the change of the activity, and the possibility of misjudgment of the edge part is effectively reduced.
After obtaining the heterogeneous pheromone residual rate mask rho (t) at the time t, an pheromone graph S (t) rho (t) S (t-1) + S (t) up to the time t can be generated, wherein matrixes rho (t), S (t-1) and S (t) are arranged on the ith row and the ith rowThe elements in column j are all operated on: sij(t)＝ρij(t)×Sij(t-1)+sij(t) of (d). Fig. 10 shows a frame of pheromone map generated when a user reads in a study.
After generating a sequence of pheromone maps that can be used to describe a user's continuous indoor activity, we use it as input to a K-nearest-neighbor algorithm (KNN) for the purpose of identifying the user's indoor activity (where each frame of pheromone map is referred to as a sample). The invention adopts an off-line training-on-line recognition mode. The user activity original data of 220 days in total are averagely divided into ten parts, each part is 22 days, one part is selected as a recognition model test set, the other nine parts are model training sets, a primary recognition library is obtained through training-testing, and then ten primary recognition libraries are obtained from all ten groups of training, and the arithmetic mean is taken as the estimation of the training result of the whole algorithm model. The K-nearest neighbor algorithm determines the class to which the input sample belongs by checking K training samples closest to the current input sample (e.g., the current pheromone map matrix s (t)) in the feature space and counting the class with the largest proportion, which is shown in fig. 11, and for the input sample (circle), it should be determined whether the input sample is a class a (square shown in fig. 11) or a class B (triangle shown in fig. 11), depending on the class with the larger proportion among the K samples closest to the input sample, when a different K value is selected, a different classification result is generated, which affects the classification accuracy, and the K value is completed in the training link, and by evaluating the classification accuracy or the false positive rate, we can select an optimal K value (integer). After a sample to be classified is input, k samples are randomly selected from identification samples to form a queue with the length of k, and the queue is sorted according to the distance between the queue and the input sample in a feature space. In the present invention, the dimension of the feature space is P ═ M × N, and the input sample a ═ NiI ═ 1, …, P }, and identify sample B ═ BjIf 1, …, P, the distance between the input sample and the identified sample can be expressed as
The maximum distance value between the input sample and the k judgment samples is recorded asLmax(let its corresponding judgment sample be Bmax). Then, all the other judging samples are traversed, the distances L between the other judging samples and the input sample are respectively calculated, and the distances L between the other judging samples and the input sample are respectively calculatedmaxComparing if L < LmaxReplacing B with the judged samplemaxAnd the queue is reordered by L. And obtaining k judgment samples which are closest to the input sample A in the whole feature space until the traversal of the judgment samples is finished, counting the activity class with the largest occupation ratio, and taking the activity class as the classification result of the A, thereby obtaining the identification result of the user activity type.
Correspondingly, the present invention further provides an indoor activity detection and identification system of a multi-sensor network corresponding to the indoor activity detection and identification method of the multi-sensor network, comprising:
a trigger module comprising a plurality of trigger sensors for placement in a target indoor environment; in this embodiment, the triggering module includes 30 total sensors including temperature, closing spring and binary sensor, which are respectively disposed in different areas to detect indoor activities independently or cooperatively, and the 30 sensors are disposed at various positions in the home, including a bed, a desk, a sofa, a door knob, etc., which can represent specific indoor activities, and when a user performs specific indoor activities such as sleeping, reading, leisure, toileting, dining, the corresponding sensors are triggered to generate a trigger record;
the recording module is used for recording the position information and the trigger information of all the trigger sensors;
and the processing module is used for identifying the human body activity type at the current trigger moment according to the position information and the trigger information of all the trigger sensors to obtain an identification result.
Further, the processing module includes:
the graph generation submodule is used for generating a corresponding two-dimensional plane graph according to the position information of all the trigger sensors recorded by the recording module;
and the central processing submodule is used for sequentially reading the trigger signals according to the trigger time sequence recorded by the recording moduleAccording to the position information of the trigger sensor corresponding to the currently read trigger information, making a single-frame picture change delta s (t) corresponding to the trigger time t on the two-dimensional plane graph, generating a sub-pheromone matrix s (t) of the trigger time t according to the single-frame picture change delta s (t) and a Gaussian convolution kernel h (t),
generating a heterogeneous pheromone residual rate mask rho (t) based on Euclidean distance according to the position information of the trigger sensor corresponding to the currently read trigger information, generating an pheromone graph matrix S (t) of the trigger time t according to a sub-pheromone matrix S (t) and the heterogeneous pheromone residual rate mask rho (t), and training a KNN nearest neighbor algorithm by a cross-folding cross-validation method to obtain a KNN identification sample and a corresponding activity type identification library, inputting the pheromone graph matrix S (t) into the KNN identification sample, calculating to obtain corresponding identification data, and combining the activity type identification library to realize the identification of the current activity type to obtain an identification result.
Compared with the prior art, the indoor activity detection and identification method and system based on the multi-sensor network have the beneficial effects that: from the aspect of data expression, the invention provides an expression method for converting an unordered and unreadable sensor trigger sequence into an ordered and highly readable pheromone graph. From the perspective of a classification method, the method classifies input samples by limited surrounding adjacent samples through a KNN nearest neighbor algorithm so as to be particularly suitable for multi-class classification and the condition that various domains are crossed and overlapped more, and the method has good applicability to household indoor activity judgment; according to the invention, the plurality of trigger sensors are arranged in the indoor environment, and the trigger information and the position information of each trigger sensor are recorded, so that the accuracy and the reliability of the activity recognition result are greatly improved.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.
Claims (9)
1. An indoor activity detection and identification method based on a multi-sensor network is characterized in that: the method comprises the following steps:
recording the position information of all the trigger sensors in the target indoor environment;
recording trigger information of all trigger sensors;
generating a corresponding two-dimensional plane graph according to the position information of all the trigger sensors;
sequentially reading trigger information according to the trigger time sequence, and making single-frame picture change delta s (t) corresponding to trigger time t on the two-dimensional plane graph according to the position information of the trigger sensor corresponding to the currently read trigger information;
generating a sub-pheromone matrix s (t) of a trigger time t according to the change delta s (t) of a single-frame picture and a Gaussian convolution kernel h (t),
generating a heterogeneous pheromone residual rate mask rho (t) based on Euclidean distance according to the position information of the trigger sensor corresponding to the currently read trigger information;
generating an pheromone graph matrix S (t) at a trigger time t according to the sub-pheromone matrix S (t) and the heterogeneous pheromone residual rate mask rho (t), wherein S (t) rho (t) S (t-1) + S (t);
training a KNN nearest neighbor algorithm by a ten-fold cross-validation method to obtain a KNN identification sample and a corresponding activity type identification library;
and inputting the pheromone graph matrix S (t) into the KNN identification sample, calculating to obtain corresponding identification data, and combining the activity type identification library to realize identification of the current activity type to obtain an identification result.
2. The indoor activity detection and identification method based on the multi-sensor network as claimed in claim 1, wherein: the recording of the trigger information is carried out by taking one day as a unit; after the position information is recorded, generating a trigger recording table of all trigger sensors in one day by combining the position information and the trigger information; and storing a plurality of trigger record tables generated by the multi-day record into the same database.
3. The indoor activity detection and identification method based on the multi-sensor network as claimed in claim 1, wherein: the position information comprises coordinate information (x, y) of each trigger sensor; and the trigger information comprises a unique label ID of each trigger type sensor, trigger time t and a trigger value ON/OFF.
4. The indoor activity detection and identification method based on the multi-sensor network as claimed in claim 1, wherein: the step of generating a heterogeneous pheromone residual rate mask rho (t) based on Euclidean distance according to the position information of the trigger sensor corresponding to the currently read trigger information includes the steps of:
establishing a blank matrix rho (M multiplied by N) with the same scale as the sub-pheromone matrix s (t);
for each grid element (M, N) in the blank matrix ρ (M × N), the position coordinates (x, y) of the currently triggered trigger sensor are used as reference points, and the geometric center coordinates (x) of the grid to be calculated are used as reference pointsm,yn) Calculating the residual rate rho of heterogeneous pheromones in the gridmnWherein M is [0, M-1 ]],n∈[0,N-1](ii) a The heterogeneous pheromone residual rate ρmnAbout the Euclidean distance
And obeys normal distribution rho to AxN (0, sigma)2) Function values of the distribution, i.e.
Wherein sigma2Is a normal distribution variance, and A is a function amplitude compensation parameter;
according to the residue rate rho of the isomeric pheromonesmnThe heterogeneous pheromone residual rate mask ρ (t) based on the triggered sensor position coordinates (x, y) is obtained at the trigger time t.
5. The indoor activity detection and identification method based on the multi-sensor network as claimed in claim 4, wherein: in the step, an information element map matrix S (t) of a trigger time t is generated according to a sub information element matrix S (t) and a heterogeneous information element residual rate mask rho (t), wherein in S (t) () rho (t) × S (t-1) + S (t), the information residual rate mask rho (t) of a trigger sensor triggered according to the trigger time t is multiplied by elements at the corresponding position of the information element map matrix of the previous trigger time t-1, and then the sub information element matrix S corresponding to the current trigger time t is superposedij(t) generating an pheromone map matrix S at a trigger time tij(t)，Sij(t)＝ρij(t)×Sij(t-1)+sij(t)。
6. The indoor activity detection and identification method based on the multi-sensor network as claimed in claim 5, wherein: the method comprises the following steps of training a KNN nearest neighbor algorithm through a ten-fold cross-validation method to obtain a KNN recognition sample and a corresponding activity type recognition library, and specifically comprises the following steps:
recording activity information which is used for representing human body activity and triggered by a trigger type sensor in 220 days, and averagely dividing the activity information of 220 days into 10 groups of activity records, wherein each group of activity records has 22 days of activity information;
sequentially selecting one group from the 10 groups of activity records as an identification model test set according to the time sequence, and using the other nine groups as model training sets to generate 10 groups of training libraries;
sequentially training the KNN nearest neighbor algorithm through 10 groups of training libraries, and after 10 times of training, obtaining 10 primary recognition libraries in total;
and performing arithmetic mean processing on the 10 preliminary recognition libraries to obtain a standard recognition library representing the training results of the 10 groups of training libraries, wherein the standard recognition library comprises the KNN recognition sample and the corresponding activity type recognition library.
7. The multi-sensor network based indoor activity detection and identification method of claim 6, wherein: the method comprises the following steps of inputting an information element map matrix S (t) into a KNN identification sample, calculating to obtain corresponding identification data, and combining an activity type identification library to realize identification of a current activity type to obtain an identification result, wherein the method specifically comprises the following steps:
inputting an pheromone map matrix S (t) into the KNN identification samples;
acquiring k judgment samples which are closest to an input pheromone map matrix S (t) in a feature space from a KNN identification sample through a KNN nearest neighbor algorithm;
searching and acquiring k activity types which are respectively in one-to-one correspondence with the k judgment samples in the activity type identification library;
and counting the number of the same activity types in the k activity types, and selecting the activity type with the largest number as the human body activity type corresponding to the current trigger time t to realize the acquisition and output of the identification result.
8. The indoor activity detection and identification method based on the multi-sensor network as claimed in claim 7, wherein: the method comprises the following steps of obtaining k judgment samples which are closest to an input pheromone map matrix S (t) in a feature space from KNN identification samples through a KNN nearest neighbor algorithm, and specifically comprises the following steps:
randomly obtaining k judgment samples from the KNN identification samples to form a judgment queue with the length of k;
respectively calculating the distances between the k judgment samples in the judgment queue and the input pheromone map matrix S (t) in the characteristic space, sequencing the calculated distances from small to large in sequence, and recording the maximum distance;
traversing other judgment samples except the k judgment samples in the KNN identification sample, sequentially calculating the distances between the other judgment samples except the k judgment samples and the input pheromone map matrix S (t) in a feature space, and replacing the judgment sample corresponding to the maximum distance with the current judgment sample if the current distance is less than the maximum distance to form a new judgment queue;
sequencing the new judgment queues again in sequence according to the distance from small to large, and recording the maximum distance again until all samples are traversed;
and acquiring k judgment samples which are closest to the input information element map matrix S (t) in the feature space from the KNN identification samples.
9. An indoor activity detection and identification system based on a multi-sensor network is characterized in that: comprises that
A trigger module comprising a plurality of trigger sensors for placement in a target indoor environment;
the recording module is used for recording the position information and the trigger information of all the trigger sensors;
the processing module is used for identifying the human body activity type at the current trigger moment according to the position information and the trigger information of all the trigger sensors to obtain an identification result;
the processing module comprises:
the graph generation submodule is used for generating a corresponding two-dimensional plane graph according to the position information of all the trigger sensors recorded by the recording module;
and a central processing submodule for sequentially reading the trigger information according to the trigger time sequence recorded by the recording module, making a single-frame picture change delta s (t) corresponding to the trigger time t on the two-dimensional plane graph according to the position information of the trigger sensor corresponding to the currently read trigger information, generating a sub-pheromone matrix s (t) of the trigger time t according to the single-frame picture change delta s (t) and a Gaussian convolution kernel h (t),
according to the trigger currently readThe method comprises the steps of generating a heterogeneous pheromone residual rate mask rho (t) based on Euclidean distance according to position information of a trigger sensor corresponding to information, generating an pheromone graph matrix S (t) of trigger time t according to a sub-pheromone matrix S (t) and the heterogeneous pheromone residual rate mask rho (t), training a KNN nearest neighbor algorithm through a cross-folding cross-validation method to obtain a KNN identification sample and a corresponding activity type identification library, inputting the pheromone graph matrix S (t) into the KNN identification sample, calculating corresponding identification data, combining the activity type identification library to realize identification of the current activity type, and obtaining an identification result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710017636.0A CN106815603B (en) | 2017-01-11 | 2017-01-11 | Indoor activity detection and identification method and system based on multiple sensor networks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710017636.0A CN106815603B (en) | 2017-01-11 | 2017-01-11 | Indoor activity detection and identification method and system based on multiple sensor networks |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106815603A CN106815603A (en) | 2017-06-09 |
| CN106815603B true CN106815603B (en) | 2020-04-14 |
Family
ID=59110051
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710017636.0A Expired - Fee Related CN106815603B (en) | 2017-01-11 | 2017-01-11 | Indoor activity detection and identification method and system based on multiple sensor networks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106815603B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111787863B (en) * | 2018-03-09 | 2023-07-25 | 松下知识产权经营株式会社 | Cognitive function assessment system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101324662A (en) * | 2008-07-21 | 2008-12-17 | 中山大学 | Human body infrared positioning apparatus and method facing to wireless sensor network |
| CN103268101A (en) * | 2013-05-06 | 2013-08-28 | 江苏科技大学 | Living energy-saving control device based on video and implementation method thereof |
| CN103268616A (en) * | 2013-04-18 | 2013-08-28 | 北京工业大学 | Multi-feature multi-sensor method for mobile robot to track moving body |
| CN103488148A (en) * | 2013-09-24 | 2014-01-01 | 华北电力大学（保定） | Intelligent livestock behavior monitoring system based on internet of things and computer vision |
| CN104142149A (en) * | 2014-07-03 | 2014-11-12 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | Human and robot identification and location method based on intelligent optical fiber floor |
| CN105577748A (en) * | 2015-11-16 | 2016-05-11 | 清华大学 | Environment information acquisition method and system based on group perception technology |
| CN105629936A (en) * | 2016-02-17 | 2016-06-01 | 南京物联传感技术有限公司 | Intelligent floor capable of controlling intelligent household equipment |
-
2017
- 2017-01-11 CN CN201710017636.0A patent/CN106815603B/en not_active Expired - Fee Related
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101324662A (en) * | 2008-07-21 | 2008-12-17 | 中山大学 | Human body infrared positioning apparatus and method facing to wireless sensor network |
| CN103268616A (en) * | 2013-04-18 | 2013-08-28 | 北京工业大学 | Multi-feature multi-sensor method for mobile robot to track moving body |
| CN103268101A (en) * | 2013-05-06 | 2013-08-28 | 江苏科技大学 | Living energy-saving control device based on video and implementation method thereof |
| CN103488148A (en) * | 2013-09-24 | 2014-01-01 | 华北电力大学（保定） | Intelligent livestock behavior monitoring system based on internet of things and computer vision |
| CN104142149A (en) * | 2014-07-03 | 2014-11-12 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | Human and robot identification and location method based on intelligent optical fiber floor |
| CN105577748A (en) * | 2015-11-16 | 2016-05-11 | 清华大学 | Environment information acquisition method and system based on group perception technology |
| CN105629936A (en) * | 2016-02-17 | 2016-06-01 | 南京物联传感技术有限公司 | Intelligent floor capable of controlling intelligent household equipment |
Non-Patent Citations (3)
| Title |
|---|
| Activity Recognition in the Home Using Simple and Ubiquitous Sensors;Emmanuel Munguia Tapia 等;《PERVASIVE 2004》;20040423;第4节，第5.1节，第6节，图2 * |
| Sensor-Based Activity Recognition;Liming Chen 等;《IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS》;20120530;第42卷(第6期);790-808 * |
| 基于手机传感器的室内用户行为识别;李晶 等;《科技展望》;20160220;143 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106815603A (en) | 2017-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liu et al. | Robotic room-level localization using multiple sets of sonar measurements | |
| Hecht et al. | Automatic identification of building types based on topographic databases–a comparison of different data sources | |
| Mushore et al. | Assessing the potential of integrated Landsat 8 thermal bands, with the traditional reflective bands and derived vegetation indices in classifying urban landscapes | |
| CN105027162B (en) | Image analysis apparatus, image analysis system, method for analyzing image | |
| CN109034077B (en) | Three-dimensional point cloud marking method and device based on multi-scale feature learning | |
| Murillo et al. | Visual door detection integrating appearance and shape cues | |
| CN104394588B (en) | Indoor orientation method based on Wi Fi fingerprints and Multidimensional Scaling | |
| CN104573744B (en) | Fine granulation classification identifies and the part of object positions and feature extracting method | |
| CN110536257B (en) | Indoor positioning method based on depth adaptive network | |
| CN112135248B (en) | WIFI fingerprint positioning method based on K-means optimal estimation | |
| CN109272554A (en) | A kind of method and system of the coordinate system positioning for identifying target and semantic map structuring | |
| CN106851937A (en) | A kind of method and device of gesture control desk lamp | |
| CN106815603B (en) | Indoor activity detection and identification method and system based on multiple sensor networks | |
| CN110443179B (en) | Off-post detection method and device and storage medium | |
| Mozos et al. | Categorization of indoor places by combining local binary pattern histograms of range and reflectance data from laser range finders | |
| WO2022237117A1 (en) | Touch control method and system for interactive electronic whiteboard, and readable medium | |
| CN112801268B (en) | Positioning method based on graph convolution and multilayer perceptron hybrid network | |
| CN110020674A (en) | A kind of cross-cutting adapting to image classification method promoting topic categories discrimination | |
| CN108388854A (en) | A kind of localization method based on improvement FAST-SURF algorithms | |
| JP5072998B2 (en) | Method, apparatus and system for determining a spatial area in which a target is located | |
| CN111132020B (en) | RSSI indoor environment sensing and detecting method based on machine learning | |
| CN102136071A (en) | Spatial correlation matrix-based image characteristic analysis model | |
| CN109583513B (en) | Method, system and device for detecting similar frame and readable storage medium | |
| Hou et al. | The indoor wireless location technology research based on WiFi | |
| Fraccaro et al. | Development and preliminary evaluation of a method for passive, privacy-aware home care monitoring based on 2D LiDAR data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200414 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |