CN108763045B - Universal continuous behavior recognition application framework containing missing data at mobile phone terminal - Google Patents
Universal continuous behavior recognition application framework containing missing data at mobile phone terminal Download PDFInfo
- Publication number
- CN108763045B CN108763045B CN201810544629.0A CN201810544629A CN108763045B CN 108763045 B CN108763045 B CN 108763045B CN 201810544629 A CN201810544629 A CN 201810544629A CN 108763045 B CN108763045 B CN 108763045B
- Authority
- CN
- China
- Prior art keywords
- data
- control unit
- sensor
- unit
- mobile phone
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3438—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment monitoring of user actions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
Abstract
A universal continuous behavior recognition application framework with missing data at a mobile phone end is applied to a smart phone and comprises an inertial sensor, a sensor control unit, a data receiving control unit, a data completion unit and a behavior recognition unit. The inertial sensor is responsible for interacting with the environment and a user to acquire real-time data; the sensor control unit is responsible for controlling the on-off and sampling frequency of the sensor according to the feedback information of the data admission control unit; the data admission control unit is responsible for judging whether the data is valid data according to the current data; and the data completion unit is responsible for completing the missing data and generating complete data to be supplied to the subsequent behavior identification unit for identification. The method is provided by considering the limitation of the energy source of the mobile phone end aiming at the inevitable invalid data and data loss problem in the behavior recognition of the mobile phone end, has good universality, easy construction, real and reliable obtained data and wide application range, and can be effectively applied to the behavior recognition of the mobile phone end under various scenes.
Description
Technical Field
The invention belongs to the field of computer application, and particularly relates to a general continuous behavior identification application framework with missing data at a mobile phone end.
Background
The behavior recognition field is a field which has been developed for years and matures day by day, the technology is mature at present, and accurate perception and recognition of user behaviors can be obtained through limited sensor information. Commonly used sensors would be: the RFID sensing equipment, the camera or the special sensing equipment judges the current behavior of the user by collecting acceleration information, position information and the like related to the user.
With the continuous development of smart phone technology, more and more sensors, such as a camera, a microphone, an acceleration sensor, etc., have been integrated on a mobile phone. The information of the user when using the mobile phone can be accurately recorded, and the current state information of the user can be reflected through the information. At present, the existing behavior recognition technology is well transplanted to a smartphone end, and usually, sufficient information related to a user is collected by collecting information of an inertial sensor (such as an accelerometer, a gyroscope, a linear accelerometer and the like without gravity) built in the smartphone, so that the behavior of the user is continuously recognized.
Generally, behavior recognition applications at the smartphone end are all run on a single mobile phone, and run locally on the mobile phone. The sensor stops working due to the scheduling of the system and other reasons, so that the reported information is continuously lost, the acquired information is incomplete, and the behavior of the user cannot be accurately identified. In addition, because the mobile phone is not located around the user at all times, when the mobile phone is not located around the user, the collected data is not an embodiment of the behavior of the user, and is called as invalid data. The current behavior recognition application at the mobile phone end does not effectively process whether the mobile phone is around the user, so that an error behavior recognition result is generated.
Meanwhile, the applications are arranged on the smart phone, but due to the limitation of the size of the device, along with the improvement of the complexity and the real-time performance of the applications, the energy consumption is also improved. The battery capacity of the handset itself tends to only support operation of the handset for a very limited time. Moreover, as the scale of the processed data is larger and the complexity of the algorithm is increased, a larger load is also generated on the CPU of the mobile phone, so when designing a specific behavior recognition application, how to deal with the missing data in blocks in the acquired data and the invalid information which may exist in the data and is not at the user's side needs to be considered, and at the same time, the energy consumption of the application is reduced as much as possible, and the design of the system is optimized.
Disclosure of Invention
Aiming at the defects in the prior art and aiming at the problems of missing data and invalid data in the existing behavior recognition application of a mobile phone terminal, the invention provides a high-efficiency and reliable universal framework for judging invalid data, completing missing data and dynamically adjusting the energy consumption of a sensor by aiming at the characteristics of a mobile phone and the characteristics of continuous data during invalid data acquisition and combining the research on the performance and the energy consumption of the sensor.
Considering that the mobile phone is not at hand and usually stays still and flat (zThe axle gravity acceleration value is equal to the gravity), and the judgment on whether the original data is invalid data is realized at the data receiving and controlling unit through a pre-trained model. Thereby filtering invalid data to facilitate obtaining a true and reliable user status.
By judging the information whether the mobile phone is at the user, when the mobile phone is not at the user, the data of a plurality of sensors are not collected with high energy consumption, the related sensors can be closed or the sampling frequency is reduced, only the data collected by the sensor with the lowest energy consumption is kept, and the judgment of whether the mobile phone is at the user is maintained.
Considering that the smart phone has block-shaped deficiency of the acquired sensor data due to reasons such as memory scheduling and the like; in addition to this, the position of the invalid data is also a missing value due to the presence of the invalid data. Due to the smoothness of continuous data and the periodicity of data inclusion, the data is organized into a reasonable matrix form, and the data is complemented through the existing data complementing algorithm to form a complete data matrix for the behavior recognition application of the upper layer. Considering that the matrix operation needs larger operation overhead, the matrix size and the calculation complexity are often exponentially expanded, and the CPU of the mobile phone is loaded greatly. Therefore, through reasonable selection of parameters, the optimal choice between the complexity and the accuracy of the calculation is selected.
Therefore, the sensor control unit, the data receiving control unit and the light-weight data completion unit are embedded into the middle of common behavior recognition application and sensor data flow, so that redundant assumption on input data is not needed, a new system framework can be conveniently constructed in the system design process, and the requirements of different applications are flexibly met.
Generally, the performance of the system is measured by two aspects of accuracy and energy consumption, and according to the characteristics of each unit, the energy consumption of the system is maintained within an acceptable range by reasonably selecting system parameters, meanwhile, sensor data are effectively and accurately acquired, invalid data are screened, and missing data are supplemented.
In order to achieve the purpose, the invention adopts the following technical scheme:
a mobile phone end common continuous behavior identification application framework containing missing data is characterized by comprising: the system comprises a data source, a sensor control unit, a data receiving control unit, a data completion unit and a behavior identification unit;
the data source comprises three inertial sensors, namely an accelerometer, a linear accelerometer and a gyroscope, and the sensors interact with the environment and a user to acquire real-time data;
the sensor control unit controls the switch and sampling frequency of the sensor according to the feedback information of the data admission control unit;
the data admission control unit judges whether the data is valid data according to the current data, if the data is valid data, the data is transmitted to the data completion unit and the sensor control unit is fed back, otherwise the sensor control unit is fed back to reduce the sampling frequency;
and the data completion unit completes the missing data to generate complete data which is supplied to the subsequent behavior identification unit for identification.
In order to optimize the technical scheme, the specific measures adopted further comprise:
the work of the application framework specifically comprises the following steps:
1) each sensor in the data source collects data independently, forms an input data stream respectively, organizes the data according to time sequence and stores the data by the sensor control unit;
2) from the present momenttAt the start of the process,the sensor control unit is according to sizewA sliding window of forward selection from among the data buffered in the respective sensorst-wTotThe data of (a); for each sensor data at the same moment, splicing the sensor data into the same vectord i The vectors within the window are then organized asD t = < d 1 , d 2 ,… , d t >Transmitting the original data to a data admission control unit; wherein the content of the first and second substances,wis a positive real number in seconds;
3) the data admission control unit comprises a pre-trained two-class classifier model and is used for judging whether the mobile phone is on the body of the user or not in the period of time; taking the vector as input and outputting as on or not on the user; if the mobile phone is judged not to be on the body of the user, judging the data in the time period to be invalid data to be discarded, simultaneously feeding back to the sensor control unit according to a determined strategy, and closing part of the sensors or reducing the sampling frequency; if the mobile phone is judged to be on the body of the user, the effective data is transmitted to the data completion unit and fed back to the sensor control unit, and the sensor needs to be turned on and the sampling frequency needs to be increased to obtain accurate data;
4) the data completion unit collects the effective data transmitted from the data admission control unit from the current timetAt the beginning, according to the sizeSAn overlap ratio ofλThe number of days isdThe sliding window of (2) is selected from the current day and the previous dayd-1Day data, selected from the current time of daytStart, select forward tot-SOrganizing the data into matrixes according to the data at a moment, marking missing values in the data matrixes by indicating the matrixes, transmitting the two matrixes as input to a preset data completion algorithm, completing effective data containing the missing values, and transmitting the completed complete data to an upper-layer behavior recognition unit; wherein the content of the first and second substances,S、λ、dis a positive real number, and the number of the real numbers,dthe unit is day;
5) and the behavior recognition unit completes corresponding perception recognition operation on the complete data set.
The specific process of the step 1) is as follows:
1.1) sensor control Unit creating a cache of predetermined sizeD；
1.2) carrying out data acquisition by each sensor according to the same sampling frequency, and storing the data obtained by each sensorDStoring the data in different sensors separately, wherein the data in each sensor are stored in sequence according to time;
1.3) definition ofDThe maximum retention time of the buffer data of each sensor ishSecond, data exceeding the retention time is written in the fixed storage device.
The specific process of the step 2) is as follows:
2.1) selecting the granularity of the sliding window in timew；
2.2) starting from the current time, according to the sizewA sliding window of forward selection from among the data buffered in the respective sensorst-wTotThe data of (a);
2.3) for the same time instantiSplicing the data of the sensors into the same vectord i Organizing the vectors in the window into shapesD t = < d 1 , d 2 ,… , d t >The vector is passed as raw data to the data admission control unit.
The specific process of the step 3) is as follows:
3.1) inputting a data vector transmitted by the sensor control unit into a classifier which is trained in advance, wherein the classifier can judge whether the mobile phone is on the body of a user according to part of acquired inertial sensor data;
3.2) if the output result is that the mobile phone is near the user, the data is delivered to the data completion unit and fed back to the sensor control unit, and the sensor needs to be opened and the sampling frequency needs to be increased to obtain accurate data;
and 3.3) if the output result is that the mobile phone is not near the user, judging the data in the time period to be invalid data and discarding the invalid data, feeding back the sensor control unit according to a determined strategy, and closing part of sensors or reducing the sampling frequency.
The specific process of the step 4) is as follows:
4.1) the data completion unit comprises a runtime cache with a predetermined size, the cache divides the historical data according to the form of each day and arranges the data of one day into a line;
4.2) data completion Unit is according to sizeSAn overlap ratio ofλThe number of days isdThe sliding window of (2) is selected from the current day and befored-1Day data, selected from the current time of daytAt the beginning, choose to forward tot-SThe data at the moment is organized into a new matrix, the matrix comprises a plurality of missing data, and a specific position value is marked to be missing through an indication matrix;
and 4.3) the data completion unit contains a packaged data completion algorithm, the data matrix and the indication matrix are used as input and transmitted to the data completion algorithm, the output of the algorithm is a complete matrix which does not contain missing values, and the complete matrix is transmitted to the upper-layer behavior identification unit.
The specific process of the step 5) is as follows:
and 5.1) the behavior recognition unit comprises a preset behavior recognition algorithm, and performs data processing on the complete data set to obtain output related to specific application.
The invention has the beneficial effects that: the system has the advantages of good universality, easiness in construction, real and reliable acquired data, wide application range and the like, and can be effectively applied to systems related to behavior recognition of the mobile phone end under various scenes.
Drawings
Fig. 1 is a system flow diagram.
Fig. 2 shows a matrix division method in the data completion unit.
Detailed Description
The present invention will now be described in further detail with reference to the accompanying drawings.
A mobile phone end common continuous behavior identification application framework containing missing data specifically comprises the following steps:
step one, a data source comprises three inertial sensorsS 1 、S 2 、S 3 The sensors are an accelerometer (including gravity), a linear accelerometer (including gravity) and a gyroscope, and each sensor collects data independently to form an input data stream, and the input data streams are organized in time sequence and stored by a sensor control unit. The process is as follows:
1.1 the sensor control Unit creates a cache of predetermined sizeD；
1.2, each sensor carries out data acquisition according to the same sampling frequency, and the data obtained by each sensor is stored inDStoring the data in different sensors separately, wherein the data in each sensor are stored in sequence according to time;
1.3, according to other units and the requirements of practical applicationDThe maximum retention time of the buffer data of each sensor ishSecond, data exceeding the retention time is written into a fixed storage device (such as a mobile phone SD card), so that the occupation of excessively old data is avoidedDThe space of (a).
Step two, from the current momenttInitially, the sensor control unit is sized tow(second) <wPositive real number, depending on the particular application) of a sliding window, selected forward from among the data buffered by each sensort-wTotThe data of (a); for each sensor data at the same moment, splicing the sensor data into the same vectord i The vectors within the window are then organized into shapesD t = < d 1 , d 2 ,… , d t >The raw data is passed to the data admission control unit. The process is as follows:
2.1, reasonably selecting the time slicing granularity of the sliding window according to other units and the requirement of practical applicationw；
2.2, starting from the current time, according to the sizew(second) <wPositive real number, depending on the particular application) of a sliding window, selected forward from among the data buffered by each sensort-wTotThe data of (a);
2.3 for the same timeiSplicing the data of the sensors into the same vectord i The vectors within the window are organized into shapesD t = < d 1 , d 2 ,… , d t >The vector is passed as raw data to the data admission control unit.
And step three, the data admission control unit comprises a pre-trained binary classifier model, and can judge whether the mobile phone is on the user or not in the period of time. The vector is taken as input and the output is either on the user or not. And if the mobile phone is judged not to be on the body of the user, judging the data in the time interval to be invalid data to be discarded, and simultaneously feeding back to the sensor control unit according to a determined strategy to close part of the sensors or reduce the sampling frequency. If the mobile phone is judged to be on the body of the user, the effective data is transmitted to the data completion unit and fed back to the sensor control unit, and the sensor needs to be turned on and the sampling frequency needs to be increased to obtain accurate data. The process is as follows:
3.1, inputting a data vector transmitted by the sensor control unit into a classifier which is trained in advance, wherein the classifier can judge whether the mobile phone is on the body of a user only by acquiring partial inertial sensor data at low cost;
3.2, if the output result is that the mobile phone is near the user, the data is delivered to the data completion unit; and fed back to the sensor control unit, requiring the sensor to be turned on and the sampling frequency to be increased to obtain accurate data.
3.3, if the output result is that the mobile phone is not near the user, judging the data in the time period as invalid data to be discarded; and feeding back the sensor control unit according to the determined strategy, and closing part of the sensors or reducing the sampling frequency.
Step four, the data complementing unit collects the effective data transmitted from the data receiving control unit, and the effective data is transmitted from the current momenttAt the beginning, according to the sizeS(in seconds) with an overlap ratio ofλThe number of days isd(day) ((S，λ，dPositive and real, depending on the particular application) selected from the current and previous daysd-1Day data, selected from the current time of daytAt the beginning, choose to forward tot- SAnd organizing the data into matrixes according to the data at the moment, marking missing values in the data matrixes by using the indicating matrixes, transmitting the two matrixes as input to a preset data completion algorithm, and completing the effective data containing the missing values. And transmitting the completed complete data to an upper-layer behavior identification unit. The process is as follows:
4.1, the data completion unit comprises a runtime cache with a predetermined size, the cache divides the historical data according to the form of each day and arranges the data of one day into a line.
4.2 data completion unit is according to sizeS(in seconds) with an overlap ratio ofλThe number of days isd(day) ((S，λ，dPositive and real, depending on the particular application) selected from the current and previous daysd-1Day data, selected from the current time of daytAt the beginning, choose to forward tot-SThe data at the time is organized into a new matrix, the matrix comprises a plurality of missing data, and the specific position is marked by an indication matrix to be missing.
And 4.3, the data completion unit comprises a packaged data completion algorithm, the data matrix and the indication matrix are used as input and transmitted to the data completion algorithm, and the output of the algorithm is a complete matrix which does not contain missing values. And transmitting the complete matrix to an upper-layer behavior identification unit.
And step five, the upper-layer behavior recognition unit determines the corresponding function according to the specific application and completes the corresponding perception recognition operation on the complete data set. The process is as follows:
and 5.1, the behavior recognition unit comprises a preset behavior recognition algorithm, and data processing is carried out on the complete data set to obtain output related to specific application.
Further explanation follows from three aspects of the hardware environment, application scenarios, and method descriptions.
Hardware environment
1. The system is deployed on a smart phone, and the smart phone needs to comprise an acceleration sensor (including gravity), a linear sensor (without gravity) and a gyroscope which can independently and continuously acquire data, and the acceleration sensor, the linear sensor and the gyroscope are converged into a continuous data stream;
2. the calculation is carried out by relying on a CPU (central processing unit) of the smart phone, and the storage is carried out by relying on a storage space of the smart phone (when the storage data volume is too large, the storage is carried out by relying on a cloud).
Second, application scenario
When the method disclosed by the invention is applied to construct a general continuous behavior recognition application framework containing missing data at a mobile phone end, firstly, sensor data acquired by the mobile phone end needs to be transmitted to a sensor control unit, and a user specifies the size of a fragment according to the requirements of specific behavior recognition applicationwBuffer sizeh. The sensor control unit organizes the sensor data at the same moment into a vector according to the time scale of the data, organizes the vector at each moment according to the time sequence, divides the vector into corresponding windows according to the preset slice size, and transmits the data to the admission control unit. And the admission control unit obtains the judgment whether the current mobile phone is around the user through a classifier trained in advance by the system according to the received sensor data. If the user is judged not to be nearby, only a few sensors can be turned on to reduce the energy consumption of the system. And transferring the data containing the missing value after the invalid data is removed to the data completion unit. The data completion unit specifies parameters according to specific applicationS，λ，dAccording to the selection of the data processing method, missing data is completed through the time smoothness and periodicity of the data according to real-time data and historical data, and complete data is transmitted to upper-layer application.
The framework adds a layer of sensor control unit, a data receiving control unit and a light-weight data completion unit in the original behavior recognition application and sensor, and invalid data can be effectively screened out and the missing data can be completed through the units, and the switch and sampling frequency of the sensor can be dynamically controlled, so that the behavior recognition application can obtain higher accuracy, and the energy consumption of the system is remarkably reduced. The framework does not interfere with the original behavior recognition application and data acquisition process, is easy to realize, and can be applied to various behavior recognition systems.
A typical application scenario of the technology related by the invention is based on the fact that a smart phone judges the motion state of a user, and judges that the user is currently in three states of static state, walking state and cycling state. The smart phone is placed close to the user, information (acceleration (without gravity), acceleration (with gravity) and a gyroscope) of an inertial sensor on the smart phone is collected, and the current action of the user is identified through a series of mode identification methods. However, although the user is expected to carry the mobile phone with him or her, in reality, the user sometimes places the mobile phone on a desk, which results in invalid data being collected, and the mobile phone may turn off the sensor when the internal memory of the scheduling system of the mobile phone is insufficient, which may result in data collection failure. Therefore, the proposed framework technology is added to the behavior recognition application, and the design of the system is optimized, so that the energy consumption of the system is remarkably optimized, and the accuracy of the system is stable.
Description of the method
The invention relates to a general continuous behavior recognition application framework containing missing data at a mobile phone end, and a data processing flow chart is shown in figure 1.
1. Data fragmentation and caching techniques
The invention relates to a general continuous behavior identification application framework method containing missing data at a mobile phone end, which is based on the technology of fragmenting and caching an input data column. In this framework, it is specified that each sensor performs sampling at the same sampling frequency when performing sampling. And organizing the data from each sensor at the same moment into a vector as a data vector at the moment. Assume that the current time istAccording to application requirements, the time slice can be determined to be of the sizewSecond, the data freshness that the behavior recognition application can tolerate ishSecond (hAn integer multiple of w). For data from sensors, after organizing them in vector form, data slicing and buffering techniques first utilize a length ofwThe second sliding window will input the data streamDSlicing into temporally disjoint data segmentsD[t-w,t], D[t-2w, t-w], ..., D[i, j], ..., D[t-h, t-h+w]WhereiniAndjrespectively, the start and end times of the data contained in the data segment. The data may be collected at different sampling frequencies at different times, so that when the data segment is cached, a dynamic storage space needs to be opened up to store the cached data. In view ofhThe value of (a) is often small, so that the caching of data can be completed in a memory.
2. Data admission control unit architecture
According to the characteristics of the data at the current moment, whether the mobile phone is horizontally placed on the table or not and whether the mobile phone moves or not can be judged, and whether the mobile phone is on the body of the user or not can be judged according to the information. Therefore, the data admission control unit trains a classifier of the state whether the mobile phone is on the body according to the thought, and the classifier is embedded into the data admission control unit as a preset classifier. The classifier can judge whether the mobile phone is on the body of the user only by collecting partial inertial sensor data at low cost.
As shown in fig. 1, the data receiving and controlling unit uses the fragmented original data segment transmitted from the sensor control module as input, and inputs the input into the embedded classifier, and the output result is two classifications, and the classification result indicates whether the mobile phone is on the body of the user.
As shown in fig. 1, the data admission control unit feeds back to the sensor control unit, and when the output result is that the user is on, the data admission control unit feeds back to the sensor control unit to turn on the sensor and increase the sampling frequency; when the output result is that the user is not present, the feedback sensor control unit controls the feedback sensor control unit to turn off the sensor and reduce the sampling frequency.
After passing through the data admission control unit, the invalid data in the original data are removed, and the valid data are transmitted to the data completion unit, wherein the removed invalid data part and the missing part in the original data are uniformly marked as missing values.
3. Light-weight data completion unit structure
Consider that temporally successive data will have a time smoothness and possibly periodicity between data for each day. Thus, the existing data completion algorithm using correlation characteristics is utilized and embedded in the data completion unit.
The lightweight data completion unit includes a runtime cache that arranges the data containing missing values passed from the data admission control in a one-day fashion, as shown in fig. 2. According to the division mode shown in FIG. 2, according to the preset parametersS，λ，dAnd segmenting the data by utilizing a sliding window technology. At the moment of timetA data matrix with missing values will be obtained. The matrix is input into an embedded data completion algorithm, and the output result is a complete data matrix which can be used by an upper-layer behavior identification unit.
The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above-mentioned embodiments, and all technical solutions belonging to the idea of the present invention belong to the protection scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may be made by those skilled in the art without departing from the principle of the invention.
Claims (4)
1. A mobile phone end common continuous behavior identification application framework containing missing data is characterized by comprising: the system comprises a data source, a sensor control unit, a data receiving control unit, a data completion unit and a behavior identification unit;
the data source comprises three inertial sensors, namely a gravity accelerometer, a linear accelerometer and a gyroscope, and the sensors interact with the environment and a user to acquire real-time data;
the sensor control unit controls the switch and sampling frequency of the sensor according to the feedback information of the data admission control unit;
the data admission control unit judges whether the data is valid data according to the current data, if the data is valid data, the data is transmitted to the data completion unit and the sensor control unit is fed back, otherwise the sensor control unit is fed back to reduce the sampling frequency;
the data completion unit completes the missing data to generate complete data which is supplied to the subsequent behavior recognition unit for recognition;
the work of the application framework specifically comprises the following steps:
1) each sensor in the data source collects data independently, forms an input data stream respectively, organizes the data according to time sequence and stores the data by the sensor control unit;
2) from the current moment t, the sensor control unit selects data from t-w to t forward from the data cached by each sensor according to a sliding window with the size of w; for each sensor data at the same moment, splicing the sensor data into the same vector diThe vectors within the window are then organized as Dt＝<d1,d2,…,dt>Transmitting the original data to a data admission control unit; wherein w is a positive real number and the unit is second;
3) the data admission control unit comprises a pre-trained two-class classifier model and is used for judging whether the mobile phone is on the body of the user or not in the period of time; taking the vector as input and outputting as on or not on the user; if the mobile phone is judged not to be on the body of the user, judging the data in the time period to be invalid data to be discarded, simultaneously feeding back to the sensor control unit according to a determined strategy, and closing part of the sensors or reducing the sampling frequency; if the mobile phone is judged to be on the body of the user, the effective data is transmitted to the data completion unit and fed back to the sensor control unit;
4) the data completion unit collects effective data transmitted from the data admission control unit, selects data of the day and d-1 days before the day from the current time t according to a sliding window with the size of S, the overlapping rate of lambda and the number of days of d, selects data from the current time t to the t-S moment forward every day, organizes the data into a matrix, marks missing values in the data matrix through an indication matrix, transmits the two matrices as input to a preset data completion algorithm, completes the effective data containing the missing values, and transmits the completed complete data to the upper-layer behavior recognition unit; wherein S, lambda and d are positive real numbers, and the unit of d is day;
5) and the behavior recognition unit completes corresponding behavior recognition operation on the complete data set.
2. The application framework for identifying the general continuous behaviors containing missing data at the mobile phone end according to claim 1, characterized in that: the specific process of the step 1) is as follows:
1.1) the sensor control unit creates a cache D with a predetermined size;
1.2) carrying out data acquisition on each sensor according to the same sampling frequency, storing data obtained by each sensor into a storage device D, and storing the data separately according to different sensors, wherein the data in each sensor are stored sequentially according to time;
1.3) defining the longest retention time of the cache data of each sensor in the D as h seconds, and writing the data exceeding the retention time into the fixed storage equipment.
3. The application framework for identifying the general continuous behaviors containing missing data at the mobile phone end according to claim 1, characterized in that: the specific process of the step 4) is as follows:
4.1) the data completion unit comprises a runtime cache with a predetermined size, the cache divides the historical data according to the form of each day and arranges the data of one day into a line;
4.2) the data complementing unit selects data from the day and d-1 days before the day according to a sliding window with the size of S, the overlapping rate of lambda and the number of days of d, selects data from the current time t in each day and selects the data from the t-S time forward, and organizes the data into a new matrix, wherein the matrix comprises a plurality of missing data, and a specific position value is marked to be missing by an indication matrix;
and 4.3) the data completion unit contains a packaged data completion algorithm, the data matrix and the indication matrix are used as input and transmitted to the data completion algorithm, the output of the algorithm is a complete matrix which does not contain missing values, and the complete matrix is transmitted to the upper-layer behavior identification unit.
4. The application framework for identifying the general continuous behaviors containing missing data at the mobile phone end according to claim 1, characterized in that: the specific process of the step 5) is as follows:
and 5.1) the behavior recognition unit comprises a preset behavior recognition algorithm, and performs data processing on the complete data set to obtain output related to specific application.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810544629.0A CN108763045B (en) | 2018-05-31 | 2018-05-31 | Universal continuous behavior recognition application framework containing missing data at mobile phone terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810544629.0A CN108763045B (en) | 2018-05-31 | 2018-05-31 | Universal continuous behavior recognition application framework containing missing data at mobile phone terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108763045A CN108763045A (en) | 2018-11-06 |
| CN108763045B true CN108763045B (en) | 2021-09-28 |
Family
ID=64000850
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810544629.0A Active CN108763045B (en) | 2018-05-31 | 2018-05-31 | Universal continuous behavior recognition application framework containing missing data at mobile phone terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108763045B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104484673A (en) * | 2014-12-05 | 2015-04-01 | 南京大学 | Data complementation method for pattern recognition application of real-time data flow |
| CN105812571A (en) * | 2016-04-26 | 2016-07-27 | 上海保橙网络科技有限公司 | Method for automatically judging user driving behavior by using mobile phone |
| CN106501834A (en) * | 2016-10-11 | 2017-03-15 | 上海保橙网络科技有限公司 | A kind of method of use mobile phone automatic decision user movement behavior |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2016212066A (en) * | 2015-05-13 | 2016-12-15 | 富士通株式会社 | Moving body terminal, sensor value interpolation method, sensor value interpolation program, behavior recognition unit and behavior recognition system |
-
2018
- 2018-05-31 CN CN201810544629.0A patent/CN108763045B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104484673A (en) * | 2014-12-05 | 2015-04-01 | 南京大学 | Data complementation method for pattern recognition application of real-time data flow |
| CN105812571A (en) * | 2016-04-26 | 2016-07-27 | 上海保橙网络科技有限公司 | Method for automatically judging user driving behavior by using mobile phone |
| CN106501834A (en) * | 2016-10-11 | 2017-03-15 | 上海保橙网络科技有限公司 | A kind of method of use mobile phone automatic decision user movement behavior |
Non-Patent Citations (2)
| Title |
|---|
| "Energy-Efficient Continuous Activity Recognition on Mobile Phones: An Activity-Adaptive Approach";Zhixian Yan 等;《2012 16th International Symposium on Wearable Computers》;20120622;第17-24页 * |
| "基于可穿戴传感器网络的人体行为识别技术研究";汪亮;《中国博士学位论文全文数据库（信息科技辑）》;20150315(第03期);第I138-51页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108763045A (en) | 2018-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108805083B (en) | Single-stage video behavior detection method | |
| US11687759B2 (en) | Neural network accelerator | |
| Keshavarzi et al. | Edge intelligence—On the challenging road to a trillion smart connected IoT devices | |
| US20240119286A1 (en) | Adaptive artificial neural network selection techniques | |
| CN111488983B (en) | Lightweight CNN model calculation accelerator based on FPGA | |
| US20210295174A1 (en) | Systems and methods for providing flexible, multi-capacity models for use of deep neural networks in mobile devices | |
| CN110991642A (en) | Network cutting optimization method based on network activation and sparsification | |
| CN108551490B (en) | Industrial stream data coding and decoding system and method | |
| CN103810246B (en) | A kind of index creation method and apparatus and search index method and apparatus | |
| Huang et al. | DeepAdapter: A collaborative deep learning framework for the mobile web using context-aware network pruning | |
| CN108923983A (en) | Prediction technique, device and the readable storage medium storing program for executing of opportunistic network link | |
| CN106130930A (en) | A kind of Frame in advance join the team process device and method | |
| CN109598250A (en) | Feature extracting method, device, electronic equipment and computer-readable medium | |
| CN108763045B (en) | Universal continuous behavior recognition application framework containing missing data at mobile phone terminal | |
| CN107168802A (en) | The merging method and device of a kind of cloud storage small file | |
| CN110187838A (en) | Data IO information processing method, analysis method, device and relevant device | |
| CN110929885A (en) | Smart campus-oriented distributed machine learning model parameter aggregation method | |
| CN110276689A (en) | Intelligent contract implementation method based on dynamic decision | |
| CN108304860B (en) | Efficient classifier stacking framework for multi-mode fusion mode recognition application | |
| CN105408824B (en) | Programmable logic controller (PLC) | |
| CN117593623A (en) | Lightweight vehicle detection method based on improved YOLOv8n model | |
| KR20220147662A (en) | A lightweight artificial intelligence layer that controls the delivery of big data | |
| CN112306951A (en) | CNN-SVM resource efficient acceleration architecture based on FPGA | |
| CN107547378A (en) | A kind of VPN route learnings method and apparatus | |
| CN110188160A (en) | Date storage method and method for reading data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |