CN107172590B

CN107172590B - Mobile terminal and activity state information processing method and device based on same

Info

Publication number: CN107172590B
Application number: CN201710527184.0A
Authority: CN
Inventors: 赵洪乐; 革明新
Original assignee: Beijing Qihoo Technology Co Ltd
Current assignee: Beijing Qihoo Technology Co Ltd
Priority date: 2017-06-30
Filing date: 2017-06-30
Publication date: 2020-07-10
Anticipated expiration: 2037-06-30
Also published as: CN107172590A

Abstract

The invention provides a mobile terminal-based activity state information processing method, a device and a mobile terminal, wherein the method comprises the following steps: acquiring a historical activity state information set for recording activity states of a user in different time periods through one or more sensors in the mobile terminal; screening out at least one activity place from the plurality of activity places, wherein the activity type belongs to a designated activity type, and the corresponding time period meets a preset condition; and determining the at least one screened activity place as a frequent stopping place of the user. Therefore, the method provided by the embodiment of the invention can be used for counting the activity rule of the user in a short time based on various user data acquired by the mobile terminal, further identifying the frequent stay place of the user, setting real-time reminding service and route planning based on the frequent stay place for the user by combining big data analysis, and better realizing the interaction between the user and the mobile terminal.

Description

Mobile terminal and activity state information processing method and device based on same

Technical Field

The present invention relates to the field of information processing technologies, and in particular, to a method and an apparatus for processing activity state information based on a mobile terminal, and a mobile terminal.

Background

With the development of internet application technology, mobile terminals (such as smart phones, tablet computers, smart watches, and the like) have been greatly appeared in people's daily life, and have become indispensable tools for people's daily life, work, and study. For example, a user can talk, take a picture, read, listen to music, play games through the mobile terminal, and can also realize rich functions including positioning, information processing, fingerprint scanning, identification card scanning, barcode scanning, alcohol content detection and the like.

However, in real life, people cannot acquire more information related to themselves through the mobile terminal, so that users cannot interact with the mobile terminal better. For example, currently, the mobile terminal can only count the walking steps through the activities of the user, and cannot acquire other information related to the user. Therefore, in the prior art, the mobile terminal cannot provide various information closely related to the activity of the mobile terminal for the user.

Disclosure of Invention

In view of the above, the present invention is proposed in order to provide an information processing method based on an activity state of a mobile terminal, a corresponding apparatus and a mobile terminal that overcome or at least partially solve the above problems.

According to an aspect of the present invention, there is provided a method for processing activity state information based on a mobile terminal, including:

acquiring a historical activity state information set for recording activity states of a user in different time periods through one or more sensors in the mobile terminal;

screening out at least one activity place from the plurality of activity places, wherein the activity type belongs to a designated activity type, and the corresponding time period meets a preset condition;

and determining the at least one screened activity place as a frequent stopping place of the user.

Optionally, after determining the at least one screened activity place as a frequent stopping place of the user, the method further includes:

marking the frequent stay place in a map on the mobile terminal by adopting a coordinate and/or text label mode;

and when an instruction for starting the map is received, opening the map interface, and loading and displaying a mark corresponding to the frequent stopping place on the map interface.

Optionally, marking the frequent stopping place in a map on the mobile terminal by using a coordinate and/or text label manner, including:

determining the longitude/latitude corresponding to the frequent stopping place;

generating coordinates corresponding to the frequent stay place by taking the longitude/latitude as coordinate values;

and marking the frequent stopping place by using the established coordinates.

acquiring a text label corresponding to the frequent stopping place, wherein the text label comprises at least one of a building name, an exclusive name generated for the frequent stopping place and a user-defined name;

and marking the frequent stopping place by using the text label.

Optionally, when the text label includes the proper name, acquiring a text label corresponding to the frequent stopping location, including:

acquiring preset related activity figure information aiming at the frequent stopping place;

and generating an exclusive name containing the information of the related active people for the frequent stopping place according to a preset rule.

Optionally, the historical activity state information set is generated as follows:

acquiring a pre-trained activity classification model, wherein the activity classification model is trained by taking a plurality of sensor data recorded on the mobile terminal and activity types respectively corresponding to the data as samples, and the sensor data comprises data recorded by at least one sensor;

reading current sensor data of the mobile terminal, and determining a first activity type corresponding to the read sensor data according to the activity classification model;

acquiring communication environment information of the mobile terminal;

correcting the first activity type according to the communication environment information to obtain a corrected second activity type;

generating the historical activity state information set according to at least one of the second activity type, the current activity location, and a time period in which the current activity location stays.

Optionally, the correcting the first activity type according to the communication environment information to obtain a corrected second activity type includes:

judging whether the first activity type is matched with the communication environment information;

if so, determining that the first activity type is the second activity type;

if not, the second activity type is determined again according to the first activity type and the communication environment information.

According to another aspect of the present invention, there is also provided a mobile terminal-based activity state information processing apparatus, including:

the mobile terminal comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is suitable for acquiring historical activity state information sets used for recording activity states of users in different time periods through one or more sensors in the mobile terminal;

the screening module is suitable for screening out at least one activity place, of which the activity type belongs to a specified activity type and the corresponding time period meets preset conditions, from the plurality of activity places according to the historical activity state information set;

and the determining module is suitable for determining the screened at least one activity place as a frequent stopping place of the user.

Optionally, the apparatus further comprises:

the marking module is suitable for marking the frequent stopping place in a map on the mobile terminal in a coordinate and/or text label mode after the screened at least one activity place is determined as the frequent stopping place of the user;

and the loading and displaying module is suitable for opening the map interface when receiving an instruction for starting the map, and loading and displaying the mark corresponding to the frequent stopping place on the map interface.

Optionally, the marking module is further adapted to:

and marking the frequent stopping place by using the established coordinates.

Optionally, the marking module is further adapted to:

and marking the frequent stopping place by using the text label.

Optionally, the marking module is further adapted to:

Optionally, the apparatus further comprises a generating module adapted to:

acquiring a pre-trained activity classification model, wherein the activity classification model is trained by taking a plurality of sensor data recorded by the mobile terminal and activity types respectively corresponding to the data as samples, and the sensor data comprises data recorded by at least one sensor;

acquiring communication environment information of the mobile terminal;

and generating the historical activity state information set according to at least one of the second activity type, the current activity place and the current time period.

Optionally, the generating module is further adapted to:

if so, determining that the first activity type is the second activity type;

According to yet another aspect of the present invention, there is provided a computer program comprising computer readable code which, when run on a computing device, causes the computing device to perform the mobile terminal based activity state information processing method.

According to a further aspect of the invention, a computer-readable medium is provided, in which said computer program is stored.

According to another aspect of the embodiments of the present invention, there is provided a mobile terminal including a processor and a memory:

the memory is used for executing the program of any one of the above information processing methods based on the activity state of the mobile terminal,

the processor is configured to execute programs stored in the memory.

According to the activity state information processing method based on the mobile terminal, provided by the embodiment of the invention, the historical activity state information set used for recording the activity state of the user in different time periods can be obtained through one or more sensors in the mobile terminal carried by the user. The historical activity state information set records activity states and related information of various users, and provides a necessary basis for subsequent targeted screening of user information. Further, at least one activity place, of which the activity type belongs to the specified activity type and the corresponding time period meets the preset conditions, is screened out from the plurality of activity places according to the acquired historical activity state information set. The screened activity sites indicate that the user frequently acts at the site, and further, at least one screened activity site is determined as a frequent stay site of the user. Therefore, the activity state information processing method based on the mobile terminal provided by the embodiment of the invention can acquire the activity state information of the user at any time and any place through the mobile terminal carried by the user, has stronger real-time performance, accuracy and comprehensiveness, and is greatly improved compared with the prior art that the step number statistical information of the user can be acquired only through the mobile terminal. Meanwhile, the method provided by the embodiment of the invention can also be used for counting the activity rule of the user in a short time and further identifying the frequent staying place of the user based on various user data acquired by the mobile terminal. Furthermore, the embodiment of the invention can also set real-time reminding service and route planning and the like based on the frequently staying places for the user according to the identified frequently staying places of the user in combination with big data analysis, better realizes the interaction between the user and the mobile terminal, and provides great convenience for the user.

The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.

The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:

fig. 1 is a process flow diagram of a mobile terminal based active state information processing method according to an embodiment of the present invention;

fig. 2 is a detailed flowchart of a method for processing activity status information based on a mobile terminal according to an embodiment of the present invention;

FIG. 3 is a flow diagram of a method of training a classification model according to one embodiment of the invention;

FIG. 4 is a schematic illustration of marking frequent stops locations on a map, according to one embodiment of the present invention;

FIG. 5 is another schematic illustration of marking frequent stops locations on a map, in accordance with one embodiment of the present invention;

FIG. 6A is a first diagram of a frequent flyover location as a search suggestion for recommendation, according to an embodiment of the invention;

FIG. 6B is a second schematic diagram of a frequent flyover location being recommended as a search suggestion, according to an embodiment of the invention;

FIG. 7 is a data trace diagram of an acceleration sensor on a mobile terminal according to one embodiment of the present invention;

FIG. 8 is a schematic diagram of a user tag, according to one embodiment of the present invention;

fig. 9 is a schematic block diagram of a mobile terminal-based activity state information processing apparatus according to one embodiment of the present invention;

fig. 10 is another schematic block diagram of a mobile terminal-based activity state information processing apparatus according to an embodiment of the present invention;

fig. 11 is another schematic block diagram of a mobile terminal-based activity state information processing apparatus according to an embodiment of the present invention;

fig. 12 is a schematic structural diagram of a mobile terminal according to an embodiment of the present invention;

fig. 13 is a block diagram of a computing device for executing a mobile terminal-based activity state information processing method according to the present invention, according to an embodiment of the present invention; and

fig. 14 is a memory unit for holding or carrying program codes for implementing a mobile terminal-based active state information processing method according to an embodiment of the present invention.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

In order to solve the above technical problem, an embodiment of the present invention provides an active state information processing method based on a mobile terminal. Fig. 1 is a process flow diagram of a mobile terminal-based activity state information processing method according to an embodiment of the present invention. As shown in fig. 1, the method for processing the activity state information based on the mobile terminal at least includes steps S102 to S108:

step S102, acquiring a historical activity state information set for recording activity states of a user in different time periods through one or more sensors in the mobile terminal;

s104, screening at least one activity place of which the activity type belongs to the specified activity type and the corresponding time period meets the preset conditions from the plurality of activity places according to the historical activity state information set;

and step S106, determining the screened at least one activity place as a frequent stopping place of the user.

According to the activity state information processing method based on the mobile terminal, provided by the embodiment of the invention, the historical activity state information set used for recording the activity state of the user in different time periods can be obtained through one or more sensors in the mobile terminal carried by the user. The historical activity state information set records the activity states of various users and provides a necessary basis for subsequent targeted screening of user information. Further, at least one activity place, of which the activity type belongs to the specified activity type and the corresponding time period meets the preset conditions, is screened out from the plurality of activity places according to the acquired historical activity state information set. The screened activity sites indicate that the user frequently acts at the site, and further, at least one screened activity site is determined as a frequent stay site of the user. Therefore, the activity state information processing method based on the mobile terminal provided by the embodiment of the invention can acquire the activity state information of the user at any time and any place through the mobile terminal carried by the user, has stronger real-time performance, accuracy and comprehensiveness, and is greatly improved compared with the prior art that the step number statistical information of the user can be acquired only through the mobile terminal. Meanwhile, the method provided by the embodiment of the invention can also be used for counting the activity rule of the user in a short time and further identifying the frequent staying place of the user based on various user data acquired by the mobile terminal. Furthermore, the embodiment of the invention can also set real-time reminding service and route planning and the like based on the frequently staying places for the user according to the identified frequently staying places of the user in combination with big data analysis, better realizes the interaction between the user and the mobile terminal, and provides great convenience for the user.

The mobile terminal mentioned in the above step S102 may include a smart phone, a tablet computer, a smart watch, a notebook computer, a mobile POS machine, etc., which are currently widely used in daily life of people. According to the embodiment of the invention, the historical activity state information used for recording the activity state of the user in different time periods can be obtained based on the mobile terminal carried by the user.

Specifically, in performing step S102, sensor data generated by the current behavior of the user may be acquired based on a sensor of the mobile terminal. The sensor of the mobile terminal may be an accelerometer (accelerometer), a magnetometer (magnetometer), a gyroscope (gyroscope), etc., and the embodiment of the present invention is not limited thereto. The sensors enable the mobile terminal to have various functions, wherein the identification of the user activity state in the embodiment of the invention uses the measurement data of each sensor in the mobile terminal. Accordingly, an accelerometer is a meter that measures the linear acceleration of a vehicle, which is the most active sensor for activity recognition, and measures the acceleration (m/s) acting in three axial directions (x, y, z) of the device²) Acceleration by gravity is included. The magnetometer measures the magnetic field strength values (A/m) acting in three axial directions (x, y, z) of the device. The gyroscope measures the speed (rad/s) at which the device rotates about three axes (x, y, z). Various sensors of the mobile terminal are capable of measuring and collecting data for different behaviors of the user.

After different sensor data generated by different activity states of a user are acquired according to a sensor of the mobile terminal, a historical activity state information set can be generated for the user according to the acquired sensor data. Specifically, in the embodiment of the present invention, a pre-trained activity classification model may be obtained, where the activity classification model is trained by using, as samples, a plurality of sensor data recorded on a mobile terminal carried by a user and an activity type corresponding to each data, and the sensor data in the embodiment of the present invention includes data recorded by at least one sensor.

Further, reading current sensor data of the mobile terminal carried by the user, and determining a first activity type corresponding to the read sensor data according to the activity classification model. In addition, in order to improve the accuracy of the determined activity type, after the first activity type is determined according to the current data of the user mobile terminal sensor, the communication environment information of the mobile terminal can be further acquired, and then the first activity type is corrected according to the communication environment information to obtain a corrected second activity type. In embodiments of the present invention, the historical activity state information set may be generated based on at least one of a second activity type, a current activity location, and a time period of stay at the current activity location.

Specifically, fig. 2 shows a flowchart of a method for processing activity status information based on a mobile terminal according to an embodiment of the present invention, and in fig. 2, the method may include at least the following steps S202 to S204:

step S202, extracting a feature vector in the current sensor data, and inputting a classification model from the feature vector to the category of the user activity state;

and step S204, identifying the category of the user activity state corresponding to the current sensor data by using the classification model.

It can be seen that the embodiment of the invention fully utilizes the sensor of the mobile terminal, reads the sensor data collected by the sensor, and can identify the daily activity state of the user using the mobile terminal based on the sensor data and the classification model. In addition, when the sensor data is processed, the feature vectors in the sensor data are extracted in a targeted manner, so that the accuracy of the identification result can be improved.

In the embodiment of the present invention, after the sensor data is read, step S202 is executed to extract the feature vector from the read sensor data, and then the extracted feature vector is input into the classification model to further identify the category of the user activity state, such as still, walking, riding, running, mountain climbing, driving, bus sitting, subway sitting, train sitting, and the like. The feature vectors in the classification model correspond to the extracted feature vectors, so the embodiment of the present invention first introduces a training step of the classification model, fig. 3 shows a flowchart of a training method of the classification model according to an embodiment of the present invention, and in fig. 3, the method at least includes the following steps S302 to S304:

step S302, collecting data collected by a sensor on the mobile terminal to obtain a data sequence;

and S304, taking the data sequence as a training sample to extract the feature vector, mapping the feature vector to the category of the user activity state, and training a classification model from the feature vector to the category of the user activity state by using a machine learning algorithm.

The above steps S302 and S304 may mainly include data collection, preprocessing, data segmentation, feature extraction, and machine learning, which are described in detail below.

In the data collection step, data collected by a sensor on the mobile terminal are collected to obtain a data sequence. That is, from the sensors described above, a sensor data sequence during a user activity can be obtained, and particularly, the data can be sampled at a constant sampling frequency (e.g., 50 hz). Furthermore, the data collected by the sensor may include data in a plurality of directions, and thus, the data sequence may also include data sequences in corresponding respective directions. For example, an accelerometer, a geomagnetic sensor, and a gyroscope all have readings in three axes, and thus for each sensor, a sequence of signal values in three axes x, y, and z is obtained. In the data collection process, the embodiment of the present invention needs to collect sufficient samples for the user activity states of the above categories, and the user activity states of the same category may also consider various different situations such as the posture of the mobile terminal.

In the preprocessing step, some mobile terminals do not necessarily support all sensors, and therefore, the embodiments of the present invention extract data of different sensor combinations from the collected data. That is, multiple types of sensors are combined to obtain multiple sets of sensors. Each group of sensors comprises one or more types of sensors, and data collected by each group of sensors on the mobile terminal are collected to obtain data sequences corresponding to each type of sensors in each group of sensors. Since accelerometers are the most dominant sensors, embodiments of the present invention select three possible combinations: (1) an accelerometer; (2) an accelerometer and a geomagnetic sensor; (3) the system comprises an accelerometer, a geomagnetic sensor and a gyroscope, and data of the three conditions are trained respectively.

In the data segmentation step, the sensor data sequence is segmented into sub-sequences, each sub-sequence being mapped to a corresponding active tag. For data sampled at a constant sampling frequency, such as accelerometers, geomagnetic sensors, and gyroscopes, embodiments of the present invention segment the entire sensor data sequence into subsequences of equal time interval. Therefore, in the embodiment of the invention, different intervals of 1 to 6 seconds are respectively selected to be segmented to obtain subsequences, data with different scales are respectively learned during machine learning classification to obtain recognition models based on different time window sizes, and then the recognition effects of the different models are evaluated to select the optimal segmentation interval. It should be noted that the time interval of 1 to 6 seconds is only illustrative and not limiting to the embodiment of the present invention, and in practical applications, the time interval of 7 or 8 seconds and the like may be selected to divide the sequence.

In the sample set obtained after the data segmentation is finished, one part of the sample set can be selected as the input of subsequent machine learning, and the rest part of the sample set can be used as a test set for evaluating the classification effect of the classification model.

In an optional embodiment of the present invention, when each time window is used to perform segmentation processing on a data sequence, each time window generates a subsequence, and the subsequences do not overlap with each other, so as to obtain a plurality of subsequences corresponding to the time window as training samples; or sliding each time window on the data sequence, generating a subsequence corresponding to the current time window every time the time window is slid, and obtaining a plurality of subsequences corresponding to each time window as training samples.

In the feature extraction step, for the segmented sensor data subsequence, the following formula is exemplified in the embodiment of the present invention to calculate the following feature quantity to obtain the feature vector of the sequence, but it should be noted that the technical solution protected by the present invention is not limited to the solution implemented by the following formula and listed in the description, and those skilled in the art can think that the technical solution of the present invention can still be implemented by reasonably changing, increasing and decreasing the proportion, adjusting the coefficient, increasing the weight, and the like, and these non-exhaustive solutions are also within the scope of the present invention.

1) Mean value (mean)

For a sequence S of size n, whose mean is the sum of the addition of n signal values divided by the sequence size, the calculation formula is as follows:

Mean(S)＝Sum(S)/n

2) standard deviation (standard deviation)

The standard deviation is the square root of the arithmetic mean of the squared deviation of each signal value from the mean, and may reflect the degree of sample variation.

For sequences x of size n₁,x₂…x_nThe mean is μ, and the standard deviation formula is:

3) median (mean)

The median is the number located at the middle position after the data sequence is arranged in sequence. For sequences with the size of n, sorting the sequences in ascending order, and if n is an odd number, taking (n +1)/2 numbers as median; if n is even number, the average of the n/2 th number and the n/2+1 th number is taken as the median.

4) Median absolute deviation (mean absolute deviation)

The median absolute deviation relative standard deviation may provide a better measure of the degree of change in some cases where there are outliers that may affect the mean. And subtracting the median of the sequence from each item of the sequence, then calculating an absolute value to obtain a new sequence, and calculating the median of the new sequence to obtain the absolute deviation of the median.

For sequence x₁,x₂…x_nThe median absolute deviation is:

MAD＝Median(|x_i-Median(X)|)

5) zero crossing (zero crossing)

The zero crossings are the number of times the signal value crosses the median position, and this characteristic quantity is used to distinguish between fast and slow motion. The calculation method is to traverse each item value of the sequence, and if the median is greater than the current item and less than the latter item, or less than the current item and greater than the latter item, the number of zero crossings is increased by 1.

6) Correlation (correlation)

The sensor comprises a multi-component sensor, wherein the correlation exists among the components of the sensor, such as the correlation among the three directions of x, y and z of the accelerometer. There may be differences in the correlation of the components of different activity states. For sequences x and y of size n, the correlation is calculated as follows:

7) peak-to-peak amplitude

This value represents the peak-to-valley difference of the signal, i.e., the difference between the maximum and minimum values of the data series, and is calculated as follows:

P2PA(S)＝Max(S)-Min(S)

8) energy of signal

The signal energy refers to the area of the region formed between the signal curve and the time axis, and for a sequence of sensor values, the signal energy can be expressed as the sum of the squared values:

although the feature vectors mainly extracted from the data sequence or the sub-sequence according to the embodiment of the present invention are described above, the present invention is not limited thereto.

Next, in the machine learning step, the embodiment of the present invention uses a supervised learning method to map the feature vector to the category of the user activity state, and establishes a classification model from the feature vector to the category of the user activity state. Here, as described above, the data sequence corresponds to the label of the category of the user activity state, so that when mapping the feature vector to the category of the user activity state, the label of the category of the user activity state corresponding to the data sequence can be used as the label of the feature vector thereof, thereby mapping the feature vector of the data sequence to the category of the user activity state.

For different sensor combinations, different classification models are trained, and then the classification models can be utilized to identify the category of the user activity state corresponding to a new sample.

Different supervised learning algorithms have obvious difference on the recognition effect of the activity, and a Support Vector Machine (SVM) is selected as a main supervised learning method in the embodiment of the invention. The objective of SVM is to maximize prediction accuracy without causing overfitting of the data, for binary classification, the SVM algorithm builds a region model in the feature space, separating the data (feature vectors) into two different classes in the multidimensional space, and making the distance between the classification boundary and the nearest data point as far as possible. The embodiment of the invention selects a one-to-one method, namely a maximum-win election method (maximum-win election) to process the activity classification problem, trains an SVM model for each pair of different activity classes to finally obtain K (K-1)/2 SVM classifiers, obtains one classification for each classifier when classifying new samples, records the number of votes obtained for each class, obtains the class winner with the most votes, and determines the class of the user activity state corresponding to the new samples.

It should be noted that the machine learning SVM algorithm listed here is only illustrative, and the embodiment of the present invention may also use algorithms such as a decision tree, an artificial neural network, and naive bayes to implement classification, and may also compare and evaluate the effects of different learning algorithms, thereby selecting a suitable learning algorithm. Specifically, the learning effect evaluation can be realized by utilizing the test sample set, the recall rate, the accuracy rate and the F-Score are selected as evaluation indexes, and the better standard is achieved by continuously adjusting relevant parameters such as training samples, feature vectors and the like, so that the recognition effect is optimized.

After training to obtain the classification model, the classification model may then be used to identify the category of the user's activity state. That is, in step S202, a feature vector is extracted from the read sensor data, a classification model from the feature vector to a category of the user activity state is input, and in step S204, the category of the user activity state corresponding to the read sensor data is identified using the classification model. Here, when reading sensor data, samples may be generated at fixed time intervals, there are two ways of generating samples:

first, assuming that the required duration of samples is 5 seconds, one sample is generated every 5 seconds, and the samples do not overlap with each other;

second, assuming that the required duration of a sample is 5 seconds, a sample is generated every 1 second, and the samples overlap each other by taking the data within the previous 5 seconds.

In extracting the feature vector, the extracted feature vector is the same as that in training, that is, the mean, standard deviation, median absolute deviation, zero crossing, correlation, peak-to-peak amplitude, and signal energy of the data sequence or subsequence can be calculated by the method mentioned in the above feature extraction step, and these feature values are taken as the feature vector of the sequence. And then, selecting a proper classification model according to the sensor supported by the mobile terminal, so that the trained classification model classifies the sample, and the class of the user activity state corresponding to the read sensor data is identified.

In an optional embodiment of the present invention, in order to further improve the accuracy of the identification, before determining the current activity state information of the user, the auxiliary identification information of the mobile terminal may be further acquired, and then the category of the identified activity state of the user may be adjusted by using the acquired auxiliary identification information. In the embodiment of the present invention, the auxiliary identification information may be communication environment information of the mobile terminal, such as network data, location information, and/or application data. More specifically, the network data of the mobile terminal may be bluetooth data and/or WIFI data, and the location information of the mobile terminal may be GPS information, which is not specifically limited in the embodiment of the present invention. Meanwhile, the method provided by the embodiment of the invention refers to the identification of the category of the user activity state by using the classification model as the initial identification, and further, the identification result of the initial identification can be adjusted by using the auxiliary identification information.

When the category of the identified user activity state is adjusted by using the auxiliary identification information, the embodiment of the present invention may determine whether a specified communication module (such as bluetooth, WIFI, etc., but the embodiment of the present invention is not limited thereto) on the mobile terminal accesses a communication device of a corresponding protocol type. If yes, the related information of the communication equipment accessed by the communication module on the mobile terminal is obtained, and the category of the identified user activity state is adjusted based on the obtained related information. This is described in detail below by way of a few examples.

Condition one, bluetooth connected state

The embodiment of the invention can acquire the Bluetooth connection state and the type of the connected Bluetooth equipment, can identify the vehicle-mounted Bluetooth equipment by checking the type of the connected Bluetooth equipment, and can confirm or correct the driving state when the mobile terminal is confirmed to be connected with the vehicle-mounted Bluetooth equipment and the preliminarily identified user activity state is the driving state or the state close to the driving state.

Second, peripheral bluetooth device

Peripheral bluetooth device lists can be obtained through bluetooth scanning, and the device type and the quantity in them are analyzed, and statistics is found that the bluetooth device quantity that scans when driving is not too many through not too much. Therefore, a number threshold may be set, and if the number exceeds the number threshold and the vehicle-mounted bluetooth device is not present, and the preliminary recognition is driving, it is considered as a false recognition.

Situation three, GPS

The current position and the moving speed can be obtained by the GPS, and whether or not the vehicle is traveling can be confirmed by setting a speed threshold. Because the GPS is used for a long time and has higher power consumption, the embodiment of the invention only uses the GPS for auxiliary judgment when the activity state of the user is possibly switched and confirmed.

Fourth, WIFI

At present, all can install WIFI on many buses, these WIFI all have more obvious characteristics (like name etc.) usually, can extract these characteristics through data acquisition and analysis, if mobile terminal has connected WIFI, through the characteristic of the WIFI of judging the current connection, can assist and judge whether on the bus.

Case five, historical status

The historical state is used for state switching, when the historical state is judged to be switched to a new state, the historical state needs to be considered, whether the new state lasts for a specified time or not is confirmed, and if yes, switching to the new state is confirmed; if not, the switching to the new state is not considered.

When the identified category of the user activity state is adjusted by using the auxiliary identification information, the embodiment of the present invention may further acquire service data of a designated application program on the mobile terminal, and further adjust the identified category of the user activity state based on the acquired service data. For example, the user opens a designated application program, opens a bicycle lock by using the designated application program, and then enters a riding state, the designated application program records the riding time of the user from unlocking to locking, and when the embodiment of the invention acquires the business data from the designated application program, the activity state of the user is confirmed to be the riding state. For another example, the user starts a function of swiping a public transportation card, and after swiping the card, the public transportation card function records the time or the station of the user taking the public transportation.

In an optional embodiment of the present invention, when an event occurs that adjusts the category of the identified user activity state, the category of the adjusted user activity state is obtained, and then the classification model is trained using the feature vector in the sensor data and the category of the adjusted user activity state to correct the classification model.

After the execution of step S102 is finished, a historical activity state information set for recording the activity state of the user in different time periods may be acquired. In the historical activity status information set, categories of user activity status may include stationary, walking, running, driving, riding in public transportation, and the like, to which the present invention is not limited. In addition, according to the embodiment of the invention, when the current activity state information of the user is acquired, whether the activity state of the user is switched can be determined. Specifically, after the category of the user activity state is identified, it may be determined whether the current identification is an initial identification operation. If so, taking the category of the user activity state identified this time as the category of the user activity state determined this time. If not, acquiring the type of the user activity state determined last time, and determining whether the activity state of the user is switched or not based on the type of the user activity state determined last time and the type of the user activity state identified this time.

Further, if the type of the user activity state determined last time is the same as the type of the user activity state identified this time, it is determined that the user activity state of this time is not switched. If the type of the user activity state determined last time is different from the type of the user activity state identified this time, the type of the user activity state determined last time is called a first type, the type of the user activity state identified this time is called a second type, and the reading and identifying operations are continued. And when the number of times that the subsequently identified category of the user activity state is the second category exceeds the threshold, determining that the corresponding user activity state is switched when the number exceeds the threshold.

For example, the category of the activity state of the user is identified as a static state through the above steps, and in order to determine whether the user activity state is switched, it is first determined whether the current identification is an initial identification operation, and if so, the category of the user activity state identified this time is taken as the category of the user activity state determined this time; if not, the type of the user activity state which is obtained from the last determination is walking, at this time, the user activity state is not immediately determined to be switched to the static state, the steps are continuously executed to identify the type of the user activity state, and if the type of the user activity state is identified to be static for multiple times, the user activity state is determined to be switched to the static state.

In an optional embodiment of the present invention, after determining that the user activity state is the first activity state, reading current sensor data from a sensor of the mobile terminal, and recognizing that the user activity state is the second activity state based on the current sensor data; and if the second activity state is different from the first activity state and the duration of the user activity state being the second activity state is greater than a first specified time threshold, determining that the user activity state is switched from the first activity state to the second activity state. The first specified time threshold herein may be determined based on the first activity state and the second activity state.

If the second activity state is different from the first activity state and the duration of the user activity state being the second activity state is smaller than the first specified time threshold, continuing to read current sensor data from a sensor of the mobile terminal, and identifying that the user activity state is a third activity state based on the current sensor data. And if the third activity state is different from the first activity state and the second activity state, and the duration of the user activity state of the third activity state is greater than a second specified time threshold, determining that the user activity state is switched from the first activity state to the third activity state. The second designated time threshold here may be determined based on the first activity state and the third activity state, or may be determined based on the first activity state, the second activity state, and the third activity state.

As can be seen from the above description, when the state changes, before entering a new state, the embodiment of the present invention needs to confirm for a while to ensure the accuracy of the state, and the following is a basic policy in state switching.

First is the initial state. Before the initial identification, the state may be set to unknown since the state cannot be determined yet.

The second is state entry. Whether the original unknown state or other recognized state, entering the new state requires a steady state for a period of time to confirm the switch to the new state.

Again, the state exits. There are generally three cases where the original state is exited:

a. entering a new state;

b. the entry into the new state is not confirmed, but the state that has been recognized for a while does not conform to the old state. For simple conditions, this time is relatively short (e.g., 30 seconds); for complex states (such as driving, public transportation, etc.), the duration of these states usually includes other states, such as waiting for a traffic light to be stationary, and therefore exits only if the state is not recognized as the original state for a long time (such as 5 minutes);

c. the system or application exits.

Finally, state maintenance. After entering the new state, the state will be maintained until the exit condition is met.

Next, the switching conditions thereof are described in detail for the respective states.

1) With respect to static state

From the unknown, walking, running, etc. state to the stationary state, the stationary state is only required to be stable for a period of time (e.g., 5-15 seconds) to confirm the switch to the stationary state.

From a driving state to a stationary state, to avoid a change in state under such conditions as traffic lights, a longer stationary stabilization duration (e.g., more than 2 minutes) is required to switch states.

From sitting the public transit state to the quiescent condition, should not directly switch to the quiescent condition from sitting the public transit state usually, for avoiding abnormal conditions, set up a longer affirmation time (for example more than 5 minutes).

2) About walking and running states

From the unknown, stationary, etc. state to the walking or running state, the switching to the new state can be confirmed only by the walking or running state remaining stable for a period of time (e.g., 5-15 seconds).

In order to avoid the situation change caused by the close state such as fast walking and slow running, the switching between walking and running needs a longer confirmation time (e.g. 15-30 seconds).

Since the driving state is complicated to the walking or running state and the driving state may be erroneously recognized as another state such as walking when passing through a bumpy road, it is necessary to ensure that the new state is stable for a long time (e.g., 30 seconds or more) when the driving state is switched to the walking or running state. There is usually a stationary state of parking (not switched to stationary for a short time) between driving and walking, and if there is a stationary state before the walking state, the confirmation time can be reduced appropriately. To ensure accuracy, GPS may also be used to confirm that the current speed does not exceed the walking speed.

From the sitting bus state to the walking and running state, when taking the bus, the user may walk in the carriage or stand, so that the user needs to keep a new state for a long time (for example, more than 30 seconds) when entering the walking and running state. To ensure accuracy, GPS may also be used to confirm that the current speed does not exceed the walking speed.

3) About driving state

From an unknown, stationary state to a driving state, it is necessary for the driving state to stabilize for a longer period of time (e.g., more than 30 seconds), and the GPS is turned on for the current speed during the confirmation period, and if the speed is too low (e.g., less than 10 kilometers per hour) or there is no GPS signal for a continuous period of time (e.g., within 15 seconds), the driving state is not entered.

From the walking or running state to the driving state, the entering condition is similar to entering driving from unknown or static, but compared with the former case, from the walking or running state to the driving state, when the vehicle is not driven yet just entering the vehicle, the static state is usually existed for a period of time (the time is not switched to static for too short), therefore, the static state judgment before the driving state needs to be additionally increased, if the static state does not exist, the confirmation time needs to be properly increased.

From the bus sitting state to the driving state, the direct switching from the bus sitting state to the driving state is generally not allowed.

4) Sit in public transit, subway

Sit the bus and drive the more complicated state than, because of its state is more unstable relatively, for example the passenger can sit and can stand, also can frequently operate mobile terminal etc. consequently only combines WIFI, GPS etc. through the state of preliminary discernment, still be not enough accurate discernment, need further combine bus stop and line data to judge.

Specifically, in an optional embodiment of the present invention, when the user activity state is identified as a preset state based on sensor data of the mobile terminal, first location information where the mobile terminal user is located is obtained; then, matching the first position information with public transport stations and line data, and if the first position information is matched with the public transport stations and line data, acquiring second position information and moving speed of a mobile terminal user after a specified time length; and then, matching the second position information with public transport stations and line data, and if the second position information is matched with the public transport stations and the line data and the moving speed meets the preset conditions, determining that the user activity state is the public transport riding state.

When the specified time length is determined, if the first position information matches the public transportation station, determining the specified time length based on the type of the matched public transportation station, wherein the type of the public transportation station comprises any one of a starting station, an intermediate station and an ending station; and if the first position information matches the public transportation line, determining the specified time length based on the related information of the matched public transportation line, wherein the related information of the public transportation line comprises traffic light conditions and/or real-time road conditions.

Since the state from the unknown, walking, running, to the riding public transportation state is usually recognized as walking when the vehicle is loaded, it is common to switch from the walking state to the riding public transportation state. During the period of confirming whether the user enters the public transportation state, the embodiment of the invention starts the GPS on the mobile terminal, and acquires the first position information of the user of the mobile terminal through the started GPS. And then, matching the first position information with public transport stations and line data, judging whether the mobile terminal user is at or near the public transport stations or whether the mobile terminal user drives along the public transport lines, if so, acquiring second position information and moving speed of the mobile terminal user after a specified time, and if so, determining that the user activity state is a public transport riding state, wherein the moving speed meets a preset condition.

From a stationary state to a public transportation state, judging whether the station is near the public transportation station and whether the station is a starting station or not by combining the positioning position with the public transportation station and line data, if the station is not getting on from the starting station, the stationary state usually does not exist, and if the stationary state exists, the duration time can be short, so that a threshold value is set, and if the stationary time exceeds the threshold value, the station is not considered to be the public transportation state; in the case of the starting station, the threshold value may be set for a long time, and it is necessary to determine the historical state before the stop, and it is necessary to identify the walking state as the riding public transportation state.

When determining whether the user activity state is switched, the embodiment of the invention can also determine based on the state change of the mobile terminal. Specifically, when the state of the mobile terminal changes, acquiring related data used for identifying the user activity state on the mobile terminal, and identifying the user activity state as an undetermined activity state based on the acquired related data; then, judging whether a first determined activity state exists before the pending activity state; and if so, determining a second activity state of the user based on the pending activity state and the first activity state. Here, the state change of the mobile terminal may be various cases, for example, sensor data is read from a sensor of the mobile terminal, and if the amplitude change of the read sensor data is greater than a magnitude threshold, it is determined that the state change of the mobile terminal occurs; for another example, when it is monitored that the designated communication module on the mobile terminal accesses the communication device of the corresponding protocol type, it is determined that the state of the mobile terminal changes, and the like, which is not limited in this embodiment of the present invention.

Further, when the second activity state of the user is determined based on the pending activity state and the first activity state, if the duration of the user activity state being the pending activity state is greater than the time threshold, the pending activity state is determined as the second activity state of the user. And if the duration of the user activity state which is the pending activity state is less than the time threshold, continuously monitoring whether the state of the mobile terminal changes.

For example, if sensor data is read from a sensor of the mobile terminal, if the amplitude change of the read sensor data is greater than the amplitude threshold, and the state of the mobile terminal changes at this time, relevant data for identifying the user activity state on the mobile terminal is obtained, and the user activity state is identified as a pending activity state, specifically, a stationary state, based on the obtained relevant data. If the first determined active state, in particular the walking state, is present before the stationary state. If the duration of the user's activity state being stationary is greater than a time threshold (e.g., 15 seconds), then the second activity state of the user is determined to be stationary. And if the duration of the user activity state in the static state is less than 15 seconds, continuously monitoring whether the state of the mobile terminal is changed. If it is monitored that a specified communication module on the mobile terminal is accessed to the communication equipment with the corresponding protocol type, and the state of the mobile terminal changes at the moment, the user activity state is identified to be a pending activity state, specifically a driving state, based on the relevant information of the accessed communication equipment. If the duration of the user activity state being the drive state is greater than a time threshold (e.g., 30 seconds), then the second activity state of the user is determined to be the drive state. And if the duration of the user activity state of the driving state is less than 30 seconds, continuously monitoring whether the state of the mobile terminal is changed, and the like.

In addition, the embodiment of the invention can also acquire the time information corresponding to the identified activity state information of the user through the mobile terminal. Further, step S104 is executed to screen out at least one activity place from the plurality of activity places, where the activity type belongs to the specified activity type and the corresponding time period meets the preset condition, according to the historical activity state information set. Accordingly, in a preferred embodiment, a time interval in which the user activity state is in a static state may be determined based on the category of the user activity state collected by the historical activity state information set and the corresponding time information, and then when the time interval is reached, the geographical location information of the user may be obtained. For example, a sensor on a mobile terminal carried by the user a records activity state data of the user a within one day, and based on the data, it is found that the activity state of the user a from ten o 'clock at night to seven o' clock at the next morning is a static state, and the geographical position of the user a is acquired as the "sigma building" in the time interval, so that the user a can know that the user a possibly lives in the "sigma building".

Further, step S106 is executed to determine the screened at least one activity place as a frequent stopping place of the user. In an alternative embodiment, the location of the obtained geographic location information of the time interval when the user is in the stationary state may be used as the frequent stopping point of the user, for example, in the above example, the "sigma building" may be used as the frequent stopping point of the user a.

In addition, after the screened at least one activity place is determined as the frequent stay place of the user, the frequent stay place can be marked in a map of the mobile terminal in a coordinate and/or text label mode. Furthermore, when an instruction for starting the map is received, the mobile terminal carried by the user can open the map interface, and load and display the corresponding mark of the user on the map for the frequent stopping place.

Specifically, when the frequent stay place is marked in the map of the mobile terminal in a coordinate and/or text label manner, the longitude/latitude corresponding to the frequent stay place may be determined first, and then the determined longitude/latitude is used as a coordinate value to generate the coordinate corresponding to the frequent stay place of the user, and further, the established coordinate is used to mark the frequent stay place, as shown in fig. 4.

In addition, when the frequent stopping place is marked in the map on the mobile terminal in a coordinate and/or text label mode, the text label corresponding to the frequent stopping place can be obtained. In the embodiment of the invention, the text label can comprise at least one of a building name, an exclusive name generated for a frequent stopping place and a user-defined name. Further, the frequently-staying places are marked by the set text labels, as shown in fig. 5, the user a marks the 'sigma buildings' where the user a lives in at the corresponding positions of the map as 'warm and comfortable houses', so that the user a can conveniently see the frequently-staying places related to the user a when opening the map without searching again, the time of the user is saved, and the user experience is improved.

More, when the text label includes the proper name, the preset related activity figure information aiming at the frequent stopping place can be obtained when the text label corresponding to the frequent stopping place is obtained, and then the proper name containing the related activity figure information is generated for the frequent stopping place according to the preset rule.

In addition, after determining the stopover points of the user, the embodiments of the present invention may also recommend the stopover points as search suggested words in the search box, as shown in fig. 6A and 6B. In fig. 6A, the position of the "input place" is a search box, the "search finding" part is a recommended frequently-stopped place, when the cursor is placed at the "input place", the user clicks "beijing post and telecommunications university", the "beijing post and telecommunications university" is copied into the search box, and then the "search" button is clicked to perform related search.

In fig. 6B, the positions of "my position" and "input end point" are both input boxes, the "search found" part is a recommended stopover point, and when the cursor is placed at the "input end point", the user clicks "moonlet", and copies "moonlet" into the input box, and initiates a route search.

In an alternative embodiment of the present invention, recommendation information related to these stopover points, such as merchant information, real-time news, etc., may also be recommended to the user periodically or aperiodically. In addition, if the current activity state of the user is a driving state, when recommending information related to the frequent stopping point for the user, the route and road condition information between the current position and the frequent stopping point can be pushed for the user, the navigation display route can be further automatically called and displayed on the interface of the mobile terminal, and more particularly, the voice interaction driving mode on the mobile terminal can be called, and the like.

The activity state information processing method based on the mobile terminal provided by the embodiment of the invention can determine the frequent stay place of the user, and can further provide corresponding map contents for different user activity states in the process of determining the frequent stay place and acquiring the specific activity states of the user, such as static state, walking, riding, running, mountain climbing, driving, public transport, subway sitting, train sitting and the like. For example, when the user activity state is a walking state, the map content containing information of shopping, dining, toilets and the like can be highlighted; when the user activity state is a bus sitting state, the map content containing information such as bus routes and stops can be highlighted; when the user activity state is a driving state, map content containing information of gas stations, maintenance stations, parking lots, and the like may be highlighted, and the like.

In addition, the method provided by the embodiment of the invention can also determine different user behaviors by combining historical data according to different user activity states when the identified user activity states are static, walking, riding, running, mountain climbing, public transportation, subway sitting, train sitting and the like, and further provide corresponding recommended contents for the user. For example, when the user activity state is a walking state, the specific behavior of the user may be determined in combination with historical data of the user's mobile terminal and corresponding time information at the time of the activity state. For example, when it is determined that the user is currently going to a breakfast shop, it may be determined that the user is ready to eat breakfast at the moment, and then a popular package of the breakfast shop may be recommended for the user to select.

With the above-provided method for processing the activity state based on the mobile terminal, there are many different application scenarios besides the above-mentioned examples. Several specific embodiments are now provided to describe in detail the user behavior determination method for a mobile terminal according to the embodiments of the present invention.

Example one

In this embodiment, the current speed of the user B obtained from the mobile terminal carried by the user B is 6km/h, and the acceleration sensor data trace diagram 7 is obtained as an example. As shown in FIG. 7, this example can acquire accelerations (m/s) in three axial directions (x, y, z) acting on the mobile terminal²). As shown in fig. 7, the acceleration trajectories in the three axis directions (x, y, z) are in a relatively gentle state. Further, the current is extractedAnd acquiring a feature vector in the acceleration sensor data, and inputting a classification model from the feature vector to the category of the user activity state. And identifying the category of the user activity state corresponding to the current sensor data by using the classification model. In this example, the active state of the current user B can be identified as walking by combining the above information of the mobile terminal of the user B.

When the activity state of the user B is obtained as walking, the current time is collected to be eight morning hours through the mobile terminal, the current position of the user B is near a Greema apartment, and the user B is moving towards a breakfast shop of California beef noodles. Moreover, through the historical track on the mobile terminal of the user, the coordinates corresponding to eight morning hours of the user are the breakfast shop of 'California beef noodles'.

According to the method provided by the invention, the 'california beef noodles' can be determined as the frequent stay place of the user B, and furthermore, special marks can be made for the 'california beef noodles' in a mobile terminal map of the user B, for example, the longitude/latitude of the 'california beef noodles' is marked on the map, and a character tag can be set by the user B in a self-defining way for marking, for example, marking 'my breakfast shop' on the corresponding coordinate of the 'california beef noodles', so that the user B can set the frequent stay place in a self-defining way.

In addition, after the method provided by the invention determines that the frequent location of the user B is 'California beef noodles', recommendation information related to 'California beef noodles' can be recommended to the user B regularly or irregularly, for example, preferential activities and new packages of the shop are recommended to the user B.

Example two

The present embodiment is based on an activity state data set of the user C at different time periods, which is recorded by a mobile terminal carried by the user C, and the data set is a historical activity state information set of the user C. In this example, it can be known from the historical activity state information set of the user C that the user C is in a stationary state from ten pm to seven pm every day, and the geographic location of the user C acquired in the time interval is "delphin cell". Further, the user C may live in the delrin cell by further judging in combination with other historical data of the user C mobile terminal, and further, the delrin cell is determined as the frequent stay place of the user C, and is marked on the map of the user C mobile terminal, and may be marked as "home" on the corresponding coordinate of the delrin cell.

According to the embodiment of the invention, after the 'Seiln district' is marked as the frequent stopping place on the map of the mobile terminal, if the user C drives home at the moment, the starting place and the destination do not need to be reset in the driving process, and the map is only opened to directly select the driving route between the current position and the frequent stopping place planned by the map application, so that convenience is provided for the user, and the driving safety of the user is further ensured.

The results provided in the above embodiments are merely examples, and the method for processing an activity state based on a mobile terminal according to the embodiments of the present invention may also be applied to other different scenarios, where the method provided in the embodiments of the present invention is not specifically limited to the above embodiments, for example, by using the method provided in the embodiments of the present invention, corresponding behaviors of a user may also be determined according to an identified location where the user frequently stays, in combination with a history track generated by the mobile terminal and other information. Further, the user may be labeled, such as office workers, food, sports, etc., as shown in fig. 8, and be given corresponding information recommendations.

Therefore, by adopting the mobile terminal-based activity state information processing method provided by the embodiment of the invention, the activity state information of the user can be acquired through the mobile terminal carried by the user at any time and any place, the real-time performance, the accuracy and the comprehensiveness are stronger, and the method is greatly improved compared with the prior art that the step number statistical information of the user can be acquired only through the mobile terminal. Meanwhile, the method provided by the embodiment of the invention can also be used for counting the activity rule of the user in a short time and further identifying the frequent staying place of the user based on various user data acquired by the mobile terminal. Furthermore, the embodiment of the invention can also set real-time reminding service and route planning and the like based on the frequently staying places for the user according to the identified frequently staying places of the user in combination with big data analysis, better realizes the interaction between the user and the mobile terminal, and provides great convenience for the user.

Based on the same inventive concept, the embodiment of the present invention further provides an active state information processing apparatus based on a mobile terminal, and fig. 9 is a first schematic block diagram of an active state information processing apparatus based on a mobile terminal according to an embodiment of the present invention. As shown in fig. 9, the mobile terminal-based activity state information processing apparatus includes at least:

an obtaining module 910, adapted to obtain, through one or more sensors in the mobile terminal, a historical activity state information set for recording activity states of a user over different time periods;

a screening module 920, coupled to the obtaining module 910, adapted to screen at least one activity place from the plurality of activity places, where an activity type belongs to a specified activity type and a corresponding time period meets a preset condition, according to the historical activity state information set;

a determining module 930, coupled to the screening module 920, adapted to determine the screened at least one activity location as a frequent location of the user.

In a preferred embodiment, as shown in fig. 10, the mobile terminal-based activity state information processing apparatus further includes:

a marking module 940, coupled to the determining module 930, adapted to mark the frequent stopping place in the map on the mobile terminal in a coordinate and/or text label manner after determining the screened at least one activity place as the frequent stopping place of the user;

and a loading and displaying module 950, coupled to the marking module 940, adapted to open the map interface and load and display a mark corresponding to the frequent stopping location on the map interface when receiving an instruction to start the map.

In a preferred embodiment, the marking module 940 is further adapted to:

determining the longitude/latitude corresponding to the constant stay place;

generating coordinates corresponding to the frequent staying places by taking the longitude/latitude as coordinate values;

and marking the frequent stopping place by using the established coordinates.

In a preferred embodiment, the marking module 940 is further adapted to:

acquiring a text label corresponding to the frequent staying place, wherein the text label comprises at least one of a building name, an exclusive name generated for the frequent staying place and a user-defined name;

and marking the frequent stopping place by using a text label.

In a preferred embodiment, the marking module 940 is further adapted to:

acquiring preset related activity figure information aiming at frequent stay places;

In a preferred embodiment, as shown in fig. 11, the apparatus for processing information based on activity status of a mobile terminal further includes a generating module 960, coupled to the obtaining module 910, and adapted to:

acquiring a pre-trained activity classification model, wherein the activity classification model is trained by taking a plurality of sensor data recorded by a mobile terminal and activity types respectively corresponding to the data as samples, and the sensor data comprises data recorded by at least one sensor;

reading current sensor data of the mobile terminal, and determining a first activity type corresponding to the read sensor data according to an activity classification model;

acquiring communication environment information of a mobile terminal;

and generating a historical activity state information set according to at least one of the second activity type, the current activity place and the current time period.

In a preferred embodiment, the generation module 960 is further adapted to:

if so, determining that the first activity type is the second activity type;

In addition, an embodiment of the present invention further provides a mobile terminal, as shown in fig. 12, including a processor 1280 and a memory 1220:

the memory 1220 is used for storing a program for executing the mobile terminal-based activity state information processing method provided in any one or a combination of the above-mentioned preferred embodiments,

the processor 1280 is configured to execute programs stored in the memory 1220.

For convenience of explanation, fig. 12 only shows a part related to the embodiment of the present invention, and details of the technology are not disclosed, please refer to the method part of the embodiment of the present invention. The mobile terminal may include any terminal device such as a mobile phone, a tablet computer, a PDA (personal digital Assistant), a POS (Point of Sales), and a vehicle-mounted computer, taking the terminal as the mobile phone as an example:

fig. 12 is a block diagram illustrating a partial structure of a mobile phone related to a mobile terminal according to an embodiment of the present invention. Referring to fig. 12, the cellular phone includes: radio Frequency (RF) circuit 1210, memory 1220, input unit 1230, display unit 1240, sensor 1250, audio circuit 1260, wireless fidelity (WiFi) module 1270, processor 1280, and power supply 1290. Those skilled in the art will appreciate that the handset configuration shown in fig. 12 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.

The following describes each component of the mobile phone in detail with reference to fig. 12:

the RF circuit 1210 may be used for receiving and transmitting signals during a message or call, and in particular, may receive downlink information of a base station and process the received downlink information to the processor 1280, and may transmit data designed for uplink to the base station. generally, the RF circuit 1210 may include, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (L w noise amplifier, &lttttransmission &l "&tttl/t &tttgtna), a duplexer, etc. furthermore, the RF circuit 1210 may communicate with a network and other devices through wireless communication.

The memory 1220 may be used to store software programs and modules, and the processor 1280 executes various functional applications and data processing of the mobile phone by operating the software programs and modules stored in the memory 1220. The memory 1220 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 1220 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The input unit 1230 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the cellular phone. Specifically, the input unit 1230 may include a touch panel 1231 and other input devices 1232. The touch panel 1231, also referred to as a touch screen, can collect touch operations of a user (e.g., operations of the user on or near the touch panel 1231 using any suitable object or accessory such as a finger, a stylus, etc.) thereon or nearby, and drive the corresponding connection device according to a preset program. Alternatively, the touch panel 1231 may include two portions, a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, and sends the touch point coordinates to the processor 1280, and can receive and execute commands sent by the processor 1280. In addition, the touch panel 1231 may be implemented by various types such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. The input unit 1230 may include other input devices 1232 in addition to the touch panel 1231. In particular, other input devices 1232 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

The Display unit 1240 may be used to Display information input by or provided to a user and various menus of the cellular phone, the Display unit 1240 may include a Display panel 1241, and optionally the Display panel 1241 may be configured in the form of a liquid crystal Display (L iquid crystal Display, L CD), an Organic light Emitting Diode (Organic L light-Emitting Diode, O L ED), and the like, further, the touch panel 1231 may cover the Display panel 1241, and when a touch operation is detected on or near the touch panel 1231, the touch panel 1231 may be transmitted to the processor 1280 to determine the type of the touch event, and then the processor 1280 may provide a corresponding visual output on the Display panel 1241 according to the type of the touch event, although in fig. 12, the touch panel 1231 and the Display panel 1241 are provided as two separate components to implement the input and input functions of the cellular phone, in some embodiments, the touch panel 1231 may be integrated with the Display panel 1241 to implement the input and output functions of the cellular phone.

The cell phone may also include at least one sensor 1250, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1241 according to the brightness of ambient light, and the proximity sensor may turn off the display panel 1241 and/or the backlight when the mobile phone moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when stationary, and can be used for applications of recognizing the posture of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which can be configured on the mobile phone, further description is omitted here.

Audio circuitry 1260, speaker 1261, and microphone 1262 can provide an audio interface between a user and a cell phone. The audio circuit 1260 can transmit the received electrical signal converted from the audio data to the speaker 1261, and the audio signal is converted into a sound signal by the speaker 1261 and output; on the other hand, the microphone 1262 converts the collected sound signals into electrical signals, which are received by the audio circuit 1260 and converted into audio data, which are processed by the audio data output processor 1280, and then passed through the RF circuit 1210 to be transmitted to, for example, another cellular phone, or output to the memory 1220 for further processing.

WiFi belongs to short-distance wireless transmission technology, and the mobile phone can help a user to receive and send e-mails, browse webpages, access streaming media and the like through the WiFi module 1270, and provides wireless broadband internet access for the user. Although fig. 12 shows the WiFi module 1270, it is understood that it does not belong to the essential constitution of the handset, and may be omitted entirely as needed within the scope not changing the essence of the invention.

The processor 1280 is a control center of the mobile phone, connects various parts of the entire mobile phone by using various interfaces and lines, and performs various functions of the mobile phone and processes data by operating or executing software programs and/or modules stored in the memory 1220 and calling data stored in the memory 1220, thereby performing overall monitoring of the mobile phone. Optionally, processor 1280 may include one or more processing units; preferably, the processor 1280 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It is to be appreciated that the modem processor described above may not be integrated into the processor 1280.

The handset also includes a power supply 1290 (e.g., a battery) for powering the various components, and preferably, the power supply may be logically connected to the processor 1280 via a power management system, so that the power management system may manage the charging, discharging, and power consumption.

Although not shown, the mobile phone may further include a camera, a bluetooth module, etc., which are not described herein.

The mobile terminal-based activity state information processing method and the mobile terminal-based activity state information processing device provided by the embodiment of the invention have the following beneficial effects that:

In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components of a mobile terminal-based active state information processing apparatus according to embodiments of the present invention. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.

For example, fig. 13 illustrates a block diagram of a computing device that may implement a mobile terminal-based activity state information processing method according to the present invention. The computing device conventionally includes a processor 1310 and a computer program product or computer-readable medium in the form of a memory 1320. The memory 1320 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. The memory 1320 has a storage space 1330 that stores program code 1331 for performing any of the method steps described above. For example, the storage space 1330 storing program codes may store the respective program codes 1331 for respectively implementing various steps in the above methods. The program code can be read from or written to one or more computer program products. These computer program products comprise a program code carrier such as a hard disk, a Compact Disc (CD), a memory card or a floppy disk. Such a computer program product is typically a portable or fixed storage unit as shown with reference to fig. 14. The storage unit may have storage segments, storage spaces, etc. arranged similarly to the memory 1320 in the computing device of fig. 13. The program code may be compressed, for example, in a suitable form. Typically, the memory unit comprises a memory unit storing computer readable program code 1331' for performing the steps of the method according to the invention, i.e. program code readable by a processor, such as 1310, for example, which when executed by a computing device, causes the computing device to perform the steps of the method described above.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

Thus, it should be appreciated by those skilled in the art that while a number of exemplary embodiments of the invention have been illustrated and described in detail herein, many other variations or modifications consistent with the principles of the invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Accordingly, the scope of the invention should be understood and interpreted to cover all such other variations or modifications.

Claims

1. A method for processing activity state information based on a mobile terminal comprises the following steps:

obtaining sensor data by one or more sensors in a mobile terminal;

acquiring network data, position information and/or application data through a corresponding application program in the mobile terminal;

determining a first activity type corresponding to the sensor data by using a pre-trained activity classification model, and adjusting the first activity type according to the network data, the position information and/or the application data;

determining an activity place for obtaining the sensor data according to the position information, and further generating a historical activity state information set according to a historical activity state and a corresponding historical activity place;

determining the screened at least one activity place as a frequent stopping place of the user;

when an instruction for starting the map is received, opening the map interface, and loading and displaying a mark corresponding to the frequent stopping place on the map interface;

and recommending the frequent stopping place as a search suggestion word of a search box of the map interface.

2. The method of claim 1, wherein marking the frequent stops in a map on the mobile terminal using coordinates and/or text labels comprises:

marking the frequent stopping place by using the generated coordinates.

3. The method of claim 1, wherein marking the frequent stops in a map on the mobile terminal using coordinates and/or text labels comprises:

and marking the frequent stopping place by using the text label.

4. The method of claim 3, wherein when the text label includes the proper name, obtaining the text label corresponding to the frequent stopping location comprises:

5. An activity state information processing apparatus based on a mobile terminal, comprising:

the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is suitable for acquiring sensor data through one or more sensors in the mobile terminal and acquiring network data, position information and/or application data through corresponding application programs in the mobile terminal;

a generation module adapted to determine a first activity type corresponding to the sensor data using a pre-trained activity classification model and adjust the first activity type according to the network data, location information, and/or application data; determining an activity place for obtaining the sensor data according to the position information, and further generating a historical activity state information set according to a historical activity state and a corresponding historical activity place;

the determination module is suitable for determining the screened at least one activity place as a frequent stopping place of the user;

the loading and displaying module is suitable for opening the map interface when receiving an instruction for starting the map, and loading and displaying a mark corresponding to the frequent stopping place on the map interface; and recommending the frequent stopping place as a search suggestion word of a search box of the map interface.

6. The apparatus of claim 5, wherein the tagging module is further adapted to:

marking the frequent stopping place by using the generated coordinates.

7. The apparatus of claim 5, wherein the tagging module is further adapted to:

and marking the frequent stopping place by using the text label.

8. The apparatus of claim 7, wherein the tagging module is further adapted to:

9. A computer readable medium having stored therein computer readable code which, when run on a computing device, causes the computing device to execute a mobile terminal based activity state information processing method according to any one of claims 1-4.

10. A mobile terminal comprising a processor and a memory:

the memory for storing a program for performing the method of any one of claims 1-4,

the processor is configured to execute programs stored in the memory.