WO2021170014A1 - Procédé et appareil d'identification de comportement d'utilisateur, dispositif informatique et support - Google Patents

Procédé et appareil d'identification de comportement d'utilisateur, dispositif informatique et support Download PDF

Info

Publication number
WO2021170014A1
WO2021170014A1 PCT/CN2021/077744 CN2021077744W WO2021170014A1 WO 2021170014 A1 WO2021170014 A1 WO 2021170014A1 CN 2021077744 W CN2021077744 W CN 2021077744W WO 2021170014 A1 WO2021170014 A1 WO 2021170014A1
Authority
WO
WIPO (PCT)
Prior art keywords
matrix
data
user behavior
state data
multiple sets
Prior art date
Application number
PCT/CN2021/077744
Other languages
English (en)
Chinese (zh)
Inventor
张夏天
Original Assignee
北京腾云天下科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京腾云天下科技有限公司 filed Critical 北京腾云天下科技有限公司
Publication of WO2021170014A1 publication Critical patent/WO2021170014A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3438—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment monitoring of user actions
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
    • G01D21/00—Measuring or testing not otherwise provided for
    • G01D21/02—Measuring two or more variables by means not covered by a single other subclass
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Definitions

  • the present disclosure relates to the field of mobile data processing technology, and in particular to a user behavior recognition method, device, computer equipment, non-transitory computer-readable storage medium, and computer program product.
  • Mobile devices such as mobile phones, tablet computers, smart wearable devices, etc.
  • Users can watch audio and video, shop, play games, record exercise status, etc. through their portable mobile devices.
  • a user behavior recognition method including: acquiring multiple sets of motion state data, each set of motion state data includes multiple pieces of state information, and the state information is used to indicate the motion state of a mobile device carried by a user; Perform decorrelation processing on the multiple sets of exercise state data to eliminate the correlation of various state information in each set of exercise state data to obtain target data; and determine the user behavior type according to the target data.
  • a user behavior recognition device including: a data acquisition module configured to acquire multiple sets of exercise state data, each set of exercise state data includes multiple pieces of state information, and the state information is used to indicate a user The movement state of the carried mobile device; a decorrelation module configured to decorrelate the above multiple sets of movement state data to eliminate the correlation of various state information in each group of movement state data to obtain target data; and The classification module is configured to determine the user behavior type according to the target data.
  • a computer device including: a memory, a processor, and a computer program stored on the memory.
  • the processor is configured to execute the above-mentioned computer program to implement the steps of the above-mentioned user behavior identification method.
  • a non-transitory computer-readable storage medium on which a computer program is stored.
  • the computer program is executed by the processor, the steps of the above-mentioned user behavior identification method are realized.
  • a computer program product including a computer program, which, when executed by a processor, implements the above-mentioned user behavior recognition method.
  • the current behavior type of the user can be determined.
  • the processing of multiple sets of motion state data includes decorrelation processing. By decorrelating multiple sets of motion state data, the correlation of each state information in each set of motion state data is eliminated, and the redundant information in the multiple sets of acquired motion state data is greatly reduced.
  • the user behavior type is determined according to the target data obtained by the decorrelation processing, which improves the efficiency and accuracy of user behavior recognition.
  • FIG. 1 is a schematic diagram illustrating an example system in which various methods described herein may be implemented according to an example embodiment
  • Fig. 2 is a flowchart illustrating a user behavior recognition method according to an exemplary embodiment
  • Fig. 3 is a schematic diagram illustrating a fuselage coordinate system according to an exemplary embodiment
  • FIG. 4 is a schematic diagram illustrating the correlation of various pieces of state information in exercise state data according to an exemplary embodiment
  • FIG. 5 is a schematic diagram illustrating the correlation of various state information after decorrelation processing is performed on the motion state data involved in FIG. 4;
  • Fig. 6 is a schematic diagram illustrating a user behavior recognition apparatus according to an exemplary embodiment
  • Fig. 7 is a structural block diagram illustrating a user behavior recognition model according to an exemplary embodiment
  • Fig. 8 is a structural block diagram illustrating another user behavior recognition model according to an exemplary embodiment
  • Fig. 9 is a schematic diagram illustrating an exemplary computer device that can be applied to the exemplary embodiment.
  • first, second, etc. to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of these elements. Such terms are only used for Distinguish one element from another.
  • first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.
  • FIG. 1 is a schematic diagram illustrating an example system 100 in which various methods described herein may be implemented according to an example embodiment.
  • the system 100 includes a mobile device 110, a server 120, and a network 130 that communicatively couples the mobile device 110 and the server 120.
  • the mobile device 110 includes a display 114 and a client application (APP) 112 that can be displayed via the display 114.
  • the client application 112 may be an application that needs to be downloaded and installed before running, or a small program (liteapp) that is a lightweight application.
  • liteapp small program
  • the client application 112 may be pre-installed on the mobile device 110 and activated.
  • the user 102 can search for the client application 112 in the host application (for example, by the name of the client application 112, etc.) or scan the graphic code of the client application 112 (for example, a barcode).
  • the mobile device 110 may be any type of mobile device, including a mobile computer, a mobile phone, a wearable computer device (such as a smart watch, a head-mounted device, including smart glasses, etc.) or other types of mobile devices.
  • the server 120 is typically a server deployed by an Internet Service Provider (ISP) or Internet Content Provider (ICP).
  • the server 120 may represent a single server, a cluster of multiple servers, a distributed system, or a cloud server that provides basic cloud services (such as cloud databases, cloud computing, cloud storage, and cloud communications). It will be understood that although the server 120 is shown in FIG. 1 to communicate with only one mobile device 110, the server 120 may provide background services for multiple mobile devices at the same time.
  • Examples of the network 130 include a combination of a local area network (LAN), a wide area network (WAN), a personal area network (PAN), and/or a communication network such as the Internet.
  • the network 130 may be a wired or wireless network.
  • technologies and/or formats including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc. are used to process data exchanged through the network 130.
  • encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. can also be used to encrypt all or some links.
  • customized and/or dedicated data communication technologies can also be used to replace or supplement the aforementioned data communication technologies.
  • a sensor for collecting motion state data of the mobile device 110 and a client application 112 are deployed in the mobile device 110.
  • the client application 112 may be a user behavior recognition application, and the application may determine the behavior type of the user carrying the mobile device 110 according to the motion state data of the mobile device 110 collected by the sensor.
  • the server 120 may be a server used with the user behavior recognition application.
  • a user behavior recognition model is deployed in the server 120 (the details of the user behavior recognition model will be detailed in the following embodiments).
  • the server 120 can provide a user behavior recognition service to the client application 112 running in the mobile device 110 based on the user behavior recognition model.
  • the server 120 receives the exercise status data sent by the mobile device 110 through the client application 112, and then transfers the exercise status data to the client application 112. Enter the user behavior recognition model to determine the type of user behavior.
  • the server 120 may also provide the user behavior recognition model to the mobile device 110, and the client application 112 running in the mobile device 110 may implement the user behavior recognition function by calling the user behavior recognition model.
  • the user behavior recognition model may be a pre-trained neural network model, and the details of the model will be described in detail in the following embodiments.
  • the server 120 may, for example, use various training or learning techniques such as back propagation of errors to train the user behavior recognition model.
  • performing backpropagation of errors may include performing truncated backpropagation through time.
  • the server 120 may employ a variety of generalization techniques (for example, weight attenuation, loss, etc.) to improve the generalization ability of the model being trained.
  • the server 120 may train the user behavior recognition model based on the set of training samples.
  • the training sample may be, for example, exercise state data marked with the type of user behavior.
  • the model can be converted into a TFLite format model file (TensorFlow Lite file, in which the model parameters are quantized into an 8-bit integer type, that is, int8) in the server 120.
  • the file is deployed in the mobile device 110, so that the client application 112 running in the mobile device 110 can implement the user behavior recognition function by calling the user behavior recognition model.
  • FIG. 2 is a flowchart illustrating a user behavior recognition method 200 according to an exemplary embodiment.
  • the method 200 may be executed at a mobile device (for example, the mobile device 110 shown in FIG. 1 ), that is, the execution subject of each step of the method 200 may be the mobile device 110 shown in FIG. 1.
  • the method 200 may be executed at a server (for example, the server 120 shown in FIG. 1).
  • the method 200 may be executed by a combination of a mobile device (e.g., mobile device 110) and a server (e.g., server 120).
  • the method 200 is described by taking the mobile device 110 as the execution subject as an example.
  • the method 200 includes steps 210 to 230.
  • step 210 multiple sets of exercise state data are obtained, and each set of exercise state data includes multiple pieces of state information, and the state information is used to indicate the exercise state of the mobile device carried by the user.
  • step 220 decorrelation processing is performed on the multiple sets of motion state data to eliminate the correlation of various state information in each set of motion state data to obtain target data.
  • step 230 the user behavior type is determined according to the target data.
  • the current behavior type of the user can be determined.
  • the processing of multiple sets of motion state data includes decorrelation processing. By decorrelating multiple sets of motion state data, the correlation of each state information in each set of motion state data is eliminated, and the redundant information in the multiple sets of acquired motion state data is greatly reduced.
  • the user behavior type is determined according to the target data obtained by the decorrelation processing, which improves the efficiency and accuracy of user behavior recognition.
  • step 210 multiple sets of exercise state data are obtained, and each set of exercise state data includes multiple pieces of state information, and the state information is used to indicate the exercise state of the mobile device carried by the user.
  • the multiple sets of motion state data described in step 210 are collected by sensors deployed in the mobile device.
  • Sensors include, but are not limited to, acceleration sensors, gyroscopes (ie, angular velocity sensors), magnetic field sensors, and the like.
  • the sensors deployed in the mobile device include an acceleration sensor and a gyroscope. Accordingly, the motion state data of the mobile device collected by the sensor includes: acceleration information of the mobile device in multiple directions; Angular velocity information in each direction.
  • the acceleration sensor can collect the acceleration information of the mobile device on the three coordinate axes of x, y and z; the gyroscope can collect the angular velocity information of the mobile device on the three coordinate axes of x, y, and z , Y, z three coordinate axis rotation angular velocity information).
  • Fig. 3 shows a schematic diagram of a fuselage coordinate system according to an exemplary embodiment. As shown in Figure 3, the origin of the body coordinate system is the center point of the screen of the mobile device, the x-axis direction is horizontal to the right in the screen plane; the y-axis direction is vertical upwards in the screen plane; the z-axis direction is perpendicular to the screen plane And point to the top of the screen.
  • step 210 includes: acquiring multiple sets of motion state data collected by sensors deployed in the mobile device within a predetermined period of time at a predetermined frequency.
  • the values of the predetermined frequency and the predetermined duration can be set by those skilled in the art according to actual conditions.
  • the predetermined frequency is set to a smaller value, thereby reducing the demand for sensor data subscription and reducing power consumption; at the same time, the predetermined duration is also set to a smaller appropriate value, thereby ensuring the recognition of user behavior
  • the accuracy is reduced while the delay is increased, the sensitivity of the response is increased, and the real-time performance of user behavior recognition is improved, that is, the current behavior type of the user is recognized in real time.
  • the predetermined frequency may be set to 20 Hz (that is, the interval time for obtaining motion state data from the sensor is 50 ms), and the predetermined duration is set to 6400 ms, then step 210 is equivalent to obtaining 128 sets of motion at a time interval of 50 ms. Status data.
  • Each group of motion state data includes the acceleration information of the mobile device in the directions of the three coordinate axes x, y, and z collected by the acceleration sensor and the angular velocity of the mobile device rotating around the three coordinate axes of x, y, and z collected by the gyroscope. Take the information as an example.
  • the predetermined frequency is set to 20Hz and the predetermined duration is set to 6400ms
  • the obtained 128 sets of motion state data d 1 to d 128 are shown in Table 1 below:
  • each row corresponds to a set of motion state data
  • each set of motion state data includes x-axis acceleration information, y-axis acceleration information, z-axis acceleration information, x-axis angular velocity information, y-axis angular velocity information, and z-axis angular velocity Information six status information.
  • the motion state data of the mobile device acquired in step 210 has a strong correlation with the behavior of the user currently carrying the mobile device, and the user behavior type can be identified accordingly.
  • User behavior types include but are not limited to being stationary, walking, running, cycling, taking a bus, taking a subway, taking a small car (such as a taxi, private car, etc.), etc.
  • step 220 is executed.
  • step 220 decorrelation processing is performed on the multiple sets of motion state data to eliminate the correlation of various state information in each set of motion state data to obtain target data.
  • Figure 4 shows the correlation of various state information in multiple sets of motion state data in the form of a matrix, where numbers 0 to 5 respectively represent x-axis acceleration information, y-axis acceleration information, z-axis acceleration information, and x-axis angular velocity.
  • Information, y-axis angular velocity information, z-axis angular velocity information, the element in the i-th row and j-th column of the matrix represents the correlation (covariance) between the information corresponding to the number i and the information corresponding to the number j.
  • the absolute value of the correlation is closer to 1 , Indicates that the stronger the correlation between the two state information, the closer the absolute value of the correlation is to 0, the weaker the correlation between the two state information.
  • the correlation is a positive number, which means that the two state information is positively correlated; the correlation is a negative number, which means that the two state information is negatively correlated.
  • the right side of Figure 4 shows an example of the corresponding relationship between correlation and gray value to show the correlation of various state information more intuitively.
  • the more correlated parts that is, the light-colored parts
  • the acceleration information of the x, y, and z axes collected by the acceleration sensor There is a large correlation between the three, and there is a large correlation between the angular velocity information of the x, y, and z axes collected by the gyroscope.
  • step 220 decorrelation processing is performed on multiple sets of motion state data to improve the efficiency and accuracy of user behavior recognition.
  • step 220 further includes the following steps 222 to 224:
  • step 222 centralization processing is performed on the multiple sets of motion state data, so that the average value of each state information after processing is 0, and centralization data is obtained.
  • multiple sets of motion state data may form an original data matrix, each row in the original data matrix represents a set of motion state data, and each column identifies one item of state information.
  • the 128 groups of motion state data in Table 1 can form a 128*6 original data matrix D, namely:
  • Each row of the original data matrix D represents a group of motion state data, and each column corresponds to a piece of state information.
  • the state information corresponding to each column is x-axis acceleration information, y-axis acceleration information, z-axis acceleration information, and x-axis angular velocity information. , Y-axis angular velocity information, z-axis angular velocity information.
  • step 224 a transformation matrix of the centralization data is determined, and the centralization data is multiplied by the transformation matrix to obtain target data, wherein the transformation matrix makes the covariance matrix of the target data Identity matrix, thereby eliminating the relevance of various status information.
  • the transformation matrix of the centralized data is determined using the following steps 2242 to 2246:
  • step 2242 the covariance matrix ⁇ of the centralized data is calculated.
  • the covariance matrix ⁇ is subjected to singular value decomposition, and the covariance matrix ⁇ is transformed into the product of the first matrix U, the second matrix S, and the transposed matrix U T of the first matrix U.
  • One matrix U is a matrix composed of eigenvectors of the covariance matrix ⁇
  • the second matrix S is a diagonal matrix composed of eigenvalues of the covariance matrix ⁇ . That is, the singular value decomposition result of the covariance matrix ⁇ is:
  • step 2246 the product of the matrices U, S -1/2 and U T is used as the transformation matrix, where the value of each element in S -1/2 is the square root of the element at the corresponding position in the second matrix S reciprocal. That is, the transformation matrix W is:
  • the centralized data C is multiplied by the transformation matrix W to obtain the target data Z, namely:
  • the covariance matrix of the target data Z is an identity matrix I with diagonal elements of 1, and other elements of 0. That is to say, in the target data Z, the covariance between different state information is 0, thereby eliminating the correlation between various state information.
  • FIG. 5 shows the correlation (covariance) of various state information after decorrelating the multiple sets of motion state data involved in FIG. 4.
  • the correlation matrix shown in Figure 5 except for the diagonal element value 1, the element values at other positions are all very small, reaching the order of 10 ⁇ (-6), 10 ⁇ (-7), almost 0 , Indicating that the correlation between various status information is eliminated.
  • step 220 the multiple sets of motion state data are decorrelated, and after the target data is obtained, step 230 is executed.
  • step 230 the user behavior type is determined according to the target data.
  • the target data obtained in step 220 has a strong correlation with the type of user behavior.
  • the user behavior type corresponding to the target data can be determined according to the correlation between the target data and the user behavior type.
  • the user behavior types include, for example, being stationary, walking, running, cycling, taking a bus, taking a subway, taking a small car (such as a taxi, a private car, etc.), but not limited thereto.
  • step 220 and step 230 may be implemented by a trained user behavior recognition model. That is, step 220 includes: inputting the multiple sets of motion state data into a preset user behavior recognition model, so that the user behavior recognition model performs decorrelation processing on the multiple sets of motion state data to obtain target data. Step 230 includes: processing the target data by the user behavior recognition model to determine the user behavior type.
  • the user behavior recognition model is trained by using the motion state data marked with the user behavior type as the training sample.
  • the trained user behavior recognition model takes exercise state data as input, and outputs the user behavior type corresponding to the exercise state data.
  • the user behavior recognition model includes a decorrelation module and a classification module.
  • the decorrelation module is configured to perform step 220, that is, decorrelate multiple sets of exercise state data to eliminate items in each group of exercise state data. Relevance of status information to get target data.
  • the classification module is configured to perform step 230, which is to determine the user behavior type according to the target data.
  • FIG. 6 is a schematic diagram illustrating a user behavior recognition apparatus 600 according to an exemplary embodiment. ⁇ 600 ⁇ Device 600. As shown in FIG. 6, the device 600 includes a data acquisition module 610, a decorrelation module 620 and a classification module 630.
  • the data acquisition module 610 is configured to acquire multiple sets of motion state data, and each set of motion state data includes multiple pieces of state information, and the state information is used to indicate the motion state of the mobile device carried by the user.
  • the decorrelation module 620 is configured to decorrelate the multiple sets of motion state data, so as to eliminate the correlation of various state information in each set of motion state data to obtain target data.
  • the classification module 630 is configured to determine the user behavior type according to the target data.
  • the current behavior type of the user can be determined.
  • the processing of multiple sets of motion state data includes decorrelation processing. By decorrelating multiple sets of motion state data, the correlation of each state information in each set of motion state data is eliminated, and the redundant information in the multiple sets of acquired motion state data is greatly reduced.
  • the user behavior type is determined according to the target data obtained by the decorrelation processing, which improves the efficiency and accuracy of user behavior recognition.
  • the data acquisition module 610 is configured to acquire multiple sets of motion state data, and each set of motion state data includes multiple pieces of state information, and the state information is used to indicate the motion state of the mobile device carried by the user.
  • multiple sets of motion state data are collected by sensors deployed in the mobile device.
  • the data acquisition module 610 is further configured to acquire multiple sets of status data collected by the sensors deployed in the mobile terminal within a predetermined period of time according to a predetermined frequency.
  • the motion state data includes: acceleration information of the mobile device in multiple directions; and angular velocity information of the mobile device in multiple directions.
  • the decorrelation module 620 is configured to decorrelate the multiple sets of motion state data, so as to eliminate the correlation of various state information in each set of motion state data to obtain target data.
  • the decorrelation module 620 is further configured to: perform centralized processing on the multiple sets of motion state data so that the average value of each item of state information after processing is 0 to obtain centralized data; determine the centralized data The transformation matrix of the data is multiplied by the centralization data and the transformation matrix to obtain the target data, wherein the transformation matrix makes the covariance matrix of the target data an identity matrix, thereby eliminating various status information Correlation.
  • the decorrelation module 620 is further configured to: calculate the covariance matrix ⁇ of the centralized data; perform singular value decomposition on the covariance matrix ⁇ , and convert the covariance matrix ⁇ into a first matrix U, The product of the second matrix S and the transposed matrix U T of the first matrix U, where the first matrix U is a matrix composed of the eigenvectors of the covariance matrix ⁇ , and the second matrix S is the eigenvalues of the covariance matrix ⁇ .
  • Diagonal matrix composed of the matrix take the product of the matrices U, S -1/2 and U T as the transformation matrix, where the value of each element in S -1/2 is the reciprocal of the square root of the element at the corresponding position in the second matrix S .
  • the classification module 630 is configured to determine the user behavior type according to the target data.
  • the user behavior types include, for example, stationary, walking, running, cycling, taking a bus, taking a subway, taking a small car, and so on.
  • each module of the apparatus 600 shown in FIG. 6 may correspond to each step in the method 200 described with reference to FIG. 2. Therefore, the operations, features, and advantages described above for the method 200 are also applicable to the device 600 and the modules included in it. For the sake of brevity, some operations, features and advantages will not be repeated here.
  • the decorrelation module 620 and the classification module 630 form a user behavior recognition model, and the user behavior recognition model is trained using exercise state data marked with user behavior types as training samples.
  • FIG. 7 shows a structural block diagram of a user behavior recognition model 700 according to an exemplary embodiment.
  • the user behavior recognition model 700 includes a decorrelation module 7100 and a classification module 7200.
  • the decorrelation module 7100 is configured to decorrelate multiple sets of motion state data to eliminate the correlation of various state information in each set of motion state data to obtain target data.
  • the decorrelation module 7100 further includes a batch normalization layer (Batch Normalization), which is configured to adjust the mean value and variance of various status information in the target data Z so that the target data Z is most suitable
  • Batch Normalization a batch normalization layer
  • the state of model training improves the efficiency of model training and subsequent classification effects. Mark the adjusted data as Z’, and the processing performed by the batch standardization layer is as follows:
  • ⁇ and ⁇ are trainable parameters, and their values are determined by training.
  • the decorrelation module 7100 obtains the target data from which the relevance of the state information is eliminated, the target data is input to the classification module 7200 for processing.
  • the classification module 7200 is configured to process the target data to determine the user behavior type and output it.
  • the classification module 7200 includes a plurality of convolution processing units 7220-1 to 7220-N.
  • Each convolution processing unit includes two depthwise separable convolution layers (depthwise separable convolution) 7221 and 7222.
  • the convolution stride of the separable convolutional layer at the first depth is 1, and the convolution stride of the separable convolutional layer at the second depth is greater than 1.
  • the convolution step length of the second depth separable convolution layer may be 2 or other values.
  • the depth separable convolutional layer has greatly reduced the number of parameters, which reduces the complexity of the model, thereby increasing the running speed of the model.
  • the convolution step size setting greater than 1 replaces the pooling layer in the traditional neural network model. Compared with setting the pooling layer, the convolution step size greater than 1 can hardly affect the accuracy of the model and avoid A large amount of information loss caused by pooling reduces the complexity of the model and the risk of overfitting, and improves the running speed of the model.
  • FIG. 8 shows a structural block diagram of another user behavior recognition model 800 according to an exemplary embodiment.
  • the user behavior recognition type 800 includes a decorrelation module 8100 and a classification module 8200.
  • the decorrelation module 8100 is configured to decorrelate the input multiple sets of motion state data to eliminate the correlation of various state information in each set of motion state data to obtain target data.
  • the decorrelation module 8100 For the detailed features and advantages of the decorrelation module 8100, please refer to the related descriptions of the aforementioned step 220, the decorrelation module 620, and the decorrelation module 7100, which will not be repeated here.
  • the input multiple sets of motion state data form a 128*6 original data matrix (a total of 128 sets of motion state data, each group of motion state data includes 6-dimensional state information), after processing by the decorrelation module 8100, 128*6 In the target data, the correlation between various status information is eliminated in the target data.
  • the target data output by the decorrelation module 8100 is input to the classification module 8200.
  • the classification module 8200 is configured to perform a series of processing on the target data, and finally output the user behavior type.
  • the classification module 8200 includes three convolution processing units, and each convolution processing unit includes a first-depth separable convolutional layer and a second-depth separable convolutional layer.
  • the convolution stride of the separable convolutional layer at the first depth is 1, and the convolution stride of the separable convolutional layer at the second depth is 2.
  • the kernel size of all depth separable convolutional layers is 5, that is, 5*1, and the number of output channels (filters) is 16.
  • the depth separable convolutional layers all use the ReLu activation function (the activation layer is not shown in FIG. 8), and the convolution kernel has only weights and no biases.
  • the first-depth separable convolutional layer 1 performs convolution processing on 128*6 target data, and outputs a 128*16 feature map;
  • the second-depth separable convolutional layer 1 performs convolution processing on the 128*16 feature map output by the first-depth separable convolutional layer 1, and outputs a 64*16 feature map;
  • the first-depth separable convolutional layer 2 performs convolution processing on the 64*16 feature map output by the second-depth separable convolutional layer 1, and outputs a 64*16 feature map;
  • the second-depth separable convolutional layer 2 performs convolution processing on the 64*16 feature map output by the first-depth separable convolutional layer 2, and outputs a 32*16 feature map;
  • the first-depth separable convolutional layer 3 performs convolution processing on the 32*16 feature map output by the second-depth separable convolutional layer 2, and outputs a 32*16 feature map;
  • the second-depth separable convolutional layer 3 performs convolution processing on the 32*16 feature map output by the first-depth separable convolutional layer 3, and outputs a 16*16 feature map;
  • the flattening layer flattens the 16*16 feature map output by the second depth separable convolutional layer 3, and outputs a 256-dimensional feature vector;
  • the fully connected softmax layer fully connects the 256-dimensional feature vector output by the flattening layer, determines the probability that the feature vector belongs to each user behavior type, and outputs the user behavior type with the highest probability as the classification result.
  • the convolution step size of the second-depth separable convolution layer is 2, which realizes down-sampling of feature data, which replaces the pooling layer in the traditional neural network model.
  • setting the convolution step size to a value greater than 1 can avoid a large amount of information loss caused by pooling, and reduce model complexity and overtime without affecting the accuracy of the model.
  • the risk of fitting improves the running speed of the model.
  • the depth separable convolutional layer greatly reduces the number of parameters that need to be trained, which reduces the complexity of the model and improves the running speed of the model.
  • the following takes the first depth separable convolution layer 1 in FIG. 8 as an example to illustrate the technical effect of the depth separable convolution layer compared with the traditional convolution layer.
  • the target data Z is a 128*6 data matrix, as shown in Table 2 below:
  • the parameters of the first depth separable convolution layer 1 are: the convolution step stride is 1, the kernel size of the convolution kernel is 5, that is, 5*1, and the number of output channels filters is 16.
  • a convolution kernel with the same size as the number of output channels of 1*the number of input channels is set, and each row of data of the feature map obtained in the previous step is convolved. That is, 16 convolution kernels with a size of 1*6 are set, and each convolution kernel convolves the feature map F separately.
  • a 1*6 convolution kernel [w 1 , w 2 , w 3 , w 4 , w 5 , w 6 ] as an example.
  • the 128 rows of data of the feature map F are convolved to obtain 128 values p 1 to p 128 , and these 128 values form a 128*1 column vector.
  • Each 1*6 convolution kernel performs convolution processing on the feature map F, and a 128*1 column vector can be obtained.
  • 16 1*6 convolution kernels perform convolution processing on the feature map F, and 16 128*1 column vectors will be obtained, that is, a 128*16 feature map will be obtained.
  • each convolution kernel also needs to have 6 channels, that is, the depth of the convolution kernel is 6, which is equivalent to setting 16 5*1*6 The convolution kernel.
  • Each 5*1*6 convolution kernel convolves the 6 input channels of the target data Z to obtain a 128*1 feature vector.
  • 16 5*1*6 convolution kernels will get 16 A 128*1 feature vector, that is, a 128*16 feature map is obtained.
  • the specific module execution action discussed herein includes the specific module itself performing the action, or alternatively the specific module calls or otherwise accesses another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, the specific module that performs an action may include the specific module itself that performs the action and/or another module that is called or accessed by the specific module to perform the action.
  • the decorrelation module 620 and the classification module 630 described above may be combined into a single module in some embodiments.
  • modules described above with respect to FIG. 6 may be implemented in hardware or in hardware combined with software and/or firmware.
  • these modules may be implemented as computer program codes/instructions configured to be executed in one or more processors and stored in a computer-readable storage medium.
  • these modules can be implemented as hardware logic/circuitry.
  • one or more of the data acquisition module 610, the decorrelation module 620, and the classification module 630 may be implemented together in a System on Chip (SoC).
  • SoC System on Chip
  • the SoC may include an integrated circuit chip (which includes a processor (for example, a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (Digital Signal Processor, DSP), etc.), a memory, a Or multiple communication interfaces, and/or one or more components in other circuits), and may optionally execute the received program code and/or include embedded firmware to perform functions.
  • a processor for example, a central processing unit (CPU), a microcontroller, a microprocessor, a digital signal processor (Digital Signal Processor, DSP), etc.
  • DSP Digital Signal Processor
  • the SoC may optionally execute the received program code and/or include embedded firmware to perform functions.
  • a computer device including a memory, a processor, and a computer program stored on the memory.
  • the processor is configured to execute a computer program to implement the steps of any method embodiment described above.
  • a non-transitory computer-readable storage medium having a computer program stored thereon, and the computer program, when executed by a processor, implements the steps of any method embodiment described above.
  • a computer program product which includes a computer program that, when executed by a processor, implements the steps of any method embodiment described above.
  • Figure 9 shows an example configuration of a computer device 900 that can be used to implement the methods described herein.
  • the mobile device 110 and/or the server 120 shown in FIG. 1 may include an architecture similar to the computer device 900.
  • the user behavior recognition apparatus 600 described above may also be fully or at least partially implemented by the computer device 900 or similar devices or systems.
  • the computer device 900 may be a variety of different types of devices, such as a server of a service provider, a device associated with a mobile device, a system on a chip, and/or any other suitable computer device or computing system.
  • Examples of computer equipment 900 include, but are not limited to: desktop computers, server computers, laptops or netbook computers, mobile devices (for example, tablet computers, cellular or other wireless phones (for example, smart phones), notebook computers, mobile stations), Wearable devices (eg, glasses, watches), entertainment devices (eg, entertainment appliances, set-top boxes communicatively coupled to display devices, game consoles), televisions or other display devices, car computers, and so on.
  • the computer device 900 can range from full-resource devices with large amounts of memory and processor resources (e.g., personal computers, game consoles) to low-resource devices with limited memory and/or processing resources (e.g., traditional set-top boxes). , Handheld game console).
  • memory and processor resources e.g., personal computers, game consoles
  • processing resources e.g., traditional set-top boxes
  • Handheld game console e.g., Handheld game console
  • the computer device 900 may include at least one processor 902, a memory 904, a communication interface(s) 906, a display device 908, and other input/output (I/O) devices capable of communicating with each other, such as through the system bus 914 or other appropriate connections. 910 and one or more mass storage devices 912.
  • the computing device 900 further includes a sensor 924 for collecting data on its motion state.
  • the sensor 924 includes, but is not limited to, an acceleration sensor, a gyroscope, and the like.
  • the processor 902 may be a single processing unit or multiple processing units, and all processing units may include a single or multiple computing units or multiple cores.
  • the processor 902 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and/or any devices that manipulate signals based on operating instructions.
  • the processor 902 may be configured to obtain and execute computer-readable instructions stored in the memory 904, the mass storage device 912, or other computer-readable media, such as the program code of the operating system 916, and the application program 918 The program code of 920, the program code of other programs 920, etc.
  • the memory 904 and the mass storage device 912 are examples of computer-readable storage media for storing instructions, which are executed by the processor 902 to implement the various functions described above.
  • the memory 904 may generally include both volatile memory and non-volatile memory (e.g., RAM, ROM, etc.).
  • the mass storage device 912 may generally include hard disk drives, solid state drives, removable media, including external and removable drives, memory cards, flash memory, floppy disks, optical disks (such as CDs, DVDs), storage arrays, and network-attached storage. , Storage Area Network, etc.
  • Both the memory 904 and the mass storage device 912 may be collectively referred to herein as a memory or a computer-readable storage medium, and may be non-transitory media capable of storing computer-readable and processor-executable program instructions as computer program codes.
  • the computer program code may be executed by the processor 902 as a specific machine configured to implement the operations and functions described in the examples herein.
  • Program modules may be stored on the mass storage device 912. These programs include an operating system 916, one or more application programs 918, other programs 920, and program data 922, and they can be loaded into the memory 904 for execution. Examples of such application programs or program modules may include, for example, computer program logic (e.g., computer program code or instructions) for implementing the following components/functions: client application 112 (including data receiving module 610, decorrelation module 620, and classification Module 630), method 200 (including any suitable steps of method 200), and/or additional embodiments described herein.
  • client application 112 including data receiving module 610, decorrelation module 620, and classification Module 630
  • method 200 including any suitable steps of method 200
  • modules 916, 918, 920, and 922 or parts thereof may be implemented using any form of computer readable media that can be accessed by the computer device 900.
  • “computer-readable media” includes at least two types of computer-readable media, namely computer storage media and communication media.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented by any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or Other data.
  • Computer storage media include but are not limited to RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital versatile disk (DVD), or other optical storage devices, magnetic cassettes, tapes, disk storage devices or other magnetic storage devices, Or any other non-transmission medium that can be used to store information for computer equipment to access.
  • a communication medium can specifically implement computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism.
  • Computer storage media as defined herein does not include communication media.
  • the computer device 900 may also include one or more communication interfaces 906 for exchanging data with other devices, such as through a network, direct connection, etc., as discussed above.
  • a communication interface may be one or more of the following: any type of network interface (for example, network interface card (NIC)), wired or wireless (such as IEEE 802.11 wireless LAN (WLAN)) wireless interface, global microwave Access interoperability (Wi-MAX) interface, Ethernet interface, universal serial bus (USB) interface, cellular network interface, Bluetooth TM interface, near field communication (NFC) interface, etc.
  • the communication interface 906 can facilitate communication within a variety of network and protocol types, including wired networks (such as LAN, cable, etc.) and wireless networks (such as WLAN, cellular, satellite, etc.), the Internet, and so on.
  • the communication interface 906 may also provide communication with external storage devices (not shown) such as in storage arrays, network-attached storage, storage area networks, and the like.
  • a display device 908 such as a monitor may be included for displaying information and images to the user.
  • the other I/O device 910 may be a device that receives various inputs from the user and provides various outputs to the user, and may include a touch input device, a gesture input device, a camera, a keyboard, a remote control, a mouse, a printer, and an audio input/ Output devices and so on.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Quality & Reliability (AREA)
  • Computer Hardware Design (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Image Analysis (AREA)

Abstract

La présente divulgation concerne un procédé et un appareil d'identification de comportement d'utilisateur, un dispositif informatique et un support. Selon la présente divulgation, le procédé d'identification de comportement d'utilisateur consiste : à obtenir une pluralité de groupes de données d'état de mouvement, chaque groupe de données d'état de mouvement comprenant une pluralité d'éléments d'informations d'état, les informations d'état étant utilisées pour représenter un état de mouvement d'un dispositif mobile porté par un utilisateur ; à effectuer un traitement de décorrélation sur la pluralité de groupes de données d'état de mouvement pour éliminer une corrélation entre les informations d'état dans chaque groupe de données d'état de mouvement de façon à obtenir des données cibles ; et déterminer un type de comportement d'utilisateur en fonction des données cibles.
PCT/CN2021/077744 2020-02-28 2021-02-24 Procédé et appareil d'identification de comportement d'utilisateur, dispositif informatique et support WO2021170014A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010127977.5 2020-02-28
CN202010127977.5A CN111427754A (zh) 2020-02-28 2020-02-28 一种用户行为识别方法及移动终端

Publications (1)

Publication Number Publication Date
WO2021170014A1 true WO2021170014A1 (fr) 2021-09-02

Family

ID=71547248

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/077744 WO2021170014A1 (fr) 2020-02-28 2021-02-24 Procédé et appareil d'identification de comportement d'utilisateur, dispositif informatique et support

Country Status (2)

Country Link
CN (1) CN111427754A (fr)
WO (1) WO2021170014A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113869299A (zh) * 2021-09-30 2021-12-31 中国平安人寿保险股份有限公司 一种银行卡识别方法、装置、计算机设备及存储介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111427754A (zh) * 2020-02-28 2020-07-17 北京腾云天下科技有限公司 一种用户行为识别方法及移动终端
CN115527373B (zh) * 2022-01-05 2023-07-14 荣耀终端有限公司 乘车工具识别方法及装置
CN114828208A (zh) * 2022-03-29 2022-07-29 Oppo广东移动通信有限公司 终端位置的识别方法及装置、计算机可读介质、电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140365164A1 (en) * 2010-10-04 2014-12-11 Numera, Inc. Fall detection system using a combination of accelerometer, audio input and magnetometer
CN108196998A (zh) * 2018-01-02 2018-06-22 联想(北京)有限公司 一种状态识别方法、移动设备及服务器
CN110367967A (zh) * 2019-07-19 2019-10-25 南京邮电大学 一种基于数据融合的便携型轻量化人脑状态检测方法
CN111427754A (zh) * 2020-02-28 2020-07-17 北京腾云天下科技有限公司 一种用户行为识别方法及移动终端

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108170274A (zh) * 2017-12-29 2018-06-15 南京邮电大学 一种基于可穿戴设备的动作识别方法
CN108245880A (zh) * 2018-01-05 2018-07-06 华东师范大学 基于多穿戴环带传感器融合的体感检测可视化方法和系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140365164A1 (en) * 2010-10-04 2014-12-11 Numera, Inc. Fall detection system using a combination of accelerometer, audio input and magnetometer
CN108196998A (zh) * 2018-01-02 2018-06-22 联想(北京)有限公司 一种状态识别方法、移动设备及服务器
CN110367967A (zh) * 2019-07-19 2019-10-25 南京邮电大学 一种基于数据融合的便携型轻量化人脑状态检测方法
CN111427754A (zh) * 2020-02-28 2020-07-17 北京腾云天下科技有限公司 一种用户行为识别方法及移动终端

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113869299A (zh) * 2021-09-30 2021-12-31 中国平安人寿保险股份有限公司 一种银行卡识别方法、装置、计算机设备及存储介质
CN113869299B (zh) * 2021-09-30 2024-06-11 中国平安人寿保险股份有限公司 一种银行卡识别方法、装置、计算机设备及存储介质

Also Published As

Publication number Publication date
CN111427754A (zh) 2020-07-17

Similar Documents

Publication Publication Date Title
WO2021170014A1 (fr) Procédé et appareil d'identification de comportement d'utilisateur, dispositif informatique et support
US11763599B2 (en) Model training method and apparatus, face recognition method and apparatus, device, and storage medium
WO2020233333A1 (fr) Procédé et dispositif de traitement d'images
US10838967B2 (en) Emotional intelligence for a conversational chatbot
WO2022022274A1 (fr) Procédé et appareil d'instruction de modèles
US20220188840A1 (en) Target account detection method and apparatus, electronic device, and storage medium
US11727270B2 (en) Cross data set knowledge distillation for training machine learning models
CN111738403B (zh) 一种神经网络的优化方法及相关设备
US20170193296A1 (en) Image integration search based on human visual pathway model
WO2021047587A1 (fr) Procédé de reconnaissance de geste, dispositif électronique, support de stockage lisible par ordinateur et puce
EP3382596B1 (fr) Procédé et appareil d'apprentissage matriciel de modèle de visage humain ainsi que support d'enregistrement
CN110335334A (zh) 虚拟头像驱动显示方法、装置、电子设备及存储介质
CN113065635A (zh) 一种模型的训练方法、图像增强方法及设备
WO2023231954A1 (fr) Procédé de débruitage de données et dispositif associé
US10133955B2 (en) Systems and methods for object recognition based on human visual pathway
US11443116B2 (en) Electronic apparatus and control method thereof
WO2022147592A1 (fr) Prédiction de données cérébrales à l'aide de modèles d'apprentissage automatique
WO2021190433A1 (fr) Procédé et dispositif de mise à jour de modèle de reconnaissance d'objet
WO2024002167A1 (fr) Procédé de prédiction d'opération et appareil associé
US20220284990A1 (en) Method and system for predicting affinity between drug and target
US20220300740A1 (en) System and method for enhancing machine learning model for audio/video understanding using gated multi-level attention and temporal adversarial training
CN110059748A (zh) 用于输出信息的方法和装置
CN109829431A (zh) 用于生成信息的方法和装置
CN113627421A (zh) 一种图像处理方法、模型的训练方法以及相关设备
CN113284206A (zh) 信息获取方法及装置、计算机可读存储介质、电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21760107

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21760107

Country of ref document: EP

Kind code of ref document: A1