KR20190085604A - Method, apparatus and computer program for recognition of a user activity - Google Patents

Abstract

According to an embodiment of the present invention, provided is a method for recognizing a user behavior using sensing data, which comprises the steps of: generating sensing data in a user smartphone and a wearable device; in a server, collecting the sensing data and applying the collected data to a machine learning framework, so that a first model of smartphone generation data, a second model of wearable device generation data, and a third model integrating the smartphone generation data and the wearable device generation data; and extracting a feature for each of the first model, the second model, and the third model to estimate a user behavior for each of the first model, the second model, and the third model.

Description

METHOD, APPARATUS AND COMPUTER PROGRAM FOR RECOGNITION OF A USER ACTIVITY,

The present invention relates to a method of recognizing a user's behavior by using sensor data collected from a user device such as a mobile device such as a smart phone and / or a wearable diabase such as a smart watch. More particularly, the present invention relates to a method and apparatus for assigning weights to a plurality of models for applying a machine learning framework to more accurately recognize a behavior of a user from collected sensor information, The present invention relates to a method for judging user behavior from data.

Recently, according to the rapid development of Internet technology, computing technology capable of recognizing the situation more accurately including the specific user's behavior is being developed from collected IoT data.

Situational cognition technology is a technology aimed at more accurately grasping the user's behavior and movement mathematically without human intervention. In general, it is an inertial sensor that attaches a sensor such as a video camera or an acceleration sensor to the body, Pressure sensor, motion sensor, or the like, and then applying the collected data to the machine learning framework to derive an estimation result.

In order to collect image data using a camera and sensor data using an object sensor, a camera, a pressure sensor, and a motion sensor device must be installed in an arbitrary space. That is, in order to recognize the action based on the camera or the object sensor, it is possible to recognize the action only in the space where the camera or the object sensor is installed. However, since the inertial sensor can be attached to a person's body, there is an advantage that the user's behavior can be perceived regardless of the place.

Further, when the user data is collected based on the inertial sensor, a plurality of sensors can be attached to the body element region, so that data that can more accurately recognize the user's behavior and / or situation can be collected. However, if this is applied to an actual user, since a plurality of sensors are attached to the user, the movement of the user becomes inconvenient, and therefore there is a problem that the user convenience is poor.

Therefore, although the sensor is not directly attached to the body of the user, a method of collecting data for sensing the behavior may be considered using a smart phone equipped with various sensors and generally positioned at a recent distance from the user. The smartphone has not only various sensors but also computing power, and it has a high user convenience.

However, in the case of data collection using only a smart phone, there is a limitation in collecting data reflecting individual characteristics of a body part that can perform various actions with only a single device, so that a wearable smart device such as a smart watch have.

The smart watch can be worn on the wearer's wrist by adopting a band-like design, but this may also be insufficient to collect data that reflects each feature of the body part.

Accordingly, the present invention collects sensing data using both wearable devices such as smart phones and smart watches, and collects the collected data from each independent device and the user behavior that can collectively estimate the data collected from each device And is provided with high reliability.

For this, according to the embodiment of the present invention, three cognitive models of a single smartphone, a single smart watch, a smartphone and a SmartWall integration are created, and weighting and integration of each model are performed to obtain a higher reliability And provide user behavior awareness.

The present invention has been made to solve the above problems. More particularly, the present invention aims at providing a framework that can estimate the behavior of a user with higher reliability by using sensing data collected from a smartphone and / or a smart watch.

A method of recognizing a user's behavior using sensing data according to an embodiment of the present invention includes generating sensing data in a user smartphone and a wearable device; The method of claim 1, wherein at the server, the sensing data is collected and the collected data is applied to a machine learning framework to generate a first model for the smartphone generated data, a second model for the wearable device generated data, And a third model for integrating the wearable device creation data; Extracting features for each of the first model, the second model, and the third model, and estimating a user behavior for each of the first model, the second model, and the third model.

Generating a weight table for applying to each of the first model, the second model, and the third model after the step of estimating the user behavior; And And applying the weight table to the user behavior estimated through the first model, the second model, and the third model.

Further, the step of generating the weight table may include applying the same first weight to each of the actions estimated through the first model, the second model, and the third model. Applying a second weight higher than the first weight to an action of less than a predetermined accuracy among the estimated actions and applying a third weight lower than the first weight to an action of a predetermined accuracy or more; And setting the weight to 0 for the action estimated through the device irrelevant to the action and increasing the weight of the action estimated only through the specific device.

And further comprising the step of comparing the priorities for the same weighted behavior after the step of applying the weighted table.

Meanwhile, a method of recognizing a user's behavior using sensing data according to an embodiment of the present invention includes: generating sensing data in a user smartphone and a wearable device; The method of claim 1, wherein at the server, the sensing data is collected and the collected data is applied to a machine learning framework to generate a first model for the smartphone generated data, a second model for the wearable device generated data, Generating a third model that integrates the wearable device creation data, extracting features for each model, and estimating a user behavior for each of the models; And generating a weight table to be applied to each of the first model, the second model, and the third model, and assigning the weight table to the user behavior estimated through the first model, the second model, Applying; And And comparing the priorities for the same weighted action.

Further, a system for recognizing user behavior according to an embodiment of the present invention includes: a user smartphone including at least one sensor; A wearable device comprising at least one sensor, the wearable device being attached to the user; A data processing server for collecting and processing sensing data generated in the smartphone and the wearable device; And applying data processed by the data processing server to a machine learning framework to generate a first model for the smartphone generated data, a second model for the wearable device generated data, and a second model for the smartphone generated data and the wearable device Generating a third model that integrates the generated data, extracting features for each model, estimating a user behavior for each of the models, and calculating a weight table for applying the first model, the second model, And an action recognition server for applying the weight table to user actions estimated through the first model, the second model, and the third model.

According to the present invention, a computer can recognize a user's behavior more accurately without human intervention.

FIG. 1 is a system configuration diagram for collecting sensing data from a smartphone and smart watch according to an embodiment of the present invention and applying it to a machine learning framework
2 is a flowchart illustrating a process of recognizing a user's behavior according to an embodiment of the present invention.
FIG. 3 is a flowchart for explaining a preprocessing process for collecting and modeling sensor data

It is to be understood that the present invention is not limited to the description of the embodiments described below, and that various modifications may be made without departing from the technical scope of the present invention. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

In the drawings, the same components are denoted by the same reference numerals. And in the accompanying drawings, some of the elements may be exaggerated, omitted or schematically illustrated. It is intended to clearly illustrate the gist of the present invention by omitting unnecessary explanations not related to the gist of the present invention.

FIG. 1 is a configuration diagram of an action recognition system for collecting sensing data from a smartphone and a smart watch according to an embodiment of the present invention and applying the same to a machine learning framework.

As shown in FIG. 1, an action recognition system according to an embodiment of the present invention includes a smartphone 10, a wearable device 20, a sensing data collection and processing server 40, And a server 60 for providing a service using the action recognition server 50 and the action recognition result.

The smartphone 10 according to the embodiment of the present invention can perform the function of transmitting the sensing data collected using the included sensor to the sensing data collection and processing server 40. [ For this purpose, the smartphone 10 may be configured to include a sensor for measuring a physical amount of specific information, for example, position, motion, brightness, etc., from an object to be measured and converting it into an electrical signal.

Examples of sensors that can be included in a smart phone include an acceleration sensor, a gravity sensor, a gyro sensor, a temperature sensor, a pressure sensor, an illuminance sensor, a proximity sensor, a GPS sensor, a geomagnetic sensor, and a direction sensor.

More specifically, the gyro sensor can measure the rotational speed of the X, Y, and Z axes to generate data that can be used to check information about the degree of inclination or rotation of the smartphone. Data can be measured. Further, the pressure sensor can generate data of atmospheric pressure measurement data, the ambient light intensity measurement data of the ambient light intensity, and the proximity sensor can measure the existence of the object to be measured and the distance to the screen without physical contact. Furthermore, the GPS sensor can generate data on the time and position at the time of utilizing the GPS satellite. The geomagnetic sensor measures the azimuth angle using the geomagnetic field, and the direction sensor measures the rotation angle to measure the gravity sensor and geomagnetic sensor Or the like, together with the location information.

Meanwhile, FIG. 1 illustrates the smartphone 10, which is merely an example, and the present invention can not be construed as being limited thereto. That is, the smartphone 10 according to the embodiment of the present invention should be construed to mean a mobile device including a predetermined sensor, which is generally portable by the user at all times. For example, a smart phone, a tablet personal computer, a mobile phone, a video phone, an e-book reader, a desktop personal computer, a laptop a personal computer, a netbook computer, a workstation, a server, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera have.

The wearable device 20 illustrated in FIG. 1 can also perform a function of transmitting sensed data collected using a sensor included in the wearable device 20 to the sensing data collection and processing server 40. To this end, the wearable device 20 may include various sensors.

More specifically, the wearable device 20 is equipped with an acceleration sensor, a gravity sensor, a gyro sensor, a temperature sensor, a pressure sensor, an illuminance sensor, a proximity sensor, a GPS sensor, a geomagnetic sensor, And can further include a heart rate monitor sensor, an oxygen measurement sensor, a skin conduction sensor, a skin temperature sensor, and the like that can utilize the characteristics attached to the user's body.

The heart rate monitor sensor can, for example, sense optical blood flow information, and the oxygen measurement sensor can generate data that can accurately determine the pulse rate by measuring the oxygen of the user's blood. Furthermore, the skin conduction sensor can generate data that can be used to measure the current response of the skin, for example, how much perspiration the user is sweating, thereby determining the intensity of physical activity and calculating calorie consumption figures .

In the example of Fig. 1, the wearable device 20 may be, for example, a smart watch, but the present invention can not be construed to be limited thereto. A wearable device according to an embodiment of the present invention may be of the accessory type such as a watch, a ring, a bracelet, a bracelet, a necklace, a pair of glasses, a contact lens or a head-mounted-device (HMD) (E.g., electronic apparel), a body attachment type, e.g., a skin pad or tattoo, or a bioimplantable, e.g., implantable circuit.

In other embodiments, the wearable device 20 may be used in a variety of medical devices, such as various portable medical measurement devices such as a blood glucose meter, a heart rate meter, a blood pressure meter, or a body temperature meter, a magnetic resonance angiography (MRA) imaging, computed tomography (CT), an imaging device, or an ultrasonic device.

The sensing data collecting and processing server 40 shown in FIG. 1 may collect sensing data generated through the smartphone 10 and / or the wearable device 20 and perform a preprocessing of the sensing data.

In the example of FIG. 1, the sensing data collection, processing server 40 and behavior awareness server 50 are shown as being separate devices, but this is only an example. That is, the sensing data collection, processing server 40, and behavior awareness server 50 according to an embodiment of the present invention may be physically one or more hardware devices and may be virtualized software.

The sensing data collection and processing server 40 may receive the sensed data transmitted by the smartphone 10 and / or the wearable device 20 and store the sensed data in the buffer memory. Further, the sensing data can be pre-processed while sequentially loading the sensing data from the buffer.

The sensing data collection and processing server 40 may collect sensing data from the smartphone 10 and / or the wearable device 20 through various types of interface protocols. For example, FTP (File Transfer Protocol), HTTP (Hyper Text Transfer Protocol), and TCP / IP (Transmission Control Protocol / Internet Protocol) may be considered as the interface protocol.

The collected sensing data may be stored in a buffer as needed and sequentially loaded in the buffer so that preprocessing processing may be performed to generate preprocessed sensing data. The preprocessed sensing data may be stored in a storage unit or transmitted to the action recognition server 50 for processing a user action result, if necessary.

The behavior awareness server 50 according to an embodiment of the present invention may include a machine learning framework. Further, a machine learning model may be generated to estimate a user's behavior by applying a machine learning framework to sensing data, and a function of estimating a user's behavior by learning a model generated in the collected data may be performed.

More specifically, the action recognition server 50 of the present invention includes a first model for learning data generated from the smartphone 10, a second model for learning data generated from the wearable device 20, 10) and the data generated from the wearable device 20 can be generated. The behavior awareness server 50 may generate at least one of the first model, the second model, and the third model, respectively.

The reason why the model applied to the data generated from the smartphone, the model applied to the data generated from the wearable device, and the model integrated into the data are separately configured in the above embodiment is that the smartphone Because the SmartWatch is attached to different locations, each sensor has perceptible and unrecognizable behaviors.

For example, in the case of "eating", it is impossible to perceive the behavior unless the sensor is located on the arm. Therefore, according to the embodiment of the present invention, an integrated model using both the smartphone and the smart watch model and the data of the two devices can be generated to recognize the behavior of the user.

For example, the first model may be generated using data of an acceleration sensor and a gyro sensor of a smartphone, and the second model may be generated using data of an acceleration sensor and a gyro sensor of a wearable device. Furthermore, the third model can be generated using data of acceleration sensors and gyro sensors of smart phones and wearable devices.

The first model recognizes an action that can be perceived by the user's overall motion. The second model recognizes an action that can be perceived by the user's arm motion. Further, the third model is used to increase the final reliability of the overall action. It can be used as a criterion for obtaining a weight that can be applied to each model.

The first model, the second model, and the third model according to an embodiment of the present invention may be generated according to a conventional behavior sensing technique through data mining. The feature values of each model can be selected from the collected data through the feature selection function of Matlab, and the features mainly used in the time domain can be used.

For example, the first model and the second model use 42 features, and the third model can use a total of 49 features. More specifically, the characteristic values include a mean, a maximum value, a minimum value, a standard deviation, a quartile, an absolute mean, an absolute median, Eight characteristic values such as a geometric mean can be used.

The formula used for each feature can be referenced to the following equation.

[Equation 1]

? The behavior awareness server 50 of FIG. 1 can generate each model for integrating and applying smartphone creation data, wearable device creation data, and data generated in a smart phone and a wearable device. Further, it is possible to extract the feature by applying the corresponding data to each model, and to weight the estimated user action and / or the extracted feature itself using the extracted feature.

More specifically, the behavior awareness server 50 according to the embodiment of the present invention may generate a weight table for applying to each model. For example, weights tables can be created using weighted voting-based algorithms that are commonly used in decision-making systems and can correct for misrecognized behaviors.

It is important to set a weighting criterion because the decision-making system using the weighted voting technique can be different depending on how the weights are applied.

For example, the action awareness server 50 according to the embodiment of the present invention may first assign a weight of 2 for each action. Second, according to the offline test result of each model, it is possible to assign a weight of 3 to a less accurate action (for example, less than 80% accuracy). Third, highly accurate actions (for example, greater than 90% accuracy) can give a weight of one.

Furthermore, for the fourth act which is not related to the act, it can be given a weight of zero. For example, in the case of an eating behavior, it can not be estimated with data generated on a smartphone. People can not eat while holding a smartphone in their hands. Therefore, weights may not be given in this case.

Finally, when a certain behavior is extracted only from a specific model, the corresponding weight can be given as 3, and a weight table can be created through such a series of processes.

In the third embodiment of the present invention, a high accuracy and a low weight are given in the third step because a highly accurate operation can be highly reliable regardless of a weight. In addition, the reason for giving the high weight to the actions that show low accuracy in the second step is that high weighting is given to the actions with low accuracy, so that it is possible to give a high competitive power to the behavior estimates derived from other models.

Furthermore, the action recognition server 50 according to the embodiment of the present invention can apply the created weight table to the action recognition result estimated by each model.

For example, when walking in the first model, walking in the second model, and walking in the third model are estimated, and α, β, and γ are applied to the respective behaviors through the weight table, , γ walk can be derived. The action-aware server may then sum the weighted actions. In the above example, the result of the act summation will be (α + γ) walking, β run.

Furthermore, the action recognition server 50 according to the embodiment of the present invention may perform the action priority comparison step. The priority comparison step can be performed when the act coefficients are the same, and can be omitted when the coefficients are different.

For example, a behavior awareness server can increase the behavior coefficient of high accuracy by one based on the accuracy of each action when the behavior coefficient is the same. The final action can be determined by combining the results of the weighted actions.

2 is a flowchart illustrating a process of recognizing a user's behavior according to an embodiment of the present invention.

A user behavior recognition system according to an embodiment of the present invention includes a smart phone and a wearable device, and user behavior data may be collected by processing sensed data through a smart phone or a wearable device. (Step 210)

A more specific example in which user behavior data is collected will be described below with reference to FIG. 3 attached hereto.

3 is a flowchart illustrating a preprocessing process for collecting and modeling sensor data.

In step 310, a smartphone and / or wearable device according to an embodiment of the present invention may generate sensing data using the included sensor and transmit sensing data to a data collection and processing server.

For this purpose, a smartphone or a wearable device may be provided with a sensor for measuring a physical amount of specific information, for example, position, motion, brightness, etc., from an object to be measured and converting it into an electric signal.

The sensing data collection and processing server may receive sensing data from a smartphone and / or a wearable device and store the sensed data in a buffer memory. Further, the sensing data can be pre-processed while sequentially loading the sensing data from the buffer. (Steps 320 and 330)

More specifically, the sensing data collection, processing server can collect sensing data from various types of interface protocols from smart phones and / or wearable devices. For example, FTP (File Transfer Protocol), HTTP (Hyper Text Transfer Protocol), and TCP / IP (Transmission Control Protocol / Internet Protocol) may be considered as the interface protocol.

The collected sensing data may be stored in a buffer as needed and sequentially loaded in the buffer so that preprocessing processing may be performed to generate preprocessed sensing data. The preprocessed sensing data may be stored in a storage unit or transmitted to the server if the need arises to process user action results. At this time, the sensing data can be synchronized with the sensing data stored in the processing server and the sensing data stored in the activity recognition server. (Step 340)

Returning to the description of FIG. 2, the action or server receiving the user action data from the sensing data collection processing server may generate at least one machine learning model by applying the user action data to the machine learning frame week. (Step 220)

More specifically, an action awareness server according to an embodiment of the present invention may include a machine learning framework. Further, a machine learning model may be generated to estimate a user's behavior by applying a machine learning framework to user behavior data, and a function of estimating a user's behavior by learning a model generated in the collected data may be performed.

For example, the action awareness server may include a first model for learning user action data generated from a smartphone, a second model for learning data generated from the wearable device, data generated from the smartphone, A third model for learning and integrating data can be generated. At least one of the first model, the second model, and the third model may be generated.

At this time, the reason why the model applied to the data generated from the smartphone, the model applied to the data generated from the wearable device, and the model applied to the data are separately configured is that the smartphone and smart watch Because each sensor is attached to different locations, each sensor has perceptible and unrecognizable behavior.

For example, in the case of "eating", it is impossible to perceive the behavior unless the sensor is located on the arm. Therefore, according to the embodiment of the present invention, an integrated model using both the smartphone and the smart watch model and the data of the two devices can be generated to recognize the behavior of the user.

For example, the first model may be generated using data of an acceleration sensor and a gyro sensor of a smartphone, and the second model may be generated using data of an acceleration sensor and a gyro sensor of a wearable device. Furthermore, the third model can be generated using data of acceleration sensors and gyro sensors of smart phones and wearable devices.

The first model can be used to recognize perceptible behavior with the user's overall motion, the second model can be used to perceive perceptible behaviors with the user's arm action, and further, And can be used as a criterion for obtaining a weight that can be applied to each model.

The first model, the second model, and the third model according to an embodiment of the present invention may be generated according to a conventional behavior sensing technique through data mining. The feature values of each model can be selected from the collected data through the feature selection function of Matlab, and the features mainly used in the time domain can be used.

For example, the first model and the second model use 42 features, and the third model can use a total of 49 features. More specifically, the characteristic values include a mean, a maximum value, a minimum value, a standard deviation, a quartile, an absolute mean, an absolute median, Eight characteristic values such as a geometric mean can be used.

Further, the behavior awareness server can estimate user behavior for each of a plurality of models at a level higher than a preset reliability level. (Step 230)

For example, the behavior awareness server generates a model for integrating and applying smartphone creation data, wearable device creation data, and data generated in a smartphone and a wearable device, and applying the data to each model, And the user behavior can be estimated using the extracted features.

The action-aware server then weights and integrates the estimated actions. (Step 240)

More specifically, the action recognizing server according to the embodiment of the present invention can generate a weight table for each model. For example, weights tables can be created using weighted voting-based algorithms that are commonly used in decision-making systems and can correct for misrecognized behaviors.

It can be important to set a weighting criterion because the decision-making system using the weighted voting technique can be different from the action recognition result depending on how the weight is applied.

For example, we assign a weight of 2 for all estimated actions, and second, we assign a weight of 3 to actions with low accuracy (for example, less than 80% accuracy) based on the results of an offline test of each model can do. Third, highly accurate actions (for example, greater than 90% accuracy) can give a weight of one.

Furthermore, it is possible to assign a weight of zero to an act estimated through a device not related to an action for the fourth time. For example, in the case of an eating behavior, it can not be estimated with data generated on a smartphone. People can not eat while holding a smartphone in their hands. Therefore, weights may not be given in this case.

Finally, when a certain behavior is extracted only from a specific model, the corresponding weight can be given as 3, and a weight table can be created through such a series of processes.

In the third embodiment of the present invention, a high accuracy and a low weight are given in the third step because a highly accurate operation can be highly reliable regardless of a weight. In addition, the reason for giving the high weight to the actions that show low accuracy in the second step is that high weighting is given to the actions with low accuracy, so that it is possible to give a high competitive power to the behavior estimates derived from other models.

The weight table thus created will be applied to the behavior recognition result estimated by each model. For example, when walking in the first model, walking in the second model, and walking in the third model are estimated, and α, β, and γ are applied to the respective behaviors through the weight table, , γ walk can be derived.

The action-aware server may then aggregate the weighted actions, i.e., combine the weights. In the above example, the result of the act summation will be (α + γ) walking, β run.

Furthermore, the action awareness server can compare the action priorities and apply the priorities to derive the final action. (Steps 250 and 260)

The priority comparison may be performed when the act coefficients are the same, and may be omitted when the coefficients are different. For example, if the act coefficients are the same, the act coefficient can be increased by 1, which shows high accuracy based on the accuracy of each action. The final action can be determined by combining the results of the weighted actions.

The embodiments of the present invention disclosed in the present specification and drawings are intended to be illustrative only and not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

Claims (8)

A method for recognizing a user's behavior using sensing data,
Generating sensing data in a user smartphone and a wearable device;
The method of claim 1, wherein at the server, the sensing data is collected and the collected data is applied to a machine learning framework to generate a first model for the smartphone generated data, a second model for the wearable device generated data, And a third model for integrating the wearable device creation data;
Extracting features for each of the first model, the second model, and the third model, and estimating a user behavior for each of the first model, the second model, and the third model, Behavior recognition method. 2. The method of claim 1, wherein, after estimating the user behavior,
Generating a weight table for applying to each of the first model, the second model, and the third model; And
And applying the weight table to user actions estimated through the first model, the second model, and the third model. 3. The method of claim 2, wherein generating the weight table comprises:
Applying the same first weight to each of the actions estimated through the first model, the second model, and the third model;
Applying a second weight higher than the first weight to an action of less than a predetermined accuracy among the estimated actions and applying a third weight lower than the first weight to an action of a predetermined accuracy or more; And
Setting a weight for a behavior estimated through a device irrelevant to a behavior to 0 and setting a weight of an action estimated only through a specific device to a higher value. 4. The method of claim 3, wherein after applying the weight table,
And comparing the priorities for actions having the same weight. A method for recognizing a user's behavior using sensing data,
Generating sensing data in a user smartphone and a wearable device;
The method of claim 1, wherein at the server, the sensing data is collected and the collected data is applied to a machine learning framework to generate a first model for the smartphone generated data, a second model for the wearable device generated data, Generating a third model that integrates the wearable device creation data, extracting features for each model, and estimating a user behavior for each of the models;
And generating a weight table to be applied to each of the first model, the second model, and the third model, and assigning the weight table to the user behavior estimated through the first model, the second model, Applying; And
And comparing the priorities for actions having the same weight. In a system for perceiving user behavior,
A user smartphone comprising at least one sensor;
A wearable device comprising at least one sensor, the wearable device being attached to the user;
A data processing server for collecting and processing sensing data generated in the smartphone and the wearable device; And
Applying data processed by the data processing server to a machine learning framework to generate a first model for the smartphone generated data, a second model for the wearable device generated data, and a second model for the smartphone generated data and the wearable device creation A third model for integrating data, extracting a feature for each model, estimating a user behavior for each of the models, and applying a weight table for applying the model to each of the first model, the second model, and the third model And applying the weight table to the user behavior estimated through the first model, the second model, and the third model. 7. The system according to claim 6,
The first model, the second model, and the third model, and for each of the behaviors equal to or less than a preset accuracy, A second weight lower than the first weight is applied to an action of a predetermined accuracy or more, a weight is set to 0 for an action estimated through a device irrelevant to the action, And setting the weight of the user action recognition system to be higher. 8. The system of claim 7,
And compares priorities for actions having the same weight.

Images