JP6461639B2 - Learning data generation system, learning data generation method and program - Google Patents

Description

  When applying semi-supervised learning to user behavior recognition, the present invention improves behavior identification accuracy and corrects reliability by correcting erroneous behavior labels based on characteristic behavior for each stay location attribute. The present invention relates to a learning data generation system, a learning data generation method, and a program for generating learning data for high behavior recognition.

  Conventionally, a method for recognizing user behavior by using motion data and audio data collected from a mobile terminal device such as a smartphone or a wearable device has been proposed. Such a method is mainly based on supervised learning, but the cost for creating a large amount of labeled data is an issue. For this problem, it is known that self-training, which is one of semi-supervised learning, is effective. Self-training makes it possible to create a large amount of labeled data from a discriminator that can be generated from a small number of labeled data and a large amount of unlabeled data.

  Here, a large amount of unlabeled data can be easily collected in daily life. However, if an incorrect action label is assigned to unlabeled data, the recognition accuracy of the classifier may be reduced. Therefore, in Non-Patent Document 1, the reliability given to the knowledge for label selection in decision tree learning is used in order to provide the reliability in the rule and determine the reliability of the assigned label. That is, an action label selected based on knowledge with high reliability is determined as a label with high reliability.

  Further, in Non-Patent Document 2, labeling with high reliability is enabled by aggregating action label selection results by a plurality of algorithms of a plurality of different learning devices by majority vote.

Exploring Semi-Supervised and Active Learning for Activity Recognition; 12th IEEE International Symposium on Wearable Computers, 2008. M. Stikic, et al. Activity Recognition Based on Semi-Supervised Learning; 13th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications; 7. Guan, et al. The Impact of Personalization Smartphone-Based Activity Recognition; Technical report, Workshop on Activity Context Representation, 2012. G. M.M. Weisss and J.M. W. Lockhart.

  However, in both of Non-Patent Documents 1 and 2, since a highly reliable label is selected based only on the label selection result of the user's own discriminator, the recognition accuracy of the discriminator is low at the initial stage. However, there is a high possibility that an incorrect label is selected, and the recognition accuracy is not improved. This is a problem caused by the limited number of samples.

  Here, in self-training, the first discriminator is generally generated from the labeled data of other users. In this respect, depending on the user, the recognition accuracy of the discriminator in the initial stage is low, and it is difficult to improve the recognition accuracy by self-training. Because, according to Non-Patent Document 3, when only the user's own labeled data is used, the action can be identified with high accuracy. On the other hand, when the other user's labeled data is used, This is because it is known that if there are users with high behavior identification accuracy, there are also users with extremely low behavior identification accuracy.

  From the above, when applying self-training to behavior recognition, if the recognition accuracy of the classifier for the user's behavior is low at the initial stage, even if the user's own unlabeled data that can be collected in daily life is used There is a problem that the action recognition accuracy of the classifier is not improved.

  Therefore, the present invention has been made in view of the above-described problems, and corrects an erroneous action label based on a characteristic action for each stay place attribute when semi-supervised learning is applied to action recognition. Accordingly, an object of the present invention is to provide a learning data generation device, a learning data generation system, a learning data generation method, and a program that improve the action identification accuracy and generate learning data for action recognition with high reliability.

  The present invention proposes the following items in order to solve the above-described problems.

  When applying self-training to behavior recognition, the present invention improves behavior identification accuracy by correcting wrong behavior labels based on characteristic behavior (behavior characteristics) for each stay location attribute, High-level learning data (labeled data) is generated.

  That is, in self-training, if the recognition accuracy of the discriminator with respect to user behavior is low in the initial stage, there is a high possibility that an incorrect label will be selected. In order to improve the recognition accuracy of such a discriminator, it is required to correct the error label. Therefore, a method (first embodiment of the present application) is proposed in which a correct action label is specified from general person action characteristics (user's own action characteristics) and an error label is corrected. The human behavior characteristic means that a person takes a characteristic behavior for each staying place attribute. For example, it is reading at a bookstore, eating at a restaurant, and watching a video at a movie theater. That is, if the stay place attribute is known, it is possible to know the behavior that frequently occurs at the place. If the error of the classifier is constant (recognizing the same behavior for a certain behavior), the error label can be corrected based on the behavioral characteristics of the person. For example, consider a case where the classifier misrecognizes at a high rate that the user is watching a video at a restaurant. Here, in a restaurant, since a person should take the action of a meal, video viewing is determined to be a meal error, and video viewing is corrected to a meal. The stay location attribute can be acquired from the user's stay point (latitude, longitude) and map information.

  However, in the proposed method, it is necessary to grasp the human behavior characteristics for each place of stay attribute. In addition, the burden on the practitioner is high in terms of maintaining map information. For this reason, a system that does not use human behavior characteristics and map information (second embodiment of the present application) is proposed. In this method, the error label is corrected by specifying a characteristic behavior (behavior characteristic) of the staying point based on the recognition result of the discriminator of other users as well as the user. When self-training is applied to action recognition, there are users with low recognition accuracy of the classifier, while there are users with high recognition accuracy. Therefore, if the recognition result for the user's action with high recognition accuracy is followed, the characteristic action at the staying point can be specified. Therefore, the action recognition results generated at the highest frequency of the user at a certain stay point are aggregated by majority vote, and the action recognition results belonging to the majority are identified as characteristic actions. Then, the label of the action recognition result belonging to the minority is corrected with the characteristic action label. Here, it is considered that there is a high possibility that the label can be correctly corrected only when the vote rate for the majority is equal to or higher than the set threshold value, and it is also possible to correct an incorrect label. That is, according to the present invention, even when the recognition accuracy of the discriminator for the user's behavior is low at the initial stage, by using the user's own unlabeled data that can be collected in daily life, the behavior recognition accuracy of the discriminator can be improved. It is possible to provide a learning data generation system, a learning data generation method, and a program that can be improved.

  (1) The present invention is a learning data generation system including a mobile terminal and a server of each user, and the user's mobile terminal inputs any of the action recognition model input from the server, motion data, or sound data. Action recognition means for recognizing the user's action based on heel, operation determination means for determining whether or not the user is operating a portable terminal, position information detection means for detecting the user's position information, and Action data generating means for generating action data of the user based on action recognition data recognized by action recognition means, operation status of the portable terminal, and position information, and the server generates the action data Action data storage means for storing the action data of each user generated by the means, and action characteristics that are characteristic actions for each place of stay Action data storage means for storing, map data storage means for storing map information, and learning data generating means for generating learning data for recognizing the user's action based on the action data, action characteristics, and map data A learning data generation system comprising: learning data storage means for storing the generated learning data; and action recognition model generation means for generating an action recognition model based on the stored learning data Has proposed.

  (2) In the learning data generation system according to (1), the learning data generation unit specifies a stay location attribute from the stay point calculated from the location information and the map data, and the stay location attribute A learning data generation system is proposed in which a characteristic behavior at a staying point is obtained from the behavior characteristics.

  (3) In the learning data generation system according to (2), the behavior data includes at least a user ID, date and time, behavior label, position information, and feature amount, and the learning data generation unit includes the feature data If the behavioral action and the recognition result by the behavior recognition means are not matched, the behavior label corresponding to the recognition result is corrected with the behavior label indicating the characteristic behavior, and the learning data We have proposed a learning data generation system characterized by generating

  (4) The present invention is a learning data generation system including a mobile terminal and a server of each user, and each of the mobile terminals of each user inputs an action recognition model input from the server and either motion data or sound data. Action recognition means for recognizing the user's action based on heel, operation determination means for determining whether or not the user is operating a portable terminal, position information detection means for detecting the user's position information, and Action data generating means for generating action data of the user based on action recognition data recognized by action recognition means, operation status of the portable terminal, and position information, and the server generates the action data Action data storage means for storing action data of each user generated by the means, and the action of the user based on the action data Learning data generating means for generating learning data for recognition, learning data storage means for storing the generated learning data, action recognition model generating means for generating an action recognition model based on the stored learning data, A learning data generation system characterized by comprising:

  (5) In the learning data generation system according to (4), the behavior data includes at least a user ID, a date and time, an action label, position information, and a feature amount, and the learning data generation means is the same stay A learning data system characterized in that the action label recognized most frequently by all users at a point is determined by majority vote, and the action label of the majority who has obtained the most votes is regarded as a characteristic action at the staying point. Has proposed.

  (6) The present invention proposes a learning data generation system according to (5), wherein the learning data generation means calculates the number of votes obtained from the majority vote from equation (1).

  (7) The present invention relates to the learning data generation system according to (6), wherein the learning data generation means corrects an action label that is a minority in the majority at the same staying point with the action label of the majority, and learns A learning data generation system characterized by generating data.

  (8) In the learning data generation system according to (7), the learning data generation unit corrects the action label of the minority only when the vote rate of majority decision is equal to or higher than a preset threshold. Thus, a learning data generation system characterized by generating learning data is proposed.

  (9) The present invention proposes a learning data generation system according to (8), wherein the learning data generation means calculates the vote rate of majority vote from equation (2).

  (10) The present invention includes each user's mobile terminal and a server, and each user's mobile terminal is based on the action recognition model input from the server and either the motion data or the sound data. Action recognition means for recognizing action, operation determination means for determining whether or not the user is operating a portable terminal, position information detection means for detecting position information of the user, and recognition by the action recognition means Action data generating means for generating action data of the user based on action recognition data, the operation status of the mobile terminal, and the position information, and each server generated by the action data generating means Behavior data storage means for storing the behavior data, behavior characteristic storage means for storing behavior characteristics that are characteristic behavior for each place of stay, Map data storage means for storing figure information, learning data generation means for generating learning data for recognizing the user's action based on the action data, action characteristics, and map data, and the generated learning data A learning data generation method in a learning data generation system comprising learning data storage means for storing and action recognition model generation means for generating an action recognition model based on the stored learning data, wherein the learning data generation A first step of acquiring the behavior data, behavior characteristics, and map data; a second step of classifying the acquired behavior data for each stay location; and staying for each stay location from the acquired map data A third step of identifying a place attribute and obtaining a characteristic behavior from the acquired behavior characteristics, and for each user for each stay point A fourth step of obtaining from the action data acquired the action recognized most frequently, and the characteristic action and the action recognized most frequently for each staying point for each user. In comparison, if there is no match, the fifth step of correcting the most frequently recognized action label with the characteristic action label, and action data including the label corrected for each user And a sixth step of generating learning data obtained by extracting the user ID, action label, and feature amount from the learning data generation method.

  (11) The present invention includes each user's mobile terminal and a server, and each user's mobile terminal is based on the action recognition model input from the server and either the motion data or the sound data. Action recognition means for recognizing action, operation determination means for determining whether or not the user is operating a portable terminal, position information detection means for detecting position information of the user, and recognition by the action recognition means Action data generating means for generating action data of the user based on action recognition data, the operation status of the mobile terminal, and the position information, and each server generated by the action data generating means Behavior data storage means for storing the behavior data, behavior characteristic storage means for storing behavior characteristics that are characteristic behavior for each place of stay, Map data storage means for storing figure information, learning data generation means for generating learning data for recognizing the user's action based on the action data, action characteristics, and map data, and the generated learning data A learning data generation method in a learning data generation system comprising learning data storage means for storing and action recognition model generation means for generating an action recognition model based on the stored learning data, wherein the learning data generation A means for acquiring behavior data from the behavior data storage means, a second step for classifying the acquired behavior data for each staying point, and for each user for each staying point, The third step to obtain from the action data acquired the action recognized the most frequently, and the highest frequency for each stay point A fourth step for determining the majority of the actions most favored by the user as perceived actions together with the vote rate, and for each staying point, if the vote rate exceeds a preset threshold, A fifth step of correcting the action label with a majority action label, and sixth learning data for extracting the user ID, action label, and feature amount from the action data including the corrected label for each user And a learning data generation method characterized by comprising steps.

  (12) The present invention includes each user's mobile terminal and a server, and each user's mobile terminal is based on the action recognition model input from the server and either the motion data or the sound data. Action recognition means for recognizing action, operation determination means for determining whether or not the user is operating a portable terminal, position information detection means for detecting position information of the user, and recognition by the action recognition means Action data generating means for generating action data of the user based on action recognition data, the operation status of the mobile terminal, and the position information, and each server generated by the action data generating means Behavior data storage means for storing the behavior data, behavior characteristic storage means for storing behavior characteristics that are characteristic behavior for each place of stay, Map data storage means for storing figure information, learning data generation means for generating learning data for recognizing the user's action based on the action data, action characteristics, and map data, and the generated learning data A program for causing a computer to execute a learning data generation method in a learning data generation system comprising: learning data storage means for storing; and action recognition model generation means for generating an action recognition model based on the stored learning data The learning data generating means includes a first step of acquiring the behavior data, behavior characteristics, and map data, a second step of classifying the acquired behavior data for each staying point, and the acquired map. A third place where a stay place attribute is specified for each stay place from the data, and a characteristic action is obtained from the acquired action characteristic And a fourth step for obtaining the behavior recognized most frequently for each user for each staying point, and the characteristic behavior for each staying point for each user. And the highest-recognized action, and if it does not match, the fifth step of correcting the highest-recognized action label with the characteristic action label, A program for causing a computer to execute a sixth step of generating learning data in which a user ID, a behavior label, and a feature amount are extracted from behavior data including a corrected label for each user is proposed.

  (13) The present invention includes each user's mobile terminal and a server, and each user's mobile terminal is based on an action recognition model input from the server and either the motion data or the sound data. Action recognition means for recognizing action, operation determination means for determining whether or not the user is operating a portable terminal, position information detection means for detecting position information of the user, and recognition by the action recognition means Action data generating means for generating action data of the user based on action recognition data, the operation status of the mobile terminal, and the position information, and each server generated by the action data generating means Behavior data storage means for storing the behavior data, behavior characteristic storage means for storing behavior characteristics that are characteristic behavior for each place of stay, Map data storage means for storing figure information, learning data generation means for generating learning data for recognizing the user's action based on the action data, action characteristics, and map data, and the generated learning data A program for causing a computer to execute a learning data generation method in a learning data generation system comprising: learning data storage means for storing; and action recognition model generation means for generating an action recognition model based on the stored learning data The learning data generation means includes a first step of acquiring action data from the action data storage means, a second step of classifying the acquired action data for each staying point, and each staying point. For each user, a third step is obtained from the action data obtained for the action recognized most frequently. And, for each staying point, a fourth step for obtaining the majority action most favored by the user as the action recognized with the highest frequency together with the vote rate, and for each staying point, the vote rate is determined in advance. If it exceeds the threshold set in step 5, the fifth step of correcting the minority action label with the majority action label, and the action data including the corrected label for each user, the user ID, action label, and A program for causing a computer to execute a sixth step of generating learning data from which feature amounts have been extracted is proposed.

  According to the present invention, when applying self-training to action recognition, the action identification accuracy is improved by correcting the wrong action label based on the characteristic action for each stay place attribute, and the reliability is improved. Generate high labeled data. Therefore, there is an effect that it is possible to improve the action recognition accuracy of the classifier even when the recognition accuracy of the classifier with respect to the user's action is low in the initial stage.

It is a figure showing the schematic structure of the learning data generation system concerning a 1st embodiment of the present invention. It is a figure which shows the structure of the learning data generation system which concerns on the 1st Embodiment of this invention. It is the figure which illustrated the data stored in the action data storage part which concerns on the 1st Embodiment of this invention. It is a processing flow figure of the learning data generation part which concerns on the 1st Embodiment of this invention. It is the figure which illustrated the data stored in the action data storage part which concerns on the 1st Embodiment of this invention. It is a figure which shows the characteristic action (behavior characteristic) which can generate | occur | produce for every stay place stored in the action characteristic memory | storage part which concerns on the 1st Embodiment of this invention. It is the figure which illustrated learning data obtained after correction concerning a 1st embodiment of the present invention. It is a figure which shows the structure of the learning data generation system which concerns on the 2nd Embodiment of this invention. It is a processing flow figure of the learning data generation part which concerns on the 2nd Embodiment of this invention. It is the figure which illustrated the data stored in the action data storage part which concerns on the 2nd Embodiment of this invention. It is the figure which illustrated the data stored in the action data storage part which concerns on the 2nd Embodiment of this invention. It is the figure which illustrated the data stored in the action data storage part which concerns on the 2nd Embodiment of this invention. It is the figure which illustrated the data stored in the action data storage part which concerns on the 2nd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Note that the constituent elements in the present embodiment can be appropriately replaced with existing constituent elements and the like, and various variations including combinations with other existing constituent elements are possible. Therefore, the description of the present embodiment does not limit the contents of the invention described in the claims.

<First Embodiment>
A learning data generation system according to the present embodiment will be described with reference to FIGS.

<Outline of learning data generation system>
As shown in FIG. 1, the learning data generation system according to the present embodiment uses highly reliable labeled data (learning) using behavior characteristics and map data representing a person's characteristic behavior for each stay location attribute. Data), and includes mobile terminals 100 to 300 and a server 500. In the learning data generation system according to the present embodiment, portable terminals 100 to 300 possessed by a user are connected to a server 500 serving as a learning data generation device via a network.

<Configuration of learning data generation system>
As illustrated in FIG. 2, the learning data generation system according to the present embodiment includes mobile terminals 100 to 300 and a server 500, and the mobile terminal 100 (the mobile terminals 200 and 300 have the same configuration) The recognition model update unit 101, the action recognition unit 102, and the motion sensor 103, which are components of the acceleration sensor 103A that senses the acceleration of the mobile terminal 100, the tilt sensor 103B that senses the tilt of the mobile terminal 100, and the voice information. It comprises a microphone 104 to be acquired, an application operation monitoring unit 105, a GPS receiving unit 106, a GPS positioning unit 107, and a behavior data generating unit 108. The server 500 includes a behavior data storage unit 501, a behavior characteristic data storage unit 502, a map data storage unit 503, a learning data generation unit 504, a learning data storage unit 505, and a behavior recognition model generation unit 506. ing.

  The behavior recognition model update unit 101 updates the behavior recognition model in a storage device (not shown) with the behavior recognition model generated by the behavior recognition model generation unit 506 in the server 500, and updates the behavior recognition model to the behavior recognition unit 102. Output. The behavior recognition model update unit 101 updates the model during a time period when the user does not use the behavior recognition unit 102, for example, during sleep. In addition, in a situation where the collected learning data is small, model updating does not make much sense, and in such a case, updating is performed not only on a daily basis but also on a weekly basis. The behavior recognition unit 102 inputs the motion data from the motion sensor 103, the sound data from the microphone 104, and the behavior recognition model from the behavior recognition model update unit 101. Then, the feature amount of the motion data from the motion sensor 103 and the sound data from the microphone 104 is extracted, and the user behavior is recognized in increments based on the inputted behavior recognition model.

  The application operation monitoring unit 105 determines whether the user is operating the mobile terminal from the operation status of the application in the mobile terminal 100. The GPS receiver 106 receives GPS signals from hygiene. The GPS positioning unit 107 acquires position information represented by latitude and longitude from the GPS signal received by the GPS receiving unit 106. The behavior data generation unit 108 generates and generates user behavior data based on the information input from the application operation monitoring unit 105, the position information input from the GPS positioning unit 107, and the information input from the behavior recognition unit 102. The action data is transmitted to the server 500 via the network. Similarly, the mobile terminals 200 and 300 transmit the action data generated in the mobile terminals 200 and 300 to the server 500 via the network.

  Specifically, the action recognition unit 102 recognizes “walking, resting, reading, eating, watching video”. Here, basic behaviors such as “walking and resting” can be recognized with sufficiently high accuracy, but applied behaviors such as “reading, eating, watching videos” may cause recognition errors. . Therefore, for example, even if the action recognition unit 102 recognizes the user action as “reading”, there is a possibility that the action is actually “meal” or “video viewing”. Therefore, the behavior recognition result is regarded as a behavior label for the feature amount, and the behavior data generation unit 108 generates behavior data including the behavior label, the feature amount, and the position information, and stores the behavior data in a behavior data storage in the server 500 described later. Stored in the section 501. However, it is assumed that the action data storage unit 501 saves only in the case of “reading, eating, watching video” in which the action recognition result may be incorrect. According to References 1 and 2, etc., the action recognition model is a machine such as a decision tree or a support vector machine using learning data formed from motion data, feature values of sound data and correct data of user actions. It can be generated by a learning algorithm.

<Reference 1>
Stefan Dembach, et al. , '' Simple and Complex Activity Recognition Through Smart Phone '', 11 th International Conference on Intelligent Enviroments, 2012.

<Reference 2>
Koji Yatani and Khai N .; Truong, '' BodyScope: A Wearable Acoustic Sensor for Activity Recognition, 14 th ACM International Conference on Ubiquitous Computing, 2012.

  The behavior data storage unit 501 in the server 500 stores behavior data including behavior labels, feature amounts, and position information generated by the behavior data generation unit 108 in each mobile terminal 100 to 300 for each mobile terminal 100 to 300. To do. The behavior characteristic data storage unit 502 stores behavior characteristics that are characteristic behaviors for each stay location attribute. For example, actions such as “reading”, “meal”, and “viewing video” are characteristic actions that occur frequently depending on the stay location attribute. As shown in FIG. 6, “reading” is a characteristic action for “bookstores”, “meal” is for “restaurants”, and “video viewing” is a characteristic action for “movie theaters”. The map data storage unit 503 stores map data in which a stay location attribute is assigned to a section divided in units of meshes. If map data is used, a stay place attribute can be specified from the overlap of a user's stay point and a division.

  The learning data generation unit 504 corrects an erroneous action label in the action data using action data, action characteristics, and map data, and generates labeled data (learning data) with high reliability. Specifically, the following processing is performed for each user action data. First, the behavior data is divided for each staying point. Next, a stay place attribute is specified for each stay point using map data, and a characteristic action is determined from the action characteristic. Moreover, the action recognized with the highest frequency is calculated for each staying point from the action data. Then, the characteristic action and the action recognized with the highest frequency are compared for each staying point, and if they do not match, the label of the action recognized with the highest frequency is corrected with the label of the characteristic action. If they match, no processing is performed. Finally, the action label and the feature amount corresponding to the action label are stored in the learning data storage unit 505 as learning data. The behavior recognition model generation unit 506 generates a behavior recognition model from the data in the learning data storage unit 505 using the signal received from the learning data generation unit 504 as a trigger. The behavior recognition model is generated during a time period when the user does not use the behavior recognition unit 102, for example, a time period during sleep. In addition, in a situation where the collected learning data is small, model updating does not make much sense, and in such a case, updating is performed not only on a daily basis but also on a weekly basis.

<Processing of learning data generation system>
Processing of the learning data generation system according to the present embodiment will be described with reference to FIGS.

  The behavior data storage unit 501 in the server 500 stores behavior data received from each portable terminal 100 to 300 for each portable terminal 100 to 300. FIG. 3 illustrates behavior data. As shown in FIG. 3, the behavior data is composed of a user ID, date and time, behavior label, position information, and feature amount. For example, there is data such as reading at a position (x0, y0) at the time of 2014/12/1 11:30, and meal at a position (x1, y1) at the time of 2014/12/1 12:00:00. Here, according to Reference 3 or the like, it is possible to specify a staying place from the position information history when the staying time near a certain point exceeds a certain threshold. However, here, for simplification, it is considered that the user is staying at the same point only in the case of the same position information. In the behavior data shown in FIG. 3, it can be seen that the user stays at the same point (x1, y1) from 2014/12/1 12:00:00 to 2014/12/1 12:09:00.

<Reference 3>
Nishino et al. Examination of action pattern extraction method from staying area transition information; Ubiquitous Computing System, 2008 (110) pp. 57-64

  4, the learning data generation unit 504 first acquires behavior data, behavior characteristics, and map data from the behavior data storage unit 501, the behavior characteristic data storage unit 502, and the map data storage unit 503 (step S101). .

  The learning data generation unit 504 divides the behavior data shown in FIG. 5 for each staying point (step S102). The learning data generation unit 504 specifies a stay location attribute for each stay location from the acquired map data, and obtains a characteristic behavior a from the acquired behavior characteristics (step S103).

  The learning data generation unit 504 obtains for each user from the action data obtained the action a 'recognized most frequently for each staying point (step S104). The learning data generation unit 504 compares a and a 'for each staying point for each user, and corrects the a' label with the a label if they do not match (step S105).

  The learning data generation unit 504 generates labeled data (learning data) obtained by extracting the user ID, the behavior label, and the feature amount from the behavior data including the corrected label for each user (step S106), and the generated labeled data is added. Data (learning data) is stored in the learning data storage unit 505 (step S107).

Specifically, using FIG. 5, the action data of the stay point (x1, y1) of the user ID (: 1) is d1 _{(x1, y1)} , and the action data of the stay point (x2, y2) is d1 _{( x2, y2),} and f (d1 _{(x1, y1))} if the function for _obtaining the action label generated at the highest frequency from the action data di _{(xj, yj)} is f (di _{(xj, yj) ).} ) = Video viewing, f (d1 _{(x2, y2)} ) = meal. Here, if the stay place attribute of (x1, y1) and (x2, y2) is both “restaurant” from the map data, the characteristic behavior of “restaurant” is “meal” from the behavior characteristics. It becomes. Therefore, only for (x1, y1), “viewing video” is corrected to “meal”, and learning data including the corrected action label is generated. The learning data generated at this time is as shown in FIG.

  As described above, according to the present embodiment, labeled data with high reliability can be obtained by improving the action identification accuracy by correcting the wrong action label based on the characteristic action for each stay location attribute. (Learning data) is generated. Therefore, even when the recognition accuracy of the classifier for the user's behavior is low in the initial stage, it is possible to improve the behavior recognition accuracy of the classifier.

<Second Embodiment>
The learning data generation system according to the present embodiment will be described with reference to FIGS. In the present embodiment, highly reliable labeled data (learning data) is generated without using the data of the behavior characteristic storage unit and the map data storage unit provided in the first embodiment.

  In 1st Embodiment, it is necessary to grasp | ascertain a person's action characteristic for every stay place attribute, and the load with respect to an implementer is high also at the point of maintenance of map information. Therefore, this embodiment proposes a method that does not use human behavior characteristics and map information. In the present embodiment, the error label is corrected by specifying the characteristic behavior of the staying point based on the recognition result of the discriminator of other users as well as the user. In general, when self-training is applied to user behavior recognition, while there are users with low recognition accuracy of the classifier, there are also users with high recognition accuracy. Then, if the recognition result with respect to a user's action with high recognition accuracy is followed, the characteristic action of a staying point can be specified. That is, the action recognition results generated at the highest frequency of the user at a certain stay point are aggregated by majority vote, and the action recognition results belonging to the majority are identified as characteristic actions. Then, the label of the action recognition result belonging to the minority is corrected with the characteristic action label. Here, it is considered that there is a high possibility that the label can be correctly corrected only when the vote rate for the majority is equal to or higher than the set threshold value, and it becomes possible to correct an incorrect label.

<Configuration of learning data generation system>
As illustrated in FIG. 8, the learning data generation system according to the present embodiment includes mobile terminals 100 to 300 and a server 500, and the mobile terminal 100 (the mobile terminals 200 and 300 have the same configuration) The recognition model update unit 101, the action recognition unit 102, and the motion sensor 103, which are components of the acceleration sensor 103A that senses the acceleration of the mobile terminal 100, the tilt sensor 103B that senses the tilt of the mobile terminal 100, and the voice information. It comprises a microphone 104 to be acquired, an application operation monitoring unit 105, a GPS receiving unit 106, a GPS positioning unit 107, and a behavior data generating unit 108. The server 500 includes a behavior data storage unit 501, a learning data storage unit 505, a behavior recognition model generation unit 506, and a learning data generation unit 510. In addition, about the component which attaches | subjects the same code | symbol as 1st Embodiment, since it has the same function, the detailed description is abbreviate | omitted.

  The learning data generation unit 510 acquires behavior data associated with each user's stay point stored in the behavior data storage unit 501 and classifies this behavior data for each stay point. Next, the action label (majority action label) recognized with the highest frequency is obtained for the most users for each staying point. Then, the minority action label is corrected with the majority action label, and the generated action label and the corresponding feature amount are stored in the learning data storage unit 505 as learning data.

<Processing of learning data generation system>
The process of the learning data generation system according to the present embodiment will be described with reference to FIGS.

  As illustrated in FIG. 9, the learning data generation unit 510 first acquires behavior data from the behavior data storage unit 501 (step S201).

  The learning data generation unit 510 divides the behavior data shown in FIGS. 5 and 10 to 13 for each staying point (step S202). The learning data generation unit 510 obtains the action recognized at the highest frequency for each user from the action data obtained for each staying point (step S203).

  The learning data generation unit 510 obtains the majority of the actions most favored by the user as the action recognized with the highest frequency for each staying point, together with the vote rate (step S204). For each staying point, the learning data generation unit 510 corrects the minority action label with the majority action label if the voting rate exceeds a preset threshold (step S205).

  The learning data generation unit 510 generates labeled data (learning data) obtained by extracting the user ID, the action label, and the feature amount from the action data including the corrected label for each user (step S206), and the generated labeled data is added. Data (learning data) is stored in the learning data storage unit 505 (step S207).

In the following, the action data at the stay point (x _j , y _j ) of the user ID = i is assumed to be di _(xj , y _j ), and the entire action data at the stay point (x _j , y _j ) is D _{(xj, yj). )} = {D1 _{(xj, yj)} ,..., Dn _{(xj, yj)} } (n: number of users, user ID = action data at a stay point (x _j , y _j ) at _i ) If it does not exist, di _{(xj, yj)} = 0), specifically, the processing of the learning data generation unit 510 will be described.

The learning data generation unit 510 divides the acquired behavior data for each staying point, so that D = {D _{(x1, y1)} , ..., D _{(xm, ym)} } (m *: the number of staying points). obtain. Next, the label of the action recognized with the highest frequency for the most users is obtained for each staying point. Here, the function for _obtaining the action label that occurs with the highest frequency from the action data di _{(xj, yj)} is f (di _{(xj, yj)} ), and g (l, di _{( xj, yj)} ) are defined as follows.

g (l, di _{(xj, yj)} ) = ifl and f (di _{(xj, yj)} ) match then lotherwise 0
At this time, the action label of the majority at the stay point (x _j , y _j ) is an action label l ′ that maximizes the following formula for calculating the number of votes (Equation 1).

  The vote rate is expressed by the following formula 2.

Then, the minority action label is corrected with the majority action label according to the vote result. _{That, f (di (xj, yj} )) ≠ l ' is a di _{(xj, yj) against, f (di (xj, yj} )) the behavior label represented by l' is corrected in.

Also, the labeled data d′ i _{(xj, yj)} is generated by extracting only the user ID, the action label, and the feature quantity necessary as the labeled data from the action data including only the corrected action label (f (di _{If (xj, yj)} ) = l ″, then d′ i _{(xj, yj)} ≠ 0).

By repeating the vote and the label correction for each staying point, the labeled data D′ i = {d′ i _{(xj, yj)} ,..., D′ i _{(xm, ym)} } ( When the user ID = i), the labeled data (learning data) is stored in the learning data storage unit 505. At this time, the reliability of the majority's action label can be given superiority or inferiority according to the vote rate for the majority at the staying point. Therefore, it is possible to obtain labeled data with higher reliability by adopting it as labeled data only when the vote rate exceeds a preset threshold value.

<Example>
Hereinafter, examples according to the present embodiment will be described with reference to FIGS. 5 and 10 to 13.
From FIG. 5, FIG. 10 to FIG.
D _{(x1, y1)} = {d1 _{(x1, y1)} , d2 _{(x1, y1)} , d3 _{(x1, y1)} , d4 _{(x1, y1)} , d5 _{(x1, y1)} }, f (d1 _{(x1, y1,) y1)} ) = viewing video, f (d2 _{(x1, y1)} ) = meal, f (d3 _{(x1, y1)} ) = meal, f (d4 _{(x1, y1)} ) = meal, f (d5 _{(x1, y1) )} ) = Meal,
D _{(x2, y2)} = {d1 _{(x2, y2)} , d2 _{(x2, y2)} , d3 _{(x2, y2)} , d4 _{(x2, y2)} } (where d5 _{(x2, y2)} ≠ 0), f (d1 _{(xj, yj)} ) = meal, f (d2 _{(xj, yj)} ) = video viewing, f (d3 _{(xj, yj)} ) = reading, f (d4 _{(xj, yj)} ) = reading .

At this time, at the stay point (x ₁ , y ₁ ), when the action label is “meal”, the number of votes is the maximum in Formula 3, and the vote rate is Formula 4.

Therefore, the action label of the majority at the stay point (x ₁ , y ₁ ) is “meal”. Similarly, at the stay point (x ₂ , y ₂ ), when the action label is “Reading”, the number of votes is the maximum in Formula 5, and the vote rate is Formula 6.

Therefore, the action label of the majority at the stay point (x ₂ , y ₂ ) is “reading”. Therefore, at the stay point (x ₁ , y ₁ ), the action label “video viewing” of the minority user ID = 1 is corrected with the action label “meal” of the majority, and learning data including the corrected action label is generated. To do. Also, at the stay point (x ₂ , y ₂ ), the minority user ID = 1 action label “meal” and the user ID = 2 action label “video viewing” are corrected with the majority action label “reading”. The learning data including the action label corrected for each user is generated.

At this time, for example, if a threshold value of 0.8 is given to the vote rate, at the stay point (x ₁ , y ₁ ), the label is corrected according to the majority action label, but the stay point (x ₂ , y 1) _{In 2} ), it is possible not to correct the label. Thereby, it is possible to obtain only labeled data with higher reliability.

  As described above, according to the present embodiment, highly reliable labeled data (learning data) is generated by correcting an erroneous action label based on a characteristic action for each stay location attribute. . Therefore, even when the recognition accuracy of the classifier for the user's behavior is low in the initial stage, it is possible to improve the behavior recognition accuracy of the classifier.

  The processing of the portable terminal and server constituting the learning data generation system is recorded on a recording medium readable by the computer system, and the program recorded on the recording medium is read into the portable terminal and server constituting the learning data generation system. The learning data generation system according to the present invention can be realized by executing. The computer system here includes an OS and hardware such as peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW (World Wide Web) system is used. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

  The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  The embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to the embodiments, and includes designs and the like that do not depart from the gist of the present invention. For example, in the first embodiment and the second embodiment, it is desirable to use action data for a sufficient period in order to select the stay location attribute with high accuracy. If one week of behavior data is used, it is possible to improve the behavior model repeatedly by continuing to generate learning data from the behavior data of the most recent one week.

  In the present embodiment, the system that performs functional division between the mobile terminal and the server has been described. However, the mobile terminal includes a behavior data storage unit 501, a behavior characteristic data storage unit 502, a map data storage unit 503, and learning data generation. Unit 504, learning data storage unit 505, behavior recognition model generation unit 506, and learning data generation unit 510, receive behavior data from other users, and store behavior data of all users in behavior data storage unit 501. By storing, learning data may be generated by each user's mobile terminal alone.

DESCRIPTION OF SYMBOLS 100; Portable terminal 101; Action recognition model update part 102; Action recognition part 103; Motion sensor 103A; Acceleration sensor 103B; Tilt sensor 104; Microphone 105; Application operation monitoring part 106; GPS reception part 107; Data generation unit 200; mobile terminal 300; mobile terminal 500; server 501; behavior data storage unit 502; behavior characteristic data storage unit 503; map data storage unit 504; learning data generation unit 505; learning data storage unit 506; Generation unit 510; learning data generation unit

Claims (10)

A learning data generation system comprising a mobile terminal and a server for each user,
The mobile terminal of each user is
Action recognition means for recognizing the action of the user based on the action recognition model input from the server and either motion data or sound data;
Operation determining means for determining whether or not the user is operating the mobile terminal;
Position information detecting means for detecting the position information of the user;
Action data generating means for generating action data of the user based on action recognition data recognized by the action recognition means, an operation status of the mobile terminal, and the position information;
With
The server is
Action data storage means for storing action data of each user generated by the action data generation means;
Behavior characteristic storage means for storing behavior characteristics that are characteristic behavior for each place of stay;
Map data storage means for storing map information;
Learning data generating means for generating learning data for recognizing the user's behavior based on the behavior data, behavior characteristics, and map data;
Learning data storage means for storing the generated learning data;
Action recognition model generation means for generating an action recognition model based on the stored learning data;
Equipped with a,
The learning data generating means specifies a stay location attribute from the stay location calculated from the location information and the map data, and obtains a characteristic behavior at the stay location from the stay location attribute and the behavior characteristics,
The behavior data is composed of at least a user ID, date and time, behavior label, position information, and feature quantity, and the learning data generation means compares the characteristic behavior with a recognition result by the behavior recognition means, A learning data generation system that generates learning data by correcting the action label corresponding to the recognition result with the action label indicating the characteristic action when they do not match . A learning data generation system comprising a mobile terminal and a server for each user,
The mobile terminal of each user is
Action recognition means for recognizing the action of the user based on the action recognition model input from the server and either motion data or sound data;
Operation determining means for determining whether or not the user is operating the mobile terminal;
Position information detecting means for detecting the position information of the user;
Action data generating means for generating action data of the user based on action recognition data recognized by the action recognition means, an operation status of the mobile terminal, and the position information;
With
The server is
Action data storage means for storing action data of each user generated by the action data generation means;
Learning data generating means for generating learning data for recognizing the user's behavior based on the behavior data;
Learning data storage means for storing the generated learning data;
Action recognition model generation means for generating an action recognition model based on the stored learning data;
With
The behavior data is composed of at least a user ID, date and time, behavior label, location information, and feature quantity, and the learning data generation means is configured to determine the majority of the behavior labels recognized by all users at the same staying point. A learning data system characterized in that the action label of the majority who has determined and obtained the most votes is regarded as a characteristic action at a staying point . The learning data generation system according to claim 2 , wherein the learning data generation unit calculates the number of votes obtained by majority vote from the equation (1).

4. The learning data according to claim 3 , wherein the learning data generation unit corrects an action label that is a minority in the majority at the same staying point with the action label of the majority, and generates learning data. 5. Generation system. The learning data generating means, only when the majority of the vote rate is equal to or greater than a threshold set in advance, by correcting the behavior label minority in claim 4, characterized in that to generate the training data The learning data generation system described. The learning data generation system according to claim 5 , wherein the learning data generation unit calculates the vote rate of majority vote from Equation 2.

Action recognition means comprising a mobile terminal of each user and a server, wherein each user's mobile terminal recognizes the action of the user based on an action recognition model input from the server and either motion data or sound data Operation determining means for determining whether or not the user is operating a portable terminal, position information detecting means for detecting position information of the user, action recognition data recognized by the action recognition means, and the portable terminal Behavior data generating means for generating behavior data of the user based on the operation status and the position information, and the server stores the behavior data of each user generated by the behavior data generating means Data storage means, behavior characteristic storage means for storing behavior characteristics that are characteristic behavior for each place of stay, and map information are stored Figure data storage means, learning data generation means for generating learning data for recognizing the user's action based on the action data, action characteristics, and map data, and learning data storage for storing the generated learning data A learning data generation method in a learning data generation system comprising: means; and behavior recognition model generation means for generating a behavior recognition model based on the stored learning data,
The learning data generating means
A first step of acquiring the behavior data, behavior characteristics, and map data;
A second step of classifying the acquired behavior data for each staying point;
A third step of identifying a stay location attribute for each stay location from the acquired map data, and obtaining a characteristic behavior from the acquired behavior characteristics;
For each user for each staying point, a fourth step for obtaining from the action data obtained for the action recognized with the highest frequency;
For each user, the characteristic behavior is compared with the highest-recognized behavior for each staying point, and if they do not match, the highest-recognized behavior label is displayed as the feature. A fifth step to correct with a typical action label;
A sixth step of generating learning data obtained by extracting a user ID, a behavior label, and a feature amount from behavior data including a corrected label for each user;
A learning data generation method characterized by comprising: Action recognition means comprising a mobile terminal of each user and a server, wherein each user's mobile terminal recognizes the action of the user based on an action recognition model input from the server and either motion data or sound data Operation determining means for determining whether or not the user is operating a portable terminal, position information detecting means for detecting position information of the user, action recognition data recognized by the action recognition means, and the portable terminal Behavior data generating means for generating behavior data of the user based on the operation status and the position information, and the server stores the behavior data of each user generated by the behavior data generating means Data storage means, behavior characteristic storage means for storing behavior characteristics that are characteristic behavior for each place of stay, and map information are stored Figure data storage means, learning data generation means for generating learning data for recognizing the user's action based on the action data, action characteristics, and map data, and learning data storage for storing the generated learning data A learning data generation method in a learning data generation system comprising: means; and behavior recognition model generation means for generating a behavior recognition model based on the stored learning data,
The learning data generating means
A first step of obtaining behavior data from the behavior data storage means;
A second step of classifying the acquired behavior data for each staying point;
A third step for obtaining from each of the staying points, from the action data obtained the action most frequently recognized for each user;
A fourth step for determining, together with the vote rate, the majority behavior most favored by the user as the most frequently recognized behavior for each stay point;
For each staying point, if the turnout exceeds a preset threshold, a fifth step of correcting the minority action label with the majority action label;
A sixth step of generating learning data obtained by extracting a user ID, a behavior label, and a feature amount from behavior data including a corrected label for each user;
A learning data generation method characterized by comprising: Action recognition means comprising a mobile terminal of each user and a server, wherein each user's mobile terminal recognizes the action of the user based on an action recognition model input from the server and either motion data or sound data Operation determining means for determining whether or not the user is operating a portable terminal, position information detecting means for detecting position information of the user, action recognition data recognized by the action recognition means, and the portable terminal Behavior data generating means for generating behavior data of the user based on the operation status and the position information, and the server stores the behavior data of each user generated by the behavior data generating means Data storage means, behavior characteristic storage means for storing behavior characteristics that are characteristic behavior for each place of stay, and map information are stored Figure data storage means, learning data generation means for generating learning data for recognizing the user's action based on the action data, action characteristics, and map data, and learning data storage for storing the generated learning data A program for causing a computer to execute a learning data generation method in a learning data generation system comprising: means and an action recognition model generation means for generating an action recognition model based on the stored learning data,
The learning data generating means
A first step of acquiring the behavior data, behavior characteristics, and map data;
A second step of classifying the acquired behavior data for each staying point;
A third step of identifying a stay location attribute for each stay location from the acquired map data, and obtaining a characteristic behavior from the acquired behavior characteristics;
For each user for each staying point, a fourth step for obtaining from the action data obtained for the action recognized with the highest frequency;
For each user, the characteristic behavior is compared with the highest-recognized behavior for each staying point, and if they do not match, the highest-recognized behavior label is displayed as the feature. A fifth step to correct with a typical action label;
A sixth step of generating learning data obtained by extracting a user ID, a behavior label, and a feature amount from behavior data including a corrected label for each user;
A program that causes a computer to execute. Action recognition means comprising a mobile terminal of each user and a server, wherein each user's mobile terminal recognizes the action of the user based on an action recognition model input from the server and either motion data or sound data Operation determining means for determining whether or not the user is operating a portable terminal, position information detecting means for detecting position information of the user, action recognition data recognized by the action recognition means, and the portable terminal Behavior data generating means for generating behavior data of the user based on the operation status and the position information, and the server stores the behavior data of each user generated by the behavior data generating means Data storage means, behavior characteristic storage means for storing behavior characteristics that are characteristic behavior for each place of stay, and map information are stored Figure data storage means, learning data generation means for generating learning data for recognizing the user's action based on the action data, action characteristics, and map data, and learning data storage for storing the generated learning data A program for causing a computer to execute a learning data generation method in a learning data generation system comprising: means and an action recognition model generation means for generating an action recognition model based on the stored learning data,
The learning data generating means
A first step of obtaining behavior data from the behavior data storage means;
A second step of classifying the acquired behavior data for each staying point;
A third step for obtaining from each of the staying points, from the action data obtained the action most frequently recognized for each user;
A fourth step for determining, together with the vote rate, the majority behavior most favored by the user as the most frequently recognized behavior for each stay point;
For each staying point, if the turnout exceeds a preset threshold, a fifth step of correcting the minority action label with the majority action label;
A sixth step of generating learning data obtained by extracting a user ID, a behavior label, and a feature amount from behavior data including a corrected label for each user;
A program that causes a computer to execute.

Images