JP2020170382A - Estimation processing device and estimation model construction device - Google Patents

Abstract

To provide an estimation processing device and estimation model construction device, capable of collecting and determining the information which can measure an attention function passively.SOLUTION: An estimation model execution device 100 includes an attention function estimation section 103 for estimating user's attention function using feature quantity generated from a sensor value of each sensor (a motion sensor) of a communication terminal 200 which is an information processor held by a user. The feature quantity is generated in the communication terminal 200 or the estimation model execution device 100.SELECTED DRAWING: Figure 1

Description

The present invention relates to an estimation processing device and an estimation model construction device for estimating a user's attention function.

Decreased attention function has become an international problem because it leads to a decrease in labor productivity. Therefore, it is considered to quantitatively measure the attention function of a person. For example, it is considered to carry out cognitive function tests such as PVT and Go-NoGo, but it takes several minutes to carry out this test, and multiple times a day to capture changes over time. It was necessary to carry out the test. Therefore, the burden on the user is large. Therefore, a system that automatically determines the level of attention function is useful.

Non-Patent Document 1 describes that a human attention function (response speed of PVT test) is estimated by a machine learning model based on features such as the number of times the screen is lit, the lighting time, and the sleeping time of a smartphone. ..

Cognitive Rhythms: Unobtrusive and Continuous Sensing of AlertnessUsing a Mobile PhoneSaeedAbdullah, Elizabeth Murnane, Mark Matthews, Matthew Kay, Julie Kientz, GeriGay, Tanzeem Choudhury, Proceedings of the 2016 ACM International JointConference on Pervasive and Ubiquitous Computing. Searched on February 6, 2014] <http://pac.cs.cornell.edu/pubs/Ubicomp2016_CognitiveRhythms.pdf>

However, the features related to how to use the smartphone, such as lighting the screen, cannot be obtained without using the smartphone, so the features are not available during work in the actual environment or when the task is concentrated (the situation where the smartphone cannot be used). It is assumed that it cannot be obtained.

Therefore, the present invention is an estimation processing device and an estimation model construction device capable of collecting and judging information capable of passively measuring the attention function even when the user is not using a terminal such as a smartphone. The purpose is to provide.

In order to solve the above-mentioned problems, the estimation processing device of the present invention estimates the user's attention function using the feature amount generated based on the sensor value of the motion sensor of the information processing device possessed by the user. It is equipped with an attention function estimation unit.

According to the present invention, the attention function of the user can be estimated from the movement of the communication terminal possessed by the user without detecting the operation of the user. Since no user operation is required, the attention function can be estimated without making the user aware of it. Note that "the user possesses the communication terminal" is not limited to the mode in which the user holds the communication terminal by holding or wearing the communication terminal, and the communication terminal can be arbitrarily selected by the user. It includes a state in which it can be operated, for example, when it is placed in a predetermined place within the reach of a user.

According to the present invention, the attention function of the user can be estimated from the movement of the communication terminal possessed by the user without detecting the operation of the user.

It is a block diagram which shows the functional structure of the estimation model execution apparatus 100 and the communication terminal 200 of this embodiment. It is a schematic diagram which shows the generation process at the time of generating the attention function score estimation model. It is a table in which the date, the attention function score, and the feature amount such as the sensor value log of each sensor value are associated with each user ID. It is a figure which shows the sensor value table of each sensor value collected by the communication terminal log collecting unit 101. It is a figure which shows the sensor value table in another example. A specific example of the screen on / off history table and the earphone connection history table is shown. Specific examples of the positioning history table and the retention point table are shown. A specific example of the application activation history table and the application information table 101b is shown. It is a flowchart which shows the construction process of attention function score estimation model. It is a flowchart in which the estimation model execution device 100 performs the estimation process of the user's attention function using the estimation model. It is a flowchart which shows the process of calculating a feature amount from an application start log. It is a figure which shows the transition of the screen history for showing an example of the terminal log which shows the operation from the screen on of a communication terminal 200 to start of an application. This is a specific example of an operation log table that stores operation logs. It is a flowchart which shows the process when the feature amount is calculated from an operation log. It is a figure which shows the example of the management table for generating an exclusion list and the exclusion list. It is a figure which shows an example of the hardware composition of the estimation model execution apparatus 100 and the communication terminal 200 which concerns on one Embodiment of this disclosure.

An embodiment of the present invention will be described with reference to the accompanying drawings. When possible, the same parts are designated by the same reference numerals and duplicate description is omitted.

FIG. 1 is a block diagram showing a functional configuration of the estimation model execution device 100 and the communication terminal 200 of the present embodiment. As shown in FIG. 1, the estimation model execution device 100 can acquire the operation log of each motion sensor provided in the communication terminal. The estimation model execution device 100 includes a communication terminal log collection unit 101 (sensor value acquisition unit, program management unit), a log table 101a (program management unit), an application information table 101b, and a feature amount calculation unit 102 (feature amount conversion unit). , Attention function estimation unit 103 (attention function estimation unit), estimation result display unit 104 (attention function presentation unit), estimation model construction unit 105 (estimation model construction unit), estimation model storage unit 106, exclusion list 107, and exclusion list. It is configured to include a generation unit 108. Further, the communication terminal 200 includes an acceleration sensor 201, an angular velocity sensor 202, a tilt sensor 203, a screen on / off sensor 204, an application use sensor 205, GPS 206, a communication unit 207, a control unit 208, and a program 209. .. Each configuration will be described below. These configurations are examples, and not all are essential configurations. For example, the exclusion list 107 and the exclusion list generation unit 108 may be omitted.

The communication terminal log collection unit 101 includes sensor values such as an acceleration sensor 201, an angular speed sensor 202, an inclination sensor 203, and a screen on / off sensor 204 provided in the communication terminal 200, and a program activation state of the application use sensor 205. This is a part for collecting terminal log information such as screen on / off and position information of the communication terminal 200. In addition, as the sensor value, the number of WIFI access points, the WIFI access rate, and the like are collected. In this case, the communication unit of the communication terminal 200 corresponds to a motion sensor. The communication terminal log collecting unit 101 can communicate with the communication terminal 200 by wire or wirelessly.

The log table 101a includes a sensor value table for storing each sensor value such as an acceleration sensor and an angular velocity sensor, a position information table, an application activation history table, and a terminal log table. The communication terminal log collecting unit 101 distributes and stores each collected sensor value and terminal log information (positioning information, application activation information, screen on / off) in each of the above tables.

The application information table 101b is a table that stores the application package name, category number, and tag in association with each other. The package name is an ID for identifying the application, and is an application ID. The category number is a number indicating the category of the application. The category number is a number indicating a category such as social / transportation / work optimization. The tag is additional information, and in the present embodiment, information indicating that the tag is a communication application such as CALL (telephone) or SMS is added.

The usage feature amount calculation unit 102 detects each sensor (acceleration sensor 201, angular velocity sensor 202, tilt sensor 203, screen on / off sensor 204, application usage sensor 205, and other communication status (WIFI access status) in the log table 101a. This is the part that calculates the feature amount based on the sensor value of the sensor and the terminal log information. For example, the feature amount is the average value of sensor values, standard deviation, maximum value, minimum value, maximum minimum difference, correlation coefficient of axis pair, total movement distance, number of screen ons, number of application startups, time zone WIFI connection on rate. , The number of observed WIFI access points. In addition, in the case of an acceleration sensor, the Pearson correlation coefficient of the value series of two of the three axes or the square root (signal power) of the sum of squares of the three axes is calculated as a feature amount. In addition, the angular velocity, inclination, and acceleration (removal of gravitational component) may be calculated in the same manner.

The feature amount calculation unit 102 can calculate the feature amount by designating the user ID and the date and acquiring the sensor value and the terminal log information for each predetermined time zone from each table. For example, the utilization feature amount calculation unit 102 uses the average value of the sensor values of the acceleration sensor every hour (every hour), the average value of the sensor values of the acceleration sensor every 6 hours, and the sensor of the acceleration sensor all day (every 24 hours). Calculate the average value of the values.

The attention function estimation unit 103 is a part that estimates the attention function of the user who owns the communication terminal 200 based on the usage feature amount and the estimation model stored in the estimation model storage unit 106. The "attention function" in the present embodiment means the user's concentration and attention, or a similar state of the user. The estimated model is a model learned in advance based on the feature quantity used and the Go-NoGo test result. The model construction process will be described later.

In the present embodiment, it is important that the attention function estimation unit 103 estimates the attention function score by inputting the sensor value into the estimation model. The sensor value in the present embodiment is a sensor value passively obtained in the communication terminal 200, and is a sensor value obtained without necessarily performing an operation by the user. By using such a sensor value, the user's attention function can be estimated even if the user does not operate the terminal.

The estimation result display unit 104 is a part that displays the attention function of the user who owns the communication terminal 200. The function is not limited to the display, and may be a function that presents the user's attention function. Further, the estimation result display unit 104 may have a function of presenting (notifying) information indicating the user's attention function to the communication terminal 200 instead of displaying the display.

The estimation model construction unit 105 uses the sensor value of the communication terminal 200 in a predetermined time unit and the feature amount based on the terminal log information (app start history, screen on / off log, position information, etc.) as explanatory variables, and sets the feature amount in the predetermined time unit. In the associative form, the communication terminal 200 constructs an estimation model by performing machine learning using the result of the Go-NoGo test obtained from the user for the communication terminal 200 as a teacher signal.

For example, the estimation model construction unit 105 uses the feature amount calculated based on the information distinguished in a predetermined time unit from the sensor value and the terminal log information (app start history, screen on / off, positioning, etc.). Obtained from the quantity calculation unit 102. On the other hand, the estimation model construction unit 105 acquires the Go-NoGo test result by the user as an index of the user's attention function. This test result may be information performed by matching the time zone with the feature amount based on the sensor value and the terminal log information in the predetermined time zone described above, but the test result is not necessarily limited to this. For example, the feature amount from 0 to 6 o'clock on the day may be associated with the attention function score performed at 12:00 on the day.

For example, the estimation model construction unit 105 obtains the feature quantity in units of 6 hours, and the user takes a Go-NoGo test every 6 hours, and the estimation model construction unit 105 acquires the result.

Then, the estimation model construction unit 105 performs learning processing using the feature amount as an explanatory variable and the Go-NoGo test result as a teacher signal. As a machine learning method, an existing method is applied. For example, SVM (Support Vector Machine), Random Forest, K-nearest neighbor method, neural network and the like can be used.

The construction process will be described with reference to FIG. FIG. 2 is a schematic diagram showing a generation process when generating an attention function score estimation model based on the sensor value of each sensor, terminal log information, and Go-NoGo test result. The input S11 indicates the sensor value of each sensor of the communication terminal 200, the application activation history, the terminal log, and the log information of the positioning history (GPS). Based on the log information such as these sensor values, the feature amount is calculated, and the feature amount shown in the output S12 is calculated.

On the other hand, the input S13 indicates that the user is taking the test in the Go-NoGo test application. The Go-NoGo test is performed by a user operating a terminal such as a communication terminal 200 and activating an application for the Go-NoGo test. The test result is the information obtained by operating the application. The output S14 is obtained by performing the scoring process based on the test result. Here, the correct answer rate, Efficiency, average value of Go reaction time, NoGo success rate, etc. of the Go-NoGo test are calculated as attention function scores. The Efficiency indicates the difference between the actually measured NoGo success rate and the assumed NoGo success rate calculated from the Go reaction speed at that time.

Then, the estimation model construction unit 105 can construct the attention function score estimation model by performing the learning process of the attention function score estimation model using the feature amount shown in the output S12 as the explanatory variable and the attention function score as the objective variable. ..

FIG. 3 shows the date, the attention function score (see output S14 in FIG. 2), the sensor value of each sensor value, and the feature amount of the terminal log information (application startup history, screen on / off, etc.) for each user ID (FIG. 2). It is a table associated with the output (see S12) of. The attention function score is indicated by one or more of the Go-NoGo accuracy rate and the like shown in S14 in FIG. For example, the Go-NoGo correct answer rate is used as this attention function score.

The estimation model construction unit 105 generates the table shown in FIG. 3 based on the feature amount calculated by the utilization feature amount calculation unit 102 and the attention function score obtained based on the Go-NoGo test result. It is possible to perform the construction process of the estimation model. The estimation model building unit 105 extracts the feature amount for each user and builds it for each user in order to generate the attention function score estimation model for each user. If the estimation model is not constructed for each user, the estimation model may be constructed using all the data.

The exclusion list generation unit 108 is a part that generates an exclusion list 107 indicating an application that is not used for attention function estimation based on the application activation history table. Details will be described later.

Next, the functional block of the communication terminal 200 will be described. The acceleration sensor 201 is a sensor that detects the acceleration given to the communication terminal 200.

The angular velocity sensor 202 is a sensor that detects an angular velocity given to the communication terminal 200.

The tilt sensor 203 is a sensor that detects the posture of the communication terminal 200 and its tilt.

The screen on / off sensor 204 is a sensor that detects whether the display unit (not shown) of the communication terminal 200 is on / off. For example, the screen on / off sensor 204 detects whether the display unit is in the sleep state and nothing is displayed or information such as the home screen is displayed.

The application usage sensor 205 is a sensor that detects which application is being used by the user.

The GPS 206 is a positioning unit for positioning the position information of the communication terminal 200.

The communication unit 207 is a part for communicating with the estimation model execution device 100 and other networks.

The control unit 208 is a part for controlling the communication terminal 200.

The program 209 is a program for the communication terminal 200 to execute a communication function and other predetermined functions.

FIG. 4 is a diagram showing a sensor value table of each sensor value collected by the communication terminal log collecting unit 101 in the estimation model execution device 100. These sensor values are information based on the information acquired by each sensor of the communication terminal 200 described above.

FIG. 4A shows a specific example of the acceleration sensor table (not shown). FIG. 4B is a table of sensor values based on the angular velocity sensor, the tilt sensor, and the acceleration sensor (after removing the gravity component).

In the present embodiment, the communication terminal 200 further includes a pressure sensor, a battery log sensor, an illuminance sensor, and the like, and the estimation model execution device 100 collects such information as sensor values of the motion sensor and puts them in the sensor value table. You may remember. Specific examples thereof are shown in FIGS. 5 (a) to 5 (c).

FIG. 6 shows a specific example of the screen on / off history table (FIG. 6 (a)) and the earphone connection history table (FIG. 6 (b)). These tables are collectively referred to as a terminal log table in FIG. In addition, the terminal log table may store an operation log table (see FIG. 13) described later. Although it is not specified that the communication terminal 200 of the present embodiment includes an earphone connection sensor, such a sensor may be provided.

FIG. 7 shows specific examples of the positioning history table and the retention point table. These tables are collectively referred to as a position information table in FIG. The positioning history table stores the position information obtained by GPS 206. The retention point table is information obtained by clustering which area the user frequently stays in based on the position of the communication terminal 200. In the figure, this table associates a retention point ID indicating the ID of the retention area, a retention point cluster ID, a stay start date and time of staying at the retention point, an end date and time thereof, and latitude and longitude.

FIG. 8 shows a specific example of the application activation history table and the application information table 101b. Since the application information stored in the application information table 101b is not log information, it is not a collection target and is prepared in advance. The application startup history table associates the startup date and time with the package name (application ID). The application information table is associated with a package name, a category number indicating an application category, and a tag indicating a communication function. The tag is information to be added later, and here, it is added as information indicating a communication function.

Next, the operation of the estimation model execution device 100 will be described. FIG. 9 is a flowchart showing the construction process of the attention function score estimation model. The communication terminal log collecting unit 101 collects sensor values and terminal log information from one or more communication terminals 200 owned by one or more users (S101). The feature amount calculation unit 102 calculates the feature amount based on the sensor value and the terminal log information (S102). The estimation model construction unit 105 inputs the feature amount of the communication terminal 200 as an explanatory variable (S103).

On the other hand, each user takes a test based on the Go-NoGo application, and the estimation model construction unit 105 acquires the test result (S105). Then, the estimation model construction unit 105 inputs an index of the attention function based on the Go-NoGo test result as an objective variable (S106).

Then, the estimation model construction unit 105 performs a construction process of the attention function score estimation model (S104), and stores the attention function score estimation model in the estimation model storage unit 106 (S107).

By this process, the attention function score estimation model can be constructed.

FIG. 10 is a flowchart in which the estimation model execution device 100 performs estimation processing of the user's attention function using the estimation model. The communication terminal log collecting unit 101 collects sensor values and log information from one communication terminal 200 and stores them in each table (S201). The feature amount calculation unit 102 acquires the sensor value and the terminal log information in the designated time unit, and calculates the feature amount based on the sensor value and the terminal log information (S202). The attention function estimation unit 103 applies the calculated feature amount to the attention function score estimation model (S203), and estimates the attention function score (S204). Thereby, the attention function of the user who possesses one communication terminal 200 can be estimated.

Next, the calculation method of the feature amount generated from the application startup log will be described. FIG. 11 is a flowchart showing a process of calculating the feature amount. This feature amount generation process is executed at the time of estimating the attention function score and at the time of constructing the attention function score estimation model.

The usage feature amount calculation unit 102 acquires the application activation log of the user's communication terminal 200 to be calculated on a predetermined date and a predetermined period from the log table 101a (S301).

The usage feature amount calculation unit 102 refers to the application information table 101b and extracts the application startup log for each category number (S302).

Then, the usage feature amount calculation unit 102 checks whether or not it corresponds to the application described in the exclusion list for each category number, and further, the tag added to the application startup log is CALL (call function) or SMS. It is checked whether or not it corresponds to (Short Message System) (S304 to S305).

Then, the usage feature amount calculation unit 102 excludes the use of the application whose tag corresponds to CALL or SMS from the use of the application whose activation time is equal to or less than the threshold value (S306), and the number of activations, activation time, total / activation time average of the application. Is calculated (S307).

Further, the usage feature amount calculation unit 102 calculates the number of activations, the activation time, and the total / average activation time for applications other than CALL or SMS (S308).

The usage feature amount calculation unit 102 uses these calculated results as feature amounts (S309), checks whether or not there is an unprocessed category number, and repeats for all applications until there is no unprocessed category number. Process (S303).

In the above process, the usage feature amount calculation unit 102 acquires the application activation log in a plurality of time units such as hourly, 6-hour units, and all-day units. Further, the usage feature amount calculation unit 102 extracts the category ID from the package name based on the application activation history table and the application information table 101b shown in FIG. 7, and calculates the activation time and the like for each category ID. For apps that have a CALL (call) tag in the app information table, the number of outgoing calls, the number of incoming calls, the total number, the total call time (outgoing / incoming / answering), and the average call are in the tag unit (CALL unit). The time (outgoing / incoming / answering) may be calculated. As a preprocessing, the history of less than 5 seconds may be removed.

Similarly, for SMS tags, the number of SMS activations, total activation time, and average activation time may be calculated for each tag. As a preprocessing, the history of less than 3 seconds may be removed.

Next, the process of generating the feature amount from the terminal log information based on the application operation will be described. FIG. 12 is a diagram showing the transition of the screen history for showing an example of the operation log showing the operation from the screen on of the communication terminal 200 to the activation of the application.

The screen G1 is a screen after a screen on event (a user operates the screen on) occurs from the standby state of the communication terminal 200 at time t1.

The screen G2 is a screen showing an operation acceptance status for subsequent unlocking.

The screen G3 shows the home screen after the unlock event (password input completion) is completed at time t2. Time Ton_unlock is the time until the screen is displayed and the lock is released. Time Ton_unlock = time t2-time t1.

The screen G4 shows the screen of the launched application when the user taps the application icon at time t3. The time Ths_tap indicates the time operated by the user from the unlocking to the start of the application, and the time Ths_tap = time t3-time t2.

The time Ton_unlock and the time Ths_tap are features based on the operation log.

In this way, using the on / off of the screen of the communication terminal 200 and the application startup log, the time from the screen on to the application startup can be calculated, and the startup time can be totaled to calculate the efficiency in user operation. it can. That is, if the user can operate efficiently, it can be determined that the user is exerting the attention function.

FIG. 13 is a specific example of an operation log table (terminal log table) that stores the above operation log. As shown in FIG. 13, a time, an event, and a value are described for each user ID (UID). As described above, the event indicates a user operation event. Value refers to an application.

A time is attached to each event, and the time Ton_unlock and the time Ths_tap can be calculated based on this time.

FIG. 14 is a flowchart showing the processing when the feature amount is calculated from the operation log in the estimation model execution device 100.

The usage feature amount calculation unit 102 acquires the operation log of the calculation target user for a predetermined period from the operation log table (S401). Then, the usage feature amount calculation unit 102 extracts a pair of Screen On and App Launch (S402).

The usage feature amount calculation unit 102 calculates the required time from the TimeStamp (time) of all the extracted Screen On and App Launch pairs (S403).

The feature amount calculation unit 102 calculates the average value of the required time (S404), and calculates the efficiency score based on the average value (S405). The efficiency score can be calculated by substituting the average value for X in any of the following equations.

Y = AX + B
Y = exp (AX + B)
Y = log (AX + B)
Further, the present invention is not limited to this, and any formula may be used as long as Y monotonically increases with respect to X. Further, A and B are arbitrarily set values in advance, but the present invention is not limited to this, and the user may be asked to conduct a questionnaire to evaluate the efficiency in five stages, and the coefficient may be set by machine learning. For this questionnaire, for example, a method based on a checklist in which the user himself / herself can input an efficiency evaluation of terminal operation such as a smartphone can be considered.

The efficiency score may be used as a feature amount, or the average value of time Ton_unlock and time Ths_tap, which are the source information of the efficiency score, may be used as a feature amount.

Next, the generation of the exclusion list described with reference to FIG. 11 will be described. The exclusion list is an application list for excluding applications such as the home screen and launcher that are not intentionally started by the user. By using this, it is possible to generate the application history that the user tried to operate.

FIG. 15A is an exclusion list associated with the average number of startups per day, the number of days activated, TF (Term Frequency), IDF (Inverse Document Frequency), and TF / IDF for each application (aggregated by category number). Shows the management table for. Here, TF indicates the average number of times each application is started in one day, and IDF indicates Log (number of days in aggregation unit / number of days activated + 1) in each application. The number of days in the aggregation unit is information for judging the importance of the application. For example, assuming that the aggregation unit is 30 days, it becomes Log (30 / number of start days + 1). The number of days is set arbitrarily. The TF is not limited to the average number of activations, and may be any information indicating the frequency of use in one day. For example, the average startup time, the total number of startups in a predetermined period, or the like may be used. Further, the IDF may be a formula indicating how much the application has been used in the number of days of the aggregation unit, and is not limited to the above formula.

By calculating the TF / IDF score for each category number, it is possible to determine important apps by weighting apps that are frequently used but not used every day.

Taking the management table of FIG. 15A as an example, the exclusion list generation unit 108 obtains TF / IDF based on the application activation history table, and excludes applications having category numbers whose TF / IDF is equal to or less than the threshold value. Register in the list. FIG. 15B is a specific example of the exclusion list, and the application of the category number in which the exclusion is true is defined as the exclusion list. The threshold value may be a predetermined numerical value, but may be a numerical value obtained by taking the average value or the median value of TF / IDF for all applications.

Next, the operation and effect of the estimation model execution device 100 of the present embodiment will be described. The estimation model execution device 100 is an estimation processing device that estimates the user's attention function, and is generated based on the sensor values of each sensor (motion sensor) of the communication terminal 200, which is an information processing device possessed by the user. The attention function estimation unit 103 for estimating the user's attention function using the obtained feature amount is provided.

In the above-described embodiment, the estimation model execution device 100 has a communication terminal log collecting unit 101 that acquires a sensor value (or a feature amount) transmitted from the communication terminal 200, and a utilization feature amount that converts the acquired sensor value into a feature amount. The calculation unit 102 and the estimation model execution device 100 are provided, but the feature amount is generated, but the feature amount is not limited to this. The communication terminal 200 may have a configuration equivalent to the usage feature amount calculation unit 102, generate a feature amount from the sensor value acquired by itself, and the estimation model execution device 100 may receive the feature amount. The same applies to the following description.

With this configuration, the user's attention function can be estimated from the sensor value of each sensor of the communication terminal 200. Therefore, the attention function of the user can be estimated from the movement of the communication terminal 200 possessed by the user without detecting the operation of the user. Since no user operation is required, the attention function can be estimated without making the user aware of it.

Further, the estimation model execution device 100 uses the attention function score estimation model constructed by using the respective sensor values of the plurality of users and the index of the attention function (for example, the test result of the Go-NoGo test), and the user's attention. Judge the function.

This attention function score estimation model may be constructed by itself using the estimation model construction unit 105, or may use a model constructed by another server (the estimation model construction unit 105 has a considerable function). ..

With this configuration, the user's attention function can be estimated using the learning model. Therefore, the estimation process can be realized by a simple process.

Further, in the estimation model execution device 100, the utilization feature amount calculation unit 102 generates a feature amount divided into predetermined time units from the sensor values acquired by the communication terminal log collection unit 101.

With this configuration, the user's attention function can be estimated in predetermined time units. By setting the predetermined time unit as both a fine time unit and a rough time unit, it is possible to accurately estimate the user's attention function.

Further, the communication terminal log collecting unit 101 acquires the logs of one or more motion sensors mounted on the communication terminal 200. In particular, by using a plurality of motion sensors, it is possible to obtain various feature quantities, and it is possible to accurately estimate the user's attention function.

Further, the estimation model execution device 100 of the present embodiment further includes a program management unit that acquires the activation state of a plurality of programs that can be executed by the communication terminal 200 as an operation log (including an application activation log). This program management unit can be realized by the communication terminal log collecting unit 101 and the terminal log table. The feature amount calculation unit 102 converts this operation log into a feature amount in addition to the sensor value of the motion sensor. The start state of the program indicates the start time in which the program is operating or the start time from a predetermined state to the start of the program.

With this configuration, it is possible to estimate the attention function based not only on the motion sensor but also on the activation operation of the user's application. Therefore, the estimation accuracy can be improved.

Further, the usage feature amount calculation unit 102 excludes the usage log of the program classified into the category defined by the predetermined method from the feature amount calculation target. Here, the programs of the categories described in the exclusion list 107 are excluded.

This configuration improves the estimation accuracy by excluding usage logs for programs that do not affect the user's attention function estimation.

For example, as an exclusion method, a category to be excluded is determined based on TF-IDF. In the present embodiment, the TF is the average number of startups per application for each application, and the IDF is the Log in each application (number of days in aggregation unit / number of days activated + 1). However, the TF is not limited to the above as long as it is an index showing the frequency of application startup and the IDF is an index showing the degree to which the application is commonly used on different days or on a person-by-person basis.

Further, the start-up state of the program includes a start-up start time from a predetermined state to start-up of the program. Further, the start state of the program includes the start count, start time, and communication time of the program used for communication among one or more started programs.

Since the user's attention function is determined according to the agility of the user's operation or the frequency with which the user is trying to make a call, the estimation accuracy can be improved.

The estimation model execution device 100 is configured to acquire sensor values and the like from the communication terminal 200, but is not limited to this. The communication terminal 200 may have the main functions of the estimation model execution device 100 (estimation model storage unit 106 and attention function estimation unit 103), and the communication terminal 200 may perform estimation processing using the estimation model. In this case, the estimation model may be acquired from the estimation model execution device 100 or other model construction device (not shown), or the communication terminal 200 may construct the estimation model by itself.

The estimation model execution device 100 may function as an estimation model construction device. When functioning as an estimation model construction device, the estimation model execution device 100 includes at least an estimation model construction unit 105 and an estimation model storage unit 106. Attention The configuration requirements for executing the function estimation process may be excluded.

The block diagram used in the description of the above embodiment shows a block of functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Further, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized by using one physically or logically connected device, or directly or indirectly (for example, two or more physically or logically separated devices). , Wired, wireless, etc.) and may be realized using these plurality of devices. The functional block may be realized by combining the software with the one device or the plurality of devices.

Functions include judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, solution, selection, selection, establishment, comparison, assumption, expectation, and assumption. There are broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, assigning, etc., but only these. I can't. For example, a functional block (constituent unit) for functioning transmission is called a transmitting unit or a transmitter. As described above, the method of realizing each of them is not particularly limited.

For example, the estimation model execution device 100 and the communication terminal 200 according to the embodiment of the present disclosure may function as a computer that processes the estimation model execution method and the estimation model construction method of the present disclosure. FIG. 16 is a diagram showing an example of the hardware configuration of the estimation model execution device 100 and the communication terminal 200 according to the embodiment of the present disclosure. The estimation model execution device 100 and the communication terminal 200 described above are physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like. May be good.

In the following description, the word "device" can be read as a circuit, a device, a unit, or the like. The hardware configuration of the estimation model execution device 100 and the communication terminal 200 may be configured to include one or more of the devices shown in the figure, or may be configured not to include some of the devices.

For each function of the estimation model execution device 100 and the communication terminal 200, the processor 1001 performs an operation by loading predetermined software (program) on hardware such as the processor 1001 and the memory 1002, and the communication device 1004 performs communication. It is realized by controlling or controlling at least one of reading and writing of data in the memory 1002 and the storage 1003.

Processor 1001 operates, for example, an operating system to control the entire computer. The processor 1001 may be configured by a central processing unit (CPU) including an interface with a peripheral device, a control device, an arithmetic unit, a register, and the like. For example, the above-mentioned utilization feature amount calculation unit 102, attention function estimation unit 103, estimation model construction unit 105, and the like may be realized by the processor 1001.

Further, the processor 1001 reads a program (program code), a software module, data, and the like from at least one of the storage 1003 and the communication device 1004 into the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least a part of the operations described in the above-described embodiment is used. For example, the feature amount calculation unit 102 and the like may be realized by a control program stored in the memory 1002 and operating in the processor 1001, and may be realized in the same manner for other functional blocks. Although it has been explained that the various processes described above are executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. Processor 1001 may be implemented by one or more chips. The program may be transmitted from the network via a telecommunication line.

The memory 1002 is a computer-readable recording medium, and is composed of at least one such as a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EPROM (Electrically Erasable Programmable ROM), and a RAM (Random Access Memory). May be done. The memory 1002 may be referred to as a register, a cache, a main memory (main storage device), or the like. The memory 1002 can store a program (program code), a software module, or the like that can be executed to execute the estimation model execution method and the estimation model construction method according to the embodiment of the present disclosure.

The storage 1003 is a computer-readable recording medium, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (for example, a compact disk, a digital versatile disk, a Blu-ray). It may consist of at least one (registered trademark) disk), smart card, flash memory (eg, card, stick, key drive), floppy (registered trademark) disk, magnetic strip, and the like. The storage 1003 may be referred to as an auxiliary storage device. The storage medium described above may be, for example, a database, server or other suitable medium containing at least one of memory 1002 and storage 1003.

The communication device 1004 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, or the like. Communication device 1004 includes, for example, a high frequency switch, a duplexer, a filter, a frequency synthesizer, and the like in order to realize at least one of frequency division duplex (FDD) and time division duplex (TDD). It may be composed of. For example, the communication terminal log collecting unit 101 described above may be realized by the communication device 1004. Further, the transmission / reception function may be physically or logically separated.

The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives an input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that outputs to the outside. The input device 1005 and the output device 1006 may have an integrated configuration (for example, a touch panel).

Further, each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be configured by using a single bus, or may be configured by using a different bus for each device.

Further, the estimation model execution device 100 and the communication terminal 200 include a microprocessor, a digital signal processor (DSP: Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field Programmable Gate Array), and the like. It may be configured to include the hardware of the above, and a part or all of each functional block may be realized by the hardware. For example, processor 1001 may be implemented using at least one of these hardware.

Notification of information is not limited to the embodiments / embodiments described in the present disclosure, and may be performed by other methods. For example, information notification includes physical layer signaling (for example, DCI (Downlink Control Information), UCI (Uplink Control Information)), upper layer signaling (for example, RRC (Radio Resource Control) signaling, MAC (Medium Access Control) signaling, etc. It may be carried out by notification information (MIB (Master Information Block), SIB (System Information Block)), other signals, or a combination thereof. Further, the RRC signaling may be referred to as an RRC message, and may be, for example, an RRC Connection Setup message, an RRC Connection Reconfiguration message, or the like.

Each aspect / embodiment described in the present disclosure includes LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), and 5G (5th generation mobile communication). system), FRA (Future Radio Access), NR (new Radio), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)) )), IEEE 802.16 (WiMAX®), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth®, and other systems that utilize and extend based on these. It may be applied to at least one of the next generation systems. Further, a plurality of systems may be applied in combination (for example, a combination of at least one of LTE and LTE-A and 5G).

The order of the processing procedures, sequences, flowcharts, and the like of each aspect / embodiment described in the present disclosure may be changed as long as there is no contradiction. For example, the methods described in the present disclosure present elements of various steps using exemplary order, and are not limited to the particular order presented.

Information and the like may be input / output via a plurality of network nodes that can be output from the upper layer (or lower layer) to the lower layer (or upper layer).

The input / output information and the like may be stored in a specific location (for example, memory), or may be managed using a management table. Input / output information and the like can be overwritten, updated, or added. The output information and the like may be deleted. The input information or the like may be transmitted to another device.

The determination may be made by a value represented by 1 bit (0 or 1), by a boolean value (Boolean: true or false), or by comparing numerical values (for example, a predetermined value). It may be done by comparison with the value).

Each aspect / embodiment described in the present disclosure may be used alone, in combination, or switched with execution. Further, the notification of predetermined information (for example, the notification of "being X") is not limited to the explicit notification, but is performed implicitly (for example, the notification of the predetermined information is not performed). May be good.

Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described in the present disclosure. The present disclosure may be implemented as an amendment or modification without departing from the purpose and scope of the present disclosure, which is determined by the description of the scope of claims. Therefore, the description of the present disclosure is for the purpose of exemplary explanation and does not have any restrictive meaning to the present disclosure.

Software is an instruction, instruction set, code, code segment, program code, program, subprogram, software module, whether called software, firmware, middleware, microcode, hardware description language, or another name. , Applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, features, etc. should be broadly interpreted to mean.

Further, software, instructions, information and the like may be transmitted and received via a transmission medium. For example, a website, where the software uses at least one of wired technology (coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), etc.) and wireless technology (infrared, microwave, etc.). When transmitted from a server, or other remote source, at least one of these wired and wireless technologies is included within the definition of transmission medium.

The information, signals, etc. described in the present disclosure may be represented using any of a variety of different techniques. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. It may be represented by a combination of.

The terms described in the present disclosure and the terms necessary for understanding the present disclosure may be replaced with terms having the same or similar meanings. For example, at least one of a channel and a symbol may be a signal (signaling). Also, the signal may be a message. Further, the component carrier (CC: Component Carrier) may be referred to as a carrier frequency, a cell, a frequency carrier, or the like.

The terms "system" and "network" used in this disclosure are used interchangeably.

Further, the information, parameters, etc. described in the present disclosure may be expressed using absolute values, relative values from predetermined values, or using other corresponding information. It may be represented. For example, the radio resource may be one indicated by an index.

The names used for the above parameters are not limited in any respect. Further, mathematical formulas and the like using these parameters may differ from those explicitly disclosed in this disclosure. Since the various channels (eg, PUCCH, PDCCH, etc.) and information elements can be identified by any suitable name, the various names assigned to these various channels and information elements are in any respect limited names. is not.

The terms "determining" and "determining" used in this disclosure may include a wide variety of actions. "Judgment" and "decision" are, for example, judgment (judging), calculation (calculating), calculation (computing), processing (processing), derivation (deriving), investigating (investigating), search (looking up, search, inquiry). It may include (eg, searching in a table, database or another data structure), ascertaining as "judgment" or "decision". Also, "judgment" and "decision" are receiving (for example, receiving information), transmitting (for example, transmitting information), input (input), output (output), and access. (Accessing) (for example, accessing data in memory) may be regarded as "judgment" or "decision". In addition, "judgment" and "decision" mean that "resolving", "selecting", "choosing", "establishing", "comparing", etc. are regarded as "judgment" and "decision". Can include. That is, "judgment" and "decision" may include that some action is regarded as "judgment" and "decision". Further, "judgment (decision)" may be read as "assuming", "expecting", "considering" and the like.

The terms "connected", "coupled", or any variation thereof, mean any direct or indirect connection or connection between two or more elements, and each other. It can include the presence of one or more intermediate elements between two "connected" or "combined" elements. The connection or connection between the elements may be physical, logical, or a combination thereof. For example, "connection" may be read as "access". As used in the present disclosure, the two elements use at least one of one or more wires, cables and printed electrical connections, and, as some non-limiting and non-comprehensive examples, the radio frequency domain. Can be considered to be "connected" or "coupled" to each other using electromagnetic energies having wavelengths in the microwave and light (both visible and invisible) regions.

The phrase "based on" as used in this disclosure does not mean "based on" unless otherwise stated. In other words, the statement "based on" means both "based only" and "at least based on".

When "include", "including" and variations thereof are used in the present disclosure, these terms are as comprehensive as the term "comprising". Is intended. Furthermore, the term "or" used in the present disclosure is intended not to be an exclusive OR.

In the present disclosure, if articles are added by translation, for example, a, an and the in English, the disclosure may include the nouns following these articles in the plural.

In the present disclosure, the term "A and B are different" may mean "A and B are different from each other". The term may mean that "A and B are different from C". Terms such as "separate" and "combined" may be interpreted in the same way as "different".

100 ... estimation model execution device, 101 ... communication terminal log collection unit, 101a ... log table, 101b ... application information table, 102 ... usage feature amount calculation unit, 103 ... attention function estimation unit, 104 ... estimation result display unit, 105 ... Estimated model construction unit, 106 ... Estimated model storage unit, 107 ... Exclusion list, 108 ... Exclusion list generation unit, 200 ... Communication terminal, 201 ... Acceleration sensor, 202 ... Angle speed sensor, 203 ... Tilt sensor, 204 ... Screen on / off Sensor, 205 ... App usage sensor, 207 ... Communication unit, 208 ... Control unit, 209 ... Program.

Claims (12)

Attention function estimation unit that estimates the user's attention function using the feature amount generated based on the sensor value of the motion sensor of the information processing device possessed by the user.
An estimation processing device comprising. The attention function estimation unit
Judgment of a user's attention function using an estimation model constructed using the respective sensor values of multiple users and the index of attention function.
The estimation processing device according to claim 1. An estimation model construction unit for constructing the estimation model is further provided by using the respective sensor values of the plurality of users and the index of the attention function.
The estimation processing device according to claim 2. The attention function estimation unit estimates using the feature amount generated by dividing the sensor value into predetermined time units.
The estimation processing apparatus according to any one of claims 1 to 3. The feature amount is generated based on the logs of one or a plurality of motion sensors mounted on the information processing device.
The estimation processing device according to any one of claims 1 to 4. The feature amount is generated based on the usage log acquired from the start state of the information processing apparatus or a plurality of programs.
The start state of the program indicates the start time during which the program is operating or the start time from a predetermined state until the program is started.
The estimation processing device according to any one of claims 1 to 5. The estimation processing device according to claim 6, wherein the feature amount is generated by excluding usage logs for programs classified into a category defined by a predetermined method among the programs. The estimation processing apparatus according to claim 7, wherein as the predetermined method, a category to be excluded is determined based on TF-IDF calculated for each category of the program. The estimation processing device according to any one of claims 6 to 8, wherein the activation state of the program includes an activation start time from a predetermined state to activation of the program. The start state of the program includes the start count, start time, and communication time of the program used for communication among one or more started programs.
The estimation processing apparatus according to any one of claims 6 to 9. The estimation processing device according to any one of claims 1 to 10, further comprising an attention function presenting unit that presents an estimated attention function. One or more feature quantities generated based on the sensor values of the motion sensors of one or more information processing devices possessed by one or more users, and the index of the attention function of the one or more users. Acquired,
An estimation model construction device that constructs an estimation model that estimates a user's attention function by machine learning using the feature amount as an explanatory variable and the index of the attention function as teacher data.