JP2023147654A - Information processing system, information processing method, and program - Google Patents

Abstract

To provide an information processing system, an information processing method, and a program which can acquire information applicable to generate personalized information.SOLUTION: An information processing system is configured to estimate a user's operation state for a terminal device. The information processing system comprises: state information acquisition means 32 which acquires information about the user's state measured by a measurement device; and operation state estimation means 35 which estimates the user's operation state for the terminal device in a prescribed period, on the basis of transition of the information about the user's state in the prescribed period.SELECTED DRAWING: Figure 4

Description

Application for application of Article 30, Paragraph 2 of the Patent Act Presentation of information processing systems, information processing methods, and programs at meetings Meeting name: Information Processing Society of Japan 84th National Convention Venue: Ehime University Johoku Campus Hybrid event Date: Reiwa March 3rd to 5th, 4th year

The present invention relates to an information processing system, an information processing method, and a program.

In recent years, with the spread of terminal devices such as mobile terminals, functions that collect information about users of terminal devices and provide personalized information to users (for example, search engines whose results change depending on the user's search history and location) Information providing services have been put into practical use, including a recommendation function that displays recommended products according to viewing history and purchase history.

For example, in the technology described in Patent Document 1, items related to the item are recommended to a user who has taken an action on the item.

Japanese Patent Application Publication No. 2011-154591

Such information provision services analyze information such as a user's search history, location, web page browsing/purchase history, etc., and generate personalized information according to the analysis results. . On the other hand, in order to generate more personalized information, it is necessary to perform analysis using more appropriate information.

The present invention has been made in view of the above problems, and aims to provide an information processing system, an information processing method, and a program that can obtain information applicable to generating personalized information. .

In order to solve the above problems, the present invention first provides an information processing system that estimates a user's operating state with respect to a terminal device, the state information acquisition system that obtains information regarding the user's state measured by a measuring device. and operation state estimating means for estimating the operation state of the user with respect to the terminal device within the predetermined period based on the transition of information regarding the user's state within the predetermined period. (Invention 1).

According to this invention (Invention 1), based on information regarding the user's state within a predetermined period, the operation state of the user on the terminal device within the predetermined period (for example, how often the terminal device is operated or the operation For example, by analyzing the estimated user's operating state, it becomes possible to generate personalized information according to the user's operating state. Thereby, information (user operation status) applicable to generating personalized information can be obtained.

In the above invention (invention 1), the operation state estimating means performs learning based on machine learning using the transition of information regarding the user's state within the predetermined period and the transition of the information regarding the user's state as learning data. The operating state of the user with respect to the terminal device within the predetermined period may be estimated based on the completed model (Invention 2).

According to this invention (invention 2), by using the learned model, it is possible to easily estimate the user's operation state on the terminal device within a predetermined period.

The above inventions (Inventions 1 and 2) further include a behavior estimation means for estimating the behavior pattern of the user within the predetermined period based on the transition of information regarding the state of the user within the predetermined period, and the operation state The estimation means may estimate the operation state of the user with respect to the terminal device within the predetermined period based on the behavior pattern of the user within the predetermined period (invention 3).

According to this invention (invention 3), it is possible to estimate the user's operation state for the terminal device within the predetermined period using the user's behavior pattern estimated based on the information regarding the user's state within the predetermined period. become.

In the above invention (invention 3), the behavior estimation means is configured to perform learning based on machine learning using the transition of information regarding the user's status within the predetermined period and the transition of the information regarding the user's status as learning data. The behavior pattern of the user within the predetermined period may be estimated based on the model (Invention 4).

According to this invention (invention 4), the user's behavior pattern within a predetermined period can be easily estimated by using the learned model.

In the above inventions (inventions 3 and 4), the behavior estimation includes a detection means for detecting a state change point indicating a timing at which an operation state of the user with respect to the terminal device changes based on information regarding the state of the user; The means may estimate the behavior pattern of the user within a period from a first state change point to a second state change point that is the next state change point after the first state change point (invention 5). ).

According to this invention (invention 5), the process of estimating the user's behavior pattern is started at the timing when the state change point is detected, and the process of estimating the user's behavior pattern is not started at any other timing. Compared to the case where the behavior pattern estimation process is started all the time, it is possible to reduce the load on the user's behavior pattern estimation process.

The above inventions (inventions 1 to 5) include a detection means for detecting a state change point indicating a timing at which the operation state of the user with respect to the terminal device changes, based on information regarding the state of the user; The estimating means estimates an operation state of the user with respect to the terminal device within a period from a first state change point to a second state change point that is a next state change point after the first state change point. Good (Invention 6).

According to this invention (invention 6), the process of estimating the user's operation state is started at the timing when the state change point is detected, and the process of estimating the user's operation state is not started at any other timing. Compared to the case where the operation state estimation process is started all the time, it is possible to reduce the load on the user's operation state estimation process.

In the above inventions (Inventions 5 and 6), the detection means determines whether the predetermined timing is the state change point when information regarding the state of the user at the predetermined timing satisfies a predetermined condition. The state change point may be detected by doing so (invention 7).

According to this invention (Invention 7), since the state change point detection process is performed at a timing when the information regarding the user's state satisfies a predetermined condition, compared to, for example, a case where the state change point detection process is always performed. , it becomes possible to reduce the load on the state change point detection process.

In the above invention (invention 7), the predetermined condition may include that a differential value of a curve indicating a transition of information regarding the user's state is 0 (invention 8).

According to this invention (invention 8), when the differential value of the curve indicating the transition of information regarding the user's state is 0, it becomes possible to perform the state change point detection process.

In the above inventions (inventions 7 and 8), the detection means detects that the predetermined timing is determined when the change point score indicating the magnitude of change in the information regarding the user's state at the predetermined timing is equal to or greater than a predetermined value. It may be determined that this is the state change point (invention 9).

According to this invention (Invention 9), by using the change point score, it becomes possible to easily detect the state change point.

The above inventions (Inventions 1 to 9) may further include a control means for controlling the operation of the terminal device according to the estimated operating state of the terminal device by the user within the predetermined period (Invention 10). .

According to this invention (Invention 10), the terminal device can be controlled by, for example, reducing the brightness of the display section in order to reduce the power consumption of the terminal device when the user is not operating the terminal device much. By doing so, it becomes possible to appropriately operate the terminal device according to the user's operating state.

The above-mentioned inventions (inventions 1 to 10) may further include providing means for providing information according to the operation state of the user with respect to the terminal device within the estimated predetermined period (invention 11).

According to this invention (Invention 11), appropriate information is provided depending on the operating state of the user, such as providing advertising information to the user via the terminal device when the user frequently operates the terminal device. It becomes possible to provide information.

The above inventions (Inventions 1 to 11) include page information acquisition means for acquiring information about a web page displayed on a web browser running on the terminal device, and the status information acquisition means acquires information regarding the state of the user measured by the measuring device when the web page is displayed on the web browser; The operating state of the user on the terminal device within a predetermined period may be estimated (invention 12).

According to this invention (invention 12), it becomes possible to estimate the user's operating state when a predetermined web page is displayed on a web browser running on a terminal device.

In the above inventions (Inventions 1 to 12), a changing means changes a display mode of a web page displayed on the web browser according to an operation state of the user on the terminal device within the estimated predetermined period. (Invention 13).

According to this invention (Invention 13), for example, it is possible to dynamically change the design, layout, etc. of a web page depending on the user's operation status when the web page is displayed on the web browser. Become.

In the above inventions (Inventions 1 to 13), the information regarding the user's state may include at least one of the user's acceleration and the user's angular velocity (Invention 14).

According to this invention (invention 14), the user's operating state can be estimated based on at least one of the user's acceleration and angular velocity.

In the above inventions (Inventions 1 to 14), the operation state of the user may include at least one of a touch state, a scroll state, a swipe state, a flick state, a pinch state, and a drag state of the user on the terminal device. Good (Invention 15).

According to this invention (Invention 15), the user's touch state (for example, the number of touch operations, the time of the touch operation, etc.) and the scroll state (for example, the number of scroll operations, the scroll operation time, etc.) are determined based on the information regarding the user's state. ), swipe status (e.g., number of swipe operations, duration of swipe operation, etc.), flick status (e.g., number of flick operations, duration of flick operation, etc.), pinch status (e.g., pinch in and/or pinch out) It becomes possible to estimate at least one of the following: the number of operations, the time of pinch-in and/or pinch-out operations, etc.) and the drag state (for example, the number of drag operations, the time of drag operations, etc.).

In the above inventions (Inventions 1 to 15), the measuring device may be provided in the terminal device (Invention 16).

According to this invention (invention 16), it becomes possible to acquire information regarding the user's condition using the measuring device provided in the terminal device.

Second, the present invention is an information processing method for estimating a user's operating state with respect to a terminal device using a computer, comprising a step in which the computer acquires information regarding the user's state measured by a measuring device. , estimating the operation state of the user with respect to the terminal device within the predetermined period based on the transition of information regarding the state of the user within the predetermined period. (Invention 17).

Thirdly, the present invention is a program for a computer to estimate a user's operating state with respect to a terminal device, the computer having a function of acquiring information regarding the user's state measured by a measuring device, and a predetermined function. A program is provided for realizing a function of estimating the operating state of the user with respect to the terminal device within the predetermined period based on the transition of information regarding the user's state within the period (invention 18).

According to the information processing system, information processing method, and program of the present invention, it is possible to obtain information applicable to generating personalized information.

1 is a diagram schematically showing the basic configuration of an information processing system according to an embodiment of the present invention. FIG. 2 is a block diagram showing the configuration of a terminal device. FIG. 1 is a block diagram showing the configuration of an information processing device. FIG. 2 is a functional block diagram for explaining functions that play a major role in the information processing system. It is a figure showing the example of composition of the 1st acquisition data. It is a figure which shows the example of a structure of 2nd acquired data. FIG. 3 is a diagram showing an example of a change over time in a user's acceleration and change point score. FIG. 3 is a diagram showing an example of the configuration of first learning data. It is a figure which shows the example of a structure of 2nd learning data. It is a figure showing an example of composition of estimation data. It is a figure showing an example of composition of control data. It is a figure showing an example of composition of provided data. (a) and (b) are diagrams showing an example of the layout of a web page displayed on a web browser according to an estimated operation state. 1 is a flowchart illustrating an example of main processing of an information processing system according to an embodiment of the present invention. It is a figure which shows the example of a structure of operation state data. It is a figure which shows the example of a structure of 3rd learning data. FIG. 3 is a diagram illustrating an example of division of functions between a terminal device and an information processing device for each function of the information processing system.

Hereinafter, one embodiment of the present invention will be described in detail with reference to the accompanying drawings. However, this embodiment is an example, and the present invention is not limited thereto.

(1) Basic configuration of information processing system FIG. 1 is a diagram schematically showing the basic configuration of an information processing system according to an embodiment of the present invention. As shown in FIG. 1, the information processing system according to the present embodiment is configured to estimate a user's operating state with respect to a terminal device 10, and the information processing device 20 is configured to estimate a user's operation state with respect to a terminal device 10. The system acquires information regarding the user's state measured by the method shown in FIG. . The terminal device 10 and the information processing device 20 are connected to a communication network NW (network) such as the Internet or a LAN (Local Area Network).

The terminal device 10 may be, for example, a device that can be worn by a user (for example, a wearable device), or a portable device that can be carried by the user. Further, the terminal device 10 is, for example, a mobile terminal, a smartphone, a PDA (Personal Digital Assistant), a personal computer, a television receiver with a two-way communication function (including a so-called multi-function smart TV), etc. It may also be a device operated by an individual user. Furthermore, the terminal device 10 may be a shared device that can be operated by multiple users.

The information processing device 20 is a device for obtaining information applicable to generating personalized information. Similar to the terminal device 10, the information processing device 20 may be a device operated by an individual user.

(2) Configuration of terminal device The configuration of the terminal device 10 will be described with reference to FIG. 2. FIG. 2 is a block diagram showing the internal configuration of the terminal device 10. As shown in FIG. 2, the terminal device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage device 14, a display processing section 15, It includes a display section 16, an input section 17, a measuring device 18, and a communication interface section 19, and is provided with a bus 10a for transmitting control signals or data signals between each section.

When the terminal device 10 is powered on, the CPU 11 loads various programs stored in the ROM 12 or the storage device 14 into the RAM 13 and executes them.

The storage device 14 is, for example, a non-volatile storage device such as a flash memory, a solid state drive (SSD), a magnetic storage device (for example, a hard disk drive (HDD), a floppy disk (registered trademark), a magnetic tape, etc.), or an optical disk. It may be a volatile storage device such as a RAM, and stores programs executed by the CPU 11 and data referenced by the CPU 11.

The display processing unit 15 displays display data given from the CPU 11 on the display unit 16. The display unit 16 is, for example, an LCD (Liquid Crystal Display) monitor including thin film transistors arranged pixel by pixel in a matrix, and displays data on the display screen by driving the thin film transistors based on display data. indicate.

When the terminal device 10 is a communication device using a button input method, the input unit 17 includes a button group including a plurality of instruction input buttons such as a direction instruction button and a decision button for accepting user operation input, and a numeric keypad, etc. It includes a button group including a plurality of instruction input buttons, and includes an interface circuit for recognizing the press (operation) input of each button and outputting it to the CPU 11.

When the terminal device 10 is a communication device using a touch panel input method, the input unit 17 mainly receives input using the touch panel method by touching the display screen with a fingertip or a pen. The touch panel input method may be a known method such as a capacitive method.

Further, when the terminal device 10 is a device capable of voice input, the input unit 17 may be configured to include a microphone for voice input, or may be configured to include voice input via an external microphone. An interface circuit for outputting the information to the CPU 11 may be provided. Furthermore, when the terminal device 10 is a communication device capable of inputting moving images and/or still images, the input unit 17 may be configured to include a digital camera or a digital video camera for inputting images. , an interface circuit for receiving image data captured by an external digital camera or digital video camera and outputting it to the CPU 11 may be provided.

The measuring device 18 is a device that measures the state of the user of the terminal device 10 continuously or intermittently (for example, at predetermined intervals (for example, every 200 milliseconds)). Here, the information regarding the user's state may be, for example, a value representing the user's state, or a value obtained by substituting a value representing the user's state into a predetermined calculation formula. Alternatively, the information may be information representing the degree of the user's condition.

The measuring device 18 measures, for example, the state of the user (for example, acceleration in three or two axes, angular velocity in three or two axes, heart rate (pulse), blood pressure, body temperature, amount of sweat, number of steps, walking speed) , posture, exercise intensity (e.g., heart rate ÷ maximum heart rate), or calories burned (e.g., acceleration sensor, gyro sensor, motion sensor, heart rate monitor, blood pressure monitor, thermometer, sweat meter, etc.) It's okay.

The communication interface unit 19 includes an interface circuit for communicating with other devices (for example, the information processing device 20) via the communication network NW.

(3) Configuration of information processing device The configuration of the information processing device 20 will be described with reference to FIG. 3. FIG. 3 is a block diagram showing the internal configuration of the information processing device 20. As shown in FIG. 3, the information processing device 20 includes a CPU 21, a ROM 22, a RAM 23, a storage device 24, a display processing section 25, a display section 26, an input section 27, and a communication interface section 28. A bus 20a is provided for transmitting control signals or data signals between each section.

When the information processing device 20 is powered on, the CPU 21 loads various programs stored in the ROM 22 or the storage device 24 into the RAM 23 and executes them. In the present embodiment, the CPU 21 reads out and executes a program stored in the ROM 22 or the storage device 24 to operate a page information acquisition means 31, a state information acquisition means 32, a detection means 33, an action estimation means 34, which will be described later. The functions of the state estimating means 35, the controlling means 36, the providing means 37, and the changing means 38 (shown in FIG. 4) are realized.

The storage device 24 may be a nonvolatile storage device such as a flash memory, an SSD, a magnetic storage device (for example, an HDD, a floppy disk (registered trademark), a magnetic tape, etc.), an optical disk, or a volatile storage device such as a RAM. It may be a physical storage device, and stores programs executed by the CPU 21 and data referenced by the CPU 21. The storage device 24 also stores first acquired data (shown in FIG. 5), second acquired data (shown in FIG. 6), first learning data (shown in FIG. 8), and second learning data (shown in FIG. 9), which will be described later. ), estimated data (shown in FIG. 10), control data (shown in FIG. 11), and provision data (shown in FIG. 12) are stored.

The input unit 27 may be an information input device such as a mouse or a keyboard, may include a microphone for audio input, or may include a digital camera or digital video camera for image input. Alternatively, an interface circuit for receiving image data captured by an external digital camera or digital video camera and outputting it to the CPU 21 may be provided.

The communication interface unit 28 includes an interface circuit for communicating via the communication network NW. The details of other parts within the information processing device 20 may be the same as those of the terminal device 10.

(4) Overview of each function in the information processing system The functions realized in the information processing system of this embodiment will be explained with reference to FIG. 4. FIG. 4 is a functional block diagram for explaining functions that play a major role in the information processing system of this embodiment. In the functional block diagram of FIG. 4, the state information acquisition means 32 and the operation state estimation means 35 correspond to the main components of the information processing system of the present invention. Other means (page information acquisition means 31, detection means 33, behavior estimation means 34, control means 36, provision means 37, and change means 38) are not necessarily essential components, but are constituent elements for making the present invention even more preferable. It is.

The page information acquisition means 31 has a function of acquiring information regarding a web page displayed on a web browser running on the terminal device 10.

Here, the information regarding the web page may include identification information of the web page (for example, a URL (Uniform Resource Locator), etc.), or may include information regarding the display state of the web page on the terminal device 10. Further, information regarding the display state of the web page includes, for example, the state in which the web page is displayed on the screen (display unit 16) of the terminal device 10 (foreground), the state in which the web page is displayed on the screen (display unit 16) of the terminal device 10, A state in which the web page is not displayed (background), a state in which the web page is pre-rendered (pre-rendering), etc. may also be included. This makes it possible to understand in what operating state the user is viewing (displaying) the web page, as well as in what state the web page is being displayed. By analyzing the user's operating state, more personalized information can be generated according to the user's operating state.

Further, the information regarding the web page may include information regarding whether the web page is being displayed online. This makes it possible to understand in what operating state a user is viewing a web page, as well as whether or not the web page is being displayed online. By analyzing, etc., it is possible to generate more personalized information according to the user's operating status.

The functions of the page information acquisition means 31 are realized, for example, as follows. First, when a web browser (e.g., Google Chrome (registered trademark), etc.) is being executed by a user's operation using the input unit 17, the CPU 11 of the terminal device 10 executes a process for a predetermined period of time (e.g., 10 seconds, etc.). ) elapses, information regarding the web page displayed on the web browser is acquired. Note that here, the information regarding the web page includes the identification information of the web page (for example, URL, etc.), the display state of the web page (for example, foreground, background, pre-rendering, etc.), and whether the web page is displayed online. It is assumed that this includes whether or not there is a person. Further, the CPU 11 may acquire information regarding a web page by executing an application configured using, for example, JavaScript (registered trademark). Then, each time the CPU 11 acquires information regarding a web page, the CPU 11 transmits the acquired information to the information processing device 20 via the communication interface unit 19 and the communication network NW. Here, the information sent to the information processing device 20 includes the acquisition date and time of information regarding the web page, and identification information of the terminal device 10 (for example, the serial number and MAC (Media Access Control) address of the terminal device 10). , may be included.

On the other hand, each time the CPU 21 of the information processing device 20 receives (acquires) information regarding a web page via the communication interface unit 28, it stores the information regarding the web page in, for example, first acquired data shown in FIG. The first acquired data includes information about the web page (in the example shown, the URL of the web page, the acquisition date and time of the information about the web page, the information about the web page This is data in which the display state of the Web page and the connection state of the terminal device 10 (whether the Web page is displayed online or not) are associated with each other. Here, as shown in FIG. 5, the acquisition date and time of information regarding the web page, the display state of the web page, and the connection state of the terminal device 10 may be associated with each URL of the web page.

The status information acquisition means 32 has a function of acquiring information regarding the status of the user of the terminal device 10 measured by the measuring device 18.

Further, the status information acquisition unit 32 may acquire information regarding the user's status measured by the measuring device 18 while the web page is displayed on the web browser.

Here, the information regarding the user's state may include at least one of the user's acceleration and the user's angular velocity. Thereby, the user's operating state can be estimated based on at least one of the user's acceleration and angular velocity.

The functions of the status information acquisition means 32 are realized, for example, as follows. Note that here, a case will be described as an example in which the status information acquisition unit 32 acquires information regarding the user's status measured by the measuring device 18 while a web page is displayed on the web browser. First, the CPU 11 of the terminal device 10 checks the state of the user every time a predetermined period of time (for example, 200 milliseconds, etc.) elapses when the web browser is executed by the user's operation using the input unit 17, for example. (For example, at least one of the user's acceleration in three or two axis directions and the user's angular velocity in three or two axis directions) is acquired. Here, the CPU 11 may obtain information regarding the user's state by executing an application configured using, for example, JavaScript (registered trademark) and operating the measuring device 18. Then, each time the CPU 11 measures information regarding the user's condition, the CPU 11 transmits the measured information regarding the user's condition to the information processing device 20 via the communication interface unit 19 and the communication network NW. Here, the information transmitted to the information processing device 20 may include the acquisition (measurement) date and time of information regarding the user's condition and the identification information of the terminal device 10 described above.

On the other hand, each time the CPU 21 of the information processing device 20 receives (acquires) information regarding the user's status via the communication interface unit 28, the CPU 21 stores the information regarding the user's status in the second acquired data shown in FIG. 6, for example. . In the second acquired data, the date and time of acquisition (measurement) of information regarding the user's condition is associated with the information regarding the user's condition for each identification information of the terminal device 10 (in the illustrated example, the terminal device ID). This is data described in the state.

The detection means 33 has a function of detecting a state change point indicating the timing at which the user's operating state of the terminal device 10 changes based on information regarding the user's state.

Further, the detection means 33 detects a state change point by determining whether or not the predetermined timing is a state change point when information regarding the user's state at a predetermined timing satisfies a predetermined condition. Good too. As a result, the state change point detection process is performed at the timing when the information regarding the user's state satisfies a predetermined condition. It becomes possible to reduce the load on the

Here, the predetermined condition may include that the differential value of the curve representing the transition of information regarding the user's state is zero. This makes it possible to perform state change point detection processing when the differential value of the curve representing the transition of information regarding the user's state is 0. Note that the predetermined condition is, for example, that the absolute value of the average amount of change in information regarding the user's state (for example, one or more of acceleration and angular velocity) within a certain period including a predetermined timing is less than a predetermined value. It may include something.

Further, the detection means 33 may determine that the predetermined timing is a state change point when a change point score indicating the magnitude of change in information regarding the user's state at a predetermined timing is equal to or greater than a predetermined value. . Thereby, by using the change point score, it becomes possible to easily detect the state change point.

The function of the detection means 33 is realized, for example, as follows. Note that here, the differential value of the curve indicating the transition of information regarding the user's condition at a predetermined timing is 0, and the change point score of the information regarding the user's condition at the predetermined timing is equal to or greater than the predetermined value. An example will be described in which the predetermined timing is detected as a state change point.

For example, the CPU 21 of the information processing device 20 determines whether a predetermined period of time (e.g., 3 seconds, etc.) has elapsed starting from a predetermined timing (e.g., the timing at which information regarding the state of the first user is received (acquired) from the terminal device 10, etc.). Each time, information regarding the user's status stored in the second acquired data within the predetermined period (information regarding the transition of information regarding the user's status within the predetermined period) is extracted from the second acquired data. Next, the CPU 21 uses the extracted information regarding the user's status within the predetermined period to calculate a quadratic curve representing a change over time of the information regarding the user's status within the predetermined period. Then, the CPU 21 determines the time when the differential value becomes 0 in the calculated quadratic curve (an example of the "predetermined timing" in the present invention) within the predetermined period (the date and time of acquisition (measurement) of information regarding the user's condition). ) is extracted.

Next, the CPU 21 of the information processing device 20 determines for each extracted time whether or not the time is a state change point. Here, the CPU 21 may determine the state change point using a well-known state change point detection method. Note that here, a case will be described in which Change Finder is used as an example of a detection method. Change Finder has characteristics such as shorter calculation time than conventional detection methods, ability to handle non-stationary data, and suitability for online processing. The CPU 21 calculates a change point score using the Change Finder for each extracted time, and detects the time when this change point score changes from less than a predetermined threshold Th to more than the threshold Th as a state change point. FIG. 7 shows an example of changes over time in acceleration, which is an example of information regarding the user's condition, and change point scores. In the example shown in FIG. 7, times t1 and t2 when the change point score goes from less than the threshold Th (20 in the example shown) to more than the threshold Th are detected as state change points.

The behavior estimation means 34 has a function of estimating the user's behavior pattern within a predetermined period based on the transition of information regarding the user's state within the predetermined period.

Furthermore, the behavior estimation means 34 calculates the behavior within a predetermined period based on the transition of information regarding the user's state within a predetermined period and the trained model based on machine learning using the transition of information regarding the user's state as learning data. The user's behavior pattern may be estimated. In this case, by using the trained model, the user's behavior pattern within a predetermined period can be easily estimated.

Furthermore, the behavior estimation means 34 estimates the user's behavior pattern within the period from the first state change point to the second state change point that is the next state change point after the first state change point. good. In this case, the process of estimating the user's behavior pattern is started at the timing when the state change point is detected, and the process of estimating the user's behavior pattern is not started at any other timing, so for example, the process of estimating the user's behavior pattern is always Compared to the case where the process is started, it is possible to reduce the load on the process of estimating the user's behavior pattern.

The function of the behavior estimation means 34 is realized, for example, as follows. Note that here, the behavior estimation means 34 changes from the first state change point (for example, time t1 shown in FIG. 7) to the second state change point (which is the next state change point after the first state change point). For example, a case will be described in which a learned model is used to estimate a user's behavior pattern during a period up to time t2) shown in FIG. 7. For example, the CPU 21 of the information processing device 20 accesses the second acquired data and determines the second state change point from the time detected as the first state change point (date and time of acquisition (measurement) of information regarding the user's state). Extract information regarding the user's status acquired within the period up to the time detected as . Then, the CPU 21 performs a first learning process based on machine learning using the extracted information regarding the user's status (information regarding the transition of the information regarding the user's status within a predetermined period) using the information regarding the user's status as the first learning data. By inputting it into the model, the user's behavior pattern within the relevant period is estimated.

An example of the first learning data is shown in FIG. The first learning data shown in FIG. ” → “Standing” → “Walking” → “Standing” → “Standing”, “Walking” → “Going up the stairs” → “Walking” → “Standing”, “Walking” → “Walking” → “Standing” → “Walking” ” → “walking,” etc.), the time transition of information regarding the user's state (for example, at least one of the user's acceleration in three axes directions and angular velocity in three axes directions) is determined based on the behavior pattern. This is data that is described in association with (behavior pattern label). Thereby, as a result of machine learning, a learned model is constructed that shows the relationship between the temporal transition of information regarding the state of a user who performs a predetermined behavior pattern and the behavior pattern. Note that in the example shown in FIG. 8, the behavior pattern is configured to include a plurality of behaviors, but the behavior pattern may be configured to include only one behavior (for example, "walking").

Note that the data described in the first learning data may include data when the same user performs different behavior patterns, or may include data when different users perform the same behavior pattern.

Further, the CPU 21 generates a model (first learned model) used to estimate the user's behavior pattern based on the information regarding the user's condition by machine learning using information regarding the user's condition as first learning data. You can learn it.

In this case, for example, when a predetermined model learning instruction is input using the input unit 27, the CPU 21 may perform model learning using the first learning data shown in FIG. The CPU 21 may perform learning using, for example, a time-series neural network model. Here, as the time-series compatible neural network, for example, RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), which is an advanced type of RNN, etc. can be applied. Further, the CPU 21 may be configured to use, for example, a graph neural network (GNN) model, a convolutional neural network (CNN) model, a support vector machine (SVM) model, a fully connected neural network (FNN) model, a gradient boosting (HGB) model, a wave network Learning may be performed using any one of a plurality of models such as the (WN) model. Further, the CPU 21 may perform learning using any one of a graph convolutional neural network (GCN) model, a graph attention network (GAT) model, and a graph convolution LSTM (GC-LSTM) model, which are derivatives of GNN.

In this way, the CPU 21 inputs information regarding the transition of information regarding the user's state within the period from the first state change point to the second state change point into the first learned model. It is possible to estimate the behavior patterns of users. Furthermore, in this embodiment, for example, each time information regarding the user's state is newly stored in the second acquired data, it is necessary to estimate the user's behavior pattern within a predetermined period from the time when the information is newly stored. Therefore, it is possible to reduce the load on the process of estimating the user's behavior pattern.

In addition, in this embodiment, the case where the CPU 21 learns a model (first learned model) provided in the information processing device 20 and estimates the user's behavior pattern using this first learned model is an example. However, the present invention is not limited to this case. For example, the first trained model used to estimate the user's behavior pattern may be provided in a device other than the information processing device 20. In this case, the CPU 21 inputs information regarding the transition of information regarding the user's state during the period from the first state change point to the second state change point into the first learned model provided in another device. , the behavioral pattern estimated by the first learned model may be received (obtained).

The operating state estimating means 35 has a function of estimating the operating state of the user with respect to the terminal device 10 within a predetermined period based on the transition of information regarding the user's state within the predetermined period.

Further, the operation state estimating means 35 calculates a predetermined period of time based on the transition of information regarding the user's status within a predetermined period and a learned model based on machine learning using the transition of information regarding the user's status as learning data. The operating state of the user with respect to the terminal device 10 within the terminal device 10 may be estimated. In this case, by using the learned model, it is possible to easily estimate the user's operation state on the terminal device 10 within a predetermined period.

Furthermore, the operating state estimating means 35 may estimate the user's operating state for the terminal device 10 within a predetermined period based on the user's behavior pattern within the predetermined period. This makes it possible to estimate the user's operating state of the terminal device 10 within the predetermined period using the user's behavior pattern estimated based on the information regarding the user's state within the predetermined period.

Furthermore, the operating state estimating means 35 may estimate the operating state of the user on the terminal device 10 within a predetermined period when the web page is displayed on the web browser. This makes it possible to estimate the user's operating state when a predetermined web page is displayed on the web browser running on the terminal device 10.

The operation state estimating means 35 also determines the operation state of the user on the terminal device 10 during the period from the first state change point to the second state change point that is the next state change point after the first state change point. may be estimated. In this case, the process of estimating the user's operating state starts at the timing when the state change point is detected, and does not start at any other timing, so for example, the process of estimating the user's operating state starts at all times. Compared to the case where the process is started, it becomes possible to reduce the load on the process of estimating the user's operation state.

Furthermore, the user's operation state may include at least one of a touch state, a scroll state, a swipe state, a flick state, a pinch state, and a drag state of the user on the terminal device 10. Based on information about the user's state, the user's touch state (e.g., number of touch operations, touch operation time, etc.), scroll state (e.g., number of scroll operations, scroll operation time, etc.), swipe state, etc. (e.g., number of swipe operations, time of swipe operation, etc.), flick status (e.g., number of flick operations, time of flick operation, etc.), pinch status (e.g., number of pinch-in and/or pinch-out operations, pinch-in and/or It becomes possible to estimate at least one of the drag state (for example, the number of drag operations, the time of a drag operation, etc.) and the drag state (for example, the number of drag operations, the time of a drag operation, etc.).

The functions of the operation state estimating means 35 are realized, for example, as follows. Note that here, the operation state estimating means 35 calculates from a first state change point (for example, time t1 shown in FIG. 7) to a second state change point that is the next state change point after the first state change point. A case will be described as an example in which the user's operation state within a period up to (for example, time t2 shown in FIG. 7) is estimated using a learned model. For example, when the CPU 21 of the information processing device 20 estimates the user's behavior pattern during the period from the first state change point to the second state change point based on the function of the behavior estimation means 34 described above, By inputting the pattern into a second learned model based on machine learning using the user's behavior pattern as second learning data, the user's operating state within the period is estimated.

An example of the second learning data is shown in FIG. The second learning data shown in FIG. 9 is based on a predetermined operation state (operation state label) of the user on the terminal device 10 (in the illustrated example, “the number of touches on the display unit 16 of the terminal device 10 is X ₀ times (X ₀ > 0) or more", "The number of touches is X ₁ times (0<X ₁ < X ₀ ) _or more and less than X ₀ times", "The number of touches is X ₂ times (0<X _{2 <} , "The number of touches is X _{3 times} (0<X _{3 <} "Lying down" → "Standing" → "Lying down" → "Standing", "Standing" → "Standing" → "Walking" → "Standing" → "Standing", "Walking" → "Going up the stairs" → This data is written in a state in which the data is associated with any of the following: "walking" → "stationary", "walking" → "walking" → "stationary" → "walking" → "walking", etc. Thereby, as a result of machine learning, a learned model indicating the relationship between the user's behavior pattern in a predetermined operating state and the operating state is configured.

Note that in the example shown in FIG. 9, the operating state is indicated by the number of times the display unit 16 is touched on the terminal device 10, but the operating state may be indicated by the time period during which the display unit 16 is touched. In addition, the operation status includes touch status (e.g., number of touch operations, duration of touch operation, etc.), scroll status (e.g., number of scroll operations, time of scroll operation, etc.), and swipe status (e.g., number of swipe operations, time of scroll operation, etc.). swipe operation time, etc.), flick status (e.g., number of flick operations, flick operation time, etc.), pinch status (e.g., number of pinch-in and/or pinch-out operations, pinch-in and/or pinch-out operation time, etc.) and the drag state (for example, the number of drag operations, the duration of drag operations, etc.). Furthermore, the operation status includes information indicating the degree of operation on the terminal device 10 (for example, "frequently operated (the number of operations such as touching is very large or the operation time is very long)", "frequently operated" "The number of touch operations is high or the operation time is long.", "The number of touch operations is small or the operation time is short.", "The number of touch operations is small or the operation time is short." The number of operations is very small or the operation time is very short), etc.).

Note that the data described in the second learning data may include operation states when the same user performs different behavior patterns, or may include operation states when different users perform the same behavior pattern. good.

Further, the CPU 21 may learn a model (second learned model) used to estimate the user's operation state based on the behavior pattern by machine learning using the behavior pattern as second learning data.

In this case, for example, when a predetermined model learning instruction is input using the input unit 27, the CPU 21 may perform model learning using the second learning data shown in FIG. The CPU 21, for example, similarly to the first trained model described above, uses a time series neural network model, a GNN model, a CNN model, an SVM model, an FNN model, an HGB model, a WN model, a GCN model, a GAT model, and a convolution LSTM. Learning may be performed using any one of a plurality of models such as the (GC-LSTM) model.

In this way, the CPU 21 inputs the user's behavior pattern during the period from the first state change point to the second state change point into the second learned model, thereby determining the user's operation state during the period. can be estimated. Furthermore, in this embodiment, for example, each time information regarding the user's state is newly stored in the second acquired data, it is necessary to estimate the user's operating state within a predetermined period from the time when the information is newly stored. Therefore, it is possible to reduce the load on the process of estimating the user's operation state.

In addition, in this embodiment, the CPU 21 learns a model provided in the information processing device 20 (second learned model), and uses this second learned model to estimate the user's behavior pattern as an example. However, the present invention is not limited to this case. For example, the second trained model used to estimate the user's operating state may be provided in a device other than the information processing device 20. In this case, the CPU 21 inputs the user's behavior pattern within the period from the first state change point to the second state change point into a second learned model provided in another device, and The user's operation state estimated by the model may be received (obtained).

Moreover, when the CPU 21 estimates the user's operating state within a predetermined period, the CPU 21 may store the information regarding the web page and the user's operating state in a manner that they are associated with each other. In this case, when the CPU 21 estimates the user's behavior within the predetermined period, the CPU 21 selects the information regarding the web page whose acquisition date and time are included within the predetermined period among the information regarding the web page corresponding to the user (terminal device ID). 1 Extract from the acquired data. Then, the CPU 21 stores the extracted information regarding the Web page and the user's operation state in a state where they are associated with each other in the estimation data shown in, for example, FIG. 10 . The estimated data is data in which information about a web page and a user's operation state are described in association with each other for each identification information of the terminal device 10 (in the illustrated example, the terminal device ID). Here, the access period in the estimated data shown in FIG. 10 indicates the period during which the corresponding web page is continuously displayed on the web browser running on the user's terminal device 10. Furthermore, in the example shown in FIG. 10, one display state, one connection state, and one operation state are associated with one access period, but among the display state, connection state, and operation state in one access period, Since at least one of them may change, a plurality of display states, connection states, and operation states may be associated with one access period.

In this case, based on the estimated data, it is possible to understand the user's operating state when a predetermined web page is displayed on the web browser running on the terminal device 10. Furthermore, based on the estimated data, it is possible to understand in what operating state the user is viewing the web page, as well as in what state the web page is being displayed. Furthermore, based on the estimated data, it is possible to understand in what operating state the user is viewing the web page, as well as whether the web page is being displayed online.

The control means 36 has a function of controlling the operation of the terminal device 10 according to the user's operating state of the terminal device 10 within the estimated predetermined period. As a result, by controlling the terminal device 10, for example, by reducing the brightness of the display unit 16 in order to reduce the power consumption of the terminal device 10 when the user is not operating the terminal device 10 much. , it becomes possible to appropriately operate the terminal device 10 according to the user's operating state.

The functions of the control means 36 are realized, for example, as follows. When the CPU 21 of the information processing device 20 estimates the user's operating state based on the function of the operating state estimating means 35, it accesses the control data shown in FIG. 11 and extracts the control content corresponding to the estimated operating state. . The control data includes the estimated operation state (in the example shown in the figure, "the number of touches is X ₀ or more", "the number of touches is X ₁ or more ₀ times, etc.) and the control details of the terminal device 10 (in the example shown in the figure, "start a predetermined application (for example, a map application, etc.)", "reduce the brightness of the display unit 16", etc.) are associated. The data is written in a state that is For example, if the operation state estimated when a predetermined web page is displayed on the web browser of the terminal device 10 is "the number of touches is X ₀ or more", the CPU 21 displays "the number of touches is X ₀ or more" The control content (here, "start a predetermined application") corresponding to the action "above" is extracted from the control data. Then, the CPU 21 sends instruction information (commands) for operating the terminal device 10 with the extracted control content (here, "start a predetermined application") to the terminal device via the communication interface unit 28 and the communication network NW. Send to 10.

On the other hand, upon receiving the instruction information from the information processing device 20, the CPU 11 of the terminal device 10 operates based on the received instruction information (here, starts a predetermined application). In this way, the terminal device 10 can be operated appropriately according to the user's operating state.

Here, an example has been described in which the control data is described in such a manner that the estimated operation state and the control content of the terminal device 10 are associated with each other for each identification information of the web page. The data may be written in such a manner that the control details of the terminal device 10 are associated with each estimated operation state (that is, the identification information of the web page does not need to be associated).

The providing means 37 has a function of providing information according to the user's operating state of the terminal device 10 within the estimated predetermined period. As a result, appropriate information is provided depending on the user's operating state, such as providing advertising information to the user via the terminal device 10 when the user frequently operates the terminal device 10. becomes possible.

The functions of the providing means 37 are realized, for example, as follows. When the CPU 21 of the information processing device 20 estimates the user's operating state based on the function of the operating state estimating means 35, it accesses the provided data shown in FIG. 12 and extracts provided information corresponding to the estimated operating state. . The provided data is based on the identification information of the web page (for example, the URL of the web page), and the estimated operating state (in the example shown, "the number of touches is X ₀ or more", "the number of touches is X ₁ or more") This is data in which provided information (in the illustrated example, "questionnaire information", "advertisement _information ", etc.) are associated with each other. For example, if the operation state estimated when a predetermined web page is displayed on the web browser of the terminal device 10 is "questionnaire information," the CPU 21 sends the provided information corresponding to the action "questionnaire information" ( Here, "questionnaire information") is extracted from the provided data. Then, the CPU 21 transmits the extracted provision information to the terminal device 10 via the communication interface unit 28 and the communication network NW.

On the other hand, upon receiving the provided information from the information processing device 20, the CPU 11 of the terminal device 10 may display the received provided information on the display unit 16, for example. Further, when the provided information is composed of audio data, the CPU 11 may output the provided information from an audio output device such as a speaker. In this way, appropriate information can be provided depending on the user's operating state.

Note that here, an example has been described in which the provided data is described in a state where the estimated operation state and the provided information are associated with each other for each identification information of the web page. The provided information may be described in association with each operating state (that is, the identification information of the web page may not be associated with it).

The changing unit 38 has a function of changing the display mode of the web page displayed on the web browser according to the user's operating state of the terminal device 10 within the estimated predetermined period. This makes it possible to dynamically change the design, layout, etc. of a web page, for example, depending on the user's operating state when the web page is displayed on the web browser.

The function of the changing means 38 is realized, for example, as follows. When the CPU 21 of the information processing device 20 estimates the user's operating state based on the function of the operating state estimating means 35, the CPU 21 acquires the web page displayed on the web browser on the terminal device 10 from, for example, a web server (not shown). However, the web page may be changed by changing the design, layout, etc. of the acquired web page according to the estimated operating state. Here, information regarding the design, layout, etc. of the web page corresponding to the estimated user operation state may be stored in the storage device 24, for example. Then, the CPU 21 transmits the changed web page to the terminal device 10 via the communication interface section 28 and the communication network NW.

On the other hand, upon receiving the changed web page from the information processing device 20, the CPU 11 of the terminal device 10 may display the changed web page on the display unit 16. FIG. 13 shows an example of a display mode of a web page displayed on the terminal device 10. FIG. 13(a) is a diagram showing an example of the layout of a web page displayed on a web browser when the estimated user operation state is "the number of touches is X ₀ times or more", and FIG. 13(b) is a diagram showing an example of the layout of a web page displayed on a web browser when the estimated user operation state is "the number of touches is X ₁ or more and less than X ₀ times". For example, if the user's operation state is estimated to be "the number of touches is X ₀ or more", as shown in FIG. 13(a), areas A, B, C, and D where text, videos, etc. The displayed web page is now displayed. Here, the layout is such that area A is displayed at the top of the screen, area B is displayed at the bottom left of the screen, and areas C and D are displayed at the bottom right of the screen. On the other hand, if it is estimated that the user's operation status is "the number of touches is X ₁ or more and less than X ₀ ", as shown in FIG. may be the same as the content displayed in FIG. 13(a)) is displayed. Here, the layout is such that area D is displayed at the top of the screen, area C is displayed at the center of the screen, and area A is displayed at the bottom of the screen. In this way, the display mode of the web page can be dynamically changed according to the user's operating state when the web page is displayed on the web browser.

(5) Flow of main processing of the information processing system of this embodiment Next, an example of the flow of main processing performed by the information processing system of this embodiment will be described with reference to the flowchart of FIG. 14.

The CPU 21 of the information processing device 20 obtains information regarding the state of the user of the terminal device 10 measured by the measuring device 18 based on the function of the state information obtaining means 32 (step S100). Here, the CPU 21 may acquire information regarding the user's state measured by the measuring device 18 while the web page is displayed on the web browser. Further, the information regarding the user's state may include at least one of the user's acceleration and the user's angular velocity.

Next, the CPU 21 of the information processing device 20 determines the user's operating state for the terminal device 10 within a predetermined period based on the transition of information regarding the user's state within the predetermined period based on the function of the operating state estimating means 35. Estimate (step S102). Here, the CPU 21 determines whether or not the terminal has been used within a predetermined period based on the transition of information regarding the user's status within a predetermined period and the learned model based on machine learning using the transition of information regarding the user's status as learning data. The operating state of the user with respect to the device 10 may be estimated. Further, the CPU 21 may estimate the user's operating state of the terminal device 10 within the predetermined period based on the user's behavior pattern within the predetermined period. Further, the CPU 21 may estimate the user's operating state of the terminal device 10 within a predetermined period when the web page is displayed on the web browser. Furthermore, the CPU 21 estimates the operating state of the user on the terminal device 10 during the period from the first state change point to the second state change point that is the next state change point after the first state change point. It's okay.

Note that the CPU 21 of the information processing device 20 transmits data including the estimated user operation state (estimated data shown in FIG. 10) to an external device (for example, personalized information) via the communication interface unit 28 and the communication network NW. The information may be transmitted (provided) to a computer (for example, a server) that generates and provides the information.

On the other hand, on the service provider side that generates and provides personalized information, based on the estimated data sent from the information processing device 20, the service provider side calculates, for example, in what operating state the user is viewing the web page. It becomes possible to understand. Furthermore, by analyzing the estimated data, it becomes possible to generate personalized information according to the user's operating state.

As described above, according to the information processing system, information processing method, and program of the present embodiment, the operation state of the user on the terminal device 10 within the predetermined period (for example, Since it is estimated how often the terminal device 10 is operated or whether it is possible to operate the terminal device 10, personalization can be performed according to the user's operating state by, for example, analyzing the estimated operating state of the user. It becomes possible to generate information based on Thereby, information (user operation status) applicable to generating personalized information can be obtained.

Further, according to the information processing system, information processing method, and program of this embodiment, since the measuring device 18 is provided in the terminal device 10, the measuring device 18 provided in the terminal device 10 is used to determine the state of the user. It becomes possible to obtain information regarding.

Modifications of the above-described embodiment will be described below.
(Modified example)
In the embodiment described above, the operation state estimating means 35 determines whether the user Although the case of estimating the operating state of is described as an example, the present invention is not limited to this case. For example, when the transition of information regarding the user's state within a predetermined period satisfies a predetermined condition corresponding to a predetermined operating state, the operating state estimating means 35 determines that the predetermined operating state is the user's operation on the terminal device 10. It may be estimated as a state. Here, the predetermined condition may be, for example, that the transition of information regarding the user's state within a predetermined period is included in the range of information regarding the user's state corresponding to a predetermined operation state.

In this case, the CPU 21 of the information processing device 20 extracts information regarding the user's state stored in the second acquired data within a predetermined period (information regarding the transition of information regarding the user's state within the predetermined period) from the second acquired data. Then, the user's operating state may be estimated by referring to the operating state data shown in FIG. The operation status data includes multiple operation statuses (in the example shown in the figure, the number of touches is X ₀ or more, the number of touches is X ₁ or more and less than X ₀ , and the number of touches is X ₂ or more and X ₁ . "The number of touches is more than X ₃ times and less than ₂ times," etc.) is associated with the time transition of the range of information related to the corresponding user's state (in the example shown in the figure, the range of 3-axis acceleration). This is data described in the state. The operation state data may be stored in the storage device 24, for example.

For example, when the time transition of the information about the user's state extracted from the second acquired data is included in the range of the information about the user's state corresponding to any operation state in the operation state data, the CPU 21 This operation state (for example, "the number of touches is X ₀ times or more") may be estimated as the user's operation state.

Further, the behavior estimation means 34 uses data (not shown) configured similarly to the operation state data shown in FIG. satisfies a predetermined condition corresponding to a predetermined behavior pattern (for example, the transition of information regarding the user's status within a predetermined period is included in the range of information regarding the user's status corresponding to the predetermined behavior pattern) In addition, it may be estimated that the user is performing the predetermined behavior pattern.

Thus, according to the information processing system, information processing method, and program according to this modification, it is possible to exhibit the same effects as the above-described embodiment.

Note that the program of the present invention may be stored in a computer-readable storage medium. The storage medium in which this program is recorded may be the ROM 12, RAM 13, or storage device 14 of the terminal device 10 shown in FIG. 2, or the ROM 22, RAM 23, or storage device 24 of the information processing device 20 shown in FIG. Good too. Further, the storage medium may be, for example, a CD-ROM or the like that can be read by being inserted into a program reading device such as a CD-ROM drive. Furthermore, the storage medium may be a magnetic tape, a cassette tape, a flexible disk, an MO/MD/DVD, or a semiconductor memory.

The embodiments and modifications described above are described to facilitate understanding of the present invention, and are not described to limit the present invention. Therefore, each element disclosed in the above embodiments and modifications is intended to include all design changes and equivalents that fall within the technical scope of the present invention.

For example, in the embodiment described above, the case where the measuring device 18 is provided in the terminal device 10 has been described as an example, but the present invention is not limited to this case. For example, the measuring device 18 may be provided while being worn by the user, or may be provided at a different position from the user and the terminal device 10. In addition, when the measuring device 18 is provided while being worn by the user, the acceleration, angular velocity, etc. of at least one body part (for example, arm, leg, etc.) of the user is acquired as information regarding the user's condition. may be done.

Furthermore, in the embodiment described above, the CPU 21 of the information processing device 20, as a function of the operation state estimating means 35, calculates the operation state of the user with respect to the terminal device 10 within a predetermined period based on the user's behavior pattern within the predetermined period. Although the case of estimation has been described as an example, the present invention is not limited to this case. For example, the CPU 21 inputs the transition of information regarding the user's status within a predetermined period into a third learned model based on machine learning that uses the transition of information regarding the user's status as third learning data. The operating state of the user within the system may be estimated.

An example of the third learning data is shown in FIG. 16. The third learning data shown in FIG. 16 is data in which the transition of information regarding the user's state and the user's operation state (operation state label) are described in a correlated manner. Thereby, as a result of machine learning, a third learned model is configured that shows the relationship between the transition of information regarding the user's state and the user's operation state. In addition, the CPU 21 uses a model (a third learned model) used to estimate the user's operation state based on the information about the user's state by machine learning using the transition of information about the user's state as third learning data. ) may be studied.

Furthermore, in the embodiment described above, the case where the operating state of the terminal device 10 by one user is estimated is explained as an example, but the operating state of the user with respect to the terminal device 10 owned by each of a plurality of users is estimated. You can.

Furthermore, in the embodiment described above, the case where a behavior pattern including one or more of "standing still", "walking", "lying down", and "climbing stairs" is estimated is explained as an example; The contents of the actions included in the pattern are not limited to these.

Further, in the embodiment described above, the case where one information processing device 20 is provided has been described as an example, but the present invention is not limited to this case. For example, a plurality of information processing devices 20 may be provided, and in this case, the operation contents, processing results, etc. on any information processing device 20 may be presented in real time on other information processing devices 20. Alternatively, the processing results of any one of the information processing apparatuses 20 may be shared among the plurality of information processing apparatuses 20.

Furthermore, in the embodiment described above, the information processing device 20 performs the page information acquisition means 31, the state information acquisition means 32, the detection means 33, the behavior estimation means 34, the operation state estimation means 35, the control means 36, the provision means 37, and the change Although the configuration is such that each function of the means 38 is realized, the present invention is not limited to this configuration. For example, a computer or the like (for example, a general-purpose personal computer, a server computer, etc.) communicatively connected to the information processing apparatus 20 via a communication network NW such as the Internet or a LAN may perform at least one of the means 31 to 37. It is also possible to adopt a configuration that realizes the functions of two means. Moreover, each function of the functional block diagram shown in FIG. 4 may be arbitrarily shared between the terminal device 10 and the information processing device 20, as shown in FIGS. 17(a) and 17(b).

For example, the CPU 11 of the terminal device 10 may estimate the user's behavior pattern based on information regarding the user's state, or may estimate the user's operating state based on the estimated behavior pattern. In this case, the CPU 11 may learn the first learned model and/or the second learned model in the embodiment described above. In this case, the CPU 11 may perform machine learning using, for example, TensorFlow (registered trademark), which is an open source machine learning library. This allows the terminal device 10 to estimate the user's operating state in real time. Furthermore, since the estimation process of the user's operation state can be performed within the terminal device 10, for example, information regarding the user's state or information regarding the user's behavior pattern is suppressed from being transmitted to the outside of the terminal device 10. can do. Therefore, it is possible to estimate the user's operating state while protecting the user's privacy.

The information processing system, information processing method, and program of the present invention as described above can be suitably used for information provision services that provide users with information personalized for each user, so that their industrial applications can be improved. The gender is extremely large.

10...Terminal device 18...Measuring device 20...Information processing device 31...Page information acquisition means 32...Status information acquisition means 33...Detection means 34...Behavior estimation means 35...Operation information estimation means 36...Control means 37...Providing means 38... means of change

Claims (18)

An information processing system that estimates a user's operation state on a terminal device,
a status information acquisition means for acquiring information regarding the user's status measured by a measuring device;
an operation state estimating means for estimating the operation state of the user with respect to the terminal device within the predetermined period based on the transition of information regarding the user's state within the predetermined period;
Information processing system. The operating state estimating means is configured to estimate the operating state based on a change in information regarding the user's state within the predetermined period and a learned model based on machine learning using the change in information regarding the user's state as learning data. The information processing system according to claim 1, wherein an operation state of the user on the terminal device within a predetermined period is estimated. comprising behavior estimation means for estimating the behavior pattern of the user within the predetermined period based on the transition of information regarding the state of the user within the predetermined period;
The information processing system according to claim 1 or 2, wherein the operation state estimation means estimates the operation state of the user with respect to the terminal device within the predetermined period based on the behavior pattern of the user within the predetermined period. . The behavior estimation means is configured to calculate the predetermined behavior based on the transition of information regarding the user's status within the predetermined period and a learned model based on machine learning using the transition of the information regarding the user's status as learning data. The information processing system according to claim 3, which estimates the user's behavior pattern within a period. comprising a detection means for detecting a state change point indicating a timing at which the operation state of the user with respect to the terminal device changes, based on information regarding the state of the user;
The behavior estimation means estimates the behavior pattern of the user within a period from a first state change point to a second state change point that is a next state change point after the first state change point. 4. The information processing system according to 3 or 4. comprising a detection means for detecting a state change point indicating a timing at which the operation state of the user with respect to the terminal device changes, based on information regarding the state of the user;
The operation state estimating means determines the operation state of the user with respect to the terminal device within a period from a first state change point to a second state change point that is the next state change point after the first state change point. The information processing system according to any one of claims 1 to 5, which estimates. The detection means detects the state change point by determining whether the predetermined timing is the state change point when information regarding the user's state at the predetermined timing satisfies a predetermined condition. , The information processing system according to claim 5 or 6. 8. The information processing system according to claim 7, wherein the predetermined condition includes that a differential value of a curve indicating a transition of information regarding the user's state is 0. The detection means determines that the predetermined timing is the state change point when a change point score indicating a change in information regarding the user's state at the predetermined timing is equal to or greater than a predetermined value. The information processing system according to item 7 or 8. The information processing system according to any one of claims 1 to 9, further comprising a control means for controlling the operation of the terminal device according to the user's operating state of the terminal device within the estimated predetermined period. The information processing system according to any one of claims 1 to 10, further comprising providing means for providing information according to the user's operation state on the terminal device within the estimated predetermined period. comprising page information acquisition means for acquiring information about a web page displayed on a web browser running on the terminal device,
The status information acquisition means acquires information regarding the user's status measured by the measuring device when the web page is displayed on the web browser,
12. The operating state estimating means estimates the operating state of the user with respect to the terminal device within the predetermined period when the web page is displayed on the web browser. Information processing system. 13. The information processing system according to claim 12, further comprising a changing unit that changes a display mode of a web page displayed on the web browser according to an estimated operation state of the user on the terminal device within the predetermined period. . The information processing system according to any one of claims 1 to 13, wherein the information regarding the user's state includes at least one of the user's acceleration and the user's angular velocity. The operation state of the user includes at least one of a touch state, a scroll state, a swipe state, a flick state, a pinch state, and a drag state of the user with respect to the terminal device. Information processing system. 16. The information processing system according to claim 1, wherein the measuring device is provided in the terminal device. An information processing method for estimating a user's operating state of a terminal device using a computer, the method comprising:
The computer,
obtaining information regarding the user's state measured by a measuring device;
estimating the operating state of the user with respect to the terminal device within the predetermined period based on the transition of information regarding the user's state within the predetermined period;
Execute each step of
Information processing method. A program for a computer to estimate a user's operating state with respect to a terminal device,
to the computer;
a function of acquiring information regarding the user's state measured by a measuring device;
a function of estimating the operation state of the user with respect to the terminal device within the predetermined period based on the transition of information regarding the state of the user within the predetermined period;
A program to make this happen.

Images