JP2022105418A - Information processing system, information processing method, and program - Google Patents

Abstract

To provide an information processing system, an information processing method, and a program capable of providing information applicable to generate personalized information.SOLUTION: An information processing system includes first acquisition means 31 for acquiring information on a Web page displayed in a Web browser executed on a terminal device, second acquisition means 32 for acquiring information on a state of a user of the terminal device measured by a prescribed measurement device when the Web page is displayed in the Web browser, estimation means 33 for estimating action of the user when the Web page is displayed in the Web browser on the basis of the acquired information on the state of the user, and storage means 34 for storing the information on the Web page and the action of the user in association with each other in a prescribed storage device.SELECTED DRAWING: Figure 4

Description

The present invention relates to an information processing system, an information processing method, and a program.

In recent years, with the spread of terminal devices such as mobile terminals, a function that collects information on the user of the terminal device and provides the user with personalized information for each user (for example, a search engine whose results change depending on the user's search history and position). An information providing service having a function, a recommendation function for displaying recommended products according to browsing history and purchase history, etc.) has been put into practical use.

For example, in the technique described in Patent Document 1, an item related to an item is recommended to a user who has taken an action on the item.

Japanese Unexamined Patent Publication No. 2011-154591

In such an information providing service, information such as a user's search history, location, or Web page browsing / purchasing history is analyzed, and personalized information is generated according to the analysis result. .. On the other hand, in order to generate more personalized information, it is necessary to perform analysis using more appropriate information.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an information processing system, an information processing method, and a program capable of providing information applicable to generate personalized information. do.

In order to solve the above problems, firstly, the present invention has a first acquisition means for acquiring information about a Web page displayed on a Web browser executed on a terminal device, and the Web page is used as the Web browser. Based on the second acquisition means for acquiring information on the user's state of the terminal device measured by a predetermined measuring device when displayed and the acquired information on the user's state, the Web page is the Web. An estimation means for estimating the behavior of the user when displayed on a browser, and a storage means for storing information about the Web page and the behavior of the user in a predetermined storage device in association with each other. Provided is an information processing system provided (Invention 1).

According to the present invention (Invention 1), the information about the Web page displayed on the Web browser executed on the terminal device and the user's behavior when the Web page is displayed on the Web browser are associated with each other. Since it is stored in a state of being stored, it is possible to grasp what kind of action the user is browsing the Web page based on the stored information. In addition, by analyzing the stored information, it is possible to generate personalized information according to the user's behavior, so it is necessary to provide information applicable to generate personalized information. Can be done.

In the above invention (invention 1), a control means for controlling the operation of the terminal device according to the estimated behavior of the user may be provided (invention 2).

According to the invention (Invention 2), for example, a Web page is created by controlling the terminal device such as activating the camera function of the terminal device when the user's behavior of holding the terminal device is estimated. Can be appropriately operated according to the user's behavior when is displayed on the Web browser.

In the above inventions (Inventions 1 and 2), a providing means for providing information according to the estimated behavior of the user may be provided (Invention 3).

According to the invention (Invention 3), a Web page is displayed on a Web browser, for example, when the user's behavior of walking is estimated, advertisement information is provided to the user via a terminal device. It becomes possible to provide appropriate information according to the user's behavior when the information is being executed.

In the above inventions (Inventions 1 to 3), a changing means for changing the display mode of the Web page displayed on the Web browser may be provided according to the estimated behavior of the user (Invention 4).

According to the present invention (Invention 4), for example, when the user's behavior changes from rest to walking, the design and layout of the Web page can be changed, so that the user when the Web page is displayed on the Web browser. It becomes possible to dynamically change the display mode of the Web page according to the behavior of.

In the above inventions (Inventions 1 to 4), the estimation means is based on the acquired information on the user's state and a trained model based on machine learning using the information on the user's state as learning data. , The user's behavior may be estimated (Invention 5).

According to the invention (Invention 5), by using the trained model, it is possible to easily estimate the user's behavior when the Web page is displayed on the Web browser.

In the above inventions (Inventions 1 to 5), the information regarding the Web page may include information regarding the display state of the Web page on the terminal device (Invention 6).

Here, the information regarding the display state of the Web page includes, for example, a state in which the Web page is displayed on the screen of the terminal device (foreground), a state in which the Web page is not displayed on the screen of the terminal device (background), and the like. A state in which the Web page is pre-rendered (pre-rendered) may be included.

According to the present invention (Invention 6), it is possible to grasp what kind of behavior the user is browsing the web page and in what state the web page is displayed. Therefore, by analyzing the stored information or the like, it is possible to generate more personalized information according to the user's behavior.

In the above inventions (Inventions 1 to 6), the information regarding the Web page may include information regarding whether or not the Web page is displayed online (Invention 7).

According to the present invention (Invention 7), it is possible to grasp what kind of behavior the user is browsing the Web page and whether or not the Web page is displayed online. By analyzing the stored information, it is possible to generate more personalized information according to the user's behavior.

In the above inventions (Inventions 1 to 7), the information regarding the state of the user may include at least one of the acceleration of the user and the angular velocity of the user (Invention 8).

According to the invention (8), the user when the web page is displayed on the web browser is based on at least one of the acceleration and the angular velocity of the user when the web page is displayed on the web browser. Behavior can be estimated.

In the above inventions (Inventions 1 to 8), the predetermined measuring device may be provided in the terminal device (Invention 9).

According to the present invention (Invention 9), it is possible to acquire information on the state of the user by using the measuring device provided in the terminal device.

Secondly, the present invention comprises a step in which a computer acquires information about a web page displayed on a web browser running on a terminal device, and a predetermined step when the web page is displayed on the web browser. Based on the step of acquiring the information about the user's state of the terminal device measured by the measuring device and the acquired information about the user's state, the user's when the Web page is displayed on the Web browser. Provided is an information processing method for executing each step of estimating an action, storing information about the Web page in a predetermined storage device in a state of associating the information with the user's action, and executing each step. Invention 10).

Thirdly, the present invention has a function of acquiring information about a web page displayed on a web browser running on a terminal device on a computer, and a predetermined method when the web page is displayed on the web browser. Based on the function to acquire the information about the user's state of the terminal device measured by the measuring device and the acquired information about the user's state, the user's when the Web page is displayed on the Web browser. Provided is a program for realizing a function of estimating an action, a function of storing information about the Web page and the action of the user in a predetermined storage device in a corresponding state (Invention 11).

According to the information processing system, information processing method, and program of the present invention, it is possible to provide information applicable to generate personalized information.

It is a figure which shows schematic the basic structure of the information processing system which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of a terminal apparatus. It is a block diagram which shows the structure of an information processing apparatus. It is a functional block diagram for explaining the function which plays a main role in an information processing system. It is a figure which shows the structural example of the 1st acquisition data. It is a figure which shows the structural example of the 2nd acquisition data. It is a figure which shows the structural example of the training data. It is a figure which shows the structural example of the 1st provided data. It is a figure which shows the structural example of the control data. It is a figure which shows the structural example of the 2nd provided data. (A) is a diagram showing an example of the layout of a Web page displayed on a Web browser when the estimated user behavior is "walking", and (b) is a diagram showing an example of the layout of the Web page where the estimated user behavior is "horizontal". It is a figure which shows an example of the layout of the Web page displayed in the Web browser in the case of "becomes". It is a flowchart which shows an example of the main processing of the information processing system which concerns on one Embodiment of this invention. It is a figure which shows the composition example of the behavior data. It is a figure which shows the example of sharing between a terminal apparatus and an information processing apparatus about each function of an information processing system.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. However, this embodiment is an example, and the present invention is not limited thereto.

(1) Basic Configuration of Information Processing System FIG. 1 is a diagram schematically showing a basic configuration of an information processing system according to an embodiment of the present invention. As shown in FIG. 1, in the information processing system according to the present embodiment, when the user is executing a Web browser (for example, Google Chrome (registered trademark)) on the terminal device 10, the information is displayed on the Web browser. The information processing device 20 acquires the information about the Web page and the information about the user's state measured by the predetermined measuring device 18 (shown in FIG. 2) when the Web page is displayed on the Web browser. It has become. The terminal device 10 and the information processing device 20 are connected to a communication network NW (network) such as the Internet or a LAN (Local Area Network).

The terminal device 10 may be, for example, a device that can be worn by the user (for example, a wearable device) or a portable device that can be possessed by the user. Further, the terminal device 10 includes, for example, a mobile terminal, a smartphone, a PDA (Personal Digital Assistant), a personal computer, a television receiver having a bidirectional communication function (including a so-called multifunctional smart television), and the like. As such, it may be a device operated by an individual user.

The information processing device 20 is a device for providing information applicable to generate personalized information. The information processing device 20 estimates the user's behavior when the web page is displayed on the web browser based on the information about the user's state acquired from the terminal device 10. Further, the information processing device 20 stores the information about the Web page acquired from the terminal device 10 and the estimated user behavior in a predetermined storage device in a state of associating them with each other. The information processing device 20 may be a device operated by an individual user, similarly to the terminal device 10.

(2) Configuration of Terminal Device The configuration of the terminal device 10 will be described with reference to FIG. FIG. 2 is a block diagram showing an internal configuration of the terminal device 10. As shown in FIG. 2, the terminal device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage device 14, and a display processing unit 15. A display unit 16, an input unit 17, a measuring device 18, and a communication interface unit 19 are provided, and a bus 10a for transmitting a control signal or a data signal between the units is provided.

When the power is turned on to the terminal device 10, the CPU 11 loads various programs stored in the ROM 12 or the storage device 14 into the RAM 13 and executes them.

The storage device 14 is, for example, a non-volatile storage device such as a flash memory, an SSD (Solid State Drive), a magnetic storage device (for example, HDD (Hard Disk Drive), floppy disk (registered trademark), magnetic tape, etc.), an optical disk, or the like. It may be a volatile storage device such as a RAM, and stores a program executed by the CPU 11 and data referred to by the CPU 11.

The display processing unit 15 displays the display data given by the CPU 11 on the display unit 16. The display unit 16 is, for example, an LCD (Liquid Crystal Display) monitor including thin film transistors arranged in a matrix in pixel units, and by driving the thin film transistors based on the display data, the displayed data is displayed on the display screen. indicate.

When the terminal device 10 is a button input type communication device, the input unit 17 includes a button group including a plurality of instruction input buttons such as a direction instruction button and a decision button for accepting a user's operation input, a ten key, and the like. It includes a button group including a plurality of instruction input buttons, and includes an interface circuit for recognizing a press (operation) input of each button and outputting it to the CPU 11.

When the terminal device 10 is a touch panel input type communication device, the input unit 17 mainly accepts touch panel type input by touching the display screen with a fingertip or a pen. The touch panel input method may be a known method such as a capacitance method.

Further, when the terminal device 10 is a device capable of inputting voice, the input unit 17 may be configured to include a microphone for voice input, or voice data input via an external microphone. May be provided with an interface circuit for outputting the data to the CPU 11. Further, when the terminal device 10 is a communication device capable of inputting a moving image and / or a still image, the input unit 17 may be configured to include a digital camera or a digital video camera for image input. An interface circuit for accepting image data captured by an external digital camera or a digital video camera and outputting the image data to the CPU 11 may be provided.

The measuring device 18 is a device that continuously or intermittently (for example, every predetermined interval (for example, 300 milliseconds, 1 second, etc.)) measures the state of the user of the terminal device 10. Here, the information regarding the user's state may be, for example, a value representing the user's state or a value obtained by substituting the value representing the user's state into a predetermined calculation formula. It may be information indicating the degree of the state of the user.

The measuring device 18 is, for example, a user's state (for example, acceleration in the triaxial or biaxial direction, angular velocity in the triaxial or biaxial direction, heart rate (pulse), blood pressure, body temperature, sweating amount, step count, walking speed). , Posture, exercise intensity (eg heart rate ÷ maximum heart rate) or calories burned (eg, accelerometer, gyro sensor, motion sensor, heart rate monitor, blood pressure monitor, thermometer, sweat meter, etc.) You may.

The communication interface unit 19 includes an interface circuit for communicating with another device (for example, the information processing device 20) via the communication network NW.

(3) Configuration of Information Processing Device The configuration of the information processing device 20 will be described with reference to FIG. FIG. 3 is a block diagram showing an internal configuration of the information processing apparatus 20. As shown in FIG. 3, the information processing device 20 includes a CPU 21, a ROM 22, a RAM 23, a storage device 24, a display processing unit 25, a display unit 26, an input unit 27, and a communication interface unit 28. A bus 20a for transmitting a control signal or a data signal between each unit is provided.

When the power is turned on to the information processing device 20, the CPU 21 loads various programs stored in the ROM 22 or the storage device 24 into the RAM 23 and executes them. In the present embodiment, the CPU 21 reads and executes a program stored in the ROM 22 or the storage device 24, so that the first acquisition means 31, the second acquisition means 32, the estimation means 33, the storage means 34, and the control means, which will be described later, are executed. 35, the functions of the providing means 36 and the changing means 37 (shown in FIG. 4) are realized.

The storage device 24 may be, for example, a non-volatile storage device such as a flash memory, SSD, a magnetic storage device (for example, HDD, floppy disk (registered trademark), magnetic tape, etc.), an optical disk, or the like, or volatilization of the RAM or the like. It may be a sex storage device, and stores a program executed by the CPU 21 and data referred to by the CPU 21. Further, in the storage device 24, the first acquired data (shown in FIG. 5), the second acquired data (shown in FIG. 6), the learning data (shown in FIG. 7), the control data (shown in FIG. 8), which will be described later, are stored in the storage device 24. The first provided data (shown in FIG. 9) and the second provided data (shown in FIG. 10) are stored.

The input unit 27 may be an information input device such as a mouse or a keyboard, may be configured to include a microphone for voice input, or may include a digital camera or a digital video camera for image input. It may be configured in the above, or it may be provided with an interface circuit for receiving image data captured by an external digital camera or a digital video camera and outputting it to the CPU 21.

The communication interface unit 28 includes an interface circuit for performing communication via the communication network NW. The details of the other parts in the information processing apparatus 20 may be the same as those of the terminal apparatus 10.

(4) Outline of Each Function in the Information Processing System The functions realized by the information processing system of the present embodiment will be described with reference to FIG. FIG. 4 is a functional block diagram for explaining a function that plays a major role in the information processing system of the present embodiment. In the functional block diagram of FIG. 4, the first acquisition means 31, the second acquisition means 32, the estimation means 33, and the storage means 34 correspond to the main configurations of the information processing system of the present invention. The other means (control means 35, providing means 36, and changing means 37) are not necessarily essential configurations, but are components for further favoring the present invention.

The first acquisition means 31 has a function of acquiring information about a Web page displayed on a Web browser executed on the terminal device 10.

Here, the information regarding the Web page may include information regarding the display state of the Web page on the terminal device 10. Further, the information regarding the display state of the Web page includes, for example, a state in which the Web page is displayed on the screen (display unit 16) of the terminal device 10 (foreground), and the Web page is the screen of the terminal device 10 (display unit 16). A state in which the Web page is not displayed (background), a state in which the Web page is pre-rendered (pre-rendered), and the like may be included. This makes it possible to grasp what kind of behavior the user is browsing the web page and what state the web page is displayed in, so that the stored information can be stored. By performing analysis or the like, it is possible to generate more personalized information according to the behavior of the user.

Further, the information about the web page may include information about whether or not the web page is displayed online. This makes it possible to understand what kind of behavior the user is browsing the web page and whether or not the web page is displayed online, so the stored information is analyzed. By doing so, it is possible to generate more personalized information according to the user's behavior.

The function of the first acquisition means 31 is realized, for example, as follows. First, the CPU 11 of the terminal device 10 displays on the Web browser every time a predetermined time (for example, 10 seconds or the like) elapses when the Web browser is executed by the user's operation using the input unit 17, for example. Acquires information about the Web page that has been created. Here, the information about the Web page includes the identification information of the Web page (for example, URL (Uniform Resource Locator), etc.), the display state of the Web page (for example, foreground, background, pre-rendering, etc.), and the Web page. It is supposed to include whether or not it is displayed online. Further, the CPU 11 may acquire information about a Web page by executing an application configured by using, for example, Javascript (registered trademark). Then, each time the CPU 11 acquires information about a Web page, the acquired information is transmitted to the information processing apparatus 20 via the communication interface unit 19 and the communication network NW. Here, the information transmitted to the information processing device 20 includes the acquisition date and time of the information related to the Web page and the identification information of the terminal device 10 (for example, the serial number of the terminal device 10 or the MAC (Media Access Control) address). , May be included.

On the other hand, each time the CPU 21 of the information processing apparatus 20 receives (acquires) information about a Web page via the communication interface unit 28, the CPU 21 stores the information about the Web page in, for example, the first acquisition data shown in FIG. The first acquired data is the information related to the Web page (in the example of the figure, the URL of the Web page, the acquisition date and time of the information related to the Web page, the Web page) for each identification information (terminal device ID in the example of the figure) of the terminal device 10. The data is described in a state in which the display state of the above and the connection state of the terminal device 10 (whether or not the Web page is displayed online) are associated with each other. Here, as shown in FIG. 5, the acquisition date and time of the information about the Web page, the display state of the Web page, and the connection state of the terminal device 10 may be associated with each URL of the Web page.

The second acquisition means 32 has a function of acquiring information regarding the state of the user of the terminal device 10 measured by the predetermined measuring device 18 when the Web page is displayed on the Web browser.

Here, the information about the user's state may include at least one of the user's acceleration and the user's angular velocity. Thereby, it is possible to estimate the user's behavior when the web page is displayed on the web browser based on at least one of the acceleration and the angular velocity of the user when the web page is displayed on the web browser. ..

The function of the second acquisition means 32 is realized, for example, as follows. First, when the CPU 11 of the terminal device 10 is executing the Web browser by the operation of the user using the input unit 17, for example, every time a predetermined time (for example, 300 milliseconds, 1 second, etc.) elapses, the CPU 11 of the terminal device 10 elapses. Acquires the user's state (for example, at least one of the user's triaxial or biaxial acceleration and the user's triaxial or biaxial angular velocity). Here, the CPU 11 may acquire information on the user's state by executing an application configured by using, for example, Javascript (registered trademark) to operate the measuring device 18. Then, each time the CPU 11 measures the information about the user's state, the CPU 11 transmits the measured information about the user's state to the information processing apparatus 20 via the communication interface unit 19 and the communication network NW. Here, the information transmitted to the information processing apparatus 20 may include the acquisition (measurement) date and time of the information regarding the user's state and the above-mentioned identification information of the terminal apparatus 10.

On the other hand, each time the CPU 21 of the information processing apparatus 20 receives (acquires) information about the user's state via the communication interface unit 28, the CPU 21 stores the information about the user's state in the second acquisition data shown in FIG. 6, for example. .. In the second acquisition data, the acquisition (measurement) date and time of the information regarding the user's state and the information regarding the user's state are associated with each identification information (terminal device ID in the example of the figure) of the terminal device 10. It is the data described in the state.

The estimation means 33 has a function of estimating the user's behavior when the Web page is displayed on the Web browser based on the acquired information on the user's state.

Further, the estimation means 33 may estimate the user's behavior based on the acquired information on the user's state and the learned model based on machine learning using the information on the user's state as learning data. In this case, by using the trained model, it is possible to easily estimate the user's behavior when the Web page is displayed on the Web browser.

The function of the estimation means 33 is realized, for example, as follows. The CPU 21 of the information processing apparatus 20 has elapsed a predetermined period (for example, 3 seconds or the like) starting from a predetermined timing (for example, a timing of receiving (acquiring) information about the state of the first user from the terminal device 10). Each time, the information about the user's state stored in the second acquisition data within the predetermined period (information about the transition of the information about the user's state within the predetermined period) is extracted from the second acquisition data. Then, the CPU 21 inputs the extracted information on the user's state (information on the transition of the information on the user's state within a predetermined period) into a trained model based on machine learning using the information on the user's state as learning data. By doing so, the behavior of the user within the predetermined period is estimated.

An example of the training data is shown in FIG. The learning data shown in FIG. 7 is a user when the user is performing a predetermined action (in the example of the figure, any one of "stationary", "walking", "climbing up and down", "lying", etc.). Information about the state of (for example, at least one of the user's acceleration in the triaxial direction and the angular velocity in the triaxial direction) is described in a state in which the time transition within a predetermined period is associated with the action (behavior label). It is data. As a result of machine learning, a learned model showing the relationship between the time transition of information about the state of the user who performs a predetermined action within a predetermined period and the action is constructed.

The data described in the learning data may include data when the same user performs different actions, or may include data when different users perform the same action.

Further, the CPU 21 may learn a model used to estimate the user's behavior based on the information on the user's state by machine learning using the information on the user's state as learning data.

In this case, for example, when a predetermined model learning instruction is input using the input unit 27, the CPU 21 may learn the model using the learning data shown in FIG. 7. The CPU 21 may be learned using, for example, a time-series correspondence type neural network model. Here, as a time-series correspondence type neural network, for example, RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory) which is an advanced type of RNN, and the like can be applied. Further, the CPU 21 has a plurality of convolutional neural network (CNN) models, support vector machine (SVM) models, fully coupled neural networks (FNN) models, gradient boosting (HGB) models, wave net (WN) models, and the like. You may learn using any of the models.

In this way, the CPU 21 inputs the information regarding the transition of the information regarding the state of the user within the predetermined period into the trained model, and thereby the behavior of the user within the predetermined period (here, “stationary” and “walking””. , "Up and down stairs", "Lie down", etc.) can be estimated.

In the present embodiment, the case where the CPU 21 learns a model provided in the information processing apparatus 20 and estimates the user's behavior using this learned model has been described as an example, but the present invention has been described. Not limited to cases. For example, the model used to estimate the user's behavior may be provided in a device other than the information processing device 20. In this case, the CPU 21 inputs information regarding the transition of information regarding the user's state within a predetermined period into a trained model provided in another device, and receives (acquires) information regarding the behavior estimated by the trained model. ) May.

The storage means 34 has a function of storing information about a Web page and a user's behavior in a storage device 24 (predetermined storage device) in a state of associating them with each other.

The function of the storage means 34 is realized, for example, as follows. When the CPU 21 of the information processing apparatus 20 estimates the user's behavior within a predetermined period based on the function of the estimation means 33, the acquisition date and time of the information about the Web page corresponding to the user (terminal device ID) is the predetermined period. Information about the Web page contained in is extracted from the first acquisition data. Then, the CPU 21 stores the information about the extracted Web page and the user's behavior in the first provided data shown in FIG. 8, for example, in a state of associating them with each other. The first provided data is data described in a state in which information about a Web page and user behavior are associated with each identification information (terminal device ID in the example of the figure) of the terminal device 10. .. Here, the access period in the first provided data shown in FIG. 8 indicates a period during which the corresponding Web page is continuously displayed on the Web browser being executed on the user's terminal device 10. Further, in the example shown in FIG. 8, the display state, the connection state, and the action are associated with each other for one access period, but at least one of the display state, the connection state, and the action is associated with one access period. Since one may change, a plurality of display states, connection states, and actions may be associated with one access period.

In this way, based on the first provided data, it becomes possible to grasp what kind of behavior the user is browsing the Web page. In addition, based on the first provided data, it becomes possible to grasp what kind of behavior the user is browsing the web page and in what state the web page is displayed. .. Further, based on the first provided data, it becomes possible to grasp what kind of behavior the user is browsing the Web page and whether or not the Web page is displayed online.

The control means 35 has a function of controlling the operation of the terminal device 10 according to the estimated user behavior. Thereby, for example, the Web page is displayed on the Web browser by controlling the terminal device 10 such as activating the camera function of the terminal device 10 when the user's behavior of holding the terminal device 10 is estimated. It becomes possible to appropriately operate the terminal device 10 according to the user's behavior at that time.

The function of the control means 35 is realized, for example, as follows. When the CPU 21 of the information processing apparatus 20 estimates the user's behavior when the Web page is displayed on the Web browser based on the function of the estimation means 33, the CPU 21 accesses the control data shown in FIG. 8 and estimates the behavior. The control content corresponding to is extracted. The control data includes the estimated behavior (in the example of the figure, "stationary", "holding the terminal device", etc.) and the control content of the terminal device 10 (for example, the URL of the web page) for each identification information of the Web page (for example, the URL of the Web page). In the example of the figure, the data is described in a state in which "start the map information application", "start the camera function", etc.) are associated with each other. For example, when the estimated action when a predetermined Web page is displayed on the Web browser of the terminal device 10 is "holding the terminal device", the CPU 21 corresponds to the action of "holding the terminal device". The control content (here, "activate the camera function") is extracted from the control data. Then, the CPU 21 issues instruction information (command) for operating the terminal device 10 with the extracted control content (here, “starting the camera function”) via the communication interface unit 28 and the communication network NW. Send to.

On the other hand, when the CPU 11 of the terminal device 10 receives the instruction information from the information processing device 20, it operates based on the received instruction information (here, the camera function is activated). In this way, the terminal device 10 can be appropriately operated according to the user's behavior when the Web page is displayed on the Web browser.

The providing means 36 may have a function of providing the information about the Web page stored in the storage device 24 (predetermined storage device) and the user's behavior in a corresponding state.

The function of the providing means 36 is realized, for example, as follows. For example, when a predetermined information provision instruction is input by the input unit 27, the CPU 21 of the information processing apparatus 20 transmits the first provided data stored in the storage device 24 via the communication interface unit 28 and the communication network NW. It may be transmitted (provided) to an external device (for example, a computer (for example, a server) that generates and provides personalized information). Further, the CPU 21 may display (provide) the first provided data stored in the storage device 24 on the display unit 26 when a predetermined information providing instruction is input by using the input unit 27. Further, the CPU 21 may convert the first provided data into voice data and output it from a voice output device such as a speaker.

Further, the providing means 36 may have a function of providing information according to the estimated user behavior. As a result, for example, when the user's behavior of walking is estimated, the user is provided with the advertisement information via the terminal device 10, and the user is displayed on the Web browser. It becomes possible to provide appropriate information according to the behavior of the user.

The function of the providing means 36 in this case is realized, for example, as follows. When the CPU 21 of the information processing apparatus 20 estimates the user's behavior when the Web page is displayed on the Web browser based on the function of the estimation means 33, it is estimated by accessing the second provided data shown in FIG. Extract the provided information corresponding to the behavior. The second provided data includes the estimated behavior (“stationary”, “walking”, etc. in the example of the figure) and the provided information (in the example of the figure, in the example of the figure) for each identification information of the web page (for example, the URL of the web page). It is data described in a state of being associated with "map information", "advertisement information", etc.). For example, if the behavior estimated when a predetermined Web page is displayed on the Web browser of the terminal device 10 is "walking", the CPU 21 provides information corresponding to the behavior of "walking" (here, here). "Advertising information") is extracted from the second provided data. Then, the CPU 21 transmits the extracted provided information to the terminal device 10 via the communication interface unit 28 and the communication network NW.

On the other hand, when the CPU 11 of the terminal device 10 receives the provided information from the information processing device 20, the received provided information may be displayed on, for example, the display unit 16. Further, when the provided information is composed of voice data, the CPU 11 may output the provided information from a voice output device such as a speaker. In this way, it is possible to provide appropriate information according to the user's behavior when the Web page is displayed on the Web browser.

The changing means 37 has a function of changing the display mode of the Web page displayed on the Web browser according to the estimated user behavior. As a result, for example, when the user's behavior changes from rest to walking, the design and layout of the web page can be changed, so that the web page can be changed according to the user's behavior when the web page is displayed on the web browser. It becomes possible to dynamically change the display mode of.

The function of the changing means 37 is realized, for example, as follows. When the CPU 21 of the information processing apparatus 20 estimates the user's behavior when the web page is displayed on the web browser based on the function of the estimation means 33, the CPU 21 displays the web page displayed on the web browser on the terminal device 10. For example, the Web page may be changed by acquiring it from a Web server (not shown) and changing the design, layout, or the like of the acquired Web page according to the estimated behavior. Here, information regarding the design, layout, and the like of the Web page corresponding to the estimated user behavior may be stored in, for example, the storage device 24. Then, the CPU 21 transmits the changed Web page to the terminal device 10 via the communication interface unit 28 and the communication network NW.

On the other hand, when the CPU 11 of the terminal device 10 receives the changed Web page from the information processing device 20, the changed Web page may be displayed on the display unit 16. FIG. 11 shows an example of the display mode of the Web page displayed on the terminal device 10. FIG. 11A is a diagram showing an example of the layout of a Web page displayed on a Web browser when the estimated user behavior is “walking”, and FIG. 11B is a diagram showing an example of the layout of the estimated user. It is a figure which shows an example of the layout of the Web page displayed in the Web browser when the action is "lying down". For example, when the user's behavior is estimated to be "walking", as shown in FIG. 11A, a Web page provided with areas A, B, C, and D in which texts, moving images, and the like are displayed is displayed. It has become so. Here, the area A is displayed at the upper part of the screen, the area B is displayed at the lower left part of the screen, and the areas C and D are laid out so as to be displayed at the lower right part of the screen. On the other hand, when the user's behavior is estimated to be "lying down", as shown in FIG. 11B, the areas D, C, and A (the information displayed in each area is displayed in FIG. 11A). A Web page provided with (may be the same as the content to be displayed) is displayed. Here, the area D is displayed at the upper part of the screen, the area C is displayed at the center of the screen, and the area A is laid out so as to be displayed at the lower part of the screen. In this way, the display mode of the Web page can be dynamically changed according to the user's behavior when the Web page is displayed on the Web browser.

(5) Main processing flow of the information processing system of the present embodiment Next, an example of the main processing flow performed by the information processing system of the present embodiment will be described with reference to the flowchart of FIG.

The CPU 21 of the information processing device 20 acquires information about a Web page displayed on a Web browser executed on the terminal device 10 (step S100). Specifically, first, when the CPU 11 of the terminal device 10 is executing the Web browser by the operation of the user using the input unit 17, for example, every time a predetermined time (for example, 10 seconds or the like) elapses. To acquire information about the Web page displayed on the Web browser. Then, each time the CPU 11 acquires information about a Web page, the acquired information is transmitted to the information processing apparatus 20 via the communication interface unit 19 and the communication network NW.

On the other hand, each time the CPU 21 of the information processing apparatus 20 receives (acquires) information about a Web page via the communication interface unit 28, the CPU 21 stores the information about the Web page in the first acquisition data.

Further, the CPU 21 of the information processing device 20 acquires information regarding the state of the user of the terminal device 10 measured by the measuring device 18 when the Web page is displayed on the Web browser (step S102). Specifically, first, when the CPU 11 of the terminal device 10 is executing the Web browser by the operation of the user using the input unit 17, for example, a predetermined time (for example, 300 milliseconds, 1 second, etc.) Acquires the user's state (for example, at least one of the user's triaxial or biaxial acceleration and the user's triaxial or biaxial angular velocity) each time. Then, each time the CPU 11 measures the information about the user's state, the CPU 11 transmits the measured information about the user's state to the information processing apparatus 20 via the communication interface unit 19 and the communication network NW.

On the other hand, each time the CPU 21 of the information processing apparatus 20 receives (acquires) information about the user's state via the communication interface unit 28, the CPU 21 stores the information about the user's state in the second acquisition data.

Next, the CPU 21 of the information processing apparatus 20 estimates the user's behavior when the Web page is displayed on the Web browser based on the acquired information regarding the user's state (step S104). Specifically, the CPU 21 of the information processing apparatus 20 starts from a predetermined timing (for example, a timing of receiving (acquiring) information about the state of the first user from the terminal device 10) for a predetermined period (for example,). Every time (3 seconds, etc.) elapses, the information about the user's state stored in the second acquisition data within the predetermined period (information about the transition of the information about the user's state within the predetermined period) is extracted from the second acquisition data. do. Then, the CPU 21 inputs the extracted information on the user's state (information on the transition of the information on the user's state within a predetermined period) into a trained model based on machine learning using the information on the user's state as learning data. By doing so, the behavior of the user within the predetermined period is estimated.

Next, the CPU 21 of the information processing device 20 stores the information about the Web page and the user's behavior in the storage device 24 in a state of associating them with each other (step S106). Specifically, when the CPU 21 of the information processing apparatus 20 estimates the behavior of the user within a predetermined period in the process of step S104, the acquisition date and time of the information related to the Web page corresponding to the user (terminal device ID) is set. Information about the Web page included in the predetermined period is extracted from the first acquired data. Then, the CPU 21 stores the information about the extracted Web page and the user's behavior in the first provided data in a state of being associated with each other.

The CPU 21 of the information processing device 20 is a computer that generates and provides the first provided data stored in the storage device 24 to an external device (for example, personalized information) via the communication interface unit 28 and the communication network NW. It may be transmitted (provided) to (for example, a server).

On the other hand, on the service provider side that generates and provides personalized information, the user browses the Web page while performing what action based on the first provided data transmitted from the information processing apparatus 20. It becomes possible to grasp whether it is. Further, by analyzing the first provided data or the like, it becomes possible to generate personalized information according to the behavior of the user.

As described above, according to the information processing system, the information processing method, and the program of the present embodiment, the information about the Web page displayed on the Web browser executed on the terminal device 10 and the Web page are displayed on the Web browser. Since the user's behavior when it is being done is stored in a state of being associated with it, it is necessary to understand what kind of behavior the user is doing while browsing the Web page based on the stored information. Will be possible. In addition, by analyzing the stored information, it is possible to generate personalized information according to the user's behavior, so it is necessary to provide information applicable to generate personalized information. Can be done.

Further, according to the information processing system, the information processing method, and the program of the present embodiment, since the measuring device 18 is provided in the terminal device 10, the user's state is obtained by using the measuring device 18 provided in the terminal device 10. It becomes possible to obtain information about.

Hereinafter, a modified example of the above-described embodiment will be described.
(Modification example)
In the above embodiment, an example is a case where the estimation means 33 estimates the user's behavior based on the information about the user's state and the trained model based on machine learning using the information about the user's state as learning data. However, the present invention is not limited to this case. For example, the estimation means 33 may presume that the user is performing the predetermined action when the transition of the information regarding the state of the user within the predetermined period satisfies the predetermined condition corresponding to the predetermined action. .. Here, the predetermined condition may be, for example, that the transition of the information regarding the state of the user within the predetermined period is included in the range of the information regarding the state of the user corresponding to the predetermined action.

In this case, the CPU 21 of the information processing apparatus 20 extracts the information about the user's state stored in the second acquisition data within the predetermined period (information about the transition of the information about the user's state within the predetermined period) from the second acquisition data. Then, the user's behavior may be estimated with reference to the behavior data shown in FIG. The behavior data is when it is estimated that the corresponding behavior is performed for each of multiple behaviors (in the example of the figure, "stationary", "walking", "stair climbing", "lying", etc.). It is the data described in the state in which the time transition of the range of information regarding the state of the user (in the example of the figure, the range of 3-axis acceleration) is associated. The behavior data may be stored in the storage device 24, for example.

When the CPU 21 is included in the time transition of the information about the user's state extracted from the second acquisition data and the range of the information about the user's state corresponding to any action in the action data, the user is one of them. It may be presumed that the behavior (for example, "stationary") is being performed.

As described above, according to the information processing system, the information processing method, and the program according to the present modification, it is possible to exert the same effects as those of the above-described embodiment.

The program of the present invention may be stored in a computer-readable storage medium. The storage medium on which this program is recorded may be the ROM 12, RAM 13 or storage device 14 of the terminal device 10 shown in FIG. 2, or the ROM 22, RAM 23 or storage device 24 of the information processing device 20 shown in FIG. May be good. Further, the storage medium may be a CD-ROM or the like that can be read by being inserted into a program reading device such as a CD-ROM drive. Further, the storage medium may be a magnetic tape, a cassette tape, a flexible disk, an MO / MD / DVD, or the like, or may be a semiconductor memory.

The embodiments and modifications described above are described for facilitating the understanding of the present invention, and are not described for limiting the present invention. Therefore, each element disclosed in the above-described embodiment and modification includes all design changes and equivalents belonging to the technical scope of the present invention.

For example, in the above-described embodiment, the case where the measuring device 18 is provided in the terminal device 10 has been described as an example, but the present invention is not limited to this case. The measuring device 18 may be provided, for example, in a state of being worn by the user, or may be provided at a position different from that of the user and the terminal device 10. Further, when the measuring device 18 is provided in a state of being worn by the user, the acceleration, the angular velocity, and the like of at least one body part (for example, arms, legs, etc.) of the user are acquired as information regarding the user's state. May be done.

Further, in the above-described embodiment, the case where the behavior of one user is estimated has been described as an example, but the behavior of each of a plurality of users may be estimated. The behavior of each of the plurality of users and the information about the Web page referred to by each of the plurality of users may be stored in the first provided data in a state of being associated with each other.

Furthermore, in the above-described embodiment, the case where any one of "stationary", "walking", "climbing up and down stairs", "lying down", and "holding the terminal" is estimated as the user's behavior is assumed. Although explained as an example, the content of the action is not limited to these.

Further, in the above-described embodiment, the case where one information processing apparatus 20 is provided has been described as an example, but the present invention is not limited to this case. For example, a plurality of information processing devices 20 may be provided, and in this case, even if the operation contents and processing results on any of the information processing devices 20 are presented in real time on the other information processing device 20. Alternatively, the processing result of any of the information processing devices 20 may be shared among the plurality of information processing devices 20.

Further, in the above-described embodiment, the information processing apparatus 20 realizes the functions of the first acquisition means 31, the second acquisition means 32, the estimation means 33, the storage means 34, the control means 35, the providing means 36, and the changing means 37. However, the present invention is not limited to this configuration. For example, at least one of the above means 31 to 37 by a computer or the like (for example, a general-purpose personal computer or a server computer) communicably connected to the information processing apparatus 20 via a communication network NW such as the Internet or a LAN. It may be configured to realize the functions of the two means. Further, each function of the functional block diagram shown in FIG. 4 may be arbitrarily shared between the terminal device 10 and the information processing device 20 as shown in FIGS. 14A and 14B.

For example, the CPU 11 of the terminal device 10 may estimate the user's behavior when the Web page is displayed on the Web browser based on the information about the user's state. Further, the CPU 11 may learn a model used to estimate the user's behavior based on the information on the user's state by machine learning using the information on the user's state as learning data. In this case, the CPU 11 may perform machine learning using, for example, TensorFlow®, which is an open source machine learning library. This makes it possible for the terminal device 10 to estimate the user's behavior in real time.

Since the information processing system, information processing method, and program of the present invention as described above can be suitably used for an information providing service that provides users with personalized information for each user, they can be used in the industry. The sex is extremely large.

10 ... Terminal device 18 ... Measuring device 20 ... Information processing device 31 ... First acquisition means 32 ... Second acquisition means 33 ... Estimating means 34 ... Storage means 35 ... Control means 36 ... Providing means 37 ... Changing means

Claims (11)

The first acquisition means for acquiring information about the Web page displayed on the Web browser executed on the terminal device, and
A second acquisition means for acquiring information regarding the user's state of the terminal device measured by a predetermined measuring device when the Web page is displayed on the Web browser.
An estimation means for estimating the behavior of the user when the Web page is displayed on the Web browser based on the acquired information on the state of the user.
A storage means for storing information about the Web page and the behavior of the user in a predetermined storage device in a state of associating them with each other.
Information processing system equipped with. The information processing system according to claim 1, further comprising a control means for controlling the operation of the terminal device according to the estimated behavior of the user. The information processing system according to claim 1 or 2, further comprising a providing means for providing information according to the estimated behavior of the user. The information processing system according to any one of claims 1 to 3, further comprising a changing means for changing the display mode of a web page displayed on the web browser according to the estimated behavior of the user. The estimation means estimates the behavior of the user based on the acquired information on the user's state and a learned model based on machine learning using the information on the user's state as learning data. The information processing system according to any one of 1 to 4. The information processing system according to any one of claims 1 to 5, wherein the information regarding the Web page includes information regarding a display state of the Web page on the terminal device. The information processing system according to any one of claims 1 to 6, wherein the information regarding the Web page includes information regarding whether or not the Web page is displayed online. The information processing system according to any one of claims 1 to 7, wherein the information regarding the state of the user includes at least one of the acceleration of the user and the angular velocity of the user. The information processing system according to any one of claims 1 to 8, wherein the predetermined measuring device is provided in the terminal device. The computer
A step to acquire information about a web page displayed on a web browser running on a terminal device, and
A step of acquiring information about a user's state of the terminal device measured by a predetermined measuring device when the Web page is displayed on the Web browser, and
A step of estimating the behavior of the user when the Web page is displayed on the Web browser based on the acquired information on the state of the user.
A step of storing information about the Web page and the behavior of the user in a predetermined storage device in a state of associating them with each other.
Information processing method that executes each step of. On the computer
A function to acquire information about a web page displayed on a web browser running on a terminal device, and
A function of acquiring information on a user's state of the terminal device measured by a predetermined measuring device when the Web page is displayed on the Web browser, and
A function of estimating the behavior of the user when the Web page is displayed on the Web browser based on the acquired information on the state of the user.
A function of storing information about the Web page and the behavior of the user in a predetermined storage device in a state of associating them with each other.
A program to realize.

Images