JP5018120B2 - Mobile terminal, program, and display screen control method for mobile terminal - Google Patents

Description

  The present invention relates to a technique for estimating various operation contents for a mobile terminal performed by a user of the mobile terminal.

  Patent Document 1 describes a portable terminal that estimates user behavior. According to Patent Document 1, the mobile terminal divides sensor data obtained from each sensor on the time axis based on a change in the value of sensor data obtained from a certain sensor, and extracts a feature amount from each divided sensor data. Thus, the user's behavior is predicted by learning the relationship between the feature amount and the service usage history in the user's mobile terminal, that is, the user's behavior. The sensor data is divided when a sensor outputs binary sensor data, and when the sensor data is output as an analog value, the value changes more than a threshold value. To do.

JP 2006-134080 A

  In the prior art, each sensor data is divided at the output change point of a certain sensor, but the meaning of dividing other sensor data at the output change point of a certain sensor, that is, divided based on a certain sensor. It is unclear whether or not the feature amount extracted from other sensor data has a causal relationship with the break of various actions performed through the user's mobile terminal, and there is a problem that the estimation error increases.

  Moreover, although the present action has a part which arises from the past action, in a prior art, only a user action is estimated from the situation information by a sensor, and an estimation error will also become large from this point. Furthermore, there is a problem that the behavior is estimated only from the sensor data of the most recent one section, and the progress of the medium- to long-term behavior is not taken into consideration.

  Therefore, the present invention solves the above-described problem, uses a mobile terminal that estimates the details of various operations performed by the user with higher accuracy, and uses the estimation results of the operation contents by the mobile terminal to provide each service for providing each service. An object is to provide a method for controlling a screen to be displayed.

According to the portable terminal of the present invention,
Means for acquiring sensor data obtained from one or more sensors and / or data relating to the state of one or more portable terminals, and storing the data as status information, and each operation content of the user's portable terminal A means for storing each piece of data as behavior information, and for each situation information, the situation information is divided into sections on the time axis to recognize similar sections, and the same identifier is assigned to the similar sections. A means for outputting information as an identifier string, a means for obtaining a relation between identifier combinations and action information based on the relation between the identifier of each identifier string and action information at each time, and an identifier corresponding to the acquired situation information And a means for estimating an operation performed by the user on the portable terminal based on the obtained identifier and the relationship between the combination of the identifier and the action information. It is characterized in.

According to another embodiment of the mobile terminal of the present invention,
The means for outputting as an identifier string comprises the first means to the Nth means, where N is an integer equal to or greater than 2, and the first means divides the status information into the first sections. Recognizing first sections that are similar to each other, and outputting the same first section as a first identifier string to which the same first identifier is assigned, the Kth means, where K is 2 or more N The following integers are input to the K-1th identifier string, and are similar to the Kth section obtained by combining the K-1th section of the K-1th identifier string. Recognized and output as a Kth identifier string with the same Kth identifier assigned to the similar Kth section, and the output of the Nth means is the output of the means for outputting the status information as an identifier string It is also preferable.

According to another embodiment of the mobile terminal of the present invention,
The K-th means performs a process on each of the K-th sections having different lengths and the number of couplings of the (K-1) -th section being different, thereby making it possible to perform a plurality of operations from one K-1-th identifier string. It is also preferable to output K identifier strings.

Furthermore, according to another embodiment of the portable terminal of the present invention,
The estimating means obtains a relationship between behaviors based on the stored behavior information, and uses the relationship between the behaviors and the acquired behavior information for estimation of operation contents performed on the mobile terminal by the user. Is also preferable.

Furthermore, according to another embodiment of the portable terminal of the present invention,
It is also preferable to have means for changing the user interface according to the estimation result of the operation content.

According to the program of the present invention,
A portable terminal is made to function as the portable terminal.

According to the method of the present invention,
In the system including the mobile terminal and the network, the network is a method for controlling a screen to be displayed on the mobile terminal, the mobile terminal transmitting an operation result estimation result to the network, and the network A step of determining a screen to be displayed on the mobile terminal based on the result, and a step of transmitting information for causing the mobile terminal to display the determined screen on the mobile terminal are provided.

  Instead of dividing other sensor data according to the data of a specific sensor, it searches for similarities for each sensor data, so that it can correspond to the characteristics unique to each user. In addition, after the first similarity search is performed using the first interval, the similarity search is repeated one or more times while combining the intervals, so that estimation in consideration of the past progress can be performed. The accuracy of estimation can be increased by searching for similarities for various sections. Furthermore, not only estimating from situation information but also estimating from the past action history in consideration of the causal relationship between actions, the estimation accuracy can be further improved.

  In addition, the user interface is changed or reconstructed based on the estimation result, or the estimation result is transmitted to the network, and a screen corresponding to the estimation result is displayed on the mobile terminal on the network side. A corresponding user interface or service can be provided.

  The best mode for carrying out the present invention will be described in detail below with reference to the drawings.

  FIG. 1 is a functional block diagram of a portable terminal according to the present invention. The portable terminal is a communication terminal that can be connected to a mobile communication network. In FIG. 1, only the portions necessary for the description of the present invention are shown, and other functional blocks such as a wireless interface are omitted. According to FIG. 1, the mobile terminal includes one or more sensors 1, a terminal state acquisition unit 2, an input / output unit 3, a log collection unit 4, a pattern extraction unit 5, an action pattern learning unit 6, an action An estimation unit 7, a user interface control unit 8, and a storage unit 9 are provided. The pattern extraction unit 5 includes a short-term pattern extraction unit 51 and a medium-term pattern extraction unit 52.

  The sensor 1 is for acquiring various conditions relating to the environment in which the current mobile terminal is placed and the biological information of the user. For example, an orientation sensor, an acceleration sensor, an environmental noise measurement sensor (microphone), a temperature sensor , A humidity sensor, an illuminance sensor, an ultraviolet sensor, a GPS (Global Positioning System) receiver, a sensor for measuring body temperature and pulse, or a part thereof. These sensors do not need to be integrated with the mobile terminal inside the mobile terminal, and the acquired sensor data related to the environment near the mobile terminal or the biometric information of the user is transmitted to the mobile terminal by wire or wirelessly. What is necessary is just to be comprised so that transmission is possible.

  The terminal state acquisition unit 2 acquires information related to the state of the mobile terminal. Information on the status of the mobile device includes the time held by the mobile device, received radio wave intensity, remaining battery level, flip / slide open / closed state, whether charging is in progress, volume, manner mode, etc., external A value related to a state that can be acquired by the mobile terminal, such as a storage medium attachment / detachment state, an earphone attachment / detachment state, a power ON / OFF state, mail reception, and an incoming call. In addition, it is a design matter about which state is specifically acquired.

  The input / output unit 3 includes output functions such as various displays for the user, including various keys for operating the mobile terminal, a microphone for voice input, a speaker for voice output, and a display for display. Provides input functions for various operations on mobile terminals.

  The log collection unit 4 performs feature amount extraction processing and quantization processing on the sensor data acquired by each sensor 1 and the information on the state of the mobile terminal acquired by the terminal state acquisition unit 2 as necessary. The processing result is output to the status information log held by the storage unit 9 as status information. In addition, the user's various actions are acquired from the input / output unit 3, the feature amount extraction process and the quantization process are performed as necessary, and the processing result is output to the action information log held in the storage unit 9 as action information. . Here, the user's various actions acquired from the input / output unit 3 are the use of various online services such as outgoing calls, telephone calls, mail transmissions, music downloads, the use of applications such as calculators, browsing of web pages, received mails. This refers to various operations that the user performs on the mobile terminal, such as browsing and responding to an incoming call.

  The feature amount extraction processing differs depending on the data to be processed. For example, for data with continuous values, such as acceleration and environmental noise, after pre-processing using a bandpass filter , FFT (Fast Fourier Transform: Fast Fourier Transform) can be used to obtain a power spectrum and use it as a feature quantity. For the position information, clustering by vector quantization or the like, or a method of dividing a map into a grid of a certain size and attaching an identifier to the address can be used. Note that the feature amount extraction processing and quantization processing are performed in order to reduce the amount of acquired raw data and simplify subsequent processing. With respect to data acquired with a simple value, the feature data extraction processing and quantization processing are not performed, and the acquired raw data may be used as situation information or behavior information as it is. Of course, it is possible to reduce the data amount by applying the feature amount extraction processing and the quantization processing to the data acquired as discrete values.

  In this embodiment, the pattern extraction unit 5 has a two-stage configuration of a short-term pattern extraction unit 51 and a medium-term pattern extraction unit 52, and divides each piece of situation information read from the situation information log into temporal sections. Recognizing sections that are similar to each other, assign the same label to similar sections, and output the status information as a label string. Note that a label means an identifier for recognizing a section, and if used for identifying a section, for example, other expressions such as index values can be used as labels in the present embodiment. It is included. Details will be described below.

  FIG. 2 is a diagram for explaining processing by the short-term pattern extraction unit 51. First, the status information read from the status information log is divided into certain time intervals called short-term sections below. As shown in FIG. 2A, in this example, it is divided into M short-term sections from # 1 to #M. Subsequently, the output probability of each short-term section data for one model data generated from all short-term sections # 1 to #M is calculated, and it is determined whether or not there is a short-term section whose output probability is less than the threshold. . If there is a short-term section whose output probability is less than the threshold, select only the short-term section whose output probability is greater than or equal to the threshold, generate model data from the selected short-term section, and select each selected model data The calculation of the output probability of the data in the short-term section and comparison with the threshold value are repeated until there is no more than the threshold value. An intermediate label for identifying the model data is given to the model data when there is no more than the threshold, and the short-term section selected when there is no more than the threshold is defined as the processed short-term section. As shown in FIG. 2B, in this example, a short-term section including # 1 and # 4 is selected, and an intermediate label “A” is given to the model data.

  Subsequently, the same processing as the above processing is performed on the sections excluding the short-term sections including # 1 and # 4 that have been processed short-term sections from the entire short-term section. As shown in FIG. 2C, in the second process, a short-term section including # 2 and # 7 is selected, and an intermediate label “A” is given to the model data.

  Further, the generation of the model data is repeated while excluding the processed short-term section in each process until all the short-term sections are processed short-term sections. As shown in FIGS. 2D and 2E, in this example, a short-term section including # 3 and # 5 in the third process, and a short-term section including # 6 and #M in the fourth process. The intermediate labels “C” and “D” are assigned to the selected model data.

  Subsequently, for each short-term section, the similarity with the model data is obtained, and the model data with the similarity equal to or higher than a predetermined threshold is selected. As shown in FIG. 2 (f), in this example, “A” and “I” are in short-term section # 1, and “I” and “D” are in short-term sections # 2 and #M. # 3 and # 4 have “a” and “c”, short-term sections # 5 and # 7 have “b” and “c”, and short-term sections # 6 have “a” and “d”. Are selected as similar model data. In FIG. 2, two similar model data are selected for each short-term section, but this is an example, and the number of model data selected for each short-term section is short-term. Depends on interval data and threshold.

  Finally, a short-term label indicating the combination is assigned to the combination of model data selected as being similar to each short-term section. As shown in FIG. 2 (g), in this example, the combination of model data #a and #a is "α", the combination of #a and #e is "β", and the combination of #a and #c. “Γ”, “ε” for the combination of #i and #c, and “ζ” for the combination of #a and #d are assigned as short-term labels. For combinations with a small number of appearances, a short-term label indicating no pattern can be assigned and data can be omitted.

  The short-term pattern extracting unit 51 stores the generated model data, the combination of the model data indicated by the short-term label, and the short-term label string expressing the situation information with the short-term label as shown in FIG. Store in the short-term pattern DB to be saved.

  The short-term pattern extraction unit 51 may divide the status information into short-term sections, determine the similarity of each short-term section, give the short-term sections similar to each other, and output the same short-term label. The processing method is not limited to the above.

  Subsequently, the medium-term pattern extraction unit 52 reads each short-term label string held in the short-term pattern DB, and performs the same processing as the processing in the short-term pattern extraction unit 51 based on the similarity in the medium-term section obtained by combining the short-term sections. . Preferably, in the medium-term pattern extraction unit 52, for example, one short-term obtained from one sensor is obtained by performing similarity recognition processing on various medium-term intervals obtained with various combinations of short-term intervals. A plurality of medium-term label sequences are generated from the label sequence. In FIG. 3, three medium-term label strings are generated for one short-term label string obtained from sensor # 1. Similarly to the short-term pattern extraction unit 5, the medium-term pattern extraction unit 52 stores the generated model data, the combination of model data indicated by the medium-term label, and the medium-term label sequence in which the short-term label sequence is expressed by the medium-term label. Stored in the medium-term pattern DB. Note that the number of medium-term label columns generated from one short-term label column does not need to be the same for each short-term label column.

  The medium-term pattern extraction unit 52 may determine the similarity of the medium-term sections, give the same medium-term label to the similar medium-term sections and output them, and is not limited to the processing method described above. Absent. Furthermore, it is possible to use a configuration of three or more stages, such as further combining the medium-term sections of the medium-term label row to perform similarity recognition processing.

  The behavior pattern learning unit 6 reads the medium-term label string of the medium-term pattern DB and the behavior information log, and obtains the relationship between the combination of the medium-term labels and the content of the behavior information at that time by modeling. For the modeling, a Bayesian network can be used, but the present invention is not limited to this.

  FIG. 3 is a diagram for explaining processing performed by the behavior pattern learning unit 6. Here, it is assumed that three sensors # 1, # 2, and # 3 are mounted, and three-system examples of medium-term labels are generated from each sensor and stored in the medium-term pattern DB. The behavior pattern learning unit 6 reads the medium term label string and the behavior information log stored in the medium term pattern DB, and examines the combination of the medium term labels at each time and the content of the behavior information at that time. For example, at the time indicated by the dotted line in FIG. 3, action # 2 corresponds to the combination of the medium-term labels “A22”, “B23”, “C12”, and “C31”. The behavior pattern learning unit 6 examines the correspondence between the combination of medium-term labels at each time and the content indicated by the behavior information, calculates a model and parameters for determining the appearance probability of each behavior from the combination of medium-term labels, and stores the results It stores in action pattern DB which the part 9 preserve | saves.

  The behavior estimation unit 7 obtains each current or latest situation information, obtains a medium term label corresponding to each obtained situation information, and uses the obtained medium term label and the model and parameters stored in the behavior pattern DB. Based on this, the mobile terminal user estimates an action to be performed using the mobile terminal, specifically, an operation content to be performed on the mobile terminal. According to a preferred embodiment, the behavior estimation unit 7 analyzes the temporal causal relationship of past behavior information from the behavior information log, and corrects the behavior estimation result based on the model and parameters stored in the behavior pattern DB. . This is because when a phone call was made, the other party did not respond, and the cause of certain actions may depend on past actions or results, such as sending an email to the other party. For example, even if the probability of performing action # 1 is calculated as 50% from the models and parameters stored in the action pattern DB, it can be estimated that the possibility of performing action # 1 is high from the past action history. Corrects the final estimation result to a value higher than 50%.

  The user interface control unit 8 performs change control of the user interface to the user, for example, reconstruction of the hierarchical structure of the menu, based on the estimation result of the operation content by the behavior estimation unit 7.

  Further, the mobile terminal transmits the estimation result to a predetermined server on the network, and the server on the network determines a screen to be displayed on the mobile terminal based on the estimation result and causes the mobile terminal to display the determined screen. Send information to your mobile device. As a result, it is possible to provide information and present services according to the behavior. The learned model stored in the behavior pattern DB may be transmitted to a predetermined server on the network and used for various services on the network side.

  As described above, according to the present invention, since other sensor data is not divided by a specific sensor but the similarity is searched for each sensor data, it is possible to deal with the characteristics unique to each user. Furthermore, since the similarity search is performed by using the medium-term section obtained by combining the short-term sections of the short-term label string with various lengths and used for estimating the behavior, the accuracy of the estimation is improved as compared with the prior art. In addition to estimating the behavior from the situation information, the accuracy can be further improved by estimating the behavior from the past behavior history in consideration of the causal relationship between the behaviors. In the present invention, since the behavior pattern is saved as a mathematical model, the behavior history of each user cannot be directly grasped from the saved data, which is preferable from the viewpoint of privacy protection.

  The present invention can be realized by a program that is read by a portable terminal having a computer or a processor and causes the computer or portable terminal to function as the portable terminal. Furthermore, the present invention can be realized not only by hardware and software but also by a combination thereof.

It is a functional block diagram of the portable terminal by this invention. It is a figure explaining the pattern extraction process by a short-term pattern extraction part. It is a figure explaining the process which an action pattern learning part performs.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Sensor 2 Terminal state acquisition part 3 Input / output part 4 Log collection part 5 Pattern extraction part 51 Short-term pattern extraction part 61 Medium-term pattern extraction part 6 Behavior pattern learning part 7 Behavior estimation part 8 User interface control part 9 Storage part

Claims (6)

Means for obtaining sensor data obtained from one or more sensors and / or data relating to the state of one or more mobile terminals, and storing each as status information;
Means for storing each piece of data indicating each operation on the mobile terminal as behavior information;
For each situation information, means for dividing the situation information into time sections and recognizing similar sections, giving the same identifier to similar sections, and outputting the situation information as an identifier string;
Based on the relationship between the identifier of each identifier string and the behavior information at each time, means for obtaining the relationship between the combination of identifiers and the behavior information,
An identifier corresponding to the acquired situation information is obtained, and a means for estimating an operation content performed by the user on the mobile terminal based on the obtained identifier and a relationship between the identifier combination and the behavior information;
I have a,
The means for outputting as an identifier string comprises 1st to Nth means, where N is an integer greater than or equal to 2,
The first means divides the status information into first sections, recognizes first sections that are similar to each other, and first identifiers that are given the same first identifier to similar first sections Output as a column,
K-th means, where K is an integer greater than or equal to 2 and less than or equal to N, is a K-th section obtained by combining the K-1th section of the K-1th identifier string with the K-1th identifier string as an input Recognizing the Kth section similar to each other and outputting the same Kth section to the similar Kth section as the Kth identifier string,
The output of the Nth means is the output of the means for outputting the status information as an identifier string.
Mobile device. The K-th means performs a process on each of the K-th sections having different lengths and the number of couplings of the (K-1) -th section being different, thereby making it possible to perform a plurality of operations from one K-1-th identifier string. Output an identifier string of K,
The mobile terminal according to claim 1 . The estimating means obtains a relationship between behaviors based on the stored behavior information, and uses the relationship between the behaviors and the acquired behavior information for estimation of operation contents performed on the mobile terminal by the user.
The portable terminal of any one of Claim 1 to 2 . The mobile terminal according to any one of claims 1 to 3 , further comprising means for changing a user interface in accordance with an operation result estimation result. The program which functions a portable terminal as a portable terminal given in any 1 paragraph of Claims 1-4 . The system including the mobile terminal and the network according to any one of claims 1 to 4 , wherein the network controls a screen displayed on the mobile terminal,
The mobile device sending the operation estimation result to the network;
The network determining a screen to display on the mobile device based on the estimation result;
The network sending information to the mobile device to display the determined screen on the mobile device;
A method comprising:

Images