JP2012088989A - Information processing apparatus and operation support method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform a series of operations having a high probability of execution through procedures less than the case where the series of operations are performed in order.SOLUTION: When a screen is changed by an operation of a user, operation information containing an identifier of an event generated by the operation and a screen identifier is generated by an operation information acquisition section 201 and stored. In the case where the screen identifier contained in the newly stored operation information is a specific screen identifier, an operation information dividing section 221 collects the stored operation information as an operation information group. An analysis section 222 calculates a probability for a series of operations represented by the operation information group to be performed. When an operation is performed, a candidate estimation section 211 estimates a series of operations to be performed next on the basis of the probability obtained by the calculation. In the case where execution of the series of estimated operations is instructed, on the basis of an event identifier contained in operation information of an operation information group corresponding to the series of operations, processing at a time when the series of operations are performed is executed.

Description

  The present invention relates to a technique for predicting and supporting an operation performed by a user.

  As an apparatus for predicting an operation performed by a user, there is a user interface generation apparatus disclosed in Patent Document 1. This apparatus stores a history of operations performed by the user. In addition, the apparatus compares the operation performed by the user with the history of the operation, predicts the next operation performed by the user, and displays a menu for executing the predicted next operation.

JP 2002-215278 A

  According to the apparatus disclosed in Patent Document 1, since a menu for performing the next operation is displayed, the user can simplify the operation and perform the next operation. However, since only the next operation is displayed in the apparatus disclosed in Patent Document 1, when performing a series of operations, the next operation predicted for each operation is selected. As for the number of times, there may be a case where there is not much difference from the case where the menus are operated in order.

  The present invention has been made under the above-described background, and an object of the present invention is to enable a series of operations with a high probability of being performed with fewer procedures than when performing a series of operations in order.

  In order to solve the above-described problems, the present invention provides an operation unit operated by a user, and a processing unit that performs a process according to an operation performed on the operation unit and displays a screen corresponding to the operation on a display unit. Operation information storage means for storing operation information in which a screen identifier for identifying a screen displayed on the display means is associated with an event identifier of an event for displaying the screen, and stored in the operation information storage means. If the screen identifier of the operation information is the screen identifier of the specific screen, the operation information stored in the operation information storage means before the operation information including the screen identifier or the operation information including the screen identifier Operation information different from the operation information dividing means for generating an operation information group in which operation information stored in the operation information storage means and operation information including the screen identifier are collected. For each group, the operation information group and the number of times the operation information group is generated are stored, and the probability that a series of operations for generating the operation information group is performed is calculated for each operation information group based on the stored number of times. When the operation means is operated, the calculation means and an estimation means for estimating a series of operations performed after the operation based on the probability obtained by the calculation means, and the processing means When an operation to instruct execution of a series of operations estimated by the estimation means is performed by the operation means, based on the event identifier included in the operation information of the operation information group corresponding to the series of operations, An information processing apparatus is provided that performs processing when the series of operations is performed.

  In the present invention, the operation information dividing means includes an operation information screen in which an operation information screen identifier newly stored in the operation information storage means is stored in the operation information storage means before the operation information. When the identifier matches, the operation information newly stored before the operation information stored in the operation information storage means, or the operation information newly stored before the operation information stored in the operation information storage means and The newly stored operation information may be configured as an operation information group.

  In the present invention, the operation information includes a list of components of a user interface constituting the screen specified by the screen identifier, and the operation information dividing unit is newly stored in the operation information storage unit. In the list of the constituent elements included in the operation information, when a predetermined constituent element is included, the operation information is stored in the operation information storage means before the operation information newly stored in the operation information storage means. Operation information, operation information newly stored before operation information stored in the operation information storage means, and operation information newly stored may be output as an operation information group.

  In the present invention, the predetermined component may be a component that displays moving image content.

  Further, in the present invention, after the operation for instructing execution of the series of operations estimated by the estimation means is performed by the operation means, the operation for instructing execution of the series of operations estimated by the estimation means again. When the operation unit performs the operation information, the configuration may include an operation information integration unit that integrates operation information groups related to a series of operations instructed to be executed.

  Further, the present invention provides a processing step in which the control unit of the information processing apparatus performs processing according to an operation performed in the operation unit, and displays a screen corresponding to the operation on the display unit, and the control unit includes the An operation information storage step for storing operation information in which a screen identifier for identifying a screen displayed on the display unit and an event identifier of an event for displaying the screen are associated with each other, and the control unit in the operation information storage step When the screen identifier of the stored operation information is the screen identifier of the specific screen, the operation information stored in the operation information storage step before the operation information including the screen identifier or the operation information including the screen identifier An operation information dividing step for generating an operation information group in which the operation information stored in the operation information storage step and the operation information including the screen identifier are collected; The control unit stores the operation information group and the number of times the operation information group is generated for each different operation information group, and the probability that a series of operations for generating the operation information group is performed based on the stored number of times. An operation step for calculating for each operation information group, and an estimation step for estimating, when the operation unit is operated, the series of operations performed after the operation based on the probability obtained in the operation step when the control unit is operated When an operation that instructs execution of a series of operations estimated in the estimation step is performed on the operation unit, the event identifier included in the operation information of the operation information group corresponding to the series of operations And an operation support method including a process execution step for executing a process when the series of operations are performed.

  In the present invention, in the operation information dividing step, the screen identifier of the operation information newly stored in the operation information storage step is the screen identifier of the operation information stored in the operation information storage step before the operation information. The operation information stored before the operation information newly stored in the operation information storage step, or the operation information newly stored before the operation information stored in the operation information storage step and the new The operation information stored in the operation information group may be used as the operation information group.

  In the present invention, the operation information includes a list of components of a user interface constituting the screen specified by the screen identifier, and the operation information dividing step is newly stored in the operation information storage step. In the list of the constituent elements included in the operation information, if the predetermined constituent elements are included, the operation information stored before the operation information newly stored in the operation information storage step, or newly The operation information stored before the operation information stored in the operation information storage step and the newly stored operation information may be configured as an operation information group.

  In the present invention, the predetermined component may be a component that displays moving image content.

  Further, in the present invention, after the operation for instructing execution of the series of operations estimated in the estimation step is performed in the operation unit, the operation for instructing execution of the series of operations estimated in the estimation step again. May be configured to include an operation information integration step of integrating operation information groups related to a series of operations instructed to be executed.

  According to the present invention, a series of operations with a high probability of being performed can be performed with fewer procedures than when performing a series of operations in order.

The figure which showed the external appearance of the information processing apparatus 1 which concerns on one Embodiment of this invention. 2 is a diagram illustrating a hardware configuration of the information processing apparatus 1. FIG. FIG. 3 is a block diagram showing a configuration of functions realized in the information processing apparatus 1. The flowchart which showed the flow of the process when the operation information acquisition part 201 outputs operation information. The flowchart which showed the flow of the process when the operation information division part 221 outputs an operation information group. The figure for demonstrating the order of a transition of a screen. The figure which showed standby screen P1. The figure which showed the starting screen P2. The figure which showed the basic screen P3. The figure which showed menu screen P4. The figure which showed the imaging | photography mode selection screen P5. The figure which showed folder list screen P6. The figure which showed the photograph list screen P7. The figure which showed the email creation screen P8. The figure which showed the screen which displayed the estimated operation. The figure for demonstrating an operation model. The flowchart which showed the flow of the process which CPU102 performs.

[Embodiment]
(Configuration of information processing apparatus 1)
FIG. 1 is a diagram illustrating an appearance of an information processing apparatus 1 according to an embodiment of the present invention. In the present embodiment, the information processing apparatus 1 is a mobile phone capable of voice communication and data communication. FIG. 2 is a diagram illustrating a hardware configuration of the information processing apparatus 1. As illustrated in FIG. 2, each unit of the information processing apparatus 1 is connected to the bus 101. Each unit of the information processing apparatus 1 exchanges data between each unit via the bus 101.

The communication unit 108 has an interface function for performing wireless communication. The communication unit 108 performs radio communication with a radio base station constituting a mobile communication network via an antenna (not shown) connected to the communication unit 108 under the control of a CPU (Central Processing Unit) 102. Perform voice communication and data communication. When the communication unit 108 receives a signal transmitted from the radio base station, the communication unit 108 outputs a voice signal related to voice communication to the call unit 109 and outputs a data signal related to data communication to the CPU 102. In addition, when the data signal output from the CPU 102 is input, the communication unit 108 transmits the data signal to the radio base station. When the audio signal output from the call unit 109 is input, the communication unit 108 receives the audio signal. Transmit to the radio base station.
The calling unit 109 has a microphone and a speaker (both not shown). When voice is input to the microphone, the call unit 109 outputs a voice signal representing the input voice to the communication unit 108. In addition, when an audio signal is input from the communication unit 108, the call unit 109 converts the audio signal into an analog signal and outputs the analog signal, and the audio is output from the speaker.

The photographing unit 110 includes an image sensor and a lens that forms an image of the photographing object on the image sensor, and generates image data representing the image formed on the image sensor under the control of the CPU 102.
The display unit 107 includes a control circuit for controlling the liquid crystal display panel and a liquid crystal display panel. Under the control of the CPU 102, a character and graphic screen, a screen serving as a user interface for operating the information processing apparatus 1, and the like Is displayed on the LCD panel.
The operation unit 106 includes keys 106A to 106C for operating the information processing apparatus 1 and a touch sensor 106D having a rectangular shape by a capacitance method. The keys 106 </ b> A to 106 </ b> C are arranged in the casing of the information processing apparatus 1, and when the user operates the keys 106 </ b> A to 106 </ b> C, the operated key is detected by the CPU 102. Further, the transparent touch sensor 106D is arranged so as to cover the entire surface of the liquid crystal display panel of the display unit 107, and transmits an image displayed on the liquid crystal display panel. The touch sensor 106D detects a position where the user touches with a finger in a rectangular area, and outputs a signal indicating the detected position to the CPU 102. CPU102 specifies the position where the user touched on the screen based on the signal output from touch sensor 106D. The CPU 102 performs various processes according to operations performed on the operation unit 106.

  A ROM (Read Only Memory) 103 stores a basic program that causes the information processing apparatus 1 to realize the basic functions of the mobile phone. The storage unit 105 includes a nonvolatile memory and stores a plurality of application programs. The storage unit 105 stores data used when the CPU 102 executes the basic program and data used when the CPU 102 executes the application program.

  A RAM (Random Access Memory) 104 is used as a work area when the CPU 102 executes a program, and temporarily stores data used when the CPU 102 executes the program. The CPU 102 is a control unit (control unit) that controls each unit of the information processing apparatus 1. When the information processing apparatus 1 is turned on, the CPU 102 reads the basic program from the ROM 103 and executes it. When the CPU 102 executes the basic program, a basic function as a mobile phone is realized, and the information processing apparatus 1 can perform voice communication, data communication, and the like. When the CPU 102 executes the basic program, it is possible to execute an application program that displays a standby screen, an application program that takes a picture with a photographing unit, an application program that transmits and receives an e-mail, and the like. Further, when the CPU 102 executes the basic program, a function of predicting a series of operations performed by the user, a function of displaying the predicted series of operations to the user, a function of executing the predicted series of operations on behalf of the user, and the like are realized. .

(Functional configuration of information processing apparatus 1)
FIG. 3 is a block diagram showing the configuration of the functions according to the present invention among the functions realized in the CPU 102 by executing the basic program.

  The processing unit 200 executes application programs and controls hardware. The processing unit 200 detects an operation performed on the operation unit 106 and executes various processes such as hardware control according to the performed operation. In addition, the processing unit 200 manages events that occur during execution of the program and components of the user interface screen displayed on the display unit 107, and is an operation target according to an operation performed on the operation unit 106. Change the program.

  The operation information acquisition unit 201 includes an identifier of a program displaying the screen to be operated (hereinafter abbreviated as a program identifier) and a list of components of the user interface screen displayed on the display unit 107 ( Hereinafter, the UI configuration list is acquired from the processing unit 200. The operation information acquisition unit 201 becomes an operation target in response to the occurrence of an event corresponding to a user operation in the program being executed or a change in the user interface displayed on the display unit 107 according to the user operation. The program identifier and UI configuration list of the program being acquired are acquired from the processing unit 200, and the set of the program identifier and UI configuration list is stored.

  Further, the operation information acquisition unit 201 compares the newly acquired set of the program identifier and UI configuration list with the set of the program identifier and UI configuration list acquired immediately before this set. Then, when either the program identifier or the UI configuration list is changed, the operation information acquisition unit 201 has a unique identifier (hereinafter, referred to as the state of the displayed user interface, that is, the state of the display screen). Screen identifiers), identifiers representing events generated in response to user operations (hereinafter referred to as event identifiers), program identifiers, and newly acquired UI configuration lists. Output operation information. In addition, when the component of a user interface has the attribute set by the user, the attribute information showing the attribute of a component is included. Here, the attribute information is data held by the constituent elements such as a character string input to the text box and a check box selection state.

FIG. 4 is a flowchart showing a processing flow when the operation information acquisition unit 201 outputs operation information. FIG. 6 is a diagram showing the order of screen transition according to the following description.
For example, it is assumed that an application program for displaying a standby screen (hereinafter referred to as a standby screen display application) is executed and the standby screen P1 (FIG. 7) is displayed on the display unit 107. Here, when the standby screen P1 (FIG. 7) is displayed on the display unit 107, an operation for instructing execution of a program for taking a photograph (hereinafter referred to as a camera application) is performed on the operation unit 106. An event for starting the camera application occurs in the processing unit 200. When this event occurs, the processing unit 200 outputs the program identifier of the standby screen display application that is the operation target at this time and the UI configuration list of the standby screen P1 to the operation information acquisition unit 201. The operation information acquisition unit 201 acquires the program identifier and UI configuration list output from the processing unit 200 (steps S101 and S102).

  When the execution of the camera application is instructed, the screen of the display unit 107 transitions to the activation screen P2 (FIG. 8) indicating that the camera application is being activated. Here, since the user interface has changed, the operation information acquisition unit 201 acquires the program identifier of the camera application that is the operation target at this time and the UI configuration list of the startup screen P2 (steps S101 and S102). Note that the operation information acquisition unit 201 changes the program identifier and UI configuration list when the standby screen P1 changes to the startup screen P2 (YES in step S103), so the screen identifier of the startup screen P2, the startup screen Operation information including an event identifier of an event instructing display of the image, a program identifier of the camera application, and a UI configuration list is output (step S105).

  Further, when a certain time has elapsed since the startup screen P2 is displayed, the screen transitions to the basic screen P3 (FIG. 9) of the camera application. Here, since the user interface has changed from the startup screen P2 to the basic screen P3, the operation information acquisition unit 201 acquires the program identifier of the camera application that is the operation target at this time and the UI configuration list of the basic screen P3. (Steps S101 and S102). Specifically, on the basic screen P3 of FIG. 9, as the components of the user interface, “MENU” button, “list” button, “shoot” button, up and down buttons represented by triangles, and the shooting unit 110 There is an element that displays an image of the image data output from. The operation information acquisition unit 201 acquires a list of these components. Next, the operation information acquisition unit 201 instructs the display of the screen identifier of the basic screen P3 and the basic screen because the UI configuration list changes when the startup screen P2 changes to the basic screen P3 (YES in step S104). Operation information including the event identifier of the event, the program identifier of the camera application, and the UI configuration list is output (step S105).

  Further, when the camera application is an operation target and the displayed “MENU” button is operated in a state where the basic screen P3 is displayed, an operation for displaying a menu screen related to shooting is performed. P4 (FIG. 10) is displayed. The operation information acquisition unit 201 acquires the program identifier of the camera application and the UI configuration list on the menu screen P4 when the UI configuration list is changed (steps S101 and S102). In the menu screen P4, as a component of the user interface, a “close” button, a “select” button, up / down / left / right buttons represented by triangles, and items such as “shooting mode” and “camera adjustment” are displayed. Since there is a list box to be displayed, the operation information acquisition unit 201 acquires a list of these components.

  Here, since the UI configuration list changes when the operation information acquisition unit 201 changes from the basic screen P3 to the menu screen P4 (YES in step S104), the screen identifier of the menu screen P4 and the display of the menu screen P4 are displayed. Operation information including the event identifier of the event instructing the camera application, the program identifier of the camera application, and the UI configuration list is output (step S105). In the menu screen P4, there is an item “camera shake correction” having an attribute set by the user as a component of the user interface. Thus, in the user interface, when there is an element to be set by the user, the attribute of the component element (information “camera shake correction: ON” in the case of the menu screen P4) is included in the operation information as attribute information.

  Further, when the “shooting mode” item is selected while the menu screen P4 is displayed, a shooting mode selection screen P5 (FIG. 11) is displayed. If any of the displayed shooting modes is selected while the shooting mode selection screen P5 is displayed, the selected shooting mode is set, and the screen returns to the basic screen P3. As described above, when the transition from the menu screen P4 to the shooting mode selection screen P5 occurs, the user interface changes (YES in step S104), and therefore the operation information acquisition unit 201 outputs the operation information. Further, when returning from the shooting mode selection screen P5 to the basic screen P3, the user interface changes (YES in step S104), so the operation information acquisition unit 201 outputs the operation information.

When the “list” button is operated while the basic screen P3 is displayed, a folder list screen (folder list screen P6 (FIG. 12)) storing image data obtained by the camera application is displayed. Is done. When an operation for selecting one of the folders is performed in a state where the folder list screen P6 is displayed, a list screen of photos represented by the image data stored in the selected folder (photo list screen P7 (FIG. 13)). )) Is displayed. When the “Mail” button is operated while the photo list screen P7 is displayed, an e-mail application program is executed to create an e-mail with the photo selected on the photo list screen P7 as an attached file. A screen (e-mail creation screen P8 (FIG. 14)) is displayed, and the program to be operated becomes an e-mail application.
As described above, when the transition is made from the basic screen P3 to the folder list screen P6, the user interface changes (YES in step S104), so the operation information acquisition unit 201 outputs the operation information. Also, when the folder list screen P6 changes to the photo list screen P7, the user interface changes (YES in step S104), so the operation information acquisition unit 201 outputs the operation information and creates an e-mail from the photo list screen P7. Even when the screen transitions to the screen P8, the user interface and the program identifier change (YES in step S104), so the operation information acquisition unit 201 outputs the operation information. Note that, in the operation information output when the transition to the e-mail creation screen P8 is made, since the operation target program is an e-mail application, the included program identifier is the program identifier of the e-mail application.

  Next, the operation information dividing unit 221 will be described. The operation information dividing unit 221 receives the operation information output from the operation information acquisition unit 201 and stores the received operation information in the temporary storage area provided in the storage unit 105 in the order received. Further, the operation information dividing unit 221 monitors the content of the operation information stored in the storage area, and when the program identifier is changed in the operation information, the screen included in the operation information newly stored in the temporary storage area When the identifier matches the screen identifier included in the operation information already stored in the temporary storage area, or when the newly acquired operation information includes a predetermined specific UI configuration list, The operation information stored in the temporary storage area is defined as one group (hereinafter referred to as an operation information group). Then, the operation information dividing unit 221 generates an identifier for uniquely identifying the operation information group (hereinafter referred to as a group identifier), and displays the generated group identifier and the operation information corresponding to the generated group identifier as the operation information group. Output as. Note that the operation information included in the operation information group is deleted from the temporary storage area.

  FIG. 5 is a flowchart showing a flow of processing when the operation information dividing unit 221 outputs an operation information group. For example, when the camera application is activated as described above, the operation information is output and stored in the storage area when the activation screen P2 is displayed (step S201), and the operation information is displayed when the basic screen P3 is next displayed. It is output and stored in the storage area. The operation information dividing unit 221 stores a UI configuration list of the basic screen P3 as a predetermined specific UI configuration list. Here, since the operation information output when the basic screen P3 is displayed includes the UI configuration list of the basic screen P3, the UI configuration list included in the operation information is a predetermined UI configuration. The operation information dividing unit 221 matches the list (YES in step S204), and the operation information division unit 221 sets the operation information output when the activation screen P2 is displayed as an operation information group (hereinafter, this operation information group is referred to as an operation information group G1). Output as.

Further, as described above, when the screen changes from the basic screen P3 → the menu screen P4 → the shooting mode selection screen P5 → the basic screen P3, when the menu screen P4 is displayed, when the shooting mode selection screen P5 is displayed. When returning to the basic screen P3, the operation information is output, and the operation information is stored in the storage area.
Note that the screen identifier of the operation information that is output when the transition from the startup screen P2 to the basic screen P3 is the screen identifier of the basic screen P3. The screen identifier of the operation information output when returning from the shooting mode selection screen P5 to the basic screen P3 is also the screen identifier of the basic screen P3. Here, since the operation information dividing unit 221 has operation information including the same screen identifier in the storage area (YES in step S203), when the basic screen P3 output before returning to the basic screen P3 is displayed. The operation information when the menu screen P4 is displayed and the operation information when the shooting mode selection screen P5 is displayed are output as an operation information group (hereinafter, this operation information group is referred to as an operation information group G11). To do.

Further, as described above, when the screen changes from the basic screen P3 → folder list screen P6 → photo list screen P7 → email creation screen P8, when the folder list screen P6 is displayed, the photo list screen P7 is displayed. When the e-mail creation screen P8 is displayed and the program identifier changes, the operation information is output, and the operation information is stored in the storage area.
Here, the operation information dividing unit 221 changes the program identifier included in the operation information when the photo list screen P7 is displayed and the operation information when the e-mail creation screen P8 is displayed (Step S1). YES in S202), the operation information when the basic screen P3 is displayed, the operation information when the folder list screen P6 is displayed, and the operation information when the photo list screen P7 is displayed are displayed in the operation information group (hereinafter referred to as this operation group). The operation information group is output as an operation information group G12).

  Next, the analysis unit 222 will be described. The analysis unit 222 receives the operation information group output from the operation information dividing unit 221 and stores the received operation information group in the storage unit 105. The analysis unit 222 constructs an operation model based on the stored operation information group. The operation model is obtained by determining the occurrence probability of a series of operations performed after the series of operations based on the operation performed by the user. The analysis unit 222 stores the constructed operation model in the storage unit 105.

  Here, an example of the operation model will be described. FIG. 16 is a diagram for explaining the operation model. For example, as described above, when an operation for instructing activation of the camera application is performed on the standby screen P1, the operation information group G1 is output from the operation information dividing unit 221. The operation information group G1 is received by the analysis unit 222 and stored in the storage unit 105. The analysis unit 222 stores the number of times this operation information group G1 has been received. For example, when the number of times the operation information group G1 is received is 10, the analysis unit 222 stores the number of times in the storage unit 105 in association with the operation information group G1 as illustrated in FIG.

  Further, when an operation for instructing activation of the game application program is performed on the standby screen P1, an operation information group including operation information related to activation of the game application program from the operation information dividing unit 221 (hereinafter, this operation information group). Is set as the operation information group G2). The operation information group G2 is received by the analysis unit 222 and stored in the storage unit 105. Further, the analysis unit 222 stores the number of times of receiving this operation information group G2. For example, if the number of times of receiving the operation information group G2 is 90, the analysis unit 222 stores the number of times in association with the operation information group G2 as shown in FIG.

  The analysis unit 222 calculates the probability that the received operation information group is output, and stores the probability obtained by the calculation. For example, the state where the standby screen P1 is displayed is the state “S”, the probability that the operation information group G1 is output by the user's operation from the state “S” is represented as P (S | G1), and the state “S” ”Represents the probability that the operation information group G2 is output from P (S | G2). The analysis unit 222 acquires the number of times the operation information group G1 is output from the state “S” (10 times) and the number of times the operation information group G2 is output from the state “S” (90 times).

  Next, the analysis unit 222 divides the output count of the operation information group G1 by the sum of the output count of the operation information group G1 and the output count of the operation information group G2. For example, if the number of times of output of the operation information group G1 (10 times) is divided by the total number of times of output of the operation information group G1 and the number of times of output of the operation information group G2 (90 times), The probability of output, that is, the probability that a series of operations represented by the operation information group G1 is performed is obtained. The analysis unit 222 stores the obtained probability as shown in FIG.

  Further, as described above, when the screen is changed by the user's operation from the basic screen P3 → the menu screen P4 → the shooting mode selection screen P5 → the basic screen P3, the operation information group G11 is output. The operation information group G11 is received by the analysis unit 222 and stored in the storage unit 105. The analysis unit 222 stores the number of times this operation information group G11 is received in association with the operation information group G11. The analysis unit 222 represents the probability that the operation information group G11 is output next after the operation information group G1 is output as P (G1 | G11), calculates the probability that the operation information group G11 is output, and the calculated probability Remember.

When the operation information group G11 is output and the screen is changed from the basic screen P3 → the folder list screen P6 → the photo list screen P7 → the e-mail creation screen P8 by the user's operation as described above, the operation information group G111 is output. The operation information group G111 is received by the analysis unit 222 and stored in the storage unit 105. The analysis unit 222 stores the number of times this operation information group G111 is received in association with the operation information group G111. The analysis unit 222 represents the probability that the operation information group G111 is output next after the operation information group G11 is output as P (G11 | G111), calculates the probability that the operation information group G111 is output, and the calculated probability Remember.
On the other hand, when the operation information group G1 is output and the screen changes from the basic screen P3 to the folder list screen P6 to the photo list screen P7 to the e-mail creation screen P8 by the user's operation as described above, the operation information group G12 is output. The operation information group G12 is received by the analysis unit 222 and stored in the storage unit 105. The analysis unit 222 stores the number of times this operation information group G12 is received in association with the operation information group G12. The analysis unit 222 represents the probability that the operation information group G12 is output next after the operation information group G1 is output as P (G1 | G12), calculates the probability that the operation information group G12 is output, and the calculated probability Remember.

  For example, the number of times the operation information group G11 is output next after the operation information group G1 is output is three times, and the number of times the operation information group G12 is output next after the operation information group G1 is output is seven times. If there is, the operation information group G1 is output by dividing the output count of the operation information group G11 (3 times) by the total output count of the operation information group G11 and the output count of the operation information group G12 (10 times). The probability that the operation information group G11 will be output later, that is, the probability that a series of operations represented by the operation information group G11 will be performed is obtained. The analysis unit 222 stores the obtained probability as shown in FIG.

  Next, the candidate estimation unit 211 will be described. The candidate estimation unit 211 receives the operation information output from the operation information acquisition unit 201 and specifies an operation information group including the received operation information. In addition, the candidate estimation unit 211 searches the operation model for the identified operation information group, and identifies operation information groups that transition from the retrieved operation information group in descending order of probability of transition. The candidate estimation unit 211 outputs the identified operation information group to the candidate display unit 202.

  The candidate display unit 202 receives the operation information group output from the candidate estimation unit 211. The candidate display unit 202 extracts operation information at the end of the received operation information group, and extracts a screen identifier included in the extracted operation information. When extracting the screen identifier, the candidate display unit 202 specifies an operation name associated with the extracted screen identifier in advance, and causes the display unit 107 to display the specified operation name.

  When the operation name displayed by the candidate display unit 202 is continuously selected, the operation information integration unit 223 integrates the operation information group related to the selected operation name, and adds the integrated operation information group to the operation model. That is, in the operation model shown in FIG. 16, when the operation information group G11 is selected and then the operation information group G111 is selected, a new operation information group G13 obtained by integrating the operation information group G11 and the operation information group G111 is created. Add to operation model.

(Operation of the embodiment)
Next, an operation when the operation model shown in FIG. 16 is stored in the storage unit 105 will be described. FIG. 17 is a flowchart showing a flow of processing performed when an operation is performed on the operation unit 106.

  First, when the operation unit 106 is operated by the user, processing corresponding to the operation is performed, and then the processing illustrated in FIG. 4 is performed. The operation information acquisition unit 201 acquires the operation information, and the acquired operation information Is output (step S1). Thereafter, the processing shown in FIG. 5 is performed, and the operation information dividing unit 221 divides the operation information stored in the storage area (step S2). Further, the analysis unit 222 stores the operation information group and the output count of the operation information group as described above, and constructs an operation model (step S3).

Next, the information processing apparatus 1 estimates an operation information group of operations performed by the user (step S4). Specifically, the candidate estimation unit 211 receives the newly output operation information, and specifies an operation information group including the received operation information based on the operation model. In addition, the candidate estimation unit 211 searches the storage unit 105 for the identified operation information group, and identifies operation information groups that transition from the retrieved operation information group in descending order of probability of transition.
For example, when an operation for starting the camera application is performed from the state in which the standby screen P1 is displayed, operation information including the screen identifier of the basic screen P3 is output when the operation transitions to the basic screen P3. In the operation model, since the operation information group G1 is stored as an operation information group including the operation information output when transitioning to the basic screen P3, the candidate estimation unit 211 specifies the operation information group G1.

  Next, the candidate estimation unit 211 searches the operation model for the identified operation information group, and identifies operation information groups that transition from the retrieved operation information group in descending order of probability of transition. For example, in FIG. 16, there are an operation information group G11 and an operation information group G12 as operation information groups transitioned from the operation information group G1. The candidate estimation unit 211 acquires the probability (P (G1 | G11)) that the operation information group G11 is output and the probability (P (G1 | G12)) that the operation information group G12 is output. Here, comparing the acquired probabilities, in the case of the operation model of FIG. 16, P (G1 | G12) = 0.7, P (G1 | G11) = 0.3, and the operation information group G12 is Since the probability of output is high, the candidate estimation unit 211 identifies the operation information group in the order of the operation information group G11 after the operation information group G12.

  Next, the candidate estimation unit 211 outputs the specified operation information group to the candidate display unit 202 in the specified order. The candidate display unit 202 receives the operation information group output from the candidate estimation unit 211, and displays an operation with a probability that the user will perform (step S5). Specifically, the operation information at the end in the received operation information group is extracted, and the screen identifier included in the extracted operation information is extracted. When extracting the screen identifier, the candidate display unit 202 specifies an operation name associated with the extracted screen identifier in advance, and causes the display unit 107 to display the specified operation name.

  For example, as described above, when the candidate display unit 202 receives the operation information group G12, the operation information at the end of the operation information group is the operation information when the photo list screen P7 is displayed. Includes the screen identifier of the photo list screen P7. In the candidate display unit 202, since an operation name “photo list display” is associated with this screen identifier in advance, a menu for selecting “photo list display” is displayed as shown in FIG.

  When the candidate display unit 202 receives the operation information group G11, the operation information at the end of the operation information group is the operation information when the shooting mode selection screen P5 is displayed. The screen identifier of the mode selection screen P5 is included. In the candidate display unit 202, since the operation name “shooting mode selection” is associated with this screen identifier in advance, a menu for selecting “shooting mode selection” is displayed as shown in FIG.

  Next, when an operation for selecting the displayed operation name has not been performed (NO in step S8), the processing in FIG. 17 is once ended, and the processing in FIG. 17 is executed again. If an operation for selecting the displayed operation name is performed (YES in step S6), each operation information included in the operation information group is extracted, and an event identifier included in each operation information is represented. The processing unit 200 generates an event. In the processing unit 200, processing corresponding to the generated event is performed, and finally a screen with a screen identifier corresponding to the selected operation name is displayed on the display unit 107 (step S7).

For example, when “photo list display” is selected on the screen of FIG. 15, from the operation information of the operation information group G12, an event instructing display of the folder list screen P6 and an event instructing display of the photo list screen P7. Is generated, and after the folder list screen P6 is displayed, the photo list screen P7 is displayed. That is, a series of operations for displaying the photo list screen P7 after displaying the folder list screen P6 performed by the user in the past is executed on behalf of the user. When the user's operation is not estimated, it is necessary to operate in the order of the basic screen P3 → folder list screen P6 → photo list screen P7. However, when the user's operation is estimated, the user performs the operations described above. Since the transition from the basic screen to the photo list screen can be performed with fewer steps than the operation, operations often performed by the user can be performed with fewer steps than in the past.
For example, when “shooting mode selection” is selected on the screen of FIG. 15, an instruction to display the menu screen P4 and a display of the shooting mode selection screen P5 are instructed from the operation information of the operation information group G11. After the event is generated and the menu screen P4 is displayed, the shooting mode selection screen P5 is displayed. Here, when the shooting mode is selected and the screen returns to the basic screen P3, in the operation model, the operation information group G11 is an operation information group including the operation information output when the shooting mode selection screen P5 is changed to the basic screen P3. Since the operation information group G111 is stored next, the candidate estimation unit 211 identifies this operation information group G111. That is, the next operation is not estimated from the current screen state, but the next operation is estimated based on the operations performed from the standby screen to the current screen state.

  The operation information integration unit 223 monitors whether the operation name displayed by the candidate display unit 202 has been selected. When the user selects the operation name displayed by the candidate display unit 202 after selecting the operation name displayed by the candidate display unit 202 (NO in step S8), the operation information related to the operation name selected previously. An operation information group obtained by integrating the group and the operation information group associated with the operation name selected later is generated and added to the operation model (step S9).

[Modification]
As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, It can implement with another various form. For example, the above-described embodiment may be modified as follows to implement the present invention, or may be implemented in combination with each modification.

  In the embodiment described above, the information processing apparatus 1 is a mobile phone, but is not limited to a mobile phone. For example, the information processing apparatus 1 may be a PDA (Personal Digital Assistant) or a smartphone. Further, the information processing apparatus is not limited to a portable information processing apparatus, and may be a desktop personal computer apparatus.

  In the embodiment described above, the operation information dividing unit 221 determines whether or not the program identifier has changed in step S202 in the process of FIG. 4, but does not perform the process of step S202 and determines the change in the program identifier. It does not have to be. Further, when determining whether or not to output the operation information group, the operation information dividing unit 221 determines whether or not to output the operation information group only by the process of step S203 without performing the processes of step S202 and step S204. Alternatively, it may be determined whether or not to output the operation information group only by the process of step S204 without performing the processes of step S202 and step S203.

In the above-described embodiment, when the operation information includes a moving image player that displays a moving image in the UI configuration list, YES may be determined in step S204 of FIG.
Also, the operation information stored in the storage area may be output up to the operation information group as an operation information group. As a result, an operation for displaying a screen having a moving image player as a component is estimated, and the operation name of the estimated operation is displayed.

  In the above-described embodiment, the operation model has a tree structure. However, the operation model is not limited to the tree structure, and a mesh-type network structure or a hash map may be used. In the operation model, the starting point of the tree structure is not limited to the state in which the standby screen P1 is displayed. For example, the state in which the menu screen is displayed may be the starting point of the tree structure.

  In the embodiment described above, when the estimated operation is displayed by the candidate display unit 202, it is displayed in the menu format. However, the estimated operation may be displayed in a transparent display in the display area related to the application program. Good. For example, when a menu screen related to an application program is displayed as in the menu screen P4, a display area for an estimated operation may be provided in addition to the menu screen.

  In the embodiment described above, when generating an operation information group, the operation information up to the previous stored operation information is set as the operation information group. However, the operation information group includes the newly stored operation information. May be included. In this case, when the candidate display unit 202 receives the operation information group G11, the corresponding operation name is displayed, and the user selects the operation, the basic screen P3 → the menu screen P4 → the shooting mode selection screen P5 → the basic screen P3. The screen changes in order. In addition, when the screen transition from the shooting mode selection screen P5 to the basic screen P3, an event represented by the event identifier included in the operation information is generated, and this event includes information indicating which item is selected. Therefore, an item equivalent to the selection item at the time of operation information generation is selected. In addition, the candidate display unit 202 extracts operation information at the end of the received operation information group, and extracts a screen identifier and an event identifier included in the extracted operation information. When the screen identifier is extracted, the candidate display unit 202 extracts an operation name previously associated with the extracted screen identifier and an operation name previously associated with the event identifier, and displays these operation names in combination. Be displayed on the unit 107. For example, when the candidate display unit 202 receives the operation information group G11 as described above, a menu for selecting “shooting mode selection (night view)” is displayed. In this modification, the operation information at the end of the previous operation information group may be included in the head of the operation information group to be newly stored.

DESCRIPTION OF SYMBOLS 1 ... Information processing apparatus, 101 ... Bus, 102 ... CPU, 103 ... ROM, 104 ... RAM, 105 ... Memory | storage part, 106 ... Operation part, 106A-106C ... Key, 106D ... Touch sensor, 107 ... Display part, 108 ... Communication unit 109 ... Communication unit 110 ... Shooting unit 200 ... Processing unit 201 ... Operation information acquisition unit 202 ... Candidate display unit 211 ... Candidate estimation unit 221 ... Operation information division unit 222 ... Analysis unit 223 ... Operation Information Integration Department

Claims (10)

Operation means operated by a user;
Processing means for performing processing according to the operation performed in the operation means, and displaying a screen corresponding to the operation on the display means;
Operation information storage means for storing operation information in which a screen identifier for identifying a screen displayed on the display means is associated with an event identifier of an event for displaying the screen;
When the screen identifier of the operation information stored in the operation information storage means is a screen identifier of a specific screen, the operation information stored in the operation information storage means before the operation information including the screen identifier, or the screen identifier Operation information dividing means for generating an operation information group in which operation information stored in the operation information storage means before the operation information including the operation information including the screen identifier is collected,
For each different operation information group, the operation information group and the number of times the operation information group is generated are stored, and the probability that a series of operations for generating the operation information group is performed based on the stored number of times is stored for each operation information group. Computing means for computing
When the operation means is operated, an estimation means for estimating a series of operations performed after the operation based on the probability obtained by the calculation means;
Have
The processing means includes
When an operation to instruct execution of a series of operations estimated by the estimation unit is performed by the operation unit, based on the event identifier included in the operation information of the operation information group corresponding to the series of operations. An information processing apparatus that performs processing when the series of operations is performed. The operation information dividing unit is configured such that the screen identifier of the operation information newly stored in the operation information storage unit matches the screen identifier of the operation information stored in the operation information storage unit before the operation information. , Operation information newly stored before the operation information stored in the operation information storage means, or operation information newly stored before the operation information stored in the operation information storage means and newly stored The information processing apparatus according to claim 1, wherein the operation information is an operation information group. The operation information includes a list of components of a user interface that constitutes a screen specified by the screen identifier,
When the operation information dividing unit includes a predetermined component in the list of the component included in the operation information newly stored in the operation information storage unit, the operation information dividing unit newly stores the operation information in the operation information storage unit. Operation information stored in the operation information storage means before the stored operation information, or operation information newly stored before the operation information stored in the operation information storage means and newly stored operation information The information processing apparatus according to claim 1, wherein the information processing apparatus outputs the information as an operation information group.   The information processing apparatus according to claim 3, wherein the predetermined component is a component that displays moving image content. After an operation for instructing execution of a series of operations estimated by the estimation means is performed by the operation means, an operation for instructing execution of a series of operations estimated by the estimation means is again performed by the operation means. 5. The information processing apparatus according to claim 1, further comprising operation information integration means for integrating operation information groups related to a series of operations instructed to be executed. A processing step in which a control unit of the information processing apparatus performs processing according to an operation performed in the operation unit, and displays a screen corresponding to the operation on the display unit;
An operation information storage step for storing operation information in which the control unit associates a screen identifier for identifying a screen displayed on the display unit with an event identifier of an event for displaying the screen;
When the screen identifier of the operation information stored in the operation information storage step is the screen identifier of the specific screen, the control unit stores the operation information stored in the operation information storage step before the operation information including the screen identifier. Or an operation information dividing step for generating an operation information group in which the operation information stored in the operation information storage step and the operation information including the screen identifier are stored before the operation information including the screen identifier;
Probability that the control unit stores an operation information group and the number of times the operation information group is generated for each different operation information group, and a series of operations for generating the operation information group is performed based on the stored number of times Calculating step for each operation information group;
When the operation unit is operated, the control unit estimates a series of operations performed after the operation based on the probability obtained in the calculation step;
When an operation that instructs execution of a series of operations estimated in the estimation step is performed on the operation unit, based on the event identifier included in the operation information of the operation information group corresponding to the series of operations And a process execution step of executing a process when the series of operations are performed. In the operation information dividing step, when the screen identifier of the operation information newly stored in the operation information storage step matches the screen identifier of the operation information stored in the operation information storage step before the operation information, Operation information newly stored before the operation information stored in the operation information storage step, or operation information newly stored before the operation information stored in the operation information storage step and newly stored operation The operation support method according to claim 6, wherein the information is an operation information group. The operation information includes a list of components of a user interface that constitutes a screen specified by the screen identifier,
In the operation information dividing step, when a predetermined component is included in the list of the component included in the operation information newly stored in the operation information storage step, the operation information dividing step is newly performed in the operation information storage step. The operation information stored before the stored operation information, or the operation information newly stored before the operation information stored in the operation information storing step and the newly stored operation information are set as an operation information group. The operation support method according to claim 6 or 7, wherein:   9. The operation support method according to claim 8, wherein the predetermined constituent element is a constituent element for displaying moving image content. After an operation instructing execution of a series of operations estimated in the estimation step is performed in the operation unit, an operation instructing execution of a series of operations estimated in the estimation step is performed again in the operation unit. The operation support method according to any one of claims 6 to 9, further comprising an operation information integration step of integrating operation information groups related to a series of operations instructed to be executed.

Images