JP2008269119A - Information processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To estimate wrong operation from an operation history, by extracting the degree of puzzlement for operation and certainty of an operation flow from the operation history of a user. <P>SOLUTION: The operation history of the user for an information processor is accumulated, the degree of puzzlement of the user is obtained by collating a rule 110 showing relation between the operation history and the degree of puzzlement of the user with the operation history; further the certainty of the operation flow of the user is obtained by collating with a model 112 expressing a general operation history for an application by a probability with the operation history; and by comparing the degree of puzzlement as a subjective element of the user with a value obtained by integrating the certainty of the operation flow as an objective element for each operation, the wrong operation is estimated from the operation history of the user. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an information processing apparatus including user support means that estimates and presents a correct operation when a user performs an incorrect operation.

  As a user support method using a user operation history from an input / output device, there are the following conventional techniques. In Patent Document 1, the user's confused state is detected from the user's operation history using data in which the user's confused state and the operation pattern are associated, and the help function is activated actively. Then, useful help information is presented to the user inferred from the system state and user operation behavior immediately before the help is activated. In Patent Document 2, it is detected that the user has made an operation error exceeding a predetermined number of operation errors in a specific operation step, and an operation manual for the operation step is presented to the user. In Patent Document 3, the system state and the required time in each state are recorded together with the user's operation history, and the user's operation history is compared by comparing the required time with a predetermined standard required time when the user requests help. Detect a place that was confused. Then, the operation history highlighting the current operation and the operation at the point where the confusion is detected is presented to the user, and the user is guided to the help information at the highlighted point.

Japanese Patent Laid-Open No. 10-154057 Japanese Unexamined Patent Publication No. 4-44113 JP 2000-231429 A

  The technique of Patent Literature 1 is effective when there is no error in the user operation history until the confusing state is detected. However, in an actual user operation, an erroneous operation is performed with certainty without being confused, and it may be confused later without knowing the operation method. In such a case, user support cannot be performed with the technique of Patent Document 1. The technique of Patent Document 2 is effective when the system can determine an operation error immediately after a user operates. However, it is difficult for the system to determine an operation error immediately after a user operation, except for an error operation in which an error message is displayed. The technique of Patent Document 3 is effective when the user is confused by the operation and the required time exceeds the standard required time at a location where the user's operation process has an error. However, there are cases where a correct operation is performed while being confused by an actual user operation. In such a case, the technology of Patent Document 3 cannot provide user support. That is, it is difficult to accurately estimate the user's operation correction location only by detecting the confusion state that is a subjective factor of the user.

  Thus, depending on the conventional technology, when there is an error in the user operation history, it is difficult to estimate the erroneous operation location.

  The present invention accurately estimates an erroneous operation location in a user's operation history, so that the degree of error in an operation that is an objective element in addition to the degree of confusion (degree of confusion) that is a subjective element of the user (degree of erroneous operation). Is used. Further, the present invention uses the correct operation degree of the operation flow probability model to estimate a correct operation at an erroneous operation location. Then, the estimated correct operation is presented to the user.

  Since the information processing apparatus according to the present invention can accurately estimate the erroneous operation location in the user operation history, the user can be guided to the erroneous operation situation. In addition, since a correct operation at an erroneous operation location can be estimated, a correct operation for accomplishing the task can be presented to the user.

Hereinafter, an embodiment of an information processing apparatus to which the present invention is applied will be described with reference to the drawings.
FIG. 1 is a conceptual configuration diagram of an information processing apparatus to which the present invention is applied. In FIG. 1, an input unit 101 is a device that sends a user request to the UI processing unit 103, and may be any input device such as a mouse or a keyboard that is used in a computer. The output unit 102 is a device that presents information received from the UI processing unit 103 to the user, and may be any display device used in a computer such as a CRT monitor or an LCD display device. The UI processing unit 103 is a device for exchanging information with an input / output device such as the input unit 101 and the output unit 102, a process of sending user input information such as commands and data received from the input unit 101 to the information processing unit 104, and A process of sending user presentation screen information such as screen information indicating the response result of the application received from the information processing unit 104 or the correction information presentation unit 109 or screen information of help contents presented to the user to the output unit 102 is performed. The information processing unit 104 is a device that executes processing corresponding to user input information received from the UI processing unit 103 and sends a response result to the UI processing unit 103. Further, the information processing unit 104 sends the user input information, information indicating the current state of the application, and the response contents of the application to the user input information to the user operation behavior acquisition unit 105.

  The user operation behavior acquisition unit 105 is a device that stores information received from the information processing unit 104 in the operation history data 111. The user confusion level detection unit 106 is an apparatus that estimates how much the user is confused from the behavior of the user's operation with respect to the system. An operation rule 110 is used. The erroneous operation degree detection unit 107 is a device that estimates the degree of erroneous operation that indicates how much an operation error is likely to occur in the user's operation from the behavior of the user's operation with respect to the system, and represents the frequency of the operation procedure in the system. The operation flow probability model 112 is used. The score integration unit 108 operates in the history that the user has operated so far when it is detected that the user is confused about the operation or when there is a request for presentation of help information from the user via the input unit 101 It is an apparatus that estimates whether there is a portion that needs to be corrected, and if there is, correct operation at the corrected portion is estimated. The correction information presenting unit 109 is a device that summarizes the correction location in the past operation of the user and the estimation result of the correct operation in the correction location as screen information and sends it to the UI processing unit 103 for presentation to the user.

  FIG. 2 is a hardware configuration diagram of an information processing apparatus to which the present invention is applied. The data unit 206 of the external storage device 205 stores a confusing operation rule 110, operation history data 111, and an operation flow probability model 112. Also, the UI processing unit 103, the information processing unit 104, the user operation behavior acquisition unit 105, the user confusion level detection unit 106, the erroneous operation level detection unit 107, the score integration unit 108, and the correction information presentation unit 109 are stored as a processing program 207 in the storage device 205. Stored in The function provided to the user by this system is realized by the processing device 203 reading the data unit 206 and the processing program 207 of the storage device 205 into the memory 204 and performing processing.

  Hereinafter, taking a media viewing / recording system for a personal computer as an example of an information processing apparatus to which the present invention is applied, a specific processing flow of the system will be described.

  FIG. 3 is a flowchart of processing in the information processing unit 104. In step 301, the user input information received from the UI processing unit 103 and the screen information of the application presented to the user by the output unit 102 are sent to the user operation behavior acquisition unit 105. FIG. 4A shows an example of user input information, and FIG. 4B shows an example of application screen information. In this example, the input device corresponding to the input unit 101 is a mouse, and the output device corresponding to the output unit 102 is an LCD display device. 4A and 4B, the lowest part is the latest data, and the older data is stored in time series as it goes upward. As user input information, there are an operation content 401 indicating a physical operation on the mouse, a start time 402 indicating a relative time elapsed from the start of the system in milliseconds, and a coordinate position 403 of the mouse on the screen of the LCD display device. It is input and sent to the user operation behavior acquisition unit 105. In addition, as the screen information of the application, the current display screen type 404, the object type 405 under the mouse pointer on the display screen, and the relative time elapsed from the start of the system in milliseconds. Time 406 is acquired and sent to the user operation behavior acquisition unit 105.

  In step 302, the user input information received from the UI processing unit 103 is identified. If the user input information is an input representing a request for help information, the process proceeds to step 305, and if it is any other input, the process proceeds to step 303. In the case of FIG. 4A, since there is no input indicating a request for help information, the process proceeds to step 303. In step 303, processing corresponding to the user input information is executed, and screen information indicating a response result is presented to the user via the UI processing unit 103. FIG. 4C is an example of a response result when FIG. 4A is user input information and FIG. 4B is application screen information. The application response result 408 is synchronized with the input information. Show. In this example, since the response result that the display screen is switched to the program search screen is obtained by clicking the program search button with the mouse, the information on the program search screen is sent to the UI processing unit 103. In step 304, the application response result for the user input information is sent to the user operation behavior acquisition unit 105, and the process returns to the state waiting for user input again. On the other hand, in step 305, the score integration unit 108 is called in order to obtain help information to be presented to the user, and the state again waits for user input.

  FIG. 5 is a flowchart of processing in the user operation behavior acquisition unit 105. In step 501, operations that can be interpreted as one operation are integrated with respect to the operation content of the user input information received from the information processing unit 104, and attributes representing the characteristics of each operation are extracted. Then, the screen information of the application and the response result of the application are integrated so as to be synchronized in time series and stored in the operation history data 111. FIG. 6 is an example of a result of integrating user input information, application screen information, and application response results. In the operation contents of the user input information, the operations “press the left mouse button” and “release the left mouse button” are integrated into “left click”. Further, a stop time in seconds is obtained from the difference from the start time of the next operation content as the attribute of the operation content “stop”, and is acquired as the attribute 605. Further, as the attribute of the operation content “move”, the moving distance in units of pixels and the number of moving pixels per second are obtained from the mouse coordinate position and acquired as attributes 605 and 606. Here, “-” in FIG. 6 means that there is no specific data. In step 502, the user's confusion level detection unit 106 is called to calculate the user's confusion level from the operation history data 111. The process proceeds to step 503 without waiting for the execution result of the user confusion level detection unit 106, and the erroneous operation level detection unit 107 is called to calculate the user's erroneous operation level from the operation history data 111. In this embodiment, the user confusion level detection unit 106 and the erroneous operation level detection unit 107 are executed in parallel, but it is not always necessary to execute them in parallel. The execution order of the user confusion level detection unit 106 and the erroneous operation level detection unit 107 may be reversed.

  FIG. 7 is a flowchart of processing in the user confusion level detection unit 106. The response result of the application in the operation history data 111 objectively indicates the user's intention to operate, and it is possible to grasp what operation flow the user is working by following the response result of the application in time series. Therefore, the degree of confusion is calculated for each response result of the application and stored in the operation history data 111. In step 701, the latest data in which the response result of the application exists is read from the operation history data 111. Data that is initially stored in the operation history data 111 always includes a response result of the application such as “display initial screen”. Therefore, the user confusion level detection unit 106 may obtain the confusion level by always paying attention only to the response results of recent applications.

  FIG. 8 is an example of the operation history data 111. The degree of confusion 0.6 stored in the uppermost data in FIG. 8 indicates the degree of confusion up to the data in which the response result of the next application exists. That is, this is a value obtained from the behavior of the operation from the operation with a start time 801 of 1050 milliseconds to the start time 801 of 10000 milliseconds in FIG. In FIG. 8, the latest user operation history is an operation with a start time 801 of 16050 milliseconds, and the latest data having a response result of an application to be read in step 701 is data with a start time 801 of 16000 milliseconds. .

  In step 702, all the latest data from the data read in step 701 is read. In the example of FIG. 8, only data with a start time 801 of 16050 milliseconds is applicable. In step 703, it is confirmed whether there is data to be read in step 702. If the data exists as in the example of FIG. 8, the process proceeds to step 704, and if the data does not exist, the process proceeds to step 706, where the degree of confusion of the data read in step 701 is set to 0, and the process of the user confusion degree detection unit 106 Exit. In the example of FIG. 8, the confusion level of data whose start time 801 is 16000 milliseconds is 0 because the data read in step 701 in the previous processing by the user confusion level detection unit 106 has the start time 801. This is because the data is 16000 milliseconds and there is no data to be read in step 702, and therefore, the degree of confusion of the data read in step 701 is set to 0 in the processing in step 706.

  In step 704, the data read in step 702 is collated with the confusing operation rule 110. FIG. 9 is an example of a puzzle operation rule. In FIG. 9, an operation pattern 901 indicates a time series pattern of operation contents in the operation history data. A condition 902 indicates an operation condition that affects the degree of confusion in each operation pattern 901. The degree of confusion 903 indicates the degree of confusion for each combination of the operation pattern 901 and the condition 902.

  Here, how to obtain each operation pattern 901, condition 902, and confusion level 903 will be described. Separately, a large amount of operation history data and video data showing the state of the operation are collected in advance, and while showing each video data to one subject, a five-level subjective evaluation of the degree of confusion as shown in FIG. 10 is performed for each operation content. Have them go. That is, if the user operating in the video feels confused, he / she is asked to enter the degree of confusion in five stages. FIG. 10 divides the numbers from 0 to 1 into five, and the greater the value, the greater the degree of confusion. By having this subjective evaluation experiment performed by as many subjects as possible, it is possible to obtain the operation pattern and conditions and the degree of confusion that are statistically attributed to the user's confusion.

  Now, the processing of step 704 will be described with an example of the operation history data in FIG. 8. The operation read in the data read in step 702 is “stop” and the stop time is “12” seconds. This data satisfies the operation pattern “stop” and the condition “8 [s] or more” of the confusion operation rule of FIG. In step 705, if the data read in step 702 does not satisfy the confusing operation rule 110 as a result of the collation in step 704, the process proceeds to step 706, and if satisfied, the process proceeds to step 707. In step 707, the degree of confusion is obtained based on the confusion operation rule 110, stored in the data read in step 701, and the process of the user confusion degree detection unit 106 is terminated. In step 704, when the data read in step 702 satisfies a plurality of rules of the puzzle operation rule 110, the total value of each puzzle level is stored in the data read in step 701.

  FIG. 11 is a flowchart of processing in the erroneous operation degree detection unit 107. In step 1101, the latest data in which an application response result exists is read from the operation history data 111. In step 1102, it is confirmed whether the response result of the application read in step 1101 matches the response result of the application read in the previous processing of the erroneous operation degree detection unit 107. If so, it can be interpreted that an objective operation has not been performed since the previous time, so the processing of the erroneous operation degree detection unit 107 is terminated. In step 1103, by grasping the current operation in the operation flow probability model 112 representing the general operation flow of the application with probability from the operation flow in which the response results of the application of the operation history data 111 are arranged in time series, In step 1101, the erroneous operation degree of the user operation corresponding to the response result of the application read is obtained.

  FIG. 12 shows an example of the operation flow probability model 112, in which the circle is called a node and the arrow is called an arc, and the numerical value attached to the arrow represents the transition probability from the node at the start point of the arrow to the node at the end point. The operation flow probability model 112 is a graph structure model having a user operation accompanied by an application response as a node, and an application response result of the operation history data 111 can be associated with a node of the operation flow probability model 112. Also, as shown in FIG. 12, the transition probability between nodes is registered in the operation flow probability model 112 with a value from 0 to 1. This inter-node transition probability can be determined by a statistical probability of an operation flow obtained from a large amount of operation history data separately collected in advance. Since this inter-node transition probability represents the objective accuracy of the user operation corresponding to the target transition, the degree of erroneous operation can be known from “1-transition probability”.

  When the operation flow probability model 112 of FIG. 12 is used and the error operation degree of the operation history data 111 shown in FIG. 8 is obtained, the operation whose response result of the application is “display program search screen” in FIG. This corresponds to the operation of transition from node 1201 “display initial screen” to node 1203 “display program search screen”. Since the node transition probability at this time is 0.35, the erroneous operation degree is 1−0.35 = 0.65. Further, in FIG. 8, the operation whose response result is “select terrestrial digital broadcast program” is selected from node 1203 “display program search screen” to node 1205 “select terrestrial digital broadcast program” in FIG. Corresponds to the operation transitioned to. Since the node transition probability at this time is 0.2, the degree of erroneous operation is 1−0.2 = 0.8. The degree of erroneous operation thus obtained is stored in the operation history data 111. Then, the processing of the erroneous operation degree detection unit 107 is ended.

  Here, in the operation flow probability model 112, the inter-node transition probability is dynamically changed according to the operation history data 111 by holding the inter-node transition probability for each operation history of the user. Detection can also be performed. FIG. 17 shows an example of the internode transition probability for each user operation history. The transition probabilities registered in the four arcs extending from the node 1703 “select program” in FIG. 17A transition from the node 1701 “display externally viewed program viewing screen” to the node 1703 “select program”. This is a value calculated from a general operation history of the operation. In addition, the transition probability in the four arcs extending from the node 1703 “select program” in FIG. 17B is the operation of transition from the node 1702 “display recorded program viewing screen” to the node 1703 “select program”. This is a value calculated from a general operation history. As described above, by holding the operation flow probability model 112 having different transition probabilities for each user operation history, it is possible to dynamically change to a transition probability corresponding to the user operation history 111.

  FIG. 13 is a flowchart of processing in the score integration unit 108. In step 1301, it is determined whether or not the score integration unit 108 has started processing. The processing of the score integration unit 108 starts when the user's processing execution request is received from the information processing unit 104 or when the degree of confusion obtained by the user confusion level detection unit 106 exceeds a predetermined threshold. Proceed to 1302. In other cases, the processing of the score integration unit 108 ends. In step 1302, the degree of erroneous operation with a high degree of accuracy is obtained by adding the values normalized by the difference between the degree of confusion in each data of the operation history data 111 and the degree of erroneous operation and the equal probability in each node of the operation flow probability model 112. Find a certain degree of integrated misoperation.

  FIG. 14 is an example of the operation history data 111 including the degree of confusion and the degree of erroneous operation. In FIG. 14, “screen type” and “object name under the mouse” are omitted from the elements of the operation history data 111. Also, the reason why the erroneous operation degree of data having a start time 1401 of 16000 milliseconds shown in FIG. 14 is not stored is that the response result after the application response result 1405 “select terrestrial digital broadcast program” is still unknown. This is because the degree of erroneous operation cannot be determined.

  A method for calculating the integrated erroneous operation degree will be described using data with a start time 1401 of 1050 milliseconds in FIG. The integrated error operation degree is an addition value 0.9 of the difference value 0.3 between the degree of confusion 0.6, the error operation degree 0.8, and the equal probability 0.5 in the node “display program search screen” of the operation flow probability model 112. Here, in the present embodiment, an added value of the degree of confusion and the normalized degree of operation error is used as the integrated operation error degree, but there is no particular limitation as long as it is a calculation method using the degree of confusion and the degree of operation error.

  In step 1303, a search is made for the maximum degree of each integrated erroneous operation in the operation history data 111. In step 1304, it is confirmed whether or not there is a plurality of pieces of data having the maximum integrated erroneous operation value. If there are a plurality of data, the process proceeds to step 1305. If there are not a plurality of data, it is estimated that the operation corresponding to the data having the maximum integrated error operation degree in step 1303 is an operation error, and the process proceeds to step 1306. In step 1305, priority is given to the data with the same degree of integrated operation error, which has a higher degree of confusion, to narrow down to one, and the operation that caused the operation error is estimated. Here, the narrowing-down method from the same integrated error operation degree is variously limited, such as increasing the priority of the operation history data closest in time from the current operation. In step 1306, the correction information presenting unit 109 is called to present the erroneous operation location and the recommended operation to the user.

  FIG. 15 is a flowchart of processing in the correction information presenting unit 109. In step 1501, a node corresponding to the erroneous operation obtained by the score integration unit 108 is found from the operation flow probability model 112, and the next node having the highest transition probability is excluded from the next nodes that can be transitioned from that node. The corresponding operation is the correct operation at this node. Referring to FIG. 12 as an example, the node corresponding to the erroneous operation obtained by the score integration unit 108 is the node 1201 “display initial screen”, and the erroneous operation node is node 1202 “display recorded program viewing screen”. At this time, the node corresponding to the correct operation among the remaining two transitionable nodes becomes the node 1203 “display program search screen” having a high transition probability. Here, the transition probability between nodes in the operation flow probability model 112 is changed by using data defining the degree of confusion obtained by the user confusion degree detection unit 106 and the weighting amount to the transition probability between nodes of the operation flow probability model 112. And more accurate operation can be obtained. In step 1502, a node corresponding to the current operation screen is found from the operation flow probability model 112, and an operation corresponding to the next node with the highest transition probability among the next nodes that can be transitioned from that node is set as a correct operation. In step 1503, the operation history data 111, the correct operation on the current operation screen, and the correct operation on the operation screen with an erroneous operation are presented to the user as help information. Here, the operation history data 111 can be displayed with an emphasis on the part indicating the latest operation or erroneous operation. Further, when the number of history operations in the operation history data 111 is large, the number of displays can be reduced by a narrowing method such as displaying only operations with a high degree of confusion. Furthermore, an operation having the highest transition probability after the correct operation currently presented can be presented by the user's selection. Similarly, by the user's selection, an operation with the highest integrated erroneous operation level can be presented after the currently present erroneous operation. In this case, it is necessary to obtain the correct operation again in step 1501. When the user selects the correct operation presented, the system state is changed to the corresponding operation screen, and the correct operation is highlighted on the screen. The correct operation highlighting means can be variously considered to enlarge the button or point to the corresponding button with an arrow, but is not limited as long as it is a conspicuous display method.

  FIG. 16 shows an example of help information presentation. The user operation history 1601 in FIG. 16 arranges the user operation history 111 that has an application response result in time series from the top, and highlights the operation with the erroneous operation and the latest operation in bold. Also, the correct operation 1602 in FIG. 16 presents the correct operation for the operation highlighted in the user operation history 1601. In addition, when the user executes the button 1603, highlighting of an operation with an erroneous operation in the user operation history 1601 is stopped, and an operation with an erroneous operation of the next candidate is highlighted. Further, when the user executes the button 1604, the correct operation in the correct operation 1602 is changed to the correct operation of the next candidate.

It is a conceptual lineblock diagram of an information processor to which the present invention is applied. It is a hardware block diagram of the information processing apparatus to which this invention is applied. It is a flowchart of the process in an information processing part. It is a data table which shows an example of the input information of an information processing part. It is a flowchart of the process in a user operation behavior acquisition part. It is a data table which shows an example of operation history data. It is a flowchart of the process in a user confusion degree detection part. It is a data table which shows an example of operation history data. It is a data table which shows an example of a puzzle operation rule. It is an example of the evaluation format used for 5 step | paragraph subjective evaluation for determining a puzzle operation rule. It is a flowchart of the process in a misoperation degree detection part. It is a graph structure figure which shows an example of an operation flow probability model. It is a flowchart of the process in a score integration part. It is a data table which shows an example of operation history data. It is a flowchart of the process in a correction information presentation part. It is an example of the user presentation screen in a correction information presentation part. It is a figure which shows an example of the transition probability between nodes for every operation history of a user.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Input part 102 Output part 103 UI process part 104 Information processing part 105 User operation behavior acquisition part 106 User confusion degree detection part 107 Error operation degree detection part 108 Score integration part 109 Correction information presentation part 110 Confusion operation rule 111 Operation history data 112 Operation Flow probability model

Claims (5)

An input device;
An output device for displaying a response result of an application to the operation by the input device;
A user operation behavior acquisition unit that accumulates an operation history by the input device;
A confusion level detection unit that calculates a degree of confusion that indicates how much the user is confused with the operation,
An erroneous operation degree detection unit for obtaining an erroneous operation degree indicating the degree of operation error in the user's operation;
A score integration unit that integrates the degree of confusion and the degree of erroneous operation, and estimates an erroneous operation location from an operation history by the input device;
A correction information presenting unit for displaying the operation history by the input device and the location of the estimated erroneous operation on the output device;
The user operation behavior acquisition unit acquires the input information from the input device, the screen information of the application, and a response result to the input information, holds the acquired information as an operation history,
The confusion level detection unit obtains the degree of confusion by checking the rule indicating the relationship between the operation pattern and the degree of confusion and the current operation history,
The erroneous operation degree detection unit obtains an erroneous operation degree by comparing a model expressing a general operation flow of an application with a probability and a current operation history,
The said score integration part calculates the value which integrated the said confusion degree and the said erroneous operation degree for every operation in the said operation history, and makes the operation location where it takes the maximum value be an erroneous operation location, The information processing apparatus characterized by the above-mentioned.   The information processing apparatus according to claim 1, wherein a probable operation at the erroneous operation location is estimated from a model expressing a general operation flow of the application with a probability.   The information processing apparatus according to claim 2, wherein the operation history and a likely operation at the erroneous operation location are displayed on the output device, and when the reliable operation is selected by the input device, the application state is changed to the erroneous operation location. An information processing apparatus characterized by being returned to the state.   The information processing apparatus according to claim 1, wherein the erroneous operation degree detection unit includes data indicating a dependency relationship between an operation flow of an application and an operation history, and a general operation flow of the application from an operation history by the input device. An information processing apparatus characterized by changing each probability value of a model in which is expressed by probability.   The information processing apparatus according to claim 1, wherein the score integration unit assigns a priority by comparing the degree of confusion when a plurality of erroneous operation locations are obtained.

Images