JP5082819B2 - Operation support apparatus, operation support method, and operation support program - Google Patents

Description

  The present invention relates to an operation support apparatus, an operation support method, and an operation support program that support user operations.

  Workers use a plurality of applications that run on a computer, such as browsers, mailers, softphones, and document creation software, to work. In business execution, specific operations are often performed repeatedly.

  For example, Patent Document 1 discloses an operation support method for automatically executing an operation with a high probability of being continuously executed by comparing past operation histories regarding operations such as mail transmission, mail browsing, and game activation performed by an operator. It is disclosed.

JP 2004-152276 A

  The problem is that it is less effective as support for operations related to actual work. In actual business, browsers and mailers are always started up, and it is not uncommon for the operation target of one application to change from time to time. Therefore, by looking at the relevance between operations by focusing on the operation of the application, even if there are many cases of “composing a new mail” after “browsing a web page”, applications such as the web page to be viewed and the destination of the mail If attention is paid to the operation target of, the high relevance is not always uniformly generated. It is also conceivable that an operation performed in connection with a certain operation is executed by different operation means for the same person, such as mail transmission or softphone transmission.

  Assuming such a situation, the relevance between operations without consciously distinguishing the operation of the application related to the operation and the operation target for one operation as described in Patent Document 1 It is unlikely that the next operation that is really relevant will be obtained.

  Note that Patent Document 1 also discloses a method of storing operation-related information together with special information and performing operation prediction based on operation history information including the special information. However, since information such as date, user type, and region is cited as an example of special information, it is said that the relevance should be obtained from information that combines the operation and the situation (environment) in which the operation was input. In other words, it is not considered that the relationship between the operation of the application and the operation target is consciously determined to obtain the relationship.

  Accordingly, an object of the present invention is to provide an operation support apparatus, an operation support method, and an operation support program that are highly effective in actual business.

An operation support apparatus according to the present invention is an operation support apparatus that supports a user operation, and includes, as operation history information of an application, operation identification information for identifying the operation, target information indicating an operation target of the operation, An operation history storage unit that stores information including information and the operation of the associated application, including operations that do not involve an active operation by an operator, are detected, and a predetermined operation and its operation target are detected in the application An application monitoring unit, an operation target conversion rule storage unit that stores a conversion rule for converting an operation target detected from the application into information expressed in a predetermined format, and a predetermined operation by the application monitoring unit. When the operation target operation is detected, compatible conversion rule and the detected operation target is acted upon conversion rule Stories When stored in the parts are the operation target which is detected in accordance with the conversion rule on converted into information represented by the predetermined format, and a target information indicating operation identification information of the detected operation and the operation target Correspondingly, the operation history recording means to be stored in the operation history storage unit, the operation to be predicted and the operation target are detected, the combination of the past operation and the operation target stored in the operation history storage unit For the target, a method for obtaining the degree of association between the detected motion of the prediction target and the combination of the motion target as a unit, and the sum of the degree of association obtained for each motion and for each motion target is the relevance of the combination determined using a plurality of methods including a method to calculate a final relevance based on relevance by each method determined, a combination final degree of association calculated higher as the predicted candidate The prediction candidate extraction means to be issued, and the operation and operation target determined based on the prediction candidate extracted by the prediction candidate extraction means are set as the operation of the application related to the next user operation and the operation target of the operation to the corresponding application. Application execution instruction means for executing is provided.

The operation support method according to the present invention is an operation support method for supporting a user operation, and monitors the operation of the associated application including the operation not accompanied by an active operation by an operator . A predetermined operation and its operation target are detected in the application, and when the predetermined operation and its operation target are detected, if a conversion rule that matches the detected operation object is stored, it is detected according to the conversion rule. on converting the by operation target information represented by a predetermined format, in association with the subject information indicating the detected operation identification information of the work operation target, it is stored as the history information of an application operation, operation and the operation target is detected, a combination of past operation as the subject being stored as the history information of the application operation of the prediction target The subject, the degree of association between a combination of operation as subject of the prediction target detected, to a method of obtaining the combination as a single unit, the relevance of combining the sum of the degree of association determined by operation and by operation target Using a plurality of methods including methods, calculating the final relevance based on the relevance of each method found , and extracting the combination with the calculated high final relevance as a prediction candidate, The operation and the operation target determined based on the extracted prediction candidates are executed by the corresponding application as the operation of the application related to the next user operation and the operation target of the operation.

An operation support program according to the present invention is an operation support program for supporting a user operation, and performs an operation of an application associated with a computer without an active operation by an operator. including monitoring, application monitoring process of detecting the predetermined operation and the operation target in the application, a predetermined operation with the operation object is detected, is detected from the application a conversion rule stored in advance If a conversion rule that matches the detected operation target exists in the conversion rule for converting the operation target into information expressed in a predetermined format, the operation target detected according to the conversion rule is after having converted the information represented by the form, the object information indicating an operation identification information of the detected operation and the operation target And association, the operation history recording process Ru is stored as history information of the application operation, the operation and the operation target is detected, the past operation as the subject being stored as the history information of the application operation of the prediction target For a combination of the above, a method for obtaining the degree of association between the detected motion of the prediction target and the combination of the motion target as a unit, and the sum of the relevance obtained by the motion and the motion target Using a plurality of methods including the method of calculating the degree of relevance, calculating the final relevance based on the relevance of each method obtained, and using the calculated combination with the highest relevance as the prediction candidate extracting predicted candidate extraction processing, and extracted the determined operation and the operation target based on the predicted candidates, movement of the application according to the following user operation And as an operation target of said operating, characterized in that to execute the application execution instruction process to be executed by the corresponding application.

  According to the present invention, it is possible to provide an operation support device, an operation support method, and an operation support program that are highly effective in actual work.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of an operation support apparatus according to the present invention. As shown in FIG. 1, the operation support apparatus according to the present invention includes an application monitoring unit 11, an operation history registration unit 12, a prediction candidate extraction unit 13, an application execution instruction unit 14, and an operation history storage unit 21. .

  The operation history storage unit 21 stores, as application operation history information, information including operation identification information for identifying the operation and target information indicating the operation target of the operation.

  The application monitoring unit 11 monitors the operation of the application associated with the application monitoring unit 11 and detects a predetermined operation and its operation target in the application.

  When the application monitoring unit 11 detects a predetermined operation and its operation target, the operation history recording unit 12 associates the operation identification information for identifying the detected operation with target information indicating the operation target. And stored in the operation history storage unit 21.

  When the motion to be predicted and the motion target are detected, the prediction candidate extraction unit 13 detects the predicted motion to be detected from the combination of the past motion and the motion target stored in the motion history storage unit 21. A combination having a high degree of association between the action and the action target combination is extracted as a prediction candidate for the action content of the application related to the next user operation.

  For example, the prediction candidate extraction unit 13 detects each combination of the past motion and the motion target as the prediction target based on the appearance frequency after the same combination as the motion and the motion target combination in the motion history. The prediction candidate may be extracted by calculating the degree of association between the performed motion and the combination of motion targets. Also, for example, each combination of past motion and motion target based on the appearance frequency by motion and the appearance frequency by motion target after the same combination as the motion and motion target combination to be predicted in the motion history The prediction candidate may be extracted by calculating the degree of association between the motion detected as the prediction target and the combination of the motion targets.

  The application execution instructing unit 14 sets the operation and the operation target determined based on the prediction candidate extracted by the prediction candidate extracting unit 13 as the application operation related to the next user operation and the operation target of the operation as a corresponding application. Let it run.

  Further, as illustrated in FIG. 2, the operation support apparatus may include a prediction candidate presentation unit 15 and an operation target conversion rule storage unit 22. The prediction candidate presenting means 15 presents a combination of an action and an action target extracted as a prediction candidate to a worker as a selection item, and selects one prediction candidate. When presenting the prediction candidate, the prediction candidate presenting means 15 presents one content of the operation or the operation target included in the combination of the operation of the application extracted as the prediction candidate and the operation target as a narrowing selection item. May be selected.

  In such a case, the application execution instructing unit 14 uses one prediction candidate selected by the operator by the prediction candidate presenting unit 15 as the application operation related to the next user operation and the operation target of the operation. The corresponding application may be executed.

  The operation target conversion rule storage unit 22 stores a conversion rule for converting an operation target detected from an application into information expressed in a predetermined format. In such a case, when the application monitoring unit detects a predetermined operation and its operation target, the operation history registration unit 12 determines the operation target according to the conversion rule stored in the operation target conversion rule storage unit. The target information converted into information expressed in a predetermined format is stored in the operation history storage unit. Note that by storing the target information in a predetermined format, the prediction candidate extraction unit 13 can determine substantially the same operation target even if the expression format of the operation target is different between applications. It is possible to calculate the relevance with high accuracy.

  Hereinafter, a more specific configuration example will be described. FIG. 3 is a block diagram showing a more specific configuration example of the operation support apparatus according to the present invention. 3 includes an information processing device 100 such as a CPU, a storage device 200 such as a memory, an input device 300 such as a keyboard, and an output device 400 such as a display device.

  The information processing apparatus 100 includes an application group 110, an application plug-in unit 120 corresponding to each application included in the application group 100, an operation history recording unit 131, a prediction candidate extraction unit 132, a prediction candidate presentation unit 133, A prediction candidate execution instruction unit 134. In addition, the storage device 200 includes an operation history storage unit 201, an operator information storage unit 202, a prediction candidate selection history storage unit 203, and an operation target conversion rule storage unit 204.

  The application group 110 includes one or more applications (more specifically, processing of an application program) used by an operator such as a browser, mailer, soft phone, and document creation software installed in the operation support apparatus. Entity). In the example shown in FIG. 3, the browser 110-1, the mailer 110-2, the soft phone 110-3, and the document creation software 110-4 are illustrated. However, for example, spreadsheet software, presentation creation, etc. Software, project management software, video viewing software such as TV can be used.

  The application plug-in unit 120 is provided in association with each application included in the application group 110. Each application plug-in unit 120 includes an application monitoring unit 121 and an application execution instruction unit 122, as shown in FIG. FIG. 4 is a block diagram illustrating a configuration example of the application plug-in unit 120. In the example shown in FIG. 4, the configuration example of the application plug-in unit 120-1 corresponding to the browser 110-1 is shown, but the application plug-in unit 120 corresponding to another application may be the same. The application plug-in unit 120 is preferably realized by a mechanism generally called a plug-in (a mechanism for expanding functions as an external device without modifying the application main body). Currently, many browsers and mailers provide a mechanism for extending functions in the form of plug-ins.

  In the example shown in FIG. 3, a browser application plug-in unit 120-1 is shown as the application plug-in unit 120 corresponding to the browser 110-1. Further, a mailer application plug-in unit 120-2 is shown as an application plug-in unit 120 corresponding to the mailer 110-2. Further, as an application plug-in unit 120 corresponding to the soft phone 110-3, a soft phone application plug-in unit 120-3 is shown. Further, as an application plug-in unit 120 corresponding to the document creation software 110-4, an application plug-in unit 120-4 for document creation software is shown.

  The application monitoring unit 121 monitors the operation of an application (hereinafter simply referred to as a “corresponding application”) associated with the application plug-in unit 120 including the application monitoring unit 121, and detects a specific operation and its operation target. . Then, information for identifying the detected operation and information indicating the detected operation target are passed to the operation history recording unit 131 described later. The application monitoring unit 121 is a processing unit corresponding to the application monitoring unit 11 described above.

  Since the operation varies depending on the application, the operation detected by the application monitoring unit 121 and the operation target are also different for each corresponding application. For example, regarding the operation, “homepage display” is detected for a browser, “send”, “receive”, “view” of mail is detected for a mailer, and “outgoing”, “receive” are detected for a soft phone. As for the operation target, for example, in the case of the “homepage display” operation of the browser, what is the “homepage” displayed at that time (specifically, URL or the like) may be detected. Also, for example, in the case of a mailer “send” operation, what is the “mail” sent at that time (specifically, a destination such as To, Cc, Bcc, a subject, a body text, etc.) can be detected. That's fine. Further, for example, in the case of a “reception” operation of a mailer, what is the “mail” received at that time (specifically, an address or the like for requesting reception) may be detected. Further, for example, in the case of the “browsing” operation of the mailer, what is the “mail” to be browsed at that time (specifically, the sender, the subject, the text, etc.) may be detected. Further, for example, in the case of a “sending” operation of a soft phone, what is necessary is to detect what “communication” is transmitted at that time (specifically, a destination telephone number or the like). Further, for example, in the case of the “incoming” operation of the softphone, what is the “communication” received at that time (specifically, the caller telephone number or the like) may be detected.

  As an example, the browser application monitoring unit 121 monitors the operation that occurs in the browser 110-1, and when the “homepage display” operation is detected, the operation is performed and displayed as the operation target. You may detect URL of the homepage made. Then, information for identifying the “homepage display” operation of the browser and a URL (for example, http://www.aaa.com) as an operation target may be notified to the operation history recording unit 131.

  Note that the operation of an application that is a history target may include not only an operation generated by an active operation such as an input by an operator but also an operation that does not involve an active operation of the operator such as an incoming call of a softphone. .

  The application execution instructing unit 122 receives the control instruction from the prediction candidate execution instructing unit 134, and executes the instructed operation content to the corresponding application as the operation of the application related to the next user operation and the operation target of the operation. Let The operation content to be executed by the corresponding application is specified by information for identifying the operation of the application and information indicating the operation target. The application execution instruction unit 122 issues an execution instruction to the corresponding application in order to execute the instructed operation content as the operation of the application related to the next user operation and the operation target of the operation. The application execution instruction unit 122 is a processing unit corresponding to the application execution instruction unit 14 described above.

  As an example, the browser application execution instruction unit 122 receives information for identifying the “homepage display” operation and a URL as an operation target (for example, http://www.aaa.com) from the prediction candidate execution instruction unit 134. When the control instruction specified as the operation content is received, execution for displaying the homepage specified by the URL “http://www.aaa.com” for the browser 110-1, which is the corresponding application, is performed. By issuing an instruction, the specified operation content is automatically executed (without user operation).

  The operation history recording unit 131 receives information for identifying a predetermined operation that has occurred in the application group 110 and information indicating an operation target at that time from each application monitoring unit 121, and sets the information as a pair, The operation history is stored in the operation history storage unit 201 as an operation history. The operation history recording unit 131 is a processing unit corresponding to the operation history recording unit 12 described above. In the present embodiment, the operation history recording unit 131 is configured to display information indicating an operation target detected from an application in a predetermined format according to the operation target conversion rule stored in the operation target conversion rule storage unit 204. And converted into the operation history storage unit 201. By converting the operation target into another information and recording it, it is possible to easily determine the relationship between the operation targets, which is difficult if the original information remains as it is. The operation target conversion procedure will be described in detail later in the description of the operation.

  When an application operation to be predicted and its operation target are detected, the prediction candidate extraction unit 132 collates the combination of the operation and the operation target with the operation history stored in the operation history storage unit 201, and the operation history Are extracted as candidates for predicting the application operation related to the next user operation from among the combinations of the operation and the operation target included in. Note that the prediction candidate extraction unit 132 is a processing unit corresponding to the prediction candidate extraction unit 13 described above. For example, the prediction candidate extraction unit 132 may extract a prediction candidate using the latest operation generated in the application group 110 as a prediction target. Note that the procedure for collating the history to determine the relevance between the motion to be predicted and the combination of motion targets will be described in detail later in the description of the motion.

  The prediction candidate presentation unit 133 presents the prediction candidate extracted by the prediction candidate extraction unit 132, and causes the operator to select one prediction candidate as an appropriate prediction candidate. The prediction candidate presentation unit 133 is a processing unit corresponding to the prediction candidate presentation unit 15 described above. The prediction candidate presenting means 133, for example, presents only one of the motions or motion targets as a narrowing selection item for the combination of motions that are the prediction candidates and the motion targets, and then selects the selected motion or Step-by-step narrowing selection may be performed in which only combinations including the operation target are presented and selected as selection items. Further, when one candidate is selected from the user, the candidate presenting means 331 notifies the prediction candidate execution instructing means 134 of the contents of the candidate (combination of action and action target).

  For example, the prediction candidate presenting means 133 displays the candidate selection screen as shown in FIG. 5 via the output device 400 to present and select the contents of the prediction candidate to the worker. FIG. 5 is an explanatory diagram illustrating an example of a candidate selection screen displayed by the prediction candidate presentation unit 133. FIG. 5A shows an example of a candidate selection screen to be displayed first.

  In FIG. 5A, for example, “narrow down by“ Hanako Suzuki ”” indicates a narrowing selection item for narrowing down the contents of the prediction candidates to only candidates that have “Hanako Suzuki” as an operation target. In FIG. 5, selection items are underlined. In addition, “Hanako Suzuki” → softphone (outgoing) presented below causes the softphone 110-3 to perform “outgoing” operation with “Hanako Suzuki” as an operation target (destination). In addition, the selection item for selecting a prediction candidate, “Filter by mailer (transmission)” is used to narrow down the contents of the prediction candidate to candidates that operate “mailer (transmission)”. The narrowing selection items are shown.

  When the operator selects a prediction candidate selection item, the prediction candidate presenting unit 133 notifies the prediction candidate execution instructing unit 134 of the content of the prediction candidate selected by the worker (a combination of the operation and the operation target) as the determined operation content. hand over. When the operator selects the narrowing selection item, the prediction candidate presenting unit 133 presents only the combination including the selected action or action target as the prediction candidate selection item. The prediction candidate presentation unit 133 may cause the prediction candidate extraction unit 132 to newly extract a prediction candidate in addition to the selected operation or operation target as a condition. Then, the prediction candidates extracted again based on the narrowing-down conditions may be presented again to the worker as prediction candidate selection items. FIG. 5B shows an example of the candidate selection screen when the narrowing selection item “Hanako Suzuki” is selected on the candidate selection screen shown in FIG. FIG. 5C shows an example of the candidate selection screen when the narrowing selection item “Browser (Display)” is selected on the candidate selection screen shown in FIG. The worker may select an appropriate candidate according to the candidate selection screen as shown in FIG.

  The prediction candidate execution instructing unit 134 receives the content of one prediction candidate determined by the prediction candidate presenting unit 133 (a set of operation and operation target), and issues a control instruction to the application execution instruction unit 122 of the corresponding application. In addition, when the target information indicating the operation target is converted into other information according to the operation target conversion rule, the prediction candidate execution instruction unit 134 converts the information into the original information (information detected from the application). It has a function of re-converting and passing it to the application execution instruction unit 122.

  The operation history storage unit 201 associates a predetermined operation generated in the application group 110 with an operation target of the operation and stores it as history information. The operation history storage unit 201 may store, for example, operation history data as shown in FIG. 6 separately from operation history data as shown in FIG. 6 and operation target history data as shown in FIG. FIG. 6 is an explanatory diagram illustrating an example of operation history data stored in the operation history storage unit 201. FIG. 7 is an explanatory diagram illustrating an example of operation target history data stored in the operation history storage unit 201. The operation history data and the operation target history data are associated with each other by a history ID. The history ID is identification information for identifying each record as history information.

  The operation history data (one record) includes, for example, a history ID, a worker ID for identifying a worker who has performed the operation, an application in which the operation has occurred, the type of the operation, and the occurrence of the operation. The time may be included. In addition, the operation target history data (one record) may include, for example, a history ID to be associated with the operation history data, a type of the operation target, and data that records the entity of the operation target.

  Further, the worker information storage unit 202 records information related to the worker. The worker information storage unit 202 may store worker information data as shown in FIG. 8, for example. FIG. 8 is an explanatory diagram illustrating an example of information stored in the worker information storage unit 202. The worker information data may include, for example, a worker ID, a first and last name, and information used as a destination in an application such as a telephone number or a mail address. Information such as the job title and organization of the worker may also be included.

  The prediction candidate selection history storage unit 203 records the prediction candidate selected by the operator from the prediction items presented by the prediction candidate presenting unit 133 as history information. The prediction candidate selection history storage unit 203 may store, for example, prediction candidate selection history data as shown in FIG. FIG. 9 is an explanatory diagram illustrating an example of prediction candidate selection history data stored in the prediction candidate selection history storage unit 203. The prediction candidate selection history data is, for example, selected by the operator ID of the worker who selected the prediction candidate, the history ID (prediction source history ID) of the history having the operation and the operation target from which the prediction candidate is presented. The history ID (selection history ID) of the history having the predicted candidate motion, the motion target, and the selected time may be included. When there are a plurality of corresponding history IDs, all of them may be registered, or only the smallest history ID may be registered. When the history ID is reused for history information having different contents by overwriting and updating the history information, not the history ID but the information indicating the operation and the operation itself may be registered.

  The operation target conversion rule storage unit 204 stores an operation target conversion rule that is a rule for converting information detected from an application into another information having a predetermined expression format for a specific operation target. The operation target conversion rule storage unit 204 may store, for example, operation target conversion rule data as shown in FIG. FIG. 10 is an explanatory diagram showing an example of operation target conversion rule data stored in the operation target conversion rule storage unit 204. The operation target conversion rule data may include, for example, an application, an operation, and an operation target type related to the operation target to be converted, information on the conversion source data, and information on the converted data.

  Next, the operation of this embodiment will be described. FIG. 11 is a flowchart illustrating an example of an operation related to registration of an operation history of the operation support apparatus according to the present embodiment. As shown in FIG. 11, first, when one of the applications in the application group 110 is operated by an operator or the like, the application monitoring unit 121 that monitors the operation of the application detects the operation (step A1). ). For example, if the browser 110-1 operates, the application monitoring unit 121 included in the browser application plug-in unit 120 detects the operation. In addition to detecting the operation, the application monitoring unit 121 acquires information on the operation target of the operation from the application, and passes the information to the operation history recording unit 131 together with information indicating the operation (step A2). For example, the application monitoring unit 121 may define a list of operations to be detected, and may include only specific operations as extraction targets.

  Next, when the operation history recording unit 131 receives the information indicating the operation from the application monitoring unit 121 and the information regarding the operation target of the operation, the operation target conversion rule data matching the operation target is stored in the operation target conversion rule storage unit. If it exists, the received information on the operation target (that is, information detected from the application) is converted into other information according to the operation target conversion rule data (step A3). Here, the information about the operation target that matches the conversion source data of the operation target conversion rule data is converted into data specified by the conversion destination data.

  Then, using the received information and the converted information, the operation history recording unit 131 records the operation in the operation history data of the operation history storage unit 201 and records the operation target in the operation target data (step A4). ).

  FIG. 12 is a flowchart illustrating an example of operations related to automatic execution from prediction candidate extraction. First, when one of the applications in the application group 110 is operated by an operator's operation or the like, information indicating an operation history as an operation history and an operation target in the operation history recording unit 131 by the same operation as steps A1 to A3 in FIG. Is generated. The operation history recording unit 131 passes the information to the prediction candidate extraction unit 132 as information indicating the operation to be predicted and the operation target. The prediction candidate extraction unit 132 acquires the motion to be predicted and information indicating the motion target from the motion history recording unit 131 (step B1).

  When the prediction candidate extraction unit 132 acquires the motion to be predicted and information indicating the motion target, the prediction candidate extraction unit 132 predicts from the history (in this case, the motion history data and the motion target history data) stored in the motion history storage unit 201. A history (one record as history information) having the same combination as the target operation and the target combination is collated (step B2). When the same combination of histories are collated, the prediction candidate extraction unit 132 extracts the histories recorded within a certain time or within a certain number from the histories (step B3). Here, the prediction candidate extraction means 132 may limit the operation | movement of the log | history to extract to the operation | movement which generate | occur | produces by active operation. In addition, as an operation target related to one operation, there may be a plurality of pieces of information such as “To”, “subject”, and “text” if the mailer is a “send” operation. In such a case, in the verification of the operation target, the operation target may be regarded as the same if a part of a plurality of pieces of information indicating the operation target matches. In this case, the operation target of the matched history may be only a part of matched information (for example, only “To” of mail).

  Next, the prediction candidate extraction unit 132 calculates the strength of association with the prediction target for each set of motion and motion target indicated by the extracted history (step B4). The stronger the relationship is, the higher the accuracy of the operator as the prediction candidate for the next operation / operation target to be executed. Here, an index indicating the strength of the relationship is referred to as the degree of relationship. The stronger the relation, the higher the degree of relation.

  Several methods can be used as a method for calculating the degree of association. Here, as an example of calculating the degree of association, the degree of association is calculated using three different methods, and the degree of relevance of each method is calculated. The method of summing up and making the final relevance will be explained.

  The first method is calculation of the relevance at the appearance frequency of the history. The degree of relevance between a history operation that frequently appears continuously in time after a history in which the same combination as the prediction target is recorded and the set of the operation target is highly evaluated. The second method is calculation of the degree of association from the past selection history. When prediction candidates are presented in the past for the same combination as the current prediction target, the degree of association between the selected motion and the motion target pair is highly evaluated. For the selection history of the operator with respect to past prediction candidates, the prediction candidate selection history data stored in the prediction candidate selection history storage unit 203 may be referred to. The third method is to calculate the degree of association from the frequency of each motion and motion target. The history recorded by the combination of the operation and the operation target is divided into the operation and the operation target, the frequency of the operation and the operation target is calculated, and the total is set as the relevance of the combination of the operation and the operation target. That is, this is a technique for highly evaluating the relevance of a pair having a high-frequency operation and a high-frequency operation target that appear after a history in which the same combination as the prediction target is recorded. A value obtained by multiplying the degree of association calculated by these three methods by the specific gravity set for each method is used as the final degree of association between the action indicated by the extracted history and the target of the action. Although three different methods have been described here, other methods such as calculating relevance based on time intervals between history occurrences may be added.

  Next, the prediction candidate extraction unit 132 extracts a combination of an action and an operation target as a prediction candidate in descending order of relevance, and passes it to the prediction candidate presenting unit 133 (step B5). In addition, the prediction candidate extraction unit 132 passes the frequency values of the motion and the motion frequency calculated by the third method to the prediction candidate presentation unit 133.

  Next, the prediction candidate presenting means 133 displays a candidate selection screen for allowing the operator to select an appropriate candidate from the extracted prediction candidates via the output display 400 (step B6). The prediction candidate presentation unit 133 displays a candidate selection screen including two types of selection items. The first type of selection item is a selection item for selecting a prediction candidate composed of a combination of the action extracted by the prediction candidate extraction unit 132 and the action target. The second type of selection item is a narrowing selection item for selecting one of an operation or an operation target included in the prediction candidate extracted by the prediction candidate extraction unit 132.

  A plurality of methods can be applied to the method of displaying the selection item of the prediction candidate and the narrowing selection item on the candidate selection screen. Here, two examples of display methods of the candidate selection screen are shown. The example of a 1st display method is an example shown in FIG. 5 already demonstrated. The prediction candidate presenting unit 133 displays a narrowing selection item for selecting an operation or an operation target in descending order according to the frequency of the operation and the operation target received together with the prediction candidate from the prediction candidate extraction unit 132. In addition, a selection item for selecting a prediction candidate having an operation or an operation target as the narrowing selection item is displayed as a subordinate item of the narrowing selection item. Here, when there are a plurality of corresponding prediction candidates, they are displayed in descending order of relevance. In the example of the candidate selection screen shown in FIG. 5, the selection item of the prediction candidate to be displayed as a subordinate item of the operation narrowing selection item (for example, “mailer (transmission)”) is “application (operation) →“ operation target ”. The action selected as a refinement selection item such as “Execution” is displayed first. Similarly, in the prediction candidate selection item that follows the refinement candidate of the action target, ““ Action Target ”→ Execute Application (Action)” As described above, the operation target is displayed first as a narrowing selection item.

  An example of the second display method is shown in FIG. FIG. 13 is an explanatory diagram illustrating another example of a candidate selection screen displayed by the prediction candidate presenting unit 133. In the second display method, prediction candidate selection items are first displayed in descending order of the degree of relevance, and the operation of the prediction candidate or the higher frequency of the operation target is displayed as a narrowing selection item immediately below. In the example illustrated in FIG. 13, the prediction candidate having the highest degree of association is ““ Hanako Suzuki ”→ mailer (transmission)”, and the combination of the prediction candidates indicates that the operation target “Hanako Suzuki” has a higher frequency. In this example, when the narrowing selection item is displayed for the same operation or operation target, the narrowing selection item may not be displayed, both of the display methods of the two candidate selection screens. In the display of the narrowing selection item, when the motion or action target selected by the narrowing selection item is narrowed down as a condition, the prediction candidate extraction unit 132 extracts in advance whether there is a newly extracted prediction candidate. If a new prediction candidate other than the currently displayed prediction candidate is not extracted, the refinement selection item may not be displayed.

  Here, the operator selects either the prediction candidate selection item or the narrowing selection item displayed on the candidate selection screen via the input device 300 (step B7). Basically, the operator selects a prediction candidate when the next candidate to be executed is present on the candidate selection screen, and selects a candidate for operation or operation target to be executed next when there is no candidate. When the selection selection item is selected by the operator (No in Step B7), the prediction candidate presenting means 133 specifies the content (operation or operation target) of the selected selection selection item as the search condition, and performs prediction. The candidate extraction unit 132 is requested again to extract prediction candidates. Upon receiving the request, the prediction candidate extraction unit 132 adds the specified operation or operation target as a narrowing condition, and extracts a set of operations and operation targets having a high degree of association as prediction candidates in the same manner as in Step B4 and Step B5. And it passes to the prediction candidate presentation means 133 (step B8). Here, for example, when the top five cases are extracted as highly relevant combinations, combinations that do not include the relevant action or action target are excluded according to the narrowing-down conditions, so that the combination that was not extracted last time is newly added. May be extracted as a prediction candidate. The prediction candidate presenting means 133 displays the candidate selection screen again with the narrowed prediction candidates (step B6).

  On the other hand, when the operator selects the selection item for the prediction candidate (Yes in Step B7), the prediction candidate presenting means 133 first includes the history ID of the history having the pair of the motion and the motion target to be predicted this time. The history candidate ID selected by the operator and the history ID of the history having the set of motion targets are associated with the selected time and stored in the prediction candidate selection history storage unit 203 (step B8). Note that instead of the history ID, information indicating the operation and the operation itself may be registered. Next, the prediction candidate presenting means 133 passes the content of the selected prediction candidate (a combination of the action and the action target) to the prediction candidate execution instructing means 134 as the determined action content.

  When the prediction candidate execution instructing unit 134 receives the content of one prediction candidate determined from the prediction candidate presenting unit 133 (a set of an operation and an operation target), the operation target conversion rule data that matches the operation target is converted into an operation target conversion. If it exists, the received information on the operation target (that is, information registered as history information) is converted into the original information (that is, the information registered as history information) according to the operation target conversion rule data. , Information detected from the application) is converted back (step B8). Here, the information of the operation target that matches the conversion destination data of the operation target conversion rule data is converted into data specified by the conversion source data. Note that both the post-conversion information and the pre-conversion information may be registered in the operation target history data, and the pre-conversion information registered may be acquired. Then, the prediction candidate execution instruction unit 134 issues a control instruction to the application execution instruction unit 122 of the corresponding application. In the control instruction, using the received information and the converted information, an operation content (a set of an operation and an operation target) to be executed by the application is specified.

  The application execution instructing unit 122 receives the control instruction from the prediction candidate execution instructing unit 134, and executes the instructed operation content to the corresponding application as the operation of the application related to the next user operation and the operation target of the operation. (Step B11).

  In addition, in the explanation of the operation of the prediction candidate extraction / automatic execution, the prediction target is one set of the motion and the motion target, but the degree of association is calculated by using two or more consecutive motion and motion target sets as the prediction target. May be. That is, operation prediction is performed by collating two or more histories having a combination of continuous movements and movement targets executed by the operator's operation. In this case, in Step B2, the same operation / operation target group as two or more operation / operation object groups is collated with a history of appearance.

  As described above, according to the present embodiment, the operation history of an application is recorded as a combination of independent information of an operation and an operation target, and the relationship between the prediction target and the combination of the operation and the operation target is further extracted in the prediction candidate extraction. Since the degree is calculated, a prediction candidate in which the worker's state is further limited by the operation target is possible as compared with the prediction candidate extraction of only the relevance between the operations. In addition, by recording an independent combination of information, the motion and the motion target are separated, and the degree of relevance to the prediction target is calculated for each. Can be extracted.

  Further, in the present embodiment, the prediction candidate presenting means 133 presents one of the actions or action targets as the narrowing selection candidate on the candidate selection screen for presenting the prediction candidate to the worker, and narrows down the prediction candidate step by step. Thus, the worker can efficiently select an appropriate prediction candidate. Also in this case, the presentation order can be changed according to the degree of association between the action and the action target. For example, if you often contact Mr. A after viewing a certain web page, even if the means (application operations) such as email, phone calls, and instant messages are different, the candidate related to Mr. A will be given priority. Can be presented.

  Hereinafter, the operation from the prediction candidate extraction to the automatic execution will be described using a more specific example. In this embodiment, the browser 110-1, the mailer 110-2, the soft phone 110-3, and the document creation software 110-4 shown in FIG. Therefore, as an application plug-in unit 120 corresponding to each application, a browser application plug-in unit 120-1, a mailer application plug-in unit 120-2, a softphone application plug-in unit 120-2, and a document creation And a software application plug-in unit 120-4.

  In this example, it is assumed that the worker information data in the worker information storage unit 202 stores the data example shown in FIG. Further, it is assumed that the data example shown in FIG. 9 is stored in the prediction candidate selection history data in the prediction candidate selection history storage unit 203. Further, it is assumed that the operation target conversion rule storage unit 204 stores the data example illustrated in FIG. Further, by executing the operation of each step shown in FIG. 11 according to the operation by the worker A in advance, the operation history storage unit 201 stores an example of the operation history data shown in FIG. 6 and FIG. Assume that an example of operation target history data is stored.

  First, it is assumed that an operator displays a homepage with a title (TITLE) “material ordering method” and a URL “http://www.jp/xxx/shisai.html” on the browser 110-1. The application monitoring unit 121 of the browser application plug-in unit 120-1 detects the operation of the browser 11-1, and the "display" operation is performed and the operation target is URL = "http: // www". .Jp / xxx / shizai.html "and TITLE =" material ordering method ". The application monitoring unit 121 passes the detected information to the operation history recording unit 131. Based on the information received from the application monitoring unit 121, the operation history recording unit 131 generates information indicating an operation as an operation history and information indicating an operation target, and records the information in the operation history data and the operation target history data. Here, it is assumed that history ID = 300 is stored in the operation history data and the operation target history data of the operation history storage unit 201.

  The information indicating the operation as the operation history and the information indicating the operation target are passed to the prediction candidate extraction unit 132 as information indicating the operation to be predicted and the operation target. Note that the application monitoring unit 121 may directly pass to the prediction candidate extraction unit 132, and the prediction candidate extraction unit 132 may convert the information as the operation history and the information indicating the operation target as necessary. Hereinafter, the notation of the information indicating the motion to be predicted and the information indicating the motion target are the operation “browser (display)”, the operation target “URL (http://www.jp/xxx/shisai.html), TITLE ( Material ordering method) ”” and simplified.

  Next, the prediction candidate extraction unit 132 collates the history having the same set as the motion to be predicted and the motion target with the motion history data and motion target history data stored in the motion history storage unit 201. Search (step B2). Here, three histories of history ID = 1, 100, 200 correspond. Next, the history recorded within a predetermined time (within 30 minutes as an example) of the history ID = 1, 100, 200 is extracted from the operation history data and the operation target history data. As a result, history ID = 2-5, 101, 201-203 are extracted (step B3).

  Next, the relationship between the motion indicated by the extracted history and the motion target group is calculated (step B4). In this embodiment, (1) relevance calculation based on history appearance frequency, (2) relevance calculation from past selection history, and (3) relevance calculation from the sum of the frequency of each operation and operation target, The degree of relevance is calculated by three methods, the degree of relevance of (1): the degree of relevance of (2): the degree of relevance of (3) is multiplied by the specific gravity of 2: 3: 1, and the final degree of relevance is calculated. To do.

  First, (1) relevance calculation based on the appearance frequency of history will be described. For the degree of relevance (1), the degree of relevance is obtained based on the number of duplicated occurrences (ie, appearance frequency) of histories that appear in duplicate. Regarding the verification of the operation target in the overlap determination, as described above, if a part of a plurality of pieces of information indicating the operation target matches, the operation target is regarded as the same. At that time, the operation target of the matched history is only a part of the matched information (for example, only “To” of mail). In addition, the smallest history ID among the duplicate histories is used as the history ID after collecting the duplicate histories. For example, it can be seen that the combinations of history ID = 2 and history ID = 202 are identical and appear overlapping. In such a case, the history ID = 2 is collected, and 2 is counted as the appearance frequency of the history ID = 2. When the relevance degree is +1 for the appearance frequency 1, it is calculated as “relevance degree of (1)” shown in FIG. FIG. 14 is an explanatory diagram illustrating a calculation result of the degree of association in the present embodiment. As shown in FIG. 14, history ID = 3 and history ID = 202 are also summarized as history ID = 3 as overlapping history, and the degree of association = 2 is calculated. For other history IDs, since the number of duplicates is 1 (appearance frequency = 1), the degree of association = 1 is calculated.

  Next, (2) relevance calculation from a past selection history will be described. When the data of the prediction candidate selection history data (FIG. 9) in the prediction candidate selection history storage unit 203 is collated, for the prediction source history ID = 1 having the same combination as the current prediction target operation and the operation target pair, There is one history in which a prediction candidate with history ID = 2 has been selected in the past. When the degree of relevance is +1 (added by the number of cases) for the number of selected cases 1, the degree of relevance is calculated as “(2) degree of relevance” shown in FIG. As shown in FIG. 14, the degree of association = 1 is calculated for the history ID = 2, and the number of selected cases is 0 for the other history IDs, so the degree of association = 0 is calculated.

  Next, (3) calculating relevance from the sum of the frequencies of the motion and the motion object will be described. First, the extracted history is divided into an operation and an operation target, and the frequency for each is calculated. Here, the frequencies shown in FIG. 15 are calculated for the frequencies related to the operations. Further, the frequencies shown in FIG. 16 are calculated as the frequencies related to the operation target. FIG. 15 is an explanatory diagram illustrating the calculation result of the appearance frequency for each operation according to the present embodiment. Moreover, FIG. 16 is explanatory drawing which shows the calculation result of the appearance frequency classified by operation target in a present Example. In FIG. 16, some items with an appearance frequency of 1 are omitted. In FIG. 16, there is an item whose operation target type is “telephone number, To” at the frequency related to the operation target. This is because the operation target type has the same conversion destination data in the operation target conversion rule data. This is because if the contents of the data are the same, it is determined that they are the same operation target. When the appearance frequency for each operation and the appearance frequency for each operation target are obtained, the frequency of the combination of the operation and the operation target indicated by the extracted history is calculated by summing the frequency of the operation included in the combination and the frequency of the operation target. To do. Since there are many types of motion targets and low duplication probability compared to motion, in this embodiment, (frequency of motion) × 0.5 + (frequency of motion target) is the total relevance. For example, since the history ID = 2 is a combination of the operation “softphone (transmission)” and the operation target “phone number (worker ID: 0002)”, the operation frequency = 2 and the operation target frequency = 3. The total degree of association = 2 × 0.5 + 3 = 4. Similarly, other combinations are also calculated as “(3) relevance” shown in FIG.

  Then, after calculating the respective degrees of association (1) to (3), the total value of the degrees of association is calculated according to the above-mentioned specific gravity. In the present embodiment, the total relevance = relevance of (1) × 2 + (2) relevance × 3 + (3). The calculation result of the total relevance is as shown in FIG.

  Next, the prediction candidate extraction unit 132 sequentially extracts a pair of operations and operation targets having a high degree of association as prediction candidates and passes them to the prediction candidate presentation unit 133 (step B5). In this embodiment, assuming that a maximum of four prediction candidates are passed in descending order of relevance, a combination of history ID = 2, 3, 101, 4 is passed. Here, a combination of “soft phone (transmission)” with history ID = 2 and “phone number (worker ID: 0002)”, “document creation software (open)” with history ID = 3, and “file name (ordering method) .Doc) ”, a combination of“ mailer (transmission) ”with history ID = 101 and“ To: mail address (worker ID: 0002) ”,“ mailer (transmission) ”with history ID = 4 and“ To: A combination of “email address (worker ID: 0003)” is passed. When the relevance is the same value, the one with the smaller history ID is given priority. In addition, the prediction candidate extraction unit 322 passes the frequencies of the motion and the motion target, that is, the information of FIGS. 15 and 16 to the prediction candidate presentation unit 133.

  Next, the prediction candidate presenting means 133 displays a candidate selection screen via the output device 400 (step B6). Here, the first display method is used from the two types of display methods described above. FIG. 17 shows an example of a candidate selection screen in the present embodiment. FIG. 17A shows an example of a candidate selection screen to be displayed first. FIG. 17B shows an example of the candidate selection screen when the narrowing selection item “Browser (display)” is selected on the candidate selection screen shown in FIG. In the example of the first display method, the narrowing selection items are presented in the descending order of the frequency of the operation and the operation target, and the selection item of the prediction candidate including the operation or the operation target is presented as the lower item. In this embodiment, referring to FIG. 15 and FIG. 16, the operation target “phone number, To (worker ID: 0002)” → the operation target “file name (ordering method.doc)” → the operation “mailer (transmission)”. -> Operation "Browser (Display)"-> Operation "Softphone (Transmission)"-> Operation "Document creation software (Open)"-> Operation target "To (Worker ID: 0003)" Here, in the case of the same frequency, the operation frequency has priority over the operation. In the case of the same movement or movement frequency, any order may be used.

  Therefore, the narrowing selection item to be displayed first is the operation target “telephone number, To (worker ID: 0002)”. Since there are two prediction candidates including the operation target, history ID = 2, 101, the prediction candidates are displayed in descending order of relevance. Note that in this embodiment, even if the refinement condition is added, no prediction candidate other than the current prediction candidate is extracted, and therefore, the refinement selection item is not displayed and only the prediction candidate is displayed.

  In this way, the selection item of the prediction candidate (history ID = 2, 101) including the operation target “phone number, To (worker ID: 0002)” having the highest appearance frequency is presented, and the next highest appearance frequency is presented. The selection item of the prediction candidate (history ID = 3) including the operation target “file name (ordering method.doc)” is presented, and the selection item of the last remaining prediction candidate (history ID = 4) is presented. If a new prediction candidate is extracted by adding a narrowing condition even if there is no current prediction candidate, only that narrowing selection item may be displayed. For example, the narrowing selection item of the operation “Browser (Display)” in the example shown in FIG. 17A corresponds to this condition.

  Here, when the selection item of the operation “browser (display)” is selected by the operator (No in step B7), the prediction candidate presenting means 133 designates the operation “browser (display)” as the narrowing-down condition. Then, the prediction candidate extraction unit 132 is made to extract prediction candidates again (step B8). In this embodiment, history ID = 5,203 is extracted. In response to the re-extraction result, the prediction candidate presenting means 133 displays a candidate selection screen as shown in FIG. 17B (step B6).

  Here, a prediction candidate having a combination of “browser (display)” with history ID = 5, “URL (http://www.jp/xxx/tosho.html), TITLE (tosho ordering method)” from the worker. Assume that a selection item is selected. The prediction candidate presenting means 133 has a history ID = 1 in the prediction candidate selection history data stored in the prediction candidate selection history storage unit 203 and the history ID having the smallest history ID in the same history ID as the prediction source history ID. Then, record ID = 5 selected this time is recorded as the selection history ID (step B8). Further, “browser (display)” and “URL (http://www.jp/xxx/tosho.html), TITLE (tosho ordering method)”, which are combinations of the prediction candidates selected by the prediction candidate execution instruction unit 134. give.

  When the prediction candidate execution instructing unit 134 receives the content of the prediction candidate (a combination of the operation and the operation target) determined from the prediction candidate presenting unit 133, the operation candidate is converted as necessary, and an application execution instruction for the corresponding application is performed. A control instruction is issued to the means 122. In this embodiment, since the conversion of the operation target is not necessary, the received operation and the set of operation targets are directly specified as the operation contents, and a control instruction is sent to the application execution instruction unit 122 of the browser application plug-in unit 120-1. put out.

  The application execution instructing unit 122 of the browser application plug-in unit 120-1 receives the control instruction from the prediction candidate execution instructing unit 134, and uses the specified operation content as the operation of the application related to the next user operation and the operation. Is executed by the browser 110-1. Here, an execution instruction is issued to the browser 110-1 in order to display a URL (http://www.jp/xxx/tosho.html) designated as an operation target.

By the above operation, the operator can execute the operation related to the next operation only by selecting the presented candidate, and therefore the operation support such as executing the operation related to the next operation in advance is performed. Can do. In addition, it can be activated with the mail title and subject already entered, and the amount of work can be reduced compared to the actual operation of the operator.
Can be planned.

  In the above-described embodiment, an operation history storage unit (for example, an operation history storage unit that stores information including operation identification information for identifying the operation and target information indicating the operation target of the operation as the operation operation history information) , An operation history storage unit 21), an application monitoring unit (for example, application monitoring unit 11) that monitors the operation of the associated application, detects a predetermined operation and its operation target in the application, and the application monitoring unit When a predetermined operation and an operation target of the operation are detected by the operation history recording, the operation history storage unit stores the operation identification information of the detected operation and the target information indicating the operation target in association with each other. Means (for example, action history recording means 12), and when the motion to be predicted and the motion target are detected, Prediction candidate extraction that extracts, as a prediction candidate, a combination having a high degree of association between the detected motion of the prediction target and the combination of the motion targets from among the combinations of the past motions and the motion targets stored in the motion history storage unit Means (for example, prediction candidate extraction means 13) and the action and operation target determined based on the prediction candidate extracted by the prediction candidate extraction means as the operation of the application related to the next user operation and the operation target of the action A configuration example of an operation support apparatus including application execution instruction means (for example, application execution instruction means 14) to be executed by a corresponding application is shown.

  Further, in the above embodiment, the prediction candidate extracting unit is configured to predict each combination of the past operation and the operation target based on the appearance frequency after the same combination as the combination of the operation to be predicted and the operation target in the history. A configuration example of an operation support apparatus that extracts prediction candidates by calculating the degree of association between a motion detected as a target and a combination of motion targets is shown. A specific example of this calculation method is shown as (1) relevance calculation based on the appearance frequency of the history described above.

  Further, in the above embodiment, the prediction candidate extraction unit is configured to obtain a past based on the appearance frequency for each operation and the appearance frequency for each operation target after the same combination as the combination of the operation to be predicted and the operation target in the history. A configuration example of an operation support apparatus that extracts prediction candidates by calculating the degree of association between each combination of the motion and the motion target and the combination of the motion detected as the prediction target and the motion target is shown. A specific example of this calculation method is shown as (2) relevance calculation from the sum of the frequencies of the motion and the motion target.

  Further, in the above-described embodiment, a prediction candidate presenting unit (for example, the prediction candidate presenting unit 15) that presents a combination of an operation and an operation target extracted as a prediction candidate to an operator and selects one prediction candidate is provided. The application execution instruction unit causes the corresponding application to execute the operation and operation target of the one prediction candidate selected by the operator by the prediction candidate presenting unit as the operation of the application related to the next operation and the operation target of the operation. A configuration example of the operation support apparatus is shown.

  Further, in the above embodiment, the prediction candidate presenting means presents one content of the operation or the operation target included in the combination of the operation of the application extracted as the prediction candidate and the operation target as a narrowing candidate, and selects it step by step. A configuration example of the operation support apparatus is shown.

  In the above-described embodiment, the operation target conversion rule storage unit (for example, the operation target conversion rule storage unit 204) stores a conversion rule for converting the operation target detected from the application into information expressed in a predetermined format. And the operation history recording means detects the predetermined action and its operation target by the application monitoring means, and determines the operation target in a predetermined format according to the conversion rule stored in the operation target conversion rule storage unit. A configuration example of an operation support apparatus that stores target information converted into expressed information in an operation history storage unit is shown.

  The present invention can be suitably applied particularly to an apparatus that performs business using a plurality of applications continuously.

It is the block diagram which showed schematic structure of the operation assistance apparatus by this invention. It is the block diagram which showed the other structural example of the operation assistance apparatus by this invention. It is a block diagram which shows the more specific structural example of the operation assistance apparatus by this invention. 3 is a block diagram illustrating a configuration example of an application plug-in unit 120. FIG. It is explanatory drawing which shows the example of the candidate selection screen which the prediction candidate presentation means 133 displays. It is explanatory drawing which shows the example of the operation history data which the operation history memory | storage part 201 memorize | stores. 6 is an explanatory diagram illustrating an example of operation target history data stored in an operation history storage unit 201. FIG. It is explanatory drawing which shows the example of the information which the worker information storage part 202 memorize | stores. It is explanatory drawing which shows the example of the prediction candidate selection log | history data which the prediction candidate selection log | history memory | storage part 203 memorize | stores. It is explanatory drawing which shows the example of the operation target conversion rule data which the operation target conversion rule memory | storage part 204 memorize | stores. It is a flowchart which shows an example of the operation | movement which concerns on registration of the operation history of an operation assistance apparatus. It is a flowchart which shows an example of the operation | movement which concerns on automatic execution from the prediction candidate extraction of an operation assistance apparatus. It is explanatory drawing which shows the other example of the candidate selection screen which the prediction candidate presentation means 133 displays. It is explanatory drawing which shows the example of a calculation result of relevance. It is explanatory drawing which shows the example of a calculation result of appearance frequency classified by operation | movement. It is explanatory drawing which shows the example of a calculation result of appearance frequency classified by operation object. It is explanatory drawing which shows an example of a candidate selection screen.

Explanation of symbols

DESCRIPTION OF SYMBOLS 120 Application plug-in part 121,11 Application monitoring means 122,14 Application execution instruction means 131,12 Operation history recording means 132,13 Prediction candidate extraction means 133,15 Prediction candidate presentation means 134 Prediction candidate execution instruction means 201,21 Operation history Storage unit 202 Worker information storage unit 203 Prediction candidate selection history storage unit 204, 22 Operation target conversion rule storage unit

Claims (8)

An operation support device that supports user operations,
An operation history storage unit that stores information including operation identification information for identifying the operation and target information indicating an operation target of the operation as application operation history information;
Application monitoring means for monitoring the operation of the associated application, including an operation not accompanied by an active operation by an operator , and detecting a predetermined operation and its operation target in the application;
An operation target conversion rule storage unit for storing a conversion rule for converting an operation target detected from an application into information expressed in a predetermined format ;
When a predetermined operation and its operation target are detected by the application monitoring means, if a conversion rule that matches the detected operation target is stored in the operation target conversion rule storage unit, it is detected according to the conversion rule. The operation history is stored in the operation history storage unit by converting the detected operation target into information expressed in a predetermined format and associating the operation identification information of the detected operation with the target information indicating the operation target. Recording means;
When the motion to be predicted and the motion target are detected, the combination of the motion to be predicted and the motion target detected for the combination of the past motion and the motion target stored in the motion history storage unit The degree of association with each other is determined by using a plurality of methods including a method for obtaining a combination as a unit and a method for determining the total degree of association obtained for each operation and operation target as a combination relevance. A prediction candidate extraction means for calculating a final relevance level based on a relevance level by the technique , and extracting a combination having a high calculated final relevance level as a prediction candidate;
Application execution instruction means for causing the corresponding application to execute the operation and the operation target determined based on the prediction candidate extracted by the prediction candidate extraction means as the operation of the application related to the next user operation and the operation target of the operation; An operation support apparatus comprising: A prediction candidate presenting means for presenting a combination of an action and an action target extracted as a prediction candidate to an operator and selecting one prediction candidate;
The application execution instruction unit causes the corresponding application to execute the operation and operation target of the one prediction candidate selected by the operator by the prediction candidate presenting unit as the operation of the application related to the next operation and the operation target of the operation. Item 4. The operation support device according to Item 1 . The operation according to claim 2 , wherein the prediction candidate presenting unit presents one content of the operation or the operation target included in the combination of the operation of the application extracted as the prediction candidate and the operation target as a narrowing candidate, and selects in stages. Support device. As the prediction candidate history information selected by the operator, the combination of the operation and the operation target that is the source of the prediction candidate and the operation of the prediction candidate selected by the operator and the combination of the operation target are stored in association with each other. A prediction candidate selection history storage unit;
Prediction candidate extraction means is intended for combinations of past operations and operation targets stored as application operation history information , and application operations and operations that appear frequently in succession after the same combination as the prediction target The first relevance calculated using a method that highly evaluates the target combination and the combination of the motion and the motion target selected when the prediction candidate is presented for the same combination as the prediction target are highly evaluated The second relevance calculated using the method, and the operation and operation target of the application that frequently appear in succession after the same combination as the prediction target are highly evaluated, and the sum of them is combined. The third degree of association calculated using the evaluation method is obtained, and the final degree of association calculated based on the obtained three types of association degrees is high. Operation support apparatus according to claim 2 or claim 3 extracts the combined viewed as prediction candidates. An operation support method for supporting a user operation,
A conversion rule for converting the operation target detected from the application into information expressed in a predetermined format is stored in advance.
Monitor the operation of the associated application, including the operation that does not involve an active operation by the operator, detect a predetermined operation and its operation target in the application,
When the predetermined operation and its operation target are detected, if a conversion rule that matches the detected operation target is stored, the operation target detected according to the conversion rule is expressed in a predetermined format. after having converted the information, in association with the subject information indicating operation identification information of the detected operation and the operation target, is stored as the history information of an application operation,
When the operation and its operation subject in prediction target is detected, the subject combination of past operation as the subject being stored as the history information of the application operation, the detected the prediction target operation and behavior of the target The degree of association with the combination was obtained using a plurality of methods including a method for obtaining the combination as one unit and a method for obtaining the total degree of association obtained for each operation and operation target as the degree of association of the combination. Calculate the final relevance based on the relevance of each method , extract the combination with the calculated high final relevance as prediction candidates,
An operation support method, characterized in that an operation and an operation target determined based on an extracted prediction candidate are caused to be executed by a corresponding application as an operation of an application related to a next user operation and an operation target of the operation. Presenting a combination of an action and an action target extracted as a prediction candidate to the worker, and selecting one prediction candidate,
The operation of the one prediction candidate selected by the worker and the operation target are caused to be executed by the corresponding application as the operation of the application related to the next operation and the operation target of the operation,
The combination of the operation and the operation target that is the source of the prediction candidate and the operation of the prediction candidate selected by the operator and the combination of the operation target are associated and stored as history information of the prediction candidate selected by the operator. ,
When the motion to be predicted and the motion target are detected, the detected motion of the prediction target and the motion target of the past motion and motion target stored as the history information of the application motion are targeted. As the degree of association with the combination, a first degree of association calculated using a method for highly evaluating the combination of the operation and the operation target of the application that frequently appears in succession after the same combination as the prediction target, A second relevance calculated using a technique that highly evaluates the combination of the motion and the motion target selected when the prediction candidate is presented for the same combination as the prediction target, and the same combination as the prediction target The application operation and the operation target that frequently appear in succession later are highly evaluated, and the total of them is used as a combination evaluation. Obtains a third degree of association is calculated using the operation according to claim 5, the final degree of association is calculated on the basis of the three kinds of the degree of association determined is extracted as prediction candidates with high combination Support method. An operation support program for supporting user operations,
On the computer,
An application monitoring process for monitoring the operation of the associated application, including an operation not accompanied by an active operation by an operator , and detecting a predetermined operation and its operation target in the application ;
When the predetermined operation and its operation target are detected, the conversion rule is stored in advance, and is included in the conversion rule for converting the operation target detected from the application into information expressed in a predetermined format. If a conversion rule that matches the detected operation target exists, the operation target detected according to the conversion rule is converted into information expressed in a predetermined format, and then the operation identification information of the detected operation and in association with the object information indicating an operation target, the operation history recording process Ru is stored as history information of the application operation,
When the operation and its operation subject in prediction target is detected, the subject combination of past operation as the subject being stored as the history information of the application operation, the detected the prediction target operation and behavior of the target The degree of association with the combination was obtained using a plurality of methods including a method for obtaining the combination as one unit and a method for obtaining the total degree of association obtained for each operation and operation target as the degree of association of the combination. Based on the degree of association by each method, the final degree of association is calculated , and the prediction candidate extraction process that extracts the combination with the calculated final degree of association as a prediction candidate , and is determined based on the extracted prediction candidate operation and the operation target, as the operation target of the operation and said operating applications according to the following user operation, a to be executed by the corresponding application An operation support program for executing application execution instruction processing. On the computer,
A prediction candidate presentation process in which a combination of an action and an action target extracted as a prediction candidate is presented to the worker and one prediction candidate is selected;
The combination of the operation and the operation target that is the source of the prediction candidate and the operation of the prediction candidate selected by the operator and the combination of the operation target are associated and stored as history information of the prediction candidate selected by the operator. Run the prediction candidate selection history recording process,
In the application execution instruction process, the operation and operation target of the one prediction candidate selected by the worker are executed by the corresponding application as the operation of the application related to the next operation and the operation target of the operation,
In the prediction candidate extraction process, as a degree of association between the detected motion of the prediction target and the combination of the motion target, a combination of the operation and the motion target of the application that appears frequently in time after the same combination as the prediction target The first relevance calculated using a method that highly evaluates the method and a method that highly evaluates the combination of the motion and the motion target selected when the prediction candidate is presented for the same combination as the prediction target The second relevance calculated in the above and the operation and operation target of the application that frequently appear in succession after the same combination as the prediction target are evaluated highly, and the sum of them is used as the evaluation of the combination. The third degree of association calculated using the method is obtained, and a combination having a high final degree of association calculated based on the three types of obtained association degrees is used as a prediction candidate. Operation support program according to claim 7 for extraction.

Images