JP7524690B2 - JUDGMENT PROGRAM, JUDGMENT METHOD, AND INFORMATION PROCESSING APPARATUS - Google Patents

Description

The present invention relates to a judgment program, etc.

As part of efforts to improve business efficiency, RPA (Robotic Process Automation), which automates simple system operations, is attracting attention.

For example, when testing the screen operation functions of a web application, workers manually perform the same operations repeatedly, but by using an RPA tool, the operations can be automated, reducing the burden on workers. Automatic playback of RPA is performed based on a scenario that records the operation commands and the operation targets.

However, if the source of the web app is modified during development, or if an attempt is made to run the same scenario on a different browser or a browser of a different size, the target of the operation may not be found and the scenario may not work correctly. For this reason, there is a demand for technology that allows autoplay to operate stably even if the scenario execution environment changes.

There is prior art 1 as a technique for playing back a scenario in an environment different from the environment in which the scenario was created. Figure 15 is a diagram for explaining prior art 1. In Figure 15, a structured document that records operation commands when creating a scenario is structured document 10, and a structured document in another environment in which the scenario is played back is structured document 20. Structured document 10 has nodes 10a to 10i. Structured document 20 has nodes 20a to 20i.

When the operation command is recorded, a scenario is generated in which a specific operation is performed on node 10g of structured document 10. In prior art 1, when such a scenario is run on structured document 20, the specific operation is executed on node 20g corresponding to node 10g.

Here, in conventional technology 1, even if the structured document is modified and the node to be operated is deleted after the scenario is created, the operation can be executed without interruption. In conventional technology 1, if the corresponding node is deleted, the parent node is substituted for the operation target, and the series of automatic processes continues.

For example, when reproducing an operation based on a scenario, a case will be described in which node 20g of structured document 20 corresponding to node 10g of structured document 10 is deleted. In this case, in prior art 1, a predetermined operation is executed on node 20b corresponding to parent node 10b of node 10g.

However, as in prior art 1, when the corresponding node has been deleted, substituting the parent node as the operation target does not mean that the node to be operated on is selected appropriately. Figure 16 is a diagram for explaining the problems with prior art 1. In Figure 16, the structured document that records the operation command when creating a scenario is structured document 15, and the structured document of another environment in which the scenario is played back is structured document 25. Structured document 15 has nodes 15a to 15i. Structured document 25 has nodes 25a to 25h.

The screen corresponding to structured document 15 is called screen 16. Screen 16 has parts 16b to 16i. Nodes 15b to 15i of structured document 15 correspond to parts 16b to 16i of screen 16.

The screen corresponding to structured document 25 is called screen 26. Screen 26 has parts 26b to 26h. Nodes 25b to 25h of structured document 25 correspond to parts 26b to 26h of screen 26.

For example, in prior art 1, a scenario will be described in which a specific operation (click operation) is performed on node 15f of structured document 15. When such a scenario is run on structured document 15, part 16f (button) on screen 16 is clicked.

Next, when the above scenario is run on structured document 25, the following operation is performed. In prior art 1, since there is no node corresponding to node 15f in structured document 25, a click operation is performed on node 25c corresponding to parent node 15c of node 15f. Here, since node 25c corresponds to part 26c (area) of screen 26, the operation of clicking on such an area is different from the operation of clicking the button described above, and no appropriate correspondence is performed.

Modifications to structured documents are not limited to the deletion of nodes as described in FIG. 15, but also include modifications that add nodes and modifications that change the structure itself. When such modifications are made, it is difficult for conventional technology 1 to appropriately select the node to be operated on.

A method using tree edit distance is known as a technique for matching elements (nodes) even when there is a change between tree structures. Tree structure editing processing indicates the degree of difference between elements in each of the tree structures being compared. The method using tree edit distance is referred to as Prior Art 2. In Prior Art 2, differences between elements in tree structures are expressed by editing operations of "insertion," "deletion," and "replacement," and costs are set for the editing operations to find the matching of elements with the minimum cost.

Figure 17 is a diagram for explaining an editing operation related to the tree structure edit distance. The tree structure before the editing operation is called tree structure 5. In tree structure 5, nodes b and c are connected below node a. Nodes d and e are connected below node c.

When the editing operation "insert" is executed on tree structure 5, tree structure 5 becomes tree structure 5a. In tree structure 5, node f is inserted under node a, and node c is connected under node f.

When the editing operation "replace" is performed on tree structure 5, tree structure 5 becomes tree structure 5b. In tree structure 5b, node c has been replaced with node f.

When the editing operation "delete" is executed on tree structure 5, tree structure 5 becomes tree structure 5c. In tree structure 5c, node c is deleted, and nodes d and e, which were connected to node c, are now connected under node a.

Figure 18 is a diagram for explaining prior art 2. In Figure 18, the structured document before editing is structured document 17, and the structured document after editing is structured document 27. In prior art 2, structured document 17 is compared with structured document 27, editing operations are performed on structured document 17, and HTML elements of structured document 17 are associated with HTML elements of structured document 27. The total cost is determined according to the number of editing operations performed on structured document 17, and the association that results in the minimum cost is obtained.

In association (A), an editing operation is performed to insert "div2" (HTML element 17-6) and an editing operation is performed to insert "item 3" (HTML element 17-9). As a result, HTML elements 17-1 to 17-5 are associated with HTML elements 27-1 to 27-5, respectively. In addition, HTML elements 17-6 and 17-7 are associated with HTML elements 27-7 and 27-8, respectively. The total cost of association (A) is "2".

In correspondence (B), an editing operation is performed to insert "div2" (HTML element 17-6) and an editing operation is performed to insert "item 2" (HTML element 17-8). Also, in correspondence (B), an editing operation is performed to replace with "item 1" (HTML element 17-7) and an editing operation is performed to replace with "item 3" (HTML element 17-9).

As a result, HTML elements 17-1 to 17-5 are associated with HTML elements 27-1 to 27-5, respectively. HTML element 17-6 is associated with HTML element 27-9. HTML element 17-7 is associated with HTML element 27-7. The total cost of association (B) is "4."

In conventional technology 2, the matching result of matching (A) that has the smallest total cost is adopted between matching (A) and matching (B).

Japanese Patent Application Publication No. 11-296544 JP 2009-211480 A JP 2012-185760 A JP 2005-266954 A

However, the above-mentioned conventional technique 2 has the problem that it is not possible to perform appropriate node matching.

The problem is whether it is acceptable to use the element correspondence found using the tree edit distance described in Prior Art 2 as is. Figures 19 to 22 are diagrams used to explain the problems with Prior Art 2.

Now let us consider Figure 19. In Figure 19, the structured document before editing is structured document 18, and the structured document after editing is structured document 28. Structured document 18 has nodes 18a to 18i. Structured document 28 has nodes 28a to 28i. In the example shown in Figure 19, "item 3" in node 18f has been replaced with "item 4" in node 28f.

The screen corresponding to structured document 18 is called screen 19. Screen 19 has parts 19b to 19i. Nodes 18b to 18i of structured document 18 correspond to parts 19b to 19i of screen 19.

The screen corresponding to structured document 28 is called screen 29. Screen 29 has parts 29b to 29i. Nodes 28b to 28i of structured document 28 correspond to parts 29b to 29i of screen 29.

When structured document 18 and structured document 28 are matched based on prior art 2, the result is as shown in Figure 20. As shown in Figure 20, nodes 18a to 18i of structured document 18 are matched to nodes 28a to 28h of structured document 28, respectively. Here, "item 3" of node 18h is similar to "item 4" of node 28h, and the other nodes (elements) are exact matches, so the matching shown in Figure 20 is valid.

However, the question remains as to whether an operation command for node 18h in the scenario can be applied directly to node 28h. In other words, the criteria for matching nodes between an unedited structured document and an edited structured document are different from the criteria for treating the matched nodes as identical and executing the same operation command. Since the purpose of matching nodes between an unedited structured document and an edited structured document is to enable automatic playback using a scenario, it is desirable to determine whether it is acceptable to execute an operation command equivalent to the environment in which the scenario was created.

In addition, prior art 2 allows matching of similar nodes to find optimal matching.

We now move on to the explanation of Figure 21. In Figure 21, the structured document before editing is structured document 30, and the structured document after editing is structured document 35. Structured document 30 has nodes 30a to 30d. Structured document 35 has nodes 35a to 35e.

The screen corresponding to structured document 30 is called screen 31. Screen 31 has parts 31b to 31d. Nodes 30b to 30d of structured document 30 correspond to parts 31b to 31d of screen 31.

The screen corresponding to structured document 35 is called screen 36. Screen 36 has parts 36b to 36e. Nodes 35b to 35e of structured document 35 correspond to parts 36b to 36e of screen 36.

When structured document 30 and structured document 35 are matched based on prior art 2, multiple candidates for matching are obtained, as shown in FIG. 22. As shown in FIG. 22, for example, in matching candidate 30-1, nodes 30b and 35b, nodes 30c and 35c, and nodes 30d and 35d are matched. In matching candidate 30-2, nodes 30b and 35b, nodes 30c and 35e, and nodes 30d and 35d are matched.

As shown in FIG. 21, the tag type of node 30c is "button," and the operation command for node 30c is to click the button, etc. In contrast, the tag type of node 35c is "text," so if the nodes are associated based on association candidate 30-1, the operation command for node 30c cannot be executed on node 35c.

On the other hand, the tag type of node 35e is "button." Therefore, when matching is performed based on matching candidate 30-2, an operation command similar to the operation command for node 30c can be executed for node 35e.

In other words, when matching candidate 30-1 and matching candidate 30-2 are compared, it can be said that matching candidate 30-2 has a more appropriate matching relationship. However, conventional technology 2 is unable to make such a judgment and is therefore unable to match the nodes appropriately.

In one aspect, the present invention aims to provide a determination program, a determination method, and an information processing device that can perform appropriate node matching.

In the first proposal, the computer executes the following process. When recording an operation command, the computer records the operation command for a plurality of first nodes included in a first tree structure. When reproducing the operation command, the computer performs a provisional correspondence between a plurality of first nodes included in the first tree structure and a plurality of second nodes included in the second tree structure based on a comparison result between the first tree structure and the second tree structure. For the first node and the second node for which a provisional correspondence has been performed, the computer determines whether or not to associate the first node and the second node for which a provisional correspondence has been performed based on the similarity between the content of the first node and the content of the second node.

The appropriate nodes can be matched.

FIG. 1 is a functional block diagram showing the configuration of an information processing apparatus according to the present embodiment. FIG. 2 is a diagram showing an example of the data structure of the structured document table. FIG. 3 is a diagram for explaining the relationship between an HTML source, a structured document, and a WEB screen. FIG. 4 is a diagram illustrating an example of a data structure of the operation history table. FIG. 5 is a diagram illustrating an example of a data structure of the node correspondence table. FIG. 6 is a flowchart showing a process for determining similarity of contents. FIG. 7 is a diagram for explaining an example of content similarity determination. FIG. 8 is a flowchart showing the process of determining compatibility of commands included in content. FIG. 9 is a diagram for explaining an example of compatibility determination of commands of contents. FIG. 10 is a flowchart showing a processing procedure when recording operations of the information processing device. FIG. 11 is a flowchart showing a processing procedure during operation playback of the information processing device. FIG. 12 is a flowchart showing the procedure of the command execution process. FIG. 13 is a diagram showing an example of implementing a scenario. FIG. 14 is a diagram illustrating an example of a hardware configuration of a computer that realizes the same functions as the information processing apparatus of the embodiment. FIG. 15 is a diagram for explaining the prior art 1. In FIG. FIG. 16 is a diagram for explaining the problem of the prior art 1. In FIG. FIG. 17 is a diagram for explaining an editing operation related to the tree edit distance. FIG. 18 is a diagram for explaining the prior art 2. In FIG. FIG. 19 is a diagram (1) for explaining the problem of the prior art 2. FIG. 20 is a diagram (2) for explaining the problem of the prior art 2. FIG. 21 is a diagram (3) for explaining the problem of the prior art 2. FIG. 22 is a diagram (4) for explaining the problem of the prior art 2.

Below, examples of the determination program, determination method, and information processing device disclosed in the present application are described in detail with reference to the drawings. Note that the present invention is not limited to these examples.

FIG. 1 is a functional block diagram showing the configuration of an information processing device according to this embodiment. As shown in FIG. 1, this information processing device 100 has a communication unit 110, an input unit 120, an output unit 130, a storage unit 140, and a control unit 150.

The communication unit 110 is realized by a NIC (Network Interface Card) or the like, and controls communication between the control unit 150 and external devices via telecommunication lines such as a LAN (Local Area Network) or the Internet.

The input unit 120 is realized using input devices such as a keyboard and a mouse, and inputs various instruction information such as operation commands to the control unit 150 in response to input operations by the operator. The output unit 130 is realized by a display device such as a liquid crystal display, a printing device such as a printer, etc.

The storage unit 140 has a structured document table 140a, an operation history table 140b, and a node correspondence table 140c. The storage unit 140 is realized, for example, by a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.

The structured document table 140a is a table that holds information about structured documents. FIG. 2 is a diagram showing an example of the data structure of the structured document table 140a. As shown in FIG. 2, this structured document table 140a associates document identification information, structured documents, and HTML (HyperText Markup Language) sources.

Document identification information is information that uniquely identifies a structured document. A structured document is a document structure (DOM: Document Object Model) obtained by analyzing an HTML source. An HTML source is a source program for displaying a web screen.

Figure 3 is a diagram for explaining the relationship between an HTML source, a structured document, and a web screen. For example, information processing device 100 analyzes HTML source So50 to obtain structured document D50. Structured document D50 has multiple nodes D50a to D50d, and each of nodes D50b to D50d corresponds to content surrounded by the corresponding tags in HTML source So50.

For example, node D50b corresponds to content 50cb. Node D50c corresponds to content 50cc. Node D50d corresponds to content 50cd. Each piece of content included in HTML source So50 has a character string and a command function for executing the command defined.

When the information processing device 100 executes the HTML source So50, a web screen I50 is displayed on the output unit 130. The web screen I50 includes parts I50b, I50c, and I50d. Here, part I50b corresponds to node D50b. Part I50c corresponds to node D50c. Part I50d corresponds to node D50d.

Returning to the explanation of FIG. 1, the operation history table 140b is a table that holds various information related to the scenario. FIG. 4 is a diagram showing an example of the data structure of the operation history table. As shown in FIG. 4, the operation history table 140b associates execution order, operation target, operation command, and document identification information.

The execution order indicates the execution order of the scenarios. For example, operation commands are executed on the operation targets in the order of S1-1, S1-2, S1-3, ..., S1-n. The operation target indicates the node on which the operation command is to be executed. The operation command indicates a command for the operation target. The operation command may be a command other than a click. The document identification information indicates the structured document that includes the node that is the operation target.

Returning to the explanation of Figure 1, the node correspondence table 140c is a table that defines the relationship between the nodes at the time when a scenario is created and the nodes at the time when the scenario is executed. In the following explanation, the time when a scenario is created will be referred to as "time of operation recording" and the time when a scenario is executed will be referred to as "time of operation playback".

Figure 5 is a diagram showing an example of the data structure of a node correspondence table. As shown in Figure 5, the node correspondence table associates a first node with a second node. The first node indicates a node included in a structured document when an operation is recorded. The second node indicates a node included in a structured document when an operation is played back.

For example, the first node "D50a" is associated with the second node "D51a". There may be multiple candidates for the second node that corresponds to the first node. For example, the first node "D50b" is associated with the second nodes "D51b" and "D51d", and is associated with one of the second nodes by processing of the control unit 150, which will be described later. Also, if there is no second node that corresponds to the first node, the second node column will be left blank.

Returning to the explanation of FIG. 1, the control unit 150 has a screen display processing unit 150a, a recording processing unit 150b, a preprocessing unit 150c, a determination unit 150d, and an execution unit 150e. The control unit 150 is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit 150 may also be executed by an integrated circuit, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The screen display processing unit 150a acquires HTML source from an external device via the network. When a URL (Uniform Resource Locator) or the like is specified by an operator, the screen display processing unit 150a accesses the URL and acquires the HTML source. The screen display processing unit 150a registers the acquired HTML source in the structured document table 140a.

The screen display processing unit 150a generates a web screen based on the acquired HTML source, and outputs the web screen information to the output unit 130 to display the web screen.

When recording an operation, the worker refers to the web screen displayed on the output unit 130 and operates the input unit 120 to repeatedly perform the task of specifying an operation target and an operation command, thereby inputting information about the scenario into the information processing device 100. When recording an operation, the worker may input a URL to request the display of the web screen.

During operation playback, the worker may operate the input unit 120 to input a URL and specify the web screen on which the scenario is to be executed. When the screen display processing unit 150a receives a specification of the web screen on which the scenario is to be executed from the worker, it displays the corresponding web screen on the output unit 130 while outputting the HTML source corresponding to the web screen to the preprocessing unit 150c.

When recording an operation, the recording processing unit 150b receives input of an operation target and an operation command from the input unit 120, creates a scenario, and records information about the created scenario in the operation history table 140b. An example of the processing of the recording processing unit 150b is described below.

The recording processing unit 150b generates a structured document of the HTML source by analyzing the HTML source executed by the screen display processing unit 150a when the web screen is displayed on the output unit 130. The recording processing unit 150b assigns unique document identification information to the generated structured document, associates the document identification information, the structured document, and the HTML source, and records them in the structured document table 140a.

When the recording processing unit 150b receives an operation command and an input of an operation target from the input unit 120 during operation recording, the recording processing unit 150b identifies the node to be selected based on the relationship between the web screen and the node described in FIG. 3. For example, when part I50c of web screen I50 is input as the operation target, the recording processing unit 150b identifies the node to be operated as node D50c.

The recording processing unit 150b associates the execution order, the operation target, the operation command, and the document identification information, and records them in the operation history table 140b.

The recording processing unit 150b repeatedly executes the above process each time it receives an input of an operation target and an operation command from the input unit 120. In the following description, the structured document of the HTML source generated by the recording processing unit 150b at the time of operation recording is referred to as the "first structured document." The first structured document corresponds to the first tree structure.

When replaying an operation, the preprocessing unit 150c provisionally associates the nodes of the structured document at the time of recording the operation with the nodes of the structured document at the time of replaying the operation, based on the results of comparing the structures of the structured document at the time of recording the operation with the structured document at the time of executing the operation. In the following description, where appropriate, the node of the structured document at the time of recording the operation (first structured document) will be referred to as the "first node." The node of the structured document at the time of replaying the operation will be referred to as the "second node."

The preprocessing unit 150c registers the relationship between the first node and the second node that have been provisionally associated in the node correspondence table 140c. An example of the processing of the preprocessing unit 150c is described below.

When the operation is being played back, the preprocessing unit 150c obtains the HTML source from the screen display processing unit 150a and analyzes the HTML source to generate a structured document of the HTML source during the operation playback. In the following description, the structured document of the HTML source during the operation playback will be referred to as the "second structured document" where appropriate. The second structured document corresponds to the second tree structure.

The preprocessing unit 150c identifies document identification information from the operation history table 140b, and obtains the first structured document corresponding to the identified document identification information from the structured document table 140a.

The preprocessing unit 150c compares the first structured document with the second structured document, and associates each first node of the first structured document with each second node of the second structured document so that the editing cost based on the tree edit distance is minimized. For example, the preprocessing unit 150c associates the first node with the second node so that the editing cost based on the tree edit distance is minimized based on the well-known technique "An Optimal Decomposition Algorithm for Tree Edit Distance". Erik Demaine, Shay Mozes, Benjamin Rossman, Oren Weimann. In Proceedings of the 34th International Colloquium on Automata, Languages and Programming (ICALP 2007)".

The preprocessing unit 150c registers the relationship between the first node and the second node in the node correspondence table 140c. When there are multiple combinations of the first node and the second node that have the smallest editing cost, the preprocessing unit 150c associates multiple second nodes with one first node and registers them in the node correspondence table 140c. The preprocessing unit 150c may also register the relationship between the first node and the second node whose editing cost is less than a threshold value in the node correspondence table 140c.

At the time of registration by the preprocessing unit 150c, the correspondence between the first node and the second node registered in the node correspondence table 140c is a tentative correspondence. Whether or not the correspondence between the first node and the second node is appropriate is determined by the determination unit 150d, which will be described later.

During operation playback, operation commands are executed on the nodes to be operated in the execution order recorded in the operation history table 140b, and the preprocessing unit 150c repeatedly executes the above process in each execution order. Note that if the first structured document to be executed last time and the first structured document to be executed this time are the same structured document, the preprocessing unit 150c may reuse the correspondence between the first node and the second node that was previously associated.

The determination unit 150d refers to the node correspondence table 140c and determines whether the correspondence between the first node and the second node, which has been provisionally associated, is an appropriate correspondence. For example, the determination unit 150d determines whether the provisionally associated first node and the second node should be associated based on the similarity between the content of the first node and the content of the second node. When determining the above correspondence, the determination unit 150d determines the correspondence between the first node that is the target of the operation in the operation history table 140b and the second node associated with the first node. An example of the process of the determination unit 150d is described below.

The determination unit 150d obtains the content corresponding to the first node from the HTML source of the first structured document. The content corresponding to the first node is referred to as the "first content" as appropriate. The determination unit 150d obtains the content corresponding to the second node from the HTML source of the second structured document. The content corresponding to the second node is referred to as the "second content" as appropriate.

The determination unit 150d performs a process of determining the similarity of the content and a process of determining the compatibility of the commands contained in the content.

The process of determining the similarity of content executed by the determination unit 150d is described below. By executing the process of determining the similarity of content, the determination unit 150d determines whether the first node and the second node are "different nodes", "potentially the same nodes", or "the same node".

Figure 6 is a flowchart showing the process of determining the similarity of content. As shown in Figure 6, the determination unit 150d of the information processing device 100 determines whether the tag of the first content and the tag of the second content are the same (step S11). If the tags are the same (step S11, Yes), the determination unit 150d proceeds to step S15. If the tags are not the same (step S11, No), the determination unit 150d proceeds to step S12.

We now move on to the explanation of step S12. The determination unit 150d determines whether the tag of the first content and the tag of the second content are tags of the same group (step S12). If they are not tags of the same group (step S12, No), the determination unit 150d determines that the first node and the second node are different nodes (step S13).

On the other hand, if the tags are from the same group (step S12, Yes), the determination unit 150d determines that the first node and the second node are possibly the same node (step S14).

We move on to an explanation of step S15. The determination unit 150d determines whether the character string of the first content and the character string of the second content are the same (step S15). If the character strings of the contents are the same (step S15, Yes), the determination unit 150d moves on to step S17.

On the other hand, if the content character strings are not the same (step S15, No), the determination unit 150d determines whether the content character strings are similar (step S16). If the content character strings are similar (step S16, Yes), the determination unit 150d determines that the first node and the second node are the same node (step S17).

On the other hand, if the content strings are not similar (step S16, No), the determination unit 150d proceeds to step S14.

The determination unit 150d executes the process shown in FIG. 6 above to determine whether the first node and the second node are "different nodes", "potentially the same nodes", or "the same node".

When the first node and the second node are different nodes, the determination unit 150d determines that the similarity between the first content of the first node and the second content of the second node is low, and determines not to associate the first node with the second node. The determination unit 150d deletes information about the second node that is determined to be a different node from the first node from the node correspondence table 140c.

For example, the following description will be given with reference to FIG. 5. If the determination unit 150d determines that the first node D50b and the second node D51b are different nodes, the determination unit 150d deletes the second node D51b associated with the first node D50b in the node correspondence table 140c.

When the first node and the second node are the same node, the determination unit 150d determines that the similarity between the first content of the first node and the second content of the second node is high, and determines to associate the first node with the second node.

When the first node and the second node are potentially the same node, the determination unit 150d executes a process to determine the compatibility of commands included in the content between the first node and the second node.

Here, an example of content similarity determination will be described. FIG. 7 is a diagram for explaining an example of content similarity determination. In FIG. 7, HTML source 40 is the HTML source when the operation is recorded. HTML source 41 is the HTML source when the operation is played back.

HTML source 40 has content 40a and content 40b. HTML source 41 has content 41a and content 41b.

The tag "span1" of content 40a and the tag "span1" of content 41a are the same tag, and the character string of content 40a and the character string of content 41a are the same. For example, when the content of the first node is content 40a and the content of the second node is content 41a, the determination unit 150d determines that the first node and the second node are "the same node."

The tag "div" in content 40b and the tag "span2" in content 41b are different tags, but since they both have the common function of "indicating the range of a text area," they are considered to be functionally similar tags. For example, information on tags with a common function is set in advance in the determination unit 150d, and the determination unit 150d determines whether the tags are similar or not based on this information. For example, if the content of the first node is content 40b and the content of the second node is content 41b, the determination unit 150d determines that the first node and the second node are "potentially the same node."

Next, the process of determining the compatibility of commands included in the content executed by the determination unit 150d will be described. The determination unit 150d determines whether a first node and a second node, which may be the same, are compatible nodes by determining the compatibility of commands included in the content. As described below, when a first node and a second node, which may be the same, are compatible nodes, the determination unit 150d associates the first node with the second node.

Figure 8 is a flowchart showing the process of determining the compatibility of commands included in content. As shown in Figure 8, the determination unit 150d of the information processing device 100 determines whether or not a command has been registered in the content of both nodes (step S21). If the command has not been registered in the content of both nodes (step S21, No), the determination unit 150d proceeds to step S22. On the other hand, if the command has been registered in the content of both nodes (step S21, Yes), the determination unit 150d proceeds to step S24.

We move on to the explanation of step S22. The judgment unit 150d judges whether or not a command is registered in the content of only one of the nodes (step S22). If a command is registered in the content of only one of the nodes (step S22, Yes), the judgment unit 150d judges that there is no compatibility (step S23).

On the other hand, if a command is not registered in the content of only one of the nodes (step S22, No), the determination unit 150d proceeds to step S26.

We move on to an explanation of step S24. The judgment unit 150d judges whether the function names of the commands are the same (step S24). If the function names of the commands are the same (step S24, Yes), the judgment unit 150d moves on to step S26.

On the other hand, if the function names of the commands are not the same (step S24, No), the determination unit 150d determines whether the function definitions of the commands are similar (step S25). If the function definitions of the commands are not similar (step S25, No), the determination unit 150d proceeds to step S23. On the other hand, if the function definitions of the commands are similar (step S25, Yes), the determination unit 150d determines that there is compatibility (step S26).

The determination unit 150d executes the process of FIG. 8 above to determine whether the first node and the second node, which may be identical, have command compatibility. If the commands are not compatible, the determination unit 150d determines that the similarity between the first content of the first node and the second content of the second node is low, and determines not to associate the first node with the second node. The determination unit 150d deletes information about the second node that is determined to be a different node from the first node from the node correspondence table 140c.

On the other hand, if the commands are compatible, the determination unit 150d determines that the similarity between the first content of the first node and the second content of the second node is high, and determines to associate the first node with the second node.

Here, an example of content command compatibility determination will be described. FIG. 9 is a diagram for explaining an example of content command compatibility determination. In FIG. 9, HTML source 42 is the HTML source at the time of operation recording. HTML source 43 is the HTML source at the time of operation playback.

The HTML source 42 includes content 42a corresponding to the first node. For example, a button with "name="hoge"" is set as a part of the web screen corresponding to content 42a, and when a click command is executed on this button, the function "cmd1()" is called. The internal script of the function "cmd1()" is the script shown in area 42b of the HTML source 42. The internal script corresponds to the definition of the function.

The HTML source 43 includes content 43a corresponding to the second node. For example, a button with "name="hoge"" is set as a part of the web screen corresponding to content 423, and when a click command is executed on this button, the function "cmd2()" is called. The internal script of the function "cmd2()" is the script shown in area 43b of the HTML source 42.

If the function names of the functions are the same, the determination unit 150d determines that the content commands are compatible.

If the function names of the functions are different, the determination unit 150d compares the internal scripts of the functions, and if the internal scripts are the same, it determines that the commands of the content are compatible. In the example shown in FIG. 9, the function names "cmd1()" and "cmd2()" are different, but because the internal scripts are the same, it is determined that they are compatible.

On the other hand, if the function names of the functions are different and the internal scripts are different, the determination unit 150d determines that the content commands are not compatible.

Note that the determination unit 150d may determine that the commands of the content are compatible if the internal scripts are similar, even if the internal scripts do not match perfectly. For example, the determination unit 150d divides the character string of the internal script into word strings, and determines that the internal scripts are similar if the number of identical words is equal to or exceeds a predetermined percentage.

As described above, the determination unit 150d executes the process of determining the similarity of the content and the process of determining the compatibility of the commands contained in the content, so that the provisional correspondence in the node correspondence table 140c becomes a confirmed correspondence.

Returning to the explanation of FIG. 1, the execution unit 150e is a processing unit that executes a scenario based on the operation history table 140b during operation playback. The execution unit 150e identifies the second node that corresponds to the first node based on the operation target (first node) corresponding to the execution order S1-m recorded in the operation history table 140b and the note correspondence table 140c. Each time the execution order moves to the next execution order, the node correspondence table 140c is updated by the preprocessing unit 150c and the determination unit 150d described above.

The execution unit 150e executes the operation command corresponding to the execution order S1-m for the second node. The execution unit 150e repeatedly executes the above process according to the execution order S1-m (m = 1, 2, 3, 4, ... n).

Next, an example of the processing procedure of the information processing device 100 according to this embodiment will be described. FIG. 10 is a flowchart showing the processing procedure when recording an operation of the information processing device. As shown in FIG. 10, the recording processing unit 150b of the information processing device 100 receives one step of a UI (User Interface) operation on the WEB screen (step S101).

The recording processing unit 150b records the node to be operated and the operation command in the operation history table 140b (step S102). The recording processing unit 150b registers the structured document at the time of the operation record in the structured document table 140a (step S103).

If the UI operation has not ended (step S104, No), the recording processing unit 150b proceeds to step S101. If the UI operation has ended (step S104, Yes), the recording processing unit 150b ends the process.

Figure 11 is a flowchart showing the processing procedure when reproducing operations of an information processing device. As shown in Figure 11, the pre-processing unit 150c of the information processing device 100 obtains one step's worth of records from the operation history table 140b (step S201).

The preprocessing unit 150c obtains the second structured document during the operation playback (step S202). The preprocessing unit 150c compares the first structured document and the second structured document, and generates a node correspondence table 140c (step S203).

The determination unit 150d of the information processing device 100 determines whether or not a second node corresponding to the first node to be operated exists based on the node correspondence table 140c (step S204). If a second node corresponding to the first node to be operated does not exist (step S205, No), the determination unit 150d proceeds to step S201. On the other hand, if a second node corresponding to the first node to be operated exists (step S205, Yes), the determination unit 150d proceeds to step S206.

The determination unit 150d and execution unit 150e of the information processing device 100 execute the command execution process (step S206). If the scenario in the operation history table has not ended (step S207, No), the information processing device 100 proceeds to step S201. On the other hand, if the scenario in the operation history table has ended (step S207, Yes), the information processing device 100 ends the process.

Next, the processing procedure of the command execution process shown in step S206 in FIG. 11 will be described. FIG. 12 is a flowchart showing the processing procedure of the command execution process. As shown in FIG. 12, the determination unit 150d of the information processing device 100 executes a similarity determination process (step S301). The processing procedure of the similarity determination process executed by the determination unit 150d corresponds to the processing procedure described in FIG. 6.

If the first node and the second node are different nodes (step S302, Yes), the execution unit 150e of the information processing device 100 proceeds to step S307. On the other hand, if the first node and the second node are not different nodes (step S302, No), the information processing device 100 proceeds to step S303.

If the first node and the second node are the same node (step S303, Yes), the execution unit 150e of the information processing device 100 executes an operation instruction (command) on the second node (step S304).

On the other hand, if the first node and the second node are not the same node (step S303, No), the determination unit 150d executes a compatibility determination process (step S305). The process procedure of the compatibility determination process executed by the determination unit 150d corresponds to the process procedure described in FIG. 8.

If the command of the first node and the command of the second node are compatible (step S306, Yes), the execution unit 150e proceeds to step S304. On the other hand, if the command of the first node and the command of the second node are incompatible (step S306, No), the execution unit 150e skips the operation command (step S307).

Next, an example of a scenario executed by the information processing device 100 will be described. FIG. 13 is a diagram showing an example of a scenario. The web screen at the time of operation recording is assumed to be web screen 50. When a worker operates the input unit 120 to generate a scenario in which button B on the web screen 50 is clicked, the recording processing unit 150b records the operation target "button B" and the operation command "click" in the operation history table 140b.

The web screen during the operation playback is assumed to be web screen 51. The execution unit 150e of the information processing device 100 identifies button B on web screen 51 that is to be operated based on the operation history table 140b, and clicks button B on web screen 51 according to the operation command.

Next, the effects of the information processing device 100 according to this embodiment will be described. During operation playback, the information processing device 100 performs a tentative association between the first node and the second node based on the results of comparing the structures of the first structured document and the second structured document. Then, based on the similarity between the content of the first node and the content of the second node, the information processing device 100 determines whether or not to associate the first node and the second node for which a tentative association has been performed. This makes it possible to appropriately associate the first node of the first structured document with the second node of the second structured document.

When the information processing device 100 determines that the first node and the second node are the same node based on the content (tag, character string) of the first node and the content of the second node, it associates the first node with the second node. This allows operation commands for the first node to be executed on the second node without any problems when replaying the operation.

When the information processing device 100 determines that the first node and the second node are different nodes based on the content (tag, character string) of the first node and the content of the second node, it cancels the association between the first node and the second node. This makes it possible to prevent an inappropriate association between a first node and a second node.

When the command of the first node and the command of the second node are compatible, the information processing device 100 associates the first node with the second node. This makes it possible to execute an operation command corresponding to the first node on the second node even if the first node and the second node are not the same node. In addition, it is possible to prevent a second node that cannot execute an operation command on the first node from being associated with the first node.

The information processing device 100 associates a first node with a second node when the internal script of a function corresponding to the first node and the internal script of a function corresponding to the second node match or are similar. This makes it possible to execute an operation command corresponding to the first node on the second node even if the first node and the second node are not the same node.

The information processing device 100 performs a provisional correspondence between the first node and the second node based on the results of comparing the structures of the first structured document and the second structured document, thereby narrowing down the second nodes to be compared with the first node, thereby reducing calculation costs.

Next, an example of the hardware configuration of a computer that realizes the same functions as the information processing device 100 shown in the above embodiment will be described. FIG. 14 is a diagram showing an example of the hardware configuration of a computer that realizes the same functions as the information processing device of the embodiment.

As shown in FIG. 14, computer 200 has a CPU 201 that executes various types of arithmetic processing, an input device 202 that accepts data input from a user, and a display 203. Computer 200 also has a communication device 204 that transmits and receives data to and from external devices, etc., via a wired or wireless network, and an interface device 205. Computer 200 also has a RAM 206 that temporarily stores various types of information, and a hard disk device 207. Each of devices 201 to 207 is connected to a bus 208.

The hard disk device 207 has a screen display processing program 207a, a recording processing program 207b, a pre-processing program 207c, a judgment program 207d, and an execution program 207e. The CPU 201 also reads out each of the programs 207a to 207e and expands them in the RAM 206.

The screen display processing program 207a functions as the screen display processing process 206a. The recording processing program 207b functions as the recording processing process 206b. The pre-processing program 207c functions as the pre-processing process 206c. The judgment program 207d functions as the judgment process 206d. The execution program 207e functions as the execution process 206e.

The processing of the screen display processing process 206a corresponds to the processing of the screen display processing unit 150a. The processing of the recording processing process 206b corresponds to the processing of the recording processing unit 150b. The processing of the pre-processing process 206c corresponds to the processing of the pre-processing unit 150c. The processing of the judgment process 206d corresponds to the processing of the judgment unit 150d. The processing of the execution process 206e corresponds to the processing of the execution unit 150e.

Note that each of the programs 207a to 207e does not necessarily have to be stored in the hard disk device 207 from the beginning. For example, each program may be stored in a "portable physical medium" such as a flexible disk (FD), CD-ROM, DVD, magneto-optical disk, or IC card that is inserted into the computer 200. Then, the computer 200 may read and execute each of the programs 207a to 207e.

The following notes are further provided with respect to the embodiments including the above examples.

(Appendix 1) To the computer,
When recording an operation command, the operation command is recorded for a plurality of first nodes included in the first tree structure;
when reproducing an operation command, a provisional correspondence is established between a plurality of first nodes included in the first tree structure and a plurality of second nodes included in the second tree structure based on a result of comparing the first tree structure with the second tree structure;
A determination program that executes a process for determining whether or not to associate a first node and a second node that have been provisionally associated, based on a similarity between the content of the first node and the content of the second node, for the first node and the second node that have been provisionally associated.

(Appendix 2) The determination program described in Appendix 1, further comprising a process for executing an operation command associated with the first node on the second node associated with the first node.

(Appendix 3) The determination program described in appendix 1 or 2, characterized in that the process of determining whether or not to associate the first node and the second node for which the provisional association has been performed is based on the similarity between a tag or character string contained in the content of the first node and a tag or character string contained in the content of the second node, for the first node and the second node for which the provisional association has been performed.

(Appendix 4) The determination program described in appendix 1 or 2, characterized in that the process of determining corresponds the first node and the second node for which the provisional correspondence has been performed if a first command included in the content of the first node is compatible with a second command included in the content of the second node.

(Appendix 5) The determination process is characterized in that the determination program described in Appendix 4 corresponds the first node and the second node that have been provisionally associated when the function name of the first command and the function name of the second command are the same, or when the function definition of the first command and the function definition of the second command are similar.

(Supplementary Note 6) A computer-implemented determination method, comprising:
When recording an operation command, the operation command is recorded for a plurality of first nodes included in the first tree structure;
when reproducing an operation command, a provisional correspondence is established between a plurality of first nodes included in the first tree structure and a plurality of second nodes included in the second tree structure based on a result of comparing the first tree structure with the second tree structure;
A determination method characterized by executing a process for determining whether or not to associate a first node and a second node for which a tentative correspondence has been performed, based on a similarity between the content of the first node and the content of the second node.

(Appendix 7) The method of determination described in Appendix 6, further comprising the step of executing an operation command associated with the first node on the second node associated with the first node.

(Appendix 8) The method of determination described in appendix 6 or 7, characterized in that the process of determining whether or not to associate the first node and the second node for which the provisional association has been performed is based on the similarity between a tag or character string contained in the content of the first node and a tag or character string contained in the content of the second node.

(Supplementary Note 9) The method of determining as described in Supplementary Note 6 or 7, characterized in that the process of determining corresponds the first node and the second node for which the provisional correspondence has been performed if a first command included in the content of the first node is compatible with a second command included in the content of the second node.

(Appendix 10) The method of determination described in Appendix 9, characterized in that the process of determining corresponds the first node and the second node that have been provisionally associated when the function name of the first command and the function name of the second command are the same, or when the function definition of the first command and the function definition of the second command are similar.

(Supplementary Note 11) A record processing unit that records an operation command for a plurality of first nodes included in a first tree structure when recording the operation command;
a pre-processing unit that performs provisional correspondence between a plurality of first nodes included in the first tree structure and a plurality of second nodes included in the second tree structure based on a comparison result between the first tree structure and the second tree structure when reproducing an operation command;
a determination unit that determines whether or not to associate the first node and the second node that have been provisionally associated with each other, based on a similarity between a content of the first node and a content of the second node;
13. An information processing device comprising:

(Appendix 12) The information processing device described in Appendix 11, further comprising an execution unit that executes an operation command associated with the first node on the second node associated with the first node.

(Supplementary Note 13) The information processing device described in Supplementary Note 11 or 12, characterized in that the determination unit determines whether or not to associate the first node and the second node for which the provisional association has been performed, based on the similarity between a tag or character string contained in the content of the first node and a tag or character string contained in the content of the second node.

(Supplementary Note 14) The information processing device described in Supplementary Note 11 or 12, characterized in that the determination unit associates the first node and the second node for which the provisional association has been performed if a first command included in the content of the first node is compatible with a second command included in the content of the second node.

(Appendix 15) The information processing device described in appendix 14 is characterized in that the determination unit associates the first node and the second node for which the provisional association has been performed when the function name of the first command and the function name of the second command are the same, or when the function definition of the first command and the function definition of the second command are similar.

REFERENCE SIGNS LIST 100 Information processing device 110 Communication unit 120 Input unit 130 Output unit 140 Storage unit 140a Structured document table 140b Operation history table 140c Node correspondence table 150 Control unit 150a Screen display processing unit 150b Recording processing unit 150c Preprocessing unit 150d Determination unit 150e Execution unit

Claims (7)

On the computer,
When recording an operation command, the operation command is recorded for a plurality of first nodes included in the first tree structure;
when reproducing an operation command, a provisional correspondence is established between a plurality of first nodes included in the first tree structure and a plurality of second nodes included in the second tree structure based on a result of comparing the first tree structure with the second tree structure;
A determination program that executes a process for determining whether or not to associate a first node and a second node that have been provisionally associated, based on a similarity between the content of the first node and the content of the second node, for the first node and the second node that have been provisionally associated. The determination program according to claim 1, further comprising a process for executing an operation command associated with the first node on the second node associated with the first node. The determination program according to claim 1 or 2, characterized in that the process of determining whether or not to associate the first node and the second node for which the provisional association has been performed is based on the similarity between a tag or character string included in the content of the first node and a tag or character string included in the content of the second node. The determination program according to claim 1 or 2, characterized in that the process of determining corresponds the first node and the second node for which the provisional correspondence has been performed if a first command included in the content of the first node is compatible with a second command included in the content of the second node. The determination program according to claim 4, characterized in that the process of determining corresponds the first node and the second node for which the provisional correspondence has been performed when the function name of the first command and the function name of the second command are the same, or when the function definition of the first command and the function definition of the second command are similar. A computer-implemented method for determining whether a
When recording an operation command, the operation command is recorded for a plurality of first nodes included in the first tree structure;
when reproducing an operation command, a provisional correspondence is established between a plurality of first nodes included in the first tree structure and a plurality of second nodes included in the second tree structure based on a result of comparing the first tree structure with the second tree structure;
A determination method characterized by executing a process for determining whether or not to associate a first node and a second node for which a tentative correspondence has been performed, based on a similarity between the content of the first node and the content of the second node. a record processing unit that records an operation command for a plurality of first nodes included in the first tree structure when recording the operation command;
a pre-processing unit that performs provisional correspondence between a plurality of first nodes included in the first tree structure and a plurality of second nodes included in the second tree structure based on a comparison result between the first tree structure and the second tree structure when reproducing an operation command;
a determination unit that determines whether or not to associate the first node and the second node that have been provisionally associated with each other, based on a similarity between a content of the first node and a content of the second node;
13. An information processing device comprising:

Images