JP5067328B2 - Evaluation apparatus, evaluation method, and program - Google Patents

Description

  The present invention relates to an evaluation apparatus, an evaluation method, and an evaluation program having evaluation items related to system usability. In particular, the present invention relates to the validity evaluation of the boundary expression that distinguishes the inside and the outside of the area to be switched for partial display switching of the screen.

  In recent years, for the purpose of mediating between a user and a system, there is a GUI (Graphical User Interface) that accepts an input for instructing an operation to the system from a user via a screen and outputs the instruction to the system. Widely used.

  If the GUI is difficult for the user to use, the user cannot easily operate the system, and the system itself is also difficult for the user to use. Therefore, in order to make the system easy for the user, it is necessary to improve the GUI.

  However, even if the GUI is improved in a state where it is impossible to accurately grasp whether the GUI is easy to use for the user, the ease of use of the GUI is not necessarily improved.

  Therefore, in order to make the system easy to use for the user, it is important to accurately evaluate the usability representing the degree of whether the GUI is easy to use for the user.

  One method for evaluating the GUI usability described above is to manually evaluate the usability.

  When evaluating usability manually, the evaluator must actually operate the GUI element displayed in the GUI screen, and the evaluator himself / herself needs to confirm and perform the evaluation.

  Therefore, the manual evaluation method has a problem that the amount of work to be performed by the evaluator when performing the evaluation becomes large. In addition, in the manual evaluation method, if the evaluator does not have a certain level of usability evaluation, or if the evaluator has overlooked defects that affect the usability evaluation results, appropriate usability There is a problem that it becomes impossible to obtain the evaluation result.

  In view of this, a technique has been considered in which usability is evaluated using an apparatus instead of an evaluator manually evaluating GUI usability (see, for example, Patent Documents 1, 2, and 3).

  In the technique described in Patent Document 1, the GUI automatic evaluation apparatus evaluates whether or not the GUI screen is configured according to the screen design guide, without relying on human hands. In this technique, the GUI automatic evaluation apparatus converts screen design guide data input from the outside into formal rules and stores the rules. Further, the GUI automatic evaluation apparatus generates GUI information in which the data of the GUI object of the evaluation target system is described as attribute information and attribute value for each window, using a product specification, a source program, a GUI construction tool, and the like. Then, the GUI automatic evaluation apparatus outputs an evaluation result indicating whether the configuration of each GUI screen is compatible with the screen design guide based on the comparison between the stored rule and the GUI information.

  Moreover, in the technique described in Patent Document 2, the test execution device executes a test for confirming whether or not the specified user sequence has been normally executed according to the specified operation procedure, without relying on human hands. In this technique, the test execution device stores a user sequence designated by the user when executing the test and a complement sequence supplemented to execute the user sequence according to the procedure in the operation DB. Yes. Then, when the test execution apparatus sequentially executes the user sequence according to the user sequence execution procedure, the test execution apparatus determines whether the user sequence can be executed. If it is determined that the execution is possible, the test execution device executes the user sequence. If it is determined that the execution is not possible, the test execution apparatus executes the user sequence after executing the appropriate complementary sequence retrieved from the operation DB. To do.

In the technique described in Patent Document 3, the Web screen creation tool and the term check tool check whether there is an error in the terminology that is easily mislabeled in the description of the source file on the Web screen, without manual intervention. To do. In this technique, terms that are easily misrepresented, such as homonyms, feed kana and synonyms, are registered in advance. Then, the Web screen creation tool and the term check tool detect terms that are easily misregistered in advance from the source files of the Web screen. When a registered term is detected from the source file, the Web screen creation tool and the term check tool display a list of correction candidates for the term.
Japanese Patent Laid-Open No. 08-241191 JP 2004-110267 A Japanese Patent Laid-Open No. 2004-234402

  In the technology disclosed in Patent Document 1 described above, if it is not possible to obtain specifications and sources such as a prototype under development or a product of another company, it is necessary to manually write the GUI information, but the GUI information is created correctly. It becomes difficult to do. In addition, in this technology, even if the specifications are available, the specifications themselves may be incorrect, so it is necessary to perform manual evaluation on the system that is the target of the evaluation that actually operates. There is a problem that.

  Moreover, in the technique disclosed in Patent Document 2, it is possible to evaluate whether or not a specified user sequence has been successfully executed. However, this technique has a problem that it is necessary to prepare a sequence according to the system to be evaluated, and it is impossible to evaluate the GUI portion or the sequence itself in sequence execution.

  In addition, the technique disclosed in Patent Document 3 has a problem that usability cannot be evaluated for terms that are not registered.

  Furthermore, there are various indexes among the indexes for evaluating the usability for the GUI. Among such indices for evaluating usability, there is an evaluation index as to whether or not the image area displayed on the GUI screen is appropriately switched.

  When the switching of the image area displayed on the GUI screen described above is evaluated, when an operation instruction input is received from the user via the GUI screen, it is displayed on the GUI screen before and after the input. When some of the image areas are switched, it is necessary to evaluate whether or not the part of the switched image areas (hereinafter referred to as “switching area”) is visually expressed appropriately. Note that that the switching area is appropriately expressed means that the range of the switching area in each of the GUI screens before and after the input is appropriately divided by a common boundary expression.

  For this reason, even when the techniques disclosed in Patent Documents 1, 2, and 3 are used, there is a common problem that usability cannot be evaluated for switching the image area in the GUI screen.

  That is, even in these technologies, when usability is evaluated for switching the image area in the GUI screen, it is necessary to perform it manually, and the amount of work to be performed by the evaluator becomes large. It will remain.

  In addition, when the usability evaluation is manually performed for switching the image area in the GUI screen, if the evaluator misses the switching area on the GUI screen, it becomes impossible to obtain an appropriate usability evaluation result. The problem remains.

  An object of this invention is to provide the evaluation apparatus, the evaluation method, and program which solve the subject mentioned above.

  In order to solve the above-described problems, the evaluation apparatus according to the present invention has a GUI (graphical) in which, when an operation instruction is input from a user, a part of the image area displayed on the screen is switched before and after the input. When an input of an operation instruction for executing an evaluation of the GUI screen is received from the user, the operation instruction is output to a GUI element included in the GUI screen. Between the trigger specifying unit that performs the operation, the rectangular switching region that has the smallest area while surrounding the partial image region, and the GUI element that does not switch the image before and after the output. In the search area, a boundary expression indicating a boundary between a search area determination unit that determines the area as a search area and a GUI element that does not switch the switching area and the image is included in the search area. And having a boundary representation judging unit that outputs an evaluation result indicating whether present.

  In order to solve the above-described problem, the evaluation method of the present invention is a GUI (graphical) in which a part of an image area displayed on the screen is switched before and after the input of an operation instruction is received from a user. A user interface is evaluated, and when an input of an operation instruction for executing an evaluation of the GUI screen is received from the user, the operation instruction is output to a GUI element included in the GUI screen. A trigger identification process to be performed, and when the operation instruction is output, a rectangular switching area that has the smallest area while surrounding the partial image area, and a GUI element in which an image is not switched before and after the output A search area determination process for determining the search area as a search area, and a boundary expression indicating a boundary between the switching area and the GUI element to which the image is not switched is an area of the search area. And having a boundary representation determining process for outputting an evaluation result indicating whether there crab.

  Also, a program to be executed by a computer, a GUI (Graphical User Interface) that switches a part of the image area displayed on the screen before and after the input of an operation instruction is received from a user. A trigger identification procedure for outputting an operation instruction to a GUI element included in the GUI screen when an evaluation instruction to be evaluated is received from the user as an operation instruction to perform evaluation of the GUI screen; When the operation instruction is output, an area between a rectangular switching area that has the smallest area while enclosing the partial image area and a GUI element that does not switch the image before and after the output is a search area. The search area determination procedure determined as follows, and the boundary expression indicating the boundary between the switching area and the GUI element to which the image is not switched are the search area. Executing a boundary representation determining step of outputting the evaluation result indicating whether present in.

  According to the present invention, when the trigger specifying unit of the evaluation apparatus receives an input of an operation instruction for executing an evaluation of the GUI screen from the user, the trigger specifying unit outputs the operation instruction to the GUI element included in the GUI screen. To do. When the trigger specifying unit outputs an operation instruction, the search region determination unit includes a rectangular switching region that has the smallest area while surrounding a part of the image region, and a GUI element that does not switch the image before and after the output. The area between is determined as a search area. Then, the boundary expression determination unit outputs an evaluation result indicating whether or not a boundary expression indicating a boundary between the switching area and a GUI element whose image is not switched exists in the search area.

  Due to such a configuration, the GUI element inside the switching area and the external GUI element are not visually divided, so that a location having a defect in the boundary expression can be specified, and as a result, the switching area on the GUI screen is separated. The validity of the boundary expression can be evaluated without human intervention.

  Hereinafter, an evaluation apparatus (including an evaluation method and a program) according to an embodiment of the present invention will be described.

  First, the configuration of the GUI evaluation device 1 (hereinafter referred to as “evaluation device 1”) will be described with reference to FIG.

  As shown in FIG. 1, the evaluation device 1 includes an input unit 11, a trigger specifying unit 12, a search area determination unit 13, a boundary expression determination unit 14, a storage unit 15, and an output unit 16. A major technical feature of the configuration of the evaluation apparatus 1 is that it includes a search area determination unit 13 and a boundary expression determination unit 14.

  The input unit 11 receives an input for instructing the evaluation target system 17 to perform an operation from the evaluator via the GUI screen WND that mediates the evaluation target system 17 and the user.

  For example, the input unit 11 receives an input of an operation instruction for executing an evaluation of the GUI screen WND from the user. Examples of the input unit 11 include a mouse and a keyboard.

  The “GUI screen WND” here is a screen configured to include the GUI element EM.

  The “GUI element EM” is a general GUI component such as a button, tab, radio button, text box, combo box, check box, list box, drop-down list, disclosure, or link.

  Hereinafter, the GUI screen WND to be evaluated is referred to as “evaluation target screen WND”. In the following, a case where the evaluation target screen WND is the window A shown in FIG. 2 will be described as an example.

  The trigger specifying unit 12 functions as a trigger TR from among the GUI elements EM included in the evaluation target screen WND when the input unit 11 receives an input of an operation instruction for executing evaluation of the GUI screen WND from the user. It plays a role of specifying the GUI element EM.

  Here, “trigger TR” refers to a GUI element EM that switches “partial image area” displayed on the evaluation target screen WND when an operation instruction to the evaluation target system 17 is input. In the following, a part of the image area switched by the trigger TR is referred to as “switching area SW”.

  Furthermore, the “operation instruction” here is, for example, an instruction to select a selection item included in the evaluation target element CND of the evaluation target system 17 (for example, an instruction corresponding to pressing in the case of a button or a group of buttons, a list). If there are any, select an unselected item.

  The trigger identifying unit 12 refers to the GUI element information 151 stored in the storage unit 15 and sequentially extracts the GUI elements EM included in the evaluation target screen WND one by one. Then, the trigger specifying unit 12 determines the extracted GUI element EM as an “evaluation target element CND” that is an evaluation target of whether or not the trigger TR is in the evaluation target screen WND.

  The “GUI element information 151” is the coordinates of the GUI element EM (text or input / output component) constituting the evaluation target screen WND, the type of the input / output component, or the text included in the evaluation target screen WND. This is information indicating the data that is the description content.

  The GUI element information 151 has a data structure shown as an example in FIG. 3 when a part of the GUI elements EM constituting the window A shown in FIG. 2 is described.

  In the GUI element information 151 shown in FIG. 3, for each GUI element EM arranged in the window A, the item number, the type of each GUI element EM, the data describing the text, and the window A A coordinate range RNG having the origin (0, 0) at the upper left of the window frame is stored.

  Here, the coordinate range RNG is a range that the image indicating each GUI element EM occupies in the window A. In this example, the coordinate range is defined by the coordinates (X1, Y1) of the upper left corner of the rectangle surrounding the GUI elements EM1 to EM10 and the coordinates (X1 and Y1) of the lower right corner of the rectangle surrounding the images of the GUI elements EM1 to EM10. X2, Y2). For example, if the radio button 1A (GUI element EM1) shown in FIG. 2 is used, the coordinate range RNG of the GUI element information 151 shown in FIG. The coordinates (50, 57) of the lower end point are stored.

  When the trigger identification unit 12 shown in FIG. 1 determines the evaluation target element CND, the GUI element EM extracted using the GUI element information 151 is the same type of GUI such as a button, a radio button, or a link. It is determined whether or not the element EM (for example, the radio buttons 1A and 2 shown in FIG. 2) is a GUI element EM functioning as the trigger TR by being continuously arranged as a group. As a result of the determination, when it is determined that the same kind of GUI element EM as the evaluation target element CND is arranged in a straight line, the trigger specifying unit 12 performs grouping of these GUI elements EM and generates the grouping A group is determined as one evaluation target element CND.

  In addition, the trigger specifying unit 12 acquires an image of the evaluation target screen WND before outputting an operation instruction to the determined evaluation target element CND. Subsequently, the trigger specifying unit 12 assigns a label indicating “before output” to the acquired image of the evaluation target screen WND.

  Thereafter, the trigger specifying unit 12 outputs an operation instruction to the determined evaluation target element CND. When the evaluation target element CND is a GUI element EM including a plurality of selection items such as a list, the trigger specifying unit 12 outputs an operation instruction for each selection item.

  Further, after outputting an operation instruction to the evaluation target element CND, the trigger specifying unit 12 acquires an image of the evaluation target screen WND. In addition, the trigger specific | specification part 12 acquires the image of the evaluation object screen WND after the output for every selection item, when operation instruction | indication is performed with respect to the evaluation object element CND which contains several selection items.

  Subsequently, the trigger specifying unit 12 assigns a label indicating “after output” to the evaluation target image WND acquired after the operation instruction is output.

  Thereafter, the trigger specifying unit 12 determines whether or not the evaluation target element CND is the trigger TR based on whether or not a part of the image area is switched on the evaluation target screen WND before and after the output of the operation instruction.

  When performing the determination, the trigger specifying unit 12 determines a difference between images indicating the evaluation target element CND before and after output. Then, based on the determined difference, it is determined whether or not the image is switched on the evaluation target screen WND before and after the output.

  In addition, the trigger specific | specification part 12 determines the difference of the image before and behind output for every selection item, when the evaluation object element CND is GUI element EM which includes several selection items like a list. A method for determining the difference between images may be a general method, and for example, “ImageDiff” that realizes the method may be used.

  If the difference between the images of the evaluation target screen WND acquired before and after the output of the operation instruction for the evaluation target element CND cannot be detected, the trigger specifying unit 12 determines that the evaluation target element CND is not the trigger TR. .

  On the other hand, when the trigger identification unit 12 detects the difference between the images of the evaluation target screen WND acquired before and after the output of the operation instruction for the evaluation target element CND, it determines that the evaluation target element CND is the trigger TR. Then, the evaluation target element CND is specified as the trigger TR.

  When the trigger specifying unit 12 specifies the evaluation target element CND as the trigger TR, the trigger specifying unit 12 outputs information on the trigger TR to the search region determining unit 13.

  The information on the trigger TR here is, for example, the coordinate range RNG of the image indicating the GUI element specified as the trigger TR, the image of the trigger TR when the display is switched according to each selection item, and the like.

  Thereafter, the trigger specifying unit 12 performs the evaluation on the GUI element EM that has not been evaluated whether the trigger specifying unit 12 is the trigger TR.

  The search area determination unit 13 plays a role of determining a “search area SRC”, which is an area where the “boundary expression BDR” should exist, from the evaluation target screen WND.

  The “boundary expression BDR” here is a switching area in which images are switched in the evaluation target screen WND before and after the output of an operation instruction to the trigger TR specified by the trigger specifying unit 12. It is an expression showing a boundary between SW and a GUI element EM that is not switched before and after the output.

  When determining the search region SRC, first, the search region determination unit 13 compares the image before the output of the operation instruction output from the trigger specifying unit 12 with the image after the output, and based on the comparison result The switching area SW where the image is switched before and after the output is determined.

  In addition, the search area determination unit 13 surrounds the determined switching area SW, does not include other GUI elements EM outside the switching area SW, and has a maximum area (hereinafter, “adjacent maximum”). Area MX ”).

  Then, the search area determination unit 13 determines the remaining area excluding the switching area SW from the specified adjacent maximum area MX (that is, the area other than the switching area SW in the adjacent maximum area MX) as the search area SRC. To do.

  In addition, when the search area SRC is determined, the search area determination unit 13 specifies the non-terminal side NOE from among the four sides of the rectangle surrounding the search area SRC.

  Here, “non-terminal side NOE” refers to a side of the determined search area SRC that is the end of the search area SRC that must be separated from the switching area SW by the boundary representation BDR.

  The boundary expression determination unit 14 determines an image indicating the boundary expression BDR from the search area SRC determined by the search area determination unit 13 based on a rule corresponding to the type of the boundary expression BDR.

  Thereby, the boundary expression determination unit 14 uses the boundary expression BDR to convert the GUI element EM that exists inside the switching area SW that is switched before and after the output of the operation instruction and the GUI element EM that exists outside the switching area SW. A portion that is not divided, that is, a portion that is not visually expressed properly and has a defect is determined.

  Then, the boundary expression determination unit 14 outputs the evaluation result RST of the boundary expression BDR to the output unit 16.

  In this example, the “boundary expression BDR” is any one of a ruled line, a background color difference, and a margin having a width equal to or greater than a certain value. However, the boundary representation BDR is not limited to these, and any boundary representation BDR may be used as long as it visually divides the switching area SW and the GUI element EM whose image is not switched before and after the input of the operation instruction.

  Further, the boundary expression determination unit 14 stores a rule serving as a reference for determining whether or not the boundary expression BDR (in this example, ruled line, background color difference or margin) is an appropriate boundary expression BDR. . Here, the rule about the evaluation criteria as to whether or not the boundary representation BDR is appropriate is arbitrary and is not particularly limited.

  The storage unit 15 stores the GUI element information 151 shown in FIG.

  The output unit 16 outputs the evaluation result RST for the boundary expression BDR output from the boundary expression determination unit 14. An example of the output unit 16 is a display.

  Next, in the evaluation apparatus 1 having the above-described configuration, the operation of the trigger specifying unit 12 specifying the trigger TR, the operation of the search region determining unit 13 determining the search region SRC, and the boundary expression determining unit 14 of the boundary expression BDR The operation for evaluating the above will be described in detail.

  First, the trigger identification unit 12 will be described in detail with respect to a method for identifying the trigger TR by evaluating whether or not the GUI element EM included in the evaluation target screen WND is the trigger TR one by one.

  When the input unit 11 receives an input of an operation instruction for executing the evaluation of the GUI screen from the user, first, the trigger specifying unit 12 refers to the GUI element information 151 in the storage unit 15, and is shown in FIG. 2. The GUI element EM that is the evaluation target element CND for determining whether the trigger TR is in the window A is extracted. In the following, the trigger specifying unit 12 includes four GUI components, that is, radio button 1 (EM1), radio button 2 (EM2), drop-down list (EM6), and tab (EM8), as evaluation target elements of trigger TR. The case of extracting as CND will be described as an example.

  First, the trigger specifying unit 12 uses a GUI element that functions as a trigger TR when GUI elements EM extracted as evaluation target elements CND are continuously arranged as a group of GUI elements EM of the same type such as radio buttons. It is determined whether it is EM.

  As a result of the determination, when it is determined that the evaluation target element CND is a GUI element EM that functions as the trigger TR by continuously arranging the evaluation target element CND, the trigger specifying unit 12 reads the coordinate range RNG described in the storage unit 15. , It is determined whether or not the same type of GUI element EM as the evaluation target element CND is arranged in a straight line.

  When it is determined that the same type of GUI element EM as the evaluation target element CND is arranged in a straight line, the trigger specifying unit 12 groups these GUI elements EM and evaluates the group generated by the grouping as one evaluation. The target element CND is determined.

  In the example of window A shown in FIG. 2, radio button 1 (EM1) and radio button 2 (EM2) are the same type of GUI elements, and radio button 1 (EM1) and radio button 2 (EM2) Are arranged in a straight line. Therefore, the trigger specifying unit 12 groups the radio button 1 (EM1) and the radio button 2 (EM2), and determines the group as one evaluation target element CND.

  Subsequently, the trigger identification unit 12 determines the evaluation target element CND determined as the three evaluation target elements CND of the group (EM1 and EM2) generated by the grouping, the drop-down list (EM6), and the tab (EM8). It is sequentially evaluated whether each evaluation target element CND is the trigger TR.

  First, the case where the group (EM1 and EM2) is evaluated will be described.

  First, the trigger specifying unit 12 acquires an image of the evaluation target screen WND before output before outputting an operation instruction for the determined evaluation target element CND (in this example, the radio button 1A and the radio button 2). Subsequently, the trigger specifying unit 12 assigns a label indicating “before output” to the image of the evaluation target screen WND acquired before output.

  Then, the trigger specifying unit 12 outputs an operation instruction to the evaluation target element CND. The trigger specifying unit 12 outputs an operation instruction (for example, an instruction corresponding to pressing in the case of a button group) for selecting a selection item included in the evaluation target element CND of the evaluation target system 17.

  In this example, when an operation instruction for selecting the radio button 2 (EM2) is output, an evaluation target screen WND shown in FIG. 4 is displayed after the output. In the evaluation target screen WND shown in FIG. 4, a GUI element EM11 is displayed instead of the GUI element EM3 shown in FIG.

  After the output, the trigger specifying unit 12 acquires an image of the evaluation target screen WND after the operation instruction is output. Subsequently, the trigger specifying unit 12 assigns a label indicating “after output” to the evaluation target image WND acquired after output.

  Further, the trigger specifying unit 12 determines whether or not a part of the image area is switched on the evaluation target screen WND before and after the output. When performing the determination, when the evaluation target element CND is a GUI element EM including a plurality of selection items such as a list, the trigger specifying unit 12 determines the difference between the images before and after output for each selection item. Then, based on the determined difference, it is determined whether or not a part of the image area is switched on the evaluation target screen WND before and after the output.

  As a result of the determination, when it is determined that a part of the image area is switched on the evaluation target screen WND, the trigger specifying unit 12 specifies the evaluation target element CND as the trigger TR. In the example shown in FIGS. 2 and 4, the GUI element EM3 is displayed before the output of the operation instruction for the radio button 2, but after the output, a part of the image displayed before the output is displayed. The area is switched and the GUI element EM11 is displayed. Therefore, the trigger specifying unit 12 specifies each of the radio button 1 (EM1) and the radio button 2 (EM2) as the trigger TR by comparing the images on the evaluation target screen WND before and after the output.

  Furthermore, when the evaluation target element CND is specified as the trigger TR, the trigger specifying unit 12 outputs information on the trigger TR to the search region determining unit 13. For example, an image indicating a GUI element specified as the trigger TR (in this example, radio buttons 1 and 2), a coordinate range RNG, an image of the evaluation target screen WND when the display is switched according to each selection item, and the like Is output. Of the images output from the trigger specifying unit 12 to the search region determining unit 13, the image indicating the “before output” is attached to the image of the evaluation target screen WND before the operation instruction is output, and the operation instruction is output. A label indicating “after output” is attached to the image of the later evaluation target screen WND.

  The trigger specifying unit 12 executes a process for specifying the trigger TR for all the GUI elements EM in the evaluation target screen WND. In the example of window A shown in FIG. 2, the selection items of the group (EM1 and EM2) are the radio button 1 (EM1) and the radio button 2 (EM2). Before the selection instruction is output, the radio button 1 ( EM1) is selected. Therefore, the trigger specifying unit 12 outputs an instruction to select an unselected item radio button 2 (EM2).

  Next, for the trigger TR specified by the trigger specifying unit 12, the search region determining unit 13 should have a boundary representation BDR (for example, ruled line or background) that separates the switching region SW and the region other than the switching region SW. A method for determining the “search area SRC” will be described.

  First, the search area determination unit 13 determines the switching area SW as a first stage for determining the search area SRC.

  When determining the switching area SW, the search area determination unit 13 determines the difference between the image areas at the same relative position for all the images on the evaluation target screen WND corresponding to the selection items included in the trigger TR. When the determination is made, the search area determination unit 13 compares the image of the evaluation target screen WND output from the trigger specifying unit 12 with the output of the operation instruction and the image of the evaluation target screen WND after the output. .

  Then, the search area determination unit 13 determines, as the switching area SW, a rectangular area that surrounds a place where a difference appears by the comparison (that is, a place where the image area is switched) and has the smallest area.

  For example, when the image of window A shown in FIG. 2 and the image of window A shown in FIG. 4 are compared, a rectangular area that includes all the differences detected by the comparison and has the smallest area is 5 is a region indicated by a broken line in FIG. Therefore, the search area determination unit 13 determines the area as the switching area SW.

  After determining the switching area SW, the search area determination unit 13 determines the search area SRC as a second stage.

  In this case, first, the search area determination unit 13 surrounds the determined switching area SW, does not include other GUI elements EM outside the switching area SW, and has a maximum area adjacent to the rectangle. Specify MX.

  In the example of the switching area SW shown in FIG. 5, a rectangle that surrounds the switching area SW, does not include other GUI elements (EM1, EM2, EM5, and EM11) outside the switching area SW and has the largest area. This area is indicated by a solid line in FIG. Therefore, the search area determination unit 13 identifies the area as the adjacent maximum area MX.

  Subsequently, the search area determination unit 13 determines the remaining area excluding the switching area SW from the specified adjacent maximum area MX as the search area SRC. In the example shown in FIG. 6A, the search area determination unit 13 searches for an area between the adjacent maximum area MX and the switching area SW, that is, a remaining area excluding the switching area SW from the adjacent maximum area MX. It determines as area | region SRC.

  That is, the determined side of the search area SRC is in contact with the GUI element EM located outside the switching area SW, is in contact with the trigger TR, or the end of the evaluation target screen WND (for example, a window frame or the like) ). The search area determination unit 13 reads out and acquires the coordinates of the GUI element EM located outside the switching area SW from the GUI element information 151.

  Finally, as a third stage, the search area determination unit 13 separates the switching area SW and other GUI elements from the four sides surrounding the determined search area SRC by the boundary expression BDR in the search area SRC. A “non-terminal side NOE” that is a side (region end of the search region SRC) is defined. In this illustrative example, the search area determination unit 13 determines the side that is in contact with the GUI element EM located outside the switching area SW among the four sides surrounding the determined search area SRC as “non-terminal side NOE”. .

  That is, the search area determination unit 13 specifies, as a non-terminal edge NOE, an edge that is in contact with the GUI element EM whose image is not switched among the four edges included in the rectangle surrounding the adjacent maximum area MX.

  As shown in FIG. 6B, among the four sides of the search area SRC shown in FIG. 6A, the upper side is in contact with the radio button 1 (EM1) and the radio button 2 (EM2) specified as the trigger TR. In addition, the left side of the search area SRC is in contact with the end of the evaluation target screen WND. Therefore, the search area determination unit 13 determines that the upper side and the left side shown in FIG. 6B are not non-terminal sides NOE.

  On the other hand, the lower side EG1 shown in FIG. 6B is in contact with the GUI element EM11, and the right side EG2 is in contact with the GUI element EM5. Therefore, the search area determination unit 13 specifies that the lower side EG1 and the right side EG2 are non-terminal sides NOE.

  Thereafter, the search area determination unit 13 outputs information regarding the search area SRC obtained in this way to the boundary expression determination unit 14.

  Here, the information related to the search area SRC includes, for example, the coordinate range of the search area SRC, the coordinate range of the switching area SW, the coordinate range of the adjacent maximum area MX, the coordinates of each end point of the non-terminal side NOE, and the output of the operation instruction It is an image of at least one of the front and rear evaluation target screens WND (for example, an image of window A after the output of an operation instruction for the radio button 2 shown in FIG. 4).

  Next, the boundary expression determination unit 14 determines a boundary expression BDR from the search area SRC determined by the search area determination unit 13 and evaluates whether the boundary expression BDR is appropriate. Explained.

  In the following description example, as a rule for a ruled line that is one of the boundary representations BDR, “a width of the switching region SW located in parallel exists, and a width of 10 px ( Pixel) The following ruled line exists ”. The method for determining the ruled line from the image may be an existing method that is generally known and used in an OCR (Optical Character Reader) product or the like.

  In this example, the rule for the background color difference is defined as “there is a boundary of a rectangular area having two colors whose background color difference is 125 or more”.

  In this example, the rule for the margin is defined as “the side of the search area SRC corresponding to the non-terminal side NOE is 100 px (pixels) or more apart”.

  Hereinafter, the boundary expression determination unit 14 evaluates whether or not the appropriate boundary expression BDR is included in the search area SRC determined by the search area determination unit 13 shown in FIG. The case will be described. The boundary expression determination unit 14 evaluates whether the boundary expression is included in the search area for each type of boundary expression BDR.

  First, the boundary expression determination unit 14 determines the boundary expression BDR included in the search area SRC for each type of boundary expression BDR (ruled line, background color difference or margin). When the boundary expression determination unit 14 determines the boundary expression BDR, the boundary expression determination unit 14 holds the coordinates and the type of the determined boundary expression BDR.

  In the example of the evaluation target screen WND illustrated in FIG. 4, the boundary expression determination unit 14 determines the ruled lines L1, L2, and L3 illustrated in FIG. 7 as the boundary expression BDR, and the ruled lines L1, L2 and Each coordinate and type of L3 are held.

  Subsequently, the boundary expression determination unit 14 determines whether or not a corresponding boundary expression BDR exists for each non-terminal side NOE. Specifically, the boundary expression determination unit 14 determines, for each of the non-terminal sides NOE (the lower side EG1 and the right side EG2 illustrated in FIG. 6B), of the two sides of the switching area SW that is parallel to the non-terminal side NOE. In the rectangular region having one side of the switching region SW closest to the non-terminal side NOE and a part of the non-terminal side NOE as a side opposite to the one side, the non-terminal side It is determined whether or not there is a boundary expression BDR in a positional relationship parallel to the side NOE and one side of the switching area SW.

  When performing the determination in the example illustrated in FIG. 6B, the boundary expression determination unit 14 indicates that the ruled lines L1 to L3 determined as the boundary expression BDR are illustrated in FIG. 7 between the non-terminal side NOE and the switching area SW. It is determined whether or not each region exists in each of the regions R1 and R2.

  In the example illustrated in FIG. 7, the boundary expression determination unit 14 determines that the boundary expressions BDR exist in both of the regions R1 and R2 by the determination. From this result, the boundary expression determination unit 14 appropriately expresses the boundary expression BDR for the image switching by the radio button 1 (EM1) and the radio button 2 (EM2) as the trigger TR (that is, there is no defect). A determination result indicating that is output to the output unit 12.

  As a result of determining whether or not there is a boundary expression BDR in a positional relationship parallel to one side of the non-terminal edge NOE and the switching area SW, the non-terminal edge NOE specified by the search area determination unit 13 is included in the non-terminal edge NOE. If there is even one non-terminal edge NOE for which no corresponding boundary expression BDR exists, the boundary expression determination unit 14 determines that “there is a defect”. Then, the boundary expression determination unit 14 outputs an image in which the position of the trigger TR and the position of the switching area SW are marked together with the determination result to the output unit 16. In addition, when performing the said marking, what is necessary is just to use the coordinate range RNG of the said trigger when the trigger specific | specification part 12 specifies trigger TR about the position of trigger TR. As for the position of the switching area SW, the coordinate range of the switching area SW when the search area determining unit 13 determines the switching area SW may be used.

  When the evaluation device 1 determines whether or not the appropriate boundary expression BDR exists for the radio buttons 1 and 2, the trigger specifying unit 12 triggers the GUI elements EM (the drop-down list EM6 and the tab EM8) that have not been evaluated yet. The evaluation of whether or not it is TR, the determination of the switching area SW and the search area SRC, and the determination of the boundary expression BDR by the boundary expression determination unit 14 are continued.

  Hereinafter, an operation when the evaluation apparatus 1 evaluates the drop-down list EM6 will be described.

  In this example, when an instruction to select the radio button 2 (EM2) shown in FIG. 2 is output, an evaluation target screen WND shown in FIG. 8 is displayed after the output. In the evaluation target screen WND shown in FIG. 8, a GUI element EM12 is displayed instead of the GUI element EM7 shown in FIG.

  In this case, the search area determination unit 13 determines an area indicated by a broken line in FIG. 8 as the switching area SW. Subsequently, the search region determination unit 13 determines a region R3 between the broken line and the solid line shown in FIG. 8 as the search region SRC. Then, the search area determination unit 13 determines the lower side EG3 of the search area SRC as a non-terminal side NOE. Then, the search area determination unit 13 outputs information on the determined switching area SW, search area SRC, and non-terminal side NOE to the boundary expression determination unit 14.

  Then, the boundary expression determination unit 14 between the non-terminal side NOE (lower side EG3) and the side of the switching area SW that is positioned closest to the non-terminal side NOE while being parallel to the non-terminal side NOE. It is determined that the boundary expression BDR does not exist in the region R3. Then, the boundary expression determination unit 14 outputs an evaluation result RST indicating that there is no appropriate boundary expression BDR (that is, there is a defect) to the output unit 16 for the image switched by the drop-down list (EM6) that is the trigger TR. Output.

  When the determination of the presence or absence of an appropriate boundary expression BDR for the drop-down list (EM6) is completed, the evaluation apparatus 1 continues to evaluate the presence or absence of the boundary expression BDR for the GUI element EM8 (tab) that has not been evaluated yet. Execute.

  Hereinafter, an operation when the evaluation apparatus 1 evaluates the presence or absence of the boundary expression BDR for the tab (EM8) will be described.

  In this example, when an instruction to select the tab 2 (EM8) shown in FIG. 2 is output, the evaluation target screen WND shown in FIG. 9 is displayed after the output. In the evaluation target screen WND shown in FIG. 9, a GUI element EM13 is displayed instead of the GUI element EM9 shown in FIG.

  In this case, the search area determination unit 13 determines an area indicated by a broken line in FIG. 9 as the switching area SW. Subsequently, the search region determination unit 13 determines a region R4 between the broken line and the solid line shown in FIG. 9 as the search region SRC. Then, the search area determination unit 13 determines the lower side EG4 of the search area SRC as a non-terminal side NOE. Then, the search area determination unit 13 outputs information on the determined switching area SW, search area SRC, and non-terminal side NOE to the boundary expression determination unit 14.

  Then, the boundary expression determination unit 14 selects one side of the switching region SW that is closest to the non-terminal side NOE among the two sides of the switching region SW parallel to the non-terminal side NOE (lower side EG4). Then, it is determined that the boundary expression BDR does not exist in the rectangular region R4 having a part of the non-terminal side NOE (lower side EG4) as the opposite side of the nearest side. Then, the boundary expression determination unit 14 outputs to the output unit 16 an evaluation result RST indicating that there is no appropriate boundary expression BDR (that is, there is a defect) with respect to image switching by the tab (EM8) that is the trigger TR. . Then, the output unit 16 outputs an image in which the position of the trigger TR and the position of the switching region SW are marked to the output unit 16 together with the evaluation result RST.

  The evaluation results RST for the radio buttons (EM1 and EM2), the drop-down list (EM6), and the tab (EM8) described above are “defects” for the radio buttons 1 and 2 (EM1 and EM2) as illustrated in FIG. “None”.

  On the other hand, for the drop-down list (EM6) and the tab (EM8), the evaluation result RST indicates that “there is a defect”. Further, regarding the drop-down list (EM6) and the tab (EM8), when the boundary expression BDR corresponding to the non-terminal side NOE does not exist in the detailed description of the evaluation result RST, the non-terminal side NOE and the switching area SW The position where there was no appropriate boundary representation BDR between the two is shown.

  Note that when the evaluation of the presence or absence of the boundary expression BDR is completed for all the GUI elements EM, the evaluation device 1 ends the process for evaluating usability.

  Next, the operation when the evaluation apparatus 1 evaluates the presence or absence of the boundary expression BDR described above will be described with reference to the flowchart shown in FIG.

  As shown in FIG. 11, first, the trigger identification unit 12 determines the GUI element EM located at the upper left in the screen among the GUI elements EM of the unevaluated trigger candidates as the evaluation target element CND ( Step S1).

  Subsequently, the trigger specifying unit 12 determines whether or not grouping with respect to the GUI element EM determined as the evaluation target element CND is possible. In this illustrative example, the trigger specifying unit 12 determines whether or not the same type of GUI element as the determined evaluation target element CND is linearly arranged in the evaluation target screen WND. If it is determined that the evaluation target element CND can be grouped by the determination, grouping is performed on the evaluation target element CND, and the generated group is determined as one evaluation target element CND (step S2).

  Thereafter, the trigger specifying unit 12 compares the image of the evaluation target screen WND before the output of the operation instruction for the evaluation target element CND with the image of the evaluation target screen WND after the output. If it is determined by the comparison that some of the displayed image areas have been switched, the evaluation target element CND is specified as the trigger TR (step S3).

  If the trigger TR is not specified in step S3 (step S4; No), the evaluation of whether or not the evaluation target element CND is the trigger TR is terminated, and the process of step S1 is executed again.

  On the other hand, when the trigger TR is specified in step S3 (step S4; Yes), the search area determination unit 13 executes the process of step S5. In step S5, the search area determination unit 13 compares the image of the evaluation target screen WND before and after the output of the operation instruction with respect to the trigger TR. A rectangular area that surrounds and has the smallest area is determined as the switching area SW.

  Subsequently, the search area determination unit 13 surrounds the determined switching area SW, does not include another GUI element EM outside the switching area SW, and has a rectangular adjacent maximum area MX having a maximum area. Identify. Subsequently, the search area determination unit 13 determines the remaining area excluding the switching area SW from the specified adjacent maximum area MX as the search area SRC (step S6).

  Further, the search area determination unit 13 determines, as a non-terminal edge NOE, a side that is in contact with the GUI element EM located outside the switching area SW among the four sides of the rectangle surrounding the search area SRC (step S7). ).

  The boundary expression determination unit 14 determines the boundary expression BDR from the search area SRC based on the rule defined for each type of boundary expression BDR stored in the memory (step S8).

  When the boundary representation BDR that separates the switching region and the non-terminal side does not exist in the search region SRC, the boundary representation determination unit 14 determines that “there is a defect”. Then, the boundary expression determination unit 14 outputs the determination result and an image in which the position of the trigger TR and the position of the switching region SW are marked to the output unit 16 (step S9).

  Furthermore, when the unevaluated GUI element EM exists in the evaluation target screen WND (step S10; Yes), the trigger specifying unit 12 executes the process of step S1. On the other hand, when there is no unevaluated GUI element in the evaluation target screen WND (step S10; No), the evaluation apparatus 1 ends a series of processes for evaluating the evaluation target screen WND.

  Next, the process of step S3 shown in FIG. 11, that is, the process of evaluating whether the evaluation target element CND is the trigger TR by the trigger specifying unit 12 will be described in detail with reference to FIG.

  The trigger specifying unit 12 acquires a still image from the evaluation target screen WND, and gives a label indicating “before output” to the acquired still image (step S31).

  Subsequently, the trigger specifying unit 12 sequentially outputs an operation instruction for selecting a selection item that has not yet been evaluated from the selection items included in the evaluation target element CND (step S32).

  Furthermore, whenever the trigger specifying unit 12 completes the output of the operation instruction for selecting the selection item included in the evaluation target element CND, the trigger specifying unit 12 acquires the image of the evaluation target screen WND after the output, Then, a label indicating “after output” is assigned (step S33).

  Subsequently, the trigger specifying unit 12 sequentially compares the pre-output image acquired in step S31 and the post-output image acquired in step S33 (step S34).

  As a result of the comparison in step S34, when one or more images in which a difference is seen in a region other than the evaluation target element CND exist (step S35), the trigger specifying unit 12 specifies the evaluation target element CND as a trigger TR ( Step S36).

  As described above, according to the present invention, the trigger specifying unit 12 outputs an operation instruction and specifies the trigger TR for switching the image area on the GUI screen WND before and after the output. Then, the search area determination unit 13 searches the search area SRC for an area between the switching area SW in which the image is switched by the trigger TR specified by the trigger specifying unit 12 and the GUI element in which the image is not switched before and after the output. Determine as. Then, the boundary representation determination unit 14 includes a boundary representation BDR between the switching region SW in which a part of the image region is switched and the GUI element EM in which the image is not switched before and after the output in the search region SRC. The evaluation result RST indicating whether or not the boundary expression BDR exists is output to the output unit 16.

  Because of such a configuration, a GUI element located within the switching area SW and a GUI element located outside the switching area SW are not visually divided, and a portion having a defect in the boundary representation BDR is evaluated. It can be specified on the screen WND. Therefore, it is possible to evaluate the presence or absence of an appropriate boundary expression BDR for separating the switching area SW included in the GUI screen WND without human intervention.

  In the present invention, the processing in the evaluation device 1 is recorded on a recording medium readable by the evaluation device 1 in addition to the processing realized by the dedicated hardware described above. The program recorded on the recording medium may be read by the evaluation apparatus 1 and executed. The recording medium readable by the evaluation apparatus 1 refers to a transfer medium such as a floppy disk (registered trademark), a magneto-optical disk, a DVD, and a CD, as well as an HDD incorporated in the evaluation apparatus 1. The program recorded on the recording medium is read by, for example, the trigger specifying unit 12, the search region determining unit 13, and the boundary expression determining unit 14 included in the evaluation device 1, and the trigger specifying unit 12, the search region determining unit 13, and the boundary The same processing as described above is performed under the control of the expression determination unit 14.

  Here, the trigger specifying unit 12, the search area determining unit 13, and the boundary expression determining unit 14 included in the evaluation device 1 operate as a computer that executes a program read from a recording medium on which the program is recorded.

  The present invention has been described above with reference to the embodiments, but the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention without departing from the gist of the present invention.

It is a figure which shows the structure of the evaluation apparatus according to embodiment of this invention. It is a figure which shows an example of a screen structure of the evaluation object screen shown in FIG. It is a figure which shows an example of the data structure of GUI element information shown in FIG. It is a figure which shows an example of the screen structure of the evaluation object screen displayed after outputting the operation instruction | indication which selects a radio button in the evaluation object screen shown in FIG. It is a figure which shows the switching area | region before outputting the operation instruction | indication with respect to a radio button, and its peripheral image. (A) It is a figure which shows the switching area | region after outputting the operation instruction | indication with respect to a radio button, and its peripheral image. (B) It is a figure which shows the non-terminal edge | side among the edges which comprise a search area | region. It is a figure which shows an example of the area | region where discrimination | determination is performed about the presence or absence of boundary expression. It is a figure which shows an example of a screen structure of the evaluation object screen displayed after outputting the operation instruction | indication which selects a drop-down list in the evaluation object screen shown in FIG. It is a figure which shows an example of a screen structure of the evaluation object screen displayed after outputting the operation instruction | indication which selects a tab in the evaluation object screen shown in FIG. It is a figure which shows an example of the data structure of the evaluation result shown in FIG. It is a flowchart which shows operation | movement when an evaluation apparatus evaluates the presence or absence of a boundary expression. FIG. 12 is a flowchart illustrating an operation when a trigger specifying unit specifies a trigger in the process illustrated in FIG. 11.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 GUI evaluation apparatus 11 Input part 12 Trigger specific | specification part 13 Search area | region determination part 14 Boundary expression determination part 15 Memory | storage part 151 GUI element information 16 Output part 17 Evaluation object system

Claims (12)

An evaluation apparatus that evaluates a GUI (graphical user interface) in which a part of an image area displayed on a screen is switched before and after the input of an operation instruction is received from a user,
A trigger identification unit that outputs an operation instruction to a GUI element included in the GUI screen when an input of an operation instruction to perform evaluation of the GUI screen is received from the user;
When the operation instruction is output, an area between a rectangular switching area that has the smallest area while enclosing the partial image area and a GUI element that does not switch the image before and after the output is a search area. A search area determination unit to determine as:
A boundary expression determination unit that outputs an evaluation result indicating whether or not a boundary expression indicating a boundary between the switching area and the GUI element to which the image is not switched exists in the search area; apparatus. The evaluation apparatus according to claim 1,
The boundary expression determination unit specifies the boundary expression according to a positional relationship between a GUI element that does not switch the image and the switching area, and determines whether the specified boundary expression exists in the search area. An evaluation apparatus that outputs an evaluation result to be displayed. In the evaluation apparatus according to claim 1 or 2,
The search area determination unit specifies a rectangular adjacent maximum area that surrounds the switching area, does not include a GUI element that does not switch the image in an area other than the switching area, and has a maximum area, An evaluation apparatus, wherein an area other than the switching area in the adjacent maximum area is determined as the search area. The evaluation apparatus according to claim 3,
The search area determination unit
Of the four sides of the rectangle surrounding the adjacent maximum region, the side that is in contact with the GUI element that does not switch the image is specified as a non-terminal side,
The boundary expression determination unit
One of the specified non-terminal side and the two sides of the switching region that are parallel to the non-terminal side is located closest to the non-terminal side, and a part of the non-terminal side is An evaluation apparatus that outputs an evaluation result indicating whether or not a boundary expression parallel to the non-terminal side exists in a rectangular area having the opposite side of the one side. An evaluation method for evaluating a GUI (graphical user interface) in which a part of an image area displayed on a screen is switched before and after the input of an operation instruction is received from a user,
A trigger specifying process for outputting an operation instruction to a GUI element included in the GUI screen when an input of an operation instruction for executing evaluation of the GUI screen is received from the user;
When the operation instruction is output, an area between a rectangular switching area that has the smallest area while enclosing the partial image area and a GUI element that does not switch the image before and after the output is a search area. Search area determination processing to be determined as
And a boundary expression determination process for outputting an evaluation result indicating whether or not a boundary expression indicating a boundary between the switching area and the GUI element to which the image is not switched exists in the search area. Method. The evaluation method according to claim 5,
In the boundary expression determination process, the boundary expression is specified according to the positional relationship between the GUI element that does not switch the image and the switching area, and whether or not the specified boundary expression exists in the search area is determined. An evaluation method characterized by outputting an evaluation result to be shown. In the evaluation method according to claim 5 or 6,
In the search area determination process, a rectangular adjacent maximum area that does not include a GUI element that surrounds the switching area and does not switch the image in an area other than the switching area and has a maximum area, An evaluation method, wherein an area other than the switching area among the adjacent maximum areas is determined as the search area. The evaluation method according to claim 7,
In the search area determination process,
Of the four sides of the rectangle surrounding the adjacent maximum region, the side that is in contact with the GUI element that does not switch the image is specified as a non-terminal side,
In the boundary expression determination process,
One of the specified non-terminal side and the two sides of the switching region that are parallel to the non-terminal side is located closest to the non-terminal side, and a part of the non-terminal side is An evaluation method, comprising: outputting an evaluation result indicating whether or not a boundary expression parallel to the non-terminal side exists in a rectangular region having the opposite side of the one side. An evaluation apparatus that evaluates a GUI (graphical user interface) in which a part of an image area displayed on the screen is switched before and after the input of an operation instruction is received from a user.
A trigger specifying procedure for outputting an operation instruction to a GUI element included in the GUI screen when an input of an operation instruction for executing evaluation of the GUI screen is received from the user;
When the operation instruction is output, an area between a rectangular switching area that has the smallest area while enclosing the partial image area and a GUI element that does not switch the image before and after the output is a search area. A search area determination procedure to be determined as
A program for executing a boundary expression determination procedure for outputting an evaluation result indicating whether a boundary expression indicating a boundary between the switching area and a GUI element to which the image is not switched exists in the search area. The program according to claim 9,
In the boundary expression determination procedure, the boundary expression is specified according to the positional relationship between the GUI element that does not switch the image and the switching area, and whether or not the specified boundary expression exists in the search area is determined. A program characterized by outputting an evaluation result to be shown. The program according to claim 9 or 10,
In the search area determination procedure, a rectangular adjacent maximum area that does not include a GUI element that surrounds the switching area but does not switch the image in an area other than the switching area and has a maximum area, A program for determining an area other than the switching area among the adjacent maximum areas as the search area. The program according to claim 11,
In the search area determination procedure,
Of the four sides of the rectangle surrounding the adjacent maximum region, the side that is in contact with the GUI element that does not switch the image is specified as a non-terminal side,
In the boundary expression determination procedure,
One of the specified non-terminal side and the two sides of the switching region that are parallel to the non-terminal side is located closest to the non-terminal side, and a part of the non-terminal side is A program that outputs an evaluation result indicating whether or not a boundary expression parallel to the non-terminal side exists in the rectangular area having the opposite side of the one side.

Images