JP2004013242A - Usability evaluation support method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology for acquiring a user's subjective evaluation result together with log data such as screen transition operated by the user so as to evaluate usability on an Internet web site. <P>SOLUTION: A question screen is previously given corresponding to each user interface screen of a system to be evaluated, having a plurality of user interface screens subject to transition. When one of the user interface screens of the system to be evaluated is displayed, the corresponding question screen is simultaneously displayed, and data input by the user on the question screen is stored. The user data input on the question screen is transmitted to an evaluation system such that the system to be evaluated is not changed for evaluation. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technology for evaluating usability of a system such as an Internet Web site, and more particularly to a method for collecting data necessary for evaluation.
[0002]
[Prior art]
As a method of evaluating usability, for example, there is a usability evaluation device described in JP-A-2001-51876. This contributes to the evaluation of the usability of the system by recording the state of the system and reproducing the history. Further, there is a user interface evaluation support device and a user interface evaluation support method described in Japanese Patent Application Laid-Open No. 8-161197. In this method, a process of operation on the user interface displayed on the screen is stored, and a problematic operation in the user interface is determined based on the stored process of operation, or each button on the user interface is determined. The degree of relevance between operations is obtained, and the result is displayed on a screen.
[0003]
[Problems to be solved by the invention]
Both the method described in JP-A-2001-51876 and the method described in JP-A-8-161197 merely collect and analyze logs such as the history of screen transitions arbitrarily operated by the user as evaluation data. In the usability evaluation, it is important to determine whether or not the user is easy to use. However, conventionally, there has been no method for acquiring the user's subjective evaluation result for the evaluation target system.
[0004]
Further, in the above-described conventional technology, it is necessary to improve a Web site to be evaluated in order to obtain evaluation data other than a log such as a subjective evaluation result of a user.
[0005]
Further, there is no consideration regarding collecting evaluable data that is elaborate according to the characteristics of the user.
[0006]
An object of the present invention is to provide a technique for acquiring a user's subjective evaluation result together with log data such as screen transitions operated by the user.
[0007]
Another object of the present invention is to provide a technique for collecting evaluation data without making any improvement to a system such as a Web site to be evaluated.
[0008]
Another object of the present invention is to provide an evaluation data collection technique capable of providing various evaluation methods to a Web page evaluation requester.
[0009]
[Means for Solving the Problems]
In order to achieve the above object, the evaluation system of the present invention obtains not only the transition log of the page but also the impressions and opinions of the user at the time of the transition in association with the transition page, thereby obtaining an evaluation system. In addition, it is possible to collect data effective for extracting a problem of the system to be evaluated.
[0010]
More specifically, a question screen is associated in advance for each user interface screen of the evaluation target system with transitions between a plurality of user interface screens, and when the user interface screen of the evaluation target system is displayed , The corresponding question screen is displayed at the same time, and the user input entered on the question screen is stored.
[0011]
Further, in order to achieve the above other object, user data other than the log input from the user terminal is sent from the user terminal to the evaluation system, and the data is transmitted to the evaluation system separate from the system to be evaluated. By enabling collection, log data and other evaluation data can be obtained without modifying the Web site to be evaluated.
[0012]
In order to achieve another object, it is possible to perform evaluation from various viewpoints at the time of evaluation execution by collecting user characteristics and evaluation data such as logs in association with each other.
[0013]
BEST MODE FOR CARRYING OUT THE INVENTION
(Example 1)
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 3 is a network configuration diagram. It has a terminal (10) used by an evaluator (hereinafter referred to as a user), a Web server (20) to be evaluated, and a proxy server (30) for storing data for evaluation experiments. FIG. 2 is a hardware block configuration diagram common to the computer systems of the terminal 10, the Web server 20, and the proxy server 30. It has data input means (16) such as a keyboard, a mouse, and a barcode reader, a CPU (13), an external device (17), a main storage means (14), a display (11), and a network connection device (15).
[0014]
The method of the present invention may be implemented on any computer of the terminal (10), the Web server (20), and the proxy server (30) used by the user, or may be implemented on another computer connected to this network. Good. In any case, the evaluation support program including the processing procedure described below with reference to FIG. 1 and related data such as a database (DB) used in the program are stored in the external storage device 17. When the evaluation is performed, the evaluation support processing is executed by sequentially reading the data from the external storage device 17 to the main storage and interpreting and executing the CPU. The evaluation support program is often stored in a storage medium readable by a computer system such as a CD-ROM (not shown) and traded. Alternatively, the information may be distributed from a server or the like via a network. In either case, the data is taken into the computer system, temporarily stored in the external storage device 17, and executed according to the above-described procedure.
[0015]
An embodiment according to the present invention (hereinafter referred to as a usability evaluation method) is performed by the method shown in FIG. First, user information is registered in step 100 of FIG. FIG. 4 shows an example of the user information registration screen. The user information registration screen allows the user to input information about the user. In the example of FIG. 4, the user is asked to enter the name, user ID, gender, age, and the like. The data input on the user information registration screen is written to the user information DB (31) in FIG. FIG. 5 shows an example of a table configuration of the user information DB. In the example of FIG. 5, the user ID (311), name (312), gender (313), age (314), Web usage (315), Web purchase (316), Web input entered on the user information registration screen The column of the usage method (317) is held.
[0016]
In a usability evaluation method, data for evaluating usability is collected while allowing a user to perform a certain task. A task that defines a task to be performed by a user is called a task, and a set of tasks ordered and defined is defined as a test.
[0017]
Step 101 of FIG. 1 sets a test. Step 101 sets which test used in the evaluation experiment is to be performed. The test content is set by reading a test ID from the test DB (32) in FIG. FIG. 6 shows an example of the table configuration of the test DB. In the example of FIG. 6, the test ID (321), the order of the tasks to be performed in the test (322), and the task ID (323) are held. The test content is selected by the experimenter. Alternatively, a method for automatically determining a test based on user information may be provided, or a method for randomly selecting a test may be provided.
[0018]
Step 111 in FIG. 1 sets a task to be performed in the test and displays an initial screen of the task (hereinafter, referred to as a task initial screen). In the usability evaluation method, a variable called “task order” holds the order of tasks performed at that time. In step 111 of FIG. 1, a task ID (323) corresponding to the test ID (321) and the order (322) that matches the "task order" in the test DB of FIG. . Next, a task initial screen is displayed. FIG. 7 shows an example of the task initial screen. The task initial screen displays tasks to be performed by the user. In the example of FIG. 7, the problem of purchasing a ticket for a business trip is displayed. The task initial screen is read from the task DB (33) in FIG. FIG. 8 shows an example of a table configuration of the task DB. In the example of FIG. 8, columns of a task ID (331), a task name (332), a task initial screen URL (333), and a task description screen URL (334) are held. In step 111 of FIG. 1, using this information, a task initial screen URL is acquired from the task ID set in step 101 of FIG. 1 and displayed.
[0019]
Step 102 in FIG. 1 writes the user operation acquired on the proxy in the user operation log DB (34). The user operation log DB holds the transition of the screen to be evaluated in the evaluator terminal (10) of FIG. 3 as a user operation record. FIG. 9 shows an example of a table configuration of the user operation log DB. In the example of FIG. 9, the log ID (341), the task ID (342), the user ID (343), the URL of the evaluation target screen (344), the transition time (345) as the time of transition to the evaluation target screen, and the next transition The column of the next transition time (346) as the time when the event occurred and the transition destination screen URL (347) from the evaluation target screen are held. In the example of FIG. 7, when the link 1021 is clicked, the transition time to the task initial screen URL is the transition time (345) in FIG. 9, and the task initial screen URL is the evaluation target screen URL (344) in FIG. The transition destination of the link 1021 of No. 7 is the transition destination screen URL (347) in FIG. 9, and the time when the transition to the transition destination screen is the next transition time (346) in FIG.
[0020]
Step 103 in FIG. 1 displays a screen (task explanation screen) for explaining a task to be performed by the user. The task explanation screen is a screen for displaying the task so that the user does not forget the task when performing the task. Also, when the user gives up the work or determines that the work has been completed, the user clicks the “GiveUp” button or the “Work end” button on the task explanation screen. FIG. 10 shows a detailed processing flow for step 103 in FIG. Step 1030 in FIG. 10 acquires the task description screen URL (334) corresponding to the task ID (331) in the task DB in FIG. A step 1031 displays the obtained task explanation screen URL in a window different from the evaluation target screen. FIG. 11 shows an example of the screen displayed in step 1031. In the example of FIG. 11, a travel navigation search screen (1033), which is an evaluation target screen, is displayed on the right side of the screen, and a task explanation screen (1032) is displayed on the left side.
[0021]
Step 104 in FIG. 1 writes the user's operation to the user operation log DB (34) as in step 102. In the example of FIG. 11, when a transition has been made from the “Travel Navigation Search Screen” on the screen 1033 to the “Station Name Search Screen” on the screen 1034, the URLs of the “Travel Navigation Search Screen” are the evaluation target URLs (344), The transition time to the “travel navigation search screen” is the transition time (345) in FIG. 9, and the URL of the “station name search screen” is the transition destination screen URL (347) in FIG. 9 and the time when the transition to the “station name search screen” is FIG. Is the next transition time (346).
[0022]
In step 105 of FIG. 1, a URL of a screen (instruction frame) when the user is asked a question and inputs an answer is acquired according to the scenario. A scenario defines a question content and an answer method for each evaluation target screen for performing a task operation. The contents of the scenario are described in the scenario DB (35) of FIG. The scenario DB (35) holds the URL of the instruction frame and the conditions for determining the conditions under which the instruction frame is issued. FIG. 12 shows an example of a table configuration of the scenario DB. In the example of FIG. 12, columns of a scenario ID (351), a task ID (352), an evaluation target screen URL (353), a Web usage (354), and an instruction frame URL (355) are held. In the example of FIG. 12, the Web usage (354) is a condition. FIG. 13 shows a detailed processing flow of step 105 in FIG. A step 1050 obtains the URL of the evaluation target screen from the user operation log DB (34) of FIG. In the user operation log DB of FIG. 9, the transition time corresponding to the user's user ID (343) acquires the latest evaluation target screen URL (344). Step 1051 in FIG. 13 acquires the corresponding user information from the user information DB (31) in FIG. In the example of FIG. 12, first, the Web usage (315) corresponding to the user ID (311) of the user is acquired from the user information DB of FIG. In step 1052 of FIG. 13, the task ID, the evaluation target screen URL extracted in step 1050, and the URL of the instruction frame corresponding to the user information extracted in step 1051 are acquired from the scenario DB (35) in FIG. In the example of FIG. 12, an instruction frame URL (355) corresponding to the evaluation target screen URL (353) and the Web usage (354) is acquired.
[0023]
Step 106 of FIG. 1 displays an answer screen. FIG. 14 shows a detailed processing flow. In step 1060 in FIG. 14, the evaluation target screen URL (353) and the instruction frame URL (355) in FIG. 12, which are the transition sources, are created as a composite screen separated by frames. Step 1061 in FIG. 14 displays the created composite screen as a response screen in a new window on the front of the user terminal. FIG. 15 shows an example of the instruction frame of the answer screen. In the example of FIG. 15, a question regarding the contents before and after the transition and an opinion regarding the transition are input.
[0024]
Step 107 in FIG. 1 writes the data input on the answer screen to the user answer DB (36) in FIG. FIG. 16 shows an example of a table configuration of the user answer DB. In the example of FIG. 16, columns such as an answer log ID (362), a user ID (363), a scenario ID (364), and an answer (365) are held.
The task ends when the user gives up the work or determines that the work is finished, by clicking a “GiveUp” button or a “Work finished” button on the task explanation screen.
[0025]
In step 108 of FIG. 1, a question is asked to the user at the end of the task and a URL of a screen (task end screen) for inputting an answer is acquired from the task end DB. The task end DB holds the URL of the task end screen under the condition at the time of task end. FIG. 17 shows an example of a table configuration of the task end DB. In the example of FIG. 17, columns of a scenario end ID (371), a task ID (372), an end button input (373), and a task end screen URL (374) are held.
[0026]
Step 109 in FIG. 1 writes the data input on the task end screen into the user answer DB (36) in FIG.
[0027]
Step 110 of FIG. 1 outputs a user operation log DB (34) and a user answer DB (36).
[0028]
According to the above-described embodiment, data is collected by associating the user's impressions and opinions (user data) with the page transition log with the page transition log. Therefore, it is possible to provide an evaluation system capable of evaluating the evaluation target system from the viewpoint of the user. .
[0029]
In addition, since user data other than the log input from the user terminal is sent from the user terminal to the evaluation system, data can be collected by an evaluation system separate from the system to be evaluated. The log data and other evaluation data can be obtained without modifying the evaluation target Web site.
[0030]
Further, by collecting the user's characteristics and evaluation data such as logs in association with each other, it is possible to perform evaluation from various viewpoints at the time of executing the evaluation.
[0031]
【The invention's effect】
The evaluation system of the present invention collects data by linking the user's impressions and opinions (user data) to the page transition log with the page transition log, and thus provides an evaluation system capable of evaluating the evaluation target system from the viewpoint of the user. it can.
[0032]
In addition, since user data other than the log input from the user terminal is sent from the user terminal to the evaluation system, data can be collected by an evaluation system separate from the system to be evaluated. Log data and other evaluation data can be acquired without modifying the evaluation target Web site.
[0033]
Further, by collecting the user's characteristics and evaluation data such as logs in association with each other, it is possible to perform evaluation from various viewpoints at the time of executing the evaluation.
[Brief description of the drawings]
FIG. 1 is a flowchart illustrating a processing procedure according to an embodiment of the present invention.
FIG. 2 is a block diagram showing a system configuration.
FIG. 3 is a configuration diagram showing the entire network system.
FIG. 4 is a diagram showing a user information registration screen.
FIG. 5 is a configuration diagram of a user information DB.
FIG. 6 is a configuration diagram of a test DB.
FIG. 7 is a diagram showing a task initial screen.
FIG. 8 is a configuration diagram of a task DB.
FIG. 9 is a flowchart showing a processing procedure at the time of user terminal work.
FIG. 10 is a diagram showing a user terminal work screen.
FIG. 11 is a configuration diagram of a user operation log DB.
FIG. 12 is a flowchart showing a scenario display processing procedure.
FIG. 13 is a flowchart illustrating a scenario display processing procedure.
FIG. 14 is a flowchart showing a procedure when answering a user terminal.
FIG. 15 is a diagram showing a screen when a user terminal answers.
FIG. 16 is a configuration diagram of a user answer DB.
FIG. 17 is a configuration diagram of a task end question DB.
[Explanation of symbols]
11 display, 13 CPU, 14 main storage device, 15 network connection device, 16 data input means, 17 external storage device.

Claims (17)

An evaluation support method for an evaluation target system involving transition between a plurality of user interface screens,
The user terminal, together with a user interface screen of the evaluation target system, a question screen for the user interface screen can be displayed on the user terminal, and user input data input to the question screen is collected. Supporting method of evaluation target system with screen transition. 2. The evaluation support method according to claim 1, further comprising collecting a transition log of the user interface screen. 2. The evaluation support method according to claim 1, wherein the question screen is displayed on the user terminal together with the corresponding user interface screen. 2. The evaluation support method according to claim 1, wherein the user input data input to the question screen is addressed to an evaluation support system different from the evaluation target system. 2. The evaluation support method according to claim 1, further comprising displaying a display screen for inputting characteristics of a user on the user terminal, and collecting the display data together with the user input data as evaluation data. . Definition that defines a question screen corresponding to a set of information to be input by the user in advance and transition information between user interface screens for each user interface screen of the evaluation target system involving transitions between a plurality of user interface screens Information, and when the user interface screen of the evaluation target system is displayed, it is associated with the transition information between the information actually input by the user and the actually generated user interface screen using the definition information. An evaluation support method comprising: displaying a question screen, and storing a user input input to the question screen. 7. The evaluation support method according to claim 1, wherein prior to collection of user input data, a description of a task to be performed by the user and a method of connecting to a first screen of the evaluation target system are provided by the user terminal. An evaluation support method characterized in that the evaluation support method is displayed on a display. 8. The evaluation support according to claim 7, wherein a plurality of tests are defined in advance by defining a combination and an order of a plurality of tasks as one test, and a user is allowed to perform a predetermined test among the plurality of tests. Method. 9. The evaluation support method according to claim 8, wherein a test is determined based on information input by a user as a method of selecting one test from a plurality of tests. 9. The evaluation support method according to claim 8, wherein a test selected at random is performed as a method of selecting one test from a plurality of tests. 7. The screen according to claim 1, wherein some of all screens that can be transitioned from the evaluation target system are defined as evaluation target screens, and when a transition to a screen other than the evaluation target screen occurs, an instruction prepared in advance is provided. An evaluation support method characterized by displaying a screen. 7. The evaluation support method according to claim 1, wherein a question screen is displayed in the foreground of the user interface screen. 8. The evaluation support method according to claim 7, wherein a screen explaining the problem is displayed beside a user interface screen of the evaluation target system. 8. The evaluation support method according to claim 7, wherein the user has a plurality of methods for declaring the end of the work performed based on the explanation information of the problem. 7. The method according to claim 1, wherein a question part and a user interface part of the evaluation target system are displayed in one screen as a question screen, and a transition source user interface screen of the user interface screen of the evaluation target system is displayed. An evaluation support method characterized by displaying. 15. The evaluation support method according to claim 14, further comprising an input when the user completes the task and an input when the user gives up completing the task. An evaluation support method for an evaluation target system involving transition between a plurality of user interface screens,
While collecting the transition log of the screen according to the instruction from the user terminal, storing the impressions and opinions of the user with respect to the screen displayed on the user terminal with respect to the screen input from the user terminal in association with the screen ID. Evaluation support method characterized by performing.

Images