JP4479343B2 - Programs and devices for system usability study support - Google Patents

Description

  The present invention relates to a technique for investigating the usefulness of a system, and more particularly to automatic creation of a questionnaire questionnaire for investigation.

  The system used by the user (user) can be used by the user in any system, ranging from a single device / software to a large system composed of many devices and software. As a result, whether it is easy for users to use is an important issue. Even if individual functions are easy to use, if the functions required by users are not sufficient, the ease of use of the system as a whole will be reduced, and if there are functions that users do not use, it will be useless. In addition, there is a problem that resources for functions necessary for the resources used by the functions are compressed. As described above, the usefulness of the entire system can be evaluated from the viewpoint of usability of the system and excess / deficiency of the functions of the system.

  Conventionally, in order to evaluate the usefulness of a system, user modeling, analysis of logs such as user operations and system responses to them, and questionnaire surveys for users, etc. must be performed with a great deal of man-hours. I had to.

  As a technique for supporting log analysis as part of such a system usability study, a technique disclosed in Patent Document 1 is known. In this conventional apparatus, the input to the interactive system is monitored by the input monitor unit and the input history is recorded, and the output of the interactive system related to this input is monitored by the output monitor unit and recorded in the output history. In this apparatus, a desired input history can be instructed and displayed on the screen by an instruction from the history reference instruction unit, and an output history associated with the displayed input history can be displayed.

  Further, although not related to evaluation of system usability, Patent Document 2 discloses an apparatus using input / output monitoring of an interactive system as disclosed in Patent Document 2. This device aims to be an active help device that presents help information at a precise timing to the user. In this device, the dialog monitor unit monitors and records the input / output signals between the user and the system and updates the dialog history, and the screen stop measurement / determination unit updates the dialog history every time the dialog history is updated. Based on this, the continuous display time for each display screen is calculated and compared with the standard display time for each display screen to evaluate whether there is an abnormal delay or stalemate in the interaction between the user and the system. To do. When an abnormal delay or the like occurs, help information is selected from the help information storage unit, and the help information is presented to the user via the help screen output unit.

JP 2001-52876 A JP 2000-227826 A

  As described above, there are techniques for automatically collecting system input and output logs, and displaying corresponding input and output in association with each other so that a user can easily analyze the log.

  However, log analysis alone is often not sufficient for evaluating the usefulness of the system, and a questionnaire survey is also used. However, since questionnaire surveys for evaluating system usability have been conducted independently of other information collection / analysis, such as log analysis, it has been difficult to identify useless question items or question items without appropriate options. There was a tendency for questionnaires to be included. If a lot of useless items are included, the survey respondents' willingness to work will be killed, and if there are items that do not have an appropriate option, the respondent will respond unreasonably. Can also reduce the reliability of questionnaire results. If the investigator narrows down the questionnaire questions and adjusts the answer options by looking at information such as log analysis, such problems will be solved considerably, but the investigator's burden is heavy. There's a problem.

According to the present invention, the computer system detects the operation performed by the user for the function provided by the investigation target system, identifies the user who performed the operation , and information on the time when the operation was performed. the at least, logging means for logging data about the operation, the analyzes recorded log data by logging means, a log analyzing means for calculating a usage index indicating a use status of the function, When the usage status index of the function satisfies a predetermined condition, information for identifying a user who has performed an operation related to the log data and the operation included in each log data recorded in the log recording unit Based on the information on the time at which the user was performed, from the time at which each operation was performed in a series of operations for use of the function performed by the same user Calculate the operation time required for the series of operations for using the function, calculate the variance of the operation time required for the series of operations for using the function by each calculated user, and calculate the variance Log analysis means for identifying a user whose operation time required for the series of operations for using the function is larger than an operation time threshold when the function is larger than a dispersion threshold , usefulness of the investigation target system Questionnaire generating means for generating questionnaire questionnaire data relating to the questionnaire, the questionnaire generating means for selecting the question items to be incorporated into the questionnaire questionnaire data based on the usage status index of the function obtained by the log analyzing means, the log When the variance of the operation time required for the series of operations for use of the function by each user calculated by the analysis means is larger than the variance threshold, the questionnaire is generated The survey questionnaire data stage is generated, the program to function as a questionnaire providing means, for the operation time required for a series of operations are provided to the operation time threshold is greater than the user for the use of the function I will provide a.

In a preferred aspect of the present invention, the computer system is a database in which question condition information indicating information of question items corresponding to the conditions is registered for each condition regarding the usage status index of the function provided by the survey target system. , to further function as, the questionnaire generating means, if usage index satisfies the condition of the features that have been determined by the log analysis unit in the database, the question items corresponding to the conditions in the questionnaire questionnaire data Select as a question item to include.

In a further preferred embodiment, the question condition information, subject information for specifying a user of interest question item corresponding to the conditions included in each of the conditions, the questionnaire generating means, said if conditions satisfying the usage index of the features that have been determined by the log analysis unit in the database, based on the subject information corresponding to the condition, and specifying the user of interest question item corresponding to the conditions Then, questionnaire questionnaire data including the question item is generated as questionnaire questionnaire data for the identified user.

  In a further preferred aspect, the question condition information is registered in the database for each function provided by the survey target system, and the log analysis unit displays a usage status index for each function provided by the survey target system. The questionnaire generating means collects, for each user, a question item for the user based on the usage status index of each function obtained by the log analyzing means and the question condition information for each function. Generate questionnaire questionnaire data.

  In another preferred aspect of the present invention, the usage status index of the function includes the number of users and the usage frequency of the function.

In another preferred embodiment of the present invention is also the questionnaire providing means, when said log analysis unit has calculated variance is less than said variance threshold, questionnaire that the questionnaire generating means has generated The vote data is provided to all users who use the function obtained from the log data recorded in the log recording means .

  In another preferred aspect of the present invention, the log analysis unit further includes the log analysis unit for each group of users further distinguished by user profile information when the usage status index of the function satisfies a predetermined condition. The usage status of the function of the group is obtained from the log, and the questionnaire generation means limits the users who are the target of the question item according to the degree of the difference in usage status for each group.

  The best mode for carrying out the present invention (hereinafter referred to as “embodiment”) will be described below with reference to the drawings.

  FIG. 1 is a diagram for explaining a schematic configuration of a system usability investigation support apparatus according to the present invention.

  In FIG. 1, a survey support device 10 is a device that provides support when investigating the usefulness of the survey target system 20.

  Here, the survey target system 20 is a system that provides a predetermined service to the user, and provides one or more functions that can be used by the user. In the illustrated example, the survey target system 20 is a service type system that provides a service to each user who operates a user PC (personal computer) 40 via a network 30 such as the Internet. For example, the core business system is an example of such a system such as groupware. However, this is an example, and the form of the survey target system 20 to which the survey support apparatus 10 can be applied is not particularly limited. The survey target system 20 may be a hardware system, a software system, a combination of both, or a scale.

  In the investigation support apparatus 10, the log recording unit 102 monitors the operation from the user to the investigation target system 20 and the response from the system 20 to the operation, and when detecting an operation instruction or a response, creates log data indicating the operation instruction or response. And stored in the operation / response log storage unit 104. The contents of the operation and response can be detected by monitoring the input and output signals or data for the investigation target system 20.

  The log data of each operation / response includes the date and time when the operation / response was performed, the identification information of the user who performed the operation (and therefore the response target), and the identification information indicating the content of the operation / response Is included. For example, “1/1 10:00 19415 Press B, Page B Open” is one operation / response log data, which is at 10:00 on January 1 ("1/1 10:00"), The user with the user ID 19415 ("19415") accesses the function B of the survey target system 20 ("Press B"), and the survey target system 20 provides the user with the function B screen (that is, ("Page B Open"). Of course, this is an example, and other data items may be included. In the above example, the operation and the response to the operation are described in one log data. However, these may be described separately.

  Such monitoring of the operation / response of the log recording unit 102 and the operation of recording the log can be realized by the same mechanism as that of the conventional apparatus disclosed in Patent Document 1.

  The log analysis unit 106 analyzes the log data group stored in the operation log storage unit 104. In the log analysis, a usage status index indicating the usage status of each function provided by the survey target system 20 is obtained. The usage status index includes, for example, the total number of times the user has used the function (usage frequency), the total number of users who have used the function (number of users), and the like. The usage status index obtained by the analysis is used to create a questionnaire questionnaire. This is just an example, and the detailed contents of log analysis will be described later.

  A user DB (database) 108 is a database in which profile information of each user is registered. The profile information includes, for example, personal feature information such as user ID, name, gender, age, title, and dominant hand.

  The question DB 114 is a database in which data of question items to be incorporated into a questionnaire questionnaire is registered. The data of each question item is registered in association with the condition that the question item should be adopted in the questionnaire. This condition is expressed as a condition regarding the usage status of the users for each function of the survey target system 20.

  FIG. 2 shows an example of data contents registered in the question DB 114. In the illustrated example, one record of the question DB 114 includes “function”, “usage condition”, “question item information”, “accompanying display information”, and “target person information”. The “function” is identification information indicating a function corresponding to the record among one or more functions provided by the survey target system 20. “Usage status condition” is data indicating a condition for a usage status index corresponding to the record. For example, in the record on the first line in the figure, the usage condition is defined in the form of a condition regarding the usage frequency and the number of users. “Question item information” is information of a question item corresponding to the record. The information on the question item includes information on an object for receiving a question sentence and an answer to the question sentence. For example, the answer object is, for example, a check box or radio button for accepting selection of answer options, or an input field for free entry. When creating a questionnaire questionnaire in the form of a web page, the information on the question item is, for example, a description of HTML (HyperText Markup Language) or a method for generating such an HTML description. In addition, the name of the information of such a question item is shown as the information of a question item in the table of illustration. The “accompanying display information” is information that should be displayed on the questionnaire questionnaire accompanying the question item indicated by the “question item information”. For example, manual information indicating an explanation of the function is an example of the accompanying display information. There are several types of accompanying display information, but the manual information indicating the explanation of the function is registered in the manual DB 116. Also, from the log data stored in the operation / response log storage unit 104, information on the usage history of a certain function of a certain individual can be created as accompanying display information. The “target person information” is information for specifying a user who is a target for presenting the question item indicated by the “question item information”. In other words, it is information that defines the conditions to be satisfied by the user to whom the question item is presented.

  Although only the record group for one function is shown in the figure, similar records are registered in the question DB 114 for each function provided by the survey target system 20.

  In other words, in this device 10, “question item information” and “accompanying” to be put on the questionnaire questionnaire by combining the “function” of the system 20 and the “usage condition” that satisfies the usage condition index for the “function”. “Display Information” is selected. When the usage status index for a certain function does not correspond to any “usage status condition” of the function registered in the question DB 114, the question item for the function is not incorporated in the questionnaire questionnaire. . In addition, among the users registered in the user DB 108, a user who satisfies the “target person information” corresponding to the combination is selected as the target of the question item.

  The questionnaire processing unit 110 is a means for executing various processes for questionnaire processing, and includes a questionnaire generation unit 112 for generating a questionnaire questionnaire.

  The questionnaire generation unit 112 obtains a question item corresponding to the usage status index for each function obtained by the analysis of the log analysis unit 106 from the question DB 114, and incorporates the obtained one or more question items. Create data for. At this time, if the accompanying display information associated with the question item to be incorporated into the questionnaire questionnaire is registered in the question DB 114, the accompanying display information is stored in the manual information in the manual DB 116 or the log information stored in the operation / response log storage unit 104. Created from and incorporated into questionnaire questionnaires. The questionnaire questionnaire data can be created, for example, as a web page described in HTML. Here, when creating a questionnaire questionnaire in which question items for a plurality of functions are collected, a customized questionnaire questionnaire in which the question items for the user are collected is created for each user. For this purpose, it is determined from “target person information” which “question item” selected by the combination of “function” and “usage condition” is intended for each, and the question item obtained as a result By organizing combinations of target users and target users for each user, it is possible to specify a question item group to be incorporated into a questionnaire questionnaire addressed to each user.

  The questionnaire processing unit 110 performs a questionnaire survey using the questionnaire questionnaire data created by the questionnaire generation unit 112. When conducting a questionnaire survey using a web page, the questionnaire processing unit 110 registers the web page of the questionnaire questionnaire generated by the questionnaire generation unit 112 in the web server, and the URL (Uniform Resource Locator) of those questionnaire surveys Is transmitted to the target user in the form of e-mail, for example. The user uses the URL to obtain the questionnaire questionnaire page at the user PC 40, inputs an answer to each question item, and transmits the input result to the web server. The questionnaire processing unit 110 summarizes the questionnaire results by counting the input results.

  The survey support apparatus 10 can display the collected questionnaire results and log analysis results on the screen, and the screen display indicates whether the investigator needs the usefulness of each function, that is, whether the use of each function or each function is necessary. It becomes a judgment material when judging.

  It is also preferable to provide the survey support apparatus 10 with a function for proposing measures for functions based on the log analysis results and the results of questionnaire responses. For example, if the log analysis shows that a certain function is not used much, and the questionnaire shows that most users do not want to use that function too much, that function is unnecessary. I can judge. Therefore, the survey support apparatus 10 conducts a questionnaire asking whether or not to use the function for which the usage frequency is determined to be less than a predetermined threshold in the log analysis to all users registered in the user DB 108. As a result, if the number of users who answered that they want to use is less than the predetermined threshold, it is likely that the function is unnecessary, so it is recommended to delete the message, etc. To do. In addition, for such a measure proposal, like the question DB 114, a measure content and conditions for adopting the measure content in the proposal may be stored in a database. The conditions for adopting the contents of the measure can be described as conditions relating to the questionnaire total result, or conditions regarding the questionnaire total result and the log analysis result.

  Next, the processing operation of the survey support apparatus 10 of the present embodiment will be described with a specific example.

  In the investigation support device 10 of the present embodiment, the log recording unit 102 constantly monitors the operation and response to the investigation target system 20 and records the log in the operation / response log storage unit 104. When the investigator instructs the apparatus 10 to start the investigation, the log analysis unit 106 first analyzes the data in the operation / response log storage unit 104, and uses the entire usage as a usage status index for each function. Find the frequency and number of users. The usage frequency of a function may be the sum of the number of log data including the function. As a result, the usage frequency of the function in the entire user is obtained. The number of users of a function may be the sum of the number of different users appearing in log data indicating the function. In this total, even if the same user uses the function a plurality of times, the user counts only once. The combination of the usage frequency and the number of users of each function can be plotted on two-dimensional coordinates with the usage frequency and the number of users as coordinate axes, as in the chart 200 of FIG.

  In a typical example, the questionnaire generation unit 112 creates a questionnaire questionnaire according to the combination of the usage frequency and the number of users for each function. Examples of questionnaires created according to the combination of the usage frequency and the number of users are shown in FIGS.

  FIG. 3 is a diagram showing a questionnaire questionnaire 300 showing question items for the function A (in the figure, the name of this function is “task report”) that has a low number of users and a low usage frequency. This corresponds to the record in the first row of the data contents of the question DB 114 illustrated in FIG. As threshold values for determining whether or not “low”, threshold values t1 and T1 are defined for the number of users and usage frequency, respectively. When the number of users is less than t1 and the use frequency is less than T1, the use condition condition of this record is satisfied, so this record is searched. In FIG. 3, the hatched portion in the chart 200 is the portion that satisfies the condition.

  The investigator may determine the threshold value. The same applies to other threshold values described in the following examples.

  If both the number of users and the frequency of use of the function are low as in this example, it may be because the function itself is not well known to the user, and the user knows the function but needs to It may be because you do not feel. Accordingly, the questionnaire generation unit 112 creates a questionnaire questionnaire 300 incorporating a question item 310 for “recognition / expectation survey” (see FIG. 2) in order to confirm whether or not the function is known. To do. The information of the question item to be incorporated is specified by searching the question DB 114 for a record corresponding to the function and the usage status index for this function.

  In the question item column 310 in the questionnaire 300, a question sentence 312 and an answer column 314 for the question sentence are shown. In the illustrated example, the answer column 314 has a format in which one corresponding item is selected from a plurality of options, and a radio button 316 is provided for each option. When creating the questionnaire as a web page, the information in the answer field 314 may be described according to the description list description method in HTML. Multiple selectable options and free entry fields can be similarly described according to HTML.

  A plurality of question sentences 312 can be incorporated in the question item column 310. In the illustrated example, in the question item column 310, in addition to the question Q1 that asks whether or not the function is known (recognition question), a question Q2 that asks the expectation of the function as an additional question to an unknown person is also included. It is. A set of these multiple questions constitutes one question item for one usage condition that “the number of users is less than t1 and the usage frequency is less than T1”. Note that the plurality of questions constituting one question item may be a combination of a plurality of independent questions as well as a combination of a main question and an accompanying question as illustrated.

  Further, when “accompanying display information” is registered in the record retrieved from the question DB 114, the accompanying display information 320 is incorporated into the questionnaire 300. In the illustrated example, a manual document for explanation of the target “task report” function is read from the manual DB 116 and incorporated in the questionnaire 300.

  As described above, the questionnaire 300 illustrated in FIG. 3 examines the possibility that the function is not well known to users, and thus is defined in the “target person information” of the record in the first line in the example of FIG. As shown, the target person is all users.

  According to the questionnaire using the questionnaire 300 illustrated in FIG. 3, first, it can be determined whether the reason why the function A is not used so much is because the users do not know the function A. If many of the users answered “I know” to the question Q1, it means that there are many people who know the function A but do not use it. In this case, there is a high possibility that the function A is not required in the first place. Therefore, the survey support apparatus 10 proposes a measure of deleting the function A. In addition, when many of the users answered “I don't know” to the question Q1, analysis of the answers to the question Q2 of the expectations of those users becomes a problem. If there are many people who answered “I want to use” to the question Q2, it can be understood that the reason why the function A is not used is simply because the function A was not known. In this case, the apparatus 10 can also propose a measure of introducing convenient usage of the function A to users. Also, if there are many people who answered “I do not want to use” in response to question Q2, there is a high possibility that function A is not needed in the first place. Can be considered.

  Next, FIG. 4 shows an example of a questionnaire questionnaire 400 showing question items for the function B (the function name is “task report” in the figure) that is frequently used but has a small number of users. This corresponds to the record in the second row of the data illustrated in FIG. As shown in this record, when the condition that the usage frequency is higher than the threshold value T2 and the number of users is lower than the threshold value t2 is satisfied, a questionnaire questionnaire 400 as shown in FIG. 4 is created. .

  A function that is frequently used but has a small number of users as in this example can be regarded as a function that is frequently used by a small number of users. About such a function, as one direction, by finding out the purpose of use of frequent users and the reason for frequent use, etc., find the selling point of this function and use it for advertising to non-users It may be useful to determine the direction of improvement of this function. Therefore, in such a case, the questionnaire generation unit 112 creates a questionnaire questionnaire 400 including a question item column 410 for “usage survey”. In this example, the question item column 410 includes one or more questions 412 that ask about the actual use of the function, such as a question Q1 that asks the purpose of use of the function and a question Q2 that asks about the good point (useful point) of the function. included. In this example, a plurality of options are shown in the answer column of each question 412, and a check box 414 is provided instead of a radio button so that a plurality of options can be selected. In addition, a free entry field 416 for entering an answer not listed as an option is also provided. As described above, the question 412 to be listed in the question item 410 and information on each option are registered in the question DB 114, and are read and incorporated in the questionnaire 400.

  In the example of FIG. 4, the question text to be incorporated in the question item column 410 and the contents of the answer column of the question text (options, etc.) are related to the content of the function that is the subject of the question item, and therefore differ for each function. Often. There are common questions such as questions that ask the purpose of use and good points, regardless of the function, but the options often have individual contents for each function. For example, the function “task report” has options as shown in FIG. 4, whereas another function “document registration” will have another option. Therefore, it is desirable to register the contents of the question items separately for each function as illustrated in FIG.

  On the other hand, in the case of a question asking whether the user knows the function as in the example of FIG. 3, “***” indicating the function name is used, such as “Do you know the“ *** ”function”? The part other than “” is a common sentence regardless of the function, and the options are not dependent on the function such as “I know” and “I don't know”. Therefore, for such items, a common question sentence template and option information are registered in the question DB 114 regardless of the function, and an individual question sentence can be obtained by inserting the function name of the function into the question sentence template. It can also be generated. In this case, information indicating a common question sentence template and option information may be registered in the “question item information” in the data content of the question DB 114 of FIG.

  In the example of FIG. 2, it is defined that “user's own usage history” is incorporated into the questionnaire as accompanying display information when the usage frequency is higher than T2 and the number of users is lower than t2. Therefore, the questionnaire generation unit 112 analyzes the log data in the operation / response log storage unit 104 to obtain the usage history chart 422 of the user who is the target of the questionnaire 400, and the accompanying display information 420 including this chart. Is incorporated into the questionnaire 400. The usage history chart 422 displays a time-series change in the number of times the function is used by the user over a predetermined period (in the illustrated example, the past 6 months).

  The user can know the usage status of his / her function B “task report” by referring to the usage history chart. This can be used, for example, in answering each question in the question item column 410. For example, the question Q1 asks the purpose of use of the function “task report”, but it is used when determining whether “group monthly report creation”, which is one of the options, corresponds to the user's own purpose. The usage pattern of the history chart 422 is helpful. For example, if it is found from the usage history chart 422 that this function is frequently used at the end of the month, the user can consider the possibility of using this function for monthly report generation.

  Thus, since the questionnaire 400 includes the usage history chart 422 which is information for each user, the questionnaire 400 is generated individually for each user. The user who is the target of the questionnaire 400 is a user who uses the function B at a high frequency as shown in the “target person information” of the question DB 114. This is the function B of each user. What is necessary is just to discriminate | determine by comparing the usage frequency of this with a threshold value. The discrimination threshold may be registered in the apparatus 10 by the investigator.

  In one aspect, according to the usage history pattern of the user, the question text to be placed in the question item column 410 of the questionnaire 400 for the user is switched, or an option to be put in the answer column even if the question text itself is the same. It is also possible to switch. For example, it is possible to automatically determine what choices of questions to ask about the purpose of use of a function are incorporated in the answer column according to the usage history pattern. That is, for example, if the usage history pattern shows that the function is frequently used at the end of each month, “Create Monthly Report” is selected as an option, and if it is frequent on each weekend, “Create Weekly Report” is selected as an option. Etc.

  According to the questionnaire based on the questionnaire 400 illustrated in FIG. 4, the purpose of using the function B and the selling point of the function B can be obtained. Accordingly, it is possible to propose measures such as advertising the purpose of use and selling points widely to users.

  Next, the example of FIG. 5 will be described. This is an example of the questionnaire questionnaire 500 when the number of users is large and the usage frequency is low or low. This corresponds to the record in the third row of the data illustrated in FIG. The usage condition here is a condition that, as shown in this record, the number of users is greater than t3 and the usage frequency is in the range of T3 to T4.

  In this way, there are many users, but the overall frequency of use is sluggish. Even though a large number of users have used the function, the function can be viewed by the entire user. The reason for this is that it is not used so frequently, and the cause may be that its functionality is not easy to use. Therefore, in this case, as shown in FIG. 5, a questionnaire questionnaire 500 including a question item column 510 in which questions regarding usability are arranged is created. In this example, a question Q1 asking whether the target function C “document registration” is easy to use, and a question Q2 asking an additional question for those who answered that this question is difficult to use as a question Q2 It is shown. As for the question Q1, the answer column 512 has a form of answering whether it is easy to use by “yes” or “no”. On the other hand, in the answer column 514 of the question Q2, problems that can be assumed for the usability of the function C are listed as options, and the corresponding items can be selected (multiple selection is possible).

  Of these questions, the question Q1 only asks whether or not the function is easy to use. Therefore, the sentence of the question sentence other than the function name and the options can be made common regardless of the function. On the other hand, as for the question Q2, the choices listed in the answer column 514 depend on the function, so that it is created separately for each function and registered in the question DB 114.

  Also, the questionnaire 500 shows manual information of the function C as the accompanying display information 520. This is provided for reference when a person who does not use the function C too much considers the cause of not using the function C. See also the third line of the data content example of FIG.

  Since such a questionnaire 500 does not make much sense even if it is sent to a person who has never used the function C, the target is limited to those who have used the function C (the third line of the data content example in FIG. 2). See the record for the eye). The user who has used the information can be obtained by analyzing log data in the operation / response log storage unit 104.

  Such usability problems may be easier to use as you get used to, and may be difficult to use no matter how many times you use them. In the latter case, it is considered that all the experienced users feel a problem. Therefore, it is preferable to have all the experienced users answer the questionnaire 500 as described above. On the other hand, in the case of the former, it is highly likely that the veteran who is familiar with the function does not feel a problem with the convenience of the function. Therefore, in order to improve the usability of the function C and reduce the stumbling at the beginner stage, it is important to answer the person who has stopped using the function at the beginner stage. Therefore, in the former case, it may be possible to limit the target of the questionnaire 500 to beginners of the function C.

  In this way, whether the target range of the question is limited to all experienced users or limited to beginners is determined by the variation in time (referred to as operation time) required for each user to use the function. be able to. In order to use one function, it is generally necessary to sequentially perform a plurality of operations. For example, in the case of the “document registration” function, the “document registration” function is called by selecting a menu, the document to be registered is designated, the registration parameters such as the registration destination folder are designated, and the designated document is designated. A series of operations of pressing the registration button for registering according to the specified parameter is required. If the user interface for using the function is easy to use, the operation time required for the series of operations will be shortened. As described above, the operation time is a determination index as to whether or not the function is easy to use.

  If the variation in operation time is large, it is considered that there is a differentiation between veterans and beginners, so the question target is limited to only beginners. On the other hand, if the variation in operation time is small even though there is a difference in the number of times each person uses function C, it may mean that it is not so easy to use even if you are used to it. I do.

  For such determination, the log analysis unit 106 calculates an operation time required for a series of operations for the use of the function C from each log data each time the user uses the function C. And the operation frequency for every use is totaled for every operation time, and the frequency of use for every operation time is calculated | required. The chart 250 in FIG. 5 is a diagram of this. In the example shown in the drawing, the variation in the operation time is large, and two peaks appear in the distribution of the number of uses arranged for each operation time. The vicinity of the peak with the shorter operation time is experienced, and the vicinity of the longer peak is considered to be a beginner. Even if the peaks are not clearly separated in this way, it can be considered that there is a difference between beginners and veterans if the variation in operation time is large.

  The degree of variation in operation time can be indicated by the dispersion of operation time for each use. Therefore, the log analysis unit 106 calculates the variance of the operation time, and if the variance is smaller than a predetermined threshold (this is registered in advance by an investigator or the like), there is no difference between the veteran and the beginner. The questionnaire 500 is provided to all the users who have used it. When the variance is larger than the threshold value, the questionnaire 500 is provided only for beginners. In this case, for example, an average value of operation times may be obtained, and a user whose average operation time is longer than the average value may be determined as a beginner. The average operation time of the user is an average of the operation time when the user uses the function C. In this example, the average value of operation times for all users is used as a threshold value for determining beginners, but this is only an example. The threshold value can also be determined by other methods such as setting the threshold value to 30% from the longer operation time.

  If a questionnaire is conducted using the questionnaire 500 in FIG. 5, it is possible to obtain a point that the user has tripped in using the function C and present it to the investigator.

  In the above, with reference to FIGS. 3 to 5, specific examples of the usage condition and the questionnaire questionnaire when it corresponds to the usage condition have been described. However, at the beginning of the provision of the function, the frequency of use itself is low, and there are many cases where it does not make sense to perform the log analysis and questionnaire generation processing as exemplified above. Therefore, the above-described processing is performed only for a function for which a certain period of time has elapsed since the start of provision to users (the threshold value of the period for determining this may be registered in the apparatus 10). It is also suitable to make it.

  3 to 5 exemplify questionnaire questionnaires 300, 400, and 500 related to one function, as described above, enumerate the question items for each function and create one questionnaire questionnaire. You can also. In this case, for each user, the question item corresponding to the user is selected and incorporated into the questionnaire questionnaire.

  As described above, according to the survey support apparatus 10, the question items to be incorporated into the questionnaire questionnaire can be switched based on the usage status of the survey target system 20 obtained by log analysis, and wasteful questions can be reduced. it can. In addition, since a questionnaire including only the question items to which the user corresponds can be created for each user, wasteful questions can be reduced also in this respect. Thus, it can be expected that a questionnaire result with higher reliability can be obtained by a questionnaire with fewer useless questions.

  In addition, the survey support apparatus 10 performs a rough analysis of obtaining the function usage frequency and the number of users from the log data, and then performs a questionnaire based on the result, or further distributes the operation time by further log analysis. We will proceed with the survey step by step, such as asking for it. Therefore, there is little waste in processing for investigation.

  The above embodiments are exemplary, and various modifications are possible within the scope of the present invention.

  For example, after analyzing the number of users and usage frequency, it is also possible to add a further analysis using user profile information and conduct a questionnaire reflecting the analysis. An example of this is shown below.

  In this example, if the analysis by the log analysis unit 106 reveals that the number of users of a certain function is large but the frequency of use is not so high (similar to the case of FIG. 5), the log analysis unit 106 further stores log data. Analyze and aggregate usage frequency by user's dominant hand. Then, the log analysis unit 106 examines whether there is a significant difference in the number of uses per user between right-handed and left-handed from the total result. For this purpose, the difference in the number of uses per person between right-handed and left-handed may be compared with a predetermined threshold. If it is greater than the threshold value, the function may be difficult for one handed person. Therefore, if a user-friendly questionnaire is conducted using the same questionnaire as shown in FIG. 5 to the dominant person, it can be confirmed whether it is really difficult for the dominant person.

  Here, analysis focusing on the dominant hand was performed, but in addition to this, users can be grouped in various groups based on profile information, such as by age group, and the usage status of each group from the log data (see above) It is possible to analyze not only the number of times of use but also other things such as operation time, for example, and determine whether there is a difference between groups. If there is a difference between groups, there may be a difference in usability of the function depending on the characteristics of the group, so a questionnaire questionnaire to confirm it may be created.

  In the above example, the online questionnaire survey using the web has been described as an example. However, the mechanism of this embodiment can be used to customize the questionnaire questionnaire even when conducting a questionnaire survey using a paper questionnaire questionnaire. Available.

  The survey support apparatus 10 described above can typically be realized by causing a computer system to execute a program that describes the operation and processing of each functional module described above.

It is a figure for demonstrating schematic structure of the system investigation assistance apparatus which concerns on this invention. It is a figure which shows an example of the data content of question DB. It is a figure which shows the example of a questionnaire questionnaire. It is a figure which shows another example of a questionnaire questionnaire. It is a figure which shows another example of a questionnaire questionnaire.

Explanation of symbols

  10 Survey support device, 20 Survey target system, 30 Network, 40 User PC, 102 Log recording unit, 104 Operation / response log storage unit, 106 Log analysis unit, 108 User DB, 110 Questionnaire processing unit, 112 Questionnaire generation Part, 114 question DB, 116 manual DB.

Claims (9)

Computer system
Detects the operation performed by the user for the function provided by the investigation target system , and includes at least information identifying the user who performed the operation and information on the time when the operation was performed. Log recording means for recording log data ,
Analyzing the log data recorded by the log recording unit, a log analyzing means for calculating a usage index indicating usage of the function, if the usage index of the function satisfies a predetermined condition, further Based on the information included in each log data recorded in the log recording means for identifying the user who performed the operation related to the log data and the information on the time when the operation was performed, the same user performs The operation time required for the series of operations for use of the function performed by the user is calculated from the time when each operation in the series of operations for use of the function is performed. When the variance of the operation time required for the series of operations for use of the function by the user is calculated and the calculated variance is larger than the dispersion threshold, the series of operations for use of the function is performed. Necessary operations Log analysis means that the time to identify the operation time threshold greater than the user,
Questionnaire generating means for generating questionnaire questionnaire data relating to the usefulness of the survey target system, wherein a question item to be incorporated into the questionnaire questionnaire data is selected based on a usage status index of the function obtained by the log analyzing means Questionnaire generation means,
Questionnaire questionnaire generated by the questionnaire generation means when the variance of the operation time required for the series of operations for use of the function by each user calculated by the log analysis means is larger than a dispersion threshold Questionnaire providing means for providing data to a user whose operation time required for the series of operations for using the function is greater than an operation time threshold;
Program to function as. The computer system;
A database in which question condition information indicating information of question items corresponding to the condition is registered for each condition about the usage status index of the function provided by the survey target system;
Further function as
The questionnaire generation unit selects a question item corresponding to the condition as a question item to be incorporated into the questionnaire questionnaire data when the condition satisfied by the function usage status index obtained by the log analysis unit is satisfied in the database. ,
The program according to claim 1. The question condition information includes target person information for identifying a user as a target of a question item corresponding to the condition for each condition,
If the condition that the usage status index of the function obtained by the log analysis unit satisfies is in the database, the questionnaire generation unit generates a question item corresponding to the condition based on target person information corresponding to the condition. Specify the target user, and generate questionnaire questionnaire data including the question items as questionnaire questionnaire data for the specified user.
The program according to claim 2. In the database, the question condition information is registered for each function provided by the survey target system,
The log analysis means obtains a usage status index for each function provided by the survey target system,
The questionnaire generating means is a questionnaire that collects the question items for each user based on the usage status index of each function obtained by the log analyzing means and the question condition information for each function. Generate questionnaire data,
The program according to claim 3.   The program according to claim 1, wherein the usage status index of the function includes the number of users and the usage frequency of the function.   The questionnaire providing means records the questionnaire questionnaire data generated by the questionnaire generating means in the log recording means when the variance calculated by the log analyzing means is not more than the variance threshold. The program according to claim 1, wherein the program is provided to all users who use the function obtained from the log data. The log analysis means, when the usage status index of the function satisfies a predetermined condition, further determines the usage status of the function of the group from the log for each user group distinguished by the user profile information. Seeking
The questionnaire generation means limits the users targeted for the question items according to the degree of difference in the usage situation for each group,
The program according to claim 1. Detects the operation performed by the user for the function provided by the investigation target system , and includes at least information identifying the user who performed the operation and information on the time when the operation was performed. Log recording means for recording log data ;
Analyzing the log data recorded by the log recording unit, a log analyzing means for calculating a usage index indicating usage of the function, if the usage index of the function satisfies a predetermined condition, further Based on the information included in each log data recorded in the log recording means for identifying the user who performed the operation related to the log data and the information on the time when the operation was performed, the same user performs The operation time required for the series of operations for use of the function performed by the user is calculated from the time when each operation in the series of operations for use of the function is performed. When the variance of the operation time required for the series of operations for use of the function by the user is calculated and the calculated variance is larger than the dispersion threshold, the series of operations for use of the function is performed. Necessary operations Time to identify the operation time threshold is greater than the user, a log analysis unit,
Questionnaire generating means for generating questionnaire questionnaire data relating to the usefulness of the survey target system, wherein a question item to be incorporated into the questionnaire questionnaire data is selected based on a usage status index of the function obtained by the log analyzing means Questionnaire generation means;
Questionnaire questionnaire generated by the questionnaire generation means when the variance of the operation time required for the series of operations for use of the function by each user calculated by the log analysis means is larger than a dispersion threshold Questionnaire providing means for providing data to a user whose operation time required for the series of operations for use of the function is greater than an operation time threshold;
System usability investigation support device comprising:   The questionnaire providing means records the questionnaire questionnaire data generated by the questionnaire generating means in the log recording means when the variance calculated by the log analyzing means is not more than the variance threshold. 9. The system usability investigation support apparatus according to claim 8, wherein the system usability investigation support apparatus is provided to all users who use the function obtained from the log data.

Images