CN105701015A - Availability evaluation method of smartphone human-computer interface on the basis of user model - Google Patents

Abstract

A user model-based usability evaluation method for smartphone man-machine interface. According to the user model, a classification standard for usability problems is established; in the process of task testing, the user's eye movement data and user operation behavior data are collected; test data and usability The mapping relationship of the problem is used to analyze the problems existing in the design and give suggestions for improvement. The invention proposes a usability problem classification standard applicable to the usability evaluation of the human-machine interface of a smart phone through the user model, which helps designers find the location of the usability problem and the cause of the error, and can provide effective suggestions for design improvement. By establishing the mapping relationship between objective data and usability problems, it helps usability evaluators to classify the found usability problems. Compared with the subjective evaluation of usability evaluation experts, the classification of usability problems by this method has higher reliability.

Description

A Usability Evaluation Method for Human-Machine Interface of Smartphone Based on User Model

technical field

The invention belongs to the field of human-computer interaction, and relates to a method for evaluating the usability of a human-machine interface, in particular to a method for evaluating the usability of a smart phone human-machine interface based on a user model.

Background technique

Although technological progress has enabled smartphones to have stronger computing power, connectivity, and interaction capabilities, there are still many limitations and challenges in the usability of smartphones due to many factors such as changing use environments, diverse interaction methods, and complex user psychology. , evaluating the usability of smartphones has also become more complex and difficult.

At present, the usability evaluation standard of smartphone human-machine interface is mainly based on the international standard ISO9241—efficiency, effectiveness and satisfaction as usability evaluation indicators, and the usability factor framework established by some ergonomic experts. On this basis, according to the characteristics of smartphones Some supplementary factors are proposed. However, most of these standards are "expert-oriented", and it is impossible to establish a "user-oriented" test method and test process based on these indicators. "Design and evaluation system issues. The degree to which the test discovers and solves usability problems mainly depends on the experience level of the testers, resulting in low reliability and validity.

The traditional usability evaluation method is based on the laboratory environment, with the desktop computer as the main object. The usability evaluation method of smartphone human-machine interface emphasizes the impact of situational factors on users. On the one hand, it is necessary to reflect the real use process as much as possible, and on the other hand, it is necessary to minimize the interference of researchers and test equipment on users. Complete and Accurate Records Collecting reliable data has been a long-standing research challenge. There are already many methods to collect test data through eye trackers, sensors, and user log data. These data are related to each other and affect each other. Most of the data analysis is based on existing usability indicators, and there is a lack of in-depth analysis of user actions and psychological motivation. Resulting in a lack of suggestions for design improvements.

Contents of the invention

The purpose of the present invention is to provide a method for assessing the usability of the human-machine interface of a smart phone based on user models, establish a unified standard throughout the design and testing process, and improve the reliability and validity of usability assessment. According to the classification and statistics of usability problems, there are Find design problems in a targeted manner and give suggestions for improvement.

To achieve the above object, the present invention adopts the following technical solutions:

A user model-based usability evaluation method for a smartphone man-machine interface, comprising the following steps:

Step 1: Establish a user model, including an action model and a cognitive model, to describe the operating characteristics of smartphone users from two aspects of the user's action model and cognitive model;

Step 2: According to the user's action model and cognitive model, establish a classification of usability problems, and divide usability problems into 7 categories: purpose conversion error, function entry cannot be found, command location cannot be found, and icon/function meaning is not understood , Can't find the next step, don't understand the operation command and the feedback information is insufficient;

Step 3: Define the data statistical indicators that need to be collected, including 2 predefined data according to the operation task: the number of standard operation steps, denoted as Cstd, and the number of function entry steps, denoted as Estd; 3 user operation behavior data: user operation The number of reaching, denoted as Cact, the number of user operation chains, denoted as N, the operation chain where the function is completed, denoted as n; the eye movement data of 2 users: the fixation points of the interest area, denoted as Ef, the average fixation time of the interest area, denoted as is Et;

Step 4: Build a usability testing platform for smartphones; use the eye-tracking system to obtain the user's eye-movement data, and use the getevent command that comes with the Android operating system to collect the user's operational behavior data through the debugging bridge;

Step 5: Design the test task and test process, and obtain the operation behavior data and eye movement data of the tested user:

Step 6: According to the measured user's operation behavior data and eye movement data obtained in step 5, count the number of operations achieved by the user for each operation task Cact, the number of operation chains N, the operation chain n where the function is completed, the number of fixation points Ef and interest in the area of interest Regional average fixation time Et; by judging the relationship of the 7 data statistical indicators defined in step 3, establishing the mapping relationship with the 7 types of usability problems in step 2, and discriminating the types of usability problems according to the usability problem discrimination process;

Step 7: According to the statistics of the types of usability problems, find design problems and give suggestions for improvement.

The action model in step 1 includes establishment of purpose, planning, implementation and evaluation.

The cognitive model in step 1 includes finding, identifying, understanding and remembering.

The test process in the step 5 is specifically as follows:

Preparation stage: The main tester will introduce the test content, distribute personal information questionnaires, and help the tested users to wear and debug the experimental equipment for normal use; the tested users fill out the personal information questionnaires to confirm that the experimental equipment does not affect the user's operation of the mobile phone. ,start testing;

Test phase: The main tester explains the test task to the user, observes and records the user's operation behavior; the user under test starts to operate after understanding the task, and asks the main tester if there is any problem or operation difficulty, and gives feedback on operation experience or suggestions after the test ;

End stage: The chief tester announces the end of the test after confirming the completion of the test project.

In the step 5, the novice user, common user and expert user are defined according to the user's experience in using the smart phone operating system.

The usability problem type discrimination process in the step 6 is as follows:

When Cact-Cstd=0, it is considered that the user has completed the operation; when Cact-Cstd<0, it is considered that the user has not completed the operation;

When Cact-Cstd=0 and n-N=0, it is considered that the user can judge that the operation has been completed;

When Cact-Cstd=0 and n-N<0, it is considered that the user still tries to operate after the operation is completed, indicating that the feedback information is insufficient;

When Cact-Cstd<0 and n-N=0, it is considered that the user uses the wrong function because he does not understand the task concept or the target command concept, which belongs to the target conversion error;

When Cact-Estd<0, it is considered that the user cannot find the function entry;

When _Cact -Estd=0 and Ef _i =0 or when Cact-Estd=0 and Et _i <100, it is considered that the user cannot find the position where the function command is; ; Et _i represents the average fixation time of the region of interest where the function icon object number is i;

When Cact-Estd=0 and Ef _i >0 or when Cact-Estd=0 and Et _i >100, it is considered that the user does not understand the meaning of the icon/function;

When Cact-Estd>0 and Ef _i >0 and Et _i <100, it is considered that the user understands the meaning of the icon/function and proceeds to the next step;

When Cact-Estd>0 and Ef _j =0 or when Cact-Estd>0 and Et _j <100, it is considered that the user cannot find the next step of operation; wherein, Ef _j represents the number of interest area fixation points for the operation command object number j, Et _j represents the average fixation time of the region of interest whose operation command object number is j;

When Cact-Estd>0 and Ef _j >0 or when Cact-Estd>0 and Et _j >100, it is considered that the user does not understand the operation command.

Compared with the prior art, the present invention has the beneficial effects:

(1) The present invention proposes usability problem classification standards applicable to the usability evaluation of smartphone man-machine interface through the user model, which helps designers find the location and error cause of usability problems, and can provide effective suggestions for design improvement.

(2) The present invention helps usability evaluators to classify discovered usability problems by establishing a mapping relationship between objective data and usability problems. Compared with the subjective evaluation of usability evaluation experts, the classification of usability problems by this method has higher reliability.

(3) The present invention has good universality and flexibility, and can evaluate various mobile phone application interfaces.

(4) In the process of task testing, use technical means such as eye movement tracking to collect data in real usage scenarios more comprehensively and without interference; establish a mapping relationship between test data and usability problems for cause analysis and problem improvement, It overcomes the problems in the prior art that data analysis is only performed on usability indicators and lacks in-depth analysis of user actions and psychological motivations, resulting in a lack of suggestions for design improvement. The invention improves the reliability and validity of usability evaluation.

Description of drawings

Figure 1 is the user cognitive model of the smart phone man-machine interface;

Figure 2 is the user action model of the smart phone man-machine interface;

Figure 3 is the usability problem classification standard;

Figure 4 is the usability testing process;

Fig. 5 is a flow chart of usability problem identification.

detailed description

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

The usability evaluation method of the smart phone man-machine interface provided by the present invention comprises the following steps:

Step 1: First of all, it is necessary to understand the basic characteristics of the user when operating the smartphone and establish a user model. The operating characteristics of smartphone users are described from two aspects of user's action model and cognitive model.

Taking a user looking for a phone icon on the main interface of a mobile phone as an example, the user's perception and cognitive process are as follows:

1) Find and discover: Tour around the main interface to see where there are icons. Vision is able to ignore all kinds of other visual information that is not relevant, and only look for icons. Different users may search according to different strategies. When the observed visual signal is basically consistent with the expected one, the vision will basically follow the signal fixedly.

2) Recognition: Distinguish the phone icon you are looking for among several similar icons. When several similar stimulus signals are found, identify the meaning of the target according to the visual feature information or detailed information.

3) Understanding: compare with the information and knowledge stored in the memory to confirm whether the icon is a phone icon.

4) Memory: store the above process in the memory.

According to the above process, the established user cognitive model is shown in Figure 1.

Similarly, taking the task of calling someone with a mobile phone as an example, the specific operation stages of the user are as follows:

1) Establish the purpose: the user determines the purpose of use according to the usage scenario and needs.

2) Plan: In order to achieve the goal, the user needs to establish an action plan, mainly considering what functions to use and how to complete them. For example, in order to make a real-time voice call with the other party, the user plans to use the call function of the mobile phone. From planning to specific operational implementation, there is a machine-oriented operational transformation process. Specifically consider where these functions are and how to operate them.

3) Implementation: When users use touch-screen smart phones, various implementation actions cannot be carried out at will, but can only be operated through the touch screen according to the interactive mode specified by the system, and feedback information can be obtained through the screen. Transforming the action plan into an operation process is mainly divided into two steps: first, transform the user’s psychological action process into an acceptable operation sequence for the smartphone; second, transform each action step into a micro operate.

4) Evaluation: It refers to comparing the feedback information given by the smart phone with the purpose of the end user to judge whether the plan is correct and whether the operation is completed.

If it meets the user's expectations and predictions, the purpose is completed; if it does not meet, it means that the plan or operation is wrong, and the action plan needs to be re-established or the operation should be re-operated.

According to the above process, the established user action model is shown in Figure 2.

Step 2: According to the user's action model and cognitive model, if the user cannot complete the task operation purpose in operating the mobile phone because the design of the mobile phone system or mobile application does not conform to the user's action characteristics or cognitive characteristics, it can be regarded as Usability issues, as shown in Table 1.

Table 1 Evaluation Criteria for Usability Questions

Based on user models and usability evaluation criteria, establish a taxonomy of usability issues appropriate for smartphone manipulation tasks. As shown in Figure 3, usability problems are divided into 7 categories: purpose conversion error, function entry cannot be found, command location cannot be found, icon/function meaning cannot be understood, next step operation cannot be found, and operation commands and feedback cannot be understood Insufficient information.

Step 3: Define the data statistical indicators that need to be collected during the test process, including 2 predefined data according to the operation task: standard operation steps (Cstd), function entry steps (Estd); 3 user operation behavior data: user The number of operations achieved (Cact), the number of user operation chains (N), the operation chain where the function is completed (n); the eye movement data of 2 users: the fixation points of the interest area (Ef) and the average fixation time of the interest area (Et), see Table 2.

Table 2 Definition of statistical data indicators

Step 4: Build a usability testing platform for smartphones. This platform includes the following equipment: a test mobile phone equipped with Android 5.0, a glasses-type eye-tracking system from SMI, and a laptop computer equipped with eye-tracking software. Use the eye-tracking system to obtain the user's eye-movement data, and use the getevent command that comes with the Android operating system to collect the user's operation behavior data through the debugging bridge.

SMI's glasses-type eye-tracking system uses glasses-type eye tracker. The advantage of using glasses-type eye tracker is that it can record the eye movement data when the user operates the real device, and simulate the user's real operation situation as much as possible. The eye tracking system includes hardware and software. The sampling rate of the hardware device is 30Hz (both eyes), the tracking resolution: <0.1°, the gaze accuracy: <0.5°; this method is equipped with 5 groups from -200° to - The 600° public pupil distance myopia lens is convenient for myopia tested users to wear. The software used is iViewETG data acquisition software and Begaze data analysis software.

Compared with the method of directly writing the operation recording program in the listener, the method of using getevent to obtain the user's click coordinates and then convert it into a click control is complicated and time-consuming. However, the advantage of this method is that the data clicked by the user globally can be obtained without modifying the underlying source code of the Android system, and the data of the physical button clicked by the user can also be obtained at the same time. This method is suitable for collecting data under multi-functional and multi-application conditions, and is suitable for evaluating various types of mobile phone applications.

Step 5: Design the test task and test process: the testers include the main tester and the user under test. The main tester helps the user wear the instrument, explains the test task to the user, observes and records the user's test operation and asks questions. The tested users include three types of users: novice users, ordinary users and expert users. Define users based on their experience with smartphone operating systems. The testing process is shown in Figure 4. The test process is as follows:

Preparation stage: The main tester will introduce the test content, distribute personal information questionnaires, and help the tested users to wear and debug the experimental equipment for normal use; the tested users fill out the personal information questionnaires to confirm that the experimental equipment does not affect the user's operation of the mobile phone. ,start testing. Before the test, the eye tracker is assisted in the calibration.

Test phase: During the test, the tested user needs to keep the distance between the mobile phone and the eyes at 30-40cm, and try to adopt a posture that can make the eyes of both eyes look at the same level, so as to improve the accuracy of eye movement data collection. The chief tester explains the test task to the user, observes the user's operation behavior, obtains the user's operation behavior data through the getevent command of the Android operating system, and uses the Begaze software to record the user's eye movement data. Provide appropriate help when the user encounters operational difficulties, and at the same time observe the accuracy of the eye movement data collection. When the accuracy of the eye movement data changes greatly, it needs to be recalibrated; the tested personnel start to operate after understanding the task. Ask the main tester if you have any questions or operational difficulties, and give feedback on the operation experience or suggestions after the test.

End stage: The main tester announces the end of the test after confirming the completion of the test project and expresses his thanks.

Step 6: Data analysis: Statistically process the user operation behavior data and eye movement data obtained in step 5, and judge the user's operation gesture and operation control object according to the click event and time point obtained by getevent; according to the interest area of Begaze software The (AOI) statistics function performs eye movement data statistics on the calibration control. Through judging the relationship of the 7 statistical indicators, see Table 3, establish the mapping relationship with the 7 types of usability problems, and judge the types of usability problems according to the usability problem identification process.

Referring to Figure 5, the usability problem identification process is as follows:

According to the test tasks, the standard operation steps Cstd and the function entry steps Estd of each test task are pre-defined before the test. After the test is completed, according to the collected user click data and eye movement data, the user's operation of each operation task is counted. The number Cact reached, the number of operation chains N, the operation chain n where the function is completed, the number of fixation points Ef of the interest area, and the average fixation time Et of the interest area; i indicates the number of the function icon object, j indicates the number of the operation command object, and add the following to different interest areas Marked distinctions, such as Ef _i , Ef _j and Et _i , Et _j .

When Cact-Cstd=0, it is considered that the user has completed the operation; when Cact-Cstd<0, it is considered that the user has not completed the operation;

When Cact-Cstd=0&n-N=0, it is considered that the user can judge that the operation has been completed;

When Cact-Cstd=0&n-N<0, it is considered that the user still tries to operate after the operation is completed, indicating that the user cannot judge whether it is completed, that is, the evaluation feedback information is insufficient;

When Cact-Cstd<0&n-N=0, it is considered that the user uses the wrong function because he does not understand the task concept or the target command concept, which belongs to the target conversion error;

When Cact-Estd<0, it is considered that the user cannot reach the function entry interface, that is, the user cannot find the function entry;

When Cact-Estd=0&Ef _i =0|Et _i <100, it is considered that the user cannot find the target function command in the function entry interface, that is, the position of the command cannot be found, and i represents the function icon object number; Ef _i represents the function icon object number Be the region of interest gaze points of i; Et _i represents the average gaze time of the region of interest that the functional icon object number is i;

When Cact-Estd=0&Ef _i >0&Et _i >100, it is considered that the user does not understand the meaning of the icon/function;

When Cact-Estd>0&Ef _i >0&Et _i <100, it is considered that the user understands the meaning of the icon/function and proceeds to the next step;

When Cact-Estd>0&Ef _{j ＝} 0|Et _j <100, it is considered that the user cannot find the next operation command; _j represents the number of the operation command object; The average fixation time of the region of interest whose operation command object number is j;

When Cact-Estd>0&Ef _j >0&Et _j >100, it is considered that the user recognizes the operation command, but because he does not understand how to operate, the final operation is not completed, that is, the user does not understand the operation command.

Table 3 Discrimination of data index relationship

Step 7: According to the statistics of the types of usability problems, find the design problems in a targeted manner, and give suggestions for improvement. See Table 4 for the design problems and improvement methods corresponding to the specific usability problems.

Table 4 Design issues and improvement methods corresponding to usability issues

The present invention improves the effectiveness and practicability of the usability evaluation method of the human-machine interface of the smart phone: according to the user model, the classification standard of the usability problem is established; in the process of task testing, the user's eye movement data and user operation behavior data are collected; The mapping relationship between test data and usability problems is used to analyze the problems in the design and give suggestions for improvement.

Claims (6)

1. A user model-based smart phone man-machine interface usability evaluation method, is characterized in that, comprises the following steps: Step 1: Establish a user model, including an action model and a cognitive model, to describe the operating characteristics of smartphone users from two aspects of the user's action model and cognitive model; Step 2: According to the user's action model and cognitive model, establish a classification of usability problems, and divide usability problems into 7 categories: purpose conversion error, function entry cannot be found, command location cannot be found, and icon/function meaning is not understood , Can't find the next step, don't understand the operation command and the feedback information is insufficient; Step 3: Define the data statistical indicators that need to be collected, including 2 predefined data according to the operation task: the number of standard operation steps, denoted as Cstd, and the number of function entry steps, denoted as Estd; 3 user operation behavior data: user operation The number of reaching, denoted as Cact, the number of user operation chains, denoted as N, the operation chain where the function is completed, denoted as n; the eye movement data of 2 users: the fixation points of the interest area, denoted as Ef, the average fixation time of the interest area, denoted as is Et; Step 4: Build a usability testing platform for smartphones; use the eye-tracking system to obtain the user's eye-movement data, and use the getevent command that comes with the Android operating system to collect the user's operational behavior data through the debugging bridge; Step 5: Design the test task and test process, and obtain the operation behavior data and eye movement data of the tested user: Step 6: According to the measured user's operation behavior data and eye movement data obtained in step 5, count the number of operations achieved by the user for each operation task Cact, the number of operation chains N, the operation chain n where the function is completed, the number of fixation points Ef and interest in the area of interest Regional average fixation time Et; by judging the relationship of the 7 data statistical indicators defined in step 3, establishing the mapping relationship with the 7 types of usability problems in step 2, and discriminating the types of usability problems according to the usability problem discrimination process; Step 7: According to the statistics of the types of usability problems, find design problems and give suggestions for improvement. 2. A method for assessing the usability of a smart phone man-machine interface based on a user model according to claim 1, wherein the action model in said step 1 includes establishing purpose, planning, implementation and evaluation. 3. a kind of smart phone man-machine interface usability assessment method based on user model according to claim 1, is characterized in that, in described step 1, cognitive model includes finding discovery, recognition, comprehension and memory. 4. a kind of smart phone man-machine interface usability evaluation method based on user model according to claim 1, is characterized in that, in described step 5, test flow is specifically as follows: Preparation stage: The main tester will introduce the test content, distribute personal information questionnaires, and help the tested users to wear and debug the experimental equipment for normal use; the tested users fill out the personal information questionnaires to confirm that the experimental equipment does not affect the user's operation of the mobile phone. ,start testing; Test phase: The main tester explains the test task to the user, observes and records the user's operation behavior; the user under test starts to operate after understanding the task, and asks the main tester if there is any problem or operation difficulty, and gives feedback on operation experience or suggestions after the test ; End stage: The chief tester announces the end of the test after confirming the completion of the test project. 5. a kind of smart phone man-machine interface usability evaluation method based on user model according to claim 1, is characterized in that, in described step 5, novice user, common user and expert user are based on user's operating system of smart phone Define users using experience. 6. a kind of smart phone man-machine interface usability evaluation method based on user model according to claim 1, is characterized in that, in the described step 6, usability problem type discrimination process is specifically as follows: When Cact-Cstd=0, it is considered that the user has completed the operation; when Cact-Cstd<0, it is considered that the user has not completed the operation; When Cact-Cstd=0 and n-N=0, it is considered that the user can judge that the operation has been completed; When Cact-Cstd=0 and n-N<0, it is considered that the user still tries to operate after the operation is completed, indicating that the feedback information is insufficient; When Cact-Cstd<0 and n-N=0, it is considered that the user uses the wrong function because he does not understand the task concept or the target command concept, which belongs to the target conversion error; When Cact-Estd<0, it is considered that the user cannot find the function entry; When _Cact -Estd=0 and Ef _i =0 or when Cact-Estd=0 and Et _i <100, it is considered that the user cannot find the position where the function command is; ; Et _i represents the average fixation time of the region of interest where the function icon object number is i; When Cact-Estd=0 and Ef _i >0 or when Cact-Estd=0 and Et _i >100, it is considered that the user does not understand the meaning of the icon/function; When Cact-Estd>0 and Ef _i >0 and Et _i <100, it is considered that the user understands the meaning of the icon/function and proceeds to the next step; When Cact-Estd>0 and Ef _j =0 or when Cact-Estd>0 and Et _j <100, it is considered that the user cannot find the next step of operation; wherein, Ef _j represents the number of interest area fixation points for the operation command object number j, Et _j represents the average fixation time of the region of interest whose operation command object number is j; When Cact-Estd>0 and Ef _j >0 or when Cact-Estd>0 and Et _j >100, it is considered that the user does not understand the operation command.