JP2005149329A - Intended extraction support apparatus, operability evaluation system using the same, and program for use in them - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an intended extraction support apparatus which extracts a user's intention or the like executed on an operation object, after the end of operation on the basis of information obtained from the user operating an information processing system. <P>SOLUTION: An intended extraction support apparatus 10 which extracts an operator's intention executed on the operation object comprises; a condition detection means 20 which detects the operation condition of the operator executed on the operation object and outputs operation condition information showing a detection result; a solid information detection means 30 which detects solid information of the operator when executing the operation object and outputs solid information showing a detection result; a storage means 50 which combines the operation condition information and the solid information and stores the result as data for intention extraction; a reproducing means 40 which reproduces the operation condition and solid information on the basis of data for intention extraction stored in the storage means; and an intention information storage means 50 wherein intention information presented by the operator is stored when reproducing data for intention extraction. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to operability evaluation support for performing operability evaluation and design support in user interface development of an information processing system and environment development including operation targets.

  In an electronic device including a computer or the like, development of user interface software that is highly convenient for a user (operator) is required. Currently, the software provided in the market is a technology-first type, and has not yet been fully discussed about human processing functions and then supports the processing functions. For example, from the viewpoint of user engineering, ISO 13407 emphasizes user analysis for the purpose of understanding the usage context in the upstream process of the iterative development cycle and usability (operability) evaluation centered on users for measures after design. ing.

  One of the methods for evaluating the ease of use of a computer or the like is to record the video and audio of the operation situation and reproduce the video and audio to evaluate the operability. For example, in Patent Document 1, in order to facilitate the extraction of utterance contents, the utterance contents that can selectively extract the utterance contents when the operator is performing a specific operation from the utterance contents of the operating operator. A content extraction device is disclosed.

  Patent Document 2 relates to an utterance data processing device, and stores utterance contents at the time of work, a time when the utterance is started, a gaze position at the time of an operator's work, and a time when the gaze position changes, and displays utterance data. A display corresponding to the gaze position is displayed, and the gaze position information is included in the utterance data.

  Japanese Patent Laid-Open No. 2004-228867 can automatically propose prompt questions with little variation to a customer by using a database in which past question items are accumulated as a method for extracting customer preference during an interview. Disclose technology.

  Furthermore, in the prior art, there are a method such as a method in which an observer sequentially asks an operator a question during an operation, and a method of thinking utterance in which the operator thinks the utterance during operation.

JP-A-2-131298 JP-A-2-39230 JP 2002-197255 A

  However, the techniques disclosed in Patent Documents 1 and 2 relate to processing and recording after acquiring utterance data, and acquire the utterance contents that are important in the previous stage, how close to the user's cognitive process. It was not considered as to what to do.

  In addition, there is a method of generating a new question item from the question items that have been revealed in the past as disclosed in Patent Document 3, but regarding whether to acquire the utterance content that is close to the recognition process for each individual user Did not consider.

  Furthermore, methods such as a method in which the observer asks the operator questions in an interview format during operation and a method of thinking utterance that causes the operator to utter the thoughts being operated are interrupted, and the operation of the operator is blocked. It was difficult to extract utterances in various operating situations. In addition, when interviews are performed sequentially during operation, it is difficult to use in combination with quantitative measurement related to the operation such as operation time.

  The present invention solves the above-described problems of the prior art, and extracts the intention of the user executed on the operation target after the operation based on the information obtained from the user who operates the information processing system. An object of the present invention is to provide a support device, an operability evaluation system using the same, and a method and a program used therefor.

  The intention extraction support apparatus for extracting the intention of the operator executed for the operation target according to the present invention detects the operation status of the operator executed for the operation target and outputs operation status information indicating the detection result The detection means, the solid information detection means for detecting the solid information of the operator when executing the operation target, and outputting the solid information indicating the detection result, the operation status information and the solid information are synthesized, and this is intended Storage means for storing as extraction data, reproduction means for reproducing operation status and solid information based on intention extraction data stored in the storage means, and intention information presented by the operator when reproducing intention extraction data And intention information storage means for storing.

  Preferably, the operation status information includes video information, audio information, and the like regarding the action of the operator executed for the operation target. The situation detection unit includes, for example, an imaging device that displays an operation situation viewed from the user's field of view, a voice input device that inputs voice of the operation environment, and the like.

  Preferably, the solid information includes information on the biological information of the operator and information on the behavior of the operator. The biological information preferably includes information related to the eye movement of the operator. In addition to this, the biological information may include information such as the brain wave of the operator at the time of operation. In addition to the biological information, the solid information may include, for example, information related to actions such as a nod and an utterance of the operator who is operating.

  Preferably, the reproduction means reproduces the intention extraction data on the display. At this time, on the display, the operation state of the operator executed for the operation target is reproduced, and at the same time, solid information (for example, line-of-sight information) is also displayed.

  Preferably, the intention information includes utterance data of the operator. That is, the operator speaks the intention and idea when he / she tried to perform the operation in a state close to the cognitive process by viewing his / her reproduced operation state and solid information. The utterance can be performed voluntarily by the operator, but may be uttered in an interview format.

  Furthermore, the operation evaluation system of the present invention includes the above-described intention extraction support device and evaluation means for analyzing intention information stored in the intention information storage means and evaluating the operability of the operation target.

  Preferably, the evaluation unit includes a classification unit that classifies the problem from the intention information, a database in which warning information about the problem of operability is registered in advance, the problem classified by the classification unit, and the warning information of the database. Determining means for comparing and determining presence / absence of corresponding warning information.

  Furthermore, the intention extraction support method for extracting the operator's intention executed on the operation target according to the present invention includes the following steps. A step of storing the operator's operation status and the operator's individual information executed for the operation target; a step of reproducing the stored operation status and the operator's individual information after the operation is completed; and the reproduced operation status and Storing the intention information presented by the operator for the solid information. In this case, the intention information preferably includes utterance data presented by the operator while viewing the reproduced operation status and the operator's individual information.

  Furthermore, by using the above-described intention extraction support method, it is possible to realize an operability evaluation method using steps of analyzing intention information stored therein and evaluating the operability of the operation target.

  Further, the intention extraction support program for extracting the operator's intention executed for the operation target is intended to extract the intention by synthesizing the operation status information of the operator executed for the operation target and the operator's solid information about the operation target. Data is generated, the operation status and the solid information are reproduced based on the intention extraction data, and the intention information presented by the operator when the intention extraction data is reproduced is stored.

  According to the present invention, after the operation on the operation target, by reproducing solid information together with the operation status, much deeper intention information (for example, utterance data) that approaches the recognition process for the operation target of the operator is obtained. ) Can be obtained. Furthermore, by analyzing the intention information from the operator, it becomes easy to determine whether the intention intended by the operator and the intention intended by the designer are the same. In some cases, the operability can be improved more than ever by solving the problem.

  The best mode for carrying out the present invention will be described below in detail with reference to the drawings. The intention extraction support apparatus according to the present embodiment is preferably used to determine the degree of recognition of user interface operations in an electronic apparatus such as an information processing system or a computer. That is, the user extracts whether the operation is completely understood, not understood at all, or what kind of intention or idea the operation is performed by a subsequent interview. More preferably, the present invention can be applied to the evaluation of whether or not the intention extracted by the intention extraction support apparatus matches an actual user interface design idea.

  FIG. 1 is a block diagram showing a configuration of an intention extraction support apparatus according to an embodiment of the present invention. The intention extraction support apparatus 10 includes a status detection unit 20 that detects a user's operation status, a solid information detection unit 30 that detects user's solid information, a playback unit 40 including an output device such as a display and a speaker, A storage unit 50 for storing data, a program memory 60 for storing a program for defining an operation of intention extraction support, a data memory 70 for temporarily storing data, a sound pressure sensor 80, and a program for the program memory 60 And a central processing unit 90 that controls these units.

  The situation detection unit 20 detects video information from the user's field of view regarding the operation target and the operation environment, audio information heard in the environment, and the like. For the situation detection unit 20, for example, a general video detection device such as a CCD camera, a detection means by signal branching from a display display screen, a general sound information detection device such as a sound collecting microphone, and the like are appropriately selected and used. Can do.

  The individual information detection unit 30 detects the individual information of the user such as information related to the operation performed by the user, biological information such as movement of the user's line of sight and brain waves during the operation, and information related to actions such as nodding and speech during the operation. To do. The individual information detection unit 30 includes a general operation device such as a button, a pointing device such as a keyboard, a touch panel, and a mouse, a general sound information detection device such as an existing biological information measurement device and a sound collection microphone, and a position. A general video detection device and detection means such as an information detection device, a motion capture, a CCD camera, and detection means by signal branching from a display display screen can be appropriately selected and used.

  Here, the biological information means, for example, information on the user's biological body such as the user's line of sight trajectory during the operation, the time for gazing at the displayed information, the degree of opening of the user's pupil, and the user's brain wave. However, the information is not limited to this as long as it is information about a user's living body when using the information processing system.

  As the storage unit 50, a general recording device such as a VTR, a semiconductor memory, a hard disk, or a flexible disk can be appropriately selected and used. The reproduction unit 40 outputs information recorded in the storage unit 50, and a general display device such as a display or a touch panel or a display device with an audio output function can be appropriately selected and used.

  FIG. 2 is a flowchart showing the operation of the intention extraction support apparatus of this embodiment. The intention extraction support operation is roughly divided into an operation on a predetermined operation target by the user from steps S101 to S105 and an interview to the user performed after the operations from step S106 to S112. FIG. 3 is a block diagram functionally representing the former operation from steps S101 to S105, and FIG. 4 is a block diagram functionally representing the latter operation from steps S106 to S112.

  First, an operation by a user for a predetermined operation target is started (step S101). The target work here may include, for example, all of a series of processes until a user completes one document in a word processor system, or character input and editing in a series of processes. Alternatively, individual processing such as file operation may be performed. Further, processing such as finer cut and paste may be performed. Alternatively, the operation is not limited to the operation on the display, and may be an operation of copying a plurality of copies of a document and closing each of them to complete a booklet in a copying machine.

  Along with the start of the operation, the status detection unit 20 detects the operation status by the user. For example, in the case of a word processor system, the operation status report to be detected is a display image of a display or the like on which work is performed, sound information heard from the surroundings, and the like. At the same time, the solid information of the user is detected by the solid information detection unit 30 (step S102). For example, the movement of the mouse during the work, the movement of the line of sight during the work, the electroencephalogram, the user's nodding and speech, and the like are detected. The movement of the line of sight can be detected by, for example, a line-of-sight tracking device.

  Next, the central processing unit 90 synthesizes the information related to the detected operation status and the solid information (step S103). These pieces of information are synthesized so as to be synchronized with each other. For example, in a work for a word processor system, an image displayed on a working display or the like to which the movement of the mouse or the movement of the line of sight is added is recorded on a recording medium such as a VTR.

  For example, in the operation on the display screen as shown in FIG. 5, the user operates the cursor B1 on the display B at time T1, then moves the cursor P2 at time T2, and moves the cursor P3 to the button at time T3. It is assumed that an operation of moving to D is performed. On the other hand, it is assumed that the user's line-of-sight position (indicated by a black circle in the figure) is on button B at time T1, is on button C at time T2, and is on button D at time T3. Using the operation start time by the user as a reference, the user's operation at each time T1, T2, and T3 and the user's line-of-sight positions E1, E2, and E3 are combined in time. The information thus synthesized in synchronism is stored in the storage unit 50 as intention extraction data (step S104).

  When the user's operation for the operation target or target environment is completed, an interview regarding the intention of the user who is operating is started (step S106). The interview, for example, describes what the user wants to do and what he / she did about the user's behavior during the operation, whether the result of the operation was an expected one, how it was different if it was different from the expected one, etc. I will listen.

  The central processing unit 90 reads the intention extraction data stored in the storage unit 50, and causes the reproduction unit 40 to reproduce it (step S107). The reproduction unit 40 reproduces the operation performed by the user and the user's individual information from the display and the speaker. For example, as shown in FIG. 5, when the user moves the cursor from P1 to P3, the user's line-of-sight position is displayed in synchronization therewith. The change in the line-of-sight position can be displayed by a black circle, but may be displayed in any form other than this.

  Next, the user speaks the situation when the operation is performed while viewing the reproduced display (step S108). For example, taking FIG. 5 as an example, “the operation of button B is completed and the operation proceeds to the next operation” at time T1, “the next operation is lost as button C or button D” at time T2, and “button D at time T3”. ” When a user tries to explain an operation performed in the past, the user is in the background of the operation in a state close to the cognitive process at that time by referring to the position change of the line of sight. The intention can be presented more deeply.

  For example, taking the operation in FIG. 5 as an example, before moving the cursor to the button D, the line-of-sight position E2 stops at the button C, and at that time, the button C is selected or the button D is selected. You may be presented with the intention of wondering what to do. If the line-of-sight position E2 cannot be referred to, there will be much less presentation of such intention.

  The intention extraction data reproduced by the reproducing unit 40 can be appropriately operated in accordance with the progress of the interview, such as stop, fast forward, and rewind. Preferably, the sound pressure sensor 80 is used so that the reading of data from the storage unit 50 is temporarily stopped when the user speaks, and the reading is resumed when the user's speech ends. Thereby, the operation | movement by the reproducing part 40 can be automated according to a user's utterance.

  The individual information detection unit 30 and the user's visual field information and the like are detected by the situation detection unit 20 from the utterance information obtained from the user during the interview (step S110). Since it is necessary to detect the sound in synchronization with the video information, the information detected by the situation detection unit 20 and the individual information detection unit 30 is synthesized in synchronism (step S111) and recorded in the storage unit 50. (Step S112). In the present embodiment, the user's utterance is exemplified as an interview after the operation. However, as a means for presenting the intention other than this, a keyboard is used, a hand signal is given, or other biological information is presented. It may be. Further, it is possible to use quantitative measurement such as operation time during operation.

  FIG. 6 is a diagram showing an utterance effect when an experiment is performed by the intention extraction support apparatus according to the present embodiment. As experimental conditions, video images to be reproduced at the time of interview were prepared with “with gaze” and “without gaze” and presented to a plurality of subjects. The horizontal axis indicates items classified according to the seven-stage model of Norman's action, the vertical axis indicates the amount of speech, “with gaze information” indicates hatching, and “without gaze information” indicates no hatching. As is clear from these figures, the amount of utterance when the line-of-sight information is displayed together is larger than when there is no line-of-sight information, and the utterance content closer to the user's cognitive process can be obtained.

  A seven-stage model of Norman's action is introduced in, for example, Kurosu et al., Introduction to User Engineering, Kyoritsu Publishing Co., Ltd., p103. FIG. 7 shows an example in which user's utterance contents are classified according to a seven-stage model of action. Compared with the conventional example (no line of sight), in the experimental state with the line of sight, it was possible to extract information evenly, but in particular, the evaluation process (perception, interpretation, The increase in information in (Evaluation) was remarkable.

  Next, a second embodiment of the present invention will be described. The operability evaluation system according to the second embodiment is configured using the intention extraction support apparatus according to the first embodiment, and its functional configuration is shown in FIG. The operability evaluation system 100 includes an intention extraction support device 10, a classification unit 110 that classifies problems based on intention extraction data stored in a storage unit of the intention extraction support device 10, and each problem for an operation target. A storage unit 120 that stores warning texts, a determination unit 130 that determines the presence or absence of a warning by comparing the result of classification by the classification unit 110 and the contents of the storage unit 120, and a warning according to the result of the determination unit 130 And a warning display unit 140 to be displayed.

  FIG. 9 is a flowchart for explaining the operation of the classification unit 110. First, the utterance data obtained by the interview is read from the storage unit of the intention extraction support apparatus 10 (step S201). Next, the read utterance data is processed by natural language processing (step S202) and biometric information processing (step S203). By natural language processing, for example, a word related to labeling included in the utterance data and a sentence end expression (not understood, lost, etc.) are extracted (step S202). Further, in the biometric information processing, for example, a portion having a long retention time, a line-of-sight pattern (comparison action, etc.) is extracted based on the amount of gaze (step S203).

  Next, these extraction results are synchronized (step S204). Based on the extraction results of natural language processing and biological information processing, problem classification is performed (step S205). For example, when there is an utterance content as shown in FIG. 7, the problem is classified roughly into a design problem or an execution process problem as shown in FIG. being classified. For example, the statement “I don't know the difference between“ margin ”and“ margin ”” is a design issue and is classified as a labeling issue. Or, the statement “Miss the way if it should be reduced” is classified as a problem of the execution process, and further classified as a problem of printing together on a single sheet.

  The determination unit 130 compares the classified problem with the problem stored in the storage unit 120, reads out a warning or the like if applicable, and displays the warning on the warning display unit 140. The designer can recognize the evaluation of the operation target from the warning displayed on the warning display unit 140.

  According to the present embodiment, it is possible to obtain deeper information based on the user's cognitive process in addition to the action actually performed by the user, and use this to evaluate the operability of the operation target. Also, by reflecting such evaluation results on operability, it is possible to develop highly convenient operability that is suitable for human processing functions that are essentially required for the user.

  The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims. Deformation / change is possible.

  In the first embodiment, the detection results from the situation detection unit and the solid information detection unit are combined and stored in the storage unit. However, the present invention is not limited to this, and detection from the situation detection unit and the solid information detection unit is performed. The result may be stored in the storage unit, and the stored information may be synthesized. Further, the device used in the present invention should not be interpreted in a limited manner, but is a concept including a system in which a plurality of devices are combined. Similarly, the system used in the present invention is interpreted in a limited manner. It is a concept that encompasses a combination of single or multiple devices.

  The intention extraction support apparatus and the operability evaluation system using the same according to the present invention can be used as a technique for performing evaluation / analysis from the viewpoint of human cognitive processing, and in particular, an information processing system, electronic It can be used for evaluation / development of user interface software for devices and the like.

It is a block diagram which shows the structure of the intention extraction assistance apparatus which concerns on the Example of this invention. It is a flowchart which shows operation | movement of the intention extraction assistance apparatus which concerns on the Example of this invention. It is a functional block diagram of the intention extraction assistance apparatus in a present Example. It is a functional block diagram of the intention extraction assistance apparatus in a present Example. It is a figure explaining the example of a user's operation condition and a look position. It is a figure which shows the speech effect based on the recognition process in a present Example. It is a figure which shows the example which classified the utterance content into seven steps of action. It is a block diagram which shows the functional structure of the operativity evaluation system which concerns on 2nd Example of this invention. It is a flowchart which shows operation | movement of the classification | category part of FIG. It is a figure which shows the relationship between utterance content and a classification of a problem.

Explanation of symbols

10: intention extraction support device 20: situation detection unit 30: solid state information detection unit 40: reproduction unit 50: storage unit 60: program memory 70: data memory 80: sound pressure sensor 90: central processing unit 100: operability evaluation system 110 Classification unit 120: storage unit 130: determination unit 140: warning display unit

Claims (17)

An intention extraction support device that extracts an operator's intention executed on an operation target,
A situation detection unit that detects an operation situation of the operator executed on the operation target and outputs operation situation information indicating the detection result;
Solid information detecting means for detecting solid information of an operator when executing the operation target and outputting solid information indicating the detection result;
Storage means for synthesizing the operation status information and the solid information and storing this as intention extraction data;
Reproducing means for reproducing the operation status and the solid information based on the intention extraction data stored in the storage means;
Intention information storage means for storing intention information presented by an operator when reproducing the intention extraction data;
An intention extraction support apparatus. The intention extraction support apparatus according to claim 1, wherein the operation status information includes video information and audio information about an operation target and an operation environment. The intention extraction support device according to claim 1, wherein the solid information includes biological information of an operator regarding an operation target. The intention extraction support apparatus according to claim 3, wherein the biological information includes an eye movement of an operator. The intention extraction support apparatus according to claim 3, wherein the reproduction unit reproduces an operation state and solid information synchronized with the operation state on the display based on the data for intention extraction. The intention extraction support apparatus according to claim 5, wherein the solid information includes an operator's line-of-sight position. The intention extraction support apparatus according to claim 1, wherein the intention information includes utterance data of an operator. The intention extraction support apparatus according to claim 1 or 7, wherein the intention information is presented by an operator in an interview format. The intention extraction support apparatus according to any one of claims 1 to 8,
Analyzing the intention information stored in the intention information storage means and evaluating the operability of the operation target;
An operability evaluation system. The evaluation means compares a classification unit that classifies problems from intention information, a database in which warning information related to operability problems is registered in advance, and problems classified by the classification unit and warning information in the database The operability evaluation system according to claim 7, further comprising determination means for determining the presence or absence of corresponding warning information. The operability evaluation system according to claim 9, further comprising warning means for warning the determination result when the determination means determines that there is warning information. An intention extraction support method for extracting an intention of an operator executed on an operation target,
Stores the operation status of the operator executed on the operation target and the individual information of the operator,
After the operation is completed, the stored operation status and the operator's individual information are reproduced,
An intention extraction support method for storing intention information presented by an operator for reproduced operation status and solid information. The intention extraction support method according to claim 12, wherein the intention information includes utterance data presented by an operator while viewing the reproduced operation status and the individual information of the operator. The intention extraction support method according to claim 12, wherein the solid information is an eye movement of an operator. The steps of claim 12;
An operability evaluation method comprising analyzing the stored intention information and evaluating the operability of an operation target. An intention extraction support program for extracting an operator's intention executed on an operation target,
Create the intention extraction data by combining the operation status information of the operator executed for the operation target and the operator's solid information about the operation target,
Based on the data for intention extraction, reproduce the operation status and solid information,
Storing intention information presented by the operator when reproducing the data for intention extraction;
Intention extraction support program. The intention extraction support program according to claim 16, further comprising a program for evaluating operability of an operation target based on intention information.

Images