JP3027521B2 - Unattended terminal device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a cash dispenser and interactive information guidance device, a unmanned terminal device unspecified user performs an operation, in particular, the right time the presentation of help information to the user It is constituted so that it can be implemented by.

[0002]

2. Description of the Related Art In this specification, a terminal device operated by a user without the help of a specialized operator is referred to as an unmanned terminal device. Instructions that give instructions are called help.

[0003] Conventional terminal devices have been provided with a function of displaying a help screen for a user who does not understand the operation procedure and teaching the operation procedure.

For example, as shown in FIG. 15, the apparatus disclosed in Japanese Patent Laid-Open No. 3-144819 has an input unit 3001 for a user to perform an input operation, a processing unit 3002 for executing an input command, and
An output unit 3003 that outputs a processing result and a help output unit 3004 that outputs help information are provided.
If an erroneous operation is performed from 01, the help output unit is sent from the processing unit 3002
A help output instruction for notifying the state of the processing unit 3002 is output to 3004, and in response, the help output unit 3004
The help information corresponding to the state of 02 is output to the output unit 3003.

The device disclosed in Japanese Patent Laid-Open No. 4-160527 is
As shown in FIG. 16, further provided is an input monitoring unit 3105 for monitoring the presence or absence of an input, and a timer unit 3106 for measuring a time during which no input operation is performed. In this device, when the input from the input unit 3101 is not performed for a time set in the timer unit 3106 in advance, the input monitoring unit 3105
It is determined that the user cannot understand the operation method and cannot perform the operation, and outputs a help output command to the help output unit 3104. Upon receiving the help output command, help output unit 3104 inquires of processing unit 3102 about the current state, and outputs help information corresponding to the state to output unit 3103.

Japanese Patent Application Laid-Open No. 1-244520 discloses that help data corresponding to the state of a processing section is selected and output as help information.

As shown in FIG. 17, when the user interrupts the operation from the input unit 3201, an input monitoring unit 3203 for monitoring the presence or absence of an input measures time using a timer unit 3204, and A device that initializes processing of the processing unit 3202 when there is no input even after a set time has elapsed is known.

[0008]

However, when these conventional configurations are applied to an unmanned terminal device, there are the following problems.

First, in the method of displaying the help information after the user has performed an erroneous operation, the erroneous operation cannot be prevented beforehand, and the user must perform the correct operation again after the erroneous operation, so that the operation efficiency is reduced. bad.

Secondly, in the method of monitoring the input and automatically outputting the help information when there is no input within a preset time, the user of the unattended terminal device continues to display the information displayed on the device. There is a possibility that help information may be output even when help is not needed and help is needed.

Third, in the conventional method, the help information is displayed in an expression method assuming an average user.
It is not an easy-to-understand expression that satisfies all wide-ranging users of the unattended terminal device. For example, for a certain person, displayed characters are small, and words are difficult to express.

Fourthly, in the method of automatically initializing the processing of the processing section when there is no input within the set time, the processing section of the processing section is operated regardless of the state in which the user is viewing the output information. Initialization may be performed.

[0013] The present invention is intended to solve such conventional problems, and its object is to provide an unattended terminal that can be given help information appropriate to the user at the right time.

[0014]

Therefore, according to the present invention, there are provided input means for performing an input operation by a user, processing means for performing a process in accordance with the input operation, and output means for outputting a processing result to the user. User in an unattended terminal device equipped with
The elements that make up the facial expression from image data showing the face of
A user situation detecting means for extracting and detecting facial expression pattern data, a help necessity determining means for determining whether a user needs help based on the detected facial expression pattern data, and a help required Help output means for outputting help according to the processing status of the processing means when the gender determination means determines that help is required.

[0015]

[0016] In addition, the personal information input means for inputting a physical handicap information <br/> report of the user, the input physical Ha
Personal information analysis means for instructing a method of expressing help conforming to the handicap information; and help output means for outputting help using the help display method instructed by the personal information analysis means.

[0017]

[0018]

[0019]

Therefore, it is possible to accurately determine the situation where the user is lost in the input operation from the user's eyes and facial expressions, and to provide help information at that stage. Therefore, the user can know the correct operation procedure before taking an erroneous operation, so that the operation efficiency is improved. In addition, in a device for inputting personal information of a user, for example, a display font is increased for an elderly user, an output sound level is increased, and the like.
Help information can be output in an expression suitable for the user.

[0020]

[0021]

【Example】

(First Embodiment) As shown in FIG. 1, an unmanned terminal device according to a first embodiment of the present invention has an input unit 101 for a user to input operation information, and is determined in advance according to the input operation information. A processing unit 102 that performs the above processing, an output unit 103 that outputs a processing result of the processing unit 102 to a user, a help output unit that outputs help information, a user situation detection unit that detects a line of sight of the user, A help necessity determining unit that determines whether help is needed from the user's eyes; a help output unit; a help processing unit 104 that performs help processing corresponding to the processing state of the processing unit 102; And a help database unit 105 that stores the user status. The user status detection unit includes an image input unit that captures the status of the user as an image.
106, a gaze detection unit 107 that detects a user's gaze from image data output by the image input unit 106, and a gaze movement detection unit 108 that detects gaze movement from gaze data output by the gaze detection unit 107. In addition, the help necessity determination unit includes a gaze movement pattern database 109 storing data of a user state corresponding to gaze movement pattern data,
By comparing the detected gaze movement pattern data with the gaze movement pattern database 109, it is determined whether the user needs help, and when it is determined that help is needed, a help output command is sent to the help processing unit 104. Output unit 110.

In this unmanned terminal device, the image input unit 106 captures the state of the user as an image, and outputs image data to the visual line detection unit 107. The line-of-sight detection unit 107 performs image processing on the image data received from the image input unit 106,
It detects the user's line of sight and outputs line-of-sight data to the line-of-sight motion detection unit 108. Eye gaze data refers to information on where the user is looking. The gaze movement detection unit 108 detects a gaze movement pattern from the gaze data received from the gaze detection unit 107, and outputs the gaze movement pattern data to the determination unit 110. The line-of-sight motion pattern refers to data indicating how the line of sight has moved during a predetermined period of time tracking the line-of-sight data. The determination unit 110 compares the gaze movement pattern data output from the gaze movement detection unit 108, the gaze movement pattern data stored in the gaze movement pattern database unit 109, and the state of the processing unit output from the processing unit 102. And determine if you need help. The details of the eye movement pattern database and the flow of processing in the determination unit 110 will be described later.

When the determination section 110 determines that help is required as a result of the collation, it outputs a help output command to the help processing section 104. The help processing unit 104 that has received the help output command selects the help data corresponding to the state of the processing unit 102 received from the processing unit 102 from the help database unit 105, and outputs the help data to the output unit 103. The user can proceed with the subsequent input operation by viewing the output help information.

The line-of-sight movement pattern database unit 109 stores a line-of-sight movement pattern database in which a line-of-sight movement pattern is associated with a user state. FIG. 2 shows an example of the table of the eye movement pattern database. For example, if the user's line of sight is unsteady for more than a certain period of time and the state of the processing unit 102 is an input screen, it is determined that the user's state does not know how to use the terminal, and help is needed. Relate to being.

The determining unit 110 determines the necessity of the help according to the procedure shown in FIG. 3 using the eye movement pattern database.

Step 1: When the gaze movement pattern data is input from the gaze movement detection unit 108, the judgment unit 110 acquires data on the state of the processing unit from the processing unit 102. Step 2: Using these data as a search key, The operation pattern database is searched. Step 3: It is determined whether or not the state of the user corresponds to a state that requires help specified in the gaze operation pattern database. If the state corresponds to a state requiring help, step 4: a help output instruction is output to the help processing unit 104, and the processing of the determination unit 110 ends. If it is determined in step 3 that no help is required, the process of the determining unit 110 ends.

As described above, in the unmanned terminal device of the first embodiment, the state of the user is detected on the unmanned terminal side based on the movement of the user's line of sight, and when the user needs help, Output help information at the right time. Therefore, the utilization efficiency of the user can be improved.

(Second Embodiment) The unmanned terminal device according to the second embodiment determines whether the user needs help from the expression of the user. This device, as shown in FIG.
The user status detection unit includes a facial expression detection unit 407 that detects the user's facial expression from the image data, and the help necessity determination unit outputs the user status data in accordance with the facial expression pattern. A stored expression pattern database 408 is provided. Other configurations are as in the first embodiment (FIG. 1).
There is no change.

In this apparatus, the expression detecting section 407 performs image processing of the image data received from the image input section 406,
The facial expression of the user is detected, and facial expression pattern data is output to the determination unit 409. In order to detect this facial expression, the facial expression detection unit 407 converts the input image data of the user's face into a wire frame, extracts the elements constituting the facial expression, and samples the facial expression of the facial expression detection unit 407 and The data is collated, and sample data representing the closest expression pattern is output as expression pattern data.

The determination unit 409 includes the expression pattern data output from the expression detection unit 408 and the expression pattern database unit 40.
8 is compared with the state of the processing unit 402 output from the processing unit 402, and when it is determined that the user needs help, the help processing unit 404
Output help output instruction to

As shown in FIG. 5, the user's state corresponding to the facial expression pattern is stored in the facial expression pattern database. For example, if the user has a confused expression, and if the state of the processing unit 402 is an input screen, the user is determined to be in a state in which he does not understand how to use the terminal,
Relate to need help. FIG.
9 shows a procedure for determining output of help, which is substantially the same as the procedure in FIG.

(Third Embodiment) The unmanned terminal device of the third embodiment can display help information in a form suitable for each user. This device has an input unit as shown in FIG.
701, a processing unit 702, an output unit 703, and a help processing unit 704 and a help database unit 705 that constitute a help output unit
In addition to the above, a personal information input unit 70 for inputting user's personal information
6 and a personal information analysis unit 707 that analyzes the input personal information.
And

In this device, when the user starts using the device, an IC card or the like in which his / her personal information is stored is connected to the personal information input unit 706. The personal information input unit 706 reads this personal information and outputs it to the personal information analysis unit 707. Personal information refers to age, gender, frequency of use, how far the user has used last time, or personal restrictions such as poor eyesight, inability to see, and distant ears.

As shown in FIG. 8, the personal information analysis unit 707 has a correspondence table between personal information and a help expression method, analyzes the personal information input from the personal information input unit 706, and converts the personal information into the personal information. The corresponding help expression method information is obtained from the table and output to the help processing unit 704.

When the user performs an input operation from the input unit 701, the processing unit 702 executes a process corresponding to the input operation information. When the user requests help from the input unit 701 during the input operation, the request is input to the help processing unit 704 and the processing unit 702, and the processing unit 702 sends the help processing unit 704 Output the processing status.

In response, the help processing unit 704 converts the help data having contents corresponding to the processing state of the processing unit 702 and corresponding to the help expression method information output from the personal information analysis unit 707 into the help database. The helper 705 retrieves the information and outputs it to the output unit 703 in an expression corresponding to the help expression method information.

For example, when the user is an elderly person, the personal information analysis unit 707 uses the table shown in FIG. Help expression part information that means "reduce
The help data is output to the help database unit 704. The help processing unit 704 searches the help database unit 705 for help data having contents corresponding to the processing state of the processing unit 702, which data is expressed in an easy-to-understand manner by reducing difficult phrases. Then, the detected help data is sent to the output unit 703 such that the display font is enlarged for easy viewing and the volume of the voice band is increased for easy listening.

As described above, in the unmanned terminal device according to the third embodiment, the help information can be output in a display method suitable for each user, so that the use efficiency and the understanding rate of the user can be improved. .

(Fourth Embodiment) The unmanned terminal device according to the fourth embodiment executes processing initialization at an appropriate timing when a user interrupts an input operation. As shown in FIG. 9, the apparatus includes an input unit 1001 for inputting operation information by a user,
Processing unit 1002 that performs processing according to input operation information
An output unit 1003 that outputs the processing result of the processing unit 1002, an image input unit 1004 that monitors the status of the user, and an image input unit 10
Based on the information from 04, a person detection unit 1005 that detects whether a person is present in front of the terminal, and a time that has elapsed since the user left the terminal, and a processing unit 1002 after a certain time has elapsed A timer unit 1006 for instructing initialization of the processing.

In this device, the image input unit 1004 captures the state of the user as an image, and outputs the image data to the human detection unit 1005. The person detection unit 1005 processes the image data, detects whether a user exists in front of the terminal, and outputs the user presence / absence data to the timer unit 1006. The timer unit 1006 starts the timer when the input user presence data indicates the absence of the user, and when the data indicating that the user exists is not input for a predetermined time, that is, If the user has not returned to the terminal, the fact is notified to the processing unit 1002, and the processing unit 1002 receives this and initializes the processing up to that point.

Thus, while the user is temporarily interrupting the operation, the processing unit 10
A malfunction in which the process of 02 is initialized can be prevented.

FIG. 10 shows the operation procedure in this device.

Step 1: Timer section 1006 is a human detection section
When the user presence data is input from 1005, step 2: check whether the data indicates the existence of the user, and if there is a user, step 3: initialize the timer, and perform the next use. Wait for input of user presence data.

In step 2, if the user presence / absence data indicates that there is no user, step 4: if the timer is not operating, step 5: start the timer.

Step 6: When the operation of the timer continues and the predetermined time is exceeded, Step 7: The timer unit 1006 outputs an initialization command to the processing unit 1002.

As described above, in the unmanned terminal device of the fourth embodiment, even if the input operation is interrupted, the process is not initialized unless the user leaves the terminal. In an unattended terminal device used by various people, a user who is unfamiliar with the operation may pause the input operation and look at the displayed information. Initialization can be prevented.

(Fifth Embodiment) In the unmanned terminal device of the fifth embodiment, as shown in FIG. 11, a pressure sensor unit 1204 detects whether or not a user is in front of the device. Other configurations are the same as those of the device of the fourth embodiment (FIG. 9). The pressure sensor is placed, for example, in front of the terminal device, and when a user performing an input operation gets on the terminal, the press sensor is detected, and user presence / absence data indicating the presence of the user is detected by the timer unit 1205.
And when the user moves away from the terminal,
The user presence / absence data indicating the absence of the user is output to the timer unit 1205. Other operations are the same as those of the device of the fourth embodiment.

FIG. 12 shows the operation procedure of this apparatus. This procedure is substantially the same as that of the fourth embodiment (FIG. 10).

(Sixth Embodiment) In the unmanned terminal device according to the sixth embodiment, as shown in FIG. 13, a user in front of the device is detected by an infrared sensor unit 1404. Other configurations are 4th
There is no difference from the apparatus of the embodiment (FIG. 9). When the user stands in front of the terminal device, the infrared sensor detects infrared rays emitted from the user, detects the presence of the user performing the input operation, and sends the user presence / absence data to inform the timer unit 1405 of the presence / absence data. Output. When the user moves away from the terminal device, the timer unit 1405 outputs user presence / absence data indicating the absence of the user. Other operations are the same as those of the device of the fourth embodiment.

FIG. 14 shows the operation procedure of this apparatus. This procedure is substantially the same as that of the fourth embodiment (FIG. 10).

[0051]

As is apparent from the above description of the embodiment, the unmanned terminal device of the present invention accurately determines whether or not the user needs help information from the user 's facial expression. Help information can be automatically output.

In a device that allows the user to input personal information, the help information can be displayed in a display form suitable for the user.

[0053]

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an unmanned terminal device according to a first embodiment of the present invention;

FIG. 2 is an example of a gaze movement pattern database in the unmanned terminal device according to the first embodiment;

FIG. 3 is a flowchart illustrating an operation procedure of the unattended terminal device according to the first embodiment;

FIG. 4 is a block diagram showing a configuration of an unmanned terminal device according to a second embodiment of the present invention;

FIG. 5 shows an example of an expression pattern database in the unattended terminal device according to the second embodiment;

FIG. 6 is a flowchart illustrating an operation procedure of the unattended terminal device according to the second embodiment;

FIG. 7 is a block diagram showing a configuration of an unmanned terminal device according to a third embodiment of the present invention;

FIG. 8 is an example of a correspondence table between personal information and help expression method information in the unattended terminal device according to the third embodiment;

FIG. 9 is a block diagram showing a configuration of an unmanned terminal device according to a fourth embodiment of the present invention;

FIG. 10 is a flowchart illustrating an operation procedure of the unmanned terminal device according to the fourth embodiment;

FIG. 11 is a block diagram showing a configuration of an unmanned terminal device according to a fifth embodiment of the present invention;

FIG. 12 is a flowchart illustrating an operation procedure of the unmanned terminal device according to the fifth embodiment;

FIG. 13 is a block diagram showing a configuration of an unmanned terminal device according to a sixth embodiment of the present invention;

FIG. 14 is a flowchart illustrating an operation procedure of the unmanned terminal device according to the sixth embodiment;

FIG. 15 is a block diagram showing a configuration of a conventional unmanned terminal device;

FIG. 16 is a block diagram showing the configuration of another conventional unmanned terminal device;

FIG. 17 is a block diagram showing a configuration of a third conventional unmanned terminal device.

[Explanation of symbols]

101, 401, 701, 1001, 1201, 1401, 3001, 3101, 3201
Input unit 102, 402, 702, 1002, 1202, 1402, 3002, 3102, 3202
Processing unit 103, 403, 703, 1003, 1203, 1403, 3003, 3103 Output unit 104, 404, 704 Help processing unit 105, 404, 705 Help database unit 106, 406, 1004 Image input unit 107 Eye gaze detection unit 108 Eye movement Detection unit 109 Eye movement pattern database unit 110, 409 Judgment unit 407 Expression detection unit 408 Expression pattern database 706 Personal information input unit 707 Personal information analysis unit 1005 Person detection unit 1204 Pressure sensor unit 1404 Infrared sensor unit 1006, 1205, 1405, 3106, 3204 Timer section 3004, 3104 Help output section 3203, 3105 Input monitoring section

Claims (2)

(57) [Claims] 1. An input means for a user to perform an input operation;
In an unmanned terminal device including processing means for performing a process according to an input operation and output means for outputting a processing result to a user, a facial expression is formed from image data showing the user 's face.
A user situation detecting means for extracting facial expression pattern data by extracting an element; a help necessity determining means for determining whether the user needs help based on the detected facial expression pattern data; and the help necessity determination. An unattended terminal device comprising: help output means for outputting help according to the processing status of the processing means when the means determines that help is required. 2. An input means by which a user performs an input operation;
Processing means for performing processing according to the input operation, and the processing result
Terminal device comprising output means for outputting a message to a user
In, personal information to enter a physical handicap information of the user
Information input means and a helper adapted to the input physical handicap information.
Personal information analyzing means for instructing a method of expressing a help, and a help display method instructed by the personal information analyzing means
And a help output means for outputting help
Unattended terminal device and butterflies.