JP2002099386A - Image display control system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately change screen displays in accordance with the movement of the line of sight. SOLUTION: An image signal from a camera 10 is supplied to a line of sight detecting part 16. The part 16 detects the direction of the line of sight from the image, and a screen display controlling part 18 detects a line of sight position on the screen of a display device 14. When the line of sight comes to the end of the screen, an arrow generating part 22 displays a scroll arrow at the end of the screen. When the line of sight moves outward from the screen after the arrow is displayed, scrolling is performed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a screen display control system, and more particularly to a system for controlling a screen display in accordance with a line of sight.

[0002]

2. Description of the Related Art Conventionally, in information processing apparatuses such as word processors, personal computers, and workstations,
A display screen is used as a means for transmitting information to a user. If this display screen is enlarged, a large amount of information can be displayed. However, the size of the screen is limited for various reasons, and the amount of information is basically not limited. Cannot be displayed on the screen.

Therefore, in order to display all the information to be displayed on the screen, the screen is scrolled or switched for display. And such scrolls,
In order to switch the screen, the user has to perform some operation each time.

Japanese Patent Application Laid-Open No. Hei 8-22385 proposes a gaze-corresponding display device that detects a movement of a gaze of a user and scrolls a display screen. According to this device, the user can scroll the screen by moving his / her line of sight, and there is an advantage that it is not necessary to operate the device each time.

[0005]

However, in the above conventional example, the moving speed of the line of sight is detected, and the screen is basically scrolled when the moving speed is higher than a certain specific speed. However, when the screen is scrolled in this manner, there is a problem that not only the user wants to scroll but also an unintended scroll is performed.

SUMMARY OF THE INVENTION The present invention has been made in consideration of the above problems, and has as its object to provide a screen display control system capable of appropriately changing a screen display in accordance with a movement of a line of sight.

[0007]

SUMMARY OF THE INVENTION The present invention relates to a screen display control system for controlling a screen display, comprising: an image pickup means for picking up an image of a user's face; Gaze detecting means for detecting the gaze, and notice display means for displaying a notice display on the screen informing that the screen display is to be changed in accordance with the output of the gaze detection means, and a gaze trend after the notice display Change the screen display based on.

Prior to a change in screen display such as scrolling, a notice is displayed. Then, the screen is changed only when a predetermined line of sight is moved after the notice.
Therefore, it is possible to confirm the user's intention and perform a correct operation while maintaining the advantage that the screen display can be changed by the line of sight.

Further, according to the output of the line-of-sight detecting means,
It is preferable to control screen scrolling. Thus, scrolling can be performed without performing an operation with a hand or the like.

[0010] When the line of sight is located at the edge of the screen, it is preferable to display a scroll arrow as a notice display. Thereby, the user can understand the content of the scroll in the direction of the arrow.

Further, according to the output of the line-of-sight detecting means,
It is preferable that a notice of a click be displayed while the cursor is moving and a predetermined item is selected by the cursor. As a result, an item can be selected by moving the line of sight, and a notice of a screen change is displayed. Therefore, a click can be made by the movement of the line of sight after the notice.

It is also preferable that the screen display is changed when the line of sight moves out of the screen after the advance notice is displayed.

[0013]

Embodiments of the present invention (hereinafter referred to as embodiments) will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the overall configuration of the system according to the embodiment. The system is, for example, a portable computer, and a camera 1 such as a CCD camera.
0, a control unit 12 for performing various data processing and a display device 14 including an LCD and the like are integrally configured.
In particular, the camera 10 is fixed near the display device, and the position and imaging direction of the camera 10 and the display device 14
Is known in advance. Note that the camera 10
If the relationship between the display device 14 and the display device 14 is known, it is not always necessary to configure the display device 14 integrally. Note that the camera 10 is usually a CCD camera, but may be any other camera as long as it can output imaging information as an electric signal. The control unit 12 is composed of a computer.

The camera 10 captures an image of the user's face, and the image signal is supplied to a line-of-sight detection unit 16 of the control unit 12. The line-of-sight detection unit 16 extracts an eye part in the image, detects the position of the pupil in the eye part, and detects the direction of the line of sight. Further, as described above, since the screen position of the display device 14, the position of the camera 10, and the imaging direction are known, which position on the screen of the display device 14 the user's line of sight sees from the detected user's line of sight. Is detected. The gaze detection is described in JP-A-8-22385.
Various techniques can be adopted as described in the publication.

The line-of-sight detection unit 16 supplies the line-of-sight detection result to the screen display control unit 18. The screen display control unit 18 writes image data supplied based on various processing results (not shown) to the display memory 20 as display data of one screen. Then, data for one screen of the display memory 20 is sequentially read out and displayed on the display device 14.

That is, the screen display control unit 18 displays the characters input by the word processing function on the screen, writes the image data of the Internet homepage obtained by the communication function into the display memory 20, and displays the screen data on the display device 14. Control.

The screen display control unit 18 determines which part of the screen of the display device 14 the gaze is pointing to based on the detection result of the gaze detection unit 16. For example, when it is determined that scrolling is necessary, , And supplies the instruction to the arrow generator 22. As a result, the arrow generator 22 generates a predetermined arrow image signal, which is displayed on the display memory 20.
Write to. Thus, an arrow is displayed on the screen of the display device 14.

Here, the scroll function performed according to the line of sight detection in the apparatus of the present embodiment will be described.
The scroll function is turned on / off by a predetermined setting operation, and the following description is an operation when the scroll function is turned on.

First, when a document is displayed as shown in FIG. 2, the line-of-sight detection unit 16 constantly monitors the line-of-sight direction. Note that a cursor indicating the line-of-sight position may be constantly displayed on the screen.

Then, when the line of sight comes to the lower end of the screen,
As shown in FIG. 3, a scroll arrow is displayed at the lower end of the screen. That is, the screen display control unit 18 is
2, and a downward arrow is displayed at the lower end of the screen. Then, when it is detected that the line of sight has moved out from the lower side of the screen, the lower scroll is performed. Therefore, the user
After the arrow appears, the screen can be consciously scrolled by shifting the line of sight greatly to the outside of the screen.

When it is detected that the line of sight has reached the upper end of the screen, an upward arrow is displayed at the upper end of the screen as shown in FIG. Then, when the line of sight goes outside from the upper side of the screen, the upper scroll is performed.

When the direction of the line of sight is returned to the inside when the scroll arrow is displayed, the arrow is erased and scrolling is not performed. Therefore, the scroll operation is not performed unless the user performs a conscious operation of shifting the line of sight out of the screen when the arrow is displayed. Therefore, it is possible to prevent scrolling that is not intended by the user while maintaining the advantage that scrolling can be performed only with the line of sight.

The horizontal scrolling is performed by displaying an arrow pointing outward in the horizontal direction when the line of sight is located at the left or right end, and scrolling when the line of sight is shifted to the outside of the screen in the direction of the arrow. The same can be done.

Next, regarding the click operation based on the line of sight,
explain. A click can be performed when scrolling in the left-right direction by the line of sight is not performed. This is made possible by setting in advance the setting for clicking.

As shown in FIG. 5, there are various displays,
When there is an item with a link to be clicked, the screen display control unit 18 is a target of the gaze when the line of sight is at the right end or the left end based on the result of the line of sight detection. The item is highlighted and flashed. It should be noted that the inverted blinking display may be all of the items or only the left box indicating that the item is clicked. Then, in this state, when the user shifts his / her line of sight from the left / right direction to the outside of the screen, the screen display control unit 18 determines that the click has been made, and changes the display to the linked screen. In this case, as in the case of the above-described scrolling, an arrow indicating that a click is made by moving the line of sight may be displayed. When the line of sight is at the center, it is preferable that the cursor be moved in accordance with the movement of the line of sight.

Here, the above operation will be described with reference to the flowchart of FIG. First, the user sets whether to use the left-right scroll function or the click function.

The screen display control unit 18 determines whether or not the line of sight is located at the edge of the screen based on the test detection result from the line of sight detection unit 16 (S11). If the determination is YES, it is determined whether the end is a horizontal end or a vertical end (S12). If it is in the horizontal direction, it is determined whether the setting made in advance is horizontal scrolling or clicking (S1).
3). If the setting is vertical in S12 or scroll is set in S13, a scroll arrow pointing outward is displayed according to the detected end (S14). Thus, the user is notified of scrolling. Then, while the scroll is displayed, it is determined whether the line of sight has moved to the outside of the screen along the direction of the arrow (S1).
5). If the determination is YES, it is recognized that the user has instructed scrolling, and the screen is scrolled in that direction (S16).

If the line of sight has not moved outward in S15, it is determined whether or not the line of sight has moved inward (S15).
17). If it has moved inward, it is determined that the user's intention is not scrolling, and the arrow is erased (S1).
8), end the process.

If a click is set in S13, the selection item corresponding to the line of sight at that time is displayed in reverse blinking (S19). Thus, the user is notified of the click. Then, in this state, it is determined whether the line of sight has moved to the outside of the screen (S20). If the user's line of sight moves outside in the state where the notice is given, it is determined that the user intends to click, the click is taken, and the arrow is erased (S21). On the other hand, S2
If NO at 0, it is determined whether the line of sight has moved inward (S22), and if YES, the reverse blinking display is stopped (S23) and the process ends. On the other hand, in S22, NO
If so, the process returns to the determination of whether the line of sight has moved outward in S20.

As described above, scrolling is performed only when the user shifts his / her line of sight out of the screen while the scroll arrow is displayed. The click is also performed when the user shifts his / her line of sight out of the screen during the reverse blinking display. Therefore, scrolling or clicking is performed after confirming the user's intention, and scrolling or clicking not intended by the user can be prevented.

[0032]

As described above, according to the present embodiment, an advance notice is displayed prior to a change in screen display such as scrolling or clicking. Then, scrolling, clicking, and the like are performed only when a predetermined line of sight moves after the notice. Therefore, it is possible to confirm the user's intention and perform a correct operation while maintaining the merit that scrolling and clicking can be performed by the line of sight.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a system.

FIG. 2 is a diagram illustrating an example of a display.

FIG. 3 is a diagram showing a downward scroll arrow display.

FIG. 4 is a diagram showing an arrow display of upward scroll.

FIG. 5 is a diagram showing a display before clicking.

FIG. 6 is a flowchart showing an operation.

[Explanation of symbols]

10 CCD camera, 12 control unit, 14 display device,
16 gaze detection unit, 18 screen display control unit, 20 display memory, 22 arrow generation unit.

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (Reference) G09G 5/08 G09G 5/08 S 5/34 5/34 MF Term (Reference) 5B087 AA09 AB14 BC05 BC12 BC13 BC26 BC32 DD03 DE03 DE06 DE07 5C082 AA15 AA21 BA02 BA12 CA72 CA76 CB05 DA86 MM02 MM09 5E501 AC37 BA02 CA03 CC08 EA01 FA02 FA09 FB32 FB44

Claims (5)

[Claims] 1. A screen display control system for controlling a screen display, comprising: an imaging unit that captures an image of a user's face; and a gaze detection unit that detects a gaze of a user from an image obtained by the imaging unit. Notice display means for displaying a notice display on the screen informing that the screen display is to be changed in accordance with the output of the gaze detection means, and changing the screen display based on the movement of the gaze after the notice display. Screen display control system. 2. The screen display control system according to claim 1, wherein a screen scroll is controlled in accordance with an output of said line-of-sight detection means. 3. The screen display control system according to claim 2, wherein a scroll arrow is displayed as an advance notice when the line of sight is located at an end of the screen. 4. The system according to claim 1, wherein a cursor moves in accordance with an output of the line-of-sight detection means.
A screen display control system for displaying an advance notice of a click when a predetermined item is selected by a cursor. 5. The screen display control system according to claim 3, wherein the screen display is changed when the line of sight moves out of the screen after the advance notice is displayed.