JP2014219712A - Operation display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation display device capable of restricting a prescribed operation such as an execution instruction of a job or transition of a screen when a user skips the contents indicated by a confirmation screen displayed on a display part.SOLUTION: A confirmation screen for requiring a prescribed confirmation is displayed on a display part. Then, the line of sight of a user is detected to determine whether or not the user has confirmed the contents indicated by the confirmation screen making the eyes of the user follow the contents. While the confirmation is incomplete, the contents indicated by the confirmation screen is recognized as not having been read through, and the execution of a prescribed operation such as an execution of a job or transition of a screen is prohibited.

Description

  The present invention relates to an operation display device that determines whether a user has confirmed display contents.

  In recent years, many methods for determining whether or not a user has confirmed display contents have been proposed. For example, in Patent Document 1, an eye for gazing at a contract sentence displayed on a display unit is photographed by a camera unit to perform line-of-sight recognition, a gaze time for the contract sentence is measured, and a prepared lack of understanding determination is prepared. A method is described in which time and gaze time are compared, and whether or not the contractor has confirmed the contract is determined based on whether or not the gaze time is equal to or greater than the underunderstanding determination time (see Patent Document 1).

  Patent Document 2 discloses a method of determining whether a web advertisement is actually viewed and measuring the actual viewing time of the advertisement when the web advertisement is viewed (see Patent Document 2). .

  By the way, in MFPs (Multi Function Peripherals) such as image forming apparatuses having functions such as printers, scanners, and facsimiles, a confirmation screen showing warnings and cautions is displayed on the display unit, and the displayed contents are displayed. It is generally widespread that the user is confirmed.

  When the confirmation screen is displayed, upon receiving a confirmation completion report from the user (for example, when a confirmation completion button is pressed), various processes such as transition to the next screen and execution of a job are started. For example, when a highly confidential document is transmitted to an external device, before transmission, information indicating the transmission destination of the document (address, device information as a transmission destination, etc.) is displayed on the display unit as a confirmation screen, After allowing the user to confirm that there is no error in the transmission destination, an instruction to start transmission is accepted. In addition, when new software is installed, a license agreement of the software is displayed on the display unit as a confirmation screen, and the user can use the software after accepting consent to the contents of the contract.

JP 2010-39646 A JP 2010-61218 A Japanese Patent Laid-Open No. 11-24549

  In general, the user often presses the confirmation completion button without confirming the contents displayed on the confirmation screen one by one, that is, skipping the content displayed on the confirmation screen (without reading it almost). If the confirmation screen is skipped, the user loses the opportunity to confirm the skipped content, and operates the MFP without grasping the content. Then, if the MFP is operated without grasping the contents (warnings and cautions) shown on the confirmation screen, problems often occur later. Therefore, it is desirable to prohibit processing such as job execution and screen transition until the user recognizes that the content shown on the confirmation screen has been confirmed (read).

  In the invention described in Patent Document 1, a criterion for determining how much time is required for the contractor to understand the contract is set, and the next contract is displayed according to the criterion. It is determined whether to prompt the rereading of the contract text being displayed. Therefore, even if the user does not actually confirm (read) the contents of the contract text, it can be skipped if the gazing time for the contract text is longer than a certain value. Also, since the invention described in Patent Document 2 also measures the viewing time of an advertisement, it is possible to determine whether or not the advertisement has been viewed, but whether or not the user has grasped the content indicated by the advertisement. It cannot be determined.

  In Patent Document 3, there is an extraction means for extracting a word / phrase from a document, and a detection means for detecting the reader's line of sight to see the displayed sentence. Whether or not the line of sight is not present at the extracted word / phrase display position is disclosed. In the method described in Patent Document 3, it is possible to recognize whether or not the line of sight is directed to each segment (word) in the document, but only a warning is given even if it is determined that the reading is skipped. The user can ignore the warning and proceed with the operation.

  The present invention is intended to solve the above-described problem, and detects whether or not the user skips the content displayed on the confirmation screen displayed on the display unit with high accuracy. An object of the present invention is to provide an operation display device that does not perform processing such as execution instructions and screen transitions.

  The gist of the present invention for achieving the object lies in the inventions of the following items.

[1] An operation unit that receives an operation from a user;
A display unit for displaying a screen corresponding to the operation received by the operation unit;
A line-of-sight detection unit for detecting the line of sight of the user;
A control unit that determines whether the user visually confirms the display content of the screen based on a detection result of the line-of-sight detection unit, and prohibits execution of a predetermined operation while the confirmation is incomplete When,
An operation display device characterized by comprising:

  In the said invention, a user's eyes | visual_axis is detected and it is determined whether the user confirmed the display content of the display part visually based on the detection result. While this confirmation is not completed, execution of a predetermined operation is prohibited. Thus, it is possible to prevent the start of a predetermined operation until it is recognized that the display content on the display unit has been read. The predetermined operation is, for example, an operation for instructing a predetermined device to start a job, an operation for instructing to resume a job, an operation for changing a screen displayed on the display unit, or a predetermined operation button displayed on the display unit. And the like.

[2] The predetermined operation includes an operation for instructing a predetermined device to start a job, an operation for instructing the device to resume a job, an operation for changing a screen displayed on the display unit, a predetermined operation The operation display device according to [1], including any one of operations for displaying an operation button on the display unit.

  In the above invention, until it is recognized that the display content of the display unit has been read, the start (resumption) of the job in the predetermined device, the transition of the screen displayed on the display unit, and the display of the predetermined button on the display unit Prohibit etc.

[3] The control unit displays a predetermined confirmation screen when a setting of a transmission job to be executed by a predetermined device is received by the operation unit, and the content of the setting includes a predetermined confirmation target item. [1] or [2], wherein the determination is performed on the confirmation screen displayed on the display unit, and execution of the transmission job is prohibited while the confirmation is incomplete. Operation display device.

  In the above invention, for example, when transmitting highly confidential data or when a transmission destination that is not suitable as a destination is designated, a predetermined confirmation screen is displayed on the display unit, and the content indicated by the confirmation screen is displayed. The transmission of the data is prohibited until the user finishes reading, and the transmission prohibition of the data is canceled when the reading is completed.

[4] The control unit performs the determination only on a specific part of the screen displayed on the display unit. Any one of [1] to [3] The operation display device described in 1.

  In the above-mentioned invention, it is not determined whether or not the user has confirmed with eyes the portion where there is no problem even if skipping. Since the determination is performed only at a specific location that the user wants to check, the processing load is reduced. Further, the user can skip parts other than the specific part.

[5] The operation display device according to [4], wherein the specific portion is a portion having an importance level higher than a predetermined value in the screen.

[6] The operation display device according to [4] or [5], wherein the specific portion is a portion in which any one of time limit information, address information, and money amount information is displayed.

[7] When the control unit receives a request for starting the predetermined operation from the user while the confirmation is not completed, the control unit emphasizes a part of the specific portion where the user's line of sight is not yet directed. The operation display device according to any one of [4] to [6], wherein the operation display device is displayed.

  In the above invention, when a request for starting a predetermined operation is received from the user even though the confirmation has not been completed, a portion of the specific portion where the user's line of sight is not yet directed is highlighted. Thereby, the user can visually recognize the unread portion in the specific portion.

[8] The operation display device according to any one of [5] to [7], wherein the importance of each part can be changed in the screen.

[9] The operation display device according to [8], wherein the importance of each part can be set for each user.

  In the said invention, the importance of each location in a screen can be set according to a user. For example, even in the same place, it can be set so that the user A cannot skip reading and the user B can skip reading.

  According to the operation display device according to the present invention, it is detected with high accuracy whether or not the user has skipped the content indicated on the confirmation screen displayed on the display unit. It is possible to control not to perform processing such as transition.

It is explanatory drawing which shows the image forming apparatus which concerns on embodiment of this invention. 1 is an explanatory diagram illustrating a state in which an image forming apparatus according to an embodiment of the present invention is connected to an external device via a network. 1 is a block diagram illustrating a schematic configuration of an image forming apparatus according to an embodiment of the present invention. It is a figure which shows the example of the content of the license agreement displayed on an operation panel. It is a figure which shows the example of the screen displayed when a user completes the content which the confirmation screen shows. 6 is a flowchart illustrating an outline of a read completion determination process performed when the image forming apparatus displays a confirmation screen. It is a figure which shows the order which a user gazes, when confirming the content which a confirmation screen shows. It is a figure which shows a mode that each content which a confirmation screen shows, and the display position on a display part are matched when displaying a confirmation screen on a display part. It is a figure which shows the positional relationship of a camera, an eyeball, and an operation panel. It is a figure which shows an eyeball, a gaze, a Purkinje image, and a light ray. It is a flowchart which shows the process performed when performing a gaze detection. It is a figure which shows the structure of a determination table. It is a figure which shows the structure of a user judgment table. It is a figure which shows the structure of a determination result table. It is a figure which shows an example of the determination table. It is a flowchart which shows the rewriting process of a determination result table. It is a flowchart which shows the continuation of the process of FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows an image forming apparatus 10 that performs both a function as an operation display apparatus according to an embodiment of the present invention and a function as an apparatus that executes a job. The image forming apparatus 10 is a copy job that optically reads a document and prints a duplicate image on a recording sheet, a scan job that saves image data of the read document as a file, or transmits the image data to an external device, and a PC ( Personal Computer) a so-called multi-function machine having a function of executing a job such as a print job for printing an image relating to data sent from a terminal 40 (see FIG. 2) on a recording sheet. The image forming apparatus 10 is connected to an external apparatus (PC terminal 40) via a LAN (Local Area Network) 3 as shown in FIG.

  In the image forming apparatus 10 according to the embodiment of the present invention, an operation panel 30 including a display unit 31 and an operation unit 32 and operating the main body of the image forming apparatus 10 is installed. A user uses the operation panel 30. Thus, it is possible to instruct various settings and job execution (see FIG. 3). A screen corresponding to the operation received by the operation unit 32 is displayed on the display unit 31. For example, when a setting operation is accepted, the screen content being displayed changes according to the set content, or the screen changes in response to an OK button operation. In the embodiment, the image forming apparatus 10 displays a confirmation screen showing text such as warnings and cautions on the display unit 31, causes the user to confirm the contents, and receives a confirmation completion report (for example, confirmation When a button indicating completion is pressed), processing such as transition to the next screen and execution of a job is started. Note that the content of the confirmation screen is not limited to text, but may be displayed as a graphic, an image, or the like, but is assumed to be text in the present embodiment.

  A line-of-sight detection unit 22 that detects the line of sight of a user watching the screen displayed on the operation panel 30 is installed on the upper part of the operation panel 30. The image forming apparatus 10 determines from the detection result detected by the line-of-sight detection unit 22 whether or not the user has confirmed the content indicated by the confirmation screen displayed on the display unit by eye. While the confirmation is incomplete (while the content shown on the confirmation screen is not read), execution of a predetermined operation such as job start (resume) or screen transition displayed on the display unit is prohibited. Accordingly, it is possible to prevent the user from starting a predetermined operation while skipping the content indicated on the confirmation screen.

  Next, the configuration of the image forming apparatus 10 will be described.

  FIG. 3 is a block diagram showing a schematic configuration of the image forming apparatus 10 according to the embodiment of the present invention. The image forming apparatus 10 includes a CPU (Central Processing Unit) 11 that comprehensively controls the operation of the image forming apparatus 10. The CPU 11 has a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a nonvolatile memory 14, a hard disk device 15, an image processing unit 16, a scanner unit 17, a printer unit 18, and a facsimile via a bus. A communication unit 19, a network communication unit 20, an authentication unit 21, a line-of-sight detection unit 22, and an operation panel 30 are provided.

  The CPU 11 is based on an OS (Operating System) program, and executes middleware, application programs, and the like. Various programs are stored in the ROM 12, and each function of the image forming apparatus 10 is realized by the CPU 11 executing various processes according to these programs. The ROM 12 stores a program for the CPU 11 to execute a series of controls of the image forming apparatus 10.

  The RAM 13 is used as a work memory for temporarily storing various data when the CPU 11 executes processing based on a program, an image memory for storing image data, and the like.

  The nonvolatile memory 14 is a memory (flash memory) whose stored contents are not destroyed even when the power is turned off, and is used for storing various setting information. The hard disk device 15 is a large-capacity nonvolatile storage device, and includes an OS program, various application programs (software), user information, print data, and image data, as well as a determination table 100, a user determination table 101, and a determination result described later. A table 102 and the like are stored.

  The image processing unit 16 performs processing such as image enlargement / reduction, rotation, rasterization processing for converting print data into image data, image data compression / decompression processing, and the like.

  The scanner unit 17 performs a function of optically reading a document and acquiring image data. The scanner unit 17 sequentially moves, for example, a light source that irradiates light to a document, a line image sensor that receives the reflected light for one line in the width direction, and a reading position in line units in the length direction of the document. An optical path composed of a moving unit, a lens or a mirror that guides the reflected light from the document to the line image sensor and forms an image, a conversion unit that converts the analog image signal output from the line image sensor into digital image data, etc. It is configured with.

  The printer unit 18 has a function of forming an image corresponding to image data on a recording sheet. Here, it has a recording paper transport device, a photosensitive drum, a charging device, a laser unit, a developing device, a transfer separation device, a cleaning device, and a fixing device, and forms an image by an electrophotographic process. It is configured as a so-called laser printer. Other methods may be used for image formation.

  The facsimile communication unit 19 performs a function of transmitting / receiving image data to / from an external device having a facsimile function through a public line.

  The network communication unit 20 performs a function of communicating data with an external device such as the PC terminal 40 through a network such as the LAN 3. For example, the image forming apparatus 10 can simultaneously transmit image data to a plurality of destinations (PC terminals 40).

  The authentication unit 21 performs a function of performing user authentication. Specifically, user authentication is performed when a user logs in, and a login user is specified. In the present embodiment, it is assumed that login is required to use the image forming apparatus 10 and user authentication is always performed before use. The user authentication method may be any suitable method such as user ID (Identification) and password authentication or fingerprint authentication.

  The line-of-sight detection unit 22 includes a detection camera 23 and an irradiation unit 24, and detects the line of sight of the user watching the screen displayed on the operation panel 30. In the embodiment, the irradiation unit 24 irradiates the user's eyeball with near-infrared rays of a point light source, and captures and uses a reflection image (hereinafter referred to as a Purkinje image) on the cornea surface of the user's eyeball. By doing so, the user's line of sight is calculated. A detailed detection method will be described later. Note that another method may be used as a method for detecting the user's line of sight.

  The operation panel 30 includes a display unit 31 and an operation unit 32. Among these, the operation unit 32 includes a switch unit 33 such as a start button and a touch panel unit 34. The display unit 31 is composed of a liquid crystal display (LCD) and the like, and has a function of displaying various operation screens, setting screens, and the like. The CPU 11 performs control related to display contents displayed on the operation panel 30 and control related to reception of operations by the operation panel 30.

  The touch panel unit 34 is provided on the display unit 31. The touch panel unit 34 detects a touch position (coordinate position) on the display unit 31 pressed by a touch pen or a finger, a flick operation, a drag operation, and the like. Further, the CPU 11 recognizes handwritten input by the user from the continuously detected change in touch position (operation information).

  Next, a confirmation screen showing a warning or a note or other text will be described with a specific example.

  FIG. 4 shows a license agreement screen 90 which is an example of a confirmation screen displayed on the display unit 31. The license agreement screen 90 is a confirmation screen displayed on the display unit 31 when new software is installed, and shows the content of the contract related to the license of the software. When the image forming apparatus 10 receives an operation indicating consent to the contract contents from the user, the image forming apparatus 10 transitions the display screen to the next screen and enables the software.

  When the image forming apparatus 10 detects the user's line of sight with respect to the license agreement screen 90 and determines that the user has visually confirmed the content of the license agreement screen 90, the image forming apparatus 10 skips the content of the license agreement screen 90. If it is recognized that the reading has been completed, the above-mentioned operation indicating consent is accepted. For example, the aforementioned operation indicating consent is invalidated until the recognition is performed. You may make it validate the operation which shows the above-mentioned consent received before the said recognition is performed, when the said recognition is performed.

  The image forming apparatus 10 displays a predetermined confirmation screen on the display unit 31 when the setting content of the transmission job includes a predetermined confirmation target item, and whether or not the user has confirmed the confirmation screen with his eyes. judge. The items to be confirmed include, for example, that the data to be transmitted is highly confidential document data and that a predetermined transmission destination is included in the destination.

  FIG. 5 shows a destination confirmation screen 91 which is an example of a confirmation screen displayed on the display unit 31 before executing the transmission job. The destination confirmation screen 91 is a confirmation screen that is displayed on the display unit 31 before transmission of highly confidential document data to an external device, and indicates information indicating the transmission destination of the data. For example, when sending by e-mail, the destination address is displayed, and when sending by facsimile, the fax number is displayed. The user presses the decision button 92 if there is no error in the information indicating the transmission destination in the destination confirmation screen 91, and presses the cancel button 93 if there is an error. The decision button 92 and the cancel button 93 are display buttons displayed in the destination confirmation screen 91. When the determination button 92 is pressed, the image forming apparatus 10 transmits the above-described data to the transmission destination indicated by the destination confirmation screen 91, and when the cancel button 93 is pressed, the transmission processing ends without transmitting the above-described document. To do.

  On the destination confirmation screen 91, until the user finishes reading the information related to the transmission destination (inside the solid line in the figure), that is, until the user's line of sight is detected and it is determined that the information has been checked by eye (reading completed). Until it is recognized, the determination button 92 and the cancel button 93 (the portion within the broken line in the figure) are not displayed. When it is determined that the user has checked the information with his / her eyes, a determination button 92 and a cancel button 93 (the portion within the broken line in the figure) are displayed. As a result, the user cannot press the enter button 92 or the cancel button 93 without reading the information related to the transmission destination. That is, the image forming apparatus 10 prohibits the execution of the transmission job until the user finishes confirming the information related to the transmission destination.

  In FIG. 6, the image forming apparatus 10 displays a confirmation screen on the display unit 31 and then starts a predetermined operation such as transition to the next screen, start (restart) of a job, display of a predetermined operation button, or the like. The outline of the process performed until now is shown.

  First, the image forming apparatus 10 displays a confirmation screen on the display unit 31 (step S101). Next, a determination process is performed as to whether or not the user has confirmed the contents shown on the confirmation screen with his / her eyes by detecting the line of sight (step S102).

  The process of step S102 is continued until it is determined that the user confirms the contents shown on the confirmation screen (reading completed) (step S103; No), and if it is determined that the confirmation is completed (step S103; Yes). Then, a predetermined operation such as transition to the next screen or execution of a job is started (step S104), and the process ends.

  As described above, the image forming apparatus 10 prohibits execution of a predetermined operation such as transition to the next screen or execution of a job until the user finishes confirming the contents shown on the confirmation screen. Accordingly, it is possible to prevent the user from starting a predetermined operation while skipping the content indicated on the confirmation screen.

  FIG. 7 shows an example of a state in which the position where the line of sight is directed moves when the user reads the content shown on the confirmation screen 95 displayed on the display unit 31. Since the size of the confirmation screen 95 is larger than the display area of the display unit 31, the display unit 31 cannot display the entire confirmation screen 95 at a time. In such a case, a part of the confirmation screen 95 is displayed on the display unit 31, and the user can confirm the entire confirmation screen 95 by scrolling the screen.

  In FIG. 7, six lines of text from the top of the confirmation screen 95 are displayed on the display unit 31. The user reads each line from top to bottom from left to right. Then, for example, after reading six lines of sentences displayed on the display unit 31, the screen is scrolled using the scroll bar 94 to display unread sentences, and the continuation is read.

  Next, a method for calculating which part of the confirmation screen the line of sight detected by the line-of-sight detection unit 22 is directed to will be described.

  The following calculation is performed by the CPU 11.

  FIG. 8 shows a procedure for calculating which part of the confirmation screen 95 the line of sight directed toward the display unit 31 is displayed after the same confirmation screen 95 as in FIG. 7 is displayed on the display unit 31. The outline of is shown.

  When the size of the confirmation screen is large and cannot be displayed on the display unit 31 at once, a part of the confirmation screen is displayed on the display unit 31. The CPU 11 performs projection conversion using a conversion table stored in the hard disk device 15, and determines which part in the confirmation screen is displayed at which position on the display unit 31.

  When the confirmation screen 95 is displayed on the display unit 31 according to the position determined by the above-described projection conversion, the line-of-sight detection unit 22 detects the user's line of sight. Then, the CPU 11 calculates to which position on the display unit 31 the detected line of sight is directed. On the display unit 31, the position where the line of sight is directed (coordinates in the screen coordinate system) is indicated by two-dimensional coordinates.

  Next, the CPU 11 calculates to which part in the confirmation screen 95 the position (the coordinates of the screen coordinate system) where the line of sight is directed on the display unit 31 corresponds. Specifically, coordinate conversion is performed on the coordinates in the screen coordinate system, and coordinates in the confirmation screen 95 (coordinates in the assigned coordinate system) are calculated. Coordinates in the confirmation screen 95 (coordinates in the assigned coordinate system) are also shown as two-dimensional coordinates.

  Thereafter, the calculated coordinates of the assigned coordinate system are compared with the determination table 100 in which the contents and the like in the confirmation screen 95 are registered, and the determination result table 102 corresponding to the determination table 100 is displayed in each of the confirmation screens 95. A determination result as to whether or not the sentence (each character) has been confirmed by eye is registered. The CPU 11 determines based on the registered determination result whether or not the entire content (sentence) shown on the confirmation screen 95 (or a part of specific parts) has been confirmed by eye (whether or not it has been read). . Details of the determination table 100 and the determination result table 102 will be described later.

  Next, a method for detecting the line of sight will be described with reference to FIGS.

  FIG. 9 shows the positional relationship among the detection camera 23, the user's eyeball 120, and the display unit 31. Position coordinates shown with the eyeball 120 as a reference are set as coordinates in the user coordinate system, and position coordinates shown with the detection camera 23 as a reference are set as coordinates in the camera coordinate system. Each coordinate in the user coordinate system and the camera coordinate system is indicated by a three-dimensional coordinate.

  Note that the coordinates in the image shown with the image captured by the detection camera 23 as a reference are the coordinates of the captured image coordinate system. The captured image coordinate system is indicated by two-dimensional coordinates. The line-of-sight detection unit 22 detects the pupil and the Purkinje image shown in the image captured by the detection camera 23 and calculates the user's line of sight using the coordinates of the center position of the Purkinje image.

  FIG. 10 is a diagram showing an enlarged model of the eyeball 120. FIG. 10 shows a state in which a light beam 122 is irradiated from the irradiation unit 24 toward the corneal curvature center 125 of the eyeball 120, and a Purkinje image 124 as a reflection image is reflected on the cornea surface of the eyeball 120. Further, the line of sight connecting the corneal curvature center 125 and the pupil center 123 is defined as a line of sight 121.

  FIG. 11 shows the flow of processing when line-of-sight detection is performed. First, the distance between the camera for detection 23 and the eyeball 120 is measured, and a translation matrix for coordinate transformation is performed by translating the coordinates of the camera coordinate system to the coordinates of the user coordinate system (step S201).

  Next, a coordinate transformation matrix is applied to the position of the Purkinje image 124 that appears in the image captured by the detection camera 23 to create a Purkinje image position vector in the camera coordinate system (step S202).

  The parallel movement matrix calculated in step S201 is applied to the position vector in the camera coordinate system created in step S202 to calculate the Purkinje image position vector in the user coordinate system (step S203).

  Using the Purkinje image position vector in the camera coordinate system and the Purkinje image position in the user coordinate system, the position coordinates of the corneal curvature center 125 in the user coordinate system are calculated (step S204).

  Next, the position vector of the user coordinate system and the camera coordinate system at a plurality of sampling points on the pupil contour is calculated by the same calculation method as in step S202 to step S203, and this calculation result is used to calculate the user coordinate system. The position coordinates of the pupil center 123 are calculated (step S205).

  A line of sight 121 is calculated from a straight line passing through the position coordinate of the corneal curvature center 125 calculated in step S204 and the position coordinate of the pupil center 123 calculated in step S205 (step S206), and the process is terminated.

  When the line of sight is calculated, the CPU 11 calculates from the position coordinates of the eyeball 120 and the position coordinates of the operation panel 30 to which position on the display unit 31 the line of sight is directed, that is, the coordinates of the screen coordinate system. . When the coordinates of the screen coordinate system are calculated, the coordinates of the assigned coordinate system (coordinates in the confirmation screen) are calculated by coordinate conversion.

  Next, the determination table 100 and the determination result table 102 will be described.

  FIG. 12 shows the structure of the determination table 100. In the determination table 100, the contents of the confirmation screen are divided and registered into a plurality of blocks in sentence (sentence) units. For each block, the block number, the coordinates of the first character in the block (coordinates in the confirmation screen), the coordinates of the last character, the number of characters in the block, and the detailed coordinates in the block (the coordinates of each character constituting the block) , Importance is registered.

  In the embodiment, the image forming apparatus 10 can skip a block (a sentence (sentence) corresponding to the importance) having a predetermined importance level or less. A block whose importance is higher than a predetermined value, that is, a block that cannot be skipped (corresponding to a sentence) corresponds to a specific portion according to the present invention. The image forming apparatus 10 determines whether or not the user has confirmed with eyes with respect to only this specific portion in the confirmation screen.

  The importance is divided into three stages of A, B, and C (A> B> C,). A cannot be skipped (high importance), and B can be skipped by the user (medium importance). , C can be skipped (low importance). In the user determination table 101 of FIG. 13, whether or not a block having an importance level B can be skipped is registered for each user (1 when skipping is possible and 2 when skipping is impossible). The CPU 11 combines the determination table 100 and the user determination table 101 to determine blocks that can be skipped by the user who is operating the image forming apparatus 10.

  FIG. 14 shows the structure of the determination result table 102. In the determination result table 102, for each block registered in the determination table 100, a determination result as to whether or not each character in the block has been confirmed visually, and whether or not the confirmation has been completed in units of blocks. The determination result is registered. If the confirmation is not completed, 0 is registered. If the confirmation is completed, 1 is registered.

  After calculating the coordinates of the assigned coordinate system, the CPU 11 compares the coordinates with the detailed coordinates in the block registered in the determination table 100. In the detailed coordinates in block, the coordinates of the upper left end and the lower right end of each character in the block are registered. If there are coordinates in the assigned coordinate system within the range of the rectangle with these two points at the upper left end and the lower right end, the determination result of the determination result table 102 corresponding to that character is changed from incomplete to complete (from 0 to 1). rewrite. When 1 (confirmation, completion) is registered as the determination result for all characters in the block, the determination result for each block is rewritten from incomplete to complete (from 0 to 1).

  Note that the CPU 11 registers 1 in the determination result regardless of whether or not the block (and each character in the block) determined to be skipped is confirmed by eye. Here, 1 is registered as an initial value in a block that can be skipped. When all 1 is registered in the determination result of each block of the determination result table 102, the CPU 11 determines that all the contents (sentences) shown on the confirmation screen have been confirmed by eye (recognized as having been read).

  The determination table 100 and the determination result table 102 are created and used according to the content of the confirmation screen when the image forming apparatus 10 displays the confirmation screen on the display unit 31. If the contents of the confirmation screen are predetermined (for example, the license agreement screen 90 in FIG. 4), the determination table 100 and the determination result table 102 corresponding to the confirmation screen are stored in advance and read out. May be used.

  FIG. 15 shows a confirmation screen 96 (FIG. 15A) and a determination table 100A (FIG. 15B) created in accordance with the confirmation screen 96. The confirmation screen 96 describes the content for prompting confirmation of the data transmission destination, and is composed of eight lines of text (two lines from the top are explanatory texts, and the third and subsequent lines indicate transmission destination information). In the determination table 100A, eight blocks are registered with the text on each line of the confirmation screen 96 as one block.

  Importance level is set for each block. The blocks corresponding to the description part (blocks 1 and 2) have importance C, and the blocks corresponding to the destination information part (blocks 3 to 8) are important. The degree is set to A or B.

  In the image forming apparatus 10, the importance of each block is automatically determined based on the content of the sentence corresponding to that block. For example, the importance of a block corresponding to a sentence including deadline information, address (destination) information, money amount information, etc. is set higher. That is, the time limit information, address (transmission destination) information, money amount information, and the like are specific places. Note that the user may set the importance of each block individually, such as changing the importance for each address.

  16 and 17 show the flow of processing when the determination result of the determination result table 102 is rewritten. In the process of FIG. 16, the determination result is rewritten for blocks that can be skipped, and in the process of FIG. The processing shown in FIGS. 16 and 17 is executed when a confirmation screen is displayed on the display unit 31.

  In the processing of FIG. 16, when the confirmation screen is displayed on the display unit 31, the CPU 11 creates the determination table 100 and the determination result table 102 (or reads out pre-stored ones) and registers them in the determination result table 102. Each determination result is initialized (all (block and character) determination results are set to 0) (step S301).

  Next, referring to the user determination table 101, the importance level that can be skipped by the user who has logged in to the image forming apparatus 10 is checked (step S302).

  In subsequent steps S303 to S307, it is checked whether or not each block of the determination table 100 can be skipped. If it can be skipped, the determination result of that block is set to 1.

  Specifically, the data of block number 1 in the determination table 100 is referred to (step S303), and it is checked whether the block can be skipped (step S304).

  If skipping is impossible (step S304; No), the process proceeds to step S306. If reading can be skipped (step S304; Yes), the determination result of the block in the determination result table 102 is rewritten to 1 (confirmation, completion) (step S305).

  When there is a block in the determination table 100 that has not been checked whether or not skipping is possible (step S306; Yes), the next numbered block (for example, the block number 2 next to the block number 1). (Step S307), the process returns to step S304 to continue the processing. When it has been checked whether or not all blocks in the determination table 100 can be skipped (step S306; No), the process proceeds to step S308 in FIG.

  The process flow of FIG. 17 will now be described. First, the CPU 11 acquires the coordinates of the assigned coordinate system from the line of sight detected by the line-of-sight detection unit 22 (step S308). Next, which block of the determination table 100 the coordinates of the allocated coordinate system acquired in step S308 correspond to is determined from the coordinates of the first character and the last character of each block (step S309).

  If the coordinates of the assigned coordinate system do not fall within any block (step S310; No), the process returns to step S308 and continues. If it falls within any block (step S310; Yes), it is determined which character in the block corresponds to the character (step S311).

  In the determination result table 102, the determination result of the character determined in step S311 is rewritten to 1 (step S312). If all the determination results of the block to which the character belongs are all 1 by rewriting in step S312, the determination result of the block is rewritten to 1.

  If even one block whose determination result is not rewritten to 1 remains (step S313; No), the process returns to step S308 and continues. When the determination results of all the blocks are rewritten to 1 (step S313; Yes), this process ends. When this processing is completed, the CPU 11 determines that the user has confirmed the contents shown on the confirmation screen by eye (recognizes that the content has been read), and starts a predetermined operation such as transition to the next screen or execution of a job. To give permission.

  As described above, the image forming apparatus 10 according to the present embodiment starts the job until it is determined that the user has confirmed the contents shown on the confirmation screen displayed on the display unit 31 with the eyes (recognized as having been read). The execution of predetermined operations such as (restart) and screen transition is prohibited. As a result, it is possible to prevent the predetermined operation from being executed in a state where the user skips the content shown on the confirmation screen. In addition, importance is set for each sentence shown on the confirmation screen, and a sentence with low importance can be skipped. For example, the content that is repeatedly displayed is set to a low importance level. Information such as time limit information, address information, and money amount information is set with high importance. The user may be able to change the importance of each block.

  Note that if a request for starting a predetermined operation is received from the user before it is determined that the content of the confirmation screen displayed on the display unit 31 has been confirmed by the user (recognized as having been read), it is skipped. Among the blocks that cannot be read, a sentence (unread sentence) corresponding to a block with a determination result of 0 (confirmation, incomplete) may be highlighted. Examples of highlighting include bold display and blinking display. Further, as an example of receiving a request for starting a predetermined operation from the user, for example, the buttons (decision button 92, cancel button 93) in the broken line in FIG. 5 are displayed from the start of display of the destination confirmation screen 91 In addition, the determination button 92 may be pressed without reading the information portion indicating the transmission destination.

  The embodiment of the present invention has been described with reference to the drawings. However, the specific configuration is not limited to that shown in the embodiment, and there are changes and additions within the scope of the present invention. Are also included in the present invention.

  In the embodiment of the present invention, the image forming apparatus 10 has been described as an example. However, the CPU 11, the ROM 12, the RAM 13, the hard disk device 15, the display unit 31, the operation unit 32, and the line-of-sight detection unit 22 are extracted. An operation display device may be used. In this case, the predetermined operation prohibited from being executed while the confirmation screen is unread includes an operation for instructing a predetermined device to start a job, an operation for instructing that device to resume a job, and a display. For example, an operation for changing the screen displayed on the unit 31, an operation for displaying a predetermined operation button on the display unit 31, and the like.

  In the embodiment of the present invention, the block that can be skipped by the user who is operating the image forming apparatus 10 is checked using both the determination table 100 and the user determination table 101. It may be created and the importance of each block may be determined (with two choices of A or C).

  In the embodiment of the present invention, when the image forming apparatus 10 determines that the content shown on the confirmation screen has been confirmed visually (recognized that the reading has been completed), the image forming apparatus 10 receives a predetermined operation start request. Although the operation has been started, the predetermined operation may be automatically started when it is determined that the content shown on the confirmation screen has been confirmed with eyes.

  In the embodiment of the present invention, the sentence (sentence) unit is divided into a plurality of blocks, but the block may be divided according to other criteria such as a word (word) unit.

  In the embodiment of the present invention, the confirmation screen is displayed when highly confidential data is transmitted. However, the confirmation screen may be displayed whenever data is transmitted even if the confidentiality is low. Note that a confirmation screen may be displayed when the transmission destination includes a specific destination.

3 ... LAN
DESCRIPTION OF SYMBOLS 10 ... Image forming apparatus 11 ... CPU
12 ... ROM
13 ... RAM
DESCRIPTION OF SYMBOLS 14 ... Non-volatile memory 15 ... Hard disk device 16 ... Image processing part 17 ... Scanner part 18 ... Printer part 19 ... Facsimile communication part 20 ... Network communication part 21 ... Authentication part 22 ... Gaze detection part 23 ... Camera for detection 24 ... Irradiation part 30 ... Operation panel 31 ... Display part 32 ... Operation part 33 ... Switch part 34 ... Touch panel part 40 ... PC terminal 90 ... License agreement screen 91 ... Destination confirmation screen 92 ... Determination button 93 ... Cancel button 94 ... Scroll bar 95 ... Confirmation screen 96 ... Confirmation screen 100, 100A ... Determination table 101 ... User determination table 102 ... Determination result table 120 ... Eyeball 121 ... Line of sight 122 ... Ray 123 ... Pupil center 124 ... Purkinje image 125 ... Corneal curvature center

Claims (9)

An operation unit that receives operations from the user;
A display unit for displaying a screen corresponding to the operation received by the operation unit;
A line-of-sight detection unit for detecting the line of sight of the user;
A control unit that determines whether the user visually confirms the display content of the screen based on a detection result of the line-of-sight detection unit, and prohibits execution of a predetermined operation while the confirmation is incomplete When,
An operation display device characterized by comprising: The predetermined operation includes an operation for instructing a predetermined device to start a job, an operation for instructing the device to restart a job, an operation for changing a screen displayed on the display unit, and a predetermined operation button. The operation display device according to claim 1, comprising any one of operations displayed on the display unit. The control unit displays a predetermined confirmation screen when the setting of a transmission job to be executed by a predetermined device is received by the operation unit and the content of the setting includes a predetermined confirmation target item. The operation display device according to claim 1, wherein the determination is performed on the confirmation screen, and the execution of the transmission job is prohibited while the confirmation is not completed. The operation display according to any one of claims 1 to 3, wherein the control unit performs the determination only on a specific part of the screen displayed on the display unit. apparatus. The operation display device according to claim 4, wherein the specific portion is a portion having an importance level higher than a predetermined value in the screen. The operation display device according to claim 4, wherein the specific portion is a portion in which any one of time limit information, address information, and money amount information is displayed. When the control unit receives a request for starting the predetermined operation from the user while the confirmation is not completed, the control unit highlights a part of the specific portion where the user's line of sight is not yet directed. The operation display device according to any one of claims 4 to 6. The operation display device according to any one of claims 5 to 7, wherein the importance of each part can be changed in the screen. The operation display device according to claim 8, wherein importance of each part can be set for each user.

Images