JP2020129269A - Image forming device and display control method - Google Patents

Abstract

To provide an image forming device capable of specifying an operation button being paid attention by an operator even when a plurality of operation buttons are in an area viewed by the operator.SOLUTION: An image forming device displays a screen including a plurality of operation buttons on an operation panel. The image forming device specifies an area in the screen viewed by an operator based on the operator's line of sight. The image forming device specifies, when two or more operation buttons are in the specified area, one operation button among two or more operation buttons existing in the area based on information input to the image forming device.SELECTED DRAWING: Figure 9

Description

The present invention relates to an image forming apparatus and a display control method in the image forming apparatus.

2. Description of the Related Art Conventionally, there is known an apparatus having a function of displaying help information such as a method of operating a device and a description of the function. For example, Patent Document 1 proposes an unmanned terminal device that detects a straying motion due to the line of sight of a user using a line-of-sight detection unit and displays help information.

Further, Patent Document 2 discloses an electronic device intended to detect a hesitation according to a user. The electronic device measures the user's stray time from the detection of the gaze by the gaze detection unit to the operation of the operation key, and when the hesitation time exceeds a specific positive direction variation allowable range Determines that he is uncertain about the operation.

Further, there is known an image forming apparatus which is a copying machine, a printer, a facsimile or a composite machine thereof. Such an image forming apparatus has an operation panel. An operation screen having a plurality of selectable operation buttons (software keys) is displayed on the operation panel.

JP-A-8-328715 JP, 2005-184698, A

On the operation panel of the image forming apparatus, many operation buttons are often displayed on one operation screen. Moreover, the display area of the operation panel is not sufficiently wide. Therefore, in the image forming apparatus, the operation buttons have to be arranged close to each other.

Therefore, the image forming apparatus may determine that the area ahead of the line of sight straddles the plurality of operation buttons. That is, the image forming apparatus may determine that there are a plurality of operation buttons in the area viewed by the operator (user). In such a case, the image forming apparatus cannot specify the operation button that the operator is paying attention to, and therefore cannot extract and display only the help information of the operation button.

The present disclosure has been made in view of the above problems, and its purpose is to allow the operator to pay attention even when there are a plurality of operation buttons in the area viewed by the operator. An object of the present invention is to provide an image forming apparatus and a display control method capable of specifying an operating button.

According to an aspect of the present disclosure, the image forming apparatus, an operation panel that accepts an input operation by an operator, a screen including a plurality of operation buttons, a display control unit that causes the operation panel to display, and an operator's line of sight, If there are two or more operation buttons in the specified area and the area specifying means for specifying the area that the operator is viewing on the screen, the area exists in the area based on the information input to the image forming apparatus. Button specifying means for specifying one operation button from one or more operation buttons.

Preferably, the display control means displays the help content corresponding to the specified operation button on the operation panel based on the specification of the operation button.

Preferably, the information is at least document information, paper information, setting contents set based on an input operation on the operation panel, past operation history for the image forming apparatus, and area information indicating an area where the image forming apparatus is used. There is one.

Preferably, the image forming apparatus further includes a scanner that reads a document. The image forming apparatus acquires document information by reading the document with a scanner.

Preferably, the manuscript information is at least one of the manuscript size, the manuscript character string, the writing direction, the character string color, and the number of manuscripts.

Preferably, the image forming apparatus stores the image data of the document by the scanner in the memory. When the operation panel receives an input operation instructing to start a printing operation after the image data is stored in the memory, the image forming apparatus performs printing based on the image data stored in the memory.

Preferably, the image forming apparatus discards the image data stored in the memory when the reading operation is performed again by the scanner.

Preferably, when there are two or more operation buttons in the area, the button identifying unit determines whether or not the content set by each operation button has a prohibition relationship with the current setting content. The button hand identifying means identifies one operation button from the remaining operation buttons excluding the operation button in which the prohibited content is set from the two or more operation buttons in the area.

Preferably, when two or more operation buttons are present in the area, the button specifying unit determines the setting content having the highest probability of being set together with the current setting content based on the past operation history. The button specifying means specifies an operation button for performing the determined setting content from the two or more operation buttons existing in the area.

Preferably, the image forming apparatus further includes a determination unit that determines the size of the document. When the size of the document is not the standard size, the button specifying means specifies the operation button corresponding to the size of the document from the two or more operation buttons existing in the area.

Preferably, the image forming apparatus further includes a recognition unit that recognizes the character string of the original by optical character recognition. The button specifying unit specifies an operation button corresponding to the information of the character string of the document from the two or more operation buttons existing in the area.

Preferably, the operation panel includes an image pickup section. The area specifying unit recognizes the line of sight of the operator based on the data obtained by the image pickup by the image pickup unit.

Preferably, the display control means causes the operation panel to display an index indicating the area specified by the area specifying means.

According to another aspect of the present disclosure, a display control method for an image forming apparatus includes a step in which a controller of the image forming apparatus receives an input operation by an operator via an operation panel of the image forming apparatus, and a controller performs a plurality of operations. The step of displaying a screen including buttons on the operation panel, the step of the controller identifying the area on the screen viewed by the operator based on the operator's line of sight, and the operation area having two or more operation buttons. If present, the controller specifies one operation button from the two or more operation buttons existing in the area based on the information input to the image forming apparatus.

According to the present disclosure, even when a plurality of operation buttons are present in the area viewed by the operator, it is possible to specify the operation button that the operator is paying attention to.

FIG. 3 is a diagram illustrating an appearance of an image forming apparatus that is an MFP. It is a figure showing the appearance of an operation panel. FIG. 3 is a schematic diagram showing an internal structure of the image forming apparatus. 3 is a block diagram illustrating an example of a hardware configuration of a main body of the image forming apparatus. FIG. It is a functional block diagram for explaining a functional configuration of the image forming apparatus. FIG. 6 is a diagram for explaining information input to the image forming apparatus. It is a figure for demonstrating the data memorize|stored in a memory|storage part. FIG. 6 is a flowchart illustrating a flow of a series of processing from setting a document on the image forming apparatus to executing a job. 9 is a flowchart for illustrating details of help display processing (step S1005 in FIG. 8) executed by image forming apparatus 1000. It is a screen transition diagram for demonstrating a 1st specific example. It is a flow figure showing details of Step S5 in the 1st example. It is a flow chart showing another example of details of Step S5. It is a screen transition diagram for explaining the second specific example. It is a figure showing the detail of step S5 in a 2nd example. It is a screen transition diagram for demonstrating a 3rd specific example. It is a figure showing the detail of step S5 in a 3rd specific example. It is a screen transition diagram for demonstrating a 4th specific example. It is a flow diagram for explaining the first half of the flow of processing in the fourth specific example. It is a flow chart for explaining the latter half of the flow of processing in the 4th example.

The image forming apparatus according to the embodiment will be described below with reference to the drawings. In the embodiments described below, when reference is made to the number, amount, etc., the scope of the present disclosure is not necessarily limited to the number, amount, etc., unless otherwise specified. The same parts or corresponding parts are designated by the same reference numerals, and duplicate description may not be repeated.

In the drawings, not shown according to the actual ratio of dimensions, but in order to facilitate understanding of the structure, there are portions where the ratio is changed so as to clarify the structure. In addition, each embodiment and each modification described below may be appropriately combined selectively.

Although an image forming apparatus as a color printer will be described below, the image forming apparatus is not limited to a color printer. For example, the image forming apparatus may be a monochrome printer, a fax machine, or a multi-functional peripheral (MFP) of a monochrome printer, a color printer, and a fax machine.

<A. Hardware configuration of image forming apparatus 1000>
(A1. Appearance of image forming apparatus 1000)
FIG. 1 is a diagram showing an appearance of an image forming apparatus 1000 which is an MFP. Referring to FIG. 1, image forming apparatus 1000 has operation panel 34. The operation panel 34 receives an input operation by a user (operator).

FIG. 2 is a diagram showing the appearance of the operation panel 34. Referring to FIG. 2, operation panel 34 has a touch screen 341, a camera 39, and a plurality of physical keys. In this example, the touch screen 341 has a structure in which a touch panel is installed on the display.

A screen is displayed on the touch screen 341. An operation screen is displayed on the touch screen 341. The operation screen is, for example, a setting screen. In one aspect, one operation screen has at least one operation button. The operation buttons are also called software keys, software buttons, selectable objects, and icons. A user of the image forming apparatus 1000 can set an item, change an item that is currently set, and the like by operating an operation button. Specific examples of the operation buttons will be described later (see FIGS. 10, 13, and 15).

The camera 39 images the user. Specifically, the camera 39 is installed at a position where the face (at least eyes) of the user looking at the operation panel 34 can be imaged. The image forming apparatus 1000 detects the line of sight of the user based on the image obtained by the camera 39. Specifically, the image forming apparatus 1000 identifies a region on the operation screen viewed by the user (hereinafter, also referred to as “gaze region”) based on the line of sight of the user. Since the line-of-sight detection is a conventionally known technique, it will not be described in detail here.

(A2. Internal structure of image forming apparatus 1000)
FIG. 3 is a schematic diagram showing the internal structure of the image forming apparatus 1000. With reference to FIG. 3, image forming apparatus 1000 includes main body 10 and post-processing device 20, as described above.

The main body 10 includes an image forming unit 11, a scanner unit 12, an automatic document feeding unit 13, paper feed trays 14A and 14B, a feeding path 15, a media sensor 16, a reverse feeding path 17, and a feeding roller. And 113. The automatic document feeder unit is also called an ADF (auto document feeder).

The main body 10 further includes a controller 31 that controls the operation of the image forming apparatus 1000. In this example, the main body 10 is a so-called tandem color printer. The main body unit 10 executes image formation based on print settings.

The automatic document feeding unit 13 automatically feeds the document placed on the document table to the reading position of the document reading unit. The scanner unit 12 reads an image of a document conveyed by the automatic document conveying unit 13 and generates image data. Further, the scanner unit 12 also reads an image of a document placed on the platen by the user without using the automatic document feeding unit 13 and generates image data. The image data of the document by the scanner unit 12 is stored in a memory (typically, the fixed storage device 32 shown in FIG. 4).

Paper P is stored in the paper feed trays 14A and 14B. The paper feed roller 113 feeds the paper P upward along the transport path 15. Each of the paper feed trays 14A and 14B includes a bottom raising plate 142 and a sensor 143. The sensor 143 detects the position of a regulation plate (not shown) in the paper feed tray, and also detects the size of the paper.

The transport path 15 is used for single-sided printing and double-sided printing. The reverse conveyance path 17 is used for double-sided printing.

The image forming unit 11 forms an image on the paper P supplied by the paper feed trays 14A and 14B based on the image data generated by the scanner unit 12 or the print data acquired from a PC (not shown).

The image forming unit 11 includes an intermediate transfer belt 101, a tension roller 102, a driving roller 103, a yellow image forming unit 104Y, a magenta image forming unit 104M, a cyan image forming unit 104C, and a black image forming unit. 104K, an image density sensor 105, a primary transfer device 111, a secondary transfer device 115, a registration roller pair 116, and a fixing device 120 including a heating roller 121 and a pressure roller 122. The tension roller 102 and the drive roller 103 hold the intermediate transfer belt 101 and rotate the intermediate transfer belt 101 in the direction A in the drawing. The registration roller pair 116 conveys the paper P conveyed by the paper feed roller 113 further downstream.

The media sensor 16 is installed in the transport path 15. The media sensor 16 realizes a paper type automatic detection function.

The post-processing device 20 further includes a punching processing device 220, a side stitching processing unit 250, a saddle stitching processing unit 260, a discharge tray 271, a discharge tray 272, and a lower discharge tray 273.

(A3. Hardware configuration of main unit 10)
FIG. 4 is a block diagram for explaining an example of the hardware configuration of the main body 10 of the image forming apparatus 1000.

Referring to FIG. 4, the main body unit 10 includes a controller 31, a fixed storage device 32, a short-range wireless IF (Inter Face) 33, a camera 39, a scanner unit 12, an operation panel 34, and a paper feed tray. 14A and 14B, the media sensor 16, the image forming unit 11, the printer controller 35, the network IF 36, and the wireless IF 37. Each unit 11, 12, 14A, 14B, 16, 32 to 37 is connected to the controller 31 via a bus 38.

The controller 31 includes a CPU (Central Processing Unit) 311, a ROM (Read Only Memory) 312 in which a control program is stored, a work S-RAM (Static Random Access Memory) 313, and various settings related to image formation. And a battery-backed NV-RAM (Non-Volatile RAM) 314 and a timepiece IC (Integrated Circuit) 315. The units 311 to 315 are connected via a bus 38.

The operation panel 34 has keys for performing various inputs and a display unit. The operation panel 34 typically includes a touch screen and hardware keys. The touch screen is a device in which a touch panel is superimposed on the display.

The camera 39 images the user as described above. The image captured by the camera 39 is sent to the controller 31. The controller 31 identifies the area (line-of-sight area) viewed by the user on the operation screen based on the image.

The network IF 36 transmits/receives various types of information to/from external devices such as a PC (not shown) and another image forming device (not shown) connected via the network NW.

The printer controller 35 generates a copy image from the print data received by the network IF 36. The image forming unit 11 forms a copy image on a sheet.

The fixed storage device 32 is typically a hard disk device. Various data is stored in the fixed storage device 32.

<B. Functional configuration>
FIG. 5 is a functional block diagram for explaining the functional configuration of the image forming apparatus 1000.

Referring to FIG. 5, image forming apparatus 1000 includes control unit 1100, document reading unit 1200, storage unit 1300, image pickup unit 3411, and display unit 3412.

The imaging unit 3411 corresponds to the camera 39. The display portion 3412 corresponds to the display of the touch screen 341 (see FIG. 2). The document reading unit 1200 corresponds to the scanner unit 12. The storage unit 1300 typically corresponds to the fixed storage device 32.

The control unit 1100 corresponds to the controller 31 (see FIG. 3). Typically, the control unit 1100 is realized by the processor (CPU 311) executing an OS (Operating System) and various programs stored in the memory.

The imaging unit 3411 images the face (at least eyes) of the user. The imaging unit 3411 sends the image data obtained by the imaging to the control unit 1100.

The display portion 3412 displays various screens. The display unit 3412 displays a screen based on the operation of the operation panel 34 by the user, for example.

The storage unit 1300 stores help content (hereinafter, also referred to as “help information”) corresponding to each operation button. The storage unit 1300 also stores various other data. The storage unit 1300 typically corresponds to the fixed storage device 32. The storage unit 1300 may include the RAM and ROM in the controller 31.

The control unit 1100 includes a gaze area specifying unit 1101, an operation button specifying unit 1102, a help information reading unit 1103, a display control unit 1104, a size determining unit 1105, and a character recognizing unit 1106.

The display control unit 1104 controls the display on the display unit 3412. When the display control unit 1104 receives a user operation via the operation panel 34, the display control unit 1104 causes the display unit 3412 to display a screen based on the user operation. In one aspect, display control unit 1104 causes display unit 3412 to display a screen including a plurality of operation buttons.

The line-of-sight region identifying unit 1101 identifies the region (line-of-sight region) viewed by the operator on the operation screen displayed on the operation panel 34 based on the line of sight of the operator. Specifically, the line-of-sight region identifying unit 1101 identifies the line-of-sight region (line-of-sight position) on the operation screen based on the image data sent from the imaging unit 3411. The line-of-sight region identification unit 1101 notifies the operation button identification unit 1102 of the identification result. Typically, the line-of-sight region specifying unit 1101 notifies the operation button specifying unit 1102 of the coordinate value indicating the specified region.

The operation button specifying unit 1102 specifies an operation button included in the specified line-of-sight region. Typically, when at least a part of the operation buttons is included in the line-of-sight region, operation button identifying unit 1102 determines that the operation button is included in the line-of-sight region.

When there are two or more operation buttons in the specified line-of-sight region, the operation button specifying unit 1102 determines whether the two or more operation buttons existing in the line-of-sight region are based on the information input to the image forming apparatus 1000. Identify one operation button from among them. The “information input to the image forming apparatus 1000” will be described later.

Specifically, the operation button specifying unit 1102 acquires information on the operation button being displayed from the display control unit 1104. Specifically, the operation button specifying unit 1102 acquires information on the position and type of the operation button from the display control unit 1104. The operation button specifying unit 1102 selects one operation button from two or more operation buttons existing in the line-of-sight region based on the information acquired from the display control unit 1104 and the information input to the image forming apparatus 1000. Specify.

The operation button specifying unit 1102 notifies the help information reading unit 1103 of the information (type) of the specified operation button.

The help information reading unit 1103 reads, from the storage unit 1300, help information corresponding to the operation button notified from the operation button specifying unit 1102. Further, the help information reading unit 1103 notifies the display control unit 1104 of the help information read from the storage unit 1300.

The display control unit 1104 causes the display unit 3412 to display a screen showing help information notified from the help information reading unit 1103 (hereinafter, also referred to as “help screen”). That is, the display control unit 1104 causes the operation panel 34 to display a help screen corresponding to the specified operation button based on the operation button being specified by the operation button specifying unit 1102.

FIG. 6 is a diagram for explaining the information input to the image forming apparatus 1000.
With reference to FIG. 6, the information input to image forming apparatus 1000 includes, for example, “original information”, “paper information”, “setting contents by operation panel”, and “operation history”. "Regional information" is mentioned. The “contents set by the operation panel” is the settings made based on the input operation on the operation panel 34. The “operation history” is a past operation history for the image forming apparatus 1000.

As "original information", "orientation (vertical/horizontal direction)", "image type (text, photo)", "document", "writing direction" (that is, the direction in which characters are written) about the original , “Number of sheets”, “color”, “size”, “character string” and the like. The writing direction is "vertical writing" and "horizontal writing". Note that these pieces of information are information obtained by operating the document reading unit 1200 (scanner unit 12). When the automatic document feeding unit 13 is used, when the user sets the document on the document tray, the image forming apparatus 1000 detects the position of the side regulating plate and the detection of the sensor provided on the tray. Detects the orientation and size of the document.

Examples of the "paper information" include "size", "type (thick paper, thin paper, plain paper)", "tray information", "orientation in the case of manual feeding", and the like regarding the paper. For example, it is assumed that which type of paper is contained in which supply tray is included in the paper information. It should be noted that the sheet information regarding these sheets can be detected by a sensor or the like installed near the sheet feed tray, or can be designated and registered in advance from the operation panel 34.

"Settings on the operation panel" include single-sided/double-sided, color printing/monochrome printing, combined copy (2in1, 4in1), magnification setting, paper tray selection, number of copies setting, "staple (also called staple)", "punch". And the like.

Examples of the “regional information” include “country name”. The area information is typically set after the image forming apparatus 1000 is installed. When the shipping destination of the image forming apparatus 1000 is determined in advance, it may be set by default. The image forming apparatus 1000 stores the set regional information in the NV-RAM 314. The image forming apparatus 1000 may store the set area information in the fixed storage device 32.

Referring again to FIG. 5, size determination unit 1105 determines the size of the document read by document reading unit 1200. The size determination unit 1105 also determines whether the document size is a standard size. Further, the size determination unit 1105 also determines the size type (A5, A4, A3, B4, B5, etc.) in the case of the standard size. The size determination unit 1105 notifies the operation button identification unit 1102 of the determination result.

In this case, if the size of the document is not the standard size, the operation button specifying unit 1102 selects the size of the document (information input to the image forming apparatus 1000 from the two or more operation buttons existing in the line-of-sight region). Operation button according to the example).

The character recognition unit 1106 recognizes the character string of the document read by the document reading unit 1200 by optical character recognition (OCR). The character recognition unit 1106 notifies the operation button identification unit 1102 of the recognition result.

In this case, the operation button specifying unit 1102 specifies an operation button corresponding to the character recognition result (an example of information input to the image forming apparatus 1000) from the two or more operation buttons existing in the line-of-sight region. ..

FIG. 7 is a diagram for explaining the data D7 stored in the storage unit 1300.
Referring to FIG. 7, data D7 stores the probability of a combination of setting contents as data based on the operation history of the job. As an example, in the case of the 2in1 setting, the probability of performing stapler, monochrome, and printing from one side to both sides is shown. The data D7 stores the probabilities of the setting of combinations not only for 2 in 1 but also for other settings (for example, 4 in 1, double-sided, color, etc.). This data D7 is calculated, for example, from the setting information of jobs that have been executed in the past, the number of executions of each combination of settings, and is calculated as a ratio to the number of executions of the job.

These probabilities are updated every time the image forming apparatus 1000 is used. Further, these probabilities may be set for each user when the user authentication of the image forming apparatus 1000 is performed. That is, the image forming apparatus 1000 may store the data D7 for each user. A configuration example (specific example) using the data D7 will be described later. In this example, the probability of each setting combination is stored as the operation history information, but the setting information of each job itself is stored as the operation history, and the operation button for the help display is specified each time. The probability of combination may be calculated.

Further, when the original reading unit 1200 performs the reading operation again, the control unit 1100 may discard the image data (output candidate image) stored in the storage unit 1300. For example, when the document is read again after the help screen is displayed, when the job (for example, print) execution button is selected, control unit 1100 deletes the output candidate image from storage unit 1300. Good. According to this, it becomes unnecessary to store unnecessary data in the storage unit 1300.

<C. Control structure>
FIG. 8 is a flowchart for explaining the flow of a series of processes from the setting of a document on image forming apparatus 1000 to the execution of a job.

Referring to FIG. 8, in step S1001, a user (operator of image forming apparatus 1000) sets a document on image forming apparatus 1000. In step S1002, the user inputs various settings (input to the operation panel 34). In step S1003, the image forming apparatus 1000 detects the line of sight of the user. In step S1004, the image forming apparatus 1000 determines whether the line of sight has not moved for a predetermined time.

If it is determined that the line of sight has not moved for a predetermined time (YES in step S1004), image forming apparatus 1000 executes a help display process in step S1005. When it is determined that the line of sight has moved within the predetermined time (NO in step S1004), image forming apparatus 1000 advances the process to step S1006.

In step S1006, the image forming apparatus 1000 determines whether the execution button (typically, the start button) for executing the job is pressed. If it is determined that the execute button has been pressed (YES in step S1006), image forming apparatus 1000 determines in step S1007 whether there is an output candidate image. If it is determined that the execute button has not been pressed (NO in step S1006), image forming apparatus 1000 ends the series of processes.

If it is determined that there is an output candidate image (YES in step S1007), image forming apparatus 1000 determines in step S1008 whether the setting has been changed or the document has been reset. When it is determined that there is no output candidate image (NO in step S1007), image forming apparatus 1000 operates document reading unit 1200 to read the document in step S1012. After that, the image forming apparatus 1000 advances the process to step S1011.

If an affirmative determination is made in step S1008 (YES), the image forming apparatus 1000 discards the output candidate image in step S1009. If a negative determination is made in step S1008 (NO), the image forming apparatus 1000 advances the process to step S1011.

In step S1010, image forming apparatus 1000 operates document reading unit 1200 to read the document. In step S1011, the image forming apparatus 1000 executes the designated job.

FIG. 9 is a flowchart for explaining details of the help display process (step S1005 in FIG. 8) executed by the image forming apparatus 1000.

Referring to FIG. 9, in step S1, controller 31 (that is, control unit 1100) specifies the line-of-sight region viewed by the user based on the image data obtained by the image pickup by camera 39. In step S2, the controller 31 identifies the operation button included in the line-of-sight region. The specified line-of-sight region may be displayed on the display unit 3412 of the operation panel 34.

In step S3, the controller 31 determines whether an operation button is present in the line-of-sight area. When the controller 31 determines that the operation button does not exist in the line-of-sight region (NO in step S3), the process returns to step S1. When it is determined that the operation button is present in the line-of-sight region (YES in step S3), the controller 31 determines in step S4 whether or not a plurality of operation buttons are present in the line-of-sight region.

When it is determined that the plurality of operation buttons are present in the line-of-sight region (YES in step S4), the controller 31 determines in step S5 that two or more operation lines are present in the line-of-sight region based on information input to the image forming apparatus 1000. One operation button is specified from among the operation buttons of. In step S6, the controller 31 displays a help screen corresponding to the operation button specified in step S5 on the operation panel 34. Specifically, the controller 31 displays on the display unit 3412 of the operation panel 34. After that, the controller 31 advances the process to step S7.

When it is determined that the operation button existing in the line-of-sight region is one (NO in step S4), the controller 31 displays the help screen corresponding to the operation button (one operation button) existing in the line-of-sight region in step S8. Is displayed on the operation panel 34. After that, the controller 31 advances the process to step S7.

In step S7, the controller 31 determines whether or not a user operation for closing the help screen has been received. When the controller 31 receives the user operation (YES in step S7), the controller 31 ends the series of processes. When the controller 31 determines that the user operation has not been accepted (NO in step S7), the process returns to step S7.

<D. Summary>
As described above, the image forming apparatus 1000 operates the operation panel 34 that accepts an input operation by the user, the display control unit 1104 that displays an operation screen including a plurality of operation buttons on the operation panel 34, and the operation based on the line of sight of the user. When there are two or more operation buttons in the line-of-sight region specifying unit 1101 that identifies the line-of-sight region that the user is looking at on the screen, the line-of-sight based on the information input to the image forming apparatus 1000. An operation button specifying unit 1102 for specifying one operation button from two or more operation buttons existing in the area. Further, the display control unit 1104 causes the operation panel 34 to display a help screen corresponding to the specified operation button based on the specification of the operation button.

With such a configuration, even when a plurality of operation buttons are present in the line-of-sight area, it is possible to specify the operation button that the user is paying attention to. Therefore, it is possible to display the help screen of the operation button that the user is paying attention to.

The information input to the image forming apparatus 1000 includes original information, paper information, setting contents set based on an input operation on the operation panel 34, past operation history of the image forming apparatus 1000, and the image forming apparatus 1000. It is at least one piece of area information indicating the area of use.

The image forming apparatus 1000 can specify one operation button from the plurality of operation buttons by using such various kinds of information. In addition, the image forming apparatus 1000 can identify one operation button from a plurality of operation buttons by appropriately combining such information.

<E. Specific example>
Hereinafter, “a process of identifying one operation button from two or more operation buttons existing in the line-of-sight region based on the information input to the image forming apparatus 1000” will be described with reference to a plurality of specific examples. To do. The specific examples shown below are merely examples, and the present invention is not limited to these. A combination of the specific examples is also possible.

(E1. First specific example)
In the first specific example, for convenience of explanation, a process focusing on “setting contents by the operation panel” as information input to the image forming apparatus will be described. Next, as a modified example, processing focusing on “setting contents by operation panel” and “operation history” will be described. Finally, the processing focusing on the “operation history” will be described.

(1) Process Focusing on Setting Contents by Operation Panel FIG. 10 is a screen transition diagram for explaining the first specific example. First, the user places a plurality of originals on the automatic original conveying unit 13 of the image forming apparatus 1000. Next, the user operates the operation panel 34 to change the setting relating to “page aggregation” to change the setting to “2 in 1”. After that, it is assumed that the operation screen shown in the state (A) of FIG. 10 is displayed on the operation panel 34 based on the user operation or the like. The operation screen includes a plurality of operation buttons 511, 512, 513 and the like.

In this case, when the user's line of sight stops, the line-of-sight region identifying unit 1101 identifies the line-of-sight region 701 on the operation screen viewed by the operator based on the line of sight. At this time, the specified line-of-sight region may be displayed on the display unit 3412 of the operation panel 34.

The operation button identifying unit 1102 determines that the line-of-sight area 701 includes two operation buttons. Specifically, the operation button specifying unit 1102 determines that the line-of-sight area 701 includes an operation button 511 for setting stapler and an operation button 512 for setting continuous enlarged shooting.

The operation button specifying unit 1102 specifies one operation button from the two or more operation buttons existing in the line-of-sight region 701 based on the information input to the image forming apparatus 1000. Specifically, the operation button specifying unit 1102 removes the enlarged continuous shooting for which the combination with “2in1” is prohibited (prohibition) based on the input for setting “2in1”. As a result, the operation button specifying unit 1102 specifies the operation button 511 for performing the staple setting from the plurality of operation buttons 511 and 512.

The display control unit 1104 displays the help screen 591 regarding the stapler on the operation panel 34, as shown in the state (B) of FIG. 10, based on the specification of the operation button 511. Typically, the display control unit 1104 pops up a help screen 591.

FIG. 11 is a flowchart showing details of step S5 in the first specific example.
Referring to FIG. 11, in step S511, controller 31 reads the setting contents currently set in image forming apparatus 1000 from storage unit 1300. In step S512, the controller 31 identifies the operation button based on the read setting information.

As described above, when there are two or more operation buttons in the line-of-sight region, the operation button identification unit 1102 determines whether the content set by each operation button has a prohibitive relationship with the current setting content. to decide. The operation button identification unit 1102 identifies one operation button from the remaining operation buttons excluding the operation button in which the prohibited contents are set from the two or more operation buttons in the line-of-sight area.

With such a configuration, it is possible to improve the identification accuracy of the operation button that the user is paying attention to, as compared with the case where the prohibition relationship is not used. Further, it is possible to prevent the help screen of the operation buttons (items) that cannot be set from being displayed, which improves convenience for the user.

(2) Processing Focusing on Setting Contents by Operation Panel and Operation History FIG. 12 is a flowchart showing another example of details of step S5.

With reference to FIG. 12, in step S511A, the controller 31 reads the operation history (data D7 and the like) from the storage unit 1300. In step S512A, the controller 31 identifies the operation button based on the read operation history.

The operation button specifying unit 1102 may specify one operation button from the plurality of operation buttons with reference to the data D7. For example, since the stapler is the most probable setting in combination with “2in1”, the operation button identifying unit 1102 identifies the operation button 511 for performing the stapler setting from the plurality of operation buttons 511 and 512. May be. By using such data D7 (past operation history), the specific accuracy is increased.

(3) Processing Focusing on Operation History In addition, when there are a plurality of operation buttons in the line-of-sight area and the setting by these operation buttons is not prohibited from being combined with the current setting, the operation button identification unit 1102. May specify one operation button from the plurality of operation buttons using the data D7. Specifically, the operation button identification unit 1102 may identify the operation button with the highest probability from the plurality of operation buttons. For example, when the “2 in 1” is set and the operation button of “monochrome” and the operation button of “print from one side to both sides” are included in the line-of-sight area, the operation button specifying unit 1102 determines that both operation buttons are operated. The operation button of “monochrome” with the highest probability is specified.

As described above, when there are two or more operation buttons in the line-of-sight region, the operation button identification unit 1102 determines the setting content that has the highest probability of being set together with the current setting content based on the past operation history. .. The operation button specifying unit 1102 specifies an operation button for performing the determined setting content from the two or more operation buttons existing in the line-of-sight area. With such a configuration, it is possible to improve the identification accuracy of the operation button that the user is paying attention to, as compared with the case where the probability data is not used.

(E2. Second specific example)
FIG. 13 is a screen transition diagram for explaining the second specific example. First, the user places a license on the platen of the image forming apparatus 1000. After that, it is assumed that the operation screen shown in the state (A) of FIG. 13 is displayed on the operation panel 34 based on the user operation or the like. The operation screen includes a plurality of operation buttons 521, 522, 523 and the like.

In this case, when the user's line of sight stops, the line-of-sight region identifying unit 1101 identifies the line-of-sight region 702 of the operator on the operation screen based on the line of sight.

The operation button identification unit 1102 determines that the line-of-sight area 702 includes two operation buttons. Specifically, the operation button identifying unit 1102 determines that the line-of-sight area 702 includes an operation button 521 for setting a card copy and an operation button 522 for setting an index document.

The operation button specifying unit 1102 specifies one operation button from the two or more operation buttons existing in the line-of-sight area 702 based on the information input to the image forming apparatus 1000. Specifically, the image forming apparatus 1000 operates the document reading unit 1200. The size determination unit 1105 determines the size of the document on the platen. The operation button specifying unit 1102 acquires the size information of the document from the size determining unit 1105. The operation button specifying unit 1102 specifies the operation button based on the acquired size information. In the case of this example, since the size of the document is determined to be the card size, the operation button 521 for setting the card copy is specified from among the plurality of operation buttons 511 and 512.

The display control unit 1104 displays the help screen 592 regarding the card copy on the operation panel 34, as shown in the state (B) of FIG. 13, based on the specification of the operation button 521. Typically, the display control unit 1104 pops up a help screen 592.

The controller 31 stores the image data of the document (license in this example) by the document reading unit 1200 in the storage unit 1300. The image data becomes the output candidate image. Image forming apparatus 1000 performs printing based on the image data stored in storage unit 1300 when operation panel 34 receives an input operation instructing to start a printing operation after image data is stored in storage unit 1300. That is, in the case of this example, the image forming apparatus 1000 prints the license.

FIG. 14 is a diagram showing details of step S5 in the second specific example.
Referring to FIG. 14, in step S521, controller 31 operates document reading unit 1200. In step S522, the controller 31 stores the image data as an output candidate image. In step S523, the controller 31 identifies the operation button based on the image data.

As described above, the image forming apparatus 1000 includes the document reading unit 1200 that reads a document and the size determination unit 1105 that determines the size of the document. When the document reading unit 1200 reads the document, the size determination unit 1105 determines (acquires) the size of the document as the document information. When the size of the document is not the standard size, the operation button specifying unit 1102 specifies the operation button corresponding to the size of the document from the two or more operation buttons existing in the line-of-sight area. With such a configuration, it is possible to improve the identification accuracy of the operation button that the user is paying attention to.

Further, the image forming apparatus 1000 stores the image data of the document by the document reading unit 1200 in the storage unit 1300. Image forming apparatus 1000 performs printing based on the image data stored in storage unit 1300 when operation panel 34 receives an input operation instructing to start a printing operation after image data is stored in storage unit 1300. With such a configuration, it is not necessary to perform the document reading operation after the input operation for instructing the start of the printing operation, so that the printing can be executed quickly.

(E3. Third specific example)
FIG. 15 is a screen transition diagram for explaining the third specific example. First, the user places a plurality of originals on the automatic original conveying unit 13 of the image forming apparatus 1000. Next, it is assumed that the user operates the operation panel 34 to display an operation screen regarding the setting of “staple” on the operation panel 34 as shown in the state (A) of FIG. 15. The operation screen includes a plurality of operation buttons 511, 512, 513 and the like.

In this case, when the user's line of sight stops, the line-of-sight region identifying unit 1101 identifies the line-of-sight region 703 on the operation screen viewed by the operator based on the line of sight.

The operation button identifying unit 1102 determines that the line-of-sight area 703 includes two operation buttons. Specifically, the operation button identifying unit 1102 determines that the line-of-sight area 703 includes an operation button 531 for stapling the upper right of the document and an operation button 532 for stapling the upper left of the document.

In this case, the controller 31 causes the automatic document feeding unit 13 to feed the document and causes the document reading unit 1200 to read the document. Character recognition unit 1106 performs character recognition processing on the read document. Further, the controller 31 reads the area information stored in the storage unit 1300. The reason for performing such processing is that the position of Hochitis differs depending on the writing direction and the country in which the image forming apparatus 1000 is used.

The operation button specifying unit 1102 specifies one operation button from the plurality of operation buttons 531 and 532 based on the read area information and the writing direction (vertical writing or horizontal writing) based on the character recognition result. To do. For example, when the read regional information is Japan and the writing direction is horizontal writing, the operation button specifying unit 1102 selects the operation button for stapling to the upper left from the plurality of operation buttons 531 and 532. 532 is specified.

The display control unit 1104 displays the help screen 593 on the operation panel 34, as shown in the state (B) of FIG. 15, based on the specification of the operation button 532. Typically, the display control unit 1104 pops up a help screen 593.

FIG. 16 is a diagram showing details of step S5 in the third specific example. Note that in FIG. 16, first, the process of identifying the operation button is executed by also referring to the past operation history, and if the operation button cannot be identified, the operation button is identified based on the image data of the document and the area information. The configuration will be described. Specifically, a configuration for identifying the operation button using the information on the writing direction obtained by performing character recognition on the image data and the area information will be described. The process based on the operation history is the same as in the first embodiment shown in FIG.

In step S531, the controller 31 reads the operation history from the storage unit 1300. In step S532, the controller 31 determines whether the operation button can be specified based on the read operation history.

When it is determined that it can be specified (YES in step S532), one operation button is specified from the plurality of operation buttons included in the line-of-sight region. After that, in step S534, the controller 31 operates the document reading unit 1200. In step S535, the controller 31 stores the image data as an output candidate image.

When it is determined that it is not identifiable (NO in step S532), controller 31 causes document reading unit 1200 to operate in step S536. In step S537, the controller 31 stores the image data as an output candidate image.

Next, in step S538, the controller 31 performs character recognition processing to specify the writing direction. Further, in step S539, the controller 31 reads the area information from the NV-RAM 314. After that, in step S540, the controller 31 specifies one operation button from the plurality of operation buttons included in the line-of-sight region based on the image data (writing direction in this example) and the area information.

As described above, the image forming apparatus 1000 includes the document reading unit 1200 that reads a document and the character recognition unit 1106 that recognizes a character string of the document by optical character recognition. When the document reading unit 1200 reads the document, the character recognition unit 1106 performs character recognition of the read document. From this, the controller 31 can acquire the information of the character string. For example, the controller 31 can determine the writing direction.

The operation button specifying unit 1102 specifies the operation button corresponding to the information of the character string of the document from the two or more operation buttons existing in the line-of-sight area. For example, the operation button identification unit 1102 identifies one operation button from the operation buttons included in the line-of-sight region based on the writing direction as the information of the character string and the area information. With such a configuration, it is possible to improve the identification accuracy of the operation button that the user is paying attention to.

(E4. Fourth specific example)
FIG. 17 is a screen transition diagram for explaining the fourth specific example. First, the user places a plurality of originals on the automatic original conveying unit 13 of the image forming apparatus 1000. After that, it is assumed that the operation screen shown in the state (A) of FIG. 17 is displayed on the operation panel 34 based on the user operation or the like. The operation screen includes a plurality of operation buttons 541, 542, 543 and the like.

In this case, when the user's line of sight stops, the line-of-sight region identifying unit 1101 identifies the line-of-sight region 704 on the operation screen viewed by the operator based on the line of sight.

The operation button identifying unit 1102 determines that the line-of-sight area 704 includes two operation buttons. More specifically, the operation button specifying unit 1102 determines that there is an operation button 541 for setting a staple and an operation button 542 for setting an enlargement of the printing magnification.

The operation button specifying unit 1102 specifies one operation button from the two or more operation buttons existing in the line-of-sight region 704, based on the information input to the image forming apparatus 1000. For example, when the document size is A4 size and A4 size paper is set in all the paper feed trays 14A and 14B, the operation button specifying unit 1102 determines that the size of the document and the size of the paper are used. , The magnification is excluded. As a result, the operation button specifying unit 1102 specifies the operation button 541 for performing the staple setting from the plurality of operation buttons 541 and 542.

The detection of the document size here can be performed by detecting the position of the side regulating plate or a sensor provided on the tray when the document size is set on the document tray of the automatic document feeding unit 13. The size of the paper set in the paper feed tray can be detected by a sensor or the like installed near the paper feed tray, or can be specified and registered in advance from the operation panel.

The display control unit 1104 displays the help screen 594 regarding the stapler on the operation panel 34, as shown in the state (B) of FIG. 17, based on the specification of the operation button 541. Typically, the display control unit 1104 pops up a help screen 594.

Since the staple processing is executed on a plurality of sheets, it is necessary that the number of originals is also plural. Therefore, a process of confirming the number of documents may be performed before specifying the operation button 541 for performing the staple setting from the plurality of operation buttons included in the line-of-sight region 704.

Specifically, the controller 31 conveys the document by the automatic document feeding unit 13 and causes the document reading unit 1200 to read the document before the final specification of the operation button. At this time, the controller 31 counts the number of documents. When the count value is 2 or more, the operation button specifying unit 1102 specifies (finally determines) the operation button 541 for stapling setting.

FIG. 18 is a flowchart for explaining the first half of the flow of processing in the fourth specific example. FIG. 19 is a flowchart for explaining the latter half of the processing flow in the fourth specific example.

The series of processing flows shown in FIGS. 18 and 19 differ from the processing flow shown in FIG. 9 in that step S5A is provided instead of step S5. The series of processing flows shown in FIGS. 18 and 19 differ from the processing flow shown in FIG. 9 in that step S9 is further included. Therefore, step S5A and step S9 will be described below.

Referring to FIG. 18, step S5A includes steps S541 to S545. In step S541, the controller 31 narrows down the operation buttons based on the information input to the image forming apparatus 1000. In step S542, the controller 31 determines whether or not one operation button has been specified.

When it is determined that it has been specified (YES in step S542), the controller 31 advances the process to step S6. When it is determined that the document cannot be specified (NO in step S542), the controller 31 operates the document reading unit 1200 to read the document in step S543. In step S544, the controller 31 narrows down the operation buttons based on the image data. In step S545, the controller 31 determines whether or not one operation button has been specified.

When it is determined that it has been specified (YES in step S545), with reference to FIG. 19, the controller 31 advances the process to step S6. When it is determined that it cannot be specified (NO in step S545), the controller 31 displays the help screens corresponding to all the operation buttons existing in the line-of-sight area on the operation panel in step S9. The controller 31 may display all the help screens on the operation panel 34 at one time, or may display all the help screens on the operation panel 34 by switching the display in order.

In the above description, based on the specification of the operation button, the image forming apparatus 1000 displays the help content corresponding to the specified operation button on the operation panel 34 as an example. However, the present invention is not limited to this, and, for example, the image forming apparatus 1000 (specifically, the display control unit 1104) changes the display color and/or the size of the specified operation button, and sets the selection of the operation button. Alternatively, the cursor may be displayed on the operation button.

The embodiments disclosed this time are to be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above description but by the scope of the claims, and is intended to include meanings equivalent to the scope of the claims and all modifications within the scope.

10 main body section, 11 image forming unit, 12 scanner unit, 13 automatic document transport unit, 14A, 14B paper feed tray, 15 transport path, 16 media sensor, 17 reverse transport path, 20 post-processing device, 31 controller, 32 fixed storage Device, 34 Operation Panel, 38 Bus, 39 Camera, 101 Intermediate Transfer Belt, 102, 103 Registration Roller, 104C, 104K, 104M, 104Y Image Forming Section, 105 Image Density Sensor, 111 Primary Transfer Device, 115 Secondary Next Transfer device, 113 paper feed roller, 120 fixing device, 220 punching device, 250 flat stitching processing unit, 260 saddle stitching processing unit, 271, 272, 273 discharge tray, 341 touch screen, 511, 512, 521, 522, 531 , 532, 541, 542 operation buttons, 591, 592, 593, 594 help screen, 701, 702, 703, 704 line-of-sight region, 1000 image forming apparatus, 1100 control unit, 1101 line-of-sight region identification unit, 1102 operation button identification unit, 1103 help information reading unit, 1104 display control unit, 1105 size determination unit, 1106 character recognition unit, 1200 document reading unit, 1300 storage unit, 3411 imaging unit, 3412 display unit, D7 data, NW network, P paper.

Claims (14)

An image forming apparatus,
An operation panel that accepts input operations by the operator,
Display control means for displaying a screen including a plurality of operation buttons on the operation panel;
Based on the operator's line of sight, area specifying means for specifying the area on the screen that the operator is looking at,
When there are two or more operation buttons in the specified area, one of the two or more operation buttons in the area is operated based on the information input to the image forming apparatus. An image forming apparatus comprising a button specifying unit that specifies a button. The image forming apparatus according to claim 1, wherein the display control unit causes the operation panel to display help content corresponding to the specified operation button based on the specification of the operation button. The information is document information, paper information, setting contents set based on the input operation on the operation panel, past operation history for the image forming apparatus, and area information indicating a use area of the image forming apparatus. The image forming apparatus according to claim 1, wherein the image forming apparatus is at least one of the following. Further comprising a scanner for reading the document,
The image forming apparatus according to claim 3, wherein information of the document is acquired by reading the document with the scanner. The image forming apparatus according to claim 4, wherein the document information is at least one of a size of the document, a character string of the document, a writing direction, a color of the character string, and a number of the document. Image data of the original document by the scanner is stored in a memory,
The printing according to the image data stored in the memory is performed when the operation panel receives an input operation instructing to start a printing operation after the image data is stored in the memory. Image forming apparatus. The image forming apparatus according to claim 6, wherein the image data stored in the memory is discarded when the reading operation is performed again by the scanner. The button specifying means is
When there are two or more operation buttons in the area, it is determined whether or not the content set by each of the operation buttons has a prohibitive relationship with the current setting content,
The one operation button is specified from the remaining operation buttons excluding the operation button in which the prohibited content is set from the two or more operation buttons in the area. 7. The image forming apparatus according to any one of items 7. The button specifying means is
When two or more operation buttons are present in the area, the setting content having the highest probability of being set together with the current setting content is determined based on the past operation history,
The image forming apparatus according to claim 3, wherein the operation button for performing the determined setting content is specified from among two or more operation buttons existing in the area. Further comprising a judging means for judging the size of the original,
The button specifying unit specifies the operation button corresponding to the size of the original document from the two or more operation buttons existing in the area when the size of the original document is not a standard size. The image forming apparatus according to any one of 1 to 9. Further comprising a recognition means for recognizing the character string of the original by optical character recognition,
11. The button specifying unit specifies the operation button according to the information of the character string of the document from the two or more operation buttons existing in the area, according to any one of claims 1 to 10. The image forming apparatus described. The operation panel includes an imaging unit,
The image forming apparatus according to claim 1, wherein the area specifying unit recognizes a line of sight of the operator based on data obtained by image pickup by the image pickup unit. The image forming apparatus according to claim 1, wherein the display control unit causes the operation panel to display an index indicating the region identified by the region identifying unit. A display control method for an image forming apparatus, comprising:
A step in which a controller of the image forming apparatus receives an input operation by an operator via an operation panel of the image forming apparatus;
A step in which the controller displays a screen including a plurality of operation buttons on the operation panel;
The controller, based on the line of sight of the operator, a step of specifying the region of the screen the operator is looking at,
When there are two or more operation buttons in the specified area, the controller selects one of the two or more operation buttons existing in the area based on the information input to the image forming apparatus. And a step of specifying one of the operation buttons.

Images