JP6572753B2 - Image forming apparatus, control method thereof, and program - Google Patents

Description

  The present disclosure relates to an image forming apparatus, a control method thereof, and a program, and in particular, an image forming apparatus that displays a screen that accepts button operations and gesture operations, a control method thereof, and the image forming apparatus. Regarding the program.

  2. Description of the Related Art Conventionally, an image forming apparatus such as an MFP (Multi-Functional Peripheral) displays a software button on a touch panel, and executes a process corresponding to the button when the software button is touched. Further, the image forming apparatus identifies the trajectory of the flick operation on the touch panel as a gesture, and executes processing corresponding to the identified gesture. Hereinafter, a flick operation that draws a locus corresponding to processing in the image forming apparatus is referred to as a “gesture operation”.

  When both button operation and gesture operation are accepted on a single screen, it is necessary to distinguish whether the current operation is a button operation or a gesture operation. This is because when the trajectory of the flick operation for the gesture operation includes a position where the software button is displayed, the image forming apparatus detects the button operation before the completion of the gesture operation. In this case, even if the user does not intend the control corresponding to the button, the image forming apparatus executes the control corresponding to the button.

  Japanese Unexamined Patent Publication No. 2013-117858 (Patent Document 1) discloses a technique for displaying a special button for switching between enabling / disabling of a touch operation in order to distinguish between a touch operation and a gesture operation on a single screen. To do. Japanese Patent Laying-Open No. 2009-151691 (Patent Document 2) discloses a technique for discriminating between a button operation and a gesture operation depending on whether or not a moving distance of a user's touch position is a predetermined distance or more. .

JP2013-117858A JP 2009-151691 A

  However, the technique disclosed in Patent Document 1 places a burden on the user to operate special buttons that are not directly related to touch operations and gesture operations.

  The technique disclosed in Patent Literature 2 is effective when an operation on a button is specified by a release trigger, that is, when the type of operation is determined when a user releases a touch. On the other hand, the technique disclosed in Patent Document 2 cannot be applied when an operation on a button is specified by a press trigger, that is, when the type of operation is determined at the start of touch. This is because when the type of operation is specified by the press trigger, the specification of the type of operation is completed before the movement distance of the touch position is specified.

  On the other hand, the press trigger has an advantage that the image forming apparatus can detect the user's button operation at an earlier stage than the release trigger.

  The present disclosure has been conceived in view of the actual situation, and an object of the present disclosure is to distinguish between a gesture operation and a button operation without imposing a burden on the user in an image forming apparatus that detects a button operation with a press trigger. That is.

  According to an aspect of the present disclosure, an image forming apparatus, a touch panel that displays a screen including one or more operation buttons, a communication unit that communicates with a terminal having an operation unit that accepts a touch operation, and the touch panel A control unit for controlling the image forming apparatus in response to an operation or an operation on the operation unit of the terminal, the control unit based on the relationship between the start position of the operation on the screen and the display position of the operation button on the screen, The operation button is configured to specify whether or not the operation button is an operation target, and the operation mode of the image forming apparatus includes one or more modes. In the first mode among the one or more modes, The control unit specifies only one of the gesture operation on the screen and the operation target operation button on the screen from the operation on the touch panel, From operation to create unit, the gesture operation on the screen, and, the operation buttons of the operation target on the screen, and is configured to identify only the other one, the image forming apparatus is provided.

Preferably, the control unit switches the operation mode between the first mode and a second mode different from the first mode, and in the second mode, from the operation on the touch panel to the gesture operation on the screen, The operation button to be operated on the screen is specified, and the contents of the first mode are displayed in the first mode.
More preferably, in the first mode, the control unit displays the content of the first mode on at least one of the touch panel and the operation unit when the screen is associated with the gesture operation and includes a press trigger button. It is configured as follows.

Preferably, the control unit switches the operation mode between the first mode and a second mode different from the first mode, and in the second mode, from the operation on the touch panel to the gesture operation on the screen, In addition, when an operation button to be operated on the screen is specified and an instruction to display a specific screen is acquired, a selection screen is displayed on the touch panel. The selection screen is a screen for selecting the first mode or the second mode . The control unit is configured to set the operation mode to the first mode or the second mode in accordance with the selection for the display of the selection screen when a specific screen is displayed on the touch panel.

Preferably, the control unit switches the operation mode between the first mode and a second mode different from the first mode, and in the second mode, from the operation on the touch panel to the gesture operation on the screen, In addition, an operation button to be operated on the screen is specified, and a selection screen is displayed on the touch panel . The selection screen is a screen for selecting the first mode or the second mode . The control unit is configured to set the operation mode to the first mode or the second mode in accordance with the selection on the display of the selection screen .

  Preferably, the control unit specifies only an operation button to be operated on the screen from an operation on the touch panel, specifies only a gesture operation on the screen from an operation on the operation unit, and includes one or more operation buttons for the terminal In an aspect indicating that the operation on is invalid, an instruction to display a screen including one or more operation buttons is transmitted.

  Preferably, when an operation other than a gesture operation on the operation unit is received from the terminal, the control unit transmits an instruction to the terminal to display that only the gesture operation on the screen is specified from the operation on the operation unit. Is configured to do.

  Preferably, the control unit acquires information for specifying a distance between the image forming apparatus and the terminal, and when the specified distance is equal to or greater than a predetermined distance, the control unit performs image formation on the terminal. An instruction to display information instructing to approach the apparatus is transmitted.

  According to another aspect of the present disclosure, a method for controlling an image forming apparatus, the step of displaying a screen including one or more operation buttons on a touch panel of the image forming apparatus, and a gesture operation on the screen from an operation on the touch panel And a step of specifying only one of the operation buttons to be operated on the screen, a step of receiving information representing the touch operation from a terminal having an operation unit that accepts the touch operation, and information received from the terminal From the gesture operation on the screen and the step of specifying only one of the operation buttons to be operated on the screen, specifying the operation button to be operated is based on the start position of the operation on the screen and the screen. Based on the relationship with the display position of the operation button, the operation button It includes identifying whether the control method of the image forming apparatus is provided.

  According to still another aspect of the present disclosure, a program executed by a computer of an image forming apparatus is provided. The program includes a step of displaying a screen including one or more operation buttons on the touch panel of the image forming apparatus on the computer, an operation on the touch panel, a gesture operation on the screen, and an operation button to be operated on the screen. A step of identifying only one of them, a step of receiving information representing the touch operation from a terminal having an operation unit that accepts a touch operation, a gesture operation on the screen, and an operation target on the screen from the information received from the terminal The step of specifying only one of the operation buttons is performed, and the operation button to be operated is specified based on the relationship between the start position of the operation on the screen and the display position of the operation button on the screen. Identifies whether the operation button is the operation target Including the door.

  According to the present disclosure, the operation button is a so-called press trigger having a characteristic that specifies whether or not the operation button is an operation target based on a relationship between a start position of the operation on the screen and a display position of the operation button on the screen. This is an operation button.

  The control unit identifies only one of the gesture operation and the operation target operation button from the operation on the touch panel, and identifies only the other from the operation on the operation unit of the terminal. That is, the control unit accepts only one type of operation among the two types of operations (gesture operation and button operation) only on the touch panel, and accepts the remaining one type of operation only on the operation unit of the terminal. Therefore, even when the button operation is specified by the press trigger, the image forming apparatus does not place a burden on the user to operate the button that is not directly related to the touch operation and the gesture operation. And can be distinguished.

FIG. 5 is a diagram for explaining an outline of an operation of the image forming apparatus according to the first embodiment. FIG. 5 is a diagram for explaining an outline of an operation of the image forming apparatus according to the first embodiment. 1 is a perspective view showing an overall configuration of MFP 100. FIG. FIG. 3 is a diagram illustrating an appearance of an operation panel 300. 2 is a block diagram showing a configuration of MFP 100. FIG. 2 is a block diagram showing a configuration of operation panel 300 of MFP 100. FIG. 3 is a block diagram showing a hardware configuration of a terminal 400. FIG. 6 is a flowchart of processing for handling a touch operation on touch panel 320 and display 404, which is executed by CPU 621 of MFP 100. It is a figure which shows an example of the screen displayed on the display 404 of the terminal 400 of 2nd Embodiment. It is a flowchart of the process which CPU621 of 2nd Embodiment performs. It is a figure which shows an example of the screen displayed in the terminal 400 of 3rd Embodiment. 10 is a flowchart of processing executed by a CPU 621 of the MFP 100 according to the third embodiment. 20 is a flowchart of processing executed by the CPU 621 of the MFP 100 according to the fourth embodiment. It is a figure which shows an example of the message displayed in the terminal 400 of 5th Embodiment. It is a flowchart of the process which CPU621 of 5th Embodiment performs. It is a figure which shows an example of the screen displayed in order to select an operation mode in 6th Embodiment. It is a flowchart of the 1st example of the process performed in 6th Embodiment. It is a flowchart of the 2nd example of the process performed in 6th Embodiment. It is a flowchart of the 3rd example of the process performed in 6th Embodiment.

  Embodiments of an image forming apparatus will be described below with reference to the drawings. In the following description, the same parts and components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, these descriptions will not be repeated.

[First Embodiment]
<1. Overview of operation of image forming apparatus>
1 and 2 are diagrams for explaining the outline of the operation of the image forming apparatus according to the first embodiment. As shown in FIG. 1, the image forming system 1 includes an MFP 100 and a terminal 400. The terminal 400 has a display 404. The MFP 100 has an operation panel 300. Operation panel 300 includes a touch panel.

  MFP 100 accepts an input of an operation on a screen displayed on the touch panel of operation panel 300 on display 404 of the touch panel and terminal 400. The method in which MFP 100 according to the first embodiment detects the operation content of the user includes two types of methods. The first type of method is a “touch operation” that determines whether a software button is displayed at a position touched by the user. The second type of method is a “gesture operation” in which the user determines whether the touch mode matches a pre-registered gesture pattern.

  The MFP 100 uses the input on the touch panel of the operation panel 300 only for one of these two types of methods, and uses the input for the display 404 of the terminal 400 only for the other two types of these methods.

  In the example illustrated in FIG. 1, the MFP 100 uses an input on the touch panel of the operation panel 300 only for detection by “touch operation”, and uses an input on the display 404 of the terminal 400 only for “gesture operation”. Hereinafter, a more specific description will be given with reference to FIG.

  An example of a screen displayed on the touch panel is shown as a screen 500 in the lower part of FIG. Screen 500 displays a menu for causing MFP 100 to execute an image forming operation as a plurality of software buttons. In the example of FIG. 1, the MFP 100 detects that the user's hand H has touched the point P10. A software button is displayed at the touched position. When MFP 100 detects a touch on point P10, MFP 100 executes control corresponding to the software button displayed on point P10.

  An example of the user's gesture operation on the display 404 of the terminal 400 is shown in the upper part of FIG. The example represents a pinch out. More specifically, two points (points P11 and P12) and two arrows (arrows A11 and A12) are shown in the display 404 of FIG. Point P11 indicates the start point of the touch of one finger of the user. An arrow A11 represents a locus of movement of the finger on the display 404. Point P12 indicates the start point of the touch of another finger of the user. An arrow A12 represents a locus of movement of the other finger on the display 404.

  Terminal 400 notifies MFP 100 that the user's touch operation is specified by points P11 and P12 and arrows A11 and A12. In response to this, MFP 100 specifies that a gesture pinch-out has been executed on display 404. FIG. 2 is a diagram illustrating an example of a display change on the touch panel of the operation panel 300.

  When pinch out is executed on the display 404 as shown in the upper part of FIG. 1, the MFP 100 displays the touch panel of the operation panel 300 from the upper screen 500A in FIG. 1 to the lower screen 500B in FIG. Change to The screen 500B is a screen that displays an enlarged upper left portion of the screen 500A.

<2. Appearance of MFP>
FIG. 3 is a perspective view showing the overall configuration of MFP 100.

  As shown in FIG. 3, MFP 100 includes an image reading unit 2 that optically reads a document to obtain image data, and an image forming unit 6 that prints an image on a sheet based on the image data. On the upper surface of the main body of the MFP 100, a feeder 4 for sending a document to the image reading unit 2 is disposed. Below the MFP 100, a plurality of paper feed units 9 for supplying printing paper to the image forming unit 6 are arranged. In the central part of the MFP 100, a tray 5 on which the printing paper on which an image is formed by the image forming unit 6 is discharged is disposed.

  An operation panel 300 is mounted on the front side of the upper part of the main body of the MFP 100. Operation panel 300 is installed in the main body of MFP 100 so as to be rotatable by hinge 300A.

  FIG. 4 is a diagram illustrating an appearance of the operation panel 300. Operation panel 300 includes a touch panel 320 and hard keys 330. Touch panel 320 displays information and accepts information input operations. As the touch panel 320, any type of touch panel such as a resistive film type or a capacitance type can be adopted.

  In the first embodiment, the operation on the operation panel 300 means that at least the hard key 330 or the touch panel 320 is operated.

<3. Hardware configuration of MFP>
FIG. 5 is a block diagram showing a configuration of MFP 100. As illustrated in FIG. 5, the MFP 100 includes a system controller 601, a memory 602, a network interface (I / F) 603, a printer engine 604, an output image processing unit 605, a storage device 606, and an imaging unit 607. An input image processing unit 608, an operation panel 300, and a power supply unit 609. The system controller 601 is connected to the memory 602, the network I / F 603, the printer engine 604, the output image processing unit 605, the storage device 606, the imaging unit 607, the input image processing unit 608, and the operation panel 300 via, for example, an internal bus. Has been.

  The power supply unit 609 is supplied with power from an external power supply. A system controller 601 controls power supply from the power supply unit 609 to each element in the MFP 100. For example, when a power button in operation panel 300 is operated, system controller 601 starts supplying power from power supply unit 609 to each element in MFP 100. For example, when the power button in operation panel 300 is operated again, system controller 601 stops the supply of power from power supply unit 609 to each element in MFP 100.

  A system controller 601 controls the entire MFP 100 for various jobs such as a scan job, a copy job, a mail transmission job, and a print job. The system controller 601 includes a CPU (Central Processing Unit) 621 and a ROM (Read Only Memory) 622.

  The CPU 621 executes a control program stored in the ROM 622. The ROM 622 stores various programs for controlling the operation of the MFP 100 and various fixed data. The CPU 621 reads data from the memory 602 and writes data to the memory 602.

  The memory 602 is, for example, a RAM (Random Access Memory), and is used, for example, for temporary storage of data or image data necessary when the CPU 621 executes a control program.

  The network I / F 603 communicates with external devices (including the terminal 400) via the network in accordance with instructions from the system controller 601. The network I / F 603 is used for communication according to a standard such as Bluetooth (registered trademark). Communication between the network I / F 603 and an external device is realized, for example, in a wireless local area network (LAN). An example of the network I / F 603 is a wired LAN interface mainly used for communication with a server or a PC (personal computer). Another example of the network I / F 603 is an interface of a wireless LAN (for example, a communication network according to the IEEE 802.11 standard) mainly used for communication with a mobile terminal or a PC.

  The printer engine 604 performs a printing process on paper or the like based on the print data processed by the output image processing unit 605. In particular, when the MFP 100 operates as a printer, the printer engine 604 prints an image. When the MFP 100 operates as a copier, the printer engine 604 prints an image read by the imaging unit 607.

  For example, when the image is printed, the output image processing unit 605 executes a conversion process for converting the data format of the image into a data format for printing.

  Storage device 606 is, for example, an HDD (Hard Disk Drive), and stores various data related to the operation of MFP 100. Storage device 606 may further store image data of a screen displayed on operation panel 300 of MFP 100.

The imaging unit 607 reads an image of the document and outputs it to the input image processing unit 608.
The input image processing unit 608 converts the format of the image data when the image capturing unit 607 reads an image (conversion processing).

  In MFP 100, the operation of MFP 100 as described in the present specification is realized by CPU 621 executing an appropriate program. The program executed by CPU 621 may be stored in ROM 622 as described above, may be stored in storage device 606, or may be stored in a storage medium that is detachable from MFP 100. . The storage medium for storing the program includes, for example, CD-ROM (Compact Disc-Read Only Memory), DVD-ROM (Digital Versatile Disk-Read Only Memory), USB (Universal Serial Bus) memory, memory card, FD (Flexible Disk), hard disk, SSD (Solid State Drive), magnetic tape, cassette tape, MO (Magnetic Optical Disc), MD (Mini Disc), IC (Integrated Circuit) card (excluding memory card), optical card, mask ROM, It is a medium for storing data in a nonvolatile manner such as EPROM and EEPROM (Electronically Erasable Programmable Read-Only Memory).

  The program according to the present disclosure may be a program module that is provided as part of an operating system (OS) of a computer and that calls necessary modules in a predetermined arrangement at a predetermined timing to execute processing. . In that case, the program itself does not include the module, and the process is executed in cooperation with the OS. A program that does not include such a module can also be included in the program according to the present invention.

  The program according to the present invention may be provided by being incorporated in a part of another program. Even in this case, the program itself does not include the module included in the other program, and the process is executed in cooperation with the other program. Such a program incorporated in another program can also be included in the program according to the present invention.

  The provided program product is installed in a program storage unit such as a hard disk and executed. The program product includes the program itself and a recording medium on which the program is recorded.

<4. Operation panel configuration>
FIG. 6 is a block diagram showing a configuration of operation panel 300 of MFP 100. The operation panel 300 includes a panel CPU 101, a ROM 102, a RAM 103, an LCD (Liquid Crystal Display) control unit 104, a VRAM (Video RAM), in addition to the touch panel 320 and the hard keys 330 described with reference to FIG. ) 105 and an input / output control unit 111.

  Panel CPU 101 controls operation panel 300 as a whole and communicates with system controller 601 (see FIG. 5) on the MFP 100 main body side.

  The ROM 102 stores a program for controlling the operation of the operation panel 300 and an image displayed on the touch panel 320. A RAM 103 is a work memory of the panel CPU 101.

  The LCD control unit 104 controls display on a display (for example, a liquid crystal display device) included in the touch panel 320. The LCD control unit 104 controls reading / writing of the VRAM 105. The image stored in the ROM 102 is stored in the VRAM 105 by the panel CPU 101 via the LCD control unit 104, read out, and output to the touch panel 320.

  Touch panel 320 includes a display and a touch sensor. The touch panel 320 specifies a touch position (or a position where the conductor is close) on the touch sensor, and outputs the specified position to the input / output control unit 111.

  Based on the display content on the display and the touch position output from the touch panel 320, the input / output control unit 111 identifies the item (menu, function, etc.) selected by the touch operation and outputs it to the panel CPU 101. The input / output control unit 111 may be realized as one function of the same processor as that constituting the panel CPU 101.

  The hard key 330 includes various keys such as a start key, a numeric keypad, and a panel reset key.

  In addition to the above elements, operation panel 300 may include various elements such as an LED (Light Emitting Diode) indicating the state of MFP 100 and a buzzer for notification.

  Input operations to the hard keys 330 and the touch panel 320 are transmitted to the system controller 601 via the input / output control unit 111 and the panel CPU 101 as necessary. Thus, MFP 100 main body operates based on an input operation on operation panel 300. For example, when a start key in the hard key 330 is pressed, information indicating that the start key is pressed is communicated to the MFP main body system control unit 100 via the hard key 330, the input / output control unit 111, and the panel CPU 101. Then, a series of copying operations is started.

<5. End-of-life configuration>
FIG. 7 is a block diagram illustrating a hardware configuration of the terminal 400. As illustrated in FIG. 7, the terminal 400 includes a CPU 401, a RAM 402, a storage device 403, a display 404, an input device 405, and a network I / F 406 as main components. The CPU 401, the RAM 402, the storage device 403, the display 404, the input device 405, and the network I / F 406 are mutually connected via an internal bus.

  The CPU 401 is an example of an arithmetic device that executes processing for controlling the overall operation of the terminal 400.

The RAM 402 functions as a work area when the CPU 401 executes processing.
The storage device 403 stores data of various programs such as an OS (Operating System) program or an application program executed by the CPU 401 and data used for executing these programs. Examples of the storage device 403 include a medium for storing data in a nonvolatile manner such as an EEPROM. The storage device 403 may be installed with a program downloaded via a network. The CPU 401 may implement the functions described in the present specification by executing a recording medium that is detachable from the main body of the terminal 400 and / or a program recorded in a storage medium on a network. obtain.

  A display 404 is a display device for displaying an image indicating a processing result of a program executed by the CPU 401.

  The input device 405 is an example of an input device for inputting information to the terminal 400 such as inputting an instruction regarding processing of an application being executed. Another example of the input device provided in the terminal 400 is a touch sensor provided on the display 404.

  Network I / F 406 communicates with an external device such as MFP 100 in accordance with a standard such as Bluetooth.

<6. Touch event detection>
CPU 621 of MFP 100 detects a touch event generated by a touch operation on touch panel 320. The CPU 401 of the terminal 400 detects a touch event generated by a touch operation on the display 404. CPU 401 transmits to MFP 100 the touch event detected by CPU 401.

<7. Button operation detection mode (2 types)>
The detection mode of button operation in MFP 100 is subdivided into two types. The first is a “press trigger” that specifies that the button is operated by the position at the start of the touch being positioned at the button display position. The second is a “release trigger” that specifies that the button has been operated when the position at the end of the touch (the position where the touch is released) is positioned at the button display position.

  Hereinafter, in the description of the detection mode in the button operation in this specification, two types of detection modes (“press trigger” and “release trigger”) can be distinguished as appropriate.

<8. Process flow>
FIG. 8 is a flowchart of a process for handling a touch operation on touch panel 320 and display 404, which is executed by CPU 621 of MFP 100.

  As shown in FIG. 8, in step S <b> 100, CPU 621 displays on touch panel 320 in accordance with an instruction from the user or an instruction from another device, or an instruction according to an execution result of a program installed in MFP 100. , Transition to a predetermined screen. Thereafter, the control proceeds to step S110.

  In step S110, CPU 621 detects a touch event on touch panel 320 or display 404. The touch event represents the start of touch or the end (release) of touch. Thereafter, the control proceeds to step S120.

  In step S120, the CPU 621 determines whether or not the touch event detected in step S110 is an operation on the display 404 of the terminal 400 (remote terminal). When CPU 621 determines that the operation is on display 404 (YES in step S120), control proceeds to step S190. On the other hand, when CPU 621 determines that the operation is not on display 404 (NO in step S120), control proceeds to step S130. If the touch event detected in step S110 is an operation on touch panel 320, control proceeds to step S130.

  In step S <b> 130, CPU 621 determines whether a touch event has occurred on a button in the screen displayed on touch panel 320. When CPU 621 determines that a touch event has occurred on the button (YES in step S130), control proceeds to step S140. When CPU 621 determines that a touch event has not occurred on the button (NO in step S130), control proceeds to step S160.

  In step S <b> 140, the CPU 621 notifies the operation panel 300 that the button determined to have been touched has been pressed. An example of the notification is a change in the display mode of the button (increase in brightness, change in color, etc.). Another example of the notification is voice output. In step S <b> 100, the CPU 621 may instruct the display 404 to display the same screen as that displayed on the touch panel 320. In this case, the CPU 621 may instruct the terminal 400 to notify the terminal 400 of pressing the button in step S140. Thereafter, the control proceeds to step S150.

  In step S150, the CPU 621 determines whether or not the button for which the touch event has been determined in step S130 is a button (press trigger button) whose presence or absence of operation is detected by the press trigger. When CPU 621 determines that the button is a press trigger button (YES in step S150), control proceeds to step S170. On the other hand, when CPU 621 determines that the button is not a press trigger button (NO in step S150), control proceeds to step S160.

  In step S160, the CPU 621 indicates that the touch event detected in step S110 indicates the release of the touch, and the release point of the touch event (the point where the touch is released) is the press point (the touch is the touch) of the previous touch event. It is determined whether or not the starting point is located on the same button. The “previous touch event” is, for example, a touch event representing the latest press point before this release. That is, it is a touch event when the touch released this time is started.

  If the CPU 621 determines that the touch event detected in step S110 indicates the release of the touch and the release point of the touch event is located on the same button as the press point (YES in step S160), the process proceeds to step S170. Advance control. In other cases (NO in step S160), CPU 621 returns control to step S110.

  If the CPU 621 determines in step S160 that the release point and the press point are different, the CPU 621 may return the control to step S110 after executing the process corresponding to the button displayed at the release point position.

  In step S170, the CPU 621 executes a process corresponding to the button displayed at the position where the touch event has occurred. For example, when the button is in the process of specifying the type of paper used for image formation, the CPU 621 sets the type of paper specified by the button as the type used for image formation. Thereafter, the control returns to step S110.

  In step S190, CPU 621 determines whether the operation on terminal 400 corresponds to a gesture operation. More specifically, it is determined whether or not the touch event received from the terminal 400 constitutes the content specified as the gesture operation. The determination target in step S190 includes a plurality of touch events. The plurality of touch events include an event indicating that the touch has started (press event), an event indicating that the touch has ended (released) (release event), and one or more events between these two events. It is. The CPU 621 can generate a trajectory of the touch position by using position information included in each of one or more events from the press event to the release event.

  A specific example of determining whether or not the operation at the terminal 400 corresponds to a gesture operation will be described. For example, the storage device 606 stores an operation pattern corresponding to one or more types of gesture operations and information for specifying processing corresponding to the gesture operations. An example of the operation pattern is an “operation for drawing a circle on the display 404”. An example of processing corresponding to the gesture operation is the start of user authentication. In this example, for example, when the user's finger draws a circle while touching display 404, CPU 401 transmits such an operation pattern on terminal 400 to MFP 100. In response to this, the CPU 621 determines whether or not the operation pattern received from the terminal 400 matches the pattern stored in the storage device 606. Determining that they match in this determination is an example of determining that the operation on the terminal 400 corresponds to a gesture operation. On the other hand, determining that the operation pattern received from the terminal 400 does not match the pattern stored in the storage device 606 is an example of determining that the operation on the terminal 400 does not correspond to the gesture operation.

  If CPU 621 determines in step S190 that the operation on terminal 400 corresponds to a gesture operation (YES in step S190), control proceeds to step S200. When CPU 621 determines that the operation on terminal 400 does not correspond to the gesture operation (NO in step S190), CPU 621 returns control to step S110.

  In step S200, the CPU 621 specifies the type of gesture operation corresponding to the operation on the terminal 400 in step S190. Thereafter, control proceeds to step S210.

  In step S210, the CPU 621 executes processing corresponding to the gesture operation specified in step S200. Thereafter, the control returns to step S110.

  In the first embodiment described above, the CPU 621 displays a button on the display screen of the touch panel 320 when the touch operation is performed on the touch panel 320 of the operation panel 300 (NO in step S120). It is determined whether or not it has occurred at a position (step S130), but it is not determined whether or not the touch operation corresponds to a gesture operation. On the other hand, when a touch operation is performed on the display 404 of the terminal 400 (YES in step S120), the CPU 621 determines whether or not the touch operation corresponds to a gesture operation (step S190), but the touch operation displays a button. It is not judged whether or not it has occurred at the position where it is placed.

  Therefore, even when the same screen as the screen displayed on the touch panel 320 is displayed on the display 404, the user needs to perform the gesture operation while avoiding the button display position when performing the gesture operation on the display 404. No. This is because even if the user touches a button on the display 404, the CPU 621 does not execute processing corresponding to the button.

[Second Embodiment]
In the second embodiment, CPU 621 of MFP 100 outputs an instruction to notify terminal 400 that a touch operation on a button on display 404 is not detected. FIG. 9 is a diagram illustrating an example of a screen displayed on the display 404 of the terminal 400 according to the second embodiment.

  In FIG. 9, an example of a screen displayed on display 404 by CPU 401 of terminal 400 in accordance with an instruction from MFP 100 is shown as screen 404A. The screen 404A displays a message “Please press the button on the main body panel and operate the gesture with this remote terminal.” This message is an example of the contents of the operation mode of MFP 100. For example, the CPU 621 may display the screen 404A on the touch panel 320 at the timing of displaying the screen 404A on the display 404. The CPU 621 may display the screen 404 </ b> A on the touch panel 320 instead of displaying it on the display 404 at the timing of displaying the screen 404 </ b> A on the display 404.

  FIG. 10 is a flowchart of processing executed by the CPU 621 according to the second embodiment. Compared with the process of the first embodiment shown in FIG. 8, the process of FIG. 10 includes the control of steps S102, S104, S106, S190, and S220.

  More specifically, the CPU 621 advances the control to step S102 after changing the display screen of the touch panel 320 in step S100.

  In step S102, CPU 621 determines whether or not the display screen of touch panel 320 is a screen for accepting a gesture operation. For example, the storage device 606 stores the screen type displayed on the touch panel 320 and the screen characteristics in association with each other. The screen characteristics include, for example, information that identifies the type of software button displayed on the screen. The characteristics of the screen may include information specifying the type of gesture operation accepted on the screen. When CPU 621 determines that the operation on the display screen of touch panel 320 corresponds to a gesture operation stored in association with the screen in storage device 606, CPU 621 executes a process associated with the gesture operation.

  In step S <b> 102, the CPU 621 determines whether or not the type of the display screen of the touch panel 320 is associated with the type of gesture operation in the storage device 606. If CPU 621 determines that the type of display screen is associated with the gesture operation (YES in step S102), CPU 621 advances the control to step S104. If CPU 621 determines that the type of display screen is not associated with the gesture operation (NO in step S102), CPU 621 advances the control to step S110.

  In step S <b> 104, the CPU 621 determines whether or not the display screen of the touch panel 320 includes a press trigger button (a button whose presence or absence of operation is detected by the press trigger) in the storage device 606. If it is determined that the display screen includes a press trigger button (YES in step S104), CPU 621 advances the control to step S106. If CPU 621 determines that the display screen does not include a press trigger button (NO in step S104), CPU 621 advances the control to step S110.

  In step S <b> 106, CPU 621 transmits an instruction to execute instruction display to terminal 400. The instruction display is, for example, to display a screen as shown as a screen 404A in FIG. 9 as described at the beginning of the second embodiment.

  In the second embodiment, when CPU 621 determines in step S160 that the release point and the press point are not located on the same button, the control proceeds to step S180.

  In step S <b> 180, CPU 621 notifies touch panel 320 of information instructing that the gesture operation is input to display 404. The notification is, for example, displaying a message on the touch panel 320 "This touch panel accepts only button operation. Please input gesture operation on the display of the remote terminal." Thereafter, the control proceeds to step S220.

  In step S220, the CPU 621 determines whether the process executed in step S170 or step S210 includes a screen transition. If the CPU 621 determines that the above process includes a screen transition (YES in step S220), the CPU 621 returns the control to step S102. On the other hand, if the CPU 621 determines that the above process does not include a screen transition (NO in step S220), the CPU 621 returns the control to step S110.

[Third Embodiment]
MFP 100 according to the third embodiment causes terminal 400 to change the display mode of buttons on the display screen, instead of causing terminal 400 to display screen 400A of FIG. 9 according to the second embodiment. FIG. 11 is a diagram illustrating an example of a screen displayed on the terminal 400 according to the third embodiment.

  In FIG. 11, the display screen of the terminal 400 is shown as a screen 400B. An example of a display screen on the touch panel 320 of the MFP 100 according to the third embodiment is a screen 500A in FIG. A screen 400B in FIG. 11 is obtained by graying out the operation buttons in the screen 500A.

  FIG. 12 is a flowchart of processing executed by the CPU 621 of the MFP 100 according to the third embodiment. The process of FIG. 12 includes the control of step S107 instead of step S106 (FIG. 10) as compared with the process of FIG.

  If it is determined that the display screen accepts a gesture operation (YES in step S102) and it is determined that the display screen includes a press trigger button (YES in step S104), CPU 621 advances the control to step S107.

  In step S <b> 107, CPU 621 outputs to terminal 400 an instruction to change the display mode of the buttons on the display screen on display 404. In response to this, the CPU 401 of the terminal 400 changes the display mode on the display 404 as shown in FIG. Thereafter, the control proceeds to step S110.

  In the third embodiment described above, the display mode on the display 404 of the terminal 400 is changed as shown, for example, as the screen 500B in FIG. A user who views the screen 500B on the display 404 of the terminal 400 can intuitively understand that the operation on the button is not accepted on the display 404.

[Fourth Embodiment]
Whenever an operation that cannot be specified as a gesture operation is performed on the display 404 of the terminal 400, the CPU 621 of the MFP 100 according to the fourth embodiment performs step S106 (the diagram of the second embodiment) on the terminal 400. Instruction display in 10) is instructed.

  FIG. 13 is a flowchart of processing executed by the CPU 621 of the MFP 100 according to the fourth embodiment. Compared to the process of the second embodiment of FIG. 10, in the process of FIG. 13, when the operation on the display 404 cannot be specified as a gesture operation in step S190, the CPU 621 returns the control to step S108. .

  In the fourth embodiment, every time an operation that cannot be specified as a gesture operation is performed, a display as shown on the screen 400A in FIG. This display indicates that the gesture operation is specified only from the operation on the display 404 of the terminal 400, and the button operation is specified only from the operation on the touch panel 320 of the MFP 100. Thereby, the user can surely understand the control contents in MFP 100.

[Fifth Embodiment]
MFP 100 according to the fifth embodiment acquires information for specifying the distance between MFP 100 and terminal 400, and when the distance is longer than a predetermined threshold, terminal 400 gives the user of terminal 400 to MFP 100. A message prompting you to approach is output.

  FIG. 14 is a diagram illustrating an example of a message displayed on the terminal 400 according to the fifth embodiment. A screen 404C is displayed on the display 404 of FIG. Screen 404C includes an example of a message that prompts the user to approach MFP 100 (button pressing requires operation on the main body panel. Move to the vicinity of the main body panel).

  For example, the CPU 621 acquires the IP (Internet Protocol) address of the terminal 400, acquires the position information of the terminal 400 from the server that manages the position information of the terminal 400 using the IP address, Using the position information of MFP 100, the distance between MFP 100 and terminal 400 is calculated. For example, the position information of MFP 100 is stored in storage device 606 in advance.

  FIG. 15 is a flowchart of processing executed by the CPU 621 according to the fifth embodiment. The process in FIG. 15 includes steps S109A and S109B instead of the control in step S106, as compared with the process in FIG.

  When CPU 621 determines in step S104 that the screen displayed on touch panel 320 includes a press trigger button, control proceeds to step S109A.

  In step S109A, CPU 621 determines whether the distance between MFP 100 and terminal 400 is equal to or less than a threshold value. When CPU 621 determines that the distance is equal to or less than the threshold (YES in step S109A), control proceeds to step S110. On the other hand, when CPU 621 determines that the distance exceeds the threshold (NO in step S109A), control proceeds to step S109B.

  In step S109B, the CPU 621 instructs the terminal 400 to display as shown as the screen 404C in FIG. Thereafter, the control proceeds to step S110.

[Sixth Embodiment]
MFP 100 according to the sixth embodiment causes the user to select an operation mode of MFP 100 and operates in the selected mode.

  FIG. 16 is a diagram illustrating an example of a screen displayed for selecting an operation mode. The screen 404X in FIG. 16 includes a message “Do you want to return to the normal remote operation mode? Or continue this operation mode as it is?”. In the sixth embodiment, the operation mode of MFP 100 includes two operation modes described below. In the message of FIG. 16, the continued operation mode is described as the first mode. The normal remote operation mode is described as the second mode.

  The operation mode of MFP 100 includes a first mode and a second mode. In the first mode, the CPU 621 specifies an operation button as a button operation target only from an operation on the touch panel 320 and specifies a gesture operation only from an operation on the display 404. In the second mode, the CPU 621 specifies both the operation button and the gesture operation that are subject to the button operation from the operation on the touch panel 320, and the operation button and the gesture operation that are subject to the button operation from the operation on the display 404. And specify both.

  In this specification, control for operating MFP 100 in the first mode is referred to as “divided touch mode”, and control for operating MFP 100 in the second mode is referred to as “normal touch mode”. There is.

  Screen 404X further includes a button 4041 for selecting the first mode (divided touch mode) and a button 4042 for selecting the second mode (normal touch mode).

  FIG. 17 is a flowchart of a first example of processing executed in the sixth embodiment. As shown in FIG. 17, when CPU 621 determines in step S102 that the display screen of touch panel 320 does not accept a gesture operation (NO in step S102), control proceeds to step S109C.

  If CPU 621 determines in step S104 that the display screen of touch panel 320 does not include a press release button (NO in step S104), the control proceeds to step S109C.

  In step S109C, CPU 621 displays, on touch panel 320 and / or display 404, a screen that accepts an input for selecting whether to continue control according to the divided touch mode. This screen is, for example, the screen 404X in FIG. Thereafter, when information for selecting continuation of control according to the divided touch mode is input to the screen display (for example, touching the button 4041), the CPU 621 advances the control to step S110. Through the control after step S110, the CPU 621 controls the MFP 100 according to the divided touch mode (first mode).

  On the other hand, when information for selecting not to continue control according to the split touch mode is input to the screen displayed in step S109C (for example, touching the button 4042), the CPU 621 in step S200, The operation mode is switched to the normal touch mode (second mode). In step S200, CPU 621 controls MFP 100 according to the normal touch mode.

  Even after switching the operation mode of the MFP 100 to the normal touch mode (second mode), the CPU 621 executes the control of Step S100 by the transition of the display on the touch panel 320, thereby dividing the operation mode of the MFP 100. It is possible to switch to the mode (first mode).

  FIG. 18 is a flowchart of a second example of processing executed in the sixth embodiment. In the process of FIG. 18, the CPU 621 advances the control to step S109C only when it is determined in step S102 that the display screen of the touch panel 320 does not accept the gesture operation (NO in step S102).

  FIG. 19 is a flowchart of a third example of processing executed in the sixth embodiment. In the process of FIG. 19, if the CPU 621 determines in step S102 that the display screen of the touch panel 320 does not accept a gesture operation (NO in step S102), the control proceeds to step S109E.

  In step S109E, it is determined whether or not the display screen of touch panel 320 includes a press trigger button. When the display screen includes a press trigger button, the CPU 621 advances the control to step S200. On the other hand, when the display screen does not include the press trigger button, the CPU 621 advances the control to step S109C.

  Each embodiment disclosed this time must be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims. In addition, the invention described in the embodiment and each modification is intended to be carried out independently or in combination as much as possible.

  1 image forming system, 100 MFP, 300 operation panel, 320 touch panel, 400 terminal, 401, 621 CPU.

Claims (10)

An image forming apparatus,
A touch panel for displaying a screen including one or more operation buttons;
A communication unit for communicating with a terminal having an operation unit for receiving a touch operation;
A control unit for controlling the image forming apparatus in response to an operation on the touch panel or an operation on the operation unit of the terminal,
The control unit is configured to identify whether or not the operation button is an operation target based on a relationship between a start position of an operation on the screen and a display position of the operation button on the screen.
The operation mode of the image forming apparatus includes one or more modes,
In the first mode among the one or more modes, the control unit includes:
From the operation on the touch panel, specify only one of the gesture operation on the screen and the operation button to be operated on the screen,
An image forming apparatus configured to specify only one of a gesture operation on a screen and an operation target operation button on the screen from an operation on the operation unit. The controller is
Switching the operation mode between the first mode and a second mode different from the first mode;
In the second mode, a gesture operation on the screen and an operation button to be operated on the screen are specified from the operation on the touch panel,
The image forming apparatus according to claim 1, wherein the image forming apparatus is configured to display contents of the first mode in the first mode. In the first mode, the control unit displays the content of the first mode on at least one of the touch panel and the operation unit when the screen is associated with a gesture operation and includes a press trigger button. The image forming apparatus according to claim 2, wherein the image forming apparatus is configured to. The controller is
Switching the operation mode between the first mode and a second mode different from the first mode;
In the second mode, a gesture operation on the screen and an operation button to be operated on the screen are specified from the operation on the touch panel,
It is configured to display a selection screen on the touch panel when an instruction to display a specific screen is acquired,
The selection screen is a screen for selecting the first mode or the second mode ,
The control unit is configured to set the operation mode to the first mode or the second mode according to a selection with respect to the display of the selection screen when the specific screen is displayed on the touch panel. The image forming apparatus according to claim 1. The controller is
Switching the operation mode between the first mode and a second mode different from the first mode;
In the second mode, a gesture operation on the screen and an operation button to be operated on the screen are specified from the operation on the touch panel,
It is configured to display a selection screen on the touch panel,
The selection screen is a screen for selecting the first mode or the second mode ,
Wherein the control unit, according to the selection for display prior Symbol screen for selection, and is configured to set the operation mode to the first mode or the second mode, the image formation according to claim 1 apparatus. The controller is
From the operation on the touch panel, specify only the operation button to be operated on the screen,
From the operation on the operation unit, specify only the gesture operation on the screen,
The terminal is configured to transmit an instruction to display a screen including the one or more operation buttons in a manner indicating that an operation on the one or more operation buttons is invalid. The image forming apparatus according to claim 1.   When the control unit receives an operation other than the gesture operation on the operation unit from the terminal, the control unit displays only the gesture operation on the screen from the operation on the operation unit. The image forming apparatus according to claim 6, wherein the image forming apparatus is configured to transmit an instruction to perform the instruction. The controller is
Obtaining information identifying the distance between the image forming apparatus and the terminal;
When the specified distance is greater than or equal to a predetermined distance, the terminal is configured to transmit an instruction to display information instructing the terminal to approach the image forming apparatus. The image forming apparatus according to any one of claims 1 to 7. An image forming apparatus control method comprising:
Displaying a screen including one or more operation buttons on the touch panel of the image forming apparatus;
Identifying only one of a gesture operation on the screen and an operation button to be operated on the screen from the operation on the touch panel;
Receiving information representing the touch operation from a terminal having an operation unit that accepts the touch operation;
Identifying from the information received from the terminal only one of the gesture operation on the screen and the operation button to be operated on the screen,
Specifying the operation button to be operated is specifying whether or not the operation button is the operation target based on the relationship between the start position of the operation on the screen and the display position of the operation button on the screen. A control method for an image forming apparatus. A program executed by a computer of an image forming apparatus,
The program is stored in the computer.
Displaying a screen including one or more operation buttons on the touch panel of the image forming apparatus;
Identifying only one of a gesture operation on the screen and an operation button to be operated on the screen from the operation on the touch panel;
Receiving information representing the touch operation from a terminal having an operation unit that accepts the touch operation;
From the information received from the terminal, to perform a gesture operation on the screen and a step of specifying only one of the operation buttons to be operated on the screen,
Specifying the operation button to be operated is specifying whether or not the operation button is the operation target based on the relationship between the start position of the operation on the screen and the display position of the operation button on the screen. Including the program.

Images