JP6435890B2 - Operation display system, operation display device, program - Google Patents

Description

  The present invention relates to an operation display system that performs display in cooperation with two operation display devices, an operation display device that performs display in cooperation with a mobile terminal, and a program.

  There is a technique in which an image forming apparatus such as a multifunction machine and a mobile terminal owned by a user are linked to display an operation panel of the image forming apparatus and a screen related to the mobile terminal juxtaposed on the panel (see Patent Document 1). . For example, when the whole of one screen cannot be displayed on the operation panel, the display area of the operation panel is substantially increased by displaying the continuation of the screen on a mobile terminal juxtaposed to the operation panel. Expanded.

  In the image forming system in which the image forming apparatus and the mobile terminal operate in cooperation, at least one of a user operation performed on the image forming apparatus and a user operation performed on the mobile terminal. According to the detected contents, one of the image forming apparatus and the mobile terminal is the main apparatus and the other is the slave apparatus, and the display contents of each apparatus is the main apparatus. There is a technique for switching in accordance with this (see Patent Document 2).

JP 2013-88506 A JP 2014-45418 A

  A mobile terminal used for cooperative display is usually not a dedicated device, but often has other functions, such as a function for wireless communication with a telephone or the outside. For this reason, during the linked display, incoming calls such as incoming calls and IM (Instant Message) / SMS (Short Message Service) occur on the mobile terminal side, or outgoing calls and messages from the mobile terminal, It may be necessary to start an application. In such a case, the mobile terminal is temporarily released from the operation panel, the cooperative display is canceled, and the screen corresponding to the event or application is automatically or manually switched. If the user is working using the display screen on the mobile terminal side during the cooperative display before such switching, the working screen disappears, which is inconvenient.

  The present invention is intended to solve the above-described problem. Even when an incoming call or the like is generated on one operation display device during linked display and the screen is switched, the operation during linked display is performed on the other display device. An object of the present invention is to provide an operation display system, an operation display device, and a program that can be continued on the operation display device.

  The gist of the present invention for achieving the object lies in the inventions of the following items.

[1] An operation display system including first and second operation display devices each including an operation unit and a display unit,
A first screen is displayed on the display unit of the first operation display device, and a second screen related to the first screen is displayed on the second operation display device juxtaposed to the first operation display device. A linked display control unit for controlling linked display;
A master-slave determination unit that determines which of the first screen and the second screen the user is mainly working during the linkage display;
A cancellation event detection unit that detects the occurrence of a predetermined event that triggers cancellation of the linkage display;
Have
The linkage control unit cancels the linkage display when the occurrence of the predetermined event is detected by the release event detection unit during the linkage display, and the determination result of the master-slave determination unit before the release Indicates that the second screen is the main screen, a screen including at least a part of the second screen is displayed on the display unit of the first operation display device after the release. Display system.

  In the above invention, when the occurrence of an event that triggers the cancellation of the cooperative display is detected, the cooperative display is canceled, and the user mainly performs the work using the second operation display device before the cancellation. If so, at least a screen including a part of the screen displayed on the second operation display device before the release is displayed on the first operation display device.

[2] The predetermined event is:
The positional relationship between the display unit of the first operation display device and the display unit of the second operation display device changes to a predetermined positional relationship unsuitable for the cooperative display;
Disconnection of communication between the first operation display device and the second operation display device;
The occurrence of an event having priority over the cooperative display in the second operation display device;
The operation display system according to [1], including at least one of receiving a linked display canceling operation in any one of the first operation display device and the second operation display device.

[3] The master-slave determination unit
Which display unit of the first operation display device or the second operation display device is the display unit last viewed by the user before the predetermined event occurs,
Which operation unit of the first operation display device or the second operation display device is the operation unit last operated by the user before the predetermined event occurs,
Which display unit of the first operation display device and the second operation display device is the display unit with a long time viewed by the user during the cooperative display,
Which of the first operation display device and the second operation display device is an operation unit with a long operation time or an operation unit with a large number of operations during linked display,
The determination is made based on at least one of the first operation display device and the second operation display device displaying a larger number of items set during cooperative display [1] Or the operation display system as described in [2].

[4] The cooperation control unit displays both the first screen and the second screen on the display unit of the first operation display device after the cancellation, by reducing at least one of the first screen and the first screen. A selection operation for either one of the screen and the second screen is received from the user, the selected screen is displayed on the display unit of the first operation display device, and the screen that has not been selected is displayed on the first operation display device The operation display system according to [1], wherein the operation display system is erased from the display unit.

  In the above invention, after canceling the cooperative display, the user receives a selection of which of the first screen and the second screen is displayed on the first operation display device.

[5] The cooperation control unit operates the first operation button displayed on the second screen before the release, and the second operation button belonging to the same group as the first operation button. Is displayed on the first screen, at least the first operation button and the second operation button are displayed on the display unit of the first operation display device after the release. The operation display system according to [1].

  In the above invention, even if the operation button is displayed on the first operation display device during the cooperative display, the second operation button belonging to the same group as the first operation button operated before the cancellation of the cooperative display is It is displayed on the first operation display device after the cancellation of cooperation.

[6] The cooperation control unit is an operation button belonging to the same group as the first operation button, and the operation button displayed on either the first screen or the second screen before the release is All the operation buttons that are displayed on the display unit of the first operation display device after the release and that do not belong to the same group as the first operation button that was displayed on the first screen before the release are The operation display system according to [5], which is not displayed on a display unit of the first operation display device.

  In the above-described invention, all the operation buttons that are displayed on the first screen or the second screen before canceling the cooperation display and belong to the same group as the operation buttons operated before the cooperation cancellation are released. The operation buttons that are displayed on the first operation display device and do not belong to the group are not displayed on the first operation display device after the cooperation is canceled.

[7] When the cooperation control unit restarts the cooperation display before the elapse of a predetermined time after the cancellation,
When the setting operation is received from the user in the first operation display device after the cancellation, the screen displayed on the display unit of the first operation display device is displayed on the display unit of the first operation display device. Display and display a screen related to the screen on the second operation display device to resume the cooperative display;
If the setting operation is not received from the user in the first operation display device after the release, the screen displayed on the display unit of the first operation display device before the release is displayed in the first operation display. Displaying the screen displayed on the display unit of the device and displayed on the display unit of the second operation display device before the release on the display unit of the second operation display device, and restarting the cooperative display. The operation display system according to any one of [1] to [6].

  In the above invention, when the operation panel is not operated during cooperation cancellation, the display is resumed by returning to the state before the cooperation cancellation, and when the operation panel is operated during cooperation cancellation, Since the operation of the first operation display device is mainly performed, the display screen of the first operation display device is maintained, the screen related to this screen is displayed on the second operation display device, and the cooperative display is resumed.

[8] Any one of [1] to [7], wherein the first operation display device is an operation panel of an image forming apparatus, and the second operation display device is a mobile terminal. Operation display system described in one.

[9] an operation unit;
A display unit;
A linked display control unit that controls a linked display that displays a first screen on the display unit and displays a second screen related to the first screen on a mobile terminal that includes an operation unit and a display unit juxtaposed to the device; ,
A master-slave determination unit that determines which of the first screen and the second screen the user is mainly working during the linkage display;
A cancellation event detection unit that detects the occurrence of a predetermined event that triggers cancellation of the linkage display;
Have
The linkage control unit cancels the linkage display when the occurrence of the predetermined event is detected by the release event detection unit during the linkage display, and the determination result of the master-slave determination unit before the release When the display indicates that the second screen is main, the operation display device displays a screen including at least a part of the second screen on the display unit after the release.

[10] The predetermined event is:
The positional relationship between the display unit of the device itself and the display unit of the mobile terminal changes to a predetermined positional relationship unsuitable for the cooperative display;
Disconnection of communication between the device and the mobile terminal;
Occurrence of an event that has priority over linked display on the mobile device,
The operation display device according to [9], including at least one of receiving a cooperative display canceling operation in either the device itself or the mobile terminal.

[11] The master-slave determination unit includes:
Whether the display unit that the user last viewed before the predetermined event occurs is the display unit of the own device or the mobile terminal,
Whether the operation unit last operated by the user before the predetermined event occurs is the operation unit of the own device or the mobile terminal,
Whether the display unit with a long time viewed by the user during linked display is the display unit of the own device or the mobile terminal,
Whether the operation unit with a long time operated by the user or the operation unit with a large number of operations during cooperation display is the operation unit of the own device or the mobile terminal,
The operation according to [9] or [10], wherein the determination is performed based on at least one of whether the device itself or the mobile terminal is displaying a larger number of items set during cooperative display Display device.

[12] The cooperation control unit displays at least one of the first screen and the second screen on the display unit of the self-device after the cancellation, and displays the first screen and the second screen. [9] characterized in that a selection operation for one of the screens is accepted from the user, the selected screen is displayed on the display unit of the own device, and the unselected screen is deleted from the display unit of the own device. The operation display device described.

[13] The cooperation control unit operates the first operation button displayed on the second screen before the release, and the second operation button belonging to the same group as the first operation button. Is displayed on the first screen, at least the first operation button and the second operation button are displayed on the display unit of the device after the release. [9] Operation display device.

[14] The cooperation control unit is an operation button belonging to the same group as the first operation button, and the operation button displayed on either the first screen or the second screen before the release is All the operation buttons that are displayed on the display unit of the device after the release and do not belong to the same group as the first operation button displayed on the first screen before the release are displayed on the display unit of the device after the release The operation display device according to [13], wherein the operation display device is not displayed.

[15] When the cooperation control unit restarts the cooperation display before the elapse of a predetermined time after the release,
When a setting operation is received from the user in the own device after the release, the screen displayed on the display unit of the own device is displayed on the display unit of the own device and a screen related to the screen is displayed on the mobile terminal. If the setting operation is not received from the user in the own device after the release, the screen displayed on the display unit of the own device before the release is displayed on the display unit of the own device, and Any one of [9] to [14], wherein the screen displayed on the display unit of the mobile terminal before the cancellation is displayed on the display unit of the mobile terminal and the cooperative display is resumed. The operation display device described in 1.

[16] An information processing apparatus
[9] A program that functions as the operation display device according to any one of [15].

  According to the operation display system, the operation display device, and the program according to the present invention, even when an incoming phone call or the like occurs in one operation display device that is in the cooperative display and the screen is switched, the operation in the cooperative display is performed on the other side. The operation display device can continue.

It is a figure which shows the structural example of the operation display system which concerns on embodiment of this invention. 1 is a block diagram illustrating a schematic electrical configuration of an image forming apparatus. It is a block diagram which shows schematic structure of a mobile terminal. 3 is a diagram illustrating an example of a state in which a mobile terminal is juxtaposed in the vicinity of an operation panel 30 of the image forming apparatus and cooperative display is executed. FIG. It is a figure which shows an example of the display state at the time of cancellation | release of cooperation in the operation display system of 1st Embodiment. 4 is a flowchart illustrating an operation related to the cancellation of cooperation between the image forming apparatus and the mobile terminal of the operation display system according to the first embodiment. It is a flowchart which shows the detail of the master-slave monitoring process (S103 and S203 of FIG. 6). It is a flowchart which shows the detail of a cooperation monitoring process (S104 and S204 of FIG. 6). It is a figure which shows an example of the display operation at the time of cancellation | release of cooperation in 2nd Embodiment. It is a figure which shows the continuation of FIG. It is a figure which shows the other example of a continuation of FIG. 12 is a flowchart illustrating an operation related to the cancellation of cooperation between the image forming apparatus and the mobile terminal of the operation display system according to the second embodiment. It is a figure which shows an example of the display operation at the time of cancellation | release of cooperation in 3rd Embodiment. It is a figure which shows the other example of the display operation at the time of the cooperation cancellation | release in 3rd Embodiment. 14 is a flowchart illustrating an operation related to cancellation of cooperation between an image forming apparatus and a mobile terminal of an operation display system according to a third embodiment. It is a figure which shows an example of the display operation at the time of the cooperation cancellation | release in 4th Embodiment. It is a figure which shows an example of the display operation in the time of cooperation cancellation | release in 4th Embodiment, and the cooperation resumption. It is a figure which shows the other example of the display operation at the time of cooperation cancellation | release in the 4th Embodiment and cooperation resumption. 14 is a flowchart illustrating operations related to the cooperation cancellation and the cooperation restart of the image forming apparatus and the mobile terminal of the operation display system according to the fourth exemplary embodiment.

  Hereinafter, various embodiments of the present invention will be described with reference to the drawings.

<First Embodiment>
FIG. 1 shows an example of an operation display system 3 according to an embodiment of the present invention. The operation display system 3 includes an image forming apparatus 10 provided with an operation panel 30 as a first operation display device, and a mobile terminal 40 as a second operation display device. In addition to being used alone, the mobile terminal 40 is juxtaposed on the operation panel 30 of the image forming apparatus 10 and used for cooperative display described later.

  The image forming apparatus 10 is a copy job that optically reads a document and prints a duplicate image on a recording sheet, a scan job that stores image data of the read document as a file, or transmits the file to an external terminal via a network, A so-called function that executes a plurality of types of jobs, such as a print job for forming and printing out an image related to print data received from a terminal device over a network on a recording paper, and a FAX job for transmitting and receiving image data according to a facsimile procedure. It is a multifunction machine. In this description, the image forming apparatus 10 is also referred to as an MFP.

  The mobile terminal 40 is obtained by installing a predetermined application program in a portable information processing apparatus such as a general-purpose tablet terminal or a smartphone, and functions related to cooperative display and the like are realized by executing the application program.

  The image forming apparatus 10 can perform cooperative display by operating in conjunction with the mobile terminal 40 when the operation panel 30 of the own apparatus and the mobile terminal 40 are in parallel and in a communicable state. In the cooperative display, screens such as an operation screen and a preview screen are displayed using the display units of both the operation panel 30 and the mobile terminal 40.

  For example, in linked display, when the entire display target screen cannot be displayed on the display unit of the operation panel 30, the portion that cannot be displayed is displayed on the mobile terminal 40 juxtaposed on the operation panel 30. Alternatively, a screen related to the screen being displayed on the operation panel 30 is displayed on the mobile terminal 40.

  FIG. 2 is a block diagram showing a schematic electrical configuration of the image forming apparatus 10. The image forming apparatus 10 includes a CPU (Central Processing Unit) 11 as a control unit that comprehensively controls the operation of the image forming apparatus 10. The CPU 11 includes a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a nonvolatile memory 14, a hard disk device 15, an automatic document feeder (ADF) 16, a network communication unit 17, and an image reading unit. 18, an image processing unit 19, a printer unit 21, a facsimile communication unit 22, an authentication unit 23, an operation panel 30, and the like are connected.

  The CPU 11 is based on an OS (Operating System) program, and executes middleware, application programs, and the like. Various programs are stored in the ROM 12, and each function of the image forming apparatus 10 (including the operation panel 30) is realized by the CPU 11 executing various processes according to these programs.

  Although the CPU 11 controls the operation panel 30 here, the operation panel 30 may be controlled by a separate CPU provided exclusively for the operation panel 30.

  The RAM 13 is used as a work memory for temporarily storing various data when the CPU 11 executes processing based on a program, an image memory for storing image data, and the like.

  The nonvolatile memory 14 is a memory (flash memory) whose stored contents are not destroyed even when the power is turned off, and is used for storing various setting information.

  The hard disk device 15 is a large-capacity nonvolatile storage device, and stores various programs and data in addition to print data and display screen data.

  The network communication unit 17 performs a function of communicating with an external terminal, a server, or the like connected to a network such as a LAN (Local Area Network).

  The image reading unit 18 performs a function of optically reading a document and acquiring image data. The image reading unit 18 sequentially, for example, a light source that irradiates light on a document, a line image sensor that receives the reflected light for one line in the width direction, and a line-by-line reading position in the length direction of the document. An optical path composed of a moving unit for moving, a lens, a mirror, and the like for guiding reflected light from the document to the line image sensor to form an image, and a conversion unit for converting an analog image signal output from the line image sensor into digital image data And so on.

  The automatic document feeder 16 feeds the document set on the platen one by one from the top one by one in order, passes the reading position of the image reading unit 18 and discharges it to a predetermined discharge position. Fulfill. The image reading unit 18 has a function of reading a document placed on the platen glass and a function of sequentially reading a document conveyed by the automatic document conveyance unit 16.

  The image processing unit 19 performs rasterization processing for converting print data into image data, image data compression and expansion processing, in addition to processing such as image enlargement / reduction and rotation.

  The printer unit 21 has a function of forming an image corresponding to image data on a recording sheet. Here, it has a recording paper transport device, a photosensitive drum, a charging device, a laser unit, a developing device, a transfer separation device, a cleaning device, and a fixing device, and forms an image by an electrophotographic process. It is configured as a so-called laser printer. Other methods may be used for image formation.

  The facsimile communication unit 22 performs a function of transmitting / receiving image data to / from an external device having a facsimile function through a telephone line.

  The authentication unit 23 authenticates a user who uses the image forming apparatus 10. The authentication method may be any password, fingerprint, vein, or the like.

  The operation panel 30 includes a display unit 31, an operation unit 32, a close proximity wireless communication unit 33, and a connection detection unit 34. The operation panel 30 has various functions such as a function for displaying various operation screens and setting screens, and a job input from a user. It fulfills the function of accepting operations.

  The display unit 31 includes a liquid crystal display (LCD ... Liquid Crystal Display) and its driver. The operation unit 32 includes various hard keys such as a start button and a numeric keypad, and a touch panel 32 a provided on the display surface of the display unit 31. The touch panel 32a detects a coordinate position where the display surface of the display unit 31 is touched with a touch pen or a finger. The operation unit 32 receives a selection operation, a flick operation, a drag operation, and the like through the touch panel 32a. The detection method of the touch panel 32a may be arbitrary, such as capacitance, analog / digital resistance film, infrared rays, ultrasonic waves, electromagnetic induction, and the like.

  The proximity wireless communication unit 33 performs a function of performing wireless communication with the mobile terminal 40 disposed in the vicinity of the operation panel 30. The communicable distance of the close proximity wireless communication unit 33 is about 50 cm. Various short-range wireless communication systems can be adopted as the communication system.

  The position detection unit 34 detects in which direction the mobile terminal 40 is juxtaposed with respect to the operation panel 30.

  FIG. 3 is a block diagram illustrating a schematic configuration of the mobile terminal 40. The mobile terminal 40 has a CPU 41 that performs overall control of the operation of the mobile terminal 40. A ROM 42, a RAM 43, a nonvolatile memory 44, an authentication unit 45, an operation unit 46, a display unit 47, a network communication unit 48, and a close proximity wireless communication unit 49 are connected to the CPU 41 through a bus.

  Various programs and data are stored in the ROM 42 and the nonvolatile memory 44, and each function of the mobile terminal 40 is realized by the CPU 41 executing various processes according to these programs. The RAM 43 is used as a work memory for temporarily storing various data when the CPU 41 executes a program and a memory for temporarily storing display data.

  The nonvolatile memory 44 is a memory that retains stored contents even when the power is turned off. The non-volatile memory 44 displays a screen for cooperative display based on a document or image file, various setting contents, user information, terminal information (network address, etc.), an application program, and an instruction from the operation panel 30. A program that performs the function is stored.

  The authentication unit 45 authenticates a user who uses the mobile terminal 40.

  The display unit 47 is configured by a liquid crystal display or the like.

  The operation unit 46 is provided on the entire display surface of the display unit 47, and includes a touch panel 46a that detects a position touched by a finger or a pen, and a small number of hard keys (operation switches).

  The network communication unit 48 has a function of communicating with various external devices by connecting to a network by wireless communication. The close proximity wireless communication unit 49 performs a function of performing wireless communication with the close proximity wireless communication unit 33 of the nearby operation panel 30. The communicable distance, communication method, and the like are the same as those of the proximity wireless communication unit 33 of the operation panel 30.

  FIG. 4 shows an example of a state in which the mobile terminal 40 is juxtaposed in the vicinity of the operation panel 30 of the image forming apparatus 10 and cooperative display is executed. In this example, a function selection screen for selecting a function to be used from among the functions of the image forming apparatus 10 is displayed in cooperation. On the screen on the operation panel 30 side, a cooperation start button 52 for instructing start of cooperation display and a cooperation release button 53 for instructing cancellation of cooperation display are displayed. In FIG. 4, since the cooperation display is already being performed, the cooperation start button 52 is displayed in a state where it cannot be pressed (grayed out).

  On the function selection screen, many operation buttons 51 for selecting a function are arranged in a horizontal row. A part of the selection screen is displayed on the operation panel 30, and a part of the selection screen that is continued to the right of the part displayed on the operation panel 30 is displayed on the mobile terminal 40. A screen displayed on the operation panel 30 is referred to as an MFP side cooperation screen 61, and a screen displayed on the mobile terminal 40 is referred to as a mobile side cooperation screen 62. When the user wants to display a portion of the function selection screen that is not displayed at the time of the current display on the operation panel 30 and the mobile terminal 40, the screen is scrolled by performing a flick operation in that direction. Yes.

  In addition, the screen displayed in cooperation is not limited to such a slider-type screen. For example, when an operation button (for example, an address book selection button) on the operation panel 30 side is pressed, a related screen (a list of addresses registered in the address book) is displayed on the mobile terminal 40. A screen having a hierarchical structure may be used and may be arbitrary.

  In the example of FIG. 4, the user operates the operation button 51 displayed on the mobile terminal 40 and works on the screen on the mobile terminal 40 side. In this state, for example, when there is an incoming call on the mobile terminal 40 side, the user moves the mobile terminal 40 away from the operation panel 30 and close to his / her ear to answer the call.

  FIG. 5 shows this state. At this time, the operation panel 30 of the image forming apparatus 10 detects that the mobile terminal 40 is far away from a position suitable for cooperative display, and cancels the cooperative display. Hereinafter, the cancellation of the cooperation display is abbreviated as the cooperation cancellation. Here, since the user was working on the mobile terminal 40 immediately before the cancellation of the cooperation, the screen (mobile-side cooperation screen 62) displayed on the mobile terminal 40 before the cancellation of the cooperation is the operation panel 30 after the cancellation of the cooperation. Is displayed on the display unit 31.

  On the other hand, when the user has not been working by looking at the mobile side cooperation screen 62 on the mobile terminal 40 side before the incoming call, for example, the user performs the work by looking at the MFP side cooperation screen 61 displayed on the operation panel 30. In such a case, the same MFP-side cooperation screen 61 as before the cooperation cancellation is displayed on the display unit 31 of the operation panel 30 even after the cooperation cancellation.

  As described above, when canceling the cooperative display in order to use the mobile terminal for purposes other than the cooperative display, it is determined whether or not the user has mainly used the mobile terminal 40 before the cooperative cancellation. If the user is mainly working with the mobile terminal 40 before canceling the cooperation, the screen displayed on the mobile terminal 40 before the cancellation of the cooperation is displayed on the operation panel 30 after the cancellation of the cooperation. Therefore, the user can use the screen displayed on the operation panel 30 to continue the work before canceling the cooperation even after canceling the cooperation.

  In addition, when the user is working mainly using the operation panel 30 before canceling the cooperation, the display content of the operation panel 30 is maintained even after canceling the cooperation. This can be done continuously even after canceling the cooperation.

  As described above, in the operation display system 3, when the cooperative display is canceled in order to use the mobile terminal for purposes other than the cooperative display, the user uses either the operation panel 30 or the mobile terminal 40 before the cooperative cancellation. Even if the operation such as the setting operation is performed, the operation can be continued on the screen displayed on the operation panel 30 after the cancellation of the cooperation.

  4 and 5 are triggered by the occurrence of an event that the user has moved the mobile terminal 40 away from the operation panel 30 in order to respond to an incoming call (change in the distance between the operation panel 30 and the mobile terminal 40). However, the event that causes the cancellation of the linkage is not limited to this. For example, there are the following. At least one of these is employed.

(1) The mobile terminal 40 is turned over.
(2) The relative positions of the operation panel 30 and the mobile terminal 40 have changed.
(3) The display has changed due to the occurrence of an event (such as an incoming call or mail reception) prioritized over the cooperative display on the mobile terminal 40.
(4) A cooperative display cancellation operation is received from the user on the operation panel 30 or the mobile terminal 40.
(5) Communication between the operation panel 30 and the mobile terminal 40 is disconnected.
In the determination of which of the MFP-side cooperation screen 61 displayed on the operation panel 30 and the mobile-side cooperation screen 62 displayed on the mobile terminal 40 are mainly used during the cooperation display, the following determination criteria are used: At least one of them is adopted.

(A) Which display unit of the operation panel 30 or the mobile terminal 40 is the display unit (screen) that the user last viewed before an event that causes the cancellation of cooperation occurs?
(B) Which operation unit of the operation panel 30 mobile terminal 40 is the operation unit last operated by the user before an event that causes the cancellation of cooperation occurs.
(C) Which display unit of the operation panel 30 or the mobile terminal 40 is the display unit with the long time viewed by the user during the cooperative display?
(D) Which operation unit of the operation panel 30 or the mobile terminal 40 is the operation unit with a long operation time or the operation unit with a large number of operations performed by the user during the cooperative display.
(E) Which of the operation panel 30 and the mobile terminal 40 is displaying more items set during cooperative display?

  FIG. 6 is a flowchart showing an operation related to the cancellation of cooperation between the image forming apparatus 10 (MFP) and the mobile terminal 40 of the operation display system 3 according to the first embodiment. When the cooperative display is started (step S101; Yes), the image forming apparatus 10 displays the MFP-side cooperative screen 61 that is handled by the operation panel 30 of the image forming apparatus 10 in the cooperative display (step S102). Similarly, when the cooperative display is started (step S201; Yes), the mobile terminal 40 displays the mobile-side cooperative screen 62 that the mobile terminal 40 takes charge of in the cooperative display (step S202).

  Here, the image forming apparatus 10 creates both the MFP-side cooperation screen 61 and the mobile-side cooperation screen 62, displays the MFP-side cooperation screen 61 on the operation panel 30, and transmits the mobile-side cooperation screen 62 to the mobile terminal 40. And displayed on the mobile terminal 40.

  When the cooperative display is started, the image forming apparatus 10 and the mobile terminal 40 respectively perform master / slave monitoring processing (processing that performs the function of the master / slave determination unit) for monitoring which of the operation panel 30 and the mobile terminal 40 the user is working on. And the cooperation monitoring process (process which fulfill | performs the function as a cancellation | release detection part) which monitors generation | occurrence | production of the event which cancel | releases a cooperation display is repeatedly performed (step S103, S104, S105; No, S203, S204, S205; No). . In the master-slave monitoring process, the monitoring status of the mobile terminal 40 is notified to the image forming apparatus 10. Further, in the cooperative monitoring process, when an event for canceling the cooperative display occurs on the own apparatus side, the image forming apparatus 10 and the mobile terminal 40 each notify the other.

  When the occurrence of an event for canceling the cooperative display is detected (step S105; Yes), the image forming apparatus 10 cancels the cooperative display and determines whether the user is mainly working on the mobile terminal 40 during the cooperative display. Is confirmed (step S106). If the user is mainly working on the mobile terminal 40 during the cooperation display (step S106; Yes), the mobile-side cooperation screen 62 displayed on the mobile terminal 40 immediately before canceling the cooperation is displayed on the operation panel 30. It displays on the display part 31 (step S107), and complete | finishes a process.

  When the user is not mainly working on the mobile terminal 40 during the cooperation display (step S106; No), the MFP-side cooperation screen 61 displayed on the operation panel 30 immediately before canceling the cooperation is displayed on the operation panel 30. The display is continued on the part 31 as it is, and the process is terminated.

  When the occurrence of an event for canceling the cooperative display is detected (step S205; Yes), the mobile terminal 40 cancels the cooperation and displays an application screen such as an incoming call screen (step S206), and ends the process. To do.

  FIG. 7 shows details of the master-slave monitoring process (S103 and S203 in FIG. 6). The mobile terminal 40 monitors whether or not an operation has been performed on the terminal itself (steps S221 and S222; No), and when the operation is detected (step S222; Yes), notifies the image forming apparatus 10 of the detection of the operation ( Step S223).

  The image forming apparatus 10 monitors whether or not an operation on the operation panel 30 has been performed and notification from the mobile terminal 40 (step S121). From the result of the monitoring, it is determined whether the user is working mainly using the operation panel 30 of the image forming apparatus 10 or is working mainly using the mobile terminal 40 (step S122). . When the user is working mainly using the operation panel 30 of the image forming apparatus 10, the fact that the MPF is the main one is returned (step S123), and this processing is terminated. If the user is working mainly using the mobile terminal 40, the return value is that the mobile terminal is the main one (step S124), and the process is terminated. In addition to the operation, the above-described master-slave relationship criterion may be monitored.

  FIG. 8 shows details of the cooperation monitoring process (S104 and S204 in FIG. 6). FIG. 8 shows a case in which cooperation cancellation is monitored by communication status, coordination cancellation is monitored by distance, position, and orientation, and coordination cancellation is monitored by a release operation or event occurrence. Each of the image forming apparatus 10 and the mobile terminal 40 notifies the other party of that when an event for canceling the cooperative display occurs on the own apparatus side.

  The image forming apparatus 10 checks whether communication with the mobile terminal 40 is maintained (step S141). If communication is not possible (step S141; No), the cooperation release is returned as a return value (step S148). The process ends. Similarly, the mobile terminal 40 checks whether communication with the operation panel 30 is maintained (step S241). If communication is not possible (step S241; No), the cooperation cancellation is returned as a return value (step S246). The process is terminated.

  If communication is possible (step S141; Yes), the image forming apparatus 10 checks the distance to the mobile terminal 40, the position and orientation of the mobile terminal 40 (step S142). When any one of the distance to the mobile terminal 40 and the position and orientation of the mobile terminal 40 cannot continue the cooperative display (Step S143; No), after notifying the mobile terminal 40 of the cooperation cancellation (Step S144), the cooperation The release is returned as a return value (step S148), and the process is terminated.

  As the position of the mobile terminal 40, for example, if the mobile terminal 40 is located at a position overlapping the operation panel 30, it is determined that the cooperative display is not possible. For example, the direction is determined to be incapable of cooperative display when, for example, it is turned over. The distance can be determined using the radio field strength of wireless communication. The distance, position, and orientation can be recognized by analyzing a camera image provided on the operation panel 30 or the like. The method for recognizing the distance, position, and orientation may be arbitrary.

  When the mobile terminal 40 receives a notification of cooperation cancellation from the image forming apparatus 10 (step S242; Yes), the mobile terminal 40 sets the cooperation cancellation as a return value (step S246) and ends the process.

  If the distance, position, and orientation are in a state where the cooperative display can be continued (step S143; Yes), the image forming apparatus 10 has received an operation for canceling the cooperation (operation of the cooperation start button 52 in FIG. 4) from the user. This is checked (step S145). When an operation for canceling the cooperation is received (step S145; Yes), the mobile terminal 40 is notified of the cancellation of the cooperation (step S146), and the process is terminated after returning the cooperation to the return value (step S148).

  When the mobile terminal 40 receives a notification of cooperation cancellation from the image forming apparatus 10 (step S242; Yes), the mobile terminal 40 sets the cooperation cancellation as a return value (step S246) and ends the process.

  In the mobile terminal 40, when an event such as an incoming call occurs (step S243; Yes), when an operation for canceling the cooperation is received from the user, or when another application program is activated (step S244; Yes). After notifying the image forming apparatus 10 of the cooperation cancellation (step S245), the cooperation cancellation is returned as a return value (step S246), and the process is terminated.

  When the image forming apparatus 10 receives the notification of the cooperation cancellation from the mobile terminal 40 (step S147; Yes), the image forming apparatus 10 sets the cooperation cancellation as a return value (step S148) and ends the processing. If the cooperation is not canceled due to any factor (step S147; No), the image forming apparatus 10 sets “no cooperation cancellation” as a return value (step S149) and ends the process. Similarly, in the case where the mobile terminal 40 does not cancel the cooperation due to any factor (step S244; No), the mobile terminal 40 sets “no cooperation cancellation” as a return value (step S247) and ends the processing.

<Second Embodiment>
The operation display system 3 according to the second embodiment inquires of the user which of the MFP-side cooperation screen 61 and the mobile-side cooperation screen 62 is to be displayed on the display unit 31 of the operation panel 30 when the cooperation is canceled. It is designed to receive a selection. A description of the same parts as those in the first embodiment will be omitted as appropriate.

  FIG. 9 shows an example of a display operation at the time of cancellation of cooperation in the second embodiment. FIG. 2A shows a state where the operation panel 30 of the image forming apparatus 10 and the mobile terminal 40 are performing the cooperative display. The user is working mainly using the mobile side cooperation screen 62 displayed on the mobile terminal 40. When the cooperation is canceled at this time, as shown in FIG. 4B, the operation panel 30 has the MFP-side cooperation screen 61 displayed on the operation panel 30 and the mobile terminal 40 displayed before the cooperation cancellation. The side cooperation screen 62 is displayed with at least one reduced.

  Here, after canceling the cooperation, a reduced screen 61a before canceling the MFP side cooperation is displayed on the right side of the screen so as to fit within the display area of the display unit 31 of the operation panel 30. On the left side of the screen, a reduced screen 62a before canceling the mobile side cooperation is displayed, which is a reduced version of the contents of the mobile side cooperation screen 62. If the operation panel 30 fits in the display area, only the MFP-side cooperation screen 61 may be reduced, and the mobile-side cooperation screen 62 may be displayed in the same size as that displayed on the mobile terminal 40.

  Next, as shown in FIG. 10 (a), when the user touches the MFP-side cooperation cancellation reduction screen 61a displayed on the operation panel 30, as shown in FIG. 10 (b), the operation panel 30 is displayed. The MFP-side cooperation screen 61a is enlarged and the same MFP-side cooperation screen 61 as that during cooperation display is displayed. The reduced screen 62a before canceling the cooperation on the mobile side is deleted.

  On the other hand, as shown in FIG. 11 (a), when the user reduces the mobile-side cooperation cancellation reduction screen 62a displayed on the operation panel 30, the operation panel 30 is displayed as shown in FIG. 11 (b). Is enlarged and displayed on the mobile-side cooperation pre-cancel reduction screen 62a. The reduction screen 61a before canceling cooperation on the MFP side is deleted.

  FIG. 12 is a flowchart showing an operation related to the cancellation of the cooperative display of the image forming apparatus 10 (MFP) and the mobile terminal 40 of the operation display system 3 according to the second embodiment. The same steps as those in FIG. 8 are denoted by the same step numbers. The operation of the mobile terminal 40 is the same as that in FIG.

  When the cooperative display is started (step S101; Yes), the image forming apparatus 10 displays the MFP-side cooperative screen (step S102). Similarly, when the cooperative display is started (step S201; Yes), the mobile terminal 40 displays the mobile side cooperative screen (step S202).

  During the linked display, the image forming apparatus 10 and the mobile terminal 40 cancel the master / slave monitoring process for monitoring which of the operation panel 30 and the mobile terminal 40 is mainly used by the user and the linked display, respectively. The cooperative monitoring process for monitoring the occurrence of the event is repeatedly executed (steps S103, S104, S105; No, S203, S204, S205; No).

  When the occurrence of an event for canceling the cooperative display is detected (step S105; Yes), the image forming apparatus 10 cancels the cooperative display and determines whether the user is mainly working on the mobile terminal 40 during the cooperative display. Is determined (step S106). If the user was mainly working on the mobile terminal 40 during the cooperation display (step S106; Yes), the mobile side cooperation screen displayed on the mobile terminal 40 before the cooperation cancellation and the operation panel 30 were displayed. Both or one of the MFP side cooperation screens is reduced, and both screens are collectively displayed on the display unit 31 of the operation panel 30 (step S161).

  Next, the user receives a selection operation from the user for either the area displayed by reducing the MFP-side cooperation screen or the area displayed by reducing the mobile-side cooperation screen (step S162). Then, the screen corresponding to the selected area is enlarged and displayed, the other screen is deleted (step S163), and the process is terminated.

  When the user is not mainly working on the mobile terminal 40 during the cooperation display (step S106; No), the MFP-side cooperation screen displayed on the operation panel 30 immediately before the cooperation is canceled is displayed on the display unit of the operation panel 30. 31 is displayed continuously as it is, and the process is terminated.

  When the occurrence of an event for canceling the cooperative display is detected (step S205; Yes), the mobile terminal 40 cancels the cooperation and displays an application screen such as an incoming call screen (step S206), and ends the process. To do.

  Even when the user mainly works on the operation panel 30 during the cooperative display, both the mobile-side cooperative screen and the MFP-side cooperative screen are collectively displayed when the cooperation is canceled, and either one of the selection operations is accepted, The selected screen may be displayed on the operation panel 30 and the other screen may be deleted.

<Third Embodiment>
In the operation display system 3 according to the third embodiment, the operation buttons are grouped. When the first operation button displayed on the mobile side cooperation screen is operated before the cooperation is released, and the second operation button belonging to the same group as the first operation button is displayed on the MFP side cooperation screen Is configured to display at least a first operation button and a second operation button on the display unit 31 of the operation panel 30 after the cancellation of cooperation. A description of the same parts as those in the first embodiment will be omitted as appropriate.

  FIG. 13 shows an example of the display operation at the time of cancellation of cooperation in the third embodiment. FIG. 2A shows a state where the operation panel 30 of the image forming apparatus 10 and the mobile terminal 40 are performing the cooperative display. Here, the operation buttons A and B displayed on the mobile terminal 40 and the operation buttons C and D displayed on the operation panel 30 belong to the same group, and the operation button E does not belong to this group.

  When the cooperation display is canceled by an incoming call or the like while the user is operating by operating the operation button B, as shown in FIG. 13B, the display unit 31 of the operation panel 30 after the cooperation is released. In addition, the operation buttons A to D belonging to the group to which the operation button B belongs are displayed, and the operation buttons E not belonging to the same group as the operation button B are controlled not to be displayed.

  FIG. 14 shows another screen example. In FIG. 5A, the operation button B displayed on the mobile terminal 40 and the operation buttons C and D displayed on the operation panel 30 belong to the same group, and the operation button A displayed on the mobile terminal 40 and The operation buttons E and F displayed on the operation panel 30 do not belong to the same group as the operation button B.

  When the cooperation display is canceled by an incoming call or the like while the user is operating by operating the operation button B, as shown in FIG. 14B, the display unit 31 of the operation panel 30 after the cooperation is released. In addition, the operation buttons B to D belonging to the group to which the operation button B belongs are displayed, and the operation buttons A, E, and F not belonging to the same group as the operation button B are controlled not to be displayed.

  In the case of FIG. 14, all of the operation buttons displayed on the mobile terminal 40 may be displayed on the operation panel 30 after the cancellation of cooperation. That is, in the example of FIG. 14, the operation buttons A to D may be displayed on the operation panel 30 after the cancellation of cooperation. Also displayed on the operation panel 30. A configuration in which at least one of the operation buttons C and D belonging to the same group as the operation button B is displayed on the screen after the cancellation of the cooperation may be employed.

  FIG. 15 is a flowchart showing an operation related to cancellation of the cooperative display of the image forming apparatus 10 (MFP) and the mobile terminal 40 of the operation display system 3 according to the third embodiment. The same steps as those in FIG. 8 are denoted by the same step numbers. Note that steps S101 to S106 on the image forming apparatus 10 side and the operation of the mobile terminal 40 are the same as those in FIG.

  In step S106, when the user is mainly working on the mobile terminal 40 during the cooperation display (step S106; Yes), the operation buttons belonging to the same group as the operation buttons operated on the mobile terminal 40 before the cooperation is released. Is displayed on the display unit 31 of the operation panel 30, and operation buttons not belonging to the group are not displayed (step S171).

  Even when the user mainly performs operations on the operation panel 30 during the cooperative display, when the cooperation is canceled, the operation buttons belonging to the same group as the operation buttons operated on the operation panel 30 before the cancellation of the cooperation are displayed. The operation buttons that do not belong to the group may be not displayed. For example, even when the operation button C is operated in FIG. 13A, the display shown in FIG.

<Fourth embodiment>
In the fourth embodiment, an operation in the case where the cooperative display is resumed after a predetermined time has elapsed after the cooperative display is canceled will be described.

  16, FIG. 17, and FIG. 18 show an example of the display operation in the fourth embodiment. FIG. 16A shows a state in which the operation panel 30 of the image forming apparatus 10 and the mobile terminal 40 perform cooperative display. The user is working mainly using the mobile side cooperation screen 62 displayed on the mobile terminal 40. For example, it is assumed that the operation button A is operated. At this time, when an incoming call occurs in the mobile terminal 40 and the cooperation is canceled, as shown in FIG. 4B, the operation panel 30 displays the mobile side displayed on the mobile terminal 40 before the cooperation is canceled. The contents of the cooperation screen 62 are displayed on the operation panel 30 after the cooperation is canceled. This movement is the same as in the first embodiment.

  Thereafter, when the mobile terminal 40 is juxtaposed next to the operation panel 30 before the predetermined time elapses and the cooperative display can be resumed, one of the following processes is selected and executed.

(1) If the user does not perform any operation on the operation panel 30 after the cancellation of the cooperation until the restart, the display is returned to the same display state as before the cancellation of the cooperation and the cooperative display is resumed (FIG. 17).

(2) If the user has operated the operation panel 30 after canceling the cooperation and restarting, the display content of the operation panel 30 is maintained as it is when the cooperative display is restarted. Then, a screen following the screen displayed on the operation panel 30 or a screen related to the screen displayed on the operation panel 30 is displayed, and the cooperative display is resumed (FIG. 18).

  FIG. 17 shows an example of the above (1). FIG. 17A shows a state in which no operation is received during cooperation cancellation, and FIG. 17B shows a display state after resumption of cooperation. FIG. 18 shows an example of the above (2). FIG. 18A shows a state in which an operation is received during cancellation of cooperation, and FIG. 18B shows a display state after resumption of cooperation. In FIG. 18B, the display content of the operation panel 30 is maintained while the cooperation is being canceled, and the operation buttons X and Y following the left side of the operation button A are displayed on the mobile terminal 40 arranged on the left side of the operation panel 30. It is displayed.

  FIG. 19 is a flowchart showing cooperation release and restart operations in the fourth embodiment. The same steps as those in FIG. 8 are denoted by the same step numbers. Note that steps S101 to S107 on the image forming apparatus 10 side and the operation of the mobile terminal 40 are the same as those in FIG.

  In step S106, when the user is mainly working on the mobile terminal 40 during the cooperation display (step S106; Yes), the mobile side cooperation screen 62 displayed on the mobile terminal 40 before the cooperation release is displayed on the operation panel 30. Are displayed on the display unit 31 and the process proceeds to step S181. If the user is not mainly working on the mobile terminal 40 during the cooperation display (step S106; No), the MFP-side cooperation screen 61 displayed on the operation panel 30 before the cooperation cancellation is continued on the operation panel 30 as it is. Then, the process proceeds to step S181.

  In step S181, the process waits for resumption of cooperative display (step S181; No). In addition, this process is complete | finished when cooperation display is not restarted before progress of the predetermined time after cooperation cancellation | release. When the cooperation display is resumed after the predetermined time elapses after the cooperation is released (step S181; Yes), it is determined whether or not an operation is received on the operation panel 30 during the cooperation release (step S182).

  If the operation panel 30 has been operated during the cancellation of cooperation (step S182; Yes), the display of the operation panel 30 is maintained as it is, and the mobile terminal 40 continues to the screen displayed on the operation panel 30. A screen or a screen related to the screen displayed on the operation panel 30 is displayed (step S183), and the process ends.

  When the operation panel 30 has not been operated during the cancellation of the cooperation (step S182; No), the display contents of the operation panel 30 and the mobile terminal 40 are returned to the state before the cancellation of the cooperation (step S184), and the process is terminated. To do.

  As described above, in the fourth embodiment, when the operation panel 30 is not operated during the cancellation of the cooperation, the display is resumed by returning to the state before the cooperation cancellation, and the operation panel 30 is operated during the cancellation of the cooperation. Since the user is working using the operation panel 30 during the cancellation of the cooperation, the display screen of the operation panel 30 is maintained, and the screen related to this screen is displayed on the mobile terminal 40 to resume the cooperative display. To do.

  The embodiment of the present invention has been described with reference to the drawings. However, the specific configuration is not limited to that shown in the embodiment, and there are changes and additions within the scope of the present invention. Are also included in the present invention.

  In the embodiment, the case where the operation panel 30 fixed to the image forming apparatus 10 and the mobile terminal 40 cooperate with each other is described as an example, but the operation panel 30 may be removable from the image forming apparatus 10. Alternatively, an individual remote operation panel may be used. Further, the present invention is also applied to the case where the operation panel 30 removed from the two mobile terminals 40 or the image forming apparatus 10 and the remote operation panel are juxtaposed and communicate with each other.

  In the present invention, the operation panel 30 (operation display device) is configured separately from the image forming apparatus 10, a control unit such as a CPU is provided in the operation panel 30, and the CPU 11 of the image forming apparatus 10 performs in the embodiment. The operation control may be performed by the operation panel 30.

  Although the operation display system 3 is exemplified in the embodiment, the present invention may be configured as the image forming apparatus 10 capable of displaying in cooperation with the mobile terminal, or a program for controlling the operation of the image forming apparatus 10. Also good.

3 ... Operation display system 10 ... Image forming apparatus 11 ... CPU
12 ... ROM
13 ... RAM
DESCRIPTION OF SYMBOLS 14 ... Non-volatile memory 15 ... Hard disk device 16 ... Automatic document conveyance part 17 ... Network communication part 18 ... Image reading part 19 ... Image processing part 21 ... Printer part 22 ... Facsimile communication part 23 ... Authentication part 30 ... Operation panel 31 ... Display part 32 ... Operation unit 32a ... Touch panel 33 ... Proximity wireless communication unit 34 ... Connection detection unit 40 ... Mobile terminal 41 ... CPU
42 ... ROM
43 ... RAM
44 ... Non-volatile memory 45 ... Authentication unit 46 ... Operation unit 46a ... Touch panel 46b ... Hard key 47 ... Display unit 48 ... Network communication unit 49 ... Nearby wireless communication unit 51 ... Operation button 52 ... Cooperation start button 53 ... Cooperation release button 61 ... MFP side cooperation screen 61a ... Reduction screen before cancellation of MFP side cooperation 62 ... Mobile side cooperation screen 62a ... Reduction screen before cancellation of mobile side cooperation

Claims (16)

An operation display system including first and second operation display devices each including an operation unit and a display unit,
A first screen is displayed on the display unit of the first operation display device, and a second screen related to the first screen is displayed on the second operation display device juxtaposed to the first operation display device. A linked display control unit for controlling linked display;
A master-slave determination unit that determines which of the first screen and the second screen the user is mainly working during the linkage display;
A cancellation event detection unit that detects the occurrence of a predetermined event that triggers cancellation of the linkage display;
Have
The linkage control unit cancels the linkage display when the occurrence of the predetermined event is detected by the release event detection unit during the linkage display, and the determination result of the master-slave determination unit before the release Indicates that the second screen is the main screen, a screen including at least a part of the second screen is displayed on the display unit of the first operation display device after the release. Display system. The predetermined event is:
The positional relationship between the display unit of the first operation display device and the display unit of the second operation display device changes to a predetermined positional relationship unsuitable for the cooperative display;
Disconnection of communication between the first operation display device and the second operation display device;
The occurrence of an event having priority over the cooperative display in the second operation display device;
2. The operation display system according to claim 1, comprising at least one of receiving a linked display canceling operation in any of the first operation display device and the second operation display device. The master-slave determination unit
Which display unit of the first operation display device or the second operation display device is the display unit last viewed by the user before the predetermined event occurs,
Which operation unit of the first operation display device or the second operation display device is the operation unit last operated by the user before the predetermined event occurs,
Which display unit of the first operation display device and the second operation display device is the display unit with a long time viewed by the user during the cooperative display,
Which of the first operation display device and the second operation display device is an operation unit with a long operation time or an operation unit with a large number of operations during linked display,
The determination is performed based on at least one of the first operation display device and the second operation display device that displays a larger number of items set during cooperative display. Or the operation display system of 2. The cooperation control unit displays both the first screen and the second screen on the display unit of the first operation display device after the cancellation, with at least one of the first screen and the first screen being reduced. A selection operation for any one of the second screens is received from the user, the selected screen is displayed on the display unit of the first operation display device, and the unselected screen is displayed on the display unit of the first operation display device The operation display system according to claim 1, wherein the operation display system is deleted. The cooperation control unit operates the first operation button displayed on the second screen before the release, and the second operation button belonging to the same group as the first operation button The display device of the first operation display device after the release displays at least the first operation button and the second operation button when displayed on one screen. The operation display system described in 1. The linkage control unit is configured to release all the operation buttons that belong to the same group as the first operation button and are displayed on either the first screen or the second screen before the release. An operation button that does not belong to the same group as the first operation button that is displayed on the display unit of the first operation display device and is displayed on the first screen before the release is released is the first button after the release. The operation display system according to claim 5, wherein the operation display system is not displayed on a display unit of the operation display device. When the cooperation control unit restarts the cooperation display before the elapse of a predetermined time after the release,
When the setting operation is received from the user in the first operation display device after the cancellation, the screen displayed on the display unit of the first operation display device is displayed on the display unit of the first operation display device. Display and display a screen related to the screen on the second operation display device to resume cooperative display, and if the setting operation is not received from the user in the first operation display device after the release, the release is performed. The screen displayed on the display unit of the first operation display device before the display is displayed on the display unit of the first operation display device, and on the display unit of the second operation display device before the release. The operation display system according to any one of claims 1 to 6, wherein the displayed screen is displayed on the display unit of the second operation display device to restart the cooperative display. The operation according to any one of claims 1 to 7, wherein the first operation display device is an operation panel of an image forming apparatus, and the second operation display device is a mobile terminal. Display system. An operation unit;
A display unit;
A linked display control unit that controls a linked display that displays a first screen on the display unit and displays a second screen related to the first screen on a mobile terminal that includes an operation unit and a display unit juxtaposed to the device; ,
A master-slave determination unit that determines which of the first screen and the second screen the user is mainly working during the linkage display;
A cancellation event detection unit that detects the occurrence of a predetermined event that triggers cancellation of the linkage display;
Have
The linkage control unit cancels the linkage display when the occurrence of the predetermined event is detected by the release event detection unit during the linkage display, and the determination result of the master-slave determination unit before the release When the display indicates that the second screen is main, the operation display device displays a screen including at least a part of the second screen on the display unit after the release. The predetermined event is:
The positional relationship between the display unit of the device itself and the display unit of the mobile terminal changes to a predetermined positional relationship unsuitable for the cooperative display;
Disconnection of communication between the device and the mobile terminal;
Occurrence of an event that has priority over linked display on the mobile device,
The operation display device according to claim 9, comprising at least one of receiving an operation of canceling cooperative display in either the device itself or the mobile terminal. The master-slave determination unit
Whether the display unit that the user last viewed before the predetermined event occurs is the display unit of the own device or the mobile terminal,
Whether the operation unit last operated by the user before the predetermined event occurs is the operation unit of the own device or the mobile terminal,
Whether the display unit with a long time viewed by the user during linked display is the display unit of the own device or the mobile terminal,
Whether the operation unit with a long time operated by the user or the operation unit with a large number of operations during cooperation display is the operation unit of the own device or the mobile terminal,
The operation display device according to claim 9 or 10, wherein the determination is performed based on at least one of the own device and the mobile terminal displaying a larger number of items set during cooperative display. . The cooperation control unit displays both the first screen and the second screen on the display unit of the self-released device after reducing at least one of the first screen and the second screen. 10. The operation according to claim 9, wherein one of the selection operations is received from a user, the selected screen is displayed on the display unit of the own device, and the unselected screen is deleted from the display unit of the own device. Display device. The cooperation control unit operates the first operation button displayed on the second screen before the release, and the second operation button belonging to the same group as the first operation button 10. The operation display according to claim 9, wherein when displayed on one screen, at least the first operation button and the second operation button are displayed on the display unit of the device after the release. apparatus. The linkage control unit is configured to release all the operation buttons that belong to the same group as the first operation button and are displayed on either the first screen or the second screen before the release. Operation buttons that do not belong to the same group as the first operation button that is displayed on the display unit of the subsequent device and that is displayed on the first screen before the release are not displayed on the display unit of the device after the release The operation display device according to claim 13. When the cooperation control unit restarts the cooperation display before the elapse of a predetermined time after the release,
When a setting operation is received from the user in the own device after the release, the screen displayed on the display unit of the own device is displayed on the display unit of the own device and a screen related to the screen is displayed on the mobile terminal. If the setting operation is not received from the user in the own device after the release, the screen displayed on the display unit of the own device before the release is displayed on the display unit of the own device, and The cooperative display is resumed by displaying the screen displayed on the display unit of the mobile terminal before the cancellation on the display unit of the mobile terminal. Operation display device. Information processing device
A program causing the operation display device according to any one of claims 9 to 15 to function.