JP6330349B2 - Display control device, display system, operation display device, program - Google Patents

Description

  The present invention relates to a display control device, a display system, an operation display device, and a program for linking two juxtaposed operation display devices.

  When an operation display device (operation panel) provided in an image forming apparatus such as a multifunction peripheral and an operation display device such as a mobile terminal owned by a user are juxtaposed to communicate with each other, these operation display devices are operated in cooperation with each other. There is an image forming apparatus that improves convenience by simultaneously displaying two related screens, for example, an operation screen and a preview screen. (See Patent Document 1).

  In this image forming apparatus, which of the operation display devices juxtaposed is to be displayed with the operation screen is compared with the specs of the display capability (the number of pixels and the size of the display unit) of the operation display devices juxtaposed, Determine the one with the higher display capacity. When a predetermined operation button is pressed on the operation display device displaying the operation screen, a screen (for example, a preview screen) corresponding to the pressed operation button is displayed on the other operation display device. It has become.

JP 2013-110470 A

  In the device disclosed in Patent Literature 1, when the juxtaposed operation display devices are operated in a coordinated manner, which device displays the operation screen and which device displays the screen corresponding to the operation is arranged in parallel. It is determined according to the display capacity specifications of the operation display device.

  However, it may be inconvenient if the above determination is made based on specifications that do not change fixedly. For example, when the battery level of an operation display device driven by a battery is low, it is desirable to display a screen on the operation display device that requires less power consumption. In an operation display device that can operate in multitasking, resources such as a CPU and a memory are often used by other tasks, and resources that can be used for cooperative operations may be limited. As described above, it is desirable to display a screen with less resources and processing burden on the operation display device with fewer resources.

  However, when it is determined which screen is displayed on which operation display device based on the fixed specifications of these operation display devices, an undesirable display result is obtained when viewed from the current device state of those operation display devices. The case that becomes.

  The present invention is to solve the above-described problem, and dynamically changes the division of display roles when linking two juxtaposed operation display devices in accordance with the status of those devices. It is an object of the present invention to provide a display control device, a display system, an operation display device, and a program that can be used.

  The gist of the present invention for achieving the object lies in the inventions of the following items.

[1] A detection unit that detects that the first and second operation display devices each including an operation unit and a display unit are juxtaposed and can communicate with each other;
A fluctuation information acquisition unit that acquires information about a predetermined apparatus state that fluctuates during operation of the first operation display device and the second operation display device;
The division of roles related to the display of the screens of the first operation display device and the second operation display device in the cooperative state is determined based on the information acquired by the variation information acquisition unit, and according to the determined division of roles A control unit that controls display of screens of the first operation display device and the second operation display device;
I have a,
The predetermined device state is a device state relating to a limit of usable power,
When the control unit displays a new screen on one of the first operation display device and the second operation display device, the power consumption required for display is displayed among the new screen and the original screen. The display control apparatus according to claim 1 , wherein the division of roles is determined so that a larger number of screens are not displayed on an operation display apparatus having a limited usable power .

In the above invention and the invention described in [ 9 ] below, the division of roles related to the display of the screens of the juxtaposed first operation display device and the second operation display device is based on a predetermined device state that fluctuates during operation. decide. The content of the role assignment is how to display which screen on which operation display device. Given device state, for example, the power state, such as power-saving mode and battery status. Also, the division of roles is determined so that the screen with the larger power consumption required for display is not displayed on the operation display device in which usable power is limited. The operation display device with limited power that can be used is an operation display device that is in a power saving mode or in a battery-powered state with a low battery level.
[2] A detection unit that detects that the first and second operation display devices each including an operation unit and a display unit are juxtaposed and can communicate with each other;
A fluctuation information acquisition unit that acquires information about a device state relating to a resource usage state of the first operation display device and the second operation display device;
The division of roles related to the display of the screens of the first operation display device and the second operation display device in the cooperative state is determined based on the information acquired by the variation information acquisition unit, and according to the determined division of roles A control unit that controls display of screens of the first operation display device and the second operation display device;
Have
When the control unit displays a new screen on either the first operation display device or the second operation display device, the usage amount of the resource required for display out of the new screen and the original screen The division of roles is determined so that the screen with more resources is not displayed on the operation display device with less resources.
A display control device characterized by that.
[3] The resource is a CPU or a memory,
When the resource is a CPU, the usage status of the resource includes the amount of processing load on the CPU,
When the resource is a memory, the usage status of the resource includes an available free memory capacity.
The display control apparatus according to [2], wherein

[ 4 ] The new screen is a screen that is displayed in response to a predetermined operation on either the first operation display device or the second operation display device, and the original screen receives the predetermined operation. The display control device according to any one of [1] to [3], wherein the display control device is a screen displayed on the operation display device.

In the above invention and the invention described in [ 12 ] below, the division of roles related to the display between the original screen (operation screen) and the screen newly displayed based on the operation is determined.

[ 5 ] The display control apparatus according to any one of [1] to [4], wherein the new screen is a screen that displays a preview image.

[ 6 ] The display control apparatus according to any one of [1] to [5], wherein the new screen is a screen that displays a moving image.

[ 7 ] When the device state changes after determining the role assignment, the control unit re-determines the role assignment based on the changed device state, and performs the control according to the re-determined role assignment. The display control device according to any one of [1] to [ 6 ], wherein:

In the above invention and the invention described in [ 15 ] below, when the device state changes, the division of roles related to the display of the screen is reviewed based on the device state after the change.

[ 8 ] First and second operation display devices each including an operation unit and a display unit;
The display control device according to any one of [1] to [ 7 ];
A display system characterized by comprising:

[ 9 ] An operation display device including an operation unit and a display unit,
A detection unit for detecting that the self-operation display device and another operation display device are juxtaposed and in a communicable cooperative state;
A fluctuation information acquisition unit that acquires information on a predetermined apparatus state that fluctuates during operation of the self-operation display apparatus and the other operation display apparatus;
The division of roles related to the display of the screen of the cooperative operation display device and the other operation display device is determined based on the information acquired by the variation information acquisition unit, and the self-operation display device according to the determined division of roles And a control unit for controlling display of the screen of the other operation display device,
I have a,
The predetermined device state is a device state relating to a limit of usable power,
When the control unit displays a new screen on either the self-operation display device or the other operation display device, the screen of the new screen and the original screen that requires more power to be displayed. The role assignment is determined so as not to be displayed on the operation display device whose usable power is limited .
[10] An operation display device including an operation unit and a display unit,
A detection unit for detecting that the self-operation display device and another operation display device are juxtaposed and in a communicable cooperative state;
A fluctuation information acquisition unit that acquires information on the device status related to the resource usage status of the self-operation display device and the other operation display device;
The division of roles related to the display of the screen of the cooperative operation display device and the other operation display device is determined based on the information acquired by the variation information acquisition unit, and the self-operation display device according to the determined division of roles And a control unit for controlling display of the screen of the other operation display device,
Have
When the control unit displays a new screen on either the self-operation display device or the other operation display device, the one that uses a larger amount of resources required for display among the new screen and the original screen. The division of roles is determined so that the screen is not displayed on the operation display device with the less resource available.
An operation display device characterized by that.
[11] The resource is a CPU or a memory,
When the resource is a CPU, the usage status of the resource includes the amount of processing load on the CPU,
When the resource is a memory, the usage status of the resource includes an available free memory capacity.
[10] The operation display device according to [10].

[ 12 ] The new screen is a screen that is displayed in response to a predetermined operation on either the own operation display device or the other operation display device, and the original screen is an operation display device that has received the predetermined operation. The operation display device according to any one of [9] to [11], wherein the operation display device is a screen displayed on the screen .

[ 13 ] The operation display device according to any one of [9] to [12], wherein the new screen is an image preview screen.

[ 14 ] The operation display device according to any one of [9] to [13], wherein the new screen is a screen for displaying a moving image.

[ 15 ] When the device state changes after determining the role assignment, the control unit re-determines the role assignment based on the changed device state, and performs the control according to the re-determined role assignment. The operation display device according to any one of [9] to [14] , wherein :

[ 16 ] An information processing device
[1] A program that causes a function as the display control device according to any one of [7].

[ 17 ] An information processing apparatus including an operation unit and a display unit,
[9] A program that functions as the operation display device according to any one of [15] .

  According to the display control device, the display system, the program, and the operation display device according to the present invention, the role of display when two juxtaposed operation display devices are linked is dynamically changed according to the state of the devices. Can be changed.

It is a figure which shows an example of the display system which concerns on embodiment of this invention. 1 is a block diagram illustrating a schematic electrical configuration of an image forming apparatus. It is a block diagram which shows schematic structure of a portable terminal. It is a flowchart which shows the display control after entering into a cooperation state. 6 is a diagram illustrating a display screen when the image forming apparatus is in a cooperative state with a mobile terminal during a power saving mode. FIG. FIG. 6 is a diagram illustrating a display screen when the image forming apparatus is in a cooperative state with a mobile terminal during a normal power mode. FIG. 10 is a diagram illustrating an example in which a preview screen is displayed on a mobile terminal so as to overlap the original screen in response to an operation of a preview display button on the mobile terminal because the operation panel is in a power saving mode. It is a figure which shows the example in which only the preview screen was displayed on the portable terminal according to operation of the preview display button in a portable terminal because the operation panel was a power saving mode. It is a figure which shows the example of a display when role assignment is re-determined as a result of the operation panel shifting to power saving mode. FIG. 5 is a flowchart showing details of a process of determining a role assignment and displaying a screen when receiving an operation (step S105 in FIG. 4). It is a figure which shows the example which displayed the new screen (preview screen) and the original screen (copy setting screen) side by side on the portable terminal. FIG. 5 is a flowchart showing details of processing (step S107 in FIG. 4) for re-determining role assignment and displaying a screen when a device state changes. It is a figure which shows the example which the display location of a preview screen switches. It is a figure which shows the other example in which the display location of a preview screen switches.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows a display system including an image forming apparatus 10 having an operation display device according to an embodiment of the present invention as an operation panel 30 and a portable terminal 40 used in parallel with the operation panel 30 of the image forming apparatus 10. 3 shows an example.

  The image forming apparatus 10 is a copy job that optically reads a document and prints a duplicate image on a recording sheet, a scan job that stores image data of the read document as a file, or transmits the file to an external terminal via a network, A so-called function that executes a plurality of types of jobs, such as a print job for forming and printing out an image related to print data received from a terminal device over a network on a recording paper, and a FAX job for transmitting and receiving image data according to a facsimile procedure. It is a multifunction machine.

  The mobile terminal 40 is a display terminal such as a mobile phone, a tablet terminal, or a smartphone.

  When the operation panel 30 and the portable terminal 40 of the image forming apparatus 10 are in a cooperating state in which the operation panel 30 and the portable terminal 40 can communicate with each other, the image forming apparatus 10 uses the display unit of one or both of the operation panel 30 and the portable terminal 40 to A screen such as a preview screen can be displayed. The operation panel 30 and the portable terminal 40 are collectively referred to as an operation display device.

  The image forming apparatus 10 acquires information regarding a predetermined apparatus state that fluctuates during operation of the operation panel 30 and the portable terminal 40, and based on this information, the screens of the operation panel 30 and the portable terminal 40 in a linked state are displayed. The division of roles related to display is determined, and display of the screen on the operation panel 30 and the portable terminal 40 is controlled according to the determined division of roles.

Specifically, the image forming apparatus 10
Whether or not to use both the operation panel 30 and the portable terminal 40;
・ When both are used, which screen is displayed on which
・ When only one is used, which screen is to be displayed on which one of the devices,
The display is controlled by determining the division of roles.

  For example, as information related to the device state, information related to restrictions on the power that can be used by the operation panel 30 and the portable terminal 40, specifically whether the power saving mode is being used, whether the remaining battery level is greater than or equal to a threshold value, etc. When information is acquired and the power that can be used by both devices is not restricted, the screen display role is divided so that both the display units of the operation panel 30 and the portable terminal 40 are used, and one of them consumes power. Is in a state where power consumption is to be restricted (during power saving mode or in a state where the remaining battery level is equal to or less than a threshold value, etc.), the role of screen display is divided so that only the other display unit is used.

  In addition, in addition to the above-described usable power, the device status that fluctuates during operation includes resource levels such as the amount of processing load on the image forming apparatus 10, the operation panel 30, and the mobile terminal 40, and the available free memory capacity. The usage status (remaining amount of resources that can be spent for display) is also included.

  Further, the division of roles is determined by comprehensive judgment in consideration of the type and contents of the screen to be displayed in addition to the apparatus status.

  FIG. 2 is a block diagram showing a schematic electrical configuration of the image forming apparatus 10. The image forming apparatus 10 includes a CPU (Central Processing Unit) 11 as a control unit that comprehensively controls the operation of the image forming apparatus 10. The CPU 11 includes a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a nonvolatile memory 14, a hard disk device 15, an automatic document feeder (ADF) 16, a network communication unit 17, and an image reading unit. 18, an image processing unit 19, a printer unit 21, a facsimile communication unit 22, an authentication unit 23, an operation panel 30, and the like are connected.

  The CPU 11 is based on an OS (Operating System) program, and executes middleware, application programs, and the like. Various programs are stored in the ROM 12, and each function of the image forming apparatus 10 (including the operation panel 30) is realized by the CPU 11 executing various processes according to these programs.

  Although the CPU 11 controls the operation panel 30 here, the operation panel 30 may be controlled by a separate CPU provided exclusively for the operation panel 30.

  The RAM 13 is used as a work memory for temporarily storing various data when the CPU 11 executes processing based on a program, an image memory for storing image data, and the like.

  The nonvolatile memory 14 is a memory (flash memory) whose stored contents are not destroyed even when the power is turned off, and is used for storing various setting information.

  The hard disk device 15 is a large-capacity nonvolatile storage device, and stores various programs and data in addition to print data and display screen data.

  The network communication unit 17 performs a function of communicating with an external terminal, a server, or the like connected to a network such as a LAN (Local Area Network).

  The image reading unit 18 performs a function of optically reading a document and acquiring image data. The image reading unit 18 sequentially, for example, a light source that irradiates light on a document, a line image sensor that receives the reflected light for one line in the width direction, and a line-by-line reading position in the length direction of the document. An optical path composed of a moving unit for moving, a lens, a mirror, and the like for guiding reflected light from the document to the line image sensor to form an image, and a conversion unit for converting an analog image signal output from the line image sensor into digital image data And so on.

  The automatic document feeder 16 feeds the document set on the platen one by one from the top one by one in order, passes the reading position of the image reading unit 18 and discharges it to a predetermined discharge position. Fulfill. The image reading unit 18 has a function of reading a document placed on the platen glass and a function of sequentially reading a document conveyed by the automatic document conveyance unit 16.

  The image processing unit 19 performs rasterization processing for converting print data into image data, image data compression and expansion processing, in addition to processing such as image enlargement / reduction and rotation.

  The printer unit 21 has a function of forming an image corresponding to image data on a recording sheet. Here, it has a recording paper transport device, a photosensitive drum, a charging device, a laser unit, a developing device, a transfer separation device, a cleaning device, and a fixing device, and forms an image by an electrophotographic process. It is configured as a so-called laser printer. Other methods may be used for image formation.

  The facsimile communication unit 22 performs a function of transmitting / receiving image data to / from an external device having a facsimile function through a telephone line.

  The authentication unit 23 authenticates a user who uses the image forming apparatus 10. The authentication method may be any password, fingerprint, vein, or the like.

  The operation panel 30 includes a display unit 31, an operation unit 32, a close proximity wireless communication unit 33, and a connection detection unit 34. The operation panel 30 has various functions such as a function for displaying various operation screens and setting screens, and a job input from a user. It fulfills the function of accepting operations.

  The display unit 31 includes a liquid crystal display (LCD ... Liquid Crystal Display) and its driver. The operation unit 32 includes various hard keys such as a start button and a numeric keypad, and a touch panel 32 a provided on the display surface of the display unit 31. The touch panel 32a detects a coordinate position where the display surface of the display unit 31 is touched with a touch pen or a finger. The operation unit 32 receives a selection operation, a flick operation, a drag operation, and the like through the touch panel 32a. The detection method of the touch panel 32a may be arbitrary, such as capacitance, analog / digital resistance film, infrared rays, ultrasonic waves, electromagnetic induction, and the like.

  The close proximity wireless communication unit 33 performs a function of performing wireless communication with the portable terminal 40 disposed in the vicinity of the operation panel 30. The communicable distance of the close proximity wireless communication unit 33 is about 50 cm. Various short-range wireless communication systems can be adopted as the communication system.

  The connection detection unit 34 detects that the portable terminal 40 is juxtaposed at a predetermined location with respect to the operation panel 30. For example, when a dock for holding the portable terminal 40 is provided on the side of the operation panel 30, the connection detection unit 34 detects that the portable terminal 40 is attached to this dock.

  FIG. 3 is a block diagram illustrating a schematic configuration of the mobile terminal 40. The mobile terminal 40 has a CPU 41 that performs overall control of the operation of the mobile terminal 40. A ROM 42, a RAM 43, a nonvolatile memory 44, an authentication unit 45, an operation unit 46, a display unit 47, a network communication unit 48, and a close proximity wireless communication unit 49 are connected to the CPU 41 through a bus.

  Various programs and data are stored in the ROM 42 and the nonvolatile memory 44, and each function of the portable terminal 40 is realized by the CPU 41 executing various processes in accordance with these programs. The RAM 43 is used as a work memory for temporarily storing various data when the CPU 41 executes a program and a memory for temporarily storing display data.

  The nonvolatile memory 44 is a memory that retains stored contents even when the power is turned off. In the nonvolatile memory 44, a program for executing a function for displaying a screen based on a document or an image file, various setting contents, user information, terminal information (such as a network address), an application program, or an instruction from the operation panel 30 Etc. are memorized.

  The authentication unit 45 authenticates a user who uses the mobile terminal 40.

  The display unit 47 is configured by a liquid crystal display or the like.

  The operation unit 46 is provided on the entire display surface of the display unit 47, and includes a touch panel 46a that detects a position touched by a finger or a pen, and a small number of hard keys (operation switches).

  The network communication unit 48 has a function of communicating with various external devices by connecting to a network by wireless communication. The close proximity wireless communication unit 49 performs a function of performing wireless communication with the close proximity wireless communication unit 33 of the nearby operation panel 30. The communicable distance, communication method, and the like are the same as those of the close proximity wireless transfer unit 33 of the operation panel 30.

  FIG. 4 is a flowchart showing display control after entering the cooperative state. Here, it is assumed that the CPU 11 of the image forming apparatus 10 performs this control. The cooperation state is a state in which the operation panel 30 and the portable terminal 40 are juxtaposed to perform communication in a state where communication is possible.

The base point that enters the linked state is
When it is detected that the operation panel 30 and the portable terminal 40 are ready for wireless communication, when the connection detection unit 34 of the operation panel 30 detects the attachment of the portable terminal 40, the operation panel 30 is carried by a predetermined sensor of the operation panel 30. For example, when the proximity of the terminal 40 is detected and it is detected that communication with the mobile terminal 40 is possible. Based on the state of the close proximity wireless transfer unit 33 and the connection detection unit 34, it is detected that the CPU 11 of the image forming apparatus 10 is in a cooperative state.

  When the cooperative state is entered, the CPU 11 of the image forming apparatus 10 operates as a fluctuation information acquisition unit. That is, the device state information indicating the device state of the device itself is acquired, and the proximity wireless communication unit 33 of the operation panel 30 is used to communicate with the mobile terminal 40 to acquire the device state information of the mobile terminal 40 (step S101). .

  Next, the CPU 11 of the image forming apparatus 10 determines the division of roles between the operation panel 30 and the mobile terminal 40 related to screen display based on the acquired apparatus state information, and the operation panel 30 and the mobile terminal 40 are determined according to the determination. It operates as a control unit that displays a screen to be displayed (step S102).

  For example, as shown in FIG. 5, when the image forming apparatus 10 is operating in the power saving mode in which the power consumption is less than that in the normal power mode (FIG. 5A), the portable terminal 40 is juxtaposed next to the operation panel 30. In the case of being in a linked state ((b) in the figure), the operation panel 30 maintains the power saving mode and does not display the screen, and determines the division of roles so that only the mobile terminal 40 displays the screen. Then, a screen such as the copy setting screen 60 is displayed on the portable terminal 40.

  Further, for example, as shown in FIG. 6, when the image forming apparatus 10 is operating in the normal power mode ((a) in FIG. 6), the portable terminal 40 is juxtaposed next to the operation panel 30 and is in a cooperative state. (FIG. 5B) displays the screen displayed on the operation panel 30 as it is and displays a new screen related to the screen displayed on the operation panel 30 on the mobile terminal 40. The division of roles is determined, and those screens are displayed on the operation panel 30 and the portable terminal 40.

  In FIG. 6, the copy setting (1) screen is displayed on the operation panel 30, and the copy setting (2) screen is displayed on the mobile terminals 40 juxtaposed therewith.

  The screen data of the screen displayed on the mobile terminal 40 is created by the image forming apparatus 10 and transmitted to the mobile terminal 40 using short-range wireless communication. The operation content received by the mobile terminal 40 is transmitted from the mobile terminal 40 to the image forming apparatus 10 using short-range wireless communication, and the image forming apparatus 10 receives this and receives the operation content received by the mobile terminal 40. Is supposed to recognize.

  Returning to FIG. 4, the description will be continued. Thereafter, it is monitored whether or not an operation is received from the user on any of the operation display devices displaying the screen (step S103) and whether or not the device state of each operation display device has changed (step S106).

  When an operation is received from the user (step S103; Yes), a new screen corresponding to the operation is created (step S104). Then, the role assignment for displaying the new screen is determined according to the current device state and screen content, and the screen is displayed according to the determination (step S105).

  For example, as shown in FIG. 7A, since the operation panel 30 is in the power saving mode, the copy setting screen is not displayed and the copy setting screen 60 is displayed only on the mobile terminal 40. It is assumed that the preview display button 61 in 60 is operated. In this case, a preview screen for previewing the copy target image is created as a new screen. Since the operation panel 30 is in the power saving mode, it remains in a non-display state, and a new screen (preview screen 62) is divided and displayed so as to overlap the original screen (copy setting screen 60). This is performed (FIG. 7B).

  In FIG. 7B, a new screen (preview screen 62) is displayed on the original screen (copy setting screen 60) displayed on the mobile terminal 40. However, in the state of FIG. When 61 is operated, as shown in FIG. 8, only the new screen (preview screen 62) may be displayed on the mobile terminal 40 without displaying the original screen.

  If the device state changes during operation (FIG. 4, step S106; Yes), the role assignment according to the changed device state is determined, and the screen is displayed accordingly (step S107).

  For example, when the operation panel 30 shifts to the power saving mode while the screen is displayed on the operation panel 30 and the portable terminal 40, the operation panel 30 does not display the screen, and the portable terminal 40 originally The role assignment is re-determined so that both the screen displayed on the portable terminal 40 and the screen displayed on the operation panel 30 before shifting to the power saving mode are displayed, and the screen is displayed accordingly. .

  In the example of FIG. 9, the result of the operation panel 30 shifting to the power saving mode when the preview screen 62 is displayed on the operation panel 30 and the copy setting screen 60 is displayed on the portable terminal 40 (FIG. 9A). As shown in FIG. 5B, the operation panel 30 is not displayed, and the mobile terminal 40 displays a preview screen 62 overlaid on the copy setting screen.

  In addition, as a change in the state of the mobile terminal 40, when the remaining battery level falls below the threshold during the display of the preview screen 62, another application (e.g., email, phone call, text reference, in-terminal information reference, etc.) is displayed on the mobile terminal 40. For example, the application may be temporarily opened. When the display in the cooperative state is continued even when such a state change occurs, the display position of the screen is switched as appropriate.

  Even when the state change occurs in the reverse direction (direction in which display is possible), the division of roles is re-determined.

  FIG. 10 shows the details of the process (step S105 in FIG. 4) for determining the role assignment and displaying the screen when an operation is received. This process is performed by the CPU 11 of the image forming apparatus 10, for example. First, the device state of the operation display device that has not been operated is checked (step S201). From this device state, it is determined whether or not the operation display device that is not the one receiving the operation is in a state where normal display is possible (normal display is possible) (step S202). For example, it is determined that normal display is possible when the normal power mode or the remaining battery level is equal to or greater than the threshold, and it is determined that normal display is not possible during the power saving mode or when the remaining battery level is less than the threshold. Whether or not normal display is possible may be determined based on the availability of resources such as the CPU and memory of each device, or in addition.

  If the operation display device that is not the user who received the operation can normally display (step S202; Yes), a new screen is displayed on the operation display device that is not the user who received the operation, and the operation display device that received the operation The original screen is displayed as it is (step S203).

  If the operation display device that is not the one that has received the operation is not normally displayable (step S202; No), a new screen is displayed on the operation display device that has received the operation (step S204). At this time, as a method of displaying the screen on the operation display device that has received the operation, as shown in FIG. 8, a single display for displaying only the new screen and an addition for displaying both the new screen and the original screen are performed. There is a display.

  In the additional display, a mode in which a new screen is displayed over the original screen as shown in FIG. 7B, a new screen (preview screen 62) and an original screen (copy setting screen 60) as shown in FIG. Are displayed side by side. Which display method and display mode to use is selected and set in advance, for example.

  FIG. 12 shows the details of the process (step S107 in FIG. 4) for re-determining the role assignment and displaying the screen when the apparatus state changes. When the normal display state changes from the normal displayable state due to a change in the device state of any of the operation display devices (step S301; Yes), the normal display is disabled. The displayed screen is displayed alone or additionally on the other operation display device (step S302), and this processing is terminated (END).

  In the single display, only the screen displayed on the operation display device that cannot be normally displayed is displayed on the other operation display device. In the additional display, the screen that was displayed on the operation display device that is normally disabled is superimposed on the screen that was displayed on the other operation display device, or the screen that was displayed on the other operation display device Are displayed side by side.

  The example of FIG. 9B corresponds to a case where the screen displayed on the operation display device that is normally disabled to display is superimposed on the screen displayed on the other operation display device.

  When the state is changed from the normal display disabled state to the normal display enabled state (step S301; No), it is checked whether there is a screen that has been displayed in a superimposed manner or a screen that has not been displayed (step S303). If there is such a screen (step S303; Yes), the screen is displayed on the operation display device that has been changed to a normal displayable state (step S304), and this processing is terminated (END). If there is no such screen (step S303; No), this process is terminated.

  Next, when a new screen created in response to an operation is displayed on the operation display device that has received the operation, and the original screen is displayed on an operation display device that is not the one that has received the operation (display switching) Will be described.

  FIG. 13A shows a state before an operation is received. The operation panel 30 is not displayed in the power saving mode, and the portable terminal 40 displays a copy setting screen 60. However, it is assumed that the operation panel 30 is in a displayable state because the backlight is turned on or turned on with reduced brightness even during power saving.

  When the preview display button 61 is operated on the portable terminal 40 in the state of FIG. 10A, the original screen (copy setting screen 60) displayed on the portable terminal 40 as shown in FIG. Is displayed on the operation panel 30, and a new screen (preview screen 62) is displayed on the mobile terminal 40.

  If the operation panel 30 can be normally displayed, the preview screen 62 is displayed on the operation panel 30 that is an operation display device that has not been operated.

  When the brightness of the backlight of the operation panel 30 is lowered, the screen displayed on the operation panel 30 is less visible than the screen displayed on the mobile terminal 40. Therefore, in the example of FIG. 13, since the screen newly displayed by the operation is a new screen, it is displayed on the operation display device with higher visibility, and the original screen has been displayed so far. Since it is recognized, it is displayed on the operation display device with lower visibility. It should be noted that it is determined whether the original screen or the new screen is a screen that needs to be clearly seen, the screen that should be clearly displayed is displayed on the portable terminal 40, and the other screen is displayed on the operation panel 30. The display role assignment may be determined by.

  In FIG. 14A, the copy setting screen (1) is displayed on the operation panel 30 and the copy setting screen (2) is displayed on the portable terminal 40. When the preview display button 61 on the copy setting screen (1) is operated in this state, a preview screen 62 is created as a new screen. Then, in consideration of the current device state of the operation panel 30 and the portable terminal 40, it is determined which is suitable for displaying the preview screen 62, and the preview screen 62 is displayed on the one suitable for displaying the preview screen 62. On the other hand, the copy setting screen (1) is displayed.

  The example shown in FIG. 14B is a case where it is determined that the operation panel 30 is suitable for displaying the preview screen 62. The preview screen 62 is displayed on the operation panel 30 and the copy setting screen (1) is displayed on the portable terminal 40. Is displayed.

  The above determination is made based on, for example, a device state relating to a restriction on power that can be used for screen display. The image forming apparatus 10 performs a display role so that the screen that consumes more power among the new screen and the original screen is not displayed on the operation display device that has limited power. Determine sharing.

  For example, in an apparatus state where the operation panel 30 operates in the normal power mode and the remaining battery level of the mobile terminal 40 is low, the power usage amount in the mobile terminal 40 should be limited. Therefore, the division of display roles is determined so that the screen with the larger power consumption required for display among the new screen and the original screen is displayed on the operation panel 30 and the other screen is displayed on the portable terminal 40.

  In the example of FIG. 14B, the preview screen 62 requires more power to display than the operation screen such as the copy setting screen, so the preview screen 62 is displayed on the operation panel 30 and the copy setting screen (1). Is displayed on the portable terminal 40 whose remaining battery level is low.

  In addition, the apparatus state used as the reference | standard which judges which screen of which operation display apparatus to display is suitable is not limited to the restriction | limiting of the electric power which can be used. For example, if one of the operation display devices has less resources such as CPU and memory, the screen that uses more resources for display is not displayed on the operation display device that has less resources. To. Further, the determination may be made in consideration of display capability such as resolution and color reproducibility.

  Further, any of the following display modes may be selected and set, and when the cooperative state is reached, the role assignment may be determined according to the set display mode.

(1) External terminal power state priority mode The image forming apparatus 10 determines the division of display roles by preferentially considering the power state of an external terminal such as the portable terminal 40 juxtaposed to the operation panel 30 of its own apparatus. That is, the display of a screen that consumes more power than the operation screen, such as the preview screen 62 or animation, is displayed when the battery level of the external terminal is less than the threshold or when the external terminal is operating in the power saving mode. The operation screen displayed on the operation panel 30 on the image forming apparatus 10 side and displaying function items and menu items is displayed on the other external terminal.

(2) Operation Panel Power Saving Mode The image forming apparatus 10 determines the division of display roles by giving priority to the power state of the own apparatus. Display of a screen such as the preview screen 62 or moving image animation that consumes more power than the operation screen is displayed on an external terminal such as the portable terminal 40 when the image forming apparatus 10 (operation panel 30) is in the power saving mode. An operation screen that displays function items and menu items is displayed on the operation panel 30. When the image forming apparatus 10 (operation panel 30) is in the power saving mode, an operation screen for displaying function items and menu items is displayed on the external terminal, and the operation panel 30 is turned off in the power saving mode. .

  As described above, in the present embodiment, the role of display when the portable terminal 40 is juxtaposed on the operation panel 30 of the image forming apparatus 10 to be able to communicate with each other is assigned to the current role of the juxtaposed operation display devices. Is determined based on the current device status (power situation, resource availability, etc.) or based on the current device status and the contents of the screen to be displayed. The screen can be displayed with an appropriate division of roles.

  The embodiment of the present invention has been described with reference to the drawings. However, the specific configuration is not limited to that shown in the embodiment, and there are changes and additions within the scope of the present invention. Are also included in the present invention.

  In the embodiment, the case where the operation panel 30 fixed to the image forming apparatus 10 and the portable terminal 40 are juxtaposed and cooperated has been described as an example. However, the operation panel 30 may be removable from the image forming apparatus 10. Alternatively, an individual remote operation panel may be used. The present invention is also applied to the case where the two portable terminals 40 or the operation panel 30 and the remote operation panel are juxtaposed and can communicate with each other.

  In the present invention, the operation panel 30 (operation display device) is configured separately from the image forming apparatus 10, a control unit such as a CPU is provided in the operation panel 30, and the CPU 11 of the image forming apparatus 10 performs in the embodiment. The operation panel 30 may perform control related to display role sharing.

  In the embodiment, the CPU 11 of the image forming apparatus 10 controls the display by determining the division of roles related to the screen display during cooperation. However, a device that performs the control is a separate display control device, for example, a communication A display control server connected to the operation panel 30 or the portable terminal 40 may be used. Further, the present invention may be a display system including two operation display devices juxtaposed and the above display control device. In addition, the present invention may be a program that causes an information processing device including an operation unit and a display unit to function as the operation display device illustrated in this embodiment, or a program that causes an information processing device to function as the display control device. Good.

3 ... Display system 10 ... Image forming apparatus 11 ... CPU
12 ... ROM
13 ... RAM
DESCRIPTION OF SYMBOLS 14 ... Nonvolatile memory 15 ... Hard disk device 16 ... Automatic document conveyance part 16a ... Touch panel 17 ... Network communication part 18 ... Image reading part 19 ... Image processing part 21 ... Printer part 22 ... Facsimile communication part 23 ... Authentication part 30 ... Operation panel 31 Display unit 32 Operation unit 32a Touch panel 33 Proximity wireless communication unit 34 Connection detection unit 40 Mobile terminal 41 CPU
42 ... ROM
43 ... RAM
44 ... Nonvolatile memory 45 ... Authentication unit 46 ... Operation unit 46a ... Touch panel 47 ... Display unit 48 ... Network communication unit 49 ... Proximity wireless communication unit 60 ... Copy setting screen 61 ... Preview display button 62 ... Preview screen

Claims (17)

A detection unit for detecting that the first and second operation display devices each including an operation unit and a display unit are juxtaposed and in a communicable state;
A fluctuation information acquisition unit that acquires information about a predetermined apparatus state that fluctuates during operation of the first operation display device and the second operation display device;
The division of roles related to the display of the screens of the first operation display device and the second operation display device in the cooperative state is determined based on the information acquired by the variation information acquisition unit, and according to the determined division of roles A control unit that controls display of screens of the first operation display device and the second operation display device;
I have a,
The predetermined device state is a device state relating to a limit of usable power,
When the control unit displays a new screen on one of the first operation display device and the second operation display device, the power consumption required for display is displayed among the new screen and the original screen. The display control apparatus according to claim 1 , wherein the division of roles is determined so that a larger number of screens are not displayed on an operation display apparatus having a limited usable power . A detection unit for detecting that the first and second operation display devices each including an operation unit and a display unit are juxtaposed and in a communicable state;
A fluctuation information acquisition unit that acquires information about a device state relating to a resource usage state of the first operation display device and the second operation display device;
The division of roles related to the display of the screens of the first operation display device and the second operation display device in the cooperative state is determined based on the information acquired by the variation information acquisition unit, and according to the determined division of roles A control unit that controls display of screens of the first operation display device and the second operation display device;
Have
When the control unit displays a new screen on either the first operation display device or the second operation display device, the usage amount of the resource required for display out of the new screen and the original screen A display control device that determines the division of roles so as not to display a screen with more resources on an operation display device with fewer resources .   The resource is a CPU or a memory;
  When the resource is a CPU, the usage status of the resource includes the amount of processing load on the CPU,
  When the resource is a memory, the usage status of the resource includes an available free memory capacity.
  The display control apparatus according to claim 2. The new screen is a screen that is displayed in response to a predetermined operation on either the first operation display device or the second operation display device, and the original screen is an operation display that has received the predetermined operation. This is the screen that was displayed on the device
  The display control apparatus according to claim 1, wherein the display control apparatus is a display control apparatus. The new screen, the display control device according to any one of claims 1 to 4, characterized in that a screen for displaying the preview image. The new screen, the display control device according to any one of claims 1 to 5, characterized in that a screen for displaying the video. The control unit re-determines the role assignment based on the changed device state when the device state changes after determining the role assignment, and performs the control according to the re-determined role assignment. the display control device according to any one of claims 1 to 6, wherein. First and second operation display devices each including an operation unit and a display unit;
A display control device according to any one of claims 1 to 7 ,
A display system characterized by comprising: An operation display device including an operation unit and a display unit,
A detection unit for detecting that the self-operation display device and another operation display device are juxtaposed and in a communicable cooperative state;
A fluctuation information acquisition unit that acquires information on a predetermined apparatus state that fluctuates during operation of the self-operation display apparatus and the other operation display apparatus;
The division of roles related to the display of the screen of the cooperative operation display device and the other operation display device is determined based on the information acquired by the variation information acquisition unit, and the self-operation display device according to the determined division of roles And a control unit for controlling display of the screen of the other operation display device,
I have a,
The predetermined device state is a device state relating to a limit of usable power,
When the control unit displays a new screen on either the self-operation display device or the other operation display device, the screen of the new screen and the original screen that requires more power to be displayed. The role assignment is determined so as not to be displayed on the operation display device whose usable power is limited . An operation display device including an operation unit and a display unit,
A detection unit for detecting that the self-operation display device and another operation display device are juxtaposed and in a communicable cooperative state;
A fluctuation information acquisition unit that acquires information on the device status related to the resource usage status of the self-operation display device and the other operation display device;
The division of roles related to the display of the screen of the cooperative operation display device and the other operation display device is determined based on the information acquired by the variation information acquisition unit, and the self-operation display device according to the determined division of roles And a control unit for controlling display of the screen of the other operation display device,
Have
When the control unit displays a new screen on either the self-operation display device or the other operation display device, the one that uses a larger amount of resources required for display among the new screen and the original screen. An operation display device that determines the division of roles so that a screen is not displayed on an operation display device with fewer resources .   The resource is a CPU or a memory;
  When the resource is a CPU, the usage status of the resource includes the amount of processing load on the CPU,
  When the resource is a memory, the usage status of the resource includes an available free memory capacity.
  The operation display device according to claim 10. The new screen is a screen that is displayed in response to a predetermined operation on either the self-operation display device or the other operation display device, and the original screen is displayed on the operation display device that has received the predetermined operation. The screen was
  The operation display device according to claim 9, wherein the operation display device is a display device. The operation display device according to claim 9 , wherein the new screen is an image preview screen. The operation display device according to claim 9 , wherein the new screen is a screen that displays a moving image. The control unit re-determines the role assignment based on the changed device state when the device state changes after determining the role assignment, and performs the control according to the re-determined role assignment. The operation display device according to any one of claims 9 to 14 , wherein Information processing device
A program that functions as the display control device according to any one of claims 1 to 7. An information processing apparatus including an operation unit and a display unit
A program causing the operation display device according to any one of claims 9 to 15 to function.

Images