JP2015141629A - Cooperative display system, display device and program of the same, and cooperative display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a cooperative display system, a display device and a program of the same, and a cooperative display method that can switch which screen displayed by a display section is made to be a primary screen depending on a situation when display sections owned by separate display devices are juxtaposed and content of display is associated.SOLUTION: When display sections of a first operation display device and a second operation display device are juxtaposed, it is determined which screen displayed by the first operation display device and the second operation display is made to be a primary screen according to a predetermined criterion. Thereafter, cooperative display is executed that makes the first operation display device and the second operation display device display a screen based on the screen determined to be the primary screen.

Description

  The present invention relates to a cooperative display system, a display device, a program thereof, and a cooperative display method for linking display contents displayed on two display units.

  The display area of the display unit is divided into two, each displaying different screens, and when the user selects which screen to display, only the selected screen is displayed in the entire display area. Japanese Patent Application Laid-Open No. H10-228561 discloses a technique for displaying using the above. Specifically, in Patent Document 1, both a television screen and a map screen are displayed on one display, and when one screen is selected, only the selected screen is expanded and displayed on the display. .

JP 2000-346651 A

  There is a method of enlarging and displaying one screen by juxtaposing a plurality of display devices and using their display units. In this method, the user has to select which of the screens displayed by the plurality of display units is to be enlarged and displayed every time the enlarged display is performed. Therefore, it is desired to automatically select which screen is enlarged and displayed. In the case where the screen displayed on the display device serving as the parent device is fixedly selected, there is a problem that the screen displayed on the display device serving as the child device cannot be enlarged. .

  The invention described in Patent Document 1 is a method of displaying one selected screen in a large size in a state where two screens are displayed on one display unit, and among the screens displayed by a plurality of display devices It does not select which display device the main screen is to be displayed.

  In the present invention, when display units of separate display devices are juxtaposed and their display contents are linked, switching which display unit is mainly displayed is switched depending on the situation. It is an object of the present invention to provide a cooperative display system, a display device, a program for the same, and a cooperative display method.

  The gist of the present invention for achieving the object lies in the inventions of the following items.

[1] First and second operation display devices including an operation unit and a display unit;
A detection unit for detecting that the display unit of the first operation display device and the display unit of the second operation display device are juxtaposed;
After the detection, a screen based on the main screen is determined based on a predetermined determination criterion as to which one of the first operation display device and the second operation display device is to be used as the main screen. A coordinated display control unit that performs coordinated display to display the first operation display device and the second operation display device
A cooperative display system characterized by comprising:

  In the invention according to the above [1] and the following [9], [17], and [18], when the display units of the first and second operation display devices are juxtaposed, the first operation display device and the second operation display device are arranged. Which of the screens to be displayed by the display device is determined according to a predetermined criterion, and a screen based on the main screen is displayed on the first operation display device and the second operation display device. . Thereby, according to the situation, it can be switched which display unit mainly displays the screen displayed. Note that the predetermined determination criteria include, for example, the operation status and display contents.

[2] Operations performed by the first operation display device and the second operation display device before the detection, or until the cooperative display is started after the detection, and the first operation display device The cooperative display system according to [1], wherein an operation received by the second operation display device is included in the predetermined determination criterion.

  In the invention according to the above [2] and the following [10] and [19], the operations received by the first and second operation display devices before the display units of the first and second operation display devices are juxtaposed, Alternatively, operations received by the first and second operation display devices after the juxtaposition and before the cooperative display is started are included in the predetermined determination criterion. For example, based on the presence / absence of the operation, the time when the operation is received, the type of operation, the operation content, etc., any one of the screens displayed on the first operation display device and the second operation display device is mainly used. Determine whether.

[3] The cooperative display control unit displays the screen displayed on the operation display device that has been operated last before the detection among the first operation display device and the second operation display device. The cooperative display system according to [1] or [2], wherein

  In the inventions according to the above [3] and the following [11] and [20], before the display units of the first and second operation display devices are juxtaposed, they are displayed on the operation display device that last received the operation. Change the main screen to the main screen. Thereby, even if a cooperation display is started, the screen of the operation display apparatus which received operation just before a cooperation display starts is displayed continuously. The operation here is not limited to a selection operation for selecting which screen is the main one, but includes a normal operation such as a setting operation.

[4] The linked display according to [3], wherein the linked display control unit does not perform the linked display when a predetermined time has elapsed since the last operation before the detection was received. system.

  In the invention according to the above [4] and the following [12] and [21], when a predetermined time has passed since the last operation before the display units of the first and second operation display devices were juxtaposed, Since the operation is highly likely to be an operation unrelated to the operation received after the cooperative display is started, the cooperative display is not performed.

[5] The display contents displayed by the first operation display device and the second operation display device before the detection are included in the predetermined determination criteria. [1] to [4] The linkage display system according to any one of the above.

  In the inventions according to [5] and [13] and [22] below, the first and second operation display devices display before the display units of the first and second operation display devices are juxtaposed. The displayed contents are included in predetermined criteria. For example, if an advertisement screen and a setting screen are displayed, if the setting screen is the main screen, the setting screen is displayed even after the start of the cooperative display. Occurrence of a situation of being reset can be prevented.

[6] The cooperative display control unit selects a screen having a portion that cannot be displayed among the screens displayed by the first operation display device and the second operation display device before the detection. The cooperative display system according to any one of [1] to [5], characterized in that:

  In the invention according to the above [6] and the following [14] and [23], the first and second operation display devices display immediately before the display units of the first and second operation display devices are juxtaposed. The main screen is the screen that cannot be displayed. A screen having a portion that cannot be displayed is a screen of a size that cannot be entirely displayed on one display portion at a time.

[7] The detection unit recognizes a relative positional relationship between the first operation display unit and the second operation display unit at the time of the detection,
The coordinated display control unit performs the coordinated display only when there is another operation display device in a direction where there is a portion that cannot be displayed with respect to the display unit displaying the main screen. [6] The cooperative display system according to [6].

  In the invention according to [7] and [15] and [24] below, when the display units of the first and second operation display devices are juxtaposed, the relative positional relationship between the two display units is recognized. Then, the cooperative display is performed only when there is the other operation display device in a direction where there is a portion that cannot be displayed on the display unit on which the main screen is displayed.

[8] The cooperative display control unit is configured to display either one of the screens displayed on the first operation display device and the second operation display device when the determination cannot be made based on the predetermined determination criterion. The linked display system according to any one of [1] to [7], wherein a selection is received from a user, and the selected screen is set as the main screen.

  In the inventions according to [8] and [16] and [25] below, when the main screen cannot be determined based on a predetermined determination criterion, the screen displayed by the first operation display device and the second operation display device Is selected from the user, and the selected screen is set as a main screen.

[9] an operation unit;
A display unit;
A communication unit that transmits and receives data to and from an external display device including an operation unit and a display unit;
A detection unit for detecting that the display unit of the device itself and the display unit of the external display device are juxtaposed;
After the detection, it is determined according to a predetermined criterion whether the screen displayed on the own device or the external display device is the main screen, and the screen based on the main screen is the display unit of the own device. A control unit that performs cooperative display to be displayed on the external display device;
A display device comprising:

[10] Operations received by the device and the external display device before the detection, or operations performed by the device and the external display device after the detection and before the linked display is started. Is included in the predetermined criterion. The display device according to [9].

[11] The cooperative display control unit makes the main screen the screen displayed on the display device that has been lastly operated before the detection among the own device and the external display device. The display device according to [9] or [10].

[12] The display device according to [11], wherein the cooperation display control unit does not perform the cooperation display when a predetermined time has elapsed since the last operation before the detection was received. .

[13] The display content displayed by the device itself and the external display device before the detection is included in the predetermined determination criterion. Display device.

[14] The linked display control unit sets, before the detection, a screen having a portion that cannot be displayed among the screens displayed by the own device and the external display device as the main screen. The display device according to any one of [9] to [13], which is characterized.

[15] At the time of the detection, the detection unit recognizes a relative positional relationship between the display unit of the device itself and the display unit of the external display device,
In the case where the screen displayed by the display unit of the device itself is the main screen, the cooperative display control unit is configured so that the external display is in a direction in which there is a portion that cannot be displayed with respect to the display unit of the device itself. Only when there is a display unit of a display device, the cooperative display is performed, and when the screen displayed by the external display device is the main screen, the display unit of the external display device is [14] The display device according to [14], wherein the cooperative display is performed only when the display unit of the own device is in a direction where there is a portion that cannot be displayed.

[16] The cooperative display control unit receives, from the user, selection of one of the screens displayed by the own device and the external display device when the determination cannot be made according to the predetermined determination criterion, The display device according to any one of [9] to [15], wherein the selected screen is the main screen.

[17] An information processing apparatus including an operation unit and a display unit,
[9] A program that functions as the display device according to any one of [16].

[18] A detection step of detecting that the display units of the first and second operation display devices including the operation unit and the display unit are juxtaposed,
After the detection, a screen based on the main screen is determined based on a predetermined determination criterion as to which one of the first operation display device and the second operation display device is to be used as the main screen. A coordinated display step for performing coordinated display for displaying the first operation display device and the second operation display device;
A cooperative display method characterized by comprising:

[19] An operation received by the first operation display device and the second operation display device before the detection, or the first operation after the detection and before the cooperative display is started. The cooperative display method according to [18], wherein an operation received by the display device and the second operation display device is included in the predetermined determination criterion.

[20] In the cooperative display step, the screen displayed on the operation display device that has been operated last before the detection among the first operation display device and the second operation display device is set as the main display. The cooperative display method according to [18] or [19], wherein the screen is a screen to be displayed.

[21] The linked display method according to [20], wherein in the linked display step, the linked display is not performed when a predetermined time has elapsed since the last operation before the detection was received. .

[22] The display contents displayed by the first operation display device and the second operation display device before the detection are included in the predetermined determination criteria. [18] to [21] The linkage display method according to any one of the above.

[23] In the cooperative display step, a screen having a portion that cannot be displayed among the screens displayed by the first operation display device and the second operation display device before the detection is defined as the main screen. The cooperative display method according to any one of [18] to [22], wherein the screen is a screen to be displayed.

[24] In the detection step, at the time of the detection, a relative positional relationship between the first operation display unit and the second operation display unit is recognized,
In the linked display step, the linked display is performed only when there is another operation display device in a direction where there is a portion that cannot be displayed with respect to the display unit displaying the main screen. The cooperative display method according to [23], which is characterized.

[25] In the cooperative display step, when the determination cannot be made based on the predetermined determination criterion, one of the screens displayed on the first operation display device and the second operation display device is selected. The linked display method according to any one of [18] to [24], wherein the selected screen is used as the main screen.

  According to the coordinated display system, the display device and the program thereof, and the coordinated display method according to the present invention, when the display units of separate display devices are juxtaposed and their display contents are coordinated, It is possible to switch between the main screen displayed by the display unit.

It is explanatory drawing which shows the image forming apparatus which concerns on embodiment of this invention, and a portable terminal. 1 is a block diagram illustrating a schematic configuration of an image forming apparatus according to an embodiment of the present invention. It is a figure which shows the example of arrangement | positioning of a sensor in an operation panel and a portable terminal. It is a block diagram which shows schematic structure of the portable terminal which concerns on embodiment of this invention. It is a figure which shows a mode that the operation panel and a portable terminal display one screen among the several screens by which the arrangement | sequence order was decided previously. It is a figure which shows a mode in the case of trying to juxtapose a portable terminal on the right side of an operation panel. It is a figure which shows a mode at the time of making a portable terminal juxtapose the right side of an operation panel. It is a figure which shows a mode that it moves toward the operation panel 30 while operating with a portable terminal. It is a figure which shows the example of a cooperation display when operation is performed to one apparatus before the start of a cooperation display. It is a figure which shows the example of a cooperation display when the priority button is selected with the portable terminal before juxtaposing It is a figure which shows the example of a cooperation display when the priority button is selected on the operation panel before juxtaposing. It is a figure which shows the example of a display in the case of receiving selection of which screen is made into a main screen in an operation panel. It is a figure which shows the example of a display in the case of receiving selection of which screen is made into a main screen in a portable terminal. It is a figure which shows the example of a display in the case of receiving selection of which screen is made into a main screen in either an operation panel or a portable terminal. It is a figure which shows a mode in the case of trying to juxtapose a portable terminal with the operation panel which is displaying the list | wrist. It is a figure which shows a mode when a portable terminal is juxtaposed on the operation panel which is displaying the list. It is a figure which shows a mode when a portable terminal is juxtaposed in the direction without a continuation part. It is a figure which shows the operation panel and portable terminal which are displaying the screen which has a continued part mutually. It is a figure which shows the example of a cooperation display in case a portable terminal is located in the direction with a continuation part of an operation panel from the state of FIG. It is a figure which shows the example of a cooperation display in case the operation panel is located in the direction with a continuation part of a portable terminal from the state of FIG. It is a figure which shows a mode in the case of making the other apparatus juxtapose in a direction with a continuing part mutually. It is a figure which shows a mode that it receives from a user the mode of receiving selection of which screen is made into a main screen, when the other apparatus is juxtaposed in the direction with a continuous part mutually. It is a figure which shows the example of screens other than a list. It is a figure which shows the example in the case of performing a cooperation display on screens other than a list. It is a flowchart which shows the process which judges which screen is made into a main screen before performing a cooperation display. FIG. 26 is a flowchart showing a continuation of the process of FIG. 25. FIG. 27 is a flowchart showing processing that is a continuation of the processing in FIG. 25 and that is branched from the processing in FIG. 26.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows a cooperative display system 5 according to an embodiment of the present invention. In the cooperative display system 5, the image forming apparatus 10 having the function as the first operation display device of the present invention and the portable terminal 40 having the function as the second operation display device of the present invention are connected to a LAN (Local Area Network). 3 are connected.

  The image forming apparatus 10 is a copy job that optically reads a document and prints a duplicate image on a recording sheet, a scan job that saves image data of the read document as a file, or transmits the image data to an external device, and a PC ( A so-called multi-function machine having a function of executing a job such as a print job (print job) for printing an image relating to data sent from a personal computer) on a recording sheet. The image forming apparatus 10 includes an operation panel 30 including a display unit 31 and an operation unit 32 (see FIG. 2). The operation panel 30 receives setting operations related to various settings and job input from the user.

  The portable terminal 40 is a portable terminal and includes an operation display unit 50 including a display unit 51 and an operation unit 52 (see FIG. 4). The portable terminal 40 receives a setting operation related to creation of a print job from the user. The portable terminal 40 has a function of transmitting a print job to the image forming apparatus 10 after creating the print job.

  In the embodiment of the present invention, when the operation panel 30 and the portable terminal 40 are juxtaposed, the coordinated display system 5 can coordinate the display contents of both devices (hereinafter referred to as coordinated display). When executing the cooperative display, the image forming apparatus 10 determines which of the operation panel 30 and the portable terminal 40 is the main screen displayed according to a predetermined determination criterion, and the main screen ( Hereinafter, the screen based on the main screen is displayed on the operation panel 30 and the portable terminal 40. For example, when the screen displayed on the operation panel 30 is a main screen, a screen corresponding to the main screen is displayed on the mobile terminal 40.

  Since it is determined which of the operation panel 30 and the portable terminal 40 is used as the main screen in accordance with a predetermined determination criterion, the cooperative display system 5 is connected to the operation panel 30 and the portable device according to the situation. It is possible to switch which of the terminals 40 performs linked display with the screen being displayed as the main screen. Details of the predetermined determination criteria and linkage display will be described later.

  FIG. 2 is a block diagram showing a schematic configuration of the image forming apparatus 10 according to the embodiment of the present invention. The image forming apparatus 10 includes a CPU (Central Processing Unit) 11 that comprehensively controls the operation of the image forming apparatus 10. The CPU 11 has a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a nonvolatile memory 14, a hard disk device 15, an image processing unit 16, an image reading unit 17, a printer unit 18, A facsimile communication unit 19, a network communication unit 20, a sensor 21, an ADF (Auto Document Feeder) 22, an authentication unit 23, and an operation panel 30 are provided.

  The CPU 11 is based on an OS (Operating System) program, and executes middleware, application programs, and the like. The CPU 11 also functions as a control unit that controls the display content of the operation panel 30.

  Various programs are stored in the ROM 12, and each function of the image forming apparatus 10 is realized by the CPU 11 executing various processes according to these programs. The ROM 12 stores a program for the CPU 11 to execute a series of controls of the image forming apparatus 10.

  The RAM 13 is used as a work memory for temporarily storing various data when the CPU 11 executes processing based on a program, an image memory for storing image data, and the like.

  The nonvolatile memory 14 is a memory (flash memory) whose stored contents are not destroyed even when the power is turned off, and is used for storing various setting information.

  The hard disk device 15 is a large-capacity nonvolatile storage device, and performs cooperative display such as an OS program, various application programs, user information, print data and image data, job history, importance of each operation, importance of each screen, and the like. In this case, various information used for determining the screen to be the main screen is stored.

  The image processing unit 16 performs processing such as image enlargement / reduction, rotation, rasterization processing for converting print data into image data, image data compression / decompression processing, and the like.

  The image reading unit 17 performs a function of optically reading a document and acquiring image data. The image reading unit 17 sequentially, for example, a light source that irradiates light on the document, a line image sensor that receives the reflected light for one line in the width direction, and a line-by-line reading position in the length direction of the document. An optical path composed of a moving unit for moving, a lens, a mirror, and the like for guiding reflected light from the document to the line image sensor to form an image, and a conversion unit for converting an analog image signal output from the line image sensor into digital image data And so on.

  The printer unit 18 has a function of forming an image corresponding to image data on a recording sheet. Here, it has a recording paper transport device, a photosensitive drum, a charging device, a laser unit, a developing device, a transfer separation device, a cleaning device, and a fixing device, and forms an image by an electrophotographic process. It is configured as a so-called laser printer. Other methods may be used for image formation.

  The facsimile communication unit 19 performs a function of transmitting / receiving image data to / from an external device having a facsimile function through a public line.

  The network communication unit 20 fulfills a function of communicating data with the mobile terminal 40 and other external devices through a network such as a LAN. In the cooperation panel mode, the network communication unit 20 transmits display data of a screen that cooperates with the screen displayed on the operation panel 30 to the mobile terminal 40.

  The ADF 22 functions as an automatic document feeder that continuously feeds a plurality of documents to the document table of the image reading unit 17 when reading an image. It is possible to set about 30 originals at a time, and it is possible to save the trouble of setting the original on the original table each time the original is read.

  The authentication unit 23 has a function of performing user authentication. Specifically, user authentication is performed when a user logs in, and a login user is specified. In the present embodiment, it is assumed that login is required to use the image forming apparatus 10 and user authentication is always performed before use. The user authentication method may be any suitable method such as user ID (Identification) and password authentication or fingerprint authentication.

  The operation panel 30 includes a display unit 31 and an operation unit 32. The operation unit 32 includes a switch unit 33 such as a start button and a return button, and a touch panel unit 34. The display unit 31 is composed of a liquid crystal display (LCD) and the like, and has a function of displaying various operation screens, setting screens, and the like. In the present embodiment, the CPU 11 performs control of display contents displayed on the operation panel 30 and control related to reception of operations by the operation panel 30.

  The touch panel unit 34 is provided on the display unit 31. The touch panel unit 34 detects a touch position (coordinate position) on the display unit 31 pressed with a touch pen or a finger. The CPU 11 recognizes a flick operation, a drag operation, and the like from the continuously detected touch position change (operation information).

  The sensors 21 are installed at a total of four locations on the side surface of the operation panel 30 one by one in different directions, up, down, left, and right, and determine whether or not the mobile terminals 40 are juxtaposed. Specifically, the sensor 21 determines whether or not the mobile terminal 40 is juxtaposed depending on whether or not communication with a sensor 47 of the mobile terminal 40 described later is possible. FIG. 3A shows an example of a place where the sensor 21 is installed.

  For example, the sensor 21 of the operation panel 30 can communicate only with the sensor 47 of the portable terminal 40 in the direction of ± 15 degrees from the central axis in the direction in which the sensor 21 faces. In the embodiment, it is assumed that the sensor 21 has a very short communicable distance and cannot communicate unless it is in contact with the sensor 47. When the sensor 21 can communicate with the sensor 47 of the mobile terminal 40, the CPU 11 determines that the mobile terminal 40 is juxtaposed in the direction in which the sensor 21 faces. The above is an example, and another sensor or method may be used for detecting that the mobile terminals 40 are juxtaposed.

  FIG. 4 is a block diagram illustrating a schematic configuration of the mobile terminal 40. The mobile terminal 40 has a CPU 41 that controls the operation of the mobile terminal 40 in an integrated manner. The CPU 41 includes a ROM 42, a RAM 43, a nonvolatile memory 44, a hard disk device 45, a network communication unit 46, a sensor 47, an authentication unit 48, and an operation display unit 50 through a bus.

  The CPU 41 is based on the OS program, and executes middleware, application programs, and the like. In addition, the CPU 41 functions as a control unit that controls the display content of the operation display unit 50.

  Various programs are stored in the ROM 42, and each function of the portable terminal 40 is realized by the CPU 41 executing various processes according to these programs. The ROM 42 stores a program for the CPU 41 to execute a series of controls of the portable terminal 40.

  The RAM 43 is used as a work memory for temporarily storing various data when the CPU 41 executes processing based on a program.

  The nonvolatile memory 44 is a memory (flash memory) whose stored contents are not destroyed even when the power is turned off, and various setting information and a program for displaying in cooperation with the operation panel 30 (this is an operation display application). Etc. are memorized.

  The hard disk device 45 is a large-capacity nonvolatile storage device, and stores an OS program, various application programs, and the like.

  The network communication unit 46 performs a function of communicating data with the image forming apparatus 10 and other external devices through a network such as the LAN 3.

  The authentication unit 48 functions to perform user authentication. Specifically, user authentication is performed when a user logs in, and a login user is specified. In the present embodiment, it is assumed that login is required to use the mobile terminal 40, and user authentication is always performed before use. Note that the user authentication method may be any appropriate method such as user ID and password authentication or fingerprint authentication.

  The operation display unit 50 includes a display unit 51 and an operation unit 52. The operation unit 52 includes a switch unit 53 such as a start button and a return button and a touch panel unit 54. The display unit 51 is configured by a liquid crystal display or the like, and fulfills a function of displaying various operation screens, setting screens, and the like.

  The touch panel unit 54 is provided on the display unit 51. The touch panel unit 54 detects a touch position (coordinate position) on the display unit 51 pressed with a touch pen or a finger. The CPU 41 recognizes a flick operation, a drag operation, a multi-touch operation, and the like from a continuously detected change in touch position.

  The sensors 47 are installed at a total of four locations on the side of the mobile terminal 40, one in each of the up, down, left, and right directions, and communicate with the sensor 21 of the operation panel 30. FIG. 3B shows an example of a place where the sensor 47 is installed.

  Note that the image forming apparatus 10 has an operation panel depending on which sensor 47 of the plurality of sensors 47 included in the mobile terminal 40 is in communication with which sensor 21 of the plurality of sensors 21 included in the operation panel 30. It is recognized in what direction the portable terminal 40 is juxtaposed with respect to the terminal 30. For example, when the sensor 21 (A) of the operation panel 30 in FIG. 3 and the sensor 47 (A) of the mobile terminal 40 can communicate, the upper side of the mobile terminal 40 faces the lower side of the operation panel 30. When the sensors 21 (A) and 47 (B) can communicate with each other, it is recognized that the right side of the portable terminal 40 faces the lower side of the operation panel 30 so as to face each other.

  In the embodiment of the present invention, it is assumed that the image forming apparatus 10 does not perform cooperative display depending on the orientation of the mobile terminal 40 juxtaposed with the operation panel 30. Specifically, the lower side of the mobile terminal 40 is above the operation panel 30, the upper side of the mobile terminal 40 is below the operation panel 30, the left side of the mobile terminal 40 is the right side of the operation panel 30, and the operation When the left side of the panel 30 is juxtaposed so that the right side of the mobile terminal 40 faces, the cooperative display is performed.

  Next, a flow until the image forming apparatus 10 (operation panel 30) and the portable terminal 40 start cooperative display will be described.

  First, the image forming apparatus 10 and the portable terminal 40 establish communication. And if communication is established, it will wait for the operation panel 30 and the portable terminal 40 to be juxtaposed. When the operation panel 30 and the portable terminal 40 are juxtaposed, it is determined which of the screens displayed on the operation panel 30 and the portable terminal 40 is the main screen. Thereafter, the image forming apparatus 10 and the portable terminal 40 shift the display mode of the own apparatus to the cooperative display mode and start cooperative display.

  Next, establishment of communication will be described. Communication can be established from either the image forming apparatus 10 or the portable terminal 40.

(1. When establishing communication from the image forming apparatus 10 side)
When establishing communication from the image forming apparatus 10 side, first, by the user's operation, the image forming apparatus 10 checks the communication status with the portable terminal 40 designated by the user. Communication with 40 is established.

(2. When establishing communication from the mobile terminal 40 side)
When establishing communication from the mobile terminal 40 side, first, the cooperative display application is started on the mobile terminal 40, the application issues a communication establishment request to the image forming apparatus 10, and the image forming apparatus 10 that has received the communication establishment request. Communication is established.

  After the mobile terminal 40 and the image forming apparatus 10 establish communication, when the user juxtaposes the mobile terminal 40 with the operation panel 30, the CPU 11 recognizes that the mobile terminal 40 and the operation panel 30 are juxtaposed using the sensor 21. To do.

  When the image forming apparatus 10 detects that the portable terminal 40 is juxtaposed on the operation panel 30, the image forming apparatus 10 determines which of the screens displayed on the operation panel 30 and the portable terminal 40 is the main screen. An information transmission request is transmitted to the portable terminal 40. Receiving this, the mobile terminal 40 transmits the information to the image forming apparatus 10. For example, the portable terminal 40 transmits to the image forming apparatus 10 the time when the last operation was received, information about the screen being displayed, and the like as a response to the transmission request. Upon receiving this information, the image forming apparatus 10 determines which of the screens displayed on the operation panel 30 and the portable terminal 40 is the main screen. Thereafter, the image forming apparatus 10 and the portable terminal 40 shift the display mode of the own apparatus to the cooperative display mode and start cooperative display.

  When the screen displayed on the operation panel 30 is to be the main screen, the image forming apparatus 10 transmits screen information (screen data or the like) corresponding to the main screen to the mobile terminal 40 and receives the mobile terminal. 40 displays a screen corresponding to the main screen based on the information.

  When the screen displayed by the mobile terminal 40 is used as the main screen, the image forming apparatus 10 acquires information on the main screen and displays a screen corresponding to the main screen on the display unit 31 based on the information.

Next, what kind of screen the operation panel 30 and the portable terminal 40 display in the cooperative display will be described. What screen is displayed on the operation panel 30 and the portable terminal 40 when performing the cooperative display is determined by the screen to be the main screen. For example,
(Pattern 1) When the main screen is one of a plurality of screens in the order of arrangement, the main screen and a screen adjacent thereto are displayed. (Pattern 2) The main screen is displayed on both devices. There are patterns such as displaying a main screen that is extended and displayed (pattern 3) and another screen associated with the main screen.

  In (Pattern 3), the corresponding screen is stored in advance in association with each screen, and when performing cooperative display, the main screen and the screen associated with the main screen are displayed on the operation panel 30 and the mobile terminal 40. Is displayed. For example, if the copy screen and the post-processing screen are associated with each other, both screens are displayed simultaneously in the cooperative display, so that settings relating to the two screens can be performed simultaneously.

  In the following, specific examples of the above (Pattern 1) and (Pattern 2) are given, and predetermined judgment criteria that can be adopted in the case of Pattern 1 and Pattern 2 will be described.

(Specific example of pattern 1)
FIG. 5 shows an example of a screen displayed on the image forming apparatus 10 and the portable terminal 40. In FIG. 5, the image forming apparatus 10 and the mobile terminal 40 display one screen among a plurality of screens (screen group 80) whose arrangement order is determined. The screen group 80 is composed of 52 screens (screen group 80) arranged in the horizontal direction. The first screen, the second screen, the third screen,... In this order from the left side (the X screen in the figure). (X is an arbitrary integer) is written as NoX.) The screens are arranged in this order.

  In FIG. 5, the image forming apparatus 10 (operation panel 30) displays the third screen, and the portable terminal 40 displays the 50th screen. In addition, the operation panel 30 and the mobile terminal 40 can advance or return the displayed screen by scrolling the screen in the horizontal direction (left and right).

  FIG. 6 shows that after the image forming apparatus 10 and the portable terminal 40 are in the cooperative display mode, the portable terminal 40 that is displaying the 50th screen is juxtaposed to the right side of the operation panel 30 that is displaying the third screen. It shows a state.

  FIG. 7 shows a screen displayed by cooperative display after the operation panel 30 and the portable terminal 40 are juxtaposed in FIG. In FIG. 7, the mobile terminal 40 is juxtaposed on the right side of the operation panel 30, and the operation panel 30 and the mobile terminal 40 display two consecutive screens in the screen group 80. Specifically, FIG. 7 is an example in which the 50th screen displayed on the mobile terminal 40 is the main screen. The mobile terminal 40 continues to display the 50th screen as it is, and the operation panel 30 is displayed on the mobile terminal. Since it is on the left side of 40, the 49th screen on the left side of the screen displayed by the mobile terminal 40 is displayed.

  When the third screen displayed on the operation panel 30 is used as the main screen, the operation panel 30 continues to display the third screen as it is, and the mobile terminal 40 is located on the right side of the operation panel 30. The fourth screen on the right side of the screen is displayed.

  Which of the left and right screens adjacent to the main screen is displayed as a screen corresponding to the main screen in the cooperative display is determined by the relative positional relationship between the operation panel 30 and the portable terminal 40. Specifically, the arrangement direction of the operation panel 30 and the portable terminal 40, the main screen, and the arrangement direction of the screens corresponding to the main screen (two consecutive screens) are determined to be the same. In FIG. 7, the operation panel 30 is located on the left side of the mobile terminal 40 that displays the main screen (the 50th screen), and therefore displays the screen on the left side of the 50th screen (the 49th screen). When the mobile terminal 40 is juxtaposed on the left side of the operation panel 30, the operation panel 30 is located on the right side of the mobile terminal 40. Display the screen.

  Next, based on the case of (Pattern 1), determination criteria 1 to 5 serving as predetermined determination criteria will be described.

<Judgment criteria 1, when the main screen is the screen displayed by the device that has received the last operation before starting the linked display>
In the criterion 1, the device that received the last operation immediately before the operation panel 30 and the portable terminal 40 are juxtaposed between the operation panel 30 and the portable terminal 40, or just after the juxtaposition, the cooperative display is started. The screen displayed by the device that has received the last operation until now is defined as the main screen. In the embodiment of the present invention, the last operation received immediately after the operation panel 30 and the portable terminal 40 are juxtaposed and before the cooperative display is started is the first operation received after the juxtaposition. Let it be an operation. The case where this criterion is adopted will be described with reference to FIGS.

  FIG. 8 shows a state in which the user approaches the operation panel 30 while operating the mobile terminal 40. Here, it is assumed that the user is moving from his / her seat toward the image forming apparatus 10 while performing a setting operation related to printing on the portable terminal 40 in an office or the like. In this case, it is assumed that the user continues the setting operation on the portable terminal 40 even after the portable terminal 40 is juxtaposed on the operation panel 30. Therefore, the image forming apparatus 10 receives the first operation when the portable terminal 40 has received the last operation immediately before the operation panel 30 and the portable terminal 40 are juxtaposed or immediately after the juxtaposition. In this case, the screen displayed on the mobile terminal 40 is set as the main screen.

  In the case where the user performs setting operation on the image forming apparatus 10, in order to improve the visibility of the operation panel 30, the portable terminal 40 is taken out and juxtaposed with the operation panel 30 to perform cooperative display. Even after the juxtaposition, it is assumed that the setting operation is continued on the image forming apparatus 10 (operation panel 30). Therefore, when the operation panel 30 receives the last operation immediately before the operation panel 30 and the portable terminal 40 are juxtaposed, or when the operation panel 30 receives the first operation immediately after the juxtaposition. Uses the screen displayed on the operation panel 30 as the main screen.

  FIG. 9 shows a screen displayed in cooperative display when the mobile terminal 40 receives the first operation immediately after the operation panel 30 and the mobile terminal 40 are juxtaposed in FIG. Here, the image forming apparatus 10 determines that the screen displayed on the mobile terminal 40 is the main screen, and the screen displayed on the operation panel 30 is the screen displayed on the mobile terminal 40 (the 50th screen). The screen is switched to the adjacent screen (the 49th screen). The portable terminal 40 continues to display the 50th screen as it is.

  Of the screens displayed on the operation panel 30 and the portable terminal 40, the determination of which screen is the main screen is received immediately after the last operation received immediately before both devices are juxtaposed. It may be performed based on any of the initial operations, and in the embodiment of the present invention, it can be set in advance by the user.

  When making the above determination based only on the last operation immediately before the both devices are juxtaposed, the portable terminal immediately after the image forming apparatus 10 detects that the portable terminal 40 has been juxtaposed with respect to the own device. 40 acquires information on the time when the last operation was performed, and compared the time indicated by the acquired information with the time when the device received the last operation, and both devices are juxtaposed. It is determined whether or not the last operation just before the operation was received.

When making the above determination based only on the first operation immediately after both devices are juxtaposed, after the image forming apparatus 10 detects that the portable terminal 40 is juxtaposed with respect to its own device, the following ( Wait until one of the conditions of A) and (B) is satisfied.
(A) The operation panel 30 receives an operation from the user. (B) A report that the user's operation has been received is received from the portable terminal 40. If (A) is established, the operation panel 30 is displayed. Set the main screen to the main screen. When (B) is established, the screen displayed on the mobile terminal 40 is set as the main screen.

  The above determination may be made based on both the last operation immediately before the two devices are juxtaposed and the first operation immediately after the two devices are juxtaposed. Specifically, after either device is juxtaposed, if either of the conditions (A) and (B) is met within a certain time (for example, within 5 seconds), The above determination is made based on the first operation, and if neither of the conditions (A) and (B) is satisfied within a certain time (for example, within 5 seconds), immediately before both devices are juxtaposed. The above determination is made based on the last operation.

  In addition, if neither of the operation panel 30 and the portable terminal 40 has received an operation immediately before being juxtaposed, it waits until one of the conditions (A) and (B) is satisfied, and is established. If the above determination is made based on the condition and an operation is received immediately before at least one of the devices is juxtaposed, the above determination may be made based on the last operation immediately before the juxtaposition. Good.

  In addition, when a certain time or more has elapsed from when the last operation is received to when the cooperative display is started, there is a high possibility that there is no relationship between the operation and the operation received after the cooperative display is started. Therefore, in the case where the above determination is made based on the last operation immediately before both devices are juxtaposed, even if the operation panel 30 and the mobile terminal 40 are juxtaposed, after receiving the last operation just before juxtaposition. If a certain amount of time has elapsed before starting linked display, linked display is not performed. For example, when both devices are juxtaposed after a certain period of time has elapsed since the operation panel 30 and the portable terminal 40 received the last operation, the cooperative display is not performed.

  In the embodiment of the present invention, when the criterion 1 is adopted, the last operation received immediately before the operation panel 30 and the portable terminal 40 are juxtaposed and the first operation received immediately after the juxtaposition are either Instead of selecting whether to display the screen displayed by the device as the main screen, normal operation such as setting operation related to printing or operation related to screen transition is performed.

<Judgment criterion 2, when the main screen is the screen displayed by a device that has received a specific operation in advance>
In the criterion 2, the screen displayed by the device that has received a specific operation before being juxtaposed between the operation panel 30 and the portable terminal 40 is set as the main screen. 10 and 11 mainly show the screens displayed by the device that has received a specific operation (the device in which the priority button 60 has been selected) before the operation panel 30 and the portable terminal 40 are juxtaposed. Shows the screen. 10 and 11, the operation panel 30 displays the copy screen 1 and the portable terminal 40 displays the FAX screen 3 before being juxtaposed.

  In FIG. 10, the operation panel 30 and the portable terminal 40 are juxtaposed with the priority button 60 selected on the screen of the portable terminal 40. When the portable terminal 40 is juxtaposed on the operation panel 30, the portable terminal 40 notifies the image forming apparatus 10 that the priority button 60 is selected. Based on the notification, the image forming apparatus 10 determines that the screen displayed on the mobile terminal 40 is the main screen. In FIG. 10, the display of the screen of the mobile terminal 40 (FAX screen 3) is maintained as it is by the cooperative display, and the screen displayed on the operation panel 30 is switched to a screen adjacent to the FAX screen 3 (FAX screen 2). Yes.

  In FIG. 11, the operation panel 30 and the portable terminal 40 are juxtaposed with the priority button 60 selected on the screen of the operation panel 30. Since priority button 60 is selected on the screen of operation panel 30, image forming apparatus 10 determines that the screen displayed on operation panel 30 is the main screen when operation panel 30 and portable terminal 40 are juxtaposed. To do. In FIG. 11, the display of the operation panel screen (copy screen 1) is maintained as it is by the cooperative display, and the screen displayed on the mobile terminal 40 is switched to a screen adjacent to the copy screen 1 (copy screen 2).

<Judgment criteria 3, when receiving a selection of the screen to be the main screen from the user>
Judgment criterion 3 is that after the operation panel 30 and the portable terminal 40 are juxtaposed, the user selects which of the two devices displays as the main screen, and the selected device displays Make the screen you used the main screen. The criterion 3 is employed when, for example, the criterion 1, the criterion 2, etc. cannot determine which of the screens displayed on the operation panel 30 and the portable terminal 40 is the main screen. . FIGS. 12-14 shows a mode that said selection is received from a user, after arranging the operation panel 30 and the portable terminal 40 side by side.

  FIG. 12 shows how the operation panel 30 displays two selection buttons (MFP button 61 and tablet button 62) for selecting the main screen. FIG. 13 shows how the mobile terminal 40 displays two selection buttons (MFP button 61 and tablet button 62) for selecting the main screen. FIG. 14 shows a state where both operation panel 30 and portable terminal 40 display two selection buttons (MFP button 61 and tablet button 62) for selecting the main screen.

  The user selects one of the selection buttons of the MFP button 61 and the tablet button 62. When the MFP button 61 is selected, it is determined that the screen of the operation panel 30 is the main screen, and when the tablet button 62 is selected, it is determined that the screen displayed on the portable terminal 40 is the main screen. .

  Note that the method of receiving a selection as to which screen is displayed as the main screen is not limited to the above example. For example, a selection by voice input may be received, or a selection instruction may be received from another device.

<Judgment criteria 4, when judging by the importance of operation>
In the determination criterion 4, it is determined which of the screens displayed on the operation panel 30 and the portable terminal 40 is the main screen based on the importance of the operation received by the operation panel 30 and the portable terminal 40. To do. For example, when the operation panel 30 and the portable terminal 40 are juxtaposed in a situation where both the portable terminal 40 and the operation panel 30 are simultaneously operated by different users, the operation that the operation panel 30 last received immediately before the juxtaposition And the screen which the apparatus of the one which received operation with high importance among the operations which the portable terminal 40 received last is made into a main screen.

  For example, when the importance is set for each type of operation, if the importance of the setting operation is set higher than the operation of browsing the file, the device that has received the operation related to the setting operation When the displayed screen becomes the main screen and the cooperative display is started, the setting contents related to the setting operation remain without being reset.

  The elements for determining the importance of the operation are not limited to the type of operation. For example, the type of setting (print setting / main unit setting / fax setting / copy setting, etc.), the number of items set, and the setting operation It may be time. Moreover, you may perform this determination combining them. For example, when the setting operation related to the print setting is received on both the operation panel 30 and the portable terminal 40, the screen displayed by the device whose number is changed from the default setting is the main screen, The screen displayed by the device with the longer operation time is used as the main screen.

<Judgment criteria 5, display contents (when judging by the importance of the screen)>
In the criterion 5, the main screen is determined based on the importance of the screen displayed on the operation panel 30 and the portable terminal 40 immediately before juxtaposition. When the criterion 5 is adopted, each screen is weighted in advance, and the importance of the screen displayed on the operation panel 30 and the portable terminal 40 when they are juxtaposed is compared. Is the main screen. For example, by setting the importance of the setting screen higher than that of the advertisement screen or home screen, it is possible to prevent a situation in which the content set on the setting screen is reset at the start of linked display .

  Next, (pattern 2: when the main screen is extended and displayed using both devices) will be described with a specific example.

(Specific example of pattern 2)
15 and 16 show an example in which one screen is extendedly displayed using two display units (31, 51). FIG. 15 shows a state in which the portable terminal 40 is juxtaposed below the operation panel 30. In FIG. 15, the operation panel 30 displays the upper part of the list 65 that is long in the vertical direction. The list 65 has a portion that cannot be displayed downward. When the list 65 is scrolled downward using the scroll bar 64 at the right end of the screen, the portion (continuation portion) is displayed on the display unit 31. .

  FIG. 16 shows an example of a screen displayed by both devices in a cooperative display after the operation panel 30 and the portable terminal 40 are juxtaposed in FIG. The operation panel 30 displays the upper part of the list 65 without change. And the portable terminal 40 is displaying the continuation part of the list | wrist 65 which the operation panel 30 is displaying.

  In this way, in the linked display of pattern 2, when the entire screen that is the main screen cannot be displayed at one time on one display unit (31 or 51), the part that cannot be displayed (the continued part) is the other. To improve visibility.

  Next, based on a specific example of (Pattern 2), a determination criterion 6 serving as a predetermined determination criterion will be described.

<Judgment criteria 6, when there is a screen with a part that cannot be displayed, and when judging based on the relative positional relationship during juxtaposition>
In the criterion 6, as shown in FIGS. 15 and 16, when there is a portion (hereinafter referred to as a continuation portion) that cannot be displayed on the screen displayed on the operation panel 30 (or the portable terminal 40), the screen is displayed. Set to main screen.

  In the embodiment of the present invention, when a screen having a continuation portion is used as the main screen, the other portion is arranged in the direction where the continuation portion is present with respect to the display unit (31 or 51) displaying the main screen. Linked display is performed only when there is a device. In FIG. 16, the continuation portion is present on the lower side in the list 65 displayed on the operation panel 30, and the mobile terminal 40 is also juxtaposed on the lower side of the operation panel 30.

  Note that, when the other device is juxtaposed in a direction other than the direction in which the continued portion is present with respect to the display unit (31 or 51) displaying the main screen, the cooperative display is not performed. FIG. 17 shows a state where the mobile terminals 40 are juxtaposed on the right side of the operation panel 30 that displays the list 65.

  In FIG. 17, the continuation part exists in the lower side in the list | wrist 65 which the operation panel 30 is displaying. However, since the portable terminal 40 is juxtaposed on the right side of the operation panel 30, no cooperative display is performed.

  Next, a method of determining which screen is the main screen when the operation panel 30 and the mobile terminal 40 display a screen having a continuation portion will be described.

  In FIG. 18, the screen (list 65) displayed on the operation panel 30 and the screen (list 66) displayed on the mobile terminal 40 both have a portion (continuous portion) that cannot be displayed on the lower side. In such a state, when the operation panel 30 and the portable terminal 40 are juxtaposed to execute cooperative display, any of the screens displayed on the operation panel 30 and the portable terminal 40 is changed depending on the relative positional relationship at the time of juxtaposition. Switch to main screen.

  Specifically, when the mobile terminal 40 is juxtaposed with the operation panel 30 in a direction where there is a continuation of the screen displayed on the operation panel 30, the screen displayed on the operation panel 30 is displayed. When the operation panel 30 is juxtaposed with the main screen in a direction in which there is a continuation of the screen displayed on the mobile terminal 40 with respect to the mobile terminal 40, the screen displayed on the mobile terminal 40 is the main screen. Make the screen.

  FIG. 19 shows an example of a screen displayed in cooperative display when the mobile terminal 40 is juxtaposed below the operation panel 30 from the state of FIG. Here, since the mobile terminal 40 is juxtaposed in a direction (lower side) where the continued portion of the list 65 displayed on the operation panel 30 is located, the screen displayed on the operation panel 30 is continuously displayed as the main screen, The terminal 40 displays a continuation of the list 65 displayed on the operation panel 30.

  FIG. 20 shows an example of a screen displayed in cooperative display when the mobile terminal 40 is juxtaposed on the upper side of the operation panel 30 from the state of FIG. Here, since the mobile terminal 40 is juxtaposed on the upper side of the operation panel 30, the operation panel 30 is positioned in a direction (lower side) where the continued portion of the list 66 displayed by the mobile terminal 40 is present. Therefore, the screen displayed on the mobile terminal 40 is continuously displayed as the main screen, and the continuation portion of the list 66 displayed on the mobile terminal 40 is displayed on the operation panel 30.

  Next, a case where the other device is juxtaposed in a direction where there is a continuation of the screen being displayed for each device will be described. In FIG. 21, there are portions (continuous portions) that cannot be displayed on the lower side of the screen (list 65) displayed on the operation panel 30 and the upper side of the screen (list 66) displayed on the mobile terminal 40. When the portable terminals 40 are juxtaposed on the lower side of the operation panel 30, the other devices are juxtaposed in a direction in which there is a continuation of the screen being displayed.

  In such a case, after the operation panel 30 and the portable terminal 40 are juxtaposed, the user receives a selection as to which screen is displayed as the main screen. For example, as shown in FIG. 12, two selection buttons (MFP button 61 and tablet button 62) for selecting the main screen are displayed on at least one of the operation panel 30 and the portable terminal 40, The selection button is selected.

  FIG. 22 shows two screens for selecting the main screen when the operation panel 30 and the portable terminal 40 are juxtaposed with each other so that the other device is positioned in a direction in which there is a continuation of the screen being displayed. An example in which selection buttons (MFP button 61 and tablet button 62) are displayed on operation panel 30 is shown. When the MFP button 61 is selected by the user, it is determined that the screen of the operation panel 30 is the main screen, and when the tablet button 62 is selected, it is determined that the screen of the portable terminal 40 is the main screen.

  The list 65 described with reference to FIGS. 15 to 22 is a file list and the list 66 is a destination list. However, other types of lists such as a history list may be used. Moreover, as long as it is a screen with a continuation part, screens other than a list | wrist (65, 66) may be sufficient. For example, when the enlarged display is performed, the entire display may not be displayed on the one display unit (31, 51) due to the enlarged display.

(Specific example 2 of pattern 2)
FIG. 23 is a specific example of pattern 2 and shows a state where the operation panel 30 displays a screen (setting screen 67) other than the list (65, 66). In FIG. 23, the operation panel 30 displays the upper part of the setting screen 67, and there is a part (continuation part) that cannot be displayed in the downward direction. FIG. 24 shows a state in which the mobile terminals 40 are juxtaposed below the operation panel 30 in FIG. Since the portable terminal 40 is juxtaposed in the direction in which the continuation part exists, the portable terminal 40 displays the continuation part by cooperative display.

  Next, in the cooperative display system 5, it is determined which one of the screens displayed on the operation panel 30 and the portable terminal 40 is the main screen, and the processing when the cooperative display is started is shown in FIG. Description will be made based on the flowchart of FIG. In FIGS. 25 to 27, the above determination is made by combining the above-mentioned determination criteria 1, 3, and 6. The determination criterion 1 in FIGS. 25 to 27 is based on the last operation received immediately before the operation panel 30 and the portable terminal 40 are juxtaposed. In the drawing, the image forming apparatus 10 is referred to as MFP.

  First, the image forming apparatus 10 waits for the portable terminal 40 to be juxtaposed (FIG. 25, Step S001; No), and when juxtaposed (Step S001; Yes), the portable terminal in any direction with respect to the operation panel 30. 40 is recognized using the sensor 21 (step S002), and the relative positional relationship between the operation panel 30 and the portable terminal 40 is grasped.

  Next, the image forming apparatus 10 acquires information on the screen displayed on the mobile terminal 40 immediately before juxtaposition and information on the operation content just before juxtaposition (step S003). When the main screen is displayed on the operation panel 30 and the portable terminal 40 that was displayed by the device that was last operated immediately before juxtaposition (step S004; Yes), the process proceeds to step S005 in FIG.

  When the screen displayed by the device that has received the last operation immediately before the juxtaposition is not the main screen (step S004; No), the process proceeds to step S013 in FIG.

  FIG. 26 is a continuation process of FIG. 25, and based on the operation received from the user, it is determined which of the operation panel 30 and the mobile terminal 40 is displayed as the main screen. . First, the image forming apparatus 10 checks whether or not a certain time has elapsed since the operation panel 30 and the portable terminal 40 were last operated (step S005).

  If the predetermined time has not elapsed (step S005; No), it is checked whether or not the last device that has been operated is the operation panel 30 (step S010). When the operation is finally received on the operation panel 30 (step S010; Yes), the screen displayed on the operation panel 30 is set as the main screen, and the portable terminal 40 is switched to display the content corresponding to the main screen. When instructed (step S011) and the portable terminal switches display contents in accordance with the instruction, the process is terminated.

  Specifically, the mobile terminal 40 receives screen data of a screen to be displayed after switching, information on a screen corresponding to the main screen, and the like from the image forming apparatus 10, and switches display contents based on these.

  If the device that has been operated last is the mobile terminal 40 (step S010; No), the screen displayed on the mobile terminal 40 is set as the main screen, and the display content of the operation panel 30 is set to the content corresponding to the main screen. Switching (step S012), the process is terminated.

  If a predetermined time or more has elapsed since the operation panel 30 and the portable terminal 40 were last operated (step S005; Yes), any one of the operation panel 30 and the portable terminal 40 displays. An option (selection buttons 61, 62) as to whether the current screen is to be the main screen is displayed on the operation panel 30 (step S006).

  When the option of selecting the screen displayed on the operation panel 30 as the main screen is selected (step S007; Yes), the screen displayed on the operation panel 30 is set as the main screen, and the display content is displayed in the content corresponding to the main screen. The portable terminal 40 is instructed to switch (step S008), and when the portable terminal switches the display content in accordance with the instruction, the process ends.

  When an option for selecting the screen displayed on the mobile terminal 40 as the main screen is selected (step S007; No), the screen displayed on the mobile terminal 40 is set as the main screen, and the display content of the operation panel 30 is set as the main screen. Switching to the content corresponding to the screen (step S009), the process is terminated.

  FIG. 27 is a continuation process of FIG. 25, and the main screen is determined based on the display contents of the operation panel 30 and the portable terminal 40 and the relative positional relationship.

  The screen displayed on the operation panel 30 has a continuation part, and the portable terminal 40 is located in a direction in which the continuation part is present with respect to the operation panel 30 (step S013; Yes). When there is a continuation part on the displayed screen and the operation panel 30 is positioned in a direction in which the continuation part is present with respect to the portable terminal 40 (step S014; Yes), the operation panel 30 and the portable terminal 40 An option (selection buttons 61 and 62) as to which of the displayed screens is to be the main screen is displayed on the operation panel 30 (step S016).

  When an option for selecting the screen displayed on the operation panel 30 as the main screen is selected (step S017; Yes), the screen currently displayed on the operation panel 30 is set as the main screen, and the display content corresponds to the main screen. The portable terminal 40 is instructed to switch to (step S018), and when the portable terminal switches display contents in accordance with the instruction, the process is terminated.

  When the option of selecting the screen displayed on the mobile terminal 40 as the main screen is selected (step S017; No), the screen displayed on the mobile terminal 40 is set as the main screen, and the display content of the operation panel 30 is set as the main screen. Switching to the content corresponding to the screen (step S019), the process is terminated.

  The screen displayed on the operation panel 30 has a continuation part, and the portable terminal 40 is located in a direction in which the continuation part is present with respect to the operation panel 30 (step S013; Yes). Even if there is no continuation part on the displayed screen or there is a continuation part, if the operation panel 30 is not located in the direction in which the continuation part is present with respect to the portable terminal 40 (step S014; No), the operation is performed. The screen displayed on the panel 30 is set as the main screen, the portable terminal 40 is instructed to switch the display content to the content corresponding to the main screen (step S015), and the processing ends when the portable terminal switches the display content in accordance with the instruction. To do.

  Even if there is no continuation part on the screen displayed on the operation panel 30 or there is a continuation part, the portable terminal 40 is not located in the direction in which the continuation part is present with respect to the operation panel 30 (step S013; No) ), And the screen displayed on the mobile terminal 40 has a continuation part, and the operation panel 30 is positioned in a direction in which the continuation part is located with respect to the mobile terminal 40 (step S020; Yes), The screen displayed on the portable terminal 40 is set as the main screen, and the display content on the operation panel 30 is switched to the content corresponding to the main screen (step S021), and the process is terminated.

  Even if there is no continuation part on the screen displayed on the operation panel 30 or there is a continuation part, the portable end 40 is not located in the direction in which the continuation part is present with respect to the operation panel 30 (step S013; No) ) And the screen displayed on the portable terminal 40 has no following part. Alternatively, even if there is a continued portion, if the operation panel 30 is not positioned in the direction in which the continued portion is present with respect to the mobile terminal 40 (step S020; No), the cooperative display is not performed and the processing is terminated.

  The process of FIGS. 25 to 27 is an example of a process for determining which one of the operation panel 30 and the portable terminal 40 displays as the main screen, and determines the main screen based on other determination criteria. May be.

  Note that the cooperative display ends when either the operation panel 30 or the portable terminal 40 receives an instruction to end the cooperative display or when the juxtaposition of the operation panel 30 and the portable terminal 40 is released.

  As described above, in the cooperative display system 5, when the operation panel 30 and the portable terminal 40 execute the cooperative display, the image forming apparatus 10 mainly uses the screen displayed by either the operation panel 30 or the portable terminal 40. The screen is determined according to a predetermined criterion, and a screen based on the main screen is displayed on the operation panel 30 and the portable terminal 40. For example, a screen displayed by the last operation of the operation panel 30 and the portable terminal 40 or a screen having a continuation portion is determined as the main screen, and linked display is performed.

  When the above operation is performed in the last operation immediately before the operation panel 30 and the portable terminal 40 are juxtaposed or in the first operation after the juxtaposition, the operation before and after the juxtaposition is an operation closely related to the cooperative display. By selecting the main screen based on the above, it is possible to automatically perform linked display according to the user's usage.

  When determining that the screen displayed by the device that received a specific operation is the main screen, the specific operation is more closely related to the linked display than the other operations, so just before juxtaposition. The last display operation or the first operation after juxtaposition is not affected by operations other than a specific operation, and linked display according to the user's usage can be automatically performed.

Of the screens displayed on the operation panel 30 and the portable terminal 40, when determining that the screen having a portion that cannot be displayed (continuation portion) is the main screen, regardless of the contents of the operations received by both devices, A portion that cannot be displayed by the device that is displaying the main screen is displayed on the other device. That is, when another operation display device is juxtaposed during display of a screen having a portion that cannot be displayed, the user can display the portion that cannot be displayed on the display unit of the other juxtaposed operation display device. Therefore, it is possible to automatically perform display according to the user's wishes by the above operation.

  When the device cannot properly determine which screen to display as the main screen among the screens displayed on the operation panel 30 and the portable terminal 40, the device mistakenly displays a screen unintended by the user when receiving a selection from the user. The possibility of selecting the main screen is reduced, and linked display according to the user's usage can be performed.

  The embodiment of the present invention has been described with reference to the drawings. However, the specific configuration is not limited to that shown in the embodiment, and there are changes and additions within the scope of the present invention. Are also included in the present invention.

  In the embodiment of the present invention, the image forming apparatus 10 constituting the cooperative display system 5 is described as an example of the first operation display apparatus, and the portable terminal 40 is described as an example of the second operation display apparatus. The operation display device is not limited to this. For example, a single tablet composed of the CPU 11, the ROM 12, the RAM 13, the nonvolatile memory 14, the network communication unit 20 and the operation panel 30 may be used as the first and second operation display devices. In addition, a single display device that functions as the image forming apparatus 10 in the embodiment, a program that causes an information processing apparatus including an operation unit and a display unit to function like the image forming apparatus 10 in the embodiment, The linked display method is also included in the present invention.

  In the cooperative display system 5 according to the embodiment of the present invention, the sensor 21 on the operation panel 30 side serves as a detection unit that detects that the operation panel 30 and the portable terminal 40 are juxtaposed. The sensor 47 on the terminal 40 side may serve as a detection unit that detects that the operation panel 30 and the portable terminal 40 are juxtaposed, or the detection unit may be provided separately.

  In the cooperative display system 5 according to the embodiment of the present invention, the image forming apparatus 10 determines which of the operation panel 30 and the portable terminal 40 is used as the main screen, and is based on the main screen. Although the role played as a cooperation display control part which performs the cooperation display which displays a screen on the operation panel 30 and the portable terminal 40, the portable terminal 40 may play the role as a cooperation display control part, and external servers, such as a server A device may be provided to serve as a cooperative display control unit.

  In the embodiment of the present invention, the image group 80 is configured by arranging a plurality of screens in the horizontal direction, but may be configured by arranging in the vertical direction.

  According to the criterion 6 of the embodiment of the present invention, a linked display is performed only when there is a continuation part on the screen and there is another device in the direction in which the continuation part is present for the apparatus displaying the screen. However, linked display may be executed with a screen having a continuation portion as a main screen regardless of the relative positional relationship with other devices.

3 ... LAN
5 ... Cooperation display system 10 ... Image forming apparatus 11 ... CPU
12 ... ROM
13 ... RAM
DESCRIPTION OF SYMBOLS 14 ... Nonvolatile memory 15 ... Hard disk device 16 ... Image processing part 17 ... Image reading part 18 ... Printer part 19 ... Facsimile communication part 20 ... Network communication part 21, 21 (A) ... Sensor 22 ... ADF
DESCRIPTION OF SYMBOLS 23 ... Authentication part 30 ... Operation panel 31 ... Display part 32 ... Operation part 33 ... Switch part 34 ... Touch panel part 40 ... Portable terminal 41 ... CPU
42 ... ROM
43 ... RAM
44 ... Non-volatile memory 45 ... Hard disk device 46 ... Network communication unit 47, 47 (A), 47 (B) ... Sensor 48 ... Authentication unit 50 ... Operation display unit 51 ... Display unit 52 ... Operation unit 53 ... Switch unit 54 ... Touch panel Numeral 60: Priority button 61 ... MFP button 62 ... Tablet button 64 ... Scroll bar 65 ... List 66 ... List 67 ... Setting screen 80 ... Screen group

Claims (25)

First and second operation display devices including an operation unit and a display unit;
A detection unit for detecting that the display unit of the first operation display device and the display unit of the second operation display device are juxtaposed;
After the detection, a screen based on the main screen is determined based on a predetermined determination criterion as to which one of the first operation display device and the second operation display device is to be used as the main screen. A coordinated display control unit that performs coordinated display to display the first operation display device and the second operation display device
A cooperative display system characterized by comprising: The operation received by the first operation display device and the second operation display device before the detection, or until the linked display is started after the detection, and the second operation display device and the second operation display device. The cooperative display system according to claim 1, wherein an operation received by the operation display device is included in the predetermined criterion. The cooperative display control unit is configured to use a screen that is displayed on the operation display device that has been lastly operated before the detection among the first operation display device and the second operation display device as the main screen. The cooperative display system according to claim 1 or 2, wherein: The cooperative display system according to claim 3, wherein the cooperative display control unit does not perform the cooperative display when a predetermined time has elapsed since the last operation before the detection was received. 5. The display content displayed by the first operation display device and the second operation display device before the detection is included in the predetermined determination criterion. 6. The linkage display system described in 1. The cooperative display control unit is configured to select a screen having a portion that cannot be displayed among the screens displayed by the first operation display device and the second operation display device before the detection as the main screen. The cooperative display system according to any one of claims 1 to 5, wherein: The detection unit recognizes a relative positional relationship between the first operation display unit and the second operation display unit during the detection,
The coordinated display control unit performs the coordinated display only when there is another operation display device in a direction where there is a portion that cannot be displayed with respect to the display unit displaying the main screen. The cooperative display system according to claim 6. The cooperative display control unit selects one of the screens displayed on the first operation display device and the second operation display device when the determination cannot be made based on the predetermined determination criterion. The cooperative display system according to any one of claims 1 to 7, wherein the selected screen is used as the main screen. An operation unit;
A display unit;
A communication unit that transmits and receives data to and from an external display device including an operation unit and a display unit;
A detection unit for detecting that the display unit of the device itself and the display unit of the external display device are juxtaposed;
After the detection, it is determined according to a predetermined criterion whether the screen displayed on the own device or the external display device is the main screen, and the screen based on the main screen is the display unit of the own device. A control unit that performs cooperative display to be displayed on the external display device;
A display device comprising: The operation received by the own device and the external display device before the detection, or the operation received by the own device and the external display device after the detection and before the linked display is started, The display device according to claim 9, wherein the display device is included in a predetermined criterion. The cooperative display control unit makes the main screen the screen displayed on the display device that has been lastly operated before the detection among the own device and the external display device. Item 11. The display device according to Item 9 or 10. The display device according to claim 11, wherein the cooperative display control unit does not perform the cooperative display when a predetermined time has elapsed since the last operation before the detection was received. The display device according to any one of claims 9 to 12, wherein display contents displayed by the device itself and the external display device before the detection are included in the predetermined criterion. The cooperative display control unit makes the main screen a screen having a portion that cannot be displayed among the screens displayed by the own device and the external display device before the detection. The display device according to claim 9. The detection unit recognizes a relative positional relationship between the display unit of the own device and the display unit of the external display device at the time of the detection,
In the case where the screen displayed by the display unit of the device itself is the main screen, the cooperative display control unit is configured so that the external display is in a direction in which there is a portion that cannot be displayed with respect to the display unit of the device itself. Only when there is a display unit of a display device, the cooperative display is performed, and when the screen displayed by the external display device is the main screen, the display unit of the external display device is The display device according to claim 14, wherein the cooperative display is performed only when the display unit of the own device is in a direction in which there is a portion that cannot be displayed. The cooperative display control unit receives a selection from the user from one of the screens displayed by the own device and the external display device when the determination cannot be made based on the predetermined determination criterion, and the selected display The display device according to claim 9, wherein the main screen is the main screen. An information processing apparatus including an operation unit and a display unit
A program that functions as the display device according to any one of claims 9 to 16. A detection step of detecting that the display units of the first and second operation display devices including the operation unit and the display unit are juxtaposed,
After the detection, a screen based on the main screen is determined based on a predetermined determination criterion as to which one of the first operation display device and the second operation display device is to be used as the main screen. A coordinated display step for performing coordinated display for displaying the first operation display device and the second operation display device;
A cooperative display method characterized by comprising: The operation received by the first operation display device and the second operation display device before the detection, or the first operation display device after the detection and before the linked display is started. The cooperative display method according to claim 18, wherein an operation received by the second operation display device is included in the predetermined determination criterion. In the linked display step, the screen displayed on the operation display device that has been last operated before the detection among the first operation display device and the second operation display device is set as the main screen. The cooperative display method according to claim 18 or 19, wherein: The linked display method according to claim 20, wherein in the linked display step, the linked display is not performed when a predetermined time has passed since the last operation before the detection was received. The display content displayed by the first operation display device and the second operation display device before the detection is included in the predetermined determination criterion. Linked display method described in 1. In the cooperative display step, a screen having a portion that cannot be displayed among the screens displayed by the first operation display device and the second operation display device before the detection is set as the main screen. The cooperative display method according to any one of claims 18 to 22, wherein: In the detection step, at the time of the detection, a relative positional relationship between the first operation display unit and the second operation display unit is recognized,
In the linked display step, the linked display is performed only when there is another operation display device in a direction where there is a portion that cannot be displayed with respect to the display unit displaying the main screen. The cooperative display method according to claim 23, wherein In the cooperative display step, when the determination cannot be made based on the predetermined determination criterion, the user selects one of the screens displayed on the first operation display device and the second operation display device from the user. 25. The cooperative display method according to claim 18, wherein the selected screen is the main screen.