JP2014107800A - Image forming system, image forming apparatus, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image forming apparatus and the related technique in which remote control on a user interface section other than a touch panel can be performed, while suppressing degradation of a user's operability.SOLUTION: The image forming apparatus has: a touch panel which displays various information and receives operation input from an operator; and a user interface section installed separately from the touch panel. The image forming apparatus generates data related to a plurality of layers including a base layer LB which includes an image indicating an operation screen to be displayed on the touch panel and at least one superposition layer LE which includes an image indicating the user interface section and is superimposed on a base layer. Furthermore, the image forming apparatus transmits the data related to the plurality of layers to an external terminal as display data for a remote control screen in the external terminal.

Description

  The present invention relates to an image forming system and related technology.

  There is a system (also referred to as an image forming system or a remote operation system) for remotely operating an image forming apparatus using an external terminal. In such a system, a remote operation screen is displayed on the display unit of the external terminal, and the operation target device is remotely operated using the remote operation screen.

  In many image forming apparatuses, a touch panel is provided in an operation panel, and an operation input by a user is performed on the touch panel. In the image forming apparatus, a hardware key may be further provided in addition to the touch panel.

  Patent Document 1 describes a technique for performing a remote operation related to a hardware key provided in a remotely operated device. Specifically, in the display window of the remote control device (personal computer), an image showing a display screen of the remote control device and a hardware key image corresponding to the hardware key of the remote control device are horizontally displayed. Techniques that are arranged and displayed are described.

JP 2008-71313 A

  However, as described above, in the technique described in Patent Document 1, the display screen image and the hardware key image are arranged in the horizontal direction in one display. For this reason, a problem may arise due to the fact that the display screen image and the hardware key image cannot each have a sufficient size. Specifically, the user's operability may be lowered, such as it becomes difficult for the user to visually recognize the contents of the display screen or to press a small hardware key image.

  In addition, the above-described problem can occur in the same manner when a user interface unit other than the hardware key is displayed on the external terminal.

  SUMMARY An advantage of some aspects of the invention is that it provides an image forming apparatus capable of performing a remote operation related to a user interface unit other than a touch panel while suppressing a decrease in user operability, and a technique related thereto.

  In order to solve the above-mentioned problem, the invention of claim 1 is an image forming apparatus that can be remotely operated by an external terminal, and includes a touch panel that displays various information and receives an operation input from an operator, and the touch panel. Plural including a user interface unit provided separately, a base layer including an image indicating an operation screen displayed on the touch panel, and at least one overlapping layer including an image indicating the user interface unit and superimposed on the base layer And generating means for generating data relating to a plurality of layers, and transmitting means for transmitting data relating to the plurality of layers to the external terminal as display data for a remote operation screen in the external terminal.

  According to a second aspect of the present invention, in the image forming apparatus according to the first aspect of the present invention, operation input information which is operation input information for the remote operation screen displayed on the external terminal based on data relating to the plurality of layers. Receiving means from the external terminal, and determining means for determining the content of the operation input based on the operation input information, wherein the determining means is based on the content of the operation input. One of the layers and the at least one superimposed layer to be updated is determined, and the generation unit selectively selects image data related to the one of the base layer and the at least one superimposed layer. The transmission means updates the image data relating to the one layer after the update to a new remote operation screen after the update of the remote operation screen. As display data, and transmits to the external terminal.

  According to a third aspect of the present invention, in the image forming apparatus according to the first aspect of the invention, the user interface unit is provided separately from the touch panel, and a hardware key that receives an operation input from the operator and the touch panel are It has at least one of a status display unit that is provided separately and displays the status of the image forming apparatus.

  According to a fourth aspect of the present invention, in the image forming apparatus according to the third aspect, the user interface unit includes both the hardware key and the status display unit, and the generation unit includes the at least one Image data relating to both the hardware key image corresponding to the hardware key and the state display image indicating the display content in the state display unit is generated as image data of the superposition layer, and the transmission means includes the at least one Data relating to the plurality of layers including one superimposed layer and the base layer is transmitted to the external terminal as the display data of the remote operation screen.

  According to a fifth aspect of the present invention, in the image forming apparatus according to the third aspect of the invention, the user interface unit includes the state display unit, and the generation unit includes the image data of the at least one superimposed layer as the image data. Generating image data relating to a state display image indicating display contents in the state display unit, wherein the transmission means transmits data relating to the plurality of layers including the at least one superimposed layer and the base layer to the data on the remote operation screen; The display data is transmitted to the external terminal.

  According to a sixth aspect of the present invention, in the image forming apparatus according to any one of the first to fifth aspects, the transmission means includes image data relating to the base layer as display data for a remote operation screen on the external terminal. And image data relating to the at least one superimposed layer are individually transmitted to the external terminal.

  According to a seventh aspect of the present invention, in the image forming apparatus according to the third aspect of the present invention, the user interface unit includes both the hardware key and the status display unit. The generation means is the state display image of both the hardware key image corresponding to the hardware key and the state display image indicating the display content in the state display unit as the image data of the at least one superimposed layer. The transmission means transmits data regarding the plurality of layers including the at least one superimposed layer and the base layer to the external terminal as the display data for the remote operation screen. It is characterized by that.

  According to an eighth aspect of the present invention, in the image forming apparatus according to the seventh aspect of the present invention, operation input information which is operation input information for the remote operation screen displayed on the external terminal based on data relating to the plurality of layers. Receiving means from the external terminal, and determining means for determining the content of the operation input based on the operation input information, wherein the status display image is a display and non-display of the hardware key image. In response to the determination that the operation on the state display image has been performed based on the operation input information, the generation unit is configured to function as a switching instruction reception area that receives a switching instruction to switch display. Image data relating to both images is generated as image data of the at least one superimposed layer, and the transmission means is configured to transmit the at least one superimposed layer. The image data relating to Ya, and transmits to the external terminal as display data of the second remote operation screen is a screen after updating the remote operation screen.

  According to a ninth aspect of the present invention, in the image forming apparatus according to the eighth aspect of the invention, the transmission unit is responsive to determining that an operation on the status display image has been performed based on the operation input information. And transmitting image data related to only the at least one superimposed layer out of the base layer and the at least one superimposed layer to the external terminal as the display data of the second remote operation screen. To do.

  According to a tenth aspect of the present invention, in the image forming apparatus according to the ninth aspect of the invention, an operation is performed on the status display image displayed together with the hardware key image on the second remote operation screen. In response to the determination, the generation unit generates image data related to only the state display image among the two images as the image data of the at least one superimposed layer, and the transmission unit includes the base layer Image data relating to only the at least one superimposed layer among the at least one superimposed layer and the external terminal as display data for a third remote operation screen, which is a screen after updating the second remote operation screen It is characterized by transmitting to.

  According to an eleventh aspect of the present invention, in a computer built in an image forming apparatus that can be remotely operated by an external terminal, a) a base layer including an image showing an operation screen displayed on the touch panel of the image forming apparatus; and the touch panel A step of generating data relating to a plurality of layers including an image showing a user interface unit provided in the image forming apparatus and including at least one superimposed layer superimposed on the base layer; b) the plurality And transmitting to the external terminal data related to the layer as display data for a remote operation screen on the external terminal.

  The invention of claim 12 is an image forming system, comprising: an image forming apparatus; and an external terminal for remotely operating the image forming apparatus using a remote operation screen, wherein the image forming apparatus displays various types of information. And a touch panel that receives an operation input from an operator, a user interface unit provided separately from the touch panel, a base layer including an image showing an operation screen displayed on the touch panel, and an image showing the user interface unit And generating means for generating data relating to a plurality of layers including at least one superposed layer superimposed on the base layer, and the external terminal using the data relating to the plurality of layers as display data for a remote operation screen in the external terminal First transmission means for transmitting to the external terminal, the external terminal First receiving means for receiving data relating to a layer, based on the data for the previous plurality of layers, and having a display means for displaying the remote operation screen.

  According to a thirteenth aspect of the present invention, in the image forming system according to the twelfth aspect of the present invention, the image forming apparatus is configured to input an operation input to the remote operation screen displayed on the external terminal based on the data regarding the plurality of layers. Receiving means for receiving operation input information as information from the external terminal, and determining means for determining the content of the operation input based on the operation input information, wherein the determining means includes the operation input Based on the content of the base layer and the at least one superimposed layer to determine one layer to be updated, and the generating means includes the one of the base layer and the at least one superimposed layer. The image data related to the layer is selectively updated, and the first transmission means updates the image data related to the one layer after the update to the remote control. As the display data for the new remote operation screen after the screen update, the data is transmitted to the external terminal, and the display means of the external terminal transmits the image data of the one layer related to the new remote operation screen and the remote operation. The new remote operation screen is displayed based on the image data of the other layer relating to the screen.

  According to a fourteenth aspect of the present invention, in the image forming system according to the twelfth aspect, the external terminal responds to an operation input to the remote operation screen displayed on the external terminal based on the data regarding the plurality of layers. Specifying an operation position on the remote operation screen, specifying means for specifying an operation target layer from the plurality of layers constituting the remote operation screen, an operation target layer specified by the specifying means, and the operation And a second transmission unit configured to transmit operation input information including information regarding the position to the image forming apparatus.

  According to a fifteenth aspect of the present invention, in the image forming system according to the twelfth aspect of the invention, the user interface unit is provided separately from the touch panel, and a hardware key for receiving an operation input from the operator and the touch panel are And a status display unit that displays a status of the image forming apparatus. The data relating to the plurality of layers includes image data relating to the base layer and image data relating to the at least one superimposed layer. The image data relating to the at least one superimposed layer includes image data relating to both a hardware key image corresponding to the hardware key and a state display image indicating display contents in the state display unit, and the external data The terminal includes the image data related to the base layer and the at least one Storage means for storing image data related to the superimposition layer, and a portion corresponding to the display target area is extracted from the image data related to the at least one superposition layer stored in the storage means, and an image of the extracted part The image processing apparatus further includes combining means for generating the remote operation screen by performing combining processing using data and image data related to the base layer.

  According to a sixteenth aspect of the present invention, in the image forming system according to the fifteenth aspect, the status display image also functions as a switching instruction receiving area for receiving a switching instruction for switching between display and non-display of the hardware key image. When the display instruction of the hardware key image is given, the synthesizing unit selects the hardware key image, the state display image, and the image data from the image data related to the at least one superimposed layer stored in the storage area. Is extracted as a part corresponding to the display target region, and the hardware key image and the state display are performed by performing a synthesis process using the image data of the extracted part and the image data related to the base layer. An image including an image and the operation screen is generated as the remote operation screen, and the hardware key is generated. When an image non-display instruction is given, a portion of the image data relating to the at least one superimposed layer stored in the storage area that does not include the hardware key image and includes the state display image is displayed. Extracting as a portion corresponding to the target area, and performing a combining process using the image data of the extracted portion and the image data related to the base layer, thereby not including the hardware key image and the state display image and the An image including an operation screen is generated as the remote operation screen.

  According to the first to sixteenth aspects of the present invention, it is possible to perform a remote operation related to a user interface unit other than the touch panel while suppressing a decrease in user operability.

  In particular, according to the second and thirteenth aspects of the present invention, it is possible to transmit only image data relating to an appropriate layer among a plurality of layers, so that the total amount of data transmission can be reduced. It is.

1 is a diagram illustrating an image forming system according to a first embodiment. It is a figure which shows the functional block of an image forming apparatus. It is a functional block diagram which shows schematic structure of an external terminal. It is a figure which shows the external appearance structure of an operation panel part. It is a figure which shows the remote operation screen displayed on the touchscreen of an external terminal. It is a figure which shows the some layer which comprises the remote control screen of FIG. It is a figure which shows the base layer containing a panel image. It is a figure which shows the superimposition layer containing an LED image and a hardware key area | region image. It is a figure which shows the remote operation screen displayed on the touchscreen of an external terminal. It is a figure which shows the some layer which comprises the remote control screen of FIG. It is a figure which shows the superimposition layer containing only an LED image. It is a figure which shows a remote operation screen (detailed setting screen). It is a figure which shows a remote operation screen (detailed setting screen). It is a figure which shows the screen transition between the some remote control screens. 3 is a timing chart showing an operation in the image forming system. 3 is a timing chart showing an operation in the image forming system. 3 is a timing chart showing an operation in the image forming system. 3 is a timing chart showing an operation in the image forming system. It is a figure which shows a remote operation screen ("copying" display screen). It is a functional block diagram which shows schematic structure of the external terminal which concerns on 2nd Embodiment. It is a figure which shows the data structure of the remote control screen which concerns on 2nd Embodiment. It is a figure which shows the part utilized when displaying a certain remote operation screen among the data of a superimposition layer. It is a figure which shows the part utilized when displaying another remote operation screen among the data of a superimposition layer. It is a timing chart which shows the operation concerning a 2nd embodiment. It is a functional block diagram which shows schematic structure of the external terminal which concerns on a modification.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<1. First Embodiment>
<1-1. Outline of configuration>
FIG. 1 is a diagram illustrating an image forming system 1 (also referred to as 1A) according to the first embodiment. As shown in FIG. 1, the image forming system 1A includes an image forming apparatus 10 (also referred to as 10A), an external terminal 50 (also referred to as 50A), and a computer 90.

  The image forming apparatus 10, the external terminal 50, and the computer 90 are connected to each other via a network NW. The network NW is configured by a LAN (Local Area Network), the Internet, and the like. Further, the connection mode to the network NW may be wired connection or wireless connection. For example, the image forming apparatus 10 and the computer 90 are each wiredly connected to the network NW, and the external terminal 50 is wirelessly connected to the network NW.

  In the image forming system 1, various operations on the image forming apparatus 10 can be performed using the external terminal 50. That is, the external terminal 50 can remotely operate the image forming apparatus 10. The image forming system 1 is also expressed as a remote operation system that performs remote operation of the image forming apparatus 10.

<1-2. Configuration of Image Forming Apparatus 10>
FIG. 2 is a diagram illustrating functional blocks of the image forming apparatus 10. Here, an MFP (Multi-Functional Peripheral) is exemplified as the image forming apparatus 10. In FIG. 2, functional blocks of the MFP 10 are shown.

  The MFP 10 is a device (also referred to as a multi-function device) having a scan function, a copy function, a facsimile function, a box storage function, and the like. Specifically, as shown in the functional block diagram of FIG. 2, the MFP 10 includes an image reading unit 2, a print output unit 3, a communication unit 4, a storage unit 5, an operation unit 6, a controller 9, and the like. Various functions are realized by operating each part of the above in a complex manner.

  The image reading unit 2 optically reads (that is, scans) a document placed at a predetermined position of the MFP 10 and generates image data of the document (also referred to as a scanned image). It is. The image reading unit 2 is also referred to as a scanning unit.

  The print output unit 3 is an output unit that prints out an image on various media such as paper based on data related to a print target.

  The communication unit 4 is a processing unit capable of performing facsimile communication via a public line or the like. Further, the communication unit 4 can perform network communication via the network NW. In this network communication, for example, various protocols such as TCP / IP (Transmission Control Protocol / Internet Protocol) are used. By using the network communication, the MFP 10 can exchange various data with a desired partner (for example, the external terminal 50).

  The communication unit 4 includes a transmission unit 4a that transmits various data and a reception unit 4b that receives various data. The communication unit 4 includes a transmission unit 4a that transmits various data and a reception unit 4b that receives various data. For example, the transmission unit 4 a transmits image data regarding a plurality of layers including the base layer LB and the superimposed layer LE to the external terminal 50 as display data for the remote operation screen RC in the external terminal 50. In addition, the receiving unit 4 b receives operation input information (operation input information) for the remote operation screen RC displayed on the external terminal 50 based on the display data from the external terminal 50.

  The storage unit 5 includes a storage device such as a hard disk drive (HDD). The storage unit 5 stores data related to the print job.

  The user interface unit (also referred to as an operation unit) 6 includes an operation input unit 6a that receives input to the MFP 10 and a display unit 6b that displays and outputs various types of information. As described below, the user interface unit 6 includes a touch panel 25, hardware keys 35, a status display unit 30, and the like.

  The MFP 10 is provided with a substantially plate-like operation panel unit 6c (see FIG. 1). Further, the operation panel unit 6c has a touch panel 25 (see FIG. 1) on the front side thereof. The touch panel 25 is configured by embedding a piezoelectric sensor or the like in a liquid crystal display panel, and can display various kinds of information and accept an operation input from an operator. For example, on the touch panel 25, menu images (including button images and the like) are displayed. The operator can set various operation contents of the image forming apparatus 10 by pressing a button (button represented by a button image) virtually arranged in the touch panel 25. The touch panel 25 functions as a part of the operation input unit 6a and also functions as a part of the display unit 6b.

  Further, a plurality of hardware keys 35 are provided on the operation panel unit 6c. Specifically, as shown in FIG. 4, a power key (sub power key) 35a, a menu key 35b, an ID key 35c, a guide key 35d, an enlargement key 35e, a voice guide key 35f, a software numeric keypad call key 35g, a start A plurality of hardware keys 35 such as a key 35h, a stop key 35i, and a reset key 35j are provided. These hardware keys 35 are also expressed as an operation input unit (user interface unit) that is provided separately from the touch panel 25 and receives an operation input from an operator.

  The MFP 10 is also provided with a display unit (hereinafter also referred to as a status display unit) 30 that displays an apparatus status (the status of the MFP 10) detected by a status detection unit 13 (described later). The status display unit 30 is also expressed as a display unit (user interface unit) provided separately from the touch panel 25. A part of the state display unit 30 is provided on the surface of the operation panel unit 6c, and the other part of the state display unit 30 is provided outside the operation panel unit 6c. The state display unit 30 includes three LED display units (display units using LEDs (light emitting diodes)) 31, 32, and 33 (see FIG. 1).

  The LED display unit 31 includes a plurality of (here, four) LEDs embedded along the left-right direction on the front surface of the main body of the MFP 10 (and outside the operation panel 6 c provided on the front surface of the main body of the MFP 10). It is configured. The LED display unit 31 is disposed below the touch panel 25. The LED display unit 31 is a display unit that displays whether the MFP 10 is in a data reception state, and is also referred to as a data reception display unit (and a data reception LED). The LED display unit 31 is also expressed as a notification unit that notifies that the MFP 10 has transitioned to the data reception state. When the MFP 10 is in a data receiving state, the LED display unit 31 is lit in blue. On the other hand, when the MFP 10 is not in the data receiving state, the LED display unit 31 is turned off. The “data reception state” includes a state in which unprocessed data is received (from the computer 90 or the like) (a state in which unprocessed data is being received), and a state in which unprocessed data is received (a state in which unprocessed data reception is completed ( Received state))) etc. are included.

  The LED display unit 32 has an elongated rectangular shape extending in the left-right direction, and is provided near the upper side of the LED display unit 31. Specifically, the LED display unit 32 includes a plurality of LEDs embedded in the front surface of the main body of the MFP 10 (and outside the operation panel unit 6c) and a diffusion plate provided on the front side of the plurality of LEDs. It is configured. The LED display unit 32 is a display unit that displays whether or not the MFP 10 is operating (in this case, during a printing operation), and is also referred to as an operating display unit (and an operating LED). The LED display unit 32 is also expressed as a notification unit that notifies that the MFP 10 has transitioned to an operating state (such as a printing operation state or a scanning operation). When the MFP 10 is operating, the LED display unit 32 is lit white. On the other hand, when the MFP 10 is not operating, the LED display unit 32 is turned off.

  The LED display unit 33 is provided in the upper left portion in the operation panel unit 6 c of the MFP 10. The LED display unit 33 includes an LED embedded in the upper left part of the operation panel unit 6c and a diffusion plate provided on the front side of the LED. The LED display unit 33 is a display unit that displays whether the MFP 10 is in an attention state (attention required state), and is also referred to as an attention display unit (and an attention LED). The LED display unit 33 is also expressed as a notification unit that notifies that the MFP 10 has transitioned to an attention state (attention required state). When the MFP 10 is in the attention state, the LED display unit 33 is lit in orange. On the other hand, when the MFP 10 is not in the attention state, the LED display unit 33 is turned off. Note that the “attention state” is a state in which there is no problem at that time (an error has not yet occurred) but an error is likely to occur if left as it is, and a state requiring attention (a state requiring attention). is there.

  As described above, the status display unit 30 includes the notification units 31, 32, and 33 that notify that the MFP 10 has transitioned to each of the three states of the data reception state, the printing operation state, and the attention state.

  The controller 9 is a control device that is built in the MFP 10 and controls the MFP 10 in an integrated manner. The controller 9 is configured as a computer system including a CPU and various semiconductor memories (RAM and ROM). The controller 9 implements various processing units by executing a predetermined software program (hereinafter also simply referred to as a program) PG1 stored in a ROM (for example, EEPROM) in the CPU. The program (specifically, a program module group) PG1 may be installed in the MFP 10 via a portable recording medium such as a USB memory, or a network NW.

  Specifically, as shown in FIG. 2, the controller 9 realizes various processing units including a communication control unit 11, an operation control unit 12, a state detection unit 13, and an image generation unit 15 by executing a program PG1. To do.

  The communication control unit 11 is a processing unit that controls communication operations with other devices (such as the external terminal 50). The communication control unit 11 displays data relating to a panel image GA (described later) indicating the display content on the touch panel 25, an LED image GB (described later) indicating the display content on the status display unit 30, and a hardware key area image GC (described later). In cooperation with the communication unit 4 or the like, the data is transmitted to the external terminal 50 as display data for the remote operation screen RC (FIG. 5 or the like) on the external terminal 50. Further, the communication control unit 11 receives operation input information (operation input information) for the remote operation screen RC displayed on the external terminal 50 from the external terminal 50 in cooperation with the communication unit 4 and the like.

  The operation control unit 12 controls various operations such as a print output operation and a scan operation in the image forming apparatus 10.

  The state detection unit 13 is a processing unit that detects the apparatus state of the MFP 10. The state detection unit 13 detects the apparatus state of the MFP 10 based on output values of various sensors.

  The image generation unit 15 is a processing unit that generates image data related to the panel image GA, the LED image GB, and the hardware key area image GC. Specifically, the image generation unit 15 generates data regarding a plurality of layers including the base layer LB and the superimposed layer LE (see FIG. 6 and the like). Here, the base layer LB is a layer including an image showing an operation screen displayed on the touch panel 25. The superimposition layer LE is a layer that is superimposed on the base layer LB, and includes an image indicating a user interface unit (the LED display unit 30 and / or the hardware key 35) other than the touch panel 25.

  The determination unit 16 is a processing unit that determines the content of the operation input by the operator. Based on the operation input information (operation input information for the remote operation screen RC displayed on the external terminal 50 based on image data related to a plurality of layers) received by the reception unit 4b, the determination unit 16 performs an operation by the operator. Determine the contents of the input.

<1-3. Configuration of External Terminal 50>
Next, the configuration of the external terminal (also referred to as a remote control device) 50 will be described.

  The external terminal 50 is a portable information input / output terminal device capable of network communication with other devices. Here, a tablet type terminal is illustrated as the external terminal 50. However, the present invention is not limited to this, and the external terminal 50 may be a smartphone or a personal computer. The external terminal 50 may be a portable device or a stationary device.

  FIG. 3 is a functional block diagram illustrating a schematic configuration of the external terminal 50.

  As shown in the functional block diagram of FIG. 3, the external terminal 50 includes a communication unit 54, a storage unit 55, an operation unit 56, a controller 59, and the like. Realize the function.

  The communication unit 54 can perform network communication via the network NW. In this network communication, for example, various protocols such as TCP / IP (Transmission Control Protocol / Internet Protocol) are used. By using the network communication, the external terminal 50 can exchange various data with a desired partner (the image forming apparatus 10 or the like). The communication unit 54 includes a transmission unit 54a that transmits various data and a reception unit 54b that receives various data. For example, the receiving unit 54b receives display data of a remote operation screen (remote operation screen) RC from the MFP 10, and the transmission unit 54a inputs an operation to the remote operation screen RC displayed on the touch panel 75 of the external terminal 50. Information (operation input information) and the like are transmitted to the MFP 10.

  The storage unit 55 includes a storage device such as a nonvolatile semiconductor memory. The storage unit 55 temporarily stores various image data transmitted from the MFP 10 (such as display data for the remote operation screen RC).

  The operation unit 56 includes an operation input unit 56a that receives an input to the external terminal 50, and a display unit 56b that displays and outputs various types of information. In the external terminal 50, a touch panel 75 (see FIG. 1) configured by embedding a piezoelectric sensor or the like in a liquid crystal display panel is provided. Specifically, on the front side of the substantially plate-like external terminal 50, the touch panel 75 is provided over almost the entire surface except for the peripheral edge portion (frame portion). The touch panel 75 functions as a part of the operation input unit 56a and also functions as a part of the display unit 56b. On the touch panel 75, the above-described remote operation screen RC is displayed, and an operation input to the remote operation screen RC is accepted.

  The controller 59 is a control device that is built in the external terminal 50 and controls the external terminal 50 in an integrated manner. The controller 59 is configured as a computer system including a CPU and various semiconductor memories (RAM and ROM). The controller 59 implements various processing units by executing a predetermined software program (hereinafter also simply referred to as a program) PG2 stored in a storage unit (semiconductor memory or the like) in the CPU. The program PG2 may be installed in the external terminal 50 via a portable recording medium such as a USB memory or a network NW.

  The program PG2 is an application software program for remotely operating the image forming apparatus 10, and implements various functions related to remote operation.

  Specifically, the controller 59 realizes various processing units including the communication control unit 61 and the input / output control unit 65 by executing the program PG2.

  The communication control unit 61 is a processing unit that controls a communication operation with the image forming apparatus 10 or the like in cooperation with the communication unit 54 or the like.

  The input / output control unit 65 is a processing unit that controls an operation input operation to the operation input unit 56a (touch panel 75 or the like) and controls a display operation in the display unit 56b (touch panel 75 or the like). The input / output control unit 65 causes the touch panel 75 to display a remote operation screen RC for remotely operating the image forming apparatus 10.

<1-4. Remote operation screen RC>
On the touch panel 75 of the external terminal 50, a remote operation screen (screen for remotely operating the MFP 10) RC is displayed based on the image data generated by the image generation unit 15 of the MFP 10.

  By the way, the external terminal 50 is not provided with the state display unit 30 described above. Therefore, even if only the same content as the touch panel 25 of the MFP 10 is displayed on the touch panel 75 of the external terminal 50, information to be transmitted to the operator by the status display unit 30 is not transmitted to the operator of the external terminal 50. Not transmitted.

  Similarly, the external terminal 50 is not provided with the hardware key 35 described above. For this reason, even if only the same content as the touch panel 25 of the MFP 10 is displayed on the touch panel 75 of the external terminal 50, the operator of the external terminal 50 cannot perform an operation input to the hardware key 35.

  Therefore, in this embodiment, an example in which not only the content similar to the touch panel 25 of the MFP 10 but also information related to the status display unit 30 is displayed on the touch panel 75 is illustrated. According to this, information to be transmitted to the operator by the status display unit 30 can be transmitted to the operator of the external terminal 50.

  In this embodiment, not only the contents similar to those of the touch panel 25 of the MFP 10 but also hardware key images corresponding to the hardware keys 35 are displayed on the touch panel 75 as necessary. According to this, it is possible to realize an operation input (remote operation input) to the hardware key 35 by an operation input using the hardware key image.

  Specifically, two types of remote operation screens RC1 (see FIG. 9) and RC2 (see FIG. 5) are used as the remote operation screen RC corresponding to the operation screen shown in FIG.

  First, the remote operation screen RC2 will be described.

  FIG. 5 is a diagram showing a remote operation screen RC2 displayed on the touch panel 75. As shown in FIG.

  As shown in FIG. 5, on the touch panel (touch screen) 75, an image GA (hereinafter also referred to as a panel image) showing a display content on the touch panel (touch screen) 25 of the MFP 10 is displayed. In addition, the touch panel 75 also displays an image (hereinafter also referred to as an LED image or a status display image) GB that simulates display contents on the status display unit 30 (LED display units 31, 32, and 33) of the MFP 10. The Further, on the touch panel 75, a hardware key area image GC is also displayed. The hardware key area image GC is an image of an area including a plurality of hardware key images each imitating a plurality of hardware keys 35 provided in the MFP 10.

  Thus, on touch panel 75, a composite image (remote operation screen RC2) in which panel image GA, LED image GB, and hardware key area image GC are combined is displayed. Specifically, a composite image obtained by superimposing both the LED image GB and the hardware key area image GC on the panel image GA is displayed.

  The panel image GA has the same content as that displayed on the touch panel 25. In the lower region in the panel image GA, a plurality of (here, seven) software keys (specifically, software key images) SK1 to SK7 are provided. The plurality of software keys SK1 to SK7 are arranged in the horizontal direction (left-right direction) in a relatively lower area in the screen.

  The hardware key area image GC is an image of an area for displaying a plurality of hardware key images KG1 to KG10. The plurality of hardware key images KG1 to KG10 are images corresponding to the plurality of hardware keys 35 of the MFP 10. Specifically, the hardware key images KG1 to KG6 are respectively a power key (sub power key) 35a, a menu key 35b, an ID key 35c, a guide key 35d, an enlargement key 35e, a voice guide, which are hardware keys of the MFP 10. It is an image corresponding to the key 35f. The hardware key image KG7 is an image corresponding to the software numeric keypad call key 35g which is a hardware key of the MFP 10. Further, the hardware key images KG8 to KG10 are images corresponding to the start key 35h, the stop key 35i, and the reset key 35j, which are hardware keys of the MFP 10, respectively. Each hardware key image KG1 to KG10 is an image showing a corresponding hardware key 35 in a simulated manner.

  The hardware key area image GC is provided so as to be superimposed on the panel image GA in the vicinity of the lower side edge portion in the panel image GA. More specifically, the hardware key area image GC is arranged further below the software key area RB in which the plurality of software keys SK1 to SK7 are arranged. The width (length in the left-right direction) of the hardware key area image GC is substantially the same as the width of the panel image GA.

  The LED image GB is provided adjacent to the hardware key area image GC on the upper side of the hardware key area image GC in the vicinity of the center in the left-right direction of the panel image GA. The LED image GB has an elongated rectangular shape extending in the left-right direction, and is arranged so as to be superimposed on the lower area of the software keys SK4, SK5 in the panel image GA. The width (length in the left-right direction) of the LED image GB is smaller than the width of the panel image GA.

  This LED image GB has three display areas L1, L2, and L3. These display areas L1, L2, and L3 are provided corresponding to the three LED display sections 31, 32, and 33 of the MFP 10, respectively.

  The display area L1 is an area provided corresponding to the LED display section (data reception display section) 31 and is an area indicating the display state of the data reception display section 31 of the MFP 10. The display area L1 is also expressed as a “data reception display area”.

  The display area L2 is an area provided corresponding to the LED display section (in-operation display section) 32, and is an area indicating the display state of the in-operation display section 32 of the MFP 10. The display area L2 is also expressed as a “display area during operation”.

  The display area L3 is an area provided corresponding to the LED display section (attention display section) 33, and is an area indicating the display state of the attention display section 33 of the MFP 10. The display area L3 is also expressed as an “attention display area”.

  Such a remote operation screen RC2 is expressed using, for example, a data file FL (also referred to as FL2) having a layer structure. As shown in FIG. 6, the data file FL includes a base layer LB and a superposition layer LE. FIG. 6 is a diagram showing a layer structure of the remote operation screen RC2.

  The base layer LB is configured to include image data representing a panel image GA (see FIG. 7) as shown in FIG.

  On the other hand, as shown in FIG. 8, the superposition layer LE2 is configured to include image data representing the LED image GB and the hardware key area image GC, respectively. Specifically, the overlapping layer LE2 is roughly divided into three strip regions RG21, RG22, and RG23. The band-shaped region RG23 is a region where the hardware key region image GC is arranged, and has the same size as the hardware key region image GC. The strip-shaped region RG22 is a region where the LED image GB is disposed, and has the same height as the LED image GB. Transparent graphic parts are arranged on both sides of the LED image GB in the band region RG22. The band-shaped region RG21 is the remaining region (a region other than the band-shaped regions RG22 and RG23), and transparent graphic parts are arranged in the band-shaped region RG21. Here, the transparent graphic parts can be compressed very efficiently by reversible compression (run-length compression or the like), and the amount of data at the time of transmission / reception can be greatly suppressed.

  In this embodiment, by arranging transparent graphic parts, the sizes of both layers LB and LE are adjusted and the corresponding positions of both layers LB and LE are adjusted. However, the present invention is not limited to this. For example, transparent graphic parts may not be arranged in the data related to the superimposed layer LE. More specifically, in the data relating to the superposition layer LE, the superposition position of the LED image GB and / or the hardware key area image GC with respect to the panel image GA (the superposition position of the LED image GB or the like in the panel image GA) is designated by the coordinate value. You may be made to do. In this way, the positional relationship among the panel image GA, the LED image GB, and the hardware key area image GC is adjusted (in other words, the corresponding positions of both layers LB and LE are adjusted). Also good.

  Such image data relating to each of the layers LB and LE is generated by the MFP 10 and transmitted to the external terminal 50. The external terminal 50 superimposes the two layers LB and LE on the touch panel 75 based on the received image data. indicate. As a result, a remote operation screen RC2 as shown in FIG.

  Next, the remote operation screen RC1 will be described.

  FIG. 9 is a diagram showing a remote operation screen RC1 displayed on the touch panel 75. As shown in FIG.

  As shown in FIG. 9, the remote operation screen RC1 includes the panel image GA and the LED image GB, but does not include the hardware key area image GC.

  As described above, the remote operation screen RC1 is a composite image obtained by superimposing only the LED image GB on the panel image GA and superimposing the LED image GB on both the images of the LED image GB and the hardware key area image GC. In the remote operation screen RC1, the LED image GB is provided so as to be superimposed on the panel image GA in the vicinity of the lower side edge portion in the panel image GA. More specifically, the LED image GB is arranged further below the software key area RB in which the plurality of software keys SK1 to SK7 are arranged in the vicinity of the center in the left-right direction of the panel image GA.

  Such a remote operation screen RC1 is also expressed using, for example, a data file FL (also referred to as FL1) having a layer structure (see FIG. 10). As shown in FIG. 10, the data file FL1 also has a base layer LB and a superimposed layer LE in the same manner as the data file FL2 (FIG. 6).

  The base layer LB (LB1) in the data file FL1 has the same contents as the base layer LB (LB2) in the data file FL2. That is, the image data related to the base layer LB2 is configured to include image data representing the panel image GA (see FIG. 7).

  On the other hand, the superimposed layer LE (LE1) is configured to include image data representing only the LED image GB as shown in FIG. This overlapping layer LE1 is roughly divided into two strip-like regions RG11 and RG12. The strip-shaped region RG12 is a region where the LED image GB is arranged, and has the same height as the LED image GB. In addition, transparent graphic parts are arranged on both sides of the LED image GB in the belt-like region RG12. The band-shaped region RG11 is the remaining region (a region other than the band-shaped region RG12), and transparent graphic parts are arranged in the band-shaped region RG11.

  Such image data relating to each of the layers LB and LE is generated by the MFP 10 and transmitted to the external terminal 50. The external terminal 50 superimposes the two layers LB1 and LE1 on the touch panel 75 based on the received image data. indicate. As a result, a remote operation screen RC1 as shown in FIG.

  As the data file FL (FL1, FL2) having such a layer structure, various types such as a PDF (Portable Document Format) format, an XPS format, and an OfficeOpenXML format may be adopted. Alternatively, a unique file may be used as the data file FL having a layer structure.

  FIG. 14 is a diagram showing screen transition and the like between the two remote operation screens RC1 and RC2.

  As shown in FIG. 14, in this embodiment, the two remote operation screens RC1 and RC2 are switched and displayed as appropriate. At the time of such screen transition, the LED image GB also functions as an area (also referred to as a switching instruction receiving area or a switching instruction receiving key) for receiving a switching instruction for switching between display and non-display of the hardware key area image GC. To do. In other words, a switching instruction (operation input) for switching between display and non-display of the hardware key area image GC is accepted via the LED image GB.

  Specifically, in the initial state immediately after the start of the remote operation, first, the remote operation screen RC1 is generated by the MFP 10 and displayed on the touch panel 75 of the external terminal 50.

  Thereafter, the remote operation screen RC2 is displayed according to the operation of the operator. Specifically, when a pressing operation is performed on the LED image GB displayed on the touch panel 75, the MFP 10 gives an instruction to switch from the remote operation screen RC1 to the remote operation screen RC2 (display instruction of the hardware key area image GC). It is determined that In response to the pressing operation, the MFP 10 generates display data for the remote operation screen RC2 and transmits the display data to the external terminal 50. The external terminal 50 displays the remote operation screen RC2 on the touch panel based on the display data. 75.

  Hereinafter, such an operation will be described in more detail.

<1-5. Operation>
<Initial display>
FIG. 15 is a timing chart showing the operation in the image forming system 1. In FIG. 15, data exchange between the MFP 10 and the external terminal 50 is shown.

  In step S <b> 11 of FIG. 15 immediately after the start of remote operation by the external terminal 50, a request for transmission of display data for the remote operation screen RC is transmitted from the external terminal 50 to the MFP 10.

  In step S12, the MFP 10 (image generation unit 15) generates a remote operation screen RC1 for initial display. Specifically, as shown in FIG. 10, image data D2 (see FIG. 7) relating to the base layer LB and image data D1 (see FIG. 11) relating to the superimposed layer LE are generated.

  In step S13, the MFP 10 (the transmission unit 4a and the like) transmits data regarding a plurality of layers including both layers LB and LE to the external terminal 50 as display data for the remote operation screen RC1. Specifically, the image data D2 related to the base layer LB and the image data D1 related to the superimposed layer LE are transmitted from the MFP 10 to the external terminal 50 as display data for the remote operation screen RC1, respectively. The external terminal 50 temporarily stores the received display data in the storage unit 55.

  In the next step S <b> 14, the external terminal 50 displays the remote operation screen RC <b> 1 (FIG. 9) on the touch panel 75 based on the data received from the MFP 10. Specifically, the external terminal 50 generates a composite image in which the LED image GB is superimposed on the panel image GA as the remote operation screen RC1, based on the image data D2 related to the base layer LB and the image data D1 related to the superimposed layer LE. The composite image (remote operation screen RC1) is displayed on the touch panel 75.

<Screen transition from remote operation screen RC1 to remote operation screen RC2>
Thereafter, the operator of the external terminal 50 performs a remote operation using the remote operation screen RC1 displayed on the touch panel 75 (step S15). In this embodiment, the LED image GB also functions as a switching instruction receiving area for receiving a switching instruction for switching between display and non-display of the hardware key area image GC. For example, the operator presses the LED image GB in the remote operation screen RC1 to give a display instruction for the hardware key area image GC (also expressed as a screen transition instruction from the remote operation screen RC1 to the remote operation screen RC2). Can be granted. When the display instruction is given, the following operation is further executed in response to the display instruction.

  Specifically, first, the external terminal 50 generates operation input information for the remote operation screen RC1, and transmits operation input information including coordinate information (X, Y) of the pressing operation position in the remote operation screen RC1 to the MFP 10. (Step S16).

  Next, in step S17, the MFP 10 (determination unit 16) determines which key the operation input (pressing operation) by the operator is based on the operation position in the operation input, and the base layer LB and the superimposed layer LE. It is determined to which of the operation inputs has been performed. More specifically, when the operation position is a position in an image part of the superimposed layer LE (specifically, an image part other than a transparent graphic part), the operation target layer (operation input target layer) is the superimposed layer LE. It is determined. On the other hand, when the operation position is a position in the transparent graphic part of the superimposed layer LE, it is determined that the operation target layer is the base layer LB.

  Here, based on the coordinate value of the pressing operation position by the operator, it is determined that the pressing operation position is a position in the LED image GB of the superimposed layer LE, and that the pressing operation (operation input) has been performed on the LED image GB. In addition, it is determined that the operation input is for the superimposed layer LE, which is the layer to which the LED image GB belongs, out of the two layers LB and LE.

  In response to the determination that the pressing operation (operation input) has been performed on the LED image GB, the MFP 10 regards the pressing operation on the LED image GB as a switching instruction from the remote operation screen RC1 to the remote operation screen RC2. Thus, display data for the remote operation screen RC2 (FIG. 5) is generated. Specifically, the image generation unit 15 of the MFP 10 determines whether the base layer LB and the superimposed layer LE are based on the contents of the operation input determined by the determination unit 16 (such as that the operation input is for the LED image GB). Of these, only the superposition layer LE is (selectively) updated. More specifically, it is determined that the pressing operation of the LED image GB is an instruction to display the hardware key area image GC, and the updated image data D1 related to the superimposed layer LE is, as shown in FIG. 6 and FIG. Image data including both the GB and the hardware key area image GC is generated.

  Further, in step S18, the MFP 10 transmits display data for the remote operation screen RC2 to the external terminal 50. Specifically, among both the image data D2 related to the base layer LB and the image data D1 related to the superimposed layer LE, the image data D1 related to the superimposed layer LE (LE2) (see FIGS. 6 and 8) updated in step S17. Only from the MFP 10 to the external terminal 50.

  In the next step S19, the external terminal 50, based on the newly received data (superimposition layer LE2 data) and the previously received data (display data for the remote operation screen RC1), the remote operation screen RC2 (FIG. 5) is displayed on the touch panel 75. Specifically, the external terminal 50 uses the image data of the base layer LB1 as the image data of the base layer LB2 among the display data of the remote operation screen RC1 received in advance, and the image of the base layer LB2 (LB1). A composite image is generated by superimposing the image data of the superposition layer LE2 newly received in step S18 on the data. Then, the external terminal 50 displays the composite image on the touch panel 75 as the remote operation screen RC2.

<Screen transition from remote operation screen RC2 to remote operation screen RC1>
Next, the operator of the external terminal 50 performs a remote operation using the remote operation screen RC (RC2) updated and displayed on the touch panel 75 (step S20). For example, the operator can give an instruction to hide the hardware key region image GC by pressing the LED image GB in the remote operation screen RC2. When the non-display instruction is given, the following operation is further executed in response to the non-display instruction (see FIG. 16).

  Specifically, first, the external terminal 50 generates operation input information for the remote operation screen RC2, and transmits operation input information including coordinate information (X, Y) of the pressing operation position in the remote operation screen RC2 to the MFP 10. (Step S21).

  Next, in step S22, a determination operation similar to that in step S17 is performed. Here, it is determined that the pressing operation position by the operator is a position in the LED image GB, and the operation input is a superposition layer that belongs to the LED image GB among the two layers LB and LE. It is determined that it is for LE.

  Then, in response to the pressing operation of the LED image GB, the MFP 10 generates display data for the remote operation screen RC1 (FIG. 9). Specifically, the image generation unit 15 of the MFP 10 determines whether the base layer LB and the superimposed layer LE are based on the contents of the operation input determined by the determination unit 16 (such as that the operation input is for the LED image GB). Of these, only the superposition layer LE is (selectively) updated. More specifically, it is determined that the pressing operation of the LED image GB is a non-display instruction for the hardware key area image GC, and the image data D1 related to the superimposed layer LE after the update includes the LED image GB as shown in FIG. Image data including only the LED image GB among the images with the hardware key area image GC is generated.

  In step S <b> 23, the MFP 10 transmits display data for the remote operation screen RC <b> 1 to the external terminal 50. Specifically, only the image data D1 related to the updated superimposed layer LE (specifically LE1) of both the image data D2 related to the base layer LB and the image data D1 related to the superimposed layer LE is transferred from the MFP 10 to the external terminal 50. Sent.

  In the next step S24, the external terminal 50, based on the newly received data (superimposition layer LE1 data) and the previously received data (base layer LB1 (LB2) data), the remote operation screen RC1 (FIG. 12) is displayed on the touch panel 75. Specifically, the external terminal 50 changes the image data of the superimposed layer LE1 (newly received in step S23) to the image data of the base layer LB1 among the display data of the remote operation screen RC1 received in advance. A superimposed composite image is generated. Then, the external terminal 50 displays the composite image on the touch panel 75 as the remote operation screen RC1.

<Screen transition from remote operation screen RC1 to remote operation screen RC3>
The operator of the external terminal 50 may give an instruction to display the detailed setting screen related to the setting item “color” by pressing the software key SK2 in the remote operation screen RC (RC2) updated and displayed in step S24. it can. When the display instruction is given in step S25, the following operation (see FIG. 16) is executed.

  Specifically, first, the external terminal 50 generates operation input information for the remote operation screen RC1, and transmits operation input information including coordinate information (X, Y) of the pressing operation position in the remote operation screen RC1 to the MFP 10. (Step S26).

  Next, in step S27, a determination operation similar to that in step S17 is performed. In this case, it is determined that the pressing operation position by the operator is a position in the software key SK2, and the operation input is a base to which the software key SK2 belongs among the two layers LB and LE. It is determined that it is for the layer LB.

  Then, in response to the pressing operation of the software key SK2, the MFP 10 generates display data for the remote operation screen RC3 (FIG. 12). Specifically, the image generation unit 15 of the MFP 10 determines whether the base layer LB and the superimposed layer LE are based on the contents of the operation input determined by the determination unit 16 (such as that the operation input is for the software key SK2). Of these, only the base layer LB is updated (selectively). More specifically, only the base layer LB including the panel image GA is updated in response to the panel image GA being changed in response to the pressing operation of the software key SK2.

  In step S28, the MFP 10 transmits display data for the remote operation screen RC3 to the external terminal 50. Specifically, only the image data D2 related to the base layer LB (specifically LB3) out of both the image data D2 related to the base layer LB and the image data D1 related to the superimposed layer LE is transmitted from the MFP 10 to the external terminal 50. . The image data D2 related to the base layer LB3 includes image data related to the panel image GA (GA3 (see FIG. 12)).

  In the next step S29, the external terminal 50, based on the data received in advance (data of the superposition layer LE1) and the data newly received in step S28 (data of the base layer LB3) (see FIG. 12) is displayed on the touch panel 75. Specifically, the external terminal 50 uses the image data of the superimposed layer LE1 as the image data of the superimposed layer LE3 among the display data of the remote operation screen RC1 received in advance, and the image of the superimposed layer LE3 (LE1). A composite image is generated by superimposing the base layer LB3 image data (newly received in step S28) on the data. Then, the external terminal 50 displays the composite image on the touch panel 75 as the remote operation screen RC3.

<Screen transition from remote operation screen RC2 to remote operation screen RC4>
In the above, the screen transition from the remote operation screen RC1 to the remote operation screen RC3 (see FIG. 12) has been illustrated, but the screen transition from the remote operation screen RC2 to the remote operation screen RC4 (see FIG. 13) is similarly performed. Is called. The remote operation screen RC4 is a screen having the same configuration as the remote operation screen RC3. However, the remote operation screen RC4 is different from the remote operation screen RC3 in that it has a hardware key area image GC. The remote operation screen RC4 is a screen having the same configuration as the remote operation screen RC2 except that the display content of the panel image GA is different.

  In the following, the operator of the external terminal 50 presses the software key SK2 in the remote operation screen RC (RC2) updated and displayed in step S19, and instructs to display the detailed setting screen regarding the setting item “color”. It shall be given (step S30). In this case, the following operation (see FIG. 17) is performed.

  Specifically, first, the external terminal 50 generates operation input information for the remote operation screen RC2, and transmits operation input information including coordinate information (X, Y) of the pressing operation position in the remote operation screen RC2 to the MFP 10. (Step S31).

  Next, in step S32, a determination operation similar to that in step S17 is performed. In this case, it is determined that the pressing operation position by the operator is a position in the software key SK2, and the operation input is a base to which the software key SK2 belongs among the two layers LB and LE. It is determined that it is for the layer LB.

  Then, in response to the pressing operation of the software key SK2, the MFP 10 generates display data for the remote operation screen RC4 (FIG. 12). Specifically, the image generation unit 15 of the MFP 10 determines whether the base layer LB and the superimposed layer LE are based on the contents of the operation input determined by the determination unit 16 (such as that the operation input is for the software key SK2). Of these, only the base layer LB is updated (selectively).

  In step S33, the MFP 10 transmits display data for the remote operation screen RC4 to the external terminal 50. Specifically, only the image data D2 related to the base layer LB (specifically LB4) out of both the image data D2 related to the base layer LB and the image data D1 related to the superimposed layer LE is transmitted from the MFP 10 to the external terminal 50. . The image data D2 related to the base layer LB4 includes image data related to the panel image GA (GA4 (see FIG. 13)).

  In the next step S34, the external terminal 50, based on the data received in advance (data of the superposition layer LE2) and the data newly received in step S33 (data of the base layer LB4), the remote operation screen RC4 (FIG. 12) is displayed on the touch panel 75. Specifically, the external terminal 50 uses the image data of the superimposed layer LE2 among the display data of the remote operation screen RC2 received in advance as the image data of the superimposed layer LE4, and the image of the superimposed layer LE4 (LE2). A composite image is generated by superimposing the image data of the base layer LB4 (newly received in step S33) on the data. Then, the external terminal 50 displays the composite image on the touch panel 75 as the remote operation screen RC4.

<Screen transition from remote operation screen RC2 to remote operation screen RC5>
When the hardware key image KG is pressed on the remote operation screen RC2, the panel image GA is changed according to the pressed hardware key image KG, and the updated remote operation screen RC is displayed on the touch panel 75. Is done.

  For example, when the hardware key image KG8 (image corresponding to the start key 35h) is pressed on the remote operation screen RC2, the remote operation screen RC5 (FIG. 5) is switched from the remote operation screen RC2 (FIG. 5) in response to the pressing operation. The screen transition to 19) is performed. The remote operation screen RC5 is a screen having the same configuration as the remote operation screen RC2 except that the display content of the panel image GA is different. In the panel image GA of the remote operation screen RC5, the word “Copying” is displayed.

  When the operator of the external terminal 50 presses the hardware key image KG8 in the remote operation screen RC (RC2) updated and displayed in step S19 and gives a start instruction (step S40), the following The operation (see FIG. 18) is executed.

  Specifically, first, the external terminal 50 generates operation input information for the remote operation screen RC2, and transmits operation input information including coordinate information (X, Y) of the pressing operation position in the remote operation screen RC2 to the MFP 10. (Step S41).

  Next, in step S42, a determination operation similar to that in step S17 is performed. Here, it is determined that the pressing operation position by the operator is a position in the hardware key image KG8, and the operation input is the layer to which the hardware key image KG8 belongs among the two layers LB and LE. It is determined that it is for a certain superimposed layer LE.

  Then, in response to the pressing operation of the software key SK2, the MFP 10 generates display data for the remote operation screen RC5 (FIG. 19). Specifically, the image generation unit 15 of the MFP 10 determines the base layer LB and the superimposed layer LE based on the content of the operation input determined by the determination unit 16 (such as that the operation input is for the hardware key image KG8). Of these, only the base layer LB is updated (selectively). More specifically, in response to the pressing of the hardware key image KG8, the panel image GA is changed to reflect the change of the panel image GA corresponding to the hardware key image KG8. Only the included base layer LB is updated.

  In step S43, the MFP 10 transmits display data for the remote operation screen RC5 to the external terminal 50. Specifically, only the image data D2 related to the base layer LB (specifically LB5) out of both the image data D2 related to the base layer LB and the image data D1 related to the superimposed layer LE is transmitted from the MFP 10 to the external terminal 50. . The image data D2 related to the base layer LB5 includes image data related to the panel image GA (GA5) (see FIG. 19).

  In the next step S44, the external terminal 50 determines the remote operation screen RC5 (FIG. 5) based on the previously received data (superimposition layer LE2 data) and the data newly received in step S43 (base layer LB5 data). 19) is displayed on the touch panel 75. Specifically, the external terminal 50 uses the image data of the superimposed layer LE2 as the image data of the superimposed layer LE5 among the display data of the remote operation screen RC2 received in advance, and the image of the superimposed layer LE5 (LE2). A composite image is generated by superimposing the image data of the base layer LB5 (newly received in step S43) on the data. Then, the external terminal 50 displays the composite image on the touch panel 75 as the remote operation screen RC5.

  In this way, the screen transition from the remote operation screen RC2 to the remote operation screen RC5 (FIG. 19) is performed in response to the pressing operation of the hardware key image KG8. Here, the case where the hardware key image KG8 is pressed is illustrated, but the same applies to the case where the other hardware key images KG1 to KG7, KG9, and KG10 are pressed. Specifically, the contents of the panel image GA are changed according to the pressed hardware key image KG, and the remote operation screen RC including the changed panel image GA is displayed on the touch panel 75.

<1-6. Effects of the embodiment>
In the above-described aspect, the MFP 10 transmits image data related to the base layer LB and the superimposed layer LE to the external terminal 50 as display data for the remote operation screen RC in the external terminal 50 (steps S13, S18, S23, (See S28, S33, and S43). Then, the external terminal 50 displays the remote operation screens RC1 (FIG. 9), RC2 (FIG. 5), RC3 (FIG. 12), RC4 (FIG. 13), and RC5 (FIG. 19) on the touch panel 75 based on the received data. indicate. Therefore, the operator can perform a remote operation related to the user interface unit 6 (specifically, the status display unit 30 and / or the hardware key 35) other than the touch panel 25. In particular, since the composite image generated by superimposing the superposition layer LE on the base layer LB is displayed on the touch panel 75, the LED image GB of the superposition layer LE is arranged inside the panel image GA of the base layer LB. Is done. Accordingly, it is possible to suppress a decrease in user operability as compared with the case where the panel image GA and the LED image GB are arranged side by side in the horizontal direction.

  In particular, the remote operation screen RC2 displayed in step S19 includes both the LED image GB and the hardware key area image GC. Therefore, information to be transmitted to the operator by the status display unit 30 is appropriately transmitted to the operator via the LED image GB, and the operator of the external terminal 50 uses the hardware key image KG. Remote operation input regarding the wear key 35 can be performed.

  Further, the remote operation screen RC1 displayed in step S14 or the like includes the LED image GB. Therefore, information to be transmitted to the operator by the status display unit 30 can be appropriately transmitted to the operator via the LED image GB. In particular, in this embodiment, in the state immediately after the start of the remote operation, the remote operation screen RC1 having only the LED image GB out of the LED image GB and the hardware key area image GC is displayed on the touch panel 75 (step S14). . Here, in an initial state immediately after remote start (in particular, a state immediately after login to the MFP 10 or a state immediately after the MFP 10 is turned on), generally, an operation on the hardware key image in the hardware key area image GC is performed. Is rarely done. By not displaying the hardware key image in such a situation, operability can be improved. Specifically, operability can be improved by suppressing unnecessary button display.

  Further, since the MFP 10 can selectively update a part of a plurality of layers constituting the remote operation screen RC and selectively transmit them to the external terminal 50, the total amount of data transmission can be reduced. (Reducing communication traffic) is possible.

  Specifically, the MFP 10 determines which one of the base layer LB and the superimposed layer LE should be updated based on operation input information that is operation input information for a certain remote operation screen RC. Specifically, when it is determined that a pressing operation has been performed on the panel image GA or the hardware key area image GC, it is further determined that the base layer LB should be updated. Further, when it is determined that the pressing operation on the LED image GB has been performed, it is further determined that the superimposed layer LE should be updated. Then, image data related to one layer to be updated is selectively generated (updated) out of the base layer LB and the superimposed layer LE, and the image data related to the one layer is updated to a new remote operation screen. It is selectively transmitted to the external terminal 50 as RC display data (specifically, part thereof). Furthermore, the external terminal 50 combines the image data related to the one layer with the image data related to the other layer received in advance, and displays a new remote operation screen RC. Therefore, it is possible to reduce the total amount of data transmission compared to the case where images of two types of layers LB and LE are always transmitted.

  More specifically, in the state transition from the remote operation screen RC1 to the remote operation screen RC2, the MFP 10 selectively generates image data of one superimposed layer LE among a plurality of layers constituting the remote operation screen RC2. Then, the data is selectively transmitted to the external terminal 50. According to this, since the data of the non-updated part (base layer LB data) is not transmitted and only the data of the updated part is substantially transmitted, the images of the two types of layers LB and LE are always transmitted. Compared to the above, it is possible to reduce the amount of data communication.

  The same applies to the state transition from the remote operation screen RC2 to the remote operation screen RC1. The MFP 10 selectively generates image data of one superimposed layer LE (including only the LED image GB) of the plurality of layers constituting the remote operation screen RC1, and selectively transmits the image data to the external terminal 50. According to this, since only the data of the updated part is transmitted substantially, it is possible to reduce the amount of data communication.

  In the state transition from the remote operation screen RC1 to the remote operation screen RC3, the MFP 10 selectively generates and selects image data of one base layer LB among a plurality of layers constituting the remote operation screen RC3. To the external terminal 50. According to this, since the data of the non-updated part (superimposition layer LE data) is not transmitted and only the data of the updated part is transmitted, it is possible to reduce the total amount of data transmission.

  Similarly, in the state transition from the remote operation screen RC2 to the remote operation screen RC4, the MFP 10 selectively generates image data of one base layer LB among a plurality of layers constituting the remote operation screen RC4. It selectively transmits to the external terminal 50. According to this, since only the data of the update part is transmitted substantially, it is possible to reduce the total amount of data transmission.

  Similarly, in the state transition from the remote operation screen RC2 to the remote operation screen RC5 (FIG. 19), the MFP 10 selectively selects the image data of one base layer LB among a plurality of layers constituting the remote operation screen RC5. And selectively transmitted to the external terminal 50. According to this, since the data of the non-updated part (superimposition layer LE data) is not transmitted and only the data of the updated part is transmitted, it is possible to reduce the total amount of data transmission.

<2. Second Embodiment>
The second embodiment is a modification of the first embodiment. Below, it demonstrates centering on difference with 1st Embodiment.

  In the first embodiment, a new superimposed layer LE (LE2 or the like) is generated by the MFP 10 and transmitted to the external terminal 50 in response to a screen transition instruction from the remote operation screen RC1 to the remote operation screen RC2. Is illustrated. However, the present invention is not limited to this. For example, image data D1 (D1b) (see FIG. 21) related to the superimposed layer LE10 as described below may be transmitted from the MFP 10 to the external terminal 50 in advance. Then, in response to a screen transition instruction from the remote operation screen RC1 to the remote operation screen RC2, the display target portion in the extension data D1b related to the superposition layer LE10 is changed to change from the remote operation screen RC1 to the remote operation screen RC2. The screen transition may be realized. According to this, screen transition from the remote operation screen RC1 to the remote operation screen RC2 is realized without transmitting new image data. In the second embodiment, such an aspect is illustrated.

  FIG. 20 is a functional block diagram showing a schematic configuration of the external terminal 50 (50B) according to the second embodiment. As illustrated in FIG. 20, the processing unit realized by the controller 59 includes an image generation unit 67. The image generation unit 67 updates the image related to the superimposed layer LE by changing the display target portion in the extension data D1b, and performs the synthesis process of the image data related to the base layer LB and the image data related to the superimposed layer LE. It is a processing unit that generates a remote operation screen RC.

  In the second embodiment, the image data related to the remote operation screen RC (RC1) received in step S13 is image data (also referred to as extension data) D1 of the superimposed layer LE (LE10) as shown in FIG. (D1b) and image data D2 of the base layer LB. FIG. 21 is a diagram illustrating a data structure of the remote operation screen RC according to the second embodiment.

  The image data (extended data) D1b of the superimposed layer LE (LE10) has data D11, D12, and D13. The data D11, D12, D13 correspond to the strip regions RG21, RG22, RG23 (see FIG. 8), respectively. The image data D1b further includes data D10. The data D10 is image data relating to a region RG20 (see FIG. 22) having the same size as the belt-shaped region RG23. Data D10 is data indicating that transparent graphic parts are arranged over the entire region RG20.

  The image data D2 of the base layer LB is the same as that in the first embodiment.

  FIG. 22 is a diagram illustrating a portion used when displaying the remote operation screen RC2 in the data D1b of the superposition layer LE10. When the remote operation screen RC2 is displayed, three data D11, D12, D13 among the four data D10, D11, D12, D13 are used. More specifically, the strip regions RG21, RG22, and RG23 are selected as display target regions, and data D11, D12, and D13 corresponding to the strip regions RG21, RG22, and RG23 are used. As a result, the LED image GB in the strip region RG22 and the hardware key region image GC in the strip region RG23 are displayed superimposed on the panel image GA at appropriate positions (see FIG. 5).

  On the other hand, FIG. 23 is a diagram illustrating a portion used when displaying the remote operation screen RC1 in the data D1b of the superposition layer LE10. When the remote operation screen RC1 is displayed, three data D10, D11, D12 among the four data D10, D11, D12, D13 are used. More specifically, the strip regions RG20, RG21, RG22 are selected as display target regions, and data D10, D11, D12 corresponding to the strip regions RG20, RG21, RG22 are used. Among these, the data D10 corresponding to the strip region RG20 and the data D11 corresponding to the strip region RG21 can represent the same contents as the strip region RG11 (see also FIG. 11). Further, the data D12 corresponding to the belt-like region RG22 can represent the same contents as the belt-like region RG12 (see also FIG. 11).

  When the image generation unit 67 described above displays the remote operation screens RC1 and RC2, the display target region (display target part) in the data D1b (image data D1b related to the superimposed layer LE) temporarily stored in the storage unit 55 is displayed. Move (change). Specifically, a portion corresponding to the display target area is extracted from the data D1b. Then, the image generation unit 67 performs a combining process using the image data D2 of the base layer LB and the updated image data of the superimposed layer (the image data of the portion extracted corresponding to the display target region). The remote operation screen RC is generated.

  As described above, the data D1b related to the remote operation screen RC is shared when the two types of remote operation screens RC1 and RC2 are switched and displayed.

  FIG. 24 is a timing chart showing an operation according to the second embodiment.

  In step S12b (FIG. 24) after step S11, image data (FIG. 21) having the data D1 (D1b) and D2 as described above is generated as display data for the remote operation screen RC.

  In the next step S <b> 13 b, the image data having the data D <b> 1 b and D <b> 2 is transmitted from the MFP 10 to the external terminal 50 and stored in the storage unit 55 of the external terminal 50. In step S13b, the MFP 10 also transmits a display instruction for the remote operation screen RC1 to the external terminal 50. The display instruction of the remote operation screen RC1 is also expressed as a non-display instruction for a hardware key image (or hardware key area image GC).

  In step S14b, as shown in FIG. 23, the remote operation screen RC1 is displayed by using the data D10, D11, D12. Specifically, in response to the instruction to hide the hardware key image, the image generation unit 67 does not include the hardware key region image GC from the image data related to the superimposed layer LE stored in the storage unit 55 and A portion including the LED image GB (a portion surrounded by a left thick frame in FIG. 23) is extracted as a portion corresponding to the display target region TG (a portion surrounded by a right thick frame in FIG. 23). Then, the image generation unit 67 performs composition processing using the image data of the extracted portion and the image data related to the base layer LB including the panel image GA, thereby not including the hardware key area image GC and the LED image. An image including the GB and the panel image GA (composite image) is generated as the remote operation screen RC1. Then, the external terminal 50 displays the remote operation screen RC1 on the touch panel 75.

  Steps S15, S16, and S17 are the same as in the first embodiment.

  In step S18 (S18b), image data is not transmitted, and only an instruction to switch from the remote operation screen RC1 to the remote operation screen RC2 (display instruction for the remote operation screen RC2) is transmitted from the MFP 10 to the external terminal 50. The display instruction of the remote operation screen RC2 is also expressed as a display instruction of a hardware key image (or hardware key area image GC).

  Thereafter, in step S19b, as shown in FIG. 22, the remote operation screen RC2 is displayed by using the data D11, D12, D13. Specifically, in response to the display instruction of the hardware key image, the image generation unit 67 selects the hardware key area image GC and the LED image GB from the image data related to the superimposed layer LE stored in the storage unit 55. (A portion surrounded by a left thick frame in FIG. 22) is extracted as a portion corresponding to the display target region TG (a portion surrounded by a right thick frame in FIG. 22). Then, the image generation unit 67 performs composition processing using the extracted portion of the image data and the image data relating to the base layer including the panel image GA, thereby performing the hardware key area image GC, the LED image GB, and the panel image. An image (a composite image) including GA is generated as a remote operation screen RC2. Then, the external terminal 50 displays the remote operation screen RC2 on the touch panel 75.

  Similar modifications can be made for other screen transition operations. For example, the same modification may be made for the screen transition from the remote operation screen RC2 to the remote operation screen RC1.

  As described above, when the two types of remote operation screens RC1 and RC2 are switched to each other, the display target region (display) in the data D1b (image data D1b related to the superimposed layer LE) temporarily stored in the storage unit 55 is displayed. By moving (changing) the target part), an image of the superimposed layer LE of the updated remote operation screen RC is generated. Then, the updated remote operation screen RC is displayed by combining the base layer LB image and the updated superimposed layer image. According to this, when the two types of remote operation screens RC1 and RC2 are switched to each other, it is not always necessary to transmit data related to the remote operation screen RC (particularly, all of the image data D1 related to the superimposed layer LE). . Therefore, it is possible to suppress the data communication amount.

  In addition, when the display content of the status display unit 30 is changed according to the change in the device status, the display content of the LED image GB may be updated by retransmitting the display data of the remote operation screen RC. preferable. In other words, when there is no change in the device state, the hardware in the external terminal 50 is transmitted by transmitting only the switching instruction of the remote operation screen RC without retransmitting the display data for the remote operation screen RC. It is preferable that the display state and non-display state of the key area image GC are switched to each other.

<3. Modified example>
Although the embodiments of the present invention have been described above, the present invention is not limited to the contents described above.

  For example, in each of the above embodiments, the operation target layer that is the target of the operation input on the remote operation screen RC is determined by the MFP 10, but the present invention is not limited to this, and the operation that is the target of the operation input on the remote operation screen RC The target layer may be determined by the external terminal 50.

  Specifically, according to the operation input in step S15 (, S20, etc.), the external terminal 50 specifies the operation position (coordinate value) on the remote operation screen RC and a plurality of layers constituting the remote operation screen RC. The operation target layer is specified from among the base layer LB and the superimposed layer LE. Specifically, when the operation position is a position in an image part (specifically, an image part other than a transparent graphic part) of the superimposed layer LE, the operation target layer is determined to be the superimposed layer LE. On the other hand, when the operation position is a position in the transparent graphic part of the superimposed layer LE, it is determined that the operation target layer is the base layer LB. Such a specifying operation is executed by the specifying unit 69 of the external terminal 50 (50C) (see FIG. 25).

  Then, the external terminal 50 (transmission unit 54a) transmits operation input information including information regarding the specified operation target layer (for example, a layer identification number representing the superimposed layer LE) and an operation position (coordinate value) to the MFP 10. To do. On the other hand, the MFP 10 (determination unit 16) determines the content of the operation input based on the received operation input information.

  In this way, the operation target layer may be determined (specified) by the external terminal 50.

  Moreover, in each said embodiment, the image data D1 regarding the superimposition layer LE and the image data D2 regarding the base layer LB are separately transmitted to the external terminal 50 as display data of the remote operation screen RC in the external terminal 50. Is illustrated. In other words, the image data D2 related to the base layer LB and the image data D1 related to the superimposed layer LE are individually transmitted in a state before being synthesized. However, the present invention is not limited to this. For example, the image data related to the base layer LB and the superimposed layer LE may be transmitted to the external terminal 50 in a state where the image data D1 related to the superimposed layer LE and the image data D2 related to the base layer LB are combined.

  In the above embodiment, two layers (specifically, a single base layer LB and a single superimposed layer LE) are exemplified as the plurality of layers, but the present invention is not limited to this. For example, two or more superimposed layers LE including a superimposed layer for the LED image GB and a superimposed layer for the hardware key area image GC may be provided in addition to the base layer LB. In other words, two or more layers including the superimposed layer for the LED image GB and the superimposed layer for the hardware key area image GC may be provided as the superimposed layer.

DESCRIPTION OF SYMBOLS 1 Image forming system 6 User interface part 6c Operation panel part 10 MFP (image forming apparatus)
25 Touch Panel 30 Status Display 35 Hardware Key 75 Touch Panel 90 Computer GA Panel Image GB LED Image GC Hardware Key Area Image KG1 to KG10 Hardware Key Image LB Base Layer LE Superimposition Layer RC Remote Operation Screen SK1 to SK7 Software Key

Claims (16)

An image forming apparatus that can be remotely operated by an external terminal,
A touch panel that displays various types of information and receives operation inputs from the operator;
A user interface provided separately from the touch panel;
Generation for generating data relating to a plurality of layers including a base layer including an image indicating an operation screen displayed on the touch panel and an image indicating the user interface unit and including at least one superimposed layer superimposed on the base layer Means,
Transmitting means for transmitting data relating to the plurality of layers to the external terminal as display data for a remote operation screen in the external terminal;
An image forming apparatus comprising: The image forming apparatus according to claim 1.
Receiving means for receiving, from the external terminal, operation input information that is operation input information for the remote operation screen displayed based on the data related to the plurality of layers at the external terminal;
Determination means for determining the content of the operation input based on the operation input information;
Further comprising
The determination means determines one layer to be updated out of the base layer and the at least one superimposed layer based on the content of the operation input,
The generation means selectively updates image data related to the one of the base layer and the at least one superimposed layer,
The transmission means transmits image data relating to the one layer after update to the external terminal as display data for a new remote operation screen after updating the remote operation screen. . The image forming apparatus according to claim 1.
The user interface unit includes a hardware key that is provided separately from the touch panel and that receives an operation input from the operator, and a status display unit that is provided separately from the touch panel and displays the device status of the image forming apparatus. An image forming apparatus having at least one of the following. The image forming apparatus according to claim 3.
The user interface unit has both the hardware key and the status display unit,
The generation means generates image data relating to both the hardware key image corresponding to the hardware key and the state display image indicating the display content in the state display unit as the image data of the at least one superimposed layer. ,
The transmission unit transmits data relating to the plurality of layers including the at least one superimposed layer and the base layer to the external terminal as the display data of the remote operation screen. . The image forming apparatus according to claim 3.
The user interface unit includes the state display unit,
The generation means generates image data relating to a state display image indicating display contents in the state display unit as image data of the at least one superimposed layer,
The transmission unit transmits data relating to the plurality of layers including the at least one superimposed layer and the base layer to the external terminal as the display data of the remote operation screen. . The image forming apparatus according to any one of claims 1 to 5,
The transmission means separately transmits image data relating to the base layer and image data relating to the at least one superimposed layer to the external terminal as display data for a remote operation screen in the external terminal. Forming equipment. The image forming apparatus according to claim 3.
The user interface unit has both the hardware key and the status display unit,
In the state immediately after the start of the remote operation, the generation means includes, as the image data of the at least one superimposed layer, a hardware key image corresponding to the hardware key, and a state display image indicating display contents on the state display unit, Image data related to only the state display image is generated, and the transmission means displays the data related to the plurality of layers including the at least one superimposed layer and the base layer on the display of the remote operation screen. An image forming apparatus that transmits the data as external data to the external terminal. The image forming apparatus according to claim 7.
Receiving means for receiving, from the external terminal, operation input information that is operation input information for the remote operation screen displayed based on the data related to the plurality of layers at the external terminal;
Determination means for determining the content of the operation input based on the operation input information;
Further comprising
The state display image also functions as a switching instruction receiving area for receiving a switching instruction for switching between display and non-display of the hardware key image,
In response to the determination that the operation on the state display image has been performed based on the operation input information, the generation unit uses the image data regarding both images as the image data of the at least one superimposed layer. The transmitting means transmits the image data relating to the at least one superimposed layer to the external terminal as display data for a second remote operation screen that is a screen after the remote operation screen is updated. An image forming apparatus. The image forming apparatus according to claim 8.
In response to the determination that the operation on the state display image has been performed based on the operation input information, the transmission unit is configured to transmit the at least one of the base layer and the at least one superimposed layer. An image forming apparatus, wherein image data relating only to a superimposition layer is transmitted to the external terminal as the display data of the second remote operation screen. The image forming apparatus according to claim 9.
In response to determining that an operation has been performed on the state display image displayed together with the hardware key image on the second remote operation screen, the generation means includes the at least one superimposition. Image data relating to only the state display image of the two images is generated as the image data of the layer, and the transmitting means is an image relating to only the at least one superimposed layer out of the base layer and the at least one superimposed layer. An image forming apparatus, wherein data is transmitted to the external terminal as display data for a third remote operation screen, which is a screen after the second remote operation screen is updated. In the computer built in the image forming device that can be remotely operated by an external terminal,
a) A base layer including an image indicating an operation screen displayed on the touch panel of the image forming apparatus, and an image indicating a user interface unit provided in the image forming apparatus separately from the touch panel and superimposed on the base layer Generating data for a plurality of layers including at least one superimposed layer;
b) transmitting data relating to the plurality of layers to the external terminal as display data for a remote operation screen on the external terminal;
A program for running An image forming system,
An image forming apparatus;
An external terminal for remotely operating the image forming apparatus using a remote operation screen;
With
The image forming apparatus includes:
A touch panel that displays various types of information and receives operation inputs from the operator;
A user interface provided separately from the touch panel;
Generation for generating data relating to a plurality of layers including a base layer including an image indicating an operation screen displayed on the touch panel and an image indicating the user interface unit and including at least one superimposed layer superimposed on the base layer Means,
First transmission means for transmitting data relating to the plurality of layers to the external terminal as display data for a remote operation screen on the external terminal;
Have
The external terminal is
First receiving means for receiving data relating to the plurality of layers;
Display means for displaying the remote operation screen based on the data related to the plurality of layers before;
An image forming system comprising: The image forming system according to claim 12.
The image forming apparatus includes:
Receiving means for receiving, from the external terminal, operation input information that is operation input information for the remote operation screen displayed based on the data related to the plurality of layers at the external terminal;
Determination means for determining the content of the operation input based on the operation input information;
Further comprising
The determination means determines one layer to be updated out of the base layer and the at least one superimposed layer based on the content of the operation input,
The generation means selectively updates image data relating to the one layer of the base layer and the at least one superimposed layer,
The first transmission means transmits the image data relating to the one layer after update to the external terminal as display data for a new remote operation screen after updating the remote operation screen,
The display means of the external terminal displays the new remote operation screen based on the image data of the one layer relating to the new remote operation screen and the image data of the other layer relating to the remote operation screen. A featured image forming system. The image forming system according to claim 12.
The external terminal is
In response to an operation input to the remote operation screen displayed on the external terminal based on the data related to the plurality of layers, the operation position on the remote operation screen is specified, and the plurality of layers constituting the remote operation screen Specifying means for specifying the operation target layer from
Second transmission means for transmitting operation input information including information related to the operation target layer specified by the specifying means and the operation position to the image forming apparatus;
And an image forming system. The image forming system according to claim 12.
The user interface unit includes both a hardware key that is provided separately from the touch panel and receives an operation input from the operator, and a status display unit that is provided separately from the touch panel and displays the apparatus status of the image forming apparatus. Have
The data relating to the plurality of layers includes image data relating to the base layer and image data relating to the at least one superimposed layer,
The image data related to the at least one superimposed layer includes image data related to both a hardware key image corresponding to the hardware key and a state display image indicating display contents in the state display unit,
The external terminal is
Storage means for storing image data relating to the base layer and image data relating to the at least one superimposed layer;
A portion corresponding to the display target region is extracted from the image data relating to the at least one superimposed layer stored in the storage means, and a composition process is performed using the extracted portion of the image data and the image data relating to the base layer. A synthesis means for generating the remote operation screen by performing
And an image forming system. The image forming system according to claim 15, wherein
The state display image also functions as a switching instruction receiving area for receiving a switching instruction for switching between display and non-display of the hardware key image,
The synthesis means includes
When an instruction to display the hardware key image is given, a portion including the hardware key image and the state display image is selected from the image data related to the at least one superimposed layer stored in the storage area. The hardware key image, the state display image, and the operation screen are extracted by extracting as a portion corresponding to the display target region and performing synthesis processing using the image data of the extracted portion and the image data related to the base layer. And generating the image including the remote control screen,
When an instruction to hide the hardware key image is given, image data relating to the at least one superimposed layer stored in the storage area does not include the hardware key image and includes the state display image A part is extracted as a part corresponding to the display target area, and the image data of the extracted part and the image data related to the base layer are used to perform synthesis processing, thereby not including the hardware key image and the state An image forming system, wherein an image including a display image and the operation screen is generated as the remote operation screen.

Images