JP2016144097A - Remote panel system, display method for remote screen, and image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a portable terminal device to display a registered initial screen different from the first screen of an image processor in the case of connecting the image processor to the portable terminal device, and to correctly perform a transition to the initial screen even when a current screen transition method at the image processor side is different from a screen transition method in initial screen registration in the case of sequentially transmitting registered operation data for screen data transmitted from the image processor in a system in which the image processor is operated from the portable terminal device by a remote panel system.SOLUTION: When receiving operation data transmitted from a portable terminal for each screen during transition from a screen in connection to an initial screen, whether or not the screen of a transition destination based on device information transmitted from a portable terminal device is the same as the screen of the transition destination based on current device information is determined on the basis of transition destination information, and when it is determined that the screens are different, the screen transition destination is corrected on the basis of the transition destination information, and the screen data of the transition destination after correction are transmitted to the portable terminal device.SELECTED DRAWING: Figure 12

Description

  The present invention provides an image processing apparatus such as an MFP (Multi Function Peripherals) that is a multifunction digital image processing apparatus having functions such as a printer function, a facsimile function, and a scan function, and a portable terminal device such as a smartphone or a tablet terminal. The present invention relates to a remote panel system provided, a remote screen display method executed by the system, and an image processing apparatus.

  As a method for operating an image processing apparatus such as the MFP as described above from a portable terminal device, there is a method called a remote panel. In this method, by transmitting the screen data of the operation screen of the image processing device to the mobile terminal device, the same screen as the operation screen is displayed as a remote screen on the screen of the mobile terminal device, and on the screen of the mobile terminal device. When the user performs an operation, the operation data is transmitted as coordinate data to the image processing apparatus. The image processing apparatus analyzes this operation data to determine which button on the screen has been operated, and transmits the screen data of the next screen corresponding to the button to the mobile terminal device. The mobile terminal device displays the received screen data and updates the screen. By repeating such processing, the processing corresponding to the button operated on the screen by the user of the mobile terminal device is finally executed in the image processing device.

  By the way, the user customizes the operation screen of the image processing device and registers one of the screens from the first screen displayed on the operation panel to the final operation screen as the initial screen of the user. The user can display a customized initial screen as the first screen.

  Therefore, even in the portable terminal device in the remote panel method described above, the remote screen halfway from the first screen displayed when connected to the image processing device to the final operation screen is registered as the initial screen of the user, Thereafter, when the image processing apparatus is connected, the registered initial screen may be displayed first.

  In this case, the registered initial screen is displayed on the mobile terminal device, but the screen data of the first screen at the time of connection is actually transmitted from the image processing device connected to the mobile terminal device. Is different from the registered initial screen. If the initial screen is not registered, the remote screen displayed on the mobile terminal device based on the first screen data is changed to the next screen by the user's operation, and eventually the same screen as the initial screen is changed. .

  Therefore, the operation data corresponding to the operation to be performed by the user on each screen from the initial screen to the initial screen at the time of connection is registered in advance in the portable terminal device and transmitted from the image processing device. For each screen data from the initial screen to the initial screen, the registered operation data is automatically sent to the image processing device in order, without the user of the mobile terminal device being aware of it, Similarly to the normal operation based on the transmission of screen data from the image processing apparatus, the screen on the image processing apparatus side and the remote screen of the mobile terminal device can be shifted to the initial screen. Thereafter, as necessary, operations such as operations on the remote screen, transmission of operation data to the image processing device, analysis of operation data by the image processing device, execution of instructed processing, update to the next screen, etc., as necessary Is done.

  However, the above-described technology that enables the display of the initial screen in the mobile terminal device has the following problems.

JP, 2014-120017, A

  In other words, the installed firmware may be upgraded in the image processing apparatus, and a new function may be added or the function may be deleted as a result of such version upgrade. When the current screen transition from the screen when connecting to the initial screen registered to the mobile terminal device and the screen transition method when the initial screen is registered to the mobile terminal device result in different There is.

  In addition, since there is not one image processing device that can be connected to the mobile terminal device, depending on the type of the image processing device, the current screen from the screen when connected to the mobile terminal device to the initial screen registered in the mobile terminal device In some cases, the screen transition method differs from the screen transition method when the initial screen is registered.

  In such a case, even if the registered operation data is sequentially transmitted with respect to the screen data transmitted from the image processing apparatus, the screen on the image processing apparatus side does not correspond to the operation data, and the transition to the initial screen is made correctly. There is a problem that can not be.

  Note that the above-described Patent Document 1 discloses a technique for appropriately displaying information on various contents on an information processing terminal without increasing a processing burden when generating a Web page in the image processing apparatus. Yes. Specifically, in the case of content that cannot be displayed on the terminal side (display with pop-up), an image is formed (bitmap) on the image processing device side and data is transmitted to the information processing terminal. is there.

  However, in this system, in the system in which the image processing device is operated from the portable terminal device by the remote panel method, when connecting to the image processing device, an initial screen different from the screen of the image processing device is displayed on the portable terminal device. The current screen transition on the image processing device side when the operation data registered from the mobile terminal device is transmitted to the screen data from the processing device and the screen data from the image processing device is automatically transitioned to the initial screen. The screen on the image processing device and the registered operation data do not correspond to each other and the screen cannot be transitioned correctly to the initial screen. It was not possible to provide a means to solve this problem.

  The present invention has been made in view of such a technical background, and in a system in which an image processing apparatus is operated from a portable terminal apparatus by a remote panel method, an image is displayed when the image processing apparatus and the portable terminal apparatus are connected. When the registered initial screen different from the first screen of the processing device is displayed on the mobile terminal device, and the operation data registered is sequentially transmitted with respect to the screen data transmitted from the image processing device, the image processing device side Provides a remote panel system, a remote screen display method, and an image processing apparatus that can make a transition to the initial screen even if the current screen transition method is different from the screen transition method at the time of initial screen registration The purpose is to do.

The above problem is solved by the following means.
(1) A remote panel system including an image processing device and a portable terminal device that can be connected to each other, wherein the image processing device includes a display unit having a touch panel and an operation of the own device displayed on the display unit. Transmitting means for transmitting screen data of the screen to the mobile terminal device, and transmitted from the mobile terminal device when the user operates the operation screen displayed as a remote screen on the mobile terminal device that has received the screen data. Receiving means for receiving operation data; and processing execution means for executing processing including screen update processing instructed by operation data based on the operation data received by the receiving means. Display means having a touch panel; and causing the display means to display the remote screen based on the screen data from the image processing apparatus. An operation screen of any one of the display control means and the image processing apparatus displayed on the display means by the display control means is registered as an initial screen, and the registration is performed from the first screen when connected to the image processing apparatus. Registration means for registering operation data for each screen necessary for screen transition up to the initial screen, registering device information of the image processing apparatus at the time of registration of the initial screen, and the device information as the image The registered operation data is transmitted to the image processing device in order for each screen from the connection screen transmitted from the image processing device to the transition to the initial screen. A storage means for storing transition destination information indicating a transition destination screen corresponding to the operation data generated for each type of model information. When the operation data transmitted from the mobile terminal device is received for each screen before the transition from the screen at the time of connection to the initial screen, the next transition destination screen that is transitioned by the operation data Judgment means for judging whether the transition destination screen based on the device information transmitted from the portable terminal device is the same as the transition destination screen based on the current device information based on the transition destination information stored in the storage means If it is determined by the determination means to be the same, the screen data of the transition destination screen is transmitted to the mobile terminal device, and if it is determined to be different, the screen transition destination is corrected based on the transition destination information. And a correction means for transmitting the corrected screen data of the transition destination to the portable terminal device.
(2) When the determination unit of the image processing apparatus determines that the transition destination screen based on the device information transmitted from the portable terminal device is different from the transition destination screen based on the current device information, the portable terminal Determine whether the transition destination screen based on the device information transmitted from the device has been deleted in the latest device information, or whether a new transition destination screen has been added as the transition destination screen based on the current device information, The remote panel system according to item 1, wherein the correction unit of the image processing apparatus performs correction according to deletion or addition of a screen.
(3) When the determination unit of the image processing apparatus determines that the transition destination screen based on the device information transmitted from the portable terminal device is deleted in the current device information, the correction unit The remote panel system according to item 2, wherein the correction is performed by ignoring operation data from the mobile terminal device and transmitting screen data of a dummy screen to the mobile terminal device.
(4) When the determination unit of the image processing apparatus determines that a new transition destination screen is added as a transition destination screen based on the current device information, the correction unit adds the added The remote panel system according to item 2, wherein the correction is performed by transmitting screen data of the screen to the mobile terminal device.
(5) The remote panel system according to any one of (1) to (4), wherein the device information is firmware version information of the image processing apparatus.
(6) The remote panel system according to any one of (1) to (4), wherein the device information is product information of the image processing apparatus.
(7) The remote panel system according to any one of (1) to (4), wherein the device information is screen size information of display means of the image processing apparatus.
(8) The remote panel system according to any one of the preceding items 1 to 7, wherein coordinate data of a screen is used as the transition destination information.
(9) The remote panel system according to any one of items 1 to 7, wherein a screen number is used as the transition destination information.
(10) The remote panel system according to any one of (1) to (9), wherein the correction unit of the image processing apparatus notifies the portable terminal device of the fact when the correction cannot be performed.
(11) A remote screen display method executed by a remote panel system including an image processing device and a portable terminal device that can be connected to each other, wherein the image processing device is displayed on a display unit having a touch panel. A transmission step of transmitting screen data of an operation screen of the own device to the mobile terminal device; and when the user operates the operation screen displayed as a remote screen on the mobile terminal device receiving the screen data, the mobile terminal A receiving step for receiving operation data transmitted from the apparatus, and a process executing step for executing a process including a screen update process instructed by the operation data based on the operation data received in the receiving step; The portable terminal device has a touch panel for displaying the remote screen based on the screen data from the image processing device. A display control step to be displayed on the display means, and an operation screen of any one of the image processing apparatuses displayed on the display means by the display control step is registered as an initial screen, and the first time when connecting to the image processing apparatus A registration step of registering operation data for each screen necessary for screen transition from the screen to the initial screen, and further registering device information of the image processing apparatus at the time of registration of the initial screen, and the device information Is transmitted to the image processing apparatus, and the registered operation data is sequentially input to each screen from the connection-time screen transmitted from the image processing apparatus to the initial screen. The image processing apparatus further displays a transition destination screen corresponding to the operation data generated for each model information. A step of storing transition destination information in a storage means; and when receiving the operation data transmitted from the mobile terminal device to each screen before transitioning from the screen at the time of connection to the initial screen, according to the operation data Whether the transition destination screen based on the device information transmitted from the mobile terminal device is the same as the transition destination screen based on the current device information is stored in the storage unit with respect to the next transition destination screen to be transitioned. If it is determined that the determination step is based on the transition destination information and the determination step is the same, the screen data of the transition destination screen is transmitted to the portable terminal device. Correcting the screen transition destination based on the transition destination information, and executing a correction step of transmitting the corrected transition destination screen data to the mobile terminal device. Remote screen display method.
(12) An image processing apparatus used in the remote panel system according to any one of items 1 to 10 above.

  According to the invention described in (1) above, when the image processing apparatus receives pre-registered operation data from the mobile terminal device for each screen before the transition from the screen at the time of connection to the initial screen, Whether the transition destination screen based on the device information transmitted from the mobile terminal device and the transition destination screen based on the current device information are the same for the next transition destination screen transitioned by the operation data is stored in the storage means. The determination is made based on the transition destination information indicating the screen of the transition destination corresponding to the operation data for each model information stored. And when it is judged that it is the same, the screen data of the transition destination screen is transmitted to a portable terminal device. If it is determined that they are different, the screen transition destination is corrected based on the transition destination information, and the corrected screen data of the transition destination is transmitted to the mobile terminal device.

  Each time operation data is transmitted from the mobile terminal device, such determination and processing are performed by the image processing device, and a transition destination screen based on device information transmitted from the mobile terminal device and current device information are used. Since the screen of the transition destination is different, even if the screen on the image processing device side does not correspond to the registered operation data from the mobile terminal device, the screen will eventually reach the initial screen by correcting the screen transition destination. Transition can be made correctly.

  According to the invention described in (2) above, when it is determined that the transition destination screen based on the device information transmitted from the mobile terminal device is different from the transition destination screen based on the current device information, the mobile terminal device Since the transition destination screen based on the device information sent from the latest device information is deleted, it is determined whether a new transition destination screen is added as the transition destination screen based on the current device information. The image processing apparatus can perform an appropriate correction according to the deletion or addition of the screen.

  According to the invention described in (3) above, when it is determined that the transition destination screen based on the device information transmitted from the mobile terminal device is deleted in the current device information, the operation data from the mobile terminal device Since the screen data of the dummy screen is transmitted to the mobile terminal device, the screen data of the erroneous operation screen corresponding to the operation data can be prevented from being transmitted to the mobile terminal device.

  According to the invention described in (4) above, when it is determined that a new transition destination screen is added as the transition destination screen based on the current device information, the screen data of the added screen is stored in the mobile terminal. Since it is transmitted to the device, it is possible to prevent screen data of an erroneous operation screen corresponding to the operation data from being transmitted to the mobile terminal device.

  According to the invention described in (5) above, the screen transition destination is corrected based on the version information of the firmware of the image processing apparatus.

  According to the invention described in item (6) above, the screen transition destination is corrected based on the product information of the image processing apparatus.

  According to the invention described in item (7), the screen transition destination is corrected based on the screen size information of the display means of the image processing apparatus.

  According to the invention described in (8), the transition destination screen is shown based on the screen coordinate data.

  According to the invention described in item (9) above, the transition destination screen is shown based on the screen number.

  According to the invention described in item (10) above, when the screen transition destination cannot be corrected, the mobile terminal device is notified to that effect, so the user of the mobile terminal device may take corresponding measures. it can.

  According to the invention described in (11) above, even if the transition destination screen based on the device information transmitted from the mobile terminal device is different from the transition destination screen based on the current device information, the screen transition destination As a result of this correction, it is possible to make a correct transition to the initial screen finally registered in the mobile terminal device.

  According to the invention described in (12) above, even if the transition destination screen based on the device information transmitted from the mobile terminal device is different from the transition destination screen based on the current device information, the screen transition destination By this correction, an image processing apparatus that can make a correct transition to the initial screen finally registered in the mobile terminal device is obtained.

It is a block diagram of the character input system which concerns on one Embodiment of this invention. It is a figure which shows the internal structure of an image processing apparatus. It is a block diagram which shows the structure of a portable terminal device. FIG. 6 is a diagram for explaining an outline of an operation when an operation screen of an image processing device is displayed as a remote screen on a mobile terminal device and a user operates this remote screen remotely from the mobile terminal device. is there. It is a figure which shows the content of the registration data registered into the portable terminal device. It is a sequence diagram which shows the exchange of the data performed between an image processing apparatus and a portable terminal device before the initial screen registered into the portable terminal device is displayed. It is a figure which shows an example when a part of operation screen of an image processing apparatus is deleted after registration of the initial screen to a portable terminal device. It is a figure which shows an example when a part of operation screen of an image processing apparatus is added after registration of the initial screen to a portable terminal device. 6 is a table in which transition destination information is collected for each screen held by the image processing apparatus in order to correct screen data. It is a figure for demonstrating the case where the screen size of an image processing apparatus differs at the time of the initial screen registration to a portable terminal device, and the present. It is a figure which shows an example of the message displayed when it cannot correct | amend with respect to the coordinate data from a portable terminal device. It is a flowchart for demonstrating operation | movement of an image processing apparatus.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a configuration diagram of a remote panel system according to an embodiment of the present invention.

  This system includes a plurality of image processing apparatuses 1, portable terminal apparatuses 2, and information processing apparatuses 3. The image forming apparatus 1, the mobile terminal apparatus 2, and the information processing apparatus 3 are all connected via a network 4.

  Here, the network 4 may be any one of a network using a dedicated line such as a LAN (Local Area Network), a network using a general line, and a network using wireless communication. The network 4 is connected to another network via an external network.

  The image forming apparatus 1 is an apparatus for forming a scanned original image and a copy image of an image generated from print data transmitted from the information processing apparatus 3 on a sheet. In this embodiment, a printer function, a facsimile function, The above-described MFP, which is a multifunction digital image processing apparatus having functions such as a scan function, is used. Hereinafter, the image processing apparatus is also referred to as an MFP. Here, the print data is a drawing instruction in a page description language obtained by converting a drawing instruction issued by the operating system or application program of the information processing apparatus 3 into a page description language that can be processed by the image forming apparatus 1 by a printer driver, or , PDF, TIFF, JPEG, XPS, and other document data described in a file format.

  The scanned document image can also be transmitted to the information processing apparatus 3 and other MFPs via the network 4. Furthermore, the document image can be transmitted to another MFP or information processing apparatus connected to another network via an external network. Furthermore, the manuscript image that has been operated and the document data received from the information processing apparatus can be stored in the fixed storage device.

  The portable terminal device 2 is a portable computer device having a CPU, a RAM, a fixed storage device (such as a hard disk device), a monitor, and a liquid crystal panel that can be touch-operated. Electronic documents can be stored and carried and viewed and edited anywhere. In addition, it has means for wireless communication, can send and receive documents to and from the MFP 1 and the information processing apparatus 3, and has means for remotely operating the MFP 1. In the following description, the mobile terminal device is also simply referred to as a mobile terminal.

  The information processing device 3 is a general computer device connected to a CPU, a RAM, a fixed storage device (such as a hard disk device), a monitor, a keyboard, and a mouse. The information processing apparatus 3 generates reprint data according to a user instruction, and transmits the generated print data to the MFP 1.

  FIG. 2 is a diagram illustrating an internal configuration of the MFP 1. The CPU 101 of the MFP 1 includes a ROM 102 in which a control program is stored, a working S-RAM (Static Random Access Memory) 103, and a battery-backed NV-RAM (nonvolatile memory) that stores various settings relating to image formation. 104 and the clock IC 105 are connected via a bus to constitute a control unit 100 that controls the entire MFP 1 in an integrated manner.

  The control unit 100 is connected via an image reading device 120 that reads a document image, various keys such as a numeric keypad that performs various inputs, a print key, and a logout key, an operation panel 130 that includes a display unit 131, and the network 4. A network interface (network I / F) 160 for transmitting / receiving various kinds of information to / from an external device such as the information processing device 3-y, and a copy image is generated from print data received by the network interface 160 A printer controller 150 and an image output device 140 for forming a copy image on a sheet are connected to each other via a bus.

  A fixed storage device 110 is connected to the control unit 100 via a bus. The fixed storage device 110 is, for example, a hard disk device. The fixed storage device 110 stores various data.

  A wireless interface (Wireless I / F) 170 and a short-range wireless interface (short-range wireless I / F) 180 are connected to the control unit 100 via a bus. The wireless interface 170 is used for communication with a network and communication with the mobile terminal device 2, and the short-range wireless interface 180 is an interface for performing short-range wireless communication such as Bluetooth or infrared data association (IrDA). It is.

  FIG. 3 is a block diagram showing the configuration of the mobile terminal 2. The mobile terminal 2 includes, for example, a tablet terminal or a mobile phone, and includes a CPU 201, a ROM 202 storing a control program, and a working RAM 203. The ROM 202 and the RAM 203 are connected to the CPU 201 via buses, respectively. Further, the CPU 201, the ROM 202, and the RAM 203 form a control unit 200 that comprehensively controls the entire mobile terminal 2.

  The control unit 200 is connected with a touch panel 205, a wireless run interface 206, and a short-range wireless interface 207, which display various types of information and perform operation input by directly touching the screen, via a bus. The wireless run interface 206 is used for communication with the network and communication with the MFP 1, and the short-range wireless interface 207 is an interface for performing short-range wireless communication such as Bluetooth or infrared communication.

  Further, a fixed storage device 204 is connected to the control unit 200 via a bus. The fixed storage device 204 is, for example, a hard disk device. The fixed storage device 204 stores various data.

  FIG. 4 shows an outline of an operation of a remote panel method in which an operation screen of the MFP 1 is displayed as a remote screen on the mobile terminal 2, and the user operates the remote screen to remotely operate the MFP 1 from the mobile terminal 2. FIG.

  When the mobile terminal 2 and the MFP 1 are wirelessly connected while the initial screen is displayed on the display unit 131 of the MFP 1, screen data of the initial screen is transmitted from the MFP 1 to the mobile terminal 2 (step S105). When the mobile terminal 2 receives the screen data (step S201), the mobile terminal 2 develops an image based on the screen data (step S202) and then draws it as a remote screen on the touch panel 205 (step S203).

  When the user performs a touch operation on the displayed remote screen on the touch panel 205 (step S204), the mobile terminal 2 detects and specifies the coordinates of the touch position (step S205), and then performs gesture type determination and coordinates. Is converted to the coordinates of the display unit 131 on the MFP 1 side (step S206), and operation information (touch event) including the coordinate data is transmitted to the MFP 1 (step S207).

  After performing the touch event reception process (step S106), the MFP 1 analyzes the content of the touch event by the touch event process (step S107), and specifies the next display screen corresponding to the analyzed touch event by the internal process. (Step S101). Then, for example, parts such as buttons necessary for configuring the specified screen are synthesized by screen creation processing (step S102) to create screen data, and then the screen data is compressed (step S103). It transmits to the terminal 2 (step S104).

  The mobile terminal 2 displays the remote screen based on the received screen data in the same manner as the initial screen. When the user performs a touch operation on the remote screen, a touch event is transmitted to the MFP 1, and the MFP 1 creates a corresponding next screen or executes a corresponding process. Thereafter, the creation of screen data based on the touch event and the execution of processing in the MFP 1 and the display of the remote screen on the portable terminal 2 and the transmission of the touch event are repeated as necessary, so that the portable terminal 2 sends the MFP 1 to the MFP 1. Is operated.

  When the MFP 1 is operated from the portable terminal 2 using such a remote screen, the user can customize the remote screen displayed as the initial screen in this embodiment. That is, the user of the portable terminal 2 registers the remote screen halfway from the first screen displayed when connected to the MFP 1 to the final operation screen as the initial screen. After registration, when the MFP 1 is connected, this initial screen is first displayed on the touch panel 205 of the portable terminal 2.

  Further, the user for each remote screen from the remote screen displayed on the mobile terminal 2 based on the initial screen data transmitted from the MFP 1 when the mobile terminal 2 and the MFP 1 are connected to the transition to the initial screen registered by the user. Operation data indicating the operation is also registered in the portable terminal 2. Such operation data is acquired from log information of the MFP 1 in this embodiment, although not limited thereto.

  That is, touch coordinate data for each screen when the user operates the remote screen to transition to the initial screen during a normal operation before registering the initial screen is stored in the MFP 1 as log information. The portable terminal 2 acquires the touch coordinate data from the MFP 1 and registers it as operation data. Further, the portable terminal 2 acquires the firmware version of the MFP 1 at the time of registration of the initial screen as device information and registers it together with operation data.

  FIG. 5 is a diagram showing the contents of registration data registered in the mobile terminal 2. In the example of FIG. 5, the initial screen is the screen G, the firmware version of the MFP 1 at the time of registration of operation data is 1234-5678, and a plurality of coordinate data is registered in order as a log (operation data). The log is obtained by touching the coordinates (100, 150) on the first remote screen displayed on the portable terminal 2 based on the screen data transmitted from the MFP when connected to the MFP 1, thereby displaying the next remote screen. , And by touching coordinates (75, 230) on this remote screen, the screen further transitions to the next remote screen. After repeating this, touch coordinates (90, 53) on the remote screen. This means that a transition to the remote screen of the initial screen G is made.

  When the initial screen G is registered, if it is connected to the MFP 1, the initial screen G is displayed as a remote screen on the portable terminal 2. Actually, the initial screen data different from the initial screen G from the MFP 1 is displayed. Is sent. On the other hand, when the mobile terminal 2 automatically transmits the coordinate data (100, 150), the mobile terminal 2 automatically transitions to the next remote screen. , 230) is further transferred to the next screen, and this is automatically repeated for the coordinate data of the log, whereby the screen on the MFP 1 side can be advanced to the same screen as the initial screen.

  Accordingly, as usual, operations such as operations on the remote screen, transmission of operation data to the MFP 1, analysis of operation data by the MFP 1, execution of instructed processing, update to the next screen, and the like are performed as necessary. .

  This state is shown in the sequence diagram of the mobile terminal 2 and the MFP 1 in FIG. When a connection request is transmitted from the portable terminal 2 to the MFP 1 (S1), there is a connection OK response from the MFP 1 (S2). When the portable terminal 2 stores the log (operation data), in other words, notifies the MFP 1 of the firmware version at the time of initial screen registration (S3), the first screen data 1 is transmitted from the MFP 1 to the portable terminal 2 (S4). .

  In response to the screen data 1, the portable terminal 2 automatically notifies the MFP 1 of the first coordinate data (X1, Y1) among the registered operation data (S5). Then, the next screen data 2 is transmitted from the MFP 1 to the portable terminal 2 in response to the operation indicated by the coordinate data (X1, Y1) (S6). In response to the screen data 2, the portable terminal 2 automatically notifies the MFP 1 of coordinate data (X2, Y2) (S7). Then, the next screen data 3 is transmitted from the MFP 1 to the portable terminal 2 in response to the operation indicated by the coordinate data (X2, Y2) (S8).

  By repeating such operations, the screen data n is transmitted from the MFP 1 to the portable terminal 2, and the initial screen G is displayed on the portable terminal 2 based on the screen data n. That is, the screen on the MFP 1 side is also advanced to the initial screen. For the initial screen G displayed on the mobile terminal 2, coordinate data of the touch position, which is operation data, is transmitted to the MFP 1 based on a user operation.

  On the other hand, the next screen data 3 (S8) transmitted from the MFP 1 in response to the operation indicated by the coordinate data (X2, Y2) (S7) in FIG. Assume that the screen data has been changed to screen data X (S11) due to addition or deletion. The portable terminal 2 notifies the next coordinate data (X3, Y3) to the remote screen based on the screen data X. However, since the screen data X and the coordinate data (X3, Y3) do not correspond, the MFP 1 Transition to a different screen, or the screen cannot be changed because the screen is different, and as a result, the process cannot proceed to the initial screen. For this reason, the user of the mobile terminal 2 cannot operate the MFP 1 from the mobile terminal 2.

  In FIG. 7, after the initial screen is registered in the mobile terminal 2, a part of the operation screen of the MFP 1 is deleted or added, so that the initial screen when connected to the mobile terminal 2 is registered in the mobile terminal 2. This shows an example of a case where the current screen transition method of the MFP 1 up to the initial screen is different from the screen transition method when the initial screen is registered in the mobile terminal 2.

  In this example, the screen is deleted, and the screen transition method at the time of registering the initial screen transitions to the screen D2 when the destination search key 131a is pressed on the screen D1, and selects the input area on the screen D2. When the search key 131b is pressed, the screen transitions to the screen D3, and when the search word is input and the search execution key 131c is pressed, the screen transitions to the screen D4 indicating the search result.

  On the other hand, changes have been made to improve operability, and the current screen transition method transitions to screen D3 when destination search key 131a is pressed on screen D1, and executes search by entering a search word. When the key 131c is pressed, the screen is changed to a screen D4 showing the search result, and the screen D2 is deleted, but the final screen is the same.

  FIG. 8 shows an example in which a screen is added. In this example, when the initial screen is registered, as shown in FIG. 5A, a screen transition method is set in which a transition to the screen D12 is made when the key A is operated on a certain screen D11. On the other hand, as a result of the addition of the function, as shown in FIG. 5B, when the key A is operated on a certain screen D11, the screen transitions to the screen D13 having the key B and the key C. When the key is pressed, the screen transitions to the screen D12, and when the key C is pressed, the screen transitions to the screen D14. That is, the screen D13 is added between the screen D11 and the screen D12, but the final screen D12 is the same as before the addition.

  Such deletion or addition of the screen is performed, for example, when the firmware is upgraded to add a function of the MFP 1.

  In this embodiment, when the screen transition method at the time of registration of the initial screen differs from the current screen transition method, the MFP 1 corrects the operation data automatically notified from the portable terminal 2. Finally, screen data corresponding to the initial screen displayed on the mobile terminal 2 is properly transmitted to the mobile terminal 2.

  FIG. 9 is a table in which transition destination information is collected for each screen held by the MFP 1 in order to correct the screen data, and this table is stored in the fixed storage device 110 or the like of the MFP 1 as a database. It is remembered.

  In this table, the firmware version as device information on which the MFP 1 is currently operating (indicated as Ver in FIG. 9 and hereinafter referred to as a new version) and a past version (hereinafter also referred to as an old version). The pattern is managed. Specifically, the screen number indicating the screen (circle number 1), the screen number of the transition destination that transitions when each transition key 1 to 4 (circle number 2) is pressed on the screen of that number, and the next Transition destination screen numbers are stored in association with the new version and the old version, respectively. The screen numbers indicated by the circle numbers 3 and 4 indicate the transition destination in the new version and the next transition destination screen, and the screen numbers indicated by the circle numbers 5 and 6 indicate the transition destination and the next in the old version. The transition destination screen is shown.

  When the mobile terminal 2 is connected to the MFP 1, the mobile terminal 2 notifies the MFP 1 of version information (shown in FIG. 5) that is device information at the time of registration of the initial screen. Further, the MFP 1 determines which transition key on which screen is pressed based on the coordinate data that is the operation data notified from the portable terminal 2. At this time, the MFP 1 compares the transition destination screen numbers (numbers 3 and 5) of the new version and the old version. If the screen numbers are the same when compared, it is determined that there is no change in the transition destination screen, and the screen data of the new version transition destination screen is generated and transmitted to the mobile terminal 2.

  If the screen numbers are different, determination is made including the next transition destination screen (circle numbers 4 and 6). If the transition destination screen of the new version (circle numeral 3) and the next transition destination screen of the old version (circle numeral 6) are the same, it is determined that the screen has been deleted, and the MFP 1 generates a dummy screen. The screen data is transmitted to the portable terminal 2. The portable terminal 2 notifies the MFP 1 of coordinate data registered for the remote screen of the dummy screen. Receiving this, the MFP 1 ignores the coordinate data for the dummy screen and transmits the screen data of the circled number 6 to the portable terminal 2.

  For example, the transition destination screen when the transition key 2 is operated on the screen with the screen number 1001 is the screen with the screen number 10105 in the new version, and the screen with the screen number 10100 in the old version, and they are different. Therefore, when the transition destination screen of the new version (circle numeral 3) and the next transition destination screen of the old version (circle numeral 6) are compared, both are the same in the screen of the screen number 10105. In this case, it is determined that the screen with the screen number 10100 in the old version has been deleted, and the screen data of the screen with the screen number 10105 is transmitted to the mobile terminal 2 through transmission of the dummy screen data.

  The new version and the old version have different transition destination screen numbers (circle numbers 3 and 5), and the new version next transition destination screen (circle number 4) and the old version transition destination screen (circle) If the numbers 5) are the same, it is determined that the screen with the screen number 3 is added in the new version. For example, when the transition key 1 is pressed for the screen number 2001, the screen number transitions to two screen numbers 20011 and 20050, which is different from the transition destination in the old version. Therefore, when the transition destination screen of the old version (circle number 5) is compared with the next transition destination screen of the new version (circle number 4), both are the same with screen number 20005, so a new screen with screen number 20011 is added. Judge that it has been.

  In this case, for the screen transition of the screen number 2001 → 2011 → 2005, the coordinate data notified from the mobile terminal 2 is not used, and the screen data of the screen with the screen number 2005 is transmitted to the mobile terminal 2.

  In this way, each time coordinate data as operation data is notified from the mobile terminal 2, the initial screen of the mobile terminal 2 is displayed by making the above-described determination regarding the transition destination based on the transition destination information in Table 9. Screen data to be transmitted can be correctly transmitted to the mobile terminal 2, and the same remote screen as the initial screen can be displayed on the mobile terminal 2 based on the screen data.

  The database shown in Table 9 may be specified in a form that covers all screens, or only specifies difference information for screens that are added or deleted to reduce the amount of data. It may be.

  The database shown in FIG. 9 exemplifies the case where the transition destination screen differs depending on the firmware version of the MFP 1, but depending on the product of the MFP, the MFP when the initial screen is registered in the mobile terminal 2 may actually be Unlike the MFP to be operated, the screen may be added or deleted in the MFP that is actually operated in response to the screen transition in the MFP when the initial screen is registered. In this case, the portable terminal 2 acquires the product information of the MFP 1 when registering the initial screen, and registers the acquired product information as device information. On the other hand, the MFP 1 maintains a database that summarizes transition destination information for each product of the MFP, and checks the transition destination screen based on the product information and coordinate data transmitted from the mobile terminal 2 when connected to the mobile terminal 2. Processing may be performed in the same manner as in the case of using the 9 database.

  Even in the same MFP, when the screen size of the display unit 131 is different due to replacement of the operation panel 130 or the like, if the layout of keys or the like is changed, the transition destination screen is different. Sometimes. For example, as shown in FIG. 10A, when the screen size of the display unit 131 is large, a large number of keys 131a can be displayed side by side at the lower end, but when the screen size is small as shown in FIG. The number of keys 131a displayed at the lower end also decreases.

  Accordingly, when the initial screen is registered in the portable terminal 2 when the screen size is the same as that shown in FIG. 5A, and the display is changed to the small-size screen display unit 131 as shown in FIG. And the coordinate data from the portable terminal 2 do not correspond to each other, and the correct transition to the initial screen cannot be performed.

  Therefore, in this case, when the initial screen is registered in the portable terminal 2, the screen size information of the display unit 131 of the MFP 1 is acquired and registered in the portable terminal 2 as device information. On the other hand, the MFP 1 holds a database that summarizes transition destination information for each size of the display unit 131, and displays a transition destination screen based on screen size information and coordinate data transmitted from the mobile terminal 2 when connected to the mobile terminal 2. In the same manner as in the case of using the database shown in FIG.

  There is a large difference in screen hierarchy between the way of the MFP 1 screen transition at the time of registration of the initial screen of the mobile terminal 2 and the way of the current screen transition of the MFP 1, and the coordinate data from the mobile terminal 2 cannot be corrected. In this case, the MFP 1 transmits a message to that effect to the portable terminal 2, and the portable terminal 2 displays the message M on the touch panel 205 as shown in FIG. As a result, the user can take necessary measures.

  FIG. 12 is a flowchart for explaining the operation of the MFP 1. This operation is executed by the CPU 101 of the MFP 1 operating according to an operation program such as the ROM 102.

  In step S01, the mobile terminal 2 is connected. At this time, device information such as version information is acquired from the portable terminal 2. Screen data is transmitted to the portable terminal 2 in step S02, and coordinate data from the portable terminal 2 is received in step S03.

  Next, in step S04, the current transition destination screen and the transition destination screen at the time of initial screen registration differ based on the received coordinate data, the device information acquired from the portable terminal 2, and the transition destination information shown in FIG. Determine if you are. If not different (NO in step S04), the process proceeds to step S05. If they are different (YES in step S04), the screen data corresponding to the coordinate data is corrected in step S07, and then the process proceeds to step S05.

  In step S05, it is determined whether or not it is the final screen of the transition destination. If it is not the final screen of the transition destination (NO in step S05), the process returns to step S02. If it is the final screen of the transition destination (YES in step S05), the screen data is transmitted to the portable terminal 2 in step S06.

  Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment. For example, in the database of transition destination information in FIG. 9, the transition destination is indicated by the screen number, but if the coordinate data is the same as the coordinate data notified from the mobile terminal 2, the transition destination screen is the same and must be different. For example, it may be determined that the transition destination screen is different.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 100 Control part 101 CPU
131 Display Unit 2 Portable Terminal Device 201 CPU
202 ROM
203 RAM
205 Display Unit 206 Wireless LAN Interface 207 Short-range Wireless Interface

Claims (12)

A remote panel system comprising an image processing device and a portable terminal device that can be connected to each other,
The image processing apparatus includes:
Display means having a touch panel;
Transmitting means for transmitting screen data of the operation screen of the own device displayed on the display means to the mobile terminal device;
Receiving means for receiving operation data transmitted from the mobile terminal device when a user operates the operation screen displayed as a remote screen on the mobile terminal device receiving the screen data;
A process execution means for executing a process including a screen update process instructed by the operation data based on the operation data received by the receiving means;
With
The portable terminal device
Display means having a touch panel;
Display control means for displaying the remote screen on the display means based on the screen data from the image processing device;
Register any operation screen of the image processing apparatus displayed on the display means by the display control means as an initial screen, and from the initial screen when connected to the image processing apparatus to the registered initial screen Registering operation data for each screen necessary for screen transition, and further registering device information of the image processing apparatus at the time of registration of the initial screen,
The device information is transmitted to the image processing device, and the registered operation data is sequentially input to each screen from the connection screen transmitted from the image processing device to the initial screen. Transmitting means for transmitting to the image processing apparatus;
With
The image processing apparatus further includes:
Storage means for storing transition destination information indicating a transition destination screen corresponding to the operation data generated for each model information;
When the operation data transmitted from the mobile terminal device is received for each screen before the transition from the screen at the time of connection to the initial screen, the mobile Determining means for determining whether the transition destination screen based on the device information transmitted from the terminal device is the same as the transition destination screen based on the current device information based on the transition destination information stored in the storage means; ,
If it is determined by the determination means to be the same, the screen data of the transition destination screen is transmitted to the portable terminal device, and if it is determined to be different, the screen transition destination is corrected based on the transition destination information. , Correction means for transmitting the corrected transition destination screen data to the mobile terminal device;
A remote panel system characterized by comprising:   When the determination unit of the image processing apparatus determines that the transition destination screen based on the device information transmitted from the mobile terminal device is different from the transition destination screen based on the current device information, the image processing device transmits from the mobile terminal device. Determining whether the transition destination screen based on the device information has been deleted in the latest device information or whether a new transition destination screen has been added as the transition destination screen based on the current device information, and the image processing The remote panel system according to claim 1, wherein the correction unit of the apparatus performs correction according to deletion or addition of a screen.   When the determination unit of the image processing apparatus determines that the transition destination screen based on the device information transmitted from the portable terminal device is deleted in the current device information, the correction unit 3. The remote panel system according to claim 2, wherein the correction is performed by ignoring the operation data from, and transmitting screen data of a dummy screen to the mobile terminal device.   When the determination unit of the image processing apparatus determines that a new transition destination screen is added as a transition destination screen based on the current device information, the correction unit is configured to display the screen of the added screen. The remote panel system according to claim 2, wherein the correction is performed by transmitting data to the mobile terminal device.   The remote panel system according to claim 1, wherein the device information is firmware version information of the image processing apparatus.   The remote panel system according to claim 1, wherein the device information is product information of the image processing apparatus.   The remote panel system according to claim 1, wherein the device information is screen size information of display means of the image processing apparatus.   The remote panel system according to claim 1, wherein screen coordinate data is used as the transition destination information.   The remote panel system according to claim 1, wherein a screen number is used as the transition destination information.   The remote panel system according to claim 1, wherein when the correction cannot be performed, the correction unit of the image processing apparatus notifies the mobile terminal device to that effect. A display method of a remote screen executed by a remote panel system including an image processing device and a portable terminal device that can be connected to each other,
The image processing apparatus includes:
A transmission step of transmitting screen data of an operation screen of the own device displayed on a display unit having a touch panel to the mobile terminal device;
A receiving step of receiving operation data transmitted from the mobile terminal device when a user operates the operation screen displayed as a remote screen on the mobile terminal device that has received the screen data;
A process execution step for executing a process including a screen update process instructed by the operation data based on the operation data received by the reception step;
Run
The portable terminal device
A display control step for displaying the remote screen on a display means having a touch panel based on the screen data from the image processing device;
The operation screen of any of the image processing devices displayed on the display means by the display control step is registered as an initial screen, and the screen transition from the initial screen to the initial screen when connected to the image processing device A registration step of registering operation data for each screen necessary for the registration, and further registering device information of the image processing apparatus at the time of registration of the initial screen;
The device information is transmitted to the image processing device, and the registered operation data is sequentially input to each screen from the connection screen transmitted from the image processing device to the initial screen. A transmission step for transmitting to the image processing device;
Run
The image processing apparatus further includes:
Storing transition destination information indicating a transition destination screen corresponding to the operation data generated for each model information in a storage unit;
When the operation data transmitted from the mobile terminal device is received for each screen before the transition from the screen at the time of connection to the initial screen, the mobile A determination step of determining whether the transition destination screen based on the device information transmitted from the terminal device is the same as the transition destination screen based on the current device information based on the transition destination information stored in the storage unit; ,
When it is determined by the determination step that it is the same, the screen data of the transition destination screen is transmitted to the mobile terminal device, and when it is determined that it is different, the screen transition destination is corrected based on the transition destination information. A correction step of transmitting the corrected transition destination screen data to the mobile terminal device;
A method for displaying a remote screen, wherein   The image processing apparatus used for the remote panel system in any one of Claims 1-10.

Images