JP6123602B2 - Image processing apparatus, remote control method, and remote control program - Google Patents

Description

  The present invention relates to an image processing device, a remote control method, and a remote control program, and more particularly, to an image processing device remotely controlled by a remote operation device, a remote control method and a remote control program executed by the image processing device.

  A technique for remotely operating a multifunction peripheral (MFP) with a portable information device such as a smartphone is known. For example, in JP 2012-93940 A, the operated terminal receives from the mobile terminal a display screen control unit that creates a mobile terminal screen in which the operated terminal screen is modified based on the display attribute information of the mobile terminal. The mobile terminal includes an encoding unit that encodes the mobile terminal screen using the encoding method and an operation processing unit that executes an operation specified by position information received from the mobile terminal. When a decoding unit that decodes the terminal screen and outputs it to the touch screen and a touch operation on the touch screen displaying the mobile terminal screen is detected, position information indicating the touch position on the mobile terminal screen is displayed as its own display attribute information. And a position information correction unit for correcting the position information on the operated terminal screen based on the display attribute information of the operated terminal received from the operated terminal. There has been described.

However, since the MFP can be used by a plurality of people, the MFP must encode and transmit the portable terminal screen to each of the plurality of portable information devices when remotely operated by the plurality of portable terminals. For this reason, the MFP has a problem that the load increases as the number of portable terminals increases.
JP 2012-93940 A

  The present invention has been made to solve the above-described problems, and one object of the present invention is to provide an image processing apparatus that prevents an increase in load due to an increase in the number of remotely operated remote control apparatuses. Is to provide.

  Another object of the present invention is to provide a remote control method and a remote control program that prevent an increase in the load on the image processing apparatus due to an increase in the number of remote control devices that are remotely controlled.

  In order to achieve the above-described object, according to one aspect of the present invention, an image processing apparatus is an image processing apparatus that can be remotely controlled by a plurality of remote operation apparatuses, and any of the plurality of remote operation apparatuses. In response to the reception of the screen request from the screen, the operation screen specified by the screen request is transmitted to the pointing device that has transmitted the screen request, and the operation screen is transmitted by the screen transmission unit. The screen identification information for identifying the operation screen and the association means for associating the device identification information for identifying the instruction device, and the screen transmission means is the device identification information of the instruction device. If different device identification information is associated with the screen identification information of the operation screen identified by the screen request, the device identification associated with the screen identification information of the operation screen identified by the screen request Requesting device determining means for determining the remote operation device specified in the report as the requesting device, and when the requesting device is determined, the operation screen specified in the screen request is specified in the screen request without being transmitted to the instruction device. A requesting device that requests the requesting device specified by the device identification information associated with the screen identification information of the operation screen to be transmitted to the instruction device of the operating screen specified by the screen request, and an association unit Associates screen identification information for identifying the operation screen with device identification information for identifying the pointing device in response to a request from the requesting device to the requesting device.

  According to this aspect, in response to receiving a screen request from any of the plurality of remote operation devices, the operation screen is transmitted to the instruction device that has transmitted the screen request, and the screen identification information of the operation screen, When the device identification information associated with the device identification information of the pointing device is associated with the screen identification information of the operation screen, the device identification associated with the screen identification information of the operation screen. The remote operation device specified by the information is determined as the request device, and the request device is requested to transmit the operation screen specified by the screen request to the instruction device. For this reason, since the requesting device transmits the operation screen to the instruction device, there is no need to execute processing for transmitting the operation screen to the instruction device. As a result, it is possible to provide an image processing apparatus that prevents an increase in load due to an increase in the number of remotely operated remote control apparatuses.

  Preferably, the associating unit further associates the time when the operation screen specified by the screen identification information is transmitted to the remote operation device of the device identification information associated with the screen identification information in association with the screen identification information. The screen transmission means, when the operation screen specified by the screen request includes a variable region that changes with time, the time associated with the device identification information of the requesting device, the time when the screen request is received, And determining means for determining whether or not the variable regions are the same, and when the request means determines that the variable regions are not the same, based on the time when the screen request is received, A variable region transmission device determining means for determining a remote operation device specified by the device identification information associated with the screen identification information of the operation screen including the region as the variable region transmission device; The non-moving area other than the changing area in the operation screen identified by the non-moving area transmitting request means for requesting transmission to the indicating apparatus, and the determined changing area transmitting apparatus is requested to transmit to the changing area indicating apparatus. Fluctuating area transmission requesting means.

  According to this aspect, when the operation screen specified by the screen request includes a variable region that changes with time, it is determined whether or not the variable region is the same. The remote operation device specified by the device identification information associated with the screen identification information of the operation screen including the same region is determined as the variable region transmission device, and transmitted to the requesting device in the non-moving region other than the variation region And requests the variable region transmitter to transmit the variable region to the pointing device. For this reason, even if a part of the screen transmitted to the requesting device changes with time, the operation screen can be displayed on the instruction device.

  Preferably, the screen transmission means is generated with the fluctuation area generation means for generating the fluctuation area when the fluctuation area transmission apparatus is not determined by the fluctuation area transmission apparatus determination means when it is determined that the fluctuation areas are not the same. And a change area transmitting means for transmitting the change area to the pointing device.

  According to this aspect, when the variable regions are not the same and the variable region transmission device is not determined, the variable region is generated and the variable region is transmitted to the pointing device. For this reason, since it is sufficient to generate only the variable region, it is possible to reduce the processing load as compared with the case where the entire operation screen is generated.

  Preferably, the screen transmission means transmits a signal indicating that there is no variable area in the instruction device when the variable area transmitter is not determined by the variable area transmitter determining means when it is determined that the variable areas are not the same. Non-existence notifying means to be included.

  According to this aspect, when the variable regions are not the same and the variable region transmitter is not determined, a signal indicating that there is no variable region is transmitted to the instruction device. For this reason, since the fluctuation region is not generated, the processing load can be reduced.

  Preferably, when there are a plurality of device identification information associated with the screen identification information of the operation screen specified by the screen request, the screen transmission unit includes a plurality of remote operation devices respectively corresponding to the plurality of device identification information. Requesting device determining means for determining, as a requesting device, a remote control device whose resolution for displaying an image approximates that of the pointing device.

  According to this aspect, since the remote control device whose resolution approximates that of the pointing device is determined as the requesting device, it is possible to reduce the processing load for converting the resolution in the pointing device.

  Preferably, when there are a plurality of pieces of device identification information associated with the screen identification information of the operation screen specified by the screen request, the requesting device determination unit includes a plurality of remote operation devices respectively corresponding to the plurality of device identification information. Of these, the remote control device with the fastest processing speed is determined as the requesting device.

  According to this aspect, since the remote operation device having the fastest processing speed is determined as the requesting device, the time for transmitting the operation screen by the requesting device can be shortened as much as possible.

  Preferably, the apparatus further comprises position detecting means for detecting the position of each of the plurality of remote control devices, and the requesting device determining means has a plurality of device identification information associated with the screen identification information of the operation screen specified by the screen request. In this case, the remote control device having the shortest distance from the pointing device among the plurality of remote control devices respectively corresponding to the plurality of device identification information is determined as the requesting device.

  According to this aspect, since the remote control device having the shortest distance to the instruction device is determined as the request device, it may be possible to shorten the time for transmitting the operation screen from the request device to the instruction device.

  Preferably, the apparatus further comprises path establishment means for establishing a communication path with the remote operation apparatus in response to a connection request received from any of the plurality of remote operation apparatuses, and the requesting apparatus determination means includes communication On the condition that the number of remote operation devices with established routes is equal to or greater than a predetermined number, the transmission of the operation screen specified by the screen request to the instruction device is requested.

  According to this aspect, the transmission of the operation screen to the instruction device is requested on the condition that the number of remote operation devices with established communication paths is equal to or greater than a predetermined number. For this reason, since the probability of determining a request apparatus becomes high, an operation screen can be efficiently transmitted to an instruction | indication apparatus.

  According to another aspect of the present invention, a remote control method is a remote control method executed by an image processing device that can be remotely controlled by a plurality of remote control devices, and any of the plurality of remote control devices. In response to receiving the screen request from the screen, the operation screen specified in the screen request is transmitted to the instruction device that has transmitted the screen request, and the operation screen is transmitted in the screen transmission step. The screen identification information for identifying the operation screen and the associating step for associating the device identification information for identifying the pointing device. The screen transmission step is the device identification information of the pointing device. When different device identification information is associated with the screen identification information of the operation screen specified by the screen request, it is associated with the screen identification information of the operation screen identified by the screen request. A request device determining step for determining the remote operation device specified by the device identification information to be a request device; and when the request device is determined, the screen is transmitted without transmitting the operation screen specified by the screen request to the instruction device. Requesting the requesting device identified by the device identification information associated with the screen identification information of the operation screen identified by the request to send the operation screen identified by the screen request to the instruction device. The associating step includes a step of associating screen identification information for identifying the operation screen with device identification information for identifying the pointing device in response to a request from the requesting device in the requesting step.

  According to this aspect, it is possible to provide a remote control method that prevents an increase in the load on the image processing apparatus due to an increase in the number of remotely operated remote control apparatuses.

  According to another aspect of the present invention, the remote control program is a remote control program that is executed by a computer that controls an image processing apparatus that can be remotely controlled by a plurality of remote control devices. In response to receiving a screen request from any of the operation devices, the operation screen specified in the screen request is transmitted to the instruction device that has transmitted the screen request, and the operation screen is displayed in the screen transmission step. In response to the transmission, the computer executes a screen identification information for identifying the operation screen and an association step for associating the device identification information for identifying the pointing device. When device identification information different from the device identification information of the device is associated with the screen identification information of the operation screen specified by the screen request, A request device determining step for determining a remote operation device specified by the device identification information associated with the screen identification information of the operation screen specified by the screen request as the request device, and if the request device is determined, the screen request The operation specified by the screen request to the requesting device specified by the device identification information associated with the screen identification information of the operation screen specified by the screen request without transmitting the specified operation screen to the instruction device A request step for requesting transmission of the screen to the instruction device, and the associating step includes screen identification information for identifying an operation screen in response to a request from the request device in the request step, and an instruction device Associating with device identification information for identifying the device.

  According to this aspect, it is possible to provide a remote control program that prevents an increase in the load on the image processing apparatus due to an increase in the number of remotely operated remote control apparatuses.

1 is a diagram showing an overview of a printing system according to one embodiment of the present invention. 1 is a perspective view showing an appearance of an MFP. 2 is a block diagram illustrating an outline of a hardware configuration of an MFP. FIG. It is a block diagram which shows the outline | summary of the hardware constitutions of a portable information device. 2 is a block diagram illustrating an example of functions of a CPU provided in an MFP. FIG. It is a figure which shows an example of the operation screen containing a consumable part status display screen as a fluctuation | variation area | region. It is a figure which shows an example of the operation screen containing a job history screen as a fluctuation | variation area | region. It is a block diagram which shows an example of the outline | summary of the function of CPU with which the portable information device in this Embodiment is provided. It is a flowchart which shows an example of the flow of a remote control process. It is a 1st flowchart which shows an example of the flow of an operation screen transmission process. It is a 2nd flowchart which shows an example of the flow of an operation screen transmission process. It is a flowchart which shows an example of the flow of a remote operation process. It is a figure which shows an example of the relevant information in the 1st step in a 1st specific example. It is a figure which shows an example of the flow of the information transmitted / received between the portable information device and MFP in the 1st specific example. It is a figure which shows an example of the relevant information in the 2nd step in a 2nd specific example. It is a figure which shows an example of the flow of the information transmitted / received between the portable information device and MFP in the 2nd specific example. It is a figure which shows an example of the relevant information in the 2nd step in a 3rd specific example. It is a figure which shows an example of the flow of the information transmitted / received between the portable information device and MFP in the 3rd specific example.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

  FIG. 1 is a diagram showing an overall outline of a printing system according to one embodiment of the present invention. Referring to FIG. 1, a print system 1 includes a multifunction peripheral (hereinafter referred to as “MFP”) 100, a personal computer (hereinafter referred to as “PC”) 300, a wireless station 5, and a portable device that function as an image processing apparatus. Information devices 200, 200A, and 200B. An MFP (Multi Function Peripheral) 100, the PC 300, and the wireless station 5 are connected to the network 3. The portable information devices 200, 200 </ b> A, 200 </ b> B are connected to the network 3 through the wireless station 5.

  The network 3 is a local area network (LAN), and the connection form may be wired or wireless. The network 3 is not limited to a LAN, and may be a wide area network (WAN), a public switched telephone network (PSTN), the Internet, or the like.

  The PC 300 is a general computer. The PC 300 is installed with a printer driver program corresponding to the MFP 100, and by executing the printer driver program, the MFP 100 can be controlled to cause the MFP 100 to execute image forming processing, document reading processing, and the like. .

  The portable information devices 200, 200A, and 200B are computers that are carried and used by users, such as smartphones and PDAs (Personal Digital Assistants). Since the portable information devices 200, 200A, and 200B have the same hardware configuration and functions, the portable information device 200 will be described as an example unless otherwise specified. Here, the portable information device 200 is a smartphone, and has a wireless LAN function and a call function. For this reason, the portable information device 200 is connected to the cellular phone network by wirelessly communicating with the cellular phone base station, and can make a call.

  In print system 1 according to the present embodiment, portable information device 200 is installed with a remote operation program for remotely controlling MFP 100 and functions as a remote operation device for remotely controlling MFP 100. On the other hand, MFP 100 is installed with a remote control program that transmits a remote operation screen to portable information device 200 and receives a remote control command from the portable information device, and executes processing in accordance with the received remote control command. The remote operation command includes, for example, position information indicating a position in the operation screen received from MFP 100.

  Further, the remote control program may be a Web server program that causes MFP 100 to function as a Web server and transmits an operation screen as a Web page described in a markup language. In this case, the remote operation program corresponding to the remote control program can be, for example, a browsing program that downloads and displays a Web page from MFP 100 that is a Web server.

  In the present embodiment, a case where a remote operation program for remotely controlling MFP 100 is installed in portable information device 200 will be described as an example. However, a remote operation program for remotely controlling MFP 100 is installed in PC 300. May be. In this case, the user can remotely control MFP 100 using PC 300 as in the case of remotely controlling MFP 100 using portable information device 200.

  FIG. 2 is a perspective view showing the appearance of the MFP. FIG. 3 is a block diagram showing an outline of the hardware configuration of the MFP. 2 and 3, MFP 100 includes a main circuit 110, a document reading unit 130 for reading a document, an automatic document transport device 120 for transporting a document to document reading unit 130, and a document reading unit. An image is formed by an image forming unit 140 for forming an image on a sheet or the like based on image data output by the manuscript 130 reading and outputting a document, a paper feeding unit 150 for supplying the image to the image forming unit 140, and an image. A post-processing unit 155 that processes paper and an operation panel 160 as a user interface are included.

  The post-processing unit 155 executes a sorting process for sorting and discharging one or more sheets on which images are formed by the image forming unit 140, a punching process for punch hole processing, and a staple process for driving staple needles.

  The main circuit 110 includes a CPU 111, a communication interface (I / F) unit 112, a ROM 113, a RAM 114, a hard disk drive (HDD) 115 as a mass storage device, a facsimile unit 116, and a CD-ROM 118. External storage device 117. CPU 111 is connected to automatic document feeder 120, document reading unit 130, image forming unit 140, paper feeding unit 150, post-processing unit 155, and operation panel 160, and controls the entire MFP 100.

  The ROM 113 stores a program executed by the CPU 111 or data necessary for executing the program. The RAM 114 is used as a work area when the CPU 111 executes a program. The RAM 114 temporarily stores read data (image data) continuously sent from the document reading unit 130.

  Operation panel 160 is provided on the upper surface of MFP 100 and includes a display unit 161 and an operation unit 163. The display unit 161 is a display device such as a liquid crystal display (LCD) or an organic ELD (Electro-Luminescence Display), and displays an operation screen including an instruction menu for the user, information about acquired image data, and the like. The operation unit 163 includes a hard key unit 167 including a plurality of keys, and accepts input of various instructions, data such as characters and numbers by user operations corresponding to the keys. The operation unit 163 further includes a touch panel 165 provided on the display unit 161.

  Communication I / F unit 112 is an interface for connecting MFP 100 to network 3. The CPU 111 communicates with the PC 300 or the portable information devices 200, 200A, and 200B via the communication I / F unit 112, and transmits and receives data. The communication I / F unit 112 can communicate with a computer connected to the Internet via the network 3.

  The facsimile unit 116 is connected to the public switched telephone network (PSTN) and transmits facsimile data to the PSTN or receives facsimile data from the PSTN. The facsimile unit 116 stores the received facsimile data in the HDD 115 or outputs it to the image forming unit 140. The image forming unit 140 prints the facsimile data received by the facsimile unit 116 on a sheet. Further, the facsimile unit 116 converts the data stored in the HDD 115 into facsimile data, and transmits the facsimile data to a facsimile machine connected to the PSTN.

  The external storage device 117 is loaded with a CD-ROM (Compact Disk ROM) 118. The CPU 111 can access the CD-ROM 118 via the external storage device 117. The CPU 111 loads the program recorded on the CD-ROM 118 attached to the external storage device 117 to the RAM 114 and executes it. The medium for storing the program executed by the CPU 111 is not limited to the CD-ROM 118, but an optical disc (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile Disc)), IC card, optical card, It may be a semiconductor memory such as a mask ROM, an EPROM (Erasable Programmable ROM), or an EEPROM (Electrically EPROM).

  Further, the program executed by the CPU 111 is not limited to the program recorded on the CD-ROM 118, and the program stored in the HDD 115 may be loaded into the RAM 114 and executed. In this case, another computer connected to the network 3 may rewrite the program stored in the HDD 115 of the MFP 100 or may add a new program and write it. Further, MFP 100 may download a program from another computer connected to network 3 and store the program in HDD 115. The program here includes not only a program directly executable by the CPU 111 but also a source program, a compressed program, an encrypted program, and the like.

  FIG. 4 is a block diagram showing an outline of the hardware configuration of the portable information device. Referring to FIG. 4, portable information device 200 in the present embodiment includes a CPU 201 for controlling the entire portable information device 200, a camera 202, a flash memory 203 for storing data in a nonvolatile manner, and a calling unit. 205 includes a wireless communication unit 204 connected to 205, a display unit 206 that displays information, an operation unit 207 that accepts user operations, a wireless LAN I / F 208, a position detection unit 209, and an external storage device 211. .

  The display unit 206 is a display device such as a liquid crystal display (LCD) or an organic ELD, and displays an instruction menu for a user, information about acquired image data, and the like. The operation unit 207 includes a main key 207A and a touch panel 207B. When the user instructs the display surface of the display unit 206, the operation unit 207 outputs the position of the display surface detected by the touch panel 207B to the CPU 201. Based on the position detected by the touch panel 207B, the CPU 201 detects a position designated by the user in the screen displayed on the display unit 206. Based on the screen displayed on the display unit 206 and the position detected by the touch panel 207 </ b> B, the CPU 201 accepts input of various instructions, data such as characters and numbers by user operations. For example, when a screen including a numeric keypad image is displayed on the display unit 206, a number corresponding to the key displayed at the position detected by the touch panel 207B is received.

  The camera 202 includes a lens and a photoelectric conversion element. The light focused by the lens is imaged on the photoelectric conversion element. The photoelectric conversion element photoelectrically converts the received light and outputs image data to the CPU 201. The photoelectric conversion element is a CMOS (Complementary Metal Oxide Semiconductor) sensor, a CCD (Charge Coupled Device) sensor, or the like. The camera 202 is arranged at a position where the imaging range includes a user who views the display unit 206. Specifically, the camera 202 is arranged such that its optical axis is substantially parallel to the normal line of the display surface of the display unit 206 and the imaging direction is the same as the direction of the display surface. In other words, when the user visually recognizes the display unit 206, the camera 202 includes the face portion of the user in the imaging range.

  The wireless communication unit 204 performs wireless communication with a mobile phone base station connected to a telephone communication network. The wireless communication unit 204 connects the portable information device 200 to a telephone communication network and enables a call using the call unit 205. The radio communication unit 204 decodes a voice signal obtained by demodulating the radio signal received from the mobile phone base station and outputs the decoded signal to the call unit 205. Further, the wireless communication unit 204 encodes the voice input from the call unit 205 and transmits it to the mobile phone base station. The call unit 205 includes a microphone and a speaker, and outputs the sound input from the wireless communication unit 204 from the speaker, and outputs the sound input from the microphone to the wireless communication unit 204. Further, the wireless communication unit 204 is controlled by the CPU 201, connects the portable information device 200 to an electronic mail server, and transmits and receives electronic mail.

  The wireless LAN I / F 208 is an interface for communicating with the wireless station 5 and connecting the portable information device 200 to the network 3. By registering the IP (Internet Protocol) addresses of PC 300 and MFP 100 in portable information device 200, portable information device 200 can communicate with PC 300 and MFP 100, and data can be transmitted and received. In the present embodiment, a case where portable information device 200 communicates with PC 300 and MFP 100 using wireless LAN I / F 208 will be described as an example, but communication may be performed using another communication method. . Specifically, when portable information device 200, PC 300, and MFP 100 are equipped with a short-range wireless device such as Bluetooth (registered trademark), portable information device 200 has one-to-one correspondence with either PC 300 or MFP 100. You may make it communicate by.

  The flash memory 203 stores a program executed by the CPU 201 or data necessary for executing the program. The CPU 201 loads the program recorded in the flash memory 203 to the RAM included in the CPU 201 and executes it.

  The position detection unit 209 detects the current position of the portable information device 200. Specifically, the position detection unit 209 is a global positioning system receiver, and measures the current position by receiving radio waves from a plurality of GPS satellites. The position detection unit 209 outputs a value indicating the measured current position, for example, latitude and longitude to the CPU 201.

  The external storage device 211 is detachable from the portable information device 200, and a CD-ROM 210A storing a remote operation program can be attached thereto. The CPU 201 can access the CD-ROM 210 </ b> A via the external storage device 211. The CPU 201 can load a remote operation program recorded on the CD-ROM 210A attached to the external storage device 211 into a RAM included in the CPU 201 and execute it.

  The program recorded in the flash memory 203 or the CD-ROM 210A has been described as a program executed by the CPU 201. However, a program in which another computer connected to the network 3 rewrites the program stored in the flash memory 203, Alternatively, it may be a new program written additionally. Further, the portable information device 200 may be a program downloaded from another computer connected to the network 3. The program here includes not only a program directly executable by the CPU 201 but also a source program, a compressed program, an encrypted program, and the like.

  The medium for storing the program executed by the CPU 201 is not limited to the CD-ROM 211A, and may be a semiconductor memory such as an optical disk (MO / MD / DVD), IC card, optical card, mask ROM, EPROM, or EEPROM. Good.

  FIG. 5 is a block diagram illustrating an example of functions of the CPU provided in the MFP. The functions shown in FIG. 5 are functions formed in CPU 111 when CPU 111 provided in MFP 100 executes a remote control program stored in ROM 113, HDD 115, and CD-ROM 118. Referring to FIG. 5, CPU 111 establishes a communication path establishing unit 51 for establishing a communication path with an external device, a screen request receiving unit 53 for receiving a screen request from the external device with which the communication path is established, and a screen. A screen transmission unit 55 that transmits the operation screen specified by the request, an association unit 57, an execution command reception unit 59 that receives an execution command from an external device with which a communication path has been established, and a process specification unit that specifies a process 61 and a remote control unit 63 that executes the specified processing.

  When the communication path establishing unit 51 controls the communication I / F unit 112 and receives a connection request from any of the portable information devices 200, 200A, and 200B, the communication path establishing unit 51 transmits the connection request among the portable information devices 200, 200A, and 200B. Establish a communication path with the connected device.

  The communication path establishment unit 51 can establish a communication path with one or more of the portable information devices 200, 200A, 200B, and communicates for each communication path using path identification information for identifying the communication path. Identify the other device. The communication path establishment unit 51 identifies device identification information for identifying a device that has established a communication route among the portable information devices 200, 200A, and 200B, and a route for identifying a communication route established between the devices. The route information associated with the identification information is stored in the RAM 114. Further, the communication path establishment unit 51 generates an establishment signal including a signal indicating that the communication path has been established and device identification information for identifying the device that has established the communication route among the portable information devices 200, 200A, and 200B. The data is output to the screen request receiving unit 53, the screen transmitting unit 55, and the execution command receiving unit 59.

  As the path identification information, for example, a session number for identifying a communication session can be used. The device identification information can be a network address assigned to the portable information device 200, 200A, 200B. The route identification information and the device identification information are not limited to these, and other information can be used as long as the communication route and the device with which the communication route is established can be identified.

  The communication path establishment unit 51 disconnects the communication path between the portable information devices 200, 200A, and 200B with the apparatus that has established the communication path. The communication path establishment unit 51 disconnects the communication path when receiving a disconnection request from a device that has established the communication path. The timing for disconnecting the communication path is not limited to this, and the communication path may be disconnected at various timings. For example, the communication path may be disconnected when data is not transmitted / received continuously from the communication path for a predetermined time or more. When the communication path establishment unit 51 disconnects the communication path with the apparatus that has established the communication path, the communication path establishment unit 51 deletes the path information including the path identification information of the disconnected communication path from the RAM 114. In addition, the communication path establishment unit 51 generates a disconnection signal including a signal indicating that the communication path has been disconnected and device identification information for identifying a device that has disconnected the communication path among the portable information devices 200, 200A, and 200B. And output to the screen request receiving unit 53, the screen transmitting unit 55, and the execution command receiving unit 59.

  Each of the screen request receiving unit 53, the screen transmitting unit 55, and the execution command receiving unit 59 has a communication path among the portable information devices 200, 200A, 200B based on the establishment signal and the disconnection signal input from the communication path establishment unit 51. Identify established devices.

  The screen request receiving unit 53 controls the communication I / F unit 112 to receive a screen request from a device with a communication path established among the portable information devices 200, 200A, and 200B. The screen request is a signal that requests transmission of an operation screen for remotely operating MFP 100. The screen request includes screen identification information that identifies one of the plurality of operation screens. When the screen request receiving unit 53 receives the screen request, the screen request receiving unit 53 outputs a transmission instruction including the screen identification information included in the screen request and the device identification information of the device that has transmitted the screen request to the screen transmission unit 55. Hereinafter, the device that has transmitted the screen request among the portable information devices 200, 200A, and 200B is referred to as an instruction device.

  The screen transmission unit 55 includes a request device determination unit 71, a real screen transmission unit 73, a determination unit 75, a request unit 77, a variable region generation unit 79, a variable region transmission unit 81, and a non-existence notification unit 83. ,including.

  First, a state in which a communication path is established with any one of the portable information devices 200, 200A, and 200B by the communication path establishing unit 51 will be described. Here, a case where a communication path is established with the portable information device 200 by the communication path establishing unit 51 will be described as an example.

  The associating unit 57 associates the device identification information of the device having the communication path established among the portable information devices 200, 200A, and 200B with the screen identification information of the operation screen displayed on the device. Specifically, the association information is stored in the RAM 114. Details of the association information will be described later, but include device identification information, screen identification information, and date / time information. Here, a communication path is established with the portable information device 200. However, before the operation screen is transmitted to the portable information device, the association unit 57 performs screen identification for the device identification information of the portable information device 200. Since no information is associated, no association information is stored in the RAM 114.

  The requesting device determination unit 71 refers to the association information stored in the RAM 114 in response to the transmission instruction input from the screen request reception unit 53, and differs from the device identification information included in the transmission instruction. It is determined whether or not the information is associated with the screen identification information included in the transmission instruction by the association unit 57. Here, since the association information is not stored in the RAM 114 by the association unit 57, the requesting device determination unit 71 determines that the association unit 57 does not associate and sends a transmission instruction input from the screen request reception unit 53. Is output to the real screen transmission unit 73.

  The real screen transmission unit 73 controls the communication I / F unit 112 in response to the transmission instruction input from the requesting device determination unit 71, and displays the operation screen specified by the screen identification information included in the transmission instruction. It transmits to the device specified by the device identification information included in the transmission instruction among the portable information devices 200, 200A, 200B. Further, the real screen transmission unit 73 outputs a set of device identification information, screen identification information, and date / time information indicating the date / time when the operation screen is transmitted, to the association unit 57.

  The associating unit 57 receives the device identification information, the screen identification information, and the date / time information from the real screen transmission unit 73 in response to the combination of the device identification information, the screen identification information, and the date / time information. Associate. Specifically, association information including device identification information, screen identification information, and date / time information is stored in the RAM 114.

  The execution command receiving unit 59 controls the communication I / F unit 112 to receive an execution command from a device that has established a communication path among the portable information devices 200, 200 </ b> A, and 200 </ b> B. The execution command is a signal for instructing MFP 100 to execute the process, and is a signal transmitted by the device that displays the operation screen among portable information devices 200, 200A, and 200B. The execution command defines a process to be executed by MFP 100 on the operation screen corresponding to the execution command. Here, the execution command is position information indicating a position in the operation screen. The execution command receiving unit 59 outputs a set of the received execution command and the device identification information of the device that has transmitted the execution command among the portable information devices 200, 200 </ b> A, 200 </ b> B to the process specifying unit 61.

  The process specifying unit 61 specifies a process in response to an input of a set of an execution command and device identification information from the execution command receiving unit 59. Specifically, the apparatus identification information input from the execution command receiving unit 59 and the screen identification information associated by the association unit 57 are acquired, and the acquired screen identification information and the execution command receiving unit 59 are input. A process is specified based on the execution command. The process specifying unit 61 specifies a button arranged at a position specified by the execution command in the operation screen specified by the screen identification information, and acquires a command assigned to the specified button.

  The command includes a screen transition command for displaying another operation screen. When the acquired command is a screen transition command, the process specifying unit 61 includes screen identification information of the operation screen after the transition specified by the screen transition command and device identification information of the device that has transmitted the execution command. The transmission instruction is output to the requesting device determination unit 71 of the screen transmission unit 55. In this case, the device that has transmitted the execution command is the pointing device. If the acquired command is a command other than the screen transition command, the process specifying unit 61 outputs the acquired command to the remote control unit 63.

  The remote control unit 63 executes the process specified by the command in response to the command input from the process specifying unit 61. The processing executed by the remote control unit 63 includes scanning processing for controlling the automatic document feeder 120 and the document reading unit 130 to read an image of the document, and controlling the image forming unit 140, the sheet feeding unit 150, and the post-processing unit 155. Image forming processing for forming an image on paper, data management processing for reading or writing data stored in HDD 115 or CD-ROM 118, facsimile processing for controlling facsimile unit 116 to transmit / receive facsimile data, communication I / F unit 112 Data transmission / reception processing for controlling data transmission / reception. The process executed by the remote control unit 63 includes a process obtained by combining the above two or more processes. For example, a copy process combining a scan process and an image forming process, a scan-to-transmit process that transmits image data obtained by reading a document image by combining a scan process and a data transmission process, and a document image combining a scan process and a data management process Includes a scan to BOX process for storing the image data read in the HDD 115.

  Next, a case where a screen request is received from the portable information device 200 in a state where the communication route establishment unit 51 establishes a communication route with each of the portable information devices 200, 200A, and 200B will be described. In this state, the associating unit 57 associates the device identification information of each of the portable information devices 200, 200A, and 200B, the screen identification information of the operation screen displayed on the device, and the date / time information. In other words, the association information for each portable information device 200, 200 </ b> A, 200 </ b> B is stored in the RAM 114 by the association unit 57.

  When the screen request reception unit 53 receives the screen request from the portable information device 200, the screen request reception unit 53 outputs a transmission instruction including the screen identification information included in the screen request and the device identification information of the portable information device 200 to the screen transmission unit 55. The portable information device 200 is an instruction device.

  The requesting device determining unit 71 refers to the association information stored in the RAM 114 in response to the transmission instruction input from the screen request receiving unit 53 or the process specifying unit 61, and is included in the transmission instruction. It is determined whether or not device identification information different from the device identification information 200 is associated with the screen identification information included in the transmission instruction by the association unit 57. The requesting device determination unit 71 extracts association information including screen identification information included in the transmission instruction from the association information stored in the RAM 114. If the requesting device determination unit 71 can extract the association information including the screen identification information included in the transmission instruction, the device identification information different from the device identification information of the portable information device 200 included in the transmission instruction is transmitted. Is determined to be associated with the screen identification information included in the association unit 57. Here, a case will be described as an example where the association information including the device identification information of portable information device 200A includes the screen identification information included in the transmission instruction. In this case, the requesting device determination unit 71 extracts association information including device identification information of the portable information device 200A, and determines the portable information device 200A as the requesting device. Request device determination unit 71 outputs the device identification information of portable information device 200 </ b> A determined as the request device to determination unit 75.

  The requesting device determination unit 71 may extract a plurality of pieces of association information including screen identification information included in the transmission instruction from the association information stored in the RAM 114. In this case, the request device determination unit 71 determines a request device based on any one of the extracted pieces of association information. For example, an apparatus having the display unit 206 having a resolution equal to or higher than the resolution of the display unit 206 of the portable information device 200 that is an instruction device is determined as the requesting device. More preferably, an apparatus having a display unit having the same resolution as that of the display unit 206 of the portable information device 200 as an instruction device is determined as the requesting device. This is because the portable information device 200 that is an instruction device does not need to perform resolution conversion processing. Further, the requesting device may be determined based on the association information with the newest date and time specified by the date and time information.

  In addition, when extracting a plurality of association information including screen identification information included in the transmission instruction, the requesting device determination unit 71 converts the association information including the device identification information of the device having the highest processing capability among the plurality of association information. The requesting device may be determined based on this. By making a device with a high processing capacity a request device, the transfer speed can be increased.

  Further, based on the association information including the device identification information of the device having the shortest distance from the pointing device among the plurality of related information by acquiring the position where the portable information device 200, 200A, 200B exists. The requesting device may be determined. This is because the communication speed between the requesting device and the instruction device may be increased by making the device closest to the instruction device the requesting device.

  The requesting device determination unit 71 may determine the requesting device when the number of devices with established communication paths is equal to or greater than a predetermined number. The predetermined number is a value that does not affect the load of the process in which the MFP 100 transmits the operation screen in order to execute other processes. This is because when the number of devices that remotely operate MFP 100 is smaller than the predetermined number, MFP 100 can execute other processing. Also, when the number of devices that remotely operate MFP 100 is greater than or equal to a predetermined number, the probability that a requesting device exists is high.

  In response to the input of the device identification information of the requesting device from the requesting device determination unit 71, the determination unit 75 causes the association unit 57 to display the screen identification information and date / time information associated with the device identification information of the requesting device. get. The determination unit 75 extracts association information including the device identification information of the requesting device from the association information stored in the RAM 114, and acquires screen identification information and date / time information included in the extracted association information. . The determination unit 75 determines whether or not the operation screen specified by the acquired screen identification information includes a variable region. The fluctuation region is a region that changes with time. The variable area is an area in which information generated when the MFP 100 executes processing is displayed.

  The variable area includes, for example, a standby job screen that displays jobs accumulated before execution, a job history screen that displays the history of executed jobs, a consumable status display screen that displays the usage or remaining amount of consumables, It includes a system screen for displaying the system status, a management information display screen for displaying management information, an error screen for displaying warnings and troubles, a destination setting screen for displaying an address book, and a browsing screen for displaying a part of data.

  The variable area included in the standby job screen displays a list of accumulated jobs. The accumulated job varies every time the job is accumulated or every time the job is executed. The variable area included in the job history screen displays a list of jobs executed by the MFP 100. The job executed by the MFP 100 changes every time the job is executed. The variable area included in the consumables status display screen displays the amount of used or remaining amount of consumables. The usage amount or remaining amount of the consumables varies depending on the image formation. The expendable supplies include toner, developing device, paper, and fixing device. The variable area included in the system screen displays the state of the system. The state of the system changes every time the CPU 111 executes a process. The system state includes the load state of the CPU 111 and the storage capacity or remaining capacity of the HDD 115. The load on the CPU 111 changes every time the CPU 111 executes a process, and the storage capacity or remaining capacity of the HDD 115 changes every time data is stored in or deleted from the HDD 115.

  The variable area included in the management information display screen displays management information. The management information includes, for example, the number of prints for a predetermined period. The number of prints varies every time MFP 100 prints. The variable area included in the error screen displays the contents of the warning or trouble. The content of the warning or trouble is determined by an error that occurs in the MFP 100.

  The variable area included in the destination setting screen displays an address book in which an e-mail address or a facsimile number for each user is associated. The change area changes when the address book is changed.

  The fluctuation area included in the browsing screen includes a fluctuation area where a plurality of pages of data is displayed for each page, or a fluctuation area where a part of the data is displayed in a scrollable state. In the variable area for displaying the data of a plurality of pages for each page, another page is displayed by accepting a page switching operation by the user. In the variable area in which a part of the data is displayed in a scrollable state, another part of the data is displayed by a scroll operation by the user.

  FIG. 6 is a diagram illustrating an example of an operation screen including a consumable state display screen as a variable region. Referring to FIG. 6, operation screen 301 includes a consumable status display screen 303. The consumables status display screen 303 includes a character string “20130801” as a date, a bar graph indicating the remaining amount of toner for each of Y (yellow), M (red purple), C (blue green), and K (black). ,including.

  FIG. 7 is a diagram illustrating an example of an operation screen including a job history screen as a variable area. Referring to FIG. 7, operation screen 311 includes a job history screen 313. Job history screen 313 includes job history information for each of the six jobs executed by MFP 100. The job information is No. Item, type item, document name / destination item, registration time item, total number of items item, and execution result item. No. In this item, a number for identifying a job is set. The type of job is set in the type item. The item of document name / destination is set to the document name of the data for which the job is executed or the data transmission destination. In the item of registration time, the time when the job is completed is set. In the item of the total number, the number of pages of data executed by the job is set. The result of executing the job is set in the execution result item. The result of executing the job includes a normal end indicating that the job has been executed normally and an abnormal end indicating that the job has not been executed normally.

  Returning to FIG. 5, the determination unit 75 outputs a full screen request instruction to the request unit 77 when determining that the operation screen specified by the acquired screen identification information does not include the fluctuation region. The full screen request instruction includes device identification information of the instruction device, device identification information of the requesting device, and screen identification information included in the transmission instruction.

  When determining that the operation screen specified by the acquired screen identification information includes a variable region, the determination unit 75 determines whether or not the variable region is temporally changed. Specifically, based on the date and time specified by the acquired date and time information, the elapsed time from the date and time specified by the date and time information is calculated, and if the elapsed time is a predetermined time or more, It is determined that there is a temporal change in the variable region, and if it is shorter than the predetermined time, it is determined that there is no temporal change in the variable region. In addition, the information included in the variable area of the operation screen is specified, and the content at the date and time specified by the acquired date and time information is compared with the content at the current date and time for the specified information. In this case, it may be determined that there is a temporal change in the fluctuation region, and if both are the same, it may be determined that there is no temporal change in the fluctuation region.

  When the determination unit 75 determines that there is no temporal change in the variable region, the determination unit 75 determines that the content of the variable region in the operation screen transmitted to the requesting device is the same as the content at the current date and time. When it is determined that there is a significant change, it is determined that the content of the variable area in the operation screen transmitted to the requesting device is not the same as the content at the current date and time.

  If the determination unit 75 determines that the content of the variable area in the operation screen transmitted to the requesting device is the same as the content at the current date and time, the determination unit 75 outputs a full screen request instruction to the request unit 77. The full screen request instruction includes device identification information of the instruction device, device identification information of the requesting device, and screen identification information included in the transmission instruction.

  If the determination unit 75 determines that the content of the variable area in the operation screen transmitted to the requesting device is not the same as the content at the current date and time, the determination unit 75 outputs a partial transmission request instruction to the request unit 77. The partial transmission request instruction includes apparatus identification information of the instruction apparatus, apparatus identification information of the request apparatus, and screen identification information included in the transmission instruction.

  The request unit 77 includes a full screen transmission request unit 91, a non-moving region transmission request unit 93, a variable region transmission device determination unit 95, and a variable region transmission request unit 97. In response to the input of the full-screen request instruction from the determination unit 75, the full-screen transmission request unit 91 is a device specified by the device identification information of the requesting device included in the full-screen request instruction, in this case, the portable information device 200A In addition, a full screen transmission request is transmitted via the communication I / F unit 112. The full screen transmission request includes the device identification information of the pointing device, here, the portable information device 200. The full-screen transmission request causes the requesting device, here, the portable information device 200 to execute processing for transmitting the operation screen displayed on the requesting device, here, the portable information device 200A, to the pointing device, here, the portable information device 200. Signal.

  In response to the partial transmission request instruction input from the determination unit 75, the non-moving area transmission request unit 93 is a device identified by the device identification information of the requesting device included in the partial transmission request instruction, in this case, the portable information device 200A In addition, the non-moving area transmission request is transmitted via the communication I / F unit 112. The non-moving region transmission request includes device identification information of the pointing device included in the partial transmission request instruction, here, device identification information of the portable information device 200, and position information indicating a position of the variable region on the operation screen. The non-moving area transmission request is an area in the operation screen displayed by the requesting device, that is, the portable information device 200A, and a non-moving area other than the variable area specified by the position information included in the partial transmission request instruction. It is a signal that causes the portable information device 200A, which is the requesting device, to execute processing to be transmitted to the device identified by the device identification information of the pointing device included in the transmission request instruction, here, the portable information device 200.

  The variable region transmission device determination unit 95 is an operation screen different from the operation screen specified by the screen identification information included in the partial transmission request instruction in response to the partial transmission request instruction input from the determination unit 75. Thus, an operation screen that includes the same variation area as the operation area identified by the screen identification information included in the partial transmission request instruction is identified as an alternative screen. An alternative screen may be associated in advance with each operation screen. There may be a plurality of alternative screens. When specifying the alternative screen, the variable region transmission device determination unit 95 extracts the association information including the screen identification information of the alternative screen from the association information stored in the RAM 114.

  If the associated area information including the screen identification information of the alternative screen can be extracted, the variable area transmitting device determination unit 95 determines whether or not the variable area of the alternative screen has changed. If the elapsed time from the date and time specified by the date and time information included in the extracted association information is equal to or longer than a predetermined time, the variable region transmission device determination unit 95 changes the temporal region of the variable region of the alternative screen. If it is determined that the change area is shorter than the predetermined time, it is determined that there is no temporal change in the change area of the alternative screen. In addition, the information included in the variation area of the alternative screen is specified, and the content at the date and time specified by the date and time information included in the association information is compared with the content at the current date and time for the specified information. If they are different, it may be determined that there is a temporal change in the variable region, and if both are the same, it may be determined that there is no temporal change in the variable region.

  When determining that the variable region of the alternative screen has not changed, the variable region transmission device determining unit 95 determines the device specified by the device identification information included in the extracted association information as the variable region transmission device. . When specifying a plurality of alternative screens, the variable region transmission device determining unit 95 may determine a variable region device corresponding to any one of the plurality of alternative screens.

  The variable region transmission device determination unit 95 outputs a variable region generation instruction to the variable region generation unit 79 when the variable region device is not determined, and when the variable region device is determined, the variable region transmission request unit 97 outputs the variable region transmission instruction. Output to. The variable region generation instruction includes device identification information of the instruction device and screen identification information included in the partial transmission request instruction. The variable region transmission instruction includes device identification information of the instruction device, device identification information of the variable region transmission device, and screen identification information of an alternative screen.

  First, the case where the variable region transmission device determination unit 95 determines a variable region device will be described as an example. Here, a case where the association information including the device identification information of the portable information device 200B includes the screen identification information of the alternative screen, and the variable region of the operation screen specified by the screen identification information is described as an example. To do. In this case, the variable region transmission device determination unit 95 extracts association information including device identification information of the portable information device 200B, and determines the portable information device 200B as a variable region transmission device. The variable region transmission device determination unit 95 outputs a variable region transmission instruction to the variable region transmission request unit 97. The variable region transmission instruction includes device identification information of the instruction device, device identification information of the portable information device 200B determined as the variable region transmission device, and screen identification information of an alternative screen.

  The variable region transmission request unit 97 is a device specified by the device identification information of the variable region transmission device included in the variable region transmission instruction in response to the input of the variable region transmission instruction from the variable region transmission device determination unit 95. Here, the variable region transmission request is transmitted to the portable information device 200B via the communication I / F unit 112. The variable region transmission request includes device identification information of the pointing device included in the variable region transmission instruction, here, device identification information of the portable information device 200, and position information indicating the position of the variable region on the operation screen. The variable area transmission request is an area on the operation screen displayed by the portable information device 200B, which is a variable area transmission device, and a variable area specified by position information included in the variable area transmission instruction is a mobile device that is an instruction device. This is a signal that causes the portable information device 200B, which is a variable region transmission device, to execute processing to be transmitted to the information device 200.

  Next, a case will be described as an example where the variable region transmission device determination unit 95 does not determine a variable region device. The variable region generation unit 79 receives a variable region generation instruction. The variable region generation instruction includes device identification information of the instruction device and screen identification information included in the partial transmission request instruction. The variable region generation unit 79 determines whether or not the variable region of the operation screen specified by the screen identification information included in the variable region generation instruction can be generated in response to the input of the variable region generation instruction. To do. The variable region generation unit 79 determines whether or not a variable region can be generated based on the load on the CPU 111 when the variable region generation instruction is input. If the load on the CPU 111 is smaller than a predetermined value, it is determined that the variable region can be generated. If the acquired load is equal to or greater than the predetermined value, it is determined that the variable region cannot be generated. Further, the variable region generation unit 79 may determine that the load on the CPU 111 is equal to or greater than a predetermined value while the job is being executed when the variable region generation instruction is input. Further, the variable region generation unit 79 may calculate the load on the CPU 111 based on the number of accumulated jobs at the time when the variable region generation instruction is input. Also, the load value may be determined according to the type of job being executed by the CPU 111.

  When determining that the variable region can be generated, the variable region generation unit 79 generates a variable region of the operation screen specified by the screen identification information included in the variable region generation instruction, and the generated variable region and the pointing device The device identification information is output to the variable region transmission unit 81. When the change area generation unit 79 determines that the change area cannot be generated, the change area generation unit 79 outputs an absence notification instruction to the absence notification unit 83. The absence notification instruction includes device identification information of the instruction device.

  In response to the input of the variable region and the device identification information of the pointing device from the variable region generation unit 79, the variable region transmitting unit 81 is a device identified by the device identification information of the pointing device, here the portable information device 200. The variable region is transmitted via the communication I / F unit 112.

  In response to the presence notification instruction input from the variable region generation unit 79, the absence notification unit 83 is connected to the instruction device identified by the device identification information included in the absence notification instruction, here the portable information device 200. Then, a non-existence signal indicating that the fluctuation region does not exist is transmitted via the communication I / F unit 112.

  Since the functions of the CPU 201 included in each of the portable information devices 200, 200A, and 200B are the same, here, the function of the CPU 201 included in the portable information device 200 will be described as an example.

  FIG. 8 is a block diagram illustrating an example of an outline of functions of a CPU provided in the portable information device according to the present embodiment. The functions shown in FIG. 6 are functions formed in the CPU 201 when the CPU 201 included in the portable information device 200 executes a remote operation program stored in the flash memory 203 or the CD-ROM 211A.

  Referring to FIG. 8, CPU 201 includes a screen request unit 251, a screen reception unit 253, a display control unit 255, and a screen transfer unit 257. Screen request unit 251 controls wireless LAN I / F 208 and transmits a screen request to MFP 100. When the user operates operation unit 207 and inputs an instruction to remotely operate MFP 100, screen request unit 251 controls wireless LAN I / F 208 to establish a communication path with MFP 100. Send a connection request to. When a communication path is established with MFP 100, screen request unit 251 transmits a screen request to MFP 100 via wireless LAN I / F 208. The screen request unit 251 outputs a signal indicating that the screen request has been transmitted to the screen reception unit 253.

  Further, when a predetermined operation by the user is accepted in a state where the operation screen is displayed on the display unit 206 by the display control unit 255, the screen request unit 251 sends a screen request to the MFP 100 via the wireless LAN I / F 208. Send to. Specifically, when the user detects the position on the display surface of the display unit 206 by the touch panel 207B, the screen request unit 251 specifies the position on the operation screen based on the position detected by the touch panel 207B. Then, an execution command including position information indicating the position in the specified operation screen is transmitted to MFP 100. The screen request unit 251 outputs a signal indicating that the execution command has been transmitted to the screen reception unit 253.

  The screen receiving unit 253 controls the wireless LAN I / F 208 to receive the screen from the outside after the signal indicating that the screen request is transmitted from the screen request unit 251 or the signal indicating that the execution command is transmitted. To do. In the present embodiment, screen receiving unit 253 receives a fixed area from mobile information device 200A in the first case of receiving an operation screen from MFP 100, in the second case of receiving an operation screen from mobile information device 200A. In the third case of receiving the variable area from the portable information device 200B, the fourth case of receiving the fixed area from the portable information device 200A, and the fourth case of receiving the variable area from the MFP 100, and from the portable information device 200A There is a fifth case of receiving a non-moving area and receiving a non-existence signal from the MFP 100.

  The screen receiving unit 253 includes a full screen receiving unit 261, a non-moving region receiving unit 263, a variable region receiving unit 365, and a nonexistence signal receiving unit 267. Full-screen receiving unit 261 receives an operation screen from MFP 100 in the first case, and receives an operation screen from portable information device 200A in the second case. The full screen receiving unit 261 outputs the received operation screen to the display control unit 255.

  In any of the third to fifth cases, the non-moving area receiving unit 263 receives the non-moving area from the portable information device 200 </ b> A and outputs the received non-moving area to the display control unit 255. Fluctuating area receiving unit 265 receives the changing area from portable information device 200B in the third case, and receives the changing area from MFP 100 in the fourth case. The fluctuation area receiving unit 265 outputs the received fluctuation area to the display control unit 255.

  Absence signal receiver 267 receives the absence signal from MFP 100 in the fifth case. When the absence signal reception unit 267 receives the absence signal, the absence signal reception unit 267 outputs the received absence signal to the display control unit 255.

  The display control unit 255 controls the display unit 206 to display an operation screen on the display unit 206. When the operation screen is input from the full screen receiving unit 261, the display control unit 255 displays the operation screen on the display unit 206.

  The display control unit 255 includes a synthesis unit 269. The synthesizing unit 269 synthesizes the non-moving area input from the non-moving area receiving unit 263 and the changing area input from the changing area receiving unit 265 to generate an operation screen. The display control unit 255 displays the operation screen generated by the combining unit 269 on the display unit 206.

  When the non-moving area is input from the non-moving area receiving unit 263 and the non-existing signal is input from the non-existing signal receiving unit 267, the display control unit 255 sets the non-moving area input from the non-moving area receiving unit 263 on the operation screen. The setting screen is displayed on the display unit 206.

  The screen transfer unit 257 controls the wireless LAN I / F 208 to transmit all or part of the operation screen displayed on the display unit 206 by the display control unit 255 to the other portable information devices 200A and 200B. The screen transfer unit 257 includes a full screen transmission request receiving unit 271, a full screen transmitting unit 273, a non-moving region transmission request receiving unit 275, a non-moving region generating unit 277, a non-moving region transmitting unit 279, and a variable region transmitting request reception. Unit 281, variable region generation unit 283, and variable region transmission unit 285.

  Full-screen transmission request receiving unit 271 controls wireless LAN I / F 208 to receive a full-screen transmission request from MFP 100. The full screen transmission request includes device identification information and screen identification information of the pointing device. Upon receiving a full screen transmission request from MFP 100, full screen transmission request reception unit 271 outputs a transmission instruction to full screen transmission unit 273. This transmission instruction includes device identification information of the instruction device.

  The full screen transmission unit 273 specifies the operation screen displayed on the display unit 206 with the device identification information included in the transmission instruction in response to the transmission instruction input from the full screen transmission request reception unit 271. The data is transmitted to the instruction device via the wireless LAN I / F 208.

  A non-moving area transmission request receiving unit 275 controls the wireless LAN I / F 208 to receive a non-moving area transmission request from the MFP 100. The non-moving area transmission request includes device identification information of the pointing device and position information indicating the position of the variable area on the operation screen. When the non-moving area transmission request receiving unit 275 receives the non-moving area transmission request from the MFP 100, the non-moving area transmission request receiving unit 275 outputs a generation instruction to the non-moving area generation unit 277 and outputs a transmission instruction to the non-moving area transmission unit 279. The generation instruction includes position information indicating the position of the change area in the operation screen. The transmission instruction includes device identification information of the instruction device.

  The non-moving region generation unit 277 generates a non-moving region in response to the generation instruction input from the non-moving region transmission request receiving unit 275, and outputs the generated non-moving region to the non-moving region transmitting unit 279. The immovable area generation unit 277 extracts an area other than the fluctuation area specified by the position information included in the generation instruction as an immovable area from the operation screen displayed on the display unit 206, and extracts the extracted immovable area as an immovable area. The data is output to the area transmission unit 279.

  The non-moving area transmission unit 279 receives the non-moving area from the non-moving area generation unit 277 after the transmission instruction is input from the non-moving area transmission request receiving unit 275, and converts the non-moving area into the non-moving area transmission request receiving unit. The data is transmitted to the instruction device specified by the device identification information included in the transmission instruction input from H.275 via the wireless LAN I / F 208.

  A variable area transmission request receiving unit 281 controls the wireless LAN I / F 208 to receive a variable area transmission request from the MFP 100. The change area transmission request includes device identification information of the pointing device and position information indicating the position of the change area in the operation screen. When the variable region transmission request receiving unit 281 receives the variable region transmission request from the MFP 100, the variable region transmission request receiving unit 281 outputs a generation instruction to the variable region generation unit 283 and outputs a transmission instruction to the variable region transmission unit 285. The generation instruction includes position information indicating the position of the change area in the operation screen. The transmission instruction includes device identification information of the instruction device.

  The variable region generation unit 283 generates a variable region in response to the generation instruction input from the variable region transmission request reception unit 281, and outputs the generated variable region to the variable region transmission unit 285. The variable region generation unit 283 extracts the region specified by the position information included in the generation instruction from the operation screen displayed on the display unit 206 as the variable region, and the extracted variable region is the variable region transmission unit 285. Output to.

  After the transmission instruction is input from the variable region transmission request receiving unit 281, the variable region transmitting unit 285 converts the variable region into the variable region transmission request receiving unit in response to the input of the variable region from the variable region generation unit 283. The data is transmitted via the wireless LAN I / F 208 to the instruction device specified by the device identification information included in the transmission instruction input from 281.

  FIG. 9 is a flowchart illustrating an example of the flow of remote control processing. The remote control process is a process executed by CPU 111 when CPU 111 provided in MFP 100 executes a remote control program stored in ROM 113, HDD 115, and CD-ROM 118.

  Referring to FIG. 9, CPU 111 determines whether a connection request is received from any of portable information devices 200, 200 </ b> A, 200 </ b> B. If communication I / F unit 112 receives a connection request, the process proceeds to step S02; otherwise, the process proceeds to step S03. Here, a case where a connection request is received from portable information device 200 will be described as an example. In step S02, a communication path is established with portable information device 200 that has transmitted the connection request, and the process proceeds to step S03. Negotiation for establishing a communication path with the portable information device 200 is executed to establish a communication path. When the communication path is established, the path identification information for identifying the communication path is associated with the device identification information of the portable information device 200 that has established the communication path, and stored in the RAM 114.

  In step S03, it is determined whether a screen request has been received from any of the devices that have established a communication path. When a communication path is established with at least one of portable information devices 200, 200A, and 200B in the stage before step S03 is executed, a screen request is received. For example, when a communication path is established with all of the portable information devices 200, 200A, 200B, a screen request is received from any of the portable information devices 200, 200A, 200B. If communication I / F unit 112 receives a screen request, the process proceeds to step S04; otherwise, the process proceeds to step S07.

  In step S04, the device that has transmitted the screen request is determined as an instruction device, and the process proceeds to step S05. Here, a case where a screen request is received from portable information device 200 will be described as an example. When a screen request is received from the portable information device 200, the portable information device 200 is determined as an instruction device.

  In step S05, a transmission target screen is specified. The operation screen specified by the screen request received in step S03 is specified as the transmission target screen. In the next step S06, an operation screen transmission process is executed, and the process proceeds to step S07. Although details of the operation screen transmission process will be described later, the operation screen transmission process is a process for transmitting or requesting the operation screen to display the operation screen specified as the transmission target screen on the instruction device.

  In step S07, it is determined whether an execution command has been received. The execution instruction is transmitted by the device that displays the operation screen among the portable information devices 200, 200A, and 200B. When a communication path is established with at least one of the portable information devices 200, 200A, and 200B before the step S07 is executed, and the operation screen is displayed on the device with the established communication path In addition, an execution instruction may be received from the device. If an execution instruction is received, the process proceeds to step S08; otherwise, the process returns to step S01.

  In step S08, the process branches depending on the content of the received execution command. If the execution command indicates a screen transition operation, the process proceeds to step S09. If the execution command indicates an execution instruction operation, the process proceeds to step S12. The screen transition operation is an operation for switching the operation screen, and specifies the operation screen after the switching. The execution instruction operation is an operation for causing MFP 100 to execute a process, and specifies a process to be executed.

  In step S09, the device that has transmitted the execution command is determined as an instruction device, and the process proceeds to step S10. Here, a case where an execution command is received from portable information device 200 will be described as an example. When an execution command is received from the portable information device 200, the portable information device 200 is determined as an instruction device.

  In step S10, a transmission handling screen is specified. The operation screen specified by the execution command received in step S07 is specified as the transmission target screen. In the next step S11, an operation screen transmission process is executed, and the process returns to step S01. On the other hand, in step S12, the process specified by the execution instruction operation is executed, and the process returns to step S01.

  10 and 11 are flowcharts showing an example of the flow of the operation screen transmission process. The operation screen transmission process is a process executed in step S06 or step S11 of FIG. Referring to FIG. 9, the instruction device is determined in step S04 or step S09 before the operation screen transmission process is executed, and the transmission target screen is specified in step S05 or step S10. Here, a case will be described as an example where portable information device 200 is determined to be the pointing device before the operation screen transmission process is executed.

  Referring to FIGS. 10 and 11, CPU 111 determines whether or not association information exists (step S21). If the association information including the screen identification information of the transmission target screen is stored in the RAM 114, it is determined that the related information exists. If related information exists, the process proceeds to step S22; otherwise, the process proceeds to step S51. The association information is stored in the RAM 114 in step S37, step S42, step S47, and step S53 described later.

  In step S51, the transmission target screen is transmitted to the instruction device, and the process proceeds to step S52. In step S52, association information is generated, and the process proceeds to step S53. Association information including device identification information of the instruction device, screen identification information of the transmission target screen, and date / time information indicating the current date / time is generated. In the next step S53, the generated association information is stored in the RAM 114, and the process returns to the remote control process.

  In step S22, the association information including the screen identification information of the transmission target screen is extracted from the association information stored in the RAM 114. In the next step S23, the requesting device is determined, and the process proceeds to step S24. The device specified by the device identification information included in the association information extracted in step S22 is determined as the requesting device. When a plurality of pieces of association information are extracted in step S22, the requesting device is determined based on any one piece of association information. For example, a device having a display unit with a resolution equal to or higher than the resolution of the display unit 206 of the pointing device is determined as the requesting device. More preferably, an apparatus having a display unit with the same resolution as the display unit 206 of the pointing device is determined as the requesting apparatus. This is because the resolution conversion processing by the instruction device is not necessary. Further, the requesting device may be determined based on the association information with the newest date and time specified by the date and time information. Here, a case where portable information device 200A is determined as the requesting device will be described as an example.

  In step S24, it is determined whether or not the transmission target screen includes a variable region. The fluctuation area is an area that changes with the passage of time, and is predetermined for each operation screen. If the transmission target screen includes a variable area, the process proceeds to step S25; otherwise, the process proceeds to step S48. In step S25, it is determined whether or not the variable region has changed. It is determined whether or not the content of the variable area in the operation screen displayed on the requesting device is different from the content at the current date and time. If the elapsed time calculated from the current date and time and the date and time specified by the date and time information included in the related information extracted in step S22 is greater than or equal to a predetermined time, it is determined that the fluctuation region has fluctuated. If the time is shorter than the predetermined time, it is determined that the change region has not changed. Note that the information included in the variable area of the operation screen is specified, and for the specified information, the contents at the date and time specified by the acquired date and time information are compared with the contents at the current date and time, and the variable area changes. It may be determined whether or not.

  In step S48, a full screen transmission request is transmitted to the requesting apparatus, and the process proceeds to step S49. A full-screen transmission request is transmitted via the communication I / F unit 112 to the requesting device determined in step S23, here, the portable information device 200A. The full screen transmission request includes the device identification information of the pointing device, here, the portable information device 200. The full screen transmission request is a signal that causes the portable information device 200A, which is the requesting device, to execute processing for transmitting the operation screen displayed on the portable information device 200A, which is the requesting device, to the portable information device 200, which is the pointing device. .

  In step S49, association information is generated, and the process proceeds to step S50. Association information including device identification information of the instruction device, screen identification information of the transmission target screen, and date / time information indicating the current date / time is generated. In the next step S50, the generated association information is stored in HDD 115, and the process returns to the remote control process.

  In step S26, a non-moving area transmission request is transmitted to the requesting apparatus determined in step S23 via the communication I / F unit 112. The non-moving area transmission request includes device identification information of the pointing device and position information indicating the position of the variable area on the operation screen. In the non-moving area transmission request, a process of transmitting the non-moving area in the operation screen displayed by the portable information device 200A that is the requesting device to the portable information device 200 that is the instruction device is transmitted to the portable information device 200A that is the requesting device. This is a signal to be executed.

  In step S27, it is determined whether an alternative screen exists. The substitute screen is an operation screen that is different from the transmission target screen and includes the same fluctuation region as the transmission target screen. If an alternative screen exists, the process proceeds to step S28; otherwise, the process proceeds to step S33. In step S <b> 28, it is determined whether or not there is association information on the alternative screen. If the association information including the screen identification information of the alternative screen is stored in the RAM 114, it is determined that the related information of the alternative screen exists. If the related information of the alternative screen exists, the process proceeds to step S29; otherwise, the process proceeds to step S33.

  In step S29, one piece of association information including the screen identification information of the alternative screen is selected as a processing target from the association information stored in the RAM 114. In the next step S30, date and time information included in the related information selected in step S29 is acquired. It is then determined whether the elapsed time is equal to or greater than a threshold value T (step S31). The elapsed time is calculated from the date and time specified by the date and time information and the current date and time. If the elapsed time is equal to or greater than threshold value T, the process proceeds to step S32. If not, the process proceeds to step S43. In step S32, it is determined whether there is association information that has not been selected as a processing target in step S29. If such association information exists, the process returns to step S29; otherwise, the process proceeds to step S33.

  In step S43, a variable region transmitter is determined. The device specified by the device identification information included in the related information selected in step S29 is determined as the variable region transmission device. Here, a case will be described as an example where portable information device 200 </ b> B is determined as the variable region transmission device. In the next step S44, a variable area transmission request is transmitted via the communication I / F unit 112 to the portable information device 200B which is a variable area transmitter. The change area transmission request includes device identification information of the portable information device 200 that is an instruction device, and position information indicating a position of the change area in the operation screen. The variable area transmission request is a process of transmitting the variable area of the operation screen displayed by the portable information device 200B, which is the variable area transmission device, to the portable information device 200, which is the pointing device. This signal is executed by the device 200B.

  In the next step S45, the date information included in the association information selected as the processing target in step S29 is set in the date information. In the next step S46, the associated additional information is generated. Association information including the device identification information of the instruction device, the screen identification information of the transmission target screen, and the date and time information set in step S46 is generated. In the next step S47, the generated association information is stored in the RAM 114, and the process returns to the remote control process.

  In step S33, it is determined whether the load on CPU 111 is equal to or greater than threshold value TH. If the load on CPU 111 is equal to or greater than threshold value TH, the process proceeds to step S34; otherwise, the process proceeds to step S38. In step S34, the absence signal is transmitted to portable information device 200, which is an instruction device, via communication I / F unit 112, and the process proceeds to step S35. The absence signal is a signal indicating that there is no fluctuation region. In step S35, a blank is set in the date / time information. In the next step S36, the associated additional information is generated. Association information including the device identification information of the instruction device, the screen identification information of the transmission target screen, and the date and time information set to blank in step S35 is generated. In the next step S37, the generated association information is stored in the RAM 114, and the process returns to the remote control process.

  In step S38, a variable area of the processing target screen is generated, and the process proceeds to step S39. In the next step S39, the generated variable area is transmitted to the portable information device 200, which is an instruction device, via the communication I / F unit 112, and the process proceeds to step S40. In step S40, the current date and time are set in the date and time information, and the process proceeds to step S41. In step S41, the associated additional information is generated. Association information including the device identification information of the instruction device, the screen identification information of the transmission target screen, and the date / time information set to the current date / time in step S40 is generated. In the next step S42, the generated association information is stored in the RAM 114, and the process returns to the remote control process.

  FIG. 12 is a flowchart illustrating an example of the flow of remote operation processing. The remote operation process is a process executed by the CPU 201 when the CPU 201 included in the portable information device 200 executes a remote operation program stored in the flash memory 203 or the CD-ROM 211A. Referring to FIG. 12, CPU 201 determines whether a remote operation instruction has been accepted (step S61). If the operation unit 207 receives a remote operation instruction input by the user, the remote operation instruction is received. The process waits until a remote operation instruction is accepted (NO in step S61). If a remote operation instruction is accepted (YES in step S61), the process proceeds to step S62.

  In step S62, a connection request is transmitted to MFP 100. The wireless LAN I / F 208 is controlled to transmit a connection request in order to establish a communication path with the MFP 100. Then, it is determined whether or not a communication path has been established (step S63). A communication path is established by negotiating with the MFP 100. The process waits until the communication path is established (NO in step S63). If the communication path is established (YES in step S63), the process proceeds to step S64.

  In step S 64, the screen request is transmitted to MFP 100 via wireless LAN I / F 208. In the next step S65, it is determined whether or not the entire screen of the operation screen has been received. If the entire screen has been received, the process proceeds to step S66; otherwise, the process proceeds to step S67. In step S66, the received full screen is set as an operation screen, and the process proceeds to step S73.

  In step S67, it is determined whether a non-moving area has been received. If a non-moving area has been received, the process proceeds to step S68; otherwise, the process proceeds to step S74. In step S68, it is determined whether or not a variable region has been received. If the variable region is received, the process proceeds to step S69; otherwise, the process proceeds to step S71. In step S69, a synthesized image is generated by synthesizing the immovable area received in step S67 and the fluctuation area received in step S68. In the next step S70, the composite image is set on the operation screen, and the process proceeds to step S73.

  In step S71, it is determined whether an absence signal has been received. If the absence signal is received, the process proceeds to step S72. If not, the process returns to step S68. In step S72, the immobile area received in step S66 is set on the operation screen, and the process proceeds to step S73.

  In step S73, the operation screen set in any of steps S66, S70, and S72 is displayed on display unit 206, and the process proceeds to step S74.

  In step S74, it is determined whether an operation by the user is accepted. When a position designated by the user is detected by touch panel 207B, an operation is accepted. A position in the setting screen is specified based on the detected position, and an operation is specified based on the position in the operation screen. If the operation is accepted, the process proceeds to step S75. If not, the process proceeds to step S78.

  In step S75, the process is branched based on the accepted operation. If the accepted operation is an operation indicating an end instruction, the process proceeds to step S76. If the accepted operation is an operation indicating an execution instruction, the process proceeds to step S77. In step S76, the communication path established in step S63 is disconnected, and the process ends.

  In step S77, an execution command is transmitted to MFP 100 via wireless LAN I / F 208, and the process proceeds to step S78. In step S78, it is determined whether a full screen transmission request is received from MFP 100 or not. If a full-screen transmission request is received, the process proceeds to step S79; otherwise, the process proceeds to step S80. In step S79, the operation screen displayed on display unit 206 is transmitted via wireless LAN I / F 208 to the instruction device specified by the device identification information included in the full-screen transmission request, and the process proceeds to step S80. .

  In step S 80, it is determined whether a non-moving area transmission request is received from MFP 100. If a fixed area transmission request is received, the process proceeds to step S81; otherwise, the process proceeds to step S83. In step S81, a non-moving area is generated. In the operation screen displayed on the display unit 206, by extracting an area other than the area specified by the position information included in the immobile area transmission request, the extracted area is set as an immobile area. In step S82, the generated immovable area is transmitted to the pointing device specified by the apparatus identification information included in the immovable area transmission request via the wireless LAN I / F 208, and the process proceeds to step S83.

  In step S83, it is determined whether a variable area transmission request is received from MFP 100 or not. If a variable area transmission request is received, the process proceeds to step S84. If not, the process returns to step S65. In step S84, a variable region is generated. In the operation screen displayed on the display unit 206, by extracting the area specified by the position information included in the variable area transmission request, the extracted area is set as the variable area. In step S85, the generated variable region is transmitted to the instruction device specified by the device identification information included in the variable region transmission request via the wireless LAN I / F 208, and the process returns to step S65.

  Next, a first specific example in which portable information devices 200, 200A, 200B remotely control MFP 100 simultaneously will be described. Here, the device identification information of the portable information devices 200, 200A, and 200B is “IP address A”, “IP address B”, and “IP address C”, respectively.

<First specific example>
In the first specific example, portable information device 200A receives and displays the first operation screen that does not include the variable region from MFP 100, and portable information device 200B displays the second operation screen that includes the variable region. A case will be described as an example where portable information device 200 requests MFP 100 for the first operation screen in the first stage received from MFP 100 and displayed.

  FIG. 13 is a diagram illustrating an example of related information in the first stage in the first specific example. Referring to FIG. 13, the related information includes related records for portable information devices 200, 200A, and 200B. The related record includes the device ID item and the operation screen No. And date and time items. In the device ID item, device identification information is set. Operation screen No. In this item, screen identification information of the operation screen is set. In the date and time item, the date and time when the operation screen is transmitted is set.

  The related record in the first line corresponds to the portable information device 200, “IP address A” is set in the item of device ID, and the operation screen No. Nothing is set in the item and date / time item. The related record in the second row corresponds to the portable information device 200A, “IP address B” is set in the item of device ID, and the operation screen No. The screen identification information “110” of the first operation screen is set in the item of “2013”, and “2013/08/01 09:45” is set in the item of date and time. The related record in the third row corresponds to the portable information device 200B, “IP address C” is set in the item of device ID, and the operation screen No. The screen identification information “140” of the second operation screen is set in the item “”, and “2013/08/01 09:59” is set in the date and time item.

  FIG. 14 is a diagram illustrating an example of a flow of information transmitted and received between the portable information device and the MFP in the first specific example. Referring to FIG. 14, the flow of time is shown from top to bottom in the vertical direction. In addition, the arrow in the figure indicates that data transmitted from the device that is the starting point of the arrow is received by the device that is the end point of the arrow.

  First, portable information device 200 </ b> A transmits a screen request for requesting transmission of the first operation screen to MFP 100. This screen request includes the screen identification information “110” of the first operation screen. In response to receiving the screen request from portable information device 200A, MFP 100 returns a first operation screen specified by screen identification information “110” included in the screen request to portable information device 200A. Portable information device 200 </ b> A displays a first operation screen received from MFP 100 on display unit 206.

  Next, portable information device 200B transmits to MFP 100 a screen request for requesting transmission of the second operation screen. This screen request includes the screen identification information “140” of the second operation screen. In response to receiving the screen request from portable information device 200B, MFP 100 returns a second operation screen specified by screen identification information “140” included in the screen request to portable information device 200B. Portable information device 200 </ b> B displays a second operation screen received from MFP 100 on display unit 206.

  In the first stage, portable information device 200 transmits a screen request for requesting transmission of the first operation screen to MFP 100. This screen request includes the screen identification information “110” of the first operation screen. In response to receiving the screen request from portable information device 200, MFP 100 refers to the related information shown in FIG. 13 and extracts a related record including screen identification information “110”. Thereby, since the related record in the second row is extracted, the portable information device 200A specified by the device identification information “IP address B” set in the device ID item is determined as the requesting device. MFP 100 transmits a full-screen transmission request to portable information device 200 </ b> A because the first operation screen does not include a variable area. The full screen transmission request includes the device identification information “IP address A” of the portable information device 200 that has transmitted the screen request.

  In response to receiving the full screen transmission request from MFP 100, portable information device 200 </ b> A displays the first operation screen displayed on display unit 206 at that time as the device identification information “IP” included in the full screen transmission request. It is transferred to the portable information device 200 specified by the address “A”. The portable information device 200 displays the first operation screen received from the portable information device 200A on the display unit 206.

<Second specific example>
In the second specific example, portable information device 200A receives and displays the third operation screen including the variation area from MFP 100, and portable information device 200B displays the second operation screen including the variation area from MFP 100. A case will be described as an example where portable information device 200 requests MFP 100 for a third operation screen in the second stage received and displayed. Further, in the second stage, the variation area of the third operation screen includes the same type of information as the variation area of the second operation screen, but the contents are different.

  FIG. 15 is a diagram illustrating an example of related information in the second stage in the second specific example. Referring to FIG. 15, the related information includes related records for portable information devices 200, 200 </ b> A, and 200 </ b> B. The related record in the first line corresponds to the portable information device 200, “IP address A” is set in the item of device ID, and the operation screen No. Nothing is set in the item and date / time item. The related record in the second row corresponds to the portable information device 200A, “IP address B” is set in the item of device ID, and the operation screen No. The screen identification information “130” of the third operation screen is set in the item “”, and “2013/08/01 09:45” is set in the date item. The related record in the third row corresponds to the portable information device 200B, “IP address C” is set in the item of device ID, and the operation screen No. The screen identification information “140” of the second operation screen is set in the item “”, and “2013/08/01 09:59” is set in the date and time item.

  FIG. 16 is a diagram illustrating an example of a flow of information transmitted and received between the portable information device and the MFP in the second specific example. Referring to FIG. 16, the flow of time is shown from top to bottom in the vertical direction. In addition, the arrow in the figure indicates that data transmitted from the device that is the starting point of the arrow is received by the device that is the end point of the arrow.

  First, portable information device 200A transmits to MFP 100 a screen request for requesting transmission of the third operation screen. This screen request includes the screen identification information “130” of the third operation screen. In response to receiving the screen request from portable information device 200A, MFP 100 returns a third operation screen specified by screen identification information “130” included in the screen request to portable information device 200A. Portable information device 200 </ b> A displays a third operation screen received from MFP 100 on display unit 206.

  Next, portable information device 200B transmits to MFP 100 a screen request for requesting transmission of the second operation screen. This screen request includes the screen identification information “140” of the second operation screen. In response to receiving the screen request from portable information device 200B, MFP 100 returns a second operation screen specified by screen identification information “140” included in the screen request to portable information device 200B. Portable information device 200 </ b> B displays a second operation screen received from MFP 100 on display unit 206.

  In the second stage, portable information device 200 transmits a screen request for requesting transmission of the third operation screen to MFP 100. The date and time at this time is “2013/08/01 10:00”. This screen request includes the screen identification information “130” of the third operation screen. In response to receiving the screen request from portable information device 200, MFP 100 refers to the related information shown in FIG. 15 and extracts a related record including screen identification information “130”. Thereby, since the related record in the second row is extracted, the portable information device 200A specified by the device identification information “IP address B” set in the device ID item is determined as the requesting device.

  Further, since the third operation screen includes the variable area, MFP 100 determines whether or not the variable area has changed based on the date / time information. The date and time in the second stage is “2013/08/01 10:00”, the date and time information in the related record in the second row is “2013/08/01 09:45”, and the third operation screen Elapsed time after transmitting is 15 minutes. If the threshold value for determining whether or not the change area has changed is 10 minutes, the change area included in the third operation screen transmitted to portable information device 200A changes in the second stage. Will be. Therefore, MFP 100 transmits a non-moving area transmission request to portable information device 200A. The non-moving area transmission request includes the device identification information “IP address A” of the portable information device 200 that has transmitted the screen request. In response to receiving the non-moving area transmission request from MFP 100, portable information device 200A extracts the non-moving area from the third operation screen displayed on display unit 206 at that time. The data is transferred to the portable information device 200 specified by the device identification information “IP address A” included in the non-moving area transmission request.

  Further, with reference to the related information shown in FIG. 15, MFP 100 displays mobile information that displays a variable area included in the third operation screen and a second operation screen that includes the same variable area to be displayed. The device 200C is specified. The date and time in the second stage is “2013/08/01 10:00”, the date and time information in the related record in the third row is “2013/08/01 09:59”, and the second operation screen Elapsed time after transmitting is 1 minute. For this reason, the change area included in the second operation screen transmitted to the portable information device 200B does not change in the second stage. Therefore, MFP 100 transmits a variable area transmission request to portable information device 200B. The variable area transmission request includes the device identification information “IP address A” of the portable information device 200 that has transmitted the screen request. In response to receiving the variable region transmission request from MFP 100, portable information device 200B extracts the variable region from the second operation screen displayed on display unit 206 at that time, and extracts the extracted variable region as The data is transferred to the portable information device 200 specified by the device identification information “IP address A” included in the change area transmission request.

  The portable information device 200 displays a composite image obtained by synthesizing the immovable region received from the portable information device 200A and the fluctuation region received from the portable information device 200C on the display unit 206 as an operation screen.

<Third specific example>
In the third specific example, portable information device 200A receives and displays the third operation screen including the variable area from MFP 100, and portable information device 200B receives and displays the fourth operation screen from MFP 100. A case will be described as an example where portable information device 200 requests MFP 100 for a third operation screen in the third stage. Further, the fourth operation screen does not include the same variation area as the variation area of the third operation screen.

  FIG. 17 is a diagram illustrating an example of related information in the second stage in the third specific example. With reference to FIG. 17, the related information includes related records for portable information devices 200, 200 </ b> A, and 200 </ b> B. The related record in the first line corresponds to the portable information device 200, “IP address A” is set in the item of device ID, and the operation screen No. Nothing is set in the item and date / time item. The related record in the second row corresponds to the portable information device 200A, “IP address B” is set in the item of device ID, and the operation screen No. The screen identification information “130” of the third operation screen is set in the item “”, and “2013/08/01 09:45” is set in the date item. The related record in the third row corresponds to the portable information device 200B, “IP address C” is set in the item of device ID, and the operation screen No. The screen identification information “150” of the fourth operation screen is set in the item of “2013”, and “2013/08/01 09:59” is set in the item of date and time.

  FIG. 18 is a diagram illustrating an example of a flow of information transmitted and received between the portable information device and the MFP in the third specific example. Referring to FIG. 18, the flow of time is shown from top to bottom in the vertical direction. In addition, the arrow in the figure indicates that data transmitted from the device that is the starting point of the arrow is received by the device that is the end point of the arrow.

  First, portable information device 200A transmits to MFP 100 a screen request for requesting transmission of the third operation screen. In response to receiving the screen request from portable information device 200A, MFP 100 returns a third operation screen specified by screen identification information “130” included in the screen request to portable information device 200A. Portable information device 200 </ b> A displays a third operation screen received from MFP 100 on display unit 206.

  Next, portable information device 200B transmits to MFP 100 a screen request for requesting transmission of the fourth operation screen. In response to receiving the screen request from portable information device 200B, MFP 100 returns a fourth operation screen specified by screen identification information “150” included in the screen request to portable information device 200B. Portable information device 200 </ b> B displays a fourth operation screen received from MFP 100 on display unit 206.

  In the third stage, portable information device 200 transmits a screen request for requesting transmission of the third operation screen to MFP 100. This screen request includes the screen identification information “130” of the third operation screen. In response to receiving the screen request from portable information device 200, MFP 100 refers to the related information shown in FIG. 17 and extracts a related record including screen identification information “130”. Thereby, since the related record in the second row is extracted, the portable information device 200A specified by the device identification information “IP address B” set in the device ID item is determined as the requesting device.

  Further, since the third operation screen includes the variable area, MFP 100 determines whether or not the variable area has changed based on the date / time information. The date and time in the second stage is “2013/08/01 10:00”, the date and time information in the related record in the second row is “2013/08/01 09:45”, and the third operation screen Elapsed time after transmitting is 15 minutes. If the threshold value for determining whether or not the change area has changed is 10 minutes, the change area included in the third operation screen transmitted to portable information device 200A changes in the third stage. Will be. Therefore, MFP 100 transmits a non-moving area transmission request to portable information device 200A. In response to receiving the non-moving area transmission request from MFP 100, portable information device 200A extracts the non-moving area from the third operation screen displayed on display unit 206 at that time. The data is transferred to the portable information device 200 specified by the device identification information “IP address A” included in the non-moving area transmission request.

  Further, MFP 100 refers to the related information shown in FIG. 17, and includes an operation including a variable area included in the third operation screen and a variable area where the information to be displayed is the same variable area and the content has not changed. It is determined whether or not a device displaying a screen exists other than portable information device 200A. Since such portable information device 200A does not exist, MFP 100 generates a variable region of the third operation screen, and transmits the generated variable region to portable information device 200 that has transmitted the screen request.

  Portable information device 200 displays a composite image obtained by combining the immovable region received from portable information device 200 </ b> A and the variable region received from MFP 100 on display unit 206 as an operation screen.

  In addition, with reference to the related information shown in FIG. 17, MFP 100 performs an operation including a variable area included in the third operation screen and a variable area where the information to be displayed is the same variable area and the content has not changed. When it is determined that there is no device displaying the screen other than the portable information device 200A, if the load on the CPU 111 is higher than the threshold value TH, a non-existence signal is sent to the portable information device 200 that has transmitted the screen request. Send.

  The portable information device 200 displays the immovable area received from the portable information device 200A on the display unit 206 as an operation screen. In this case, the fluctuation region is not displayed and is, for example, black or white. Further, a message indicating that the fluctuation area is not displayed may be displayed.

  As described above, MFP 100 according to the present embodiment functions as an image processing apparatus that can be remotely controlled by portable information devices 200, 200A, and 200B. In response to receiving a screen request from any of portable information devices 200, 200A, and 200B, MFP 100 transmits a screen for operation specified by the screen request to an instruction device that has transmitted the screen request. And an association means for associating screen identification information for identifying the operation screen with apparatus identification information for identifying the pointing device in response to the operation screen being transmitted by the screen transmission means, and a plurality of remote Based on the execution command received from any of the operation devices, based on the execution command and the operation screen specified by the screen identification information associated with the device identification information of the remote operation device that has transmitted the execution command A process specifying means for specifying the process and a remote control means for executing the specified process, wherein the screen transmitting means is a device identification different from the device identification information of the pointing device. If the information is associated with the screen identification information of the operation screen specified by the screen request, the remote operation device specified by the device identification information associated with the screen identification information of the operation screen specified by the screen request Request device determination means to determine the request device, and when the request device is determined, the operation screen specified by the screen request is associated with the screen identification information of the operation screen specified by the screen request without being transmitted to the instruction device. Requesting device that requests the requesting device specified by the device identification information to be transmitted to the instruction device of the operation screen specified by the screen request, and the associating device requests the requesting device by the requesting device. In response, the screen identification information for identifying the operation screen is associated with the device identification information for identifying the pointing device.

  According to this aspect, in response to a screen request being received from any of portable information devices 200, 200A, 200B, the operation screen is transmitted to the instruction device that has transmitted the screen request, and the screen identification of the operation screen is performed. The association information that associates the information with the device identification information of the instruction device is stored in the RAM 114. For example, when a screen request is received from the portable information device 200, the portable information device 200 is set as an instruction device. And when the device identification information different from the device identification information of the instruction device is associated with the screen identification information of the operation screen, the remote operation device identified by the device identification information associated with the screen identification information of the operation screen For example, the portable information device 200A is determined as the requesting device, and the portable information device 200A that is the requesting device is requested to transmit the operation screen specified by the screen request to the portable information device 200 that is the instruction device. For this reason, since portable information device 200A as a requesting device transmits an operation screen to portable information device 200 as an instruction device, there is no need to execute processing for transmitting the operation screen to the instruction device. Therefore, it is possible to prevent an increase in the load due to the transmission of the operation screen due to an increase in the number of remotely operated remote control devices.

  In addition, when the operation screen specified by the screen request received from the portable information device 200 that is the instruction device includes a variable area that changes with time, the MFP 100 displays the portable information device 200A that is the requesting device. Device identification information associated with the screen identification information of the operation screen including the same area as the fluctuation area when the fluctuation area of the current operation screen is the same as the fluctuation area at the current time. The mobile information device 200B, such as the portable information device 200B, is determined as the variable region transmitting device, and is transmitted to the portable information device 200A, which is the requesting device, to the portable information device 200, which is an instruction device for the non-moving region other than the variable region. And requests the portable information device 200B, which is a variable region transmission device, to transmit to the portable information device 200, which is a variable region instruction device. Therefore, even when a part of the screen transmitted to the portable information device 200A as the requesting device changes with time, the operation screen can be displayed on the portable information device 200 as the instruction device. .

  In addition, when the operation screen specified by the screen request received from the portable information device 200 that is the instruction device includes a variable area that changes with time, the MFP 100 displays the portable information device 200A that is the requesting device. If the variable area of the operation screen being displayed is not the same as the variable area at the current time, and if the variable area transmitting device is not determined, the variable area is generated and the variable area is transmitted to the portable information device 200 that is the pointing device. For this reason, MFP 100 only needs to generate a variable region, so that the processing load can be reduced compared to the case where the entire operation screen is generated.

  In addition, when the operation screen specified by the screen request received from the portable information device 200 that is the instruction device includes a variable area that changes with time, the MFP 100 displays the portable information device 200A that is the requesting device. If the variable area of the operation screen being displayed is not the same as the variable area at the current time, or if the variable area transmitting device is not determined, and if the load on the CPU 111 is greater than or equal to a predetermined value, the portable information device 200 as the pointing device has A non-existence signal indicating non-existence is transmitted. For this reason, since MFP 100 does not generate a variable region, the processing load can be reduced.

  Further, when there are a plurality of device identification information associated with the screen identification information of the operation screen specified by the screen request, the mobile information device 200 that is an instruction device among the remote operation devices specified by the plurality of device identification information. The remote control device that approximates the resolution of the display unit 206 is determined as the requesting device. For this reason, in the portable information device 200 that is an instruction device, it is not necessary to execute a process of converting the resolution on the operation screen received from the requesting device, so that the load can be reduced.

  Further, when there are a plurality of device identification information associated with the screen identification information of the operation screen specified by the screen request, the remote operation device having the fastest processing speed among the remote operation devices specified by the plurality of device identification information is selected. Decide on the requesting device. For this reason, the time for transmitting the operation screen by the requesting apparatus can be shortened as much as possible.

  Further, when there are a plurality of device identification information associated with the screen identification information of the operation screen specified by the screen request, the portable information device 200 that is an instruction device among the remote operation devices specified by the plurality of device identification information The remote control device with the shortest distance is determined as the requesting device. For this reason, it may be possible to shorten the time for transmitting the operation screen from the requesting device to the portable information device 200 that is an instruction device.

  Further, on the condition that the number of remote control devices having established communication paths among the portable information devices 200, 200A, 200B is equal to or greater than a predetermined number, an operation screen is displayed on the requesting device of the portable information devices 200, 200A, 200B. Request transmission to the pointing device. For this reason, there is a high probability that there is a requesting device displaying an operation screen to be displayed on the pointing device. Since it is not necessary to determine whether or not there is a requesting device in a state where there is a low probability that there is a requesting device displaying an operation screen to be displayed on the instruction device, the efficiency of the system can be improved.

  In the above-described embodiment, the MFP 100 is described as an example of the image processing apparatus, and the portable information devices 200, 200A, and 200B are illustrated as examples of the remote operation apparatus. However, the remote information illustrated in FIGS. Remote control method for causing MFP 100 to execute control processing, remote control program for causing CPU 111 to control MFP 100 to execute the remote control method, and remote control method for causing portable information devices 200, 200A, and 200B to execute the processing illustrated in FIG. Needless to say, the invention can be understood as a remote operation program that causes the CPU 201 to control the portable information devices 200, 200A, and 200B to execute the remote control method.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

<Appendix>
(1) Screen identification information associated with the execution command and device identification information of the remote control device that has transmitted the execution command in response to an execution command received from any of the plurality of remote control devices A process specifying means for specifying a process based on the operation screen specified in
The image processing apparatus according to claim 1, further comprising remote control means for executing the specified process.
(2) The association means includes screen identification information for identifying the operation screen and device identification information for identifying the instruction device in response to a request from the requesting device to the requesting device. The image processing apparatus according to claim 1, wherein the knight is associated with the image processing apparatus.
(3) In response to receiving an execution command from any of the plurality of remote control devices, the execution command and screen identification information associated with device identification information of the remote control device that has transmitted the execution command A process specifying step for specifying a process based on the operation screen specified in
The remote control method according to claim 9, further comprising a remote control step of executing the specified process.
(4) The association step includes screen identification information for identifying the operation screen and device identification information for identifying the instruction device in response to a request from the requesting device in the requesting step. The remote control method according to claim 9 or (3), further comprising the step of associating.
(5) Screen identification information associated with the execution command and device identification information of the remote control device that has transmitted the execution command in response to the execution command being received from any of the plurality of remote control devices. A process specifying step for specifying a process based on the operation screen specified in
The remote control program according to claim 10, further causing the computer to execute a remote control step of executing the specified process.
(6) The association step includes screen identification information for identifying the operation screen and device identification information for identifying the instruction device in response to a request from the requesting device in the requesting step. The remote control program according to claim 10 or (5), further causing the computer to execute a step of associating.

  1 Print System, 3 Network, 100 MFP, 200, 200A, 200B Portable Information Device, 300 PC, 5 Wireless Station, 110 Main Circuit, 111 CPU, 112 Communication I / F Unit, 113 ROM, 114 RAM, 115 HDD115 Facsimile Unit 117 External storage device, 118 CD-ROM, 120 Automatic document feeder, 130 Document reading unit, 140 Image forming unit, 150 Paper feed unit, 155 Post-processing unit, 160 Operation panel, 161 Display unit, 163 Operation unit, 165 Touch panel, 167 Hard key section, 201 CPU, 202 Camera, 203 Flash memory, 204 Wireless communication section, 205 Call section, 206 Display section, 207 Operation section, 207A Main key, 207B Touch panel, 208 Wireless LAN I / F, 209 position detection unit, 211 external storage device, 51 communication path establishment unit, 53 screen request reception unit, 55 screen transmission unit, 57 association unit, 59 execution command reception unit, 61 process identification unit, 63 remote control unit, 71 request Device determination unit, 73 actual screen transmission unit, 75 determination unit, 77 request unit, 79 variable region generation unit, 81 variable region transmission unit, 83 non-existence notification unit, 91 full screen transmission request unit, 93 non-moving region transmission request unit, 95 Fluctuating area transmitting device determining section, 97 Fluctuating area transmission requesting section, 251 Screen request section, 253 Screen receiving section, 255 Display control section, 257 Screen transfer section, 261 Full screen receiving section, 263 Non-moving area receiving section, 265 Fluctuating area Receiving unit, 267 absence signal receiving unit, 269 combining unit, 271 full screen transmission request receiving unit, 273 full screen transmitting unit, 275 non-moving area transmission request receiving A transmission unit, a 277 immovable region generation unit, a 279 immovable region transmission unit, a 281 variable region transmission request reception unit, an 283 variable region generation unit, and a 285 variable region transmission unit.

Claims (10)

An image processing device that can be remotely controlled by a plurality of remote control devices,
Screen transmission means for transmitting an operation screen specified by the screen request to an instruction device that has transmitted the screen request in response to a screen request being received from any of the plurality of remote operation devices;
And an association means for associating screen identification information for identifying the operation screen with apparatus identification information for identifying the pointing device in response to the operation screen being transmitted by the screen transmission means. ,
The screen transmission means, when device identification information different from the device identification information of the pointing device is associated with screen identification information of the operation screen specified by the screen request, the operation screen specified by the screen request Requesting device determining means for determining the remote operation device specified by the device identification information associated with the screen identification information of the requesting device;
When the request device is determined, the device identification information associated with the screen identification information of the operation screen specified by the screen request is transmitted without transmitting the operation screen specified by the screen request to the instruction device. An image processing apparatus comprising: a requesting unit that requests the specified requesting device to transmit the operation screen specified by the screen request to the instruction device. The association means further associates the time when the operation screen specified by the screen identification information is transmitted to the remote operation device of the device identification information associated with the screen identification information in association with the screen identification information,
When the operation screen specified by the screen request includes a variable region that changes with time, the screen transmission means receives the time associated with the device identification information of the requesting device and the screen request. And a determination means for determining whether or not the variation areas are the same based on
When it is determined that the variable areas are not the same, the requesting unit is configured to identify a device associated with the screen identification information of the operation screen including the same area as the variable area based on the time when the screen request is received. A variable region transmitting device determining means for determining a remote control device specified by information as a variable region transmitting device;
A non-moving area transmission requesting unit that requests the requesting apparatus to transmit to the pointing device a non-moving area other than the variable area of the operation screen specified by the screen request;
The image processing apparatus according to claim 1, further comprising: a change area transmission requesting unit that requests the determined change area transmitting apparatus to transmit the change area to the instruction device. The screen transmitting means, when it is determined that the variable areas are not the same, if the variable area transmitter is not determined by the variable area transmitter determining means, a variable area generating means for generating a variable area;
The image processing apparatus according to claim 2, further comprising: a fluctuation area transmitting unit that transmits the generated fluctuation area to the pointing device.   The screen transmission means indicates that the change area does not exist in the instruction device when the change area transmitter is not determined by the change area transmitter determining means when it is determined that the change areas are not the same. The image processing apparatus according to claim 2, further comprising non-existence notifying means for transmitting a signal.   When there are a plurality of pieces of device identification information associated with the screen identification information of the operation screen specified by the screen request, the screen transmission unit is configured to display an image among a plurality of remote control devices respectively corresponding to the plurality of device identification information. 5. The image processing apparatus according to claim 1, further comprising: a requesting device determining unit that determines, as the requesting device, a remote control device that has a resolution for displaying an image that approximates the pointing device.   The requesting device determination means, when there are a plurality of device identification information associated with the screen identification information of the operation screen specified by the screen request, out of a plurality of remote operation devices respectively corresponding to the plurality of device identification information The image processing apparatus according to claim 1, wherein a remote operation apparatus having the fastest processing speed is determined as the requesting apparatus. Position detecting means for detecting the position of each of the plurality of remote control devices, further comprising:
The requesting device determination means, when there are a plurality of device identification information associated with the screen identification information of the operation screen specified by the screen request, out of a plurality of remote operation devices respectively corresponding to the plurality of device identification information The image processing apparatus according to claim 1, wherein a remote operation apparatus having a shortest distance from the instruction apparatus is determined as the request apparatus. Path establishment means for establishing a communication path with the remote control device in response to a connection request received from any of the plurality of remote control devices;
The requesting device determining means requests transmission of the operation screen specified by the screen request to the instruction device on the condition that the number of remote operation devices with which the communication path is established is a predetermined number or more. The image processing apparatus according to claim 1. A remote control method executed by an image processing apparatus capable of being remotely controlled by a plurality of remote control devices,
A screen transmission step of transmitting an operation screen specified by the screen request to an instruction device that has transmitted the screen request in response to a screen request being received from any of the plurality of remote operation devices;
An association step for associating screen identification information for identifying the operation screen with device identification information for identifying the pointing device in response to the operation screen being transmitted in the screen transmission step. ,
In the screen transmission step, when device identification information different from the device identification information of the instruction device is associated with screen identification information of the operation screen specified by the screen request, the operation screen specified by the screen request A request device determining step for determining a remote operation device specified by the device identification information associated with the screen identification information of the request device as a request device;
When the request device is determined, the device identification information associated with the screen identification information of the operation screen specified by the screen request is transmitted without transmitting the operation screen specified by the screen request to the instruction device. And a requesting step of requesting the specified requesting device to transmit the operation screen specified by the screen request to the instruction device. A remote control program executed by a computer for controlling an image processing apparatus that can be remotely controlled by a plurality of remote control devices,
A screen transmission step of transmitting an operation screen specified by the screen request to an instruction device that has transmitted the screen request in response to a screen request being received from any of the plurality of remote operation devices;
An association step for associating screen identification information for identifying the operation screen with device identification information for identifying the pointing device in response to the operation screen being transmitted in the screen transmission step; Let the computer run,
In the screen transmission step, when device identification information different from the device identification information of the instruction device is associated with screen identification information of the operation screen specified by the screen request, the operation screen specified by the screen request A request device determining step for determining a remote operation device specified by the device identification information associated with the screen identification information of the request device as a request device;
When the request device is determined, the device identification information associated with the screen identification information of the operation screen specified by the screen request is transmitted without transmitting the operation screen specified by the screen request to the instruction device. A requesting step of requesting the specified requesting device to transmit the operation screen specified by the screen request to the pointing device.

Images