JP2016072784A - Image processing device, mobile device, and system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology for suppressing complication of display in making information on an image processing device be displayed on a touch panel of a mobile device.SOLUTION: A MFP 100 transmits image data to a mobile device 200 to make an image of the image data be displayed on a touch panel 61 of the mobile device 200. In displaying the image of the image data, the MFP 100 acquires information on a display size of the touch panel 61 of the mobile device 200 from the mobile device 200. When a prescribed size determined in advance is smaller than a prescribed ratio of a size obtained from the size information, the MFP 100 transmits, to the mobile device 200, a first instruction to make the image of the image data and an operation screen, which has a prescribed size and makes it possible to perform operation on the image, be displayed on the touch panel 61 using one screen.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing device, a mobile device, and a system. More specifically, the present invention relates to a technique for remotely operating an image processing apparatus by operating a mobile device.

  Conventionally, in a system including an image processing device such as a multifunction peripheral and a mobile device such as a mobile phone, the mobile device acquires image data from the image processing device, and the image of the image data and the A technique for remotely operating an image processing apparatus by displaying an operation screen that allows a user to operate image data and operating the operation screen using a touch panel of a mobile device is known.

  For example, Patent Literature 1 is a literature related to remote control. Japanese Patent Application Laid-Open No. 2004-133620 is a multifunction device that prints and outputs an image according to an instruction from a mobile device, acquires display size information of a touch panel of the mobile device, and converts the image size of the image by a magnification based on the display size information. A configuration is disclosed in which a converted image is displayed on a mobile device as a preview image.

JP 2004-140420 A

  However, the conventional technique described above has the following problems. That is, if a lot of information such as images and operation screens are displayed on the touch panel of the mobile device, the display tends to be complicated. This complication of display in the mobile device becomes more significant as the display size of the touch panel of the mobile device is smaller, and there is room for improvement.

  The present invention has been made to solve the above-described problems of the prior art. That is, an object of the present invention is to provide a technology that suppresses complication of display when information of an image processing device is displayed on a touch panel of a mobile device.

  An image processing apparatus for solving this problem includes a communication unit capable of communicating with a mobile device having a touch panel, and a control unit, and the control unit is wirelessly connected to the mobile device via the communication unit. In the acquired state, the acquisition request for acquiring the display size information of the touch panel of the mobile device is transmitted to the mobile device using the communication unit, and the acquisition request is transmitted in the acquisition request transmission processing. In the display size receiving process for receiving the display size information transmitted from the mobile device using the communication unit, and wirelessly connected to the mobile device via the communication unit, Image data transmission processing for transmitting data to the mobile device using the communication unit, and a predetermined size determined by the display size information If the image data is smaller than a predetermined ratio, the user's operation using the touch panel of the mobile device with respect to the image of the image data transmitted in the image transmission process and the image of the image data of the predetermined size is performed. And a first instruction transmission process for transmitting a first instruction to display the operation screen to be enabled on the touch panel of the mobile device on a single screen to the mobile device using the communication unit. Yes.

  An image processing device disclosed in the present specification transmits image data to a mobile device, and displays an image of the image data on a touch panel of the mobile device. The image processing apparatus acquires display size information of the touch panel of the mobile device from the mobile device when displaying the image of the image data on the mobile device. Then, when the predetermined size is smaller than a predetermined ratio of the size based on the display size information, the image processing device makes an image of the image data and an operation on the image having the predetermined size for the mobile device. A first instruction to display the operation screen that enables the touch screen on one screen is transmitted. Operations for image data include image processing settings such as pen writing, text input, color adjustment, and trimming, input of instructions to enter the processing mode for performing the processing, and processing execution instructions. Applicable.

  That is, when the predetermined size is larger than a predetermined ratio of the display size information size of the touch panel of the mobile device, the information displayed is complicated when the mobile device displays the image of the image data and the operation screen of the predetermined size on one screen. It is difficult to confirm information. Therefore, in the image processing apparatus disclosed in the present specification, when the predetermined size is smaller than the predetermined ratio of the size of the display size information, the image of the image data and the operation screen are displayed on one screen on the touch panel of the mobile device. . This can reduce the possibility of complicated display.

  The image processing apparatus disclosed in the present specification includes an operation unit, and the control unit transmits the image in the image transmission process when the predetermined size is larger than a predetermined ratio of the size according to the display size information. Second instruction transmission processing for transmitting a second instruction to display the image of the image data on the touch panel of the mobile device to the mobile device via the communication unit, and the predetermined size is the display size information A main body side operation process for allowing an operation on the image of the image data displayed on the touch panel of the mobile device to be operated via the operation unit when the size is larger than a predetermined ratio of The operation information transmission process which transmits the information operated by the process to the mobile device via the communication unit may be executed. When the predetermined size is smaller than the predetermined ratio of the size based on the display size information, the display ratio of the image data is small, and the image of the image data is difficult to see. According to this configuration, when the predetermined size is larger than the predetermined ratio of the size of the display size information on the touch panel of the mobile device, the image processing device does not transmit the operation screen display instruction to the mobile device, and By making the operation possible on the main body side, it becomes easier to view images on the mobile device.

  In addition, when the control unit receives the fact that the first instruction is rejected via the communication unit after transmitting the first instruction by the first instruction transmission process, the control unit receives the second instruction, It may be transmitted to the mobile device via a communication unit. Even if the image processing apparatus transmits the first instruction, the operation screen may not be displayed because the mobile device does not support the operation screen. According to this configuration, even if the first instruction is rejected, the possibility that at least an image of the image data can be displayed is increased by transmitting the second instruction.

  In addition, the operation unit includes a touch panel, and the control unit, in the main body side operation process, outputs at least a processing command for an image of the image data transmitted in the image transmission process and an output of the image data for the image. It is good to display the main body side operation screen which enables a user's input using the said touch panel of the said operation part on the said touch panel of the said operation part. By inputting image data processing commands, etc., from the touch panel on the main unit, the display burden on the mobile touch panel can be reduced.

  The control unit may execute the first instruction when the process execution command received from the mobile device is a process execution command that does not require display of an image of image data on the touch panel of the mobile device. If the execution instruction of the process that executes the transmission process and is received from the mobile device is not the execution instruction of the process that does not require the display of the image data on the touch panel of the mobile device, the first instruction transmission process Should not be executed. If the execution command is a process that does not need to display an image of the image data on the touch panel of the mobile device, it is preferable not to transmit the first display instruction because it is not necessary to display the image of the image data or the operation screen.

  A control method, a system, a computer program, and a computer-readable storage medium storing the computer program for realizing the functions of the image processing apparatus are also novel and useful.

  ADVANTAGE OF THE INVENTION According to this invention, the technique which suppresses the complication of a display when displaying the information of an image processing apparatus on the touchscreen of a mobile apparatus is implement | achieved.

1 is a block diagram illustrating a configuration of an image processing system according to an embodiment. FIG. 6 is a sequence diagram illustrating a copying procedure in the image processing system according to the embodiment. It is a figure which shows the operation screen of a mobile apparatus provided with a large sized touch panel. It is a figure which shows the operation screen of a mobile apparatus provided with a small size touch panel. FIG. 6 is a diagram illustrating an operation screen of the multifunction machine. 6 is a flowchart illustrating a first half procedure of remote execution processing executed by the multifunction peripheral according to the embodiment; 6 is a flowchart illustrating a second half of a remote execution process executed by the multifunction peripheral according to the embodiment. 6 is a flowchart illustrating the first half of a screen display process executed by the multifunction peripheral according to the embodiment. It is a flowchart which shows the procedure of the remote request | requirement process which the mobile apparatus concerning embodiment performs. It is a flowchart which shows the procedure of the second half of the 1st display process which the mobile apparatus concerning embodiment performs. It is a flowchart which shows the procedure of the second half of the 2nd display process which the mobile apparatus concerning embodiment performs.

  Hereinafter, the image processing apparatus according to the present embodiment will be described in detail with reference to the accompanying drawings. In the present embodiment, the present invention is applied to a multi function peripheral (MFP) having a reading function, a printing function, and a facsimile function.

[Overall configuration of image processing system]
As shown in FIG. 1, the image processing system 900 of this embodiment includes an MFP 100 and a mobile device 200. Any number of devices that communicate with the MFP 100 may be connected in addition to the mobile device 200 illustrated. MFP 100 is an example of an image processing apparatus. As the mobile device 200, for example, a smartphone or a tablet PC is applicable.

  In the image processing system 900 of this embodiment, data can be exchanged between the mobile device 200 and the MFP 100 by wireless communication. Data can be exchanged via an access point (not shown) or directly between devices.

[MFP configuration]
Next, a schematic configuration of MFP 100 will be described. As illustrated in FIG. 1, the MFP 100 includes a controller 30 including a CPU 31, a ROM 32, a RAM 33, and an NVRAM (Non Volatile RAM) 34. The controller 30 may be a part of the ASIC, or the controller 30 may include the ASIC. The controller 30 in FIG. 1 is a collective term for hardware used for controlling the MFP 100, such as the CPU 31, and is not necessarily represented as a single piece of hardware that actually exists in the MFP 100.

  The ROM 32 stores firmware, which is a control program for controlling the MFP 100, various settings, initial values, and the like. The RAM 33 and the NVRAM 34 are used as a work area from which various control programs are read or as a storage area for temporarily storing various data such as image data.

  The CPU 31 controls each component of the MFP 100 while storing the processing result in the RAM 33 or the NVRAM 34 according to a control program read from the ROM 32 and signals sent from various sensors. The CPU 31 is an example of a control unit. The controller 30 may be an example of a control unit.

  The MFP 100 includes an image forming unit 10 that prints an image on a sheet, an image reading unit 11 that reads an image of a document, an operation unit 40, a wireless communication interface 36, and a telephone interface 37, which are CPU 31. Controlled by. In addition to these, the MFP 100 may include, for example, a wired LAN interface or a USB interface.

  The image forming unit 10 may be capable of color printing or only monochrome printing. Also, the printing method may be an electrophotographic method or an ink jet method. Further, the image reading unit 11 may be capable of color scanning or only monochrome scanning. Also, the reading mechanism may be a CCD or a CIS.

  The wireless communication interface 36 is hardware that enables communication with an external device by wireless communication. The wireless communication interface 36 according to the present embodiment has a configuration that enables wireless communication by the WiFi system based on the IEEE 802.11 standard and standards equivalent thereto. Note that the wireless communication standard is not limited to WiFi, and may be another standard such as NFC. The MFP 100 receives data transmitted from the mobile device 200 via the wireless communication interface 36. Further, the MFP 100 transmits data to the mobile device 200 via the wireless communication interface 36. The wireless communication interface 36 is an example of a communication unit.

  The telephone line interface 37 includes an NCU that controls telephone line connection and a modem that modulates voice signals and digital signals, and is an interface that enables communication using a telephone line. The MFP 100 performs facsimile communication with an external device via the telephone line interface 37.

  The operation unit 40 is hardware that enables messages and various settings to be displayed and various information from the user to be input, and is provided on the exterior of the MFP 100. The operation unit 40 includes a touch panel 41 having a display function and an input function, and a button group 42 including various input means such as a numeric keypad, an arrow key, an OK button, and a cancel button.

[Configuration of mobile device]
Next, a schematic configuration of the mobile device 200 will be described. As shown in FIG. 1, the mobile device 200 includes a CPU 51 that executes various processes, a ROM 52 that stores a startup process program (BIOS) executed by the CPU 51 when the mobile device 200 is started, and the CPU 51 performs various processes. The controller 50 includes a RAM 53 that is used as a temporary storage area when performing, and a hard disk drive (HDD) 54 that stores various programs and data. The controller 50 in FIG. 1 is a collective term for hardware used for controlling the mobile device 200, such as the CPU 51, and does not necessarily represent a single piece of hardware that actually exists in the mobile device 200. Absent.

  The HDD 54 of the mobile device 200 incorporates an OS, a browser for browsing files on the Internet, device drivers for controlling various devices, and the like. In addition, an application program (hereinafter referred to as “application”) that receives an execution instruction for causing MFP 100 to execute printing, reading, data transfer, and the like is also stored.

  The CPU 51 performs various processes while storing the calculation results in the RAM 53 or the HDD 54 in accordance with the control program read from the ROM 52 or the program read from the HDD 54. The application operation described above is also processed by the CPU 51.

  The mobile device 200 includes a wireless communication interface 56, an operation unit 60, and a telephone unit 70, which are controlled by the controller 50. In addition to these, the mobile device 200 may include, for example, a USB interface.

  The operation unit 60 is hardware that allows a message and various settings to be displayed and various information from the user to be input, and is provided on the front surface of the mobile device 200. The operation unit 60 includes a touch panel 61 having a display function and an input function, and a button group 62 including various input means such as an OK button and a cancel button. For example, a job for printing an image or a job for reading an image is input to the mobile device 200 by an operation on the touch panel 61 or the button group 62 by the user.

  The wireless communication interface 56 is hardware that enables communication with an external device. The wireless communication interface 56 of this embodiment has a configuration that enables wireless communication based on the same standard as the wireless communication interface 36 of the MFP 100. Therefore, the mobile device 200 can input various instructions to the MFP 100 via the wireless communication interface 56. The mobile device 200 can accept various instructions from the MFP 100 via the wireless communication interface 56. The wireless communication interface 56 is an example of a communication unit.

  The telephone unit 70 is configured with hardware for realizing a telephone function such as a microphone, a speaker, a telephone line interface, and the like, and conforms to a mobile phone communication system (for example, W-CDMA) with a base station (not shown). Enables wireless communication based on the standard. The telephone unit 70 is used for telephone calls, Internet connection via a base station, and the like.

[Overview of remote communication]
Next, an operation of performing remote communication between the MFP 100 and the mobile device 200 in the image processing system 900 of this embodiment will be described with reference to the sequence diagram of FIG. First, mobile device 200 establishes a session in which communication with MFP 100 is permitted in order to perform remote communication with MFP 100.

  Specifically, when the MFP 100 is used from the mobile device 200, the above-described application is activated on the mobile device 200. The mobile device 200 displays a screen provided by the application on the touch panel 61 and accepts a request to start using the MFP 100 by operating the touch panel 61.

  When the mobile device 200 accepts the request, the mobile device 200 transmits an establishment request for establishing a session to the MFP 100 via the wireless communication interface 56. Then, when a session is established with MFP 100, mobile device 200 causes touch panel 61 to display a process selection screen that is provided by the application and that allows the user to select a process to be used in MFP 100. .

  When the MFP 100 receives a session establishment request from the mobile device 200 via the wireless communication interface 36, the MFP 100 establishes a session on condition that a condition for permitting communication is satisfied. The MFP 100 satisfies a condition for permitting communication when a session is not established with another device or when establishment of a session with the mobile device 200 is not restricted.

  The MFP 100 receives a process execution command from the mobile device 200 via the wireless communication interface 36 while the session with the mobile device 200 is established. In other words, MFP 100 does not receive a process execution command from mobile device 200 when a session with mobile device 200 is not established. When the session with the mobile device 200 is established, the MFP 100 displays a screen displaying a message indicating that communication with another device is in progress on the touch panel 41, and inputs an execution command by operating the operation unit 40. Will not accept.

  In the mobile device 200, the process to be used in the MFP 100 is selected by the user operating the touch panel 61 while the process selection screen is displayed. For example, the MFP 100 can perform print processing, scan processing, copy processing, and facsimile transmission processing, and one of these processing is selected. Here, the description will be continued assuming that the copy process is selected.

  When the copy process is selected, the mobile device 200 transmits a copy process execution command to the MFP 100 via the wireless communication interface 56. When the MFP 100 receives a copy processing execution command from the mobile device 200 via the wireless communication interface 36, the MFP 100 causes the image reading unit 20 to start reading a document set on the MFP 100. Then, the MFP 100 transmits the image data of the document read by the image reading unit 20 to the mobile device 200 via the wireless communication interface 36. After transmitting the image data, the MFP 100 does not start printing the image of the image data immediately, but waits for a start command input by the user. When the mobile device 200 receives the image data, the mobile device 200 stores the image data in the RAM 53.

  When the MFP 100 receives a copy processing execution command from the mobile device 200, the MFP 100 transmits an acquisition request for acquiring display size information on the touch panel 61 of the mobile device 200 via the wireless communication interface 36, and the mobile device 200 Display size information is received via the wireless communication interface 36. The size included in the display size information does not need to be the physical size of the touch panel 61, and may be the size of the portion involved in the display.

  Then, after receiving the display size information, the MFP 100 transmits a display instruction corresponding to the size indicated in the display size information to the mobile device 200 via the wireless communication interface 36. Specifically, MFP 100 determines a display instruction depending on whether a predetermined size is smaller than a predetermined ratio of the size based on the display size information, that is, the display size of touch panel 61 of mobile device 200.

  When the predetermined size is smaller than a predetermined ratio of the display size of the touch panel 61, the display size of the touch panel 61 is very large with respect to the predetermined size. Therefore, the MFP 100 instructs the mobile device 200 to display an image of the image data and an operation screen for shifting to the processing mode for performing the processing on the image of the image data on one screen. The first instruction is transmitted via the wireless communication interface 36. Examples of the processing include pen writing, character insertion, color adjustment, and trimming. The operation screen of this embodiment is equivalent to the above-mentioned predetermined size.

  When the mobile device 200 receives the first instruction via the wireless communication interface 56, the mobile device 200 displays the received image data image 63 and the operation screen 64 on one screen on the touch panel 61 as shown in FIG. Let The operation screen 64 includes a processing button 641 for inputting an instruction to shift to a processing mode that is a mode that enables processing of the image 63 of the image data, a start button 642 for starting printing, and a cancel button for canceling the copy processing. 643 is displayed. The image that accepts user input displayed on the operation screen 64 is not limited to a button but may be a specific symbol such as an icon.

  When the processing button 641 is touched by a user operation on the touch panel 61, the mobile device 200 switches the content of the operation screen 64 to, for example, a processing selection screen (not shown) that allows selection of the type of processing. Further, when the type of machining process is selected by the user's touch operation on the touch panel 61, the mobile device 200 switches to a machining setting screen (not shown) that allows input of settings relating to the selected type of machining process. Note that the input to the mobile device 200 may be performed using the button group 62 in addition to the touch panel 61. Further, the display may be switched from the process selection screen or the process setting screen to the first operation screen 64 shown in FIG.

  When the start button 642 is touched by a user operation on the touch panel 61, the mobile device 200 transmits a print start command to the MFP 100 via the wireless communication interface 56. When the processing of the image data is performed before the start button 642 is touched, the mobile device 200 adds the processed image data that is the processed image data to the above-described start command. When the MFP 100 receives the start command via the wireless communication interface 36, the MFP 100 causes the image forming unit 10 to start printing the image of the read image data or the processed image data added to the start command.

  In addition, after transmitting the start command, the mobile device 200 transmits a cancel request for canceling the session to the MFP 100 via the wireless communication interface 56. When the MFP 100 receives a cancellation request from the mobile device 200 via the wireless communication interface 36, the MFP 100 cancels the session established with the mobile device 200. As a result, another device can establish a session with MFP 100. In addition, the MFP 100 returns the display on the touch panel 41 to the initial screen, and also cancels the input restriction of the process execution command from the operation panel 40.

  If there is no response to an inquiry from the MFP 100 to the mobile device 200 in addition to a session cancellation request from the mobile device 200, or if there is no request from the mobile device 200 for a certain period of time, the MFP 100 cancels it via the button group 42. When the request is input, MFP 100 voluntarily releases the session.

  On the other hand, when the predetermined size is larger than a predetermined ratio of the display size of the touch panel 61, the display size of the touch panel 61 is not so large with respect to the predetermined size. For this reason, when the operation screen 64 is displayed on the touch panel 61, the area other than the operation screen 64 can be displayed, and there is a high possibility that information other than the information in the operation screen 64 such as the image 63 of the image data becomes difficult to see. Therefore, the MFP 100 transmits a second instruction, which is an instruction to display an image of the image data, to the mobile device 200 via the wireless communication interface 36.

  When the mobile device 200 receives the second instruction via the wireless communication interface 56, the mobile device 200 displays the image 63 of the received image data on the touch panel 61 as shown in FIG. That is, the mobile device 200 does not display the operation screen 64 as shown in FIG. Instead, the MFP 100 displays an operation screen 44 corresponding to the operation screen 64 of the mobile device 200 on the touch panel 41 of the MFP 100 as shown in FIG.

  The layout of the operation screen 44 is the same as the operation screen 64 of the mobile device 200 shown in FIG. That is, the operation screen 44 also displays a processing button 441 for inputting an instruction to shift to the processing mode, a start button 442 for starting printing, and a cancel button 443 for canceling processing. When the processing button 41 is touched by a user operation on the touch panel 41, the MFP 100 transmits a processing mode transition instruction to the mobile device 200 via the wireless communication interface 36.

  When the mobile device 200 receives a processing mode transition instruction from the MFP 100 via the wireless communication interface 56, the mobile device 200 causes the touch panel 61 to display a processing selection screen (not shown) on which, for example, a processing type can be selected. Furthermore, when a type of processing is selected by a user operation on the touch panel 61, the mobile device 200 displays a touch panel on a processing setting screen (not shown) that allows input of settings regarding the selected type of processing. Switch the display of 61. It should be noted that the MFP 100 may select up to the type of processing process, and the MFP 100 may transmit a processing mode transition process with the type of processing process added. In this case, the mobile device 200 causes the touch panel 61 to display a processing setting screen instead of the processing selection screen.

  When the mobile device 200 performs the processing, the mobile device 200 transmits the processed image data to the MFP 100 via the wireless communication interface 56. When the MFP 100 receives the processed image data via the wireless communication interface 36, the MFP 100 stores the processed image data in the RAM 33.

  When the start button 442 is touched by a user operation on the touch panel 41, the MFP 100 prints an image of the processed image data when the processed image data is received before the start button 442 is touched. When the processed image data is not received, the image forming unit 10 starts printing the image of the read image data.

  After the copy process is completed, the MFP 100 transmits an end notification to the mobile device 200 via the wireless communication interface 36. When the mobile device 200 receives the end notification via the wireless communication interface 56, the mobile device 200 transmits a release request to the MFP 100 via the wireless communication interface 56. After receiving the release request via the wireless communication interface 36, the MFP 100 releases the session established with the mobile device 200.

[Remote communication procedure]
Next, individual operations of MFP 100 and mobile device 200 that realize the above-described remote communication will be described.

[MFP operation]
First, as an operation of the MFP 100, a remote execution process that receives an execution instruction of a process from an external device and executes an image process will be described with reference to the flowcharts of FIGS. The remote execution process is executed by the CPU 31 when a session with the external device is established. In the following description of the remote execution process, it is assumed that a session with the mobile device 200 is established as an external device.

  In the remote execution process, first, as shown in FIG. 6, the CPU 31 determines whether or not a process execution command has been received from the mobile device 200 (S101). When the execution command has not been received (S101: NO), the CPU 31 determines whether or not an end command has been received (S102). When the end command is received (S102: YES), the remote execution process is ended. If an end command has not been received (S102: NO), the process proceeds to S101, and the CPU 31 waits for an execution command or an end command.

  When the execution instruction is received (S101: YES), the CPU 31 determines whether or not the process instructed to execute is a process that requires reading (S111). Examples of processing that requires reading include scan processing, copy processing, and fax transmission processing. An example of processing that does not need to be read is print processing. Even in the FAX transmission process, when the image data to be transmitted or the reference destination of the image data is added to the execution command, the process is not required to be read. In the case of processing that requires reading (S111: YES), the CPU 31 causes the image reading unit 20 to start reading an image of the document (S112).

  In the case of processing that does not require reading (S111: NO), the CPU 31 determines whether image data is added to the execution command (S121). When the image data is added (S121: YES), the CPU 31 acquires the image data (S131). Then, processing is executed using the image data (S132). In S132, for example, in the case of a printing process, the image forming unit 10 is caused to print an image of the image data. In the case of FAX transmission processing, facsimile transmission based on the image data is performed using the telephone interface 37. After S132, the remote execution process is terminated.

  On the other hand, when the image data is not added (S121: NO), the execution instruction is added with the information of the reference destination of the image data to be processed, that is, the storage destination where the image data is stored. Therefore, the CPU 31 reads image data from the reference destination into the RAM 33 (S122). The image data storage destination may be the NVRAM 34 of the MFP 100 itself or an external device.

  After S112 or S122, the CPU 31 transmits the acquired image data to the mobile device 200 (S113). S113 is an example of image data transmission processing. Then, the CPU 31 temporarily stores the image data to be transmitted in a predetermined storage area of the RAM 33 (S114). The storage destination is not limited to the RAM 33, and may be the NVRAM 34 or an external device.

  After S114, the CPU 31 determines whether the preview function is valid (S141). When the MFP 100 acquires the image data by reading or when reading and acquiring the image data from the reference destination, as described above, the image of the acquired image data is not printed immediately, and the acquired image data is temporarily displayed on the mobile device 200. Then, when the user confirms the content and touches the start button, the processing for the image data is continued. The mobile device 200 can add a setting for enabling or disabling such a preview function to the execution command. Therefore, the CPU 31 determines that the preview function is invalid when a setting for invalidating the preview function is added. Further, for example, in the scan process, when the scan process that does not perform the preview is the direct scan process and is distinguished from the scan process that performs the preview, the invalidation of the preview function may be determined depending on the type of the process. In the case of facsimile transmission, since the necessity for processing tends to be low, the preview function may be determined to be invalid.

  When the preview function is invalid (S141: NO), the CPU 31 continues the process without waiting for the input of the start command (S151). For example, printing is started for copy processing, image data is saved for scanning processing, and facsimile transmission based on the image data is started for FAX transmission processing. After the process is completed, the CPU 31 deletes the image data stored in S114 (S152) and ends the remote execution process.

  On the other hand, when the preview function is valid (S141: YES), the CPU 31 executes a screen display process for realizing the preview function (S142). The screen display process will be described with reference to the flowchart of FIG.

  In the screen display process of S142, the CPU 31 first transmits an acquisition request for requesting acquisition of display size information on the touch panel 61 of the mobile device 200 to the mobile device 200 (S001). S001 is an example of an acquisition request transmission process. Then, the CPU 31 receives and acquires display size information transmitted from the mobile device 200 (S002). S002 is an example of display size reception processing. Further, the CPU 31 reads and acquires a predetermined size determined in advance from the ROM 32 or the NVRAM 34 (S003).

  The predetermined size is defined by the size of the operation screen 64 displayed on the mobile device 200. In this embodiment, the predetermined size is made equal to the size of the operation screen 64. Note that the predetermined size may be a predetermined size of the operation screen 64. In addition, the MFP 100 ensures a size that allows various buttons such as the processing button 641 to be easily operated by a user's finger as a predetermined size. For example, in the present embodiment, since three buttons are displayed in a horizontal row on the operation screen 64, the predetermined size is a size that can display at least one button in the vertical direction and at least three buttons in the horizontal direction. Become.

After S003, the CPU 31 determines whether or not the predetermined size is smaller than a predetermined ratio of the size based on the display size information received in S002 (S021). For example, if the area based on the predetermined size is S1, the area based on the display size of the touch panel 61 is S2, and the predetermined ratio is R, it is determined whether or not the following relationship (1) is satisfied.
S1 <S2 × R (1)

  When the predetermined size defined by the size of the operation screen 64 is not smaller than the predetermined ratio of the size based on the display size information on the touch panel 61, that is, when the relationship (1) is not satisfied (S021: NO), the CPU 31 A second instruction for displaying the image of the image data, that is, a second instruction for not displaying the operation screen 64 is transmitted to the mobile device 200 (S022). S022 is an example of a second instruction transmission process.

  Furthermore, the CPU 31 displays the operation screen 44 on the touch panel 41 as shown in FIG. 5 (S023). S023 is an example of the main body side operation process. By sharing the display between the mobile device 200 and the MFP 100, the image 63 of the image data and the operation screen 44 can be displayed in an easily viewable manner. S022 and S023 may be in reverse order or simultaneously.

  On the other hand, when the predetermined size is smaller than the predetermined ratio of the size, that is, when the above relationship (1) is satisfied (S021: YES), the CPU 31 displays the image 63 of the image data and the operation screen 64 on one screen. 1 instruction is transmitted to the mobile device 200 (S031). S031 is an example of a first instruction transmission process.

  In addition, after transmitting the first instruction, it is determined whether or not a message indicating that the display of the first instruction is rejected is received from the mobile device 200 (S032). Even if the mobile device 200 receives the first instruction, if the mobile device 200 does not support the display of the operation screen 64, for example, a program for displaying the operation screen 64 is not incorporated, the mobile device 200 responds that the display is rejected. .

  When the CPU 31 receives that the display of the first instruction is rejected (S032: YES), the CPU 31 transmits the second instruction to the mobile device 200 (S022), and displays the operation screen 44 on the touch panel 41 (S023). . This increases the possibility that the image 63 of the image data can be displayed on the mobile device 200. After S023, or when the fact that the display of the first instruction is rejected is not received (S032: NO), the image display process is terminated and the process returns to the remote execution process of FIG.

  Returning to the description of the remote execution process, the process proceeds to FIG. 7 after S142, and the CPU 31 determines whether or not a start command is input (S161). In S161, if the operation screen 64 is displayed on the touch panel 61 of the mobile device 200, it is determined whether or not the start command transmitted from the mobile device 200 is received in response to the touch on the start button 642, and the operation screen 64 If 44 is displayed on touch panel 41 of MFP 100, it is determined whether start button 442 has been touched or not.

  If the start command has not been input (S161: NO), the CPU 31 determines whether or not the operation screen 44 is being displayed on the touch panel 41 (S162). When the operation screen 44 is being displayed (S162: YES), the CPU 31 determines whether or not the processing button 441 included in the operation screen 44 has been touched (S163).

  When the processing button 441 is touched (S163: YES), the CPU 31 transmits a processing mode transition instruction to the mobile device 200 (S164). S164 is an example of operation information transmission processing. The mobile device 200 shifts to the machining mode upon receiving the machining mode transition instruction and accepts input of various machining processing settings or execution commands. Then, the CPU 31 receives the processed image data transmitted from the mobile device 200 and stores it in the RAM 33 (S165). After S165, or when the processing button 441 is not touched (S163: NO), or when the operation screen 44 is not displayed on the touch panel 41 (S162: NO), the process proceeds to S161, and the CPU 31 performs various inputs. Wait for.

  When the start command is input (S161: YES), the CPU 31 determines whether or not the processed image data is added to the start command (S171). If the processed image data has been added (S171: YES), the CPU 31 receives the processed image data and stores it in the RAM 33 (S172).

  After S172 or when processed image data is not added (S171: NO), the CPU 31 determines whether or not the processed image data is stored in the RAM 33 (S181). The processed image data is temporarily stored in the RAM 33 in S165 or S172.

  When the processed image data is stored (S181: YES), the CPU 31 continues the processing using the processed image data (S183). That is, an image of processed image data is output. Then, after the processing is completed, the CPU 31 deletes the processed image data from the RAM 33 (S184). On the other hand, when the processed image data is not stored (S181: NO), the CPU 31 continues the processing using the image data saved in S114 (S191). That is, an image of the image data obtained in S112 or S122 is output. After S184 or after S191, the CPU 31 also deletes the image data stored in S114 (S185).

  After S185, the CPU 31 determines whether or not the operation screen 44 is being displayed on the touch panel 41 (S186). When the operation screen 44 is being displayed (S186: YES), the CPU 31 returns the display of the touch panel 41 to the initial screen (S187), and further transmits an end notification to the mobile device 200 (S188). After S188 or if the operation screen 44 is not being displayed (S186: NO), the remote execution process is terminated.

  Note that when an end command transmitted from the mobile device 200 is received in response to the touch of the cancel button 642 displayed on the touch panel 61 of the mobile device 200, or the operation screen 44 is being displayed on the touch panel 41. If the cancel button 443 is touched before the start button 441 is touched, the CPU 31 deletes the image data stored in S114, and if the processed image data is received, the processed image data is also deleted. . Furthermore, an end notification is transmitted to the mobile device 200, and the remote execution process is ended.

[Mobile device operation]
Next, as an operation of the mobile device 200, remote request processing for requesting execution of processing will be described with reference to the flowchart of FIG. The remote request process is executed by the CPU 51 when the application is activated and a use start request for the MFP 100 is received by an operation on the touch panel 61.

  In the remote request process, the CPU 51 first transmits a session establishment request to the MFP 100 to establish a session with the MFP 100 (S201). When the session is established, CPU 51 causes touch panel 61 to display a screen for accepting an execution command for processing to be executed by MFP 100 and an application end command. Processing execution instructions are accepted for each type of processing. If session establishment fails, retry may be performed, or an error may be displayed and the application may be terminated.

  After S201, the CPU 51 determines whether or not an execution command for processing to be executed by the MFP 100 has been received (S202). When the execution command has not been received (S202: NO), the CPU 51 determines whether an end command has been received (S203). If an end command has not been received (S203: NO), the process proceeds to S202, and the CPU 51 waits for an execution command or an end command. When the termination command is received (S203: YES), the CPU 51 transmits a session cancellation request to the MFP 100 to cancel the session (S291), and ends the remote request processing.

  When the execution command is received (S202: YES), the CPU 51 transmits an execution command related to the received process to the MFP 100 (S211). S211 is an example of an execution command transmission process.

  After S211, the CPU 51 determines whether the preview function is valid (S212). In S212, the determination may be made based on whether the preview function is enabled or disabled, or may be determined based on the type of processing. The setting for enabling or disabling the preview function is input by the user. If the preview function is disabled (S212: NO), the mobile device 200 does not display or process the image data, so the session is canceled (S291), and the remote request process is terminated.

  When the preview function is valid (S212: YES), the CPU 51 receives the image data transmitted from the MFP 100 when the MFP 100 starts executing the process (S213). S213 is an example of image reception processing. After S213, the CPU 51 transmits the display size information of the touch panel 61 to the MFP 100 in response to the acquisition request transmitted from the MFP 100 (S214). Then, the CPU 51 receives a display instruction transmitted from the MFP 100 that has received the display size information (S215). The display instruction is the first instruction or the second instruction described above. S215 is an example of a first instruction reception process.

  After S215, the CPU 51 determines whether or not the display instruction is the first instruction (S221). When the MFP 100 is the first instruction (S221: YES), the MFP 100 executes a first display process for displaying the image 63 of the image data and the operation screen 64 on one screen (S222), and is not the first instruction. In other words, if the instruction is the second instruction (S221: NO), the second display process of displaying the image 63 of the image data and not displaying the operation screen 64 is executed (S223).

  FIG. 10 shows the procedure of the first display process of S222. In the first display process, the CPU 51 first displays the image 63 of the image data and the operation screen 64 on the touch panel 61 as shown in FIG. 3 (S231). S231 is an example of a first display process. When the image 63 of the image data and the operation screen 64 are displayed on one screen, a lot of information can be displayed on one screen. Therefore, it is convenient for the user. Note that the CPU 51 may enlarge or reduce the image 63 of the image data in accordance with the display size of the touch panel 61.

  After S231, the CPU 51 determines whether or not the processing button 641 included in the operation screen 64 has been touched (S232). When the processing button 641 is touched (S232: YES), the CPU 51 shifts to the processing mode, switches the screen of the touch panel 61 to a processing selection screen in which the type of processing processing can be selected, and further selects the type of processing processing. Then, by switching to a processing setting screen that enables input of settings relating to the selected type of processing, various settings of the processing are accepted (S233). For example, if the type of processing is character insertion, at least two settings of a character and a character position are accepted. For example, if the type of processing is trimming, the setting of the trimmed range is accepted.

  After S233, the CPU 51 executes the processing according to the setting received in S233, generates processed image data, and stores it in the RAM 53 (S234). Then, the CPU 51 returns the touch panel screen to the operation screen 64 and displays the processed image data image 63 on the touch panel 61 (S235).

  After S235 or when the processing button 641 is not touched (S232: NO), the CPU 51 determines whether or not the start button 642 included in the operation screen 64 is touched (S236). When the start button 642 is not touched (S236: NO), the process proceeds to S232, and the CPU 51 waits for various inputs.

  When the start button 642 is touched (S236: YES), the CPU 51 determines whether or not the processed image data is stored in the RAM 53 (S271). That is, in S271, it is determined whether or not the machining process in the machining mode has been executed. When the processed image data is stored (S271: YES), the CPU 51 transmits a start command with the processed image data added to the MFP 100 (S273). On the other hand, when the processed image data is not stored (S271: NO), the CPU 51 transmits a start command for not adding the processed image data to the MFP 100 (S272). After S272 or S273, the first display process is terminated, and the process returns to the remote request process of FIG.

  FIG. 11 shows the procedure of the second display process in S223. In the second display process, the CPU 51 first displays the image 63 of the image data on the touch panel 61 as shown in FIG. 4 (S241). In S241, the operation screen 64 is not displayed on the touch panel 61. Then, the CPU 51 determines whether or not a processing mode transition instruction has been received from the MFP 100 (S242).

  When the machining mode shift instruction is received (S242: YES), the CPU 51 shifts to the machining mode and accepts various settings of the machining process (S251), similarly to S233. Then, the CPU 51 executes the processing according to the setting received in S251, and generates processed image data (S252). If the processing setting is added to the processing mode transition instruction, the CPU 51 may generate the processing image data according to the processing setting without shifting to the processing mode. After S252, the CPU 51 transmits the processed image data generated in S252 to the MFP 100 (S253).

  After S253 or when the processing mode shift instruction has not been received (S242: NO), the CPU 51 determines whether or not an end notification has been received from the MFP 100 (S243). When the end notification has not been received (S243: NO), the process proceeds to S242, and the CPU 51 waits for various receptions. When the end notification is received (S243: YES), the second display process is ended and the process returns to the remote request process of FIG.

  Returning to the description of FIG. 9, after the first display process of S222 or the second display process of S223 is completed, the CPU 51 releases the session (S291) and ends the remote request process.

  If the CPU 51 does not receive the next command or instruction even after a predetermined time has elapsed after receiving the command or instruction from the MFP 100, the CPU 51 forcibly releases the session and ends the remote request processing. When the cancel button 463 is touched while the operation screen 64 is being displayed, the CPU 51 transmits an end command instead of the start command, cancels the session, and ends the remote request processing.

  As described above in detail, in the MFP 100 according to the present embodiment, when display of an image of the image data on the touch panel 61 of the mobile device 200 is necessary, the display size information of the touch panel 61 is acquired from the mobile device 200 and the predetermined size is displayed. When the size information is smaller than a predetermined ratio of the size, a first instruction for displaying the image 63 of the image data and the operation screen 64 on the touch panel 61 of the mobile device 200 on one screen is transmitted. When the predetermined size is smaller than a predetermined ratio of the size of the display size information, even if the operation screen 64 having the predetermined size is displayed, information other than the operation screen 64 such as the image 63 of the image data may be difficult to see. Is low. Therefore, it is possible to avoid complicated display by transmitting the first instruction in such a case.

  Note that this embodiment is merely an example, and does not limit the present invention. Therefore, the present invention can naturally be improved and modified in various ways without departing from the gist thereof. For example, the image processing apparatus only needs to have at least one function for performing image processing, and may be, for example, a printer, a scanner, a facsimile apparatus, or a copier in addition to the MFP.

  In the embodiment, while MFP 100 establishes a session with mobile device 200, a session with another external device is not established, but a session may be established. However, it is preferable not to establish a session in order to avoid a conflict between a request from the mobile device 200 and a request from another external device. Further, an upper limit of the number of sessions to be established may be set, and if the number of external devices with which sessions are established is less than the upper limit, the session may be established.

  In the embodiment, while the session with the mobile device 200 is established, the MFP 100 restricts the input from the operation unit 40, but the input may not be restricted. However, it is preferable to limit the input in order to avoid a conflict between a request from the mobile device 200 and a request by direct input to the own device.

  In the embodiment, the process in which the preview display on the mobile device is set to invalid or the specific process such as the print process is a process that does not require the display of the image data. Processing in which the processing process in the mobile device is set to be invalid may be used.

  In the embodiment, when the second instruction is transmitted, the MFP 100 displays the operation screen 44 on its touch panel 41 and accepts an operation on the image of the image data. However, the button 100 may accept the operation. . In this case, the operation screen 44 may not be displayed on the touch panel 41.

  In the embodiment, an instruction to shift to the machining mode is accepted on the operation screen 64 to the operation screen 44, and thereafter the screen is switched to accept various settings for the machining process. Various processing settings may be received directly at.

  The processing disclosed in the embodiments may be executed by a single CPU, a plurality of CPUs, hardware such as an ASIC, or a combination thereof. Further, the processing disclosed in the embodiment can be realized in various modes such as a recording medium or a method recording a program for executing the processing.

36 Wireless communication interface 41 Touch panel 56 Wireless communication interface 61 Touch panel 100 MFP
200 Mobile device 900 Image processing system

Claims (7)

A communication unit capable of communicating with a mobile device having a touch panel;
A control unit;
With
The controller is
Acquisition request transmission processing for transmitting an acquisition request for acquiring display size information of the touch panel of the mobile device to the mobile device using the communication unit in a state of being wirelessly connected to the mobile device via the communication unit When,
A display size reception process for receiving the display size information transmitted from the mobile device using the communication unit after transmitting the acquisition request in the acquisition request transmission process;
Image data transmission processing for transmitting image data to the mobile device using the communication unit in a state of being wirelessly connected to the mobile device via the communication unit;
When the predetermined size is smaller than a predetermined ratio of the size according to the display size information, the image of the image data transmitted by the image transmission process and the image of the predetermined size and the image data A first instruction to display an operation screen that enables a user operation using the touch panel of the mobile device on the touch panel of the mobile device on a single screen is transmitted to the mobile device using the communication unit. 1 instruction transmission processing,
An image processing apparatus characterized by executing The image processing apparatus according to claim 1,
It has an operation part,
The controller is
When the predetermined size is larger than a predetermined ratio of the size based on the display size information, a second instruction for displaying the image of the image data transmitted in the image transmission process on the touch panel of the mobile device is displayed. A second instruction transmission process for transmitting to the mobile device via a unit;
When the predetermined size is larger than a predetermined ratio of the size according to the display size information, the main body side that allows the operation of the image data displayed on the touch panel of the mobile device to be operated via the operation unit Operation processing,
Operation information transmission processing for transmitting information operated by the main body side operation processing to the mobile device via the communication unit;
An image processing apparatus characterized by executing The image processing apparatus according to claim 2,
The controller is
After transmitting the first instruction by the first instruction transmission process, when receiving the rejection of the first instruction via the communication unit, the second instruction is transmitted via the communication unit. An image processing apparatus characterized by transmitting to a mobile device. In the image processing device according to claim 2 or 3,
The operation unit includes a touch panel,
The controller is
In the main body side operation process, at least a processing command for an image of the image data transmitted in the image transmission process and an output command for the image data image are input by a user using the touch panel of the operation unit. An image processing apparatus that displays a main body side operation screen on the touch panel of the operation unit. In the image processing device according to any one of claims 1 to 4,
The controller is
If the execution instruction of the process received from the mobile device is an execution instruction of a process that does not require display of an image of image data on the touch panel of the mobile device, the first instruction transmission process is executed,
When the execution instruction of the process received from the mobile device is not an execution instruction of a process that does not require image display of image data on the touch panel of the mobile device, the first instruction transmission process is not executed. An image processing apparatus. A communication unit capable of communicating with the image processing apparatus;
A touch panel;
A control unit;
With
The controller is
Execution command transmission for transmitting to the image processing device, using the communication unit, an execution command for processing that causes the image processing device to perform image processing while wirelessly connected to the image processing device via the communication unit Processing,
An image reception process for receiving image data from the image processing apparatus using the communication unit after transmitting the execution command in the execution command transmission process;
Enables a user operation using the touch panel for the image of the image data received in the image reception process and the image of the image data while being wirelessly connected to the image processing apparatus via the communication unit. A first instruction receiving process for receiving, from the image processing apparatus, a first instruction to display an operation screen on one screen on the touch panel;
When the first instruction is received in the first reception process, a first display screen for displaying the image of the image data received in the image reception process and the operation screen on one screen is displayed on the touch panel. A first display process to be performed;
A mobile device characterized in that An image processing device;
A mobile device with a touch panel;
In a system comprising
In a state where the image processing device and the mobile device are wirelessly connected, an acquisition request for acquiring display size information of the touch panel of the mobile device is transmitted from the image processing device to the mobile device, and the display size information A display size transmission unit for transmitting from the mobile device to the image processing device;
An image data transmission unit for transmitting image data from the image processing device to the mobile device after transmitting the display size information from the mobile device to the image processing device;
When the predetermined size is smaller than a predetermined ratio of the size based on the display size information, the image of the image data transmitted by the image data transmitting unit and the image of the predetermined size and the image data A display unit that displays on the touch panel of the mobile device a first display screen that displays an operation screen that enables a user operation using the touch panel of the mobile device on a single screen;
A system comprising:

Images