JP5907002B2 - Image processing cooperation system, image processing device, and portable terminal device - Google Patents

Description

  The present invention relates to an image processing cooperation system comprising an image processing device such as an MFP (Multi Function Peripherals) which is a multifunction digital multi-function peripheral, and a mobile terminal device such as a smartphone or a tablet terminal connectable to the image processing device, The present invention relates to an image processing device and a mobile terminal device that are preferably used in the system.

  In recent years, portable terminal devices such as smartphones and tablet terminals can store digitized documents, and can browse and edit documents regardless of time and place. In addition, a file edited by the portable terminal device can be transmitted to an image processing apparatus such as the MFP, and the image processing apparatus can execute processing such as printing or saving. Furthermore, a file composed of image data of a document read by the scanner unit of the image processing apparatus can be transmitted to the mobile terminal device, and can be edited by the mobile terminal device.

  However, it is necessary to operate the mobile terminal device and the image processing device separately, and there is a problem that the operation becomes complicated.

  Therefore, recently, an image processing apparatus for displaying an operation screen for the image processing apparatus for using the function of the image processing apparatus on the display unit of the mobile terminal apparatus, and enabling the image processing apparatus to be operated from the mobile terminal apparatus. And a portable terminal device have been proposed. In this cooperative system, the operation screen of the image processing device displayed on the display unit of the mobile terminal device is displayed based on display data transmitted from the image processing device.

  By the way, in such a cooperative system, the portable terminal device is configured to be detachable from the image processing device, and when the portable terminal device is attached to the image processing device, both are wiredly connected via USB or the like. It has been proposed to switch to a wireless connection when the device is removed from the image processing device.

  Further, Patent Document 1 uses an image format conversion unit mounted on another device connected to a communication network, thereby reducing the load of the format conversion process and the distribution of the format conversion unit in a communication network. In order to reduce the cost of the format conversion processing equipment per apparatus, the image processing apparatus that processes the image data in the second format transfers the image to the image supply apparatus that stores the image data in the first format. When a transmission request is issued, the image supply device adds information related to the image processing device to the image data in the first format and transmits it to the image conversion device that forms and converts the image data into the second format, and the format of the image data A technique for requesting conversion and transmission of image data of the second format to the image processing apparatus is disclosed.

Japanese Patent Laid-Open No. 08-166915

  As described above, when the portable terminal device is attached to the image processing device, both are connected by wire, and when the portable terminal device is removed from the image processing device, the system can be switched to wireless connection as described below. was there.

  That is, since the communication measure is generally faster in the wired connection state than in the wireless connection, in order to display the operation screen on the image processing apparatus side on the mobile terminal device, a plurality of screen generation data that are constituent elements of the operation screen are displayed. Transmit to a highly versatile portable terminal device to improve the convenience of image display, and after transmission, the received data for screen generation is synthesized in the portable terminal device to reproduce the screen on the image processing device side And display it.

  However, when the mobile terminal device is detached from the image processing device and switched to wireless connection, the communication speed decreases. In addition, when a plurality of screen generation data that are constituent elements of the operation screen are transmitted to the mobile terminal device, and the screen generation data is generated and displayed by combining the screen generation data on the mobile terminal device side Since the screen cannot be generated unless the reception of the data for generating the screen is completed, it takes time until the reception of the data for generating the screen is completed due to the decrease in the communication speed. There is a problem that the screen is displayed slowly.

  Note that the technique described in Patent Document 1 described above cannot provide a solution to the above problem.

  The present invention has been made in view of such circumstances, and when the mobile terminal device is attached to the image processing device, both are connected by wire, and when the mobile terminal device is detached from the image processing device, the wireless connection is established. Provided is an image processing cooperation system that can shorten the time until the screen on the image processing apparatus side is displayed on the portable terminal device when switching, and further provides an image processing apparatus and a portable terminal device that are preferably used in the cooperation system. The issue is to provide.

The above problem is solved by the following means.
(1) An image processing cooperation system comprising an image processing device and a mobile terminal device that is detachably attached to the image processing device, wherein the image processing device is attached to the mobile terminal device, According to the wired connection means that can be connected to the portable terminal device by wire, the wireless connection means that can be connected to the portable terminal device wirelessly with the portable terminal device removed, and the attachment / detachment of the portable terminal device A connection switching unit that switches between a wired connection and a wireless connection, and the connection switching unit switches the connection with the portable terminal device to a wireless connection, and uses one or a plurality of screen generation data to carry In the state where the connection with the portable terminal device is switched to the wired connection by the screen generation unit that generates the screen on the image processing device side displayed on the terminal device and the connection switching unit, When the screen generation data is transmitted to the mobile terminal device and the connection with the mobile terminal device is switched to the wireless connection, screen data for displaying the screen generated by the screen generation means is displayed on the mobile terminal device. And the mobile terminal device is detached from the image processing device and wired connection means that can be connected to the image processing device in a wired manner while being attached to the image processing device. A wireless connection means that is wirelessly connectable to the image processing apparatus, a reception means for receiving the screen generation data or screen data transmitted from the image processing apparatus, and attachment / detachment to / from the image processing apparatus The connection switching unit that switches between wired connection and wireless connection, the display unit, and the connection switching unit switch the connection with the image processing apparatus to a wired connection. The screen generation means for generating the screen on the image processing apparatus side displayed on the display means using the data for screen generation received by the receiving means, and the connection to the image processing apparatus by the connection switching means Is displayed on the display means when the screen is switched to the wired connection, and the reception is displayed when the connection with the image processing apparatus is switched to the wireless connection. An image processing cooperation system comprising: display control means for displaying on the display means a screen on the image processing apparatus side based on screen data received by the means.
(2) The portable terminal device includes an operation information notifying unit for notifying the image processing device of operation information indicating a content of an operation performed on the screen on the image processing device side displayed on the display unit, and the image processing Request means for requesting the device to transmit the screen generation data or screen data to the own device, and when the screen generation data or screen data is compressed data, the screen generation data or screen data is expanded. 2. The image processing cooperation system according to claim 1, further comprising: decompression means for receiving and reception completion notifying means for notifying the image processing apparatus of completion of reception of the screen generation data or screen data.
(3) The screen generation data includes a plurality of part data constituting a part of the screen, and the screen generation means of the mobile terminal device synthesizes the plurality of part data at the time of wired connection. 3. The image processing apparatus side screen is generated by the method, and when wirelessly connected, the screen generation means of the image processing apparatus generates the image processing apparatus side screen by combining the plurality of parts data. Image processing collaboration system.
(4) The screen generation data includes image data to be displayed as a preview image. When a wired connection is established, the screen generation means of the mobile terminal device edits and uses the image data to generate a preview screen. 4. The image processing cooperation system according to any one of items 1 to 3 described above.
(5) The portable terminal device includes operation information notification means for notifying the image processing apparatus of operation information indicating the content of an operation performed on the screen on the image processing apparatus side displayed on the display means of the own apparatus. The image processing apparatus includes: an analysis unit that analyzes the operation information notified from the mobile terminal device; a selection unit that selects data for screen generation based on an analysis result of the analysis unit; and the selection unit Compression means for compressing selected screen generation data or screen data for displaying a screen generated by the screen generation means using the selected screen generation data. 5. The image processing cooperation system according to any one of items 1 to 4, wherein the transmission unit transmits the screen generation data or the screen data compressed by the compression unit to the portable terminal device.
(6) In the state where the connection with the portable terminal device is switched to the wireless connection by the connection switching unit, the transmission unit of the image processing device also transmits the screen generation data in parallel with the screen data. The screen generation means of the mobile terminal device is displayed on the display means using the screen generation data received by the reception means when the reception of the screen generation data by the reception means is completed. The display control means of the portable terminal device displays the screen on the image processing apparatus side based on the screen data received by the receiving means on the display means until the screen is generated by the screen generating means. Then, after the screen is generated by the screen generation unit, the screen on the image processing apparatus side generated by the screen generation unit of the portable terminal device is displayed on the display unit. 5 Image processing cooperation system according to any one of.
(7) The portable terminal device includes a reception completion notifying unit that notifies the image processing device of completion of reception of screen generation data, and the image processing device receives a reception completion notification from the portable terminal device. 7. The image processing cooperation system according to item 6, wherein transmission of the screen data is stopped.
(8) The image processing apparatus includes a communication speed determining unit that determines whether or not a communication speed at the time of the wireless connection is faster than a communication speed at the time of the wired connection. When it is determined that the communication speed is faster than the communication speed at the time of wired connection, the transmission unit of the image processing apparatus transmits screen generation data to the portable terminal apparatus even if the transmission unit is in a wireless connection state. The image processing cooperation system according to any one of the above.
(9) The image processing apparatus includes a size determination unit that determines whether or not a total size of screen generation data to be transmitted to the mobile terminal device during the wireless connection is equal to or less than a predetermined value. When it is determined that the total size of the generation data is equal to or less than a predetermined value, the transmission unit of the image processing device transmits the screen generation data to the portable terminal device even if the transmission unit is in a wireless connection state. The image processing cooperation system according to any one of the above.
(10) When the user performs an editing operation on the image data displayed on the display unit of the mobile terminal device, the screen generation unit of the mobile terminal device has already received the image data from the image processing device during wired connection. A screen corresponding to the user's editing operation is generated using the completed image data, and the display control means of the mobile terminal device displays the generated screen on the display means. The screen corresponding to the user's editing operation is generated and the screen data is transferred to the mobile terminal device, and the image data is transferred to the mobile terminal device in parallel with the screen data. Screen data corresponding to the user's editing operation transferred from the image processing device is displayed on the display means, and after the transfer of the image data transferred in parallel with the screen data is completed, the screen data is displayed. It means image processing cooperation system according to any one of items 1 to 9 to generate a screen corresponding to the user's editing operation by using the image data, and displays on the display means.
(11) An image processing apparatus in which a mobile terminal device is detachably mounted, and a wired connection means that can be connected to the mobile terminal device in a wired manner with the mobile terminal device mounted, and the mobile terminal device With the wireless connection means wirelessly connectable with the mobile terminal device, the connection switching means for switching between the wired connection and the wireless connection according to the attachment / detachment of the mobile terminal device, and the connection switching means, Screen generation means for generating a screen on the side of the image processing device displayed on the mobile terminal device using one or a plurality of screen generation data in a state where the connection with the mobile terminal device is switched to the wireless connection; In the state where the connection with the portable terminal device is switched to the wired connection by the connection switching unit, the screen generation data is transmitted to the portable terminal device, and the connection with the portable terminal device is performed. There in a state switched to a wireless connection, an image processing apparatus characterized by comprising a transmitting means for transmitting the screen data for displaying a screen generated by the screen generating means to said portable terminal device.
(12) The screen generation data includes a plurality of part data constituting a part of the screen, and when connected by wire, transmits the plurality of parts data to the mobile terminal device, and when connected wirelessly, 12. The image processing device according to item 11, wherein the screen generation unit generates a screen on the image processing device side by combining the plurality of parts data.
(13) The image processing apparatus according to item 11 or 12, wherein the screen generation data includes image data to be displayed as a preview image.
(14) Analyzing means for analyzing operation information notified from the portable terminal apparatus and indicating operation contents performed on the screen of the image processing apparatus displayed on the portable terminal apparatus, and an analysis result of the analyzing means To select the screen generation data based on the screen generation data, the screen generation data selected by the selection means, or the screen generated by the screen generation means using the selected screen generation data Compression means for compressing the screen data, wherein the transmission means transmits the screen generation data or the screen data compressed by the compression means to the portable terminal device. Image processing apparatus.
(15) The transmission unit also transmits screen generation data to the mobile terminal device in parallel with the screen data in a state where the connection switching unit switches the connection with the mobile terminal device to a wireless connection. 15. The image processing apparatus according to any one of items 11 to 14 above.
(16) The image processing device according to item 15, wherein when the notification of completion of reception of the screen generation data made from the mobile terminal device is received, transmission of the screen data is stopped.
(17) Communication speed judging means for judging whether or not the communication speed at the time of wireless connection is faster than the communication speed at the time of wired connection, and the communication speed at the time of wireless connection by the communication speed judgment means is The image processing device according to any one of the preceding items 11 to 16, wherein when the transmission unit is determined to be faster than the communication speed, the transmission unit transmits the screen generation data to the portable terminal device even in a wireless connection state.
(18) Size determination means for determining whether the total size of the screen generation data to be transmitted to the mobile terminal device at the time of the wireless connection is equal to or less than a predetermined value, and the size determination means determines the size of the screen generation data. The image according to any one of the above items 11 to 16, wherein when the total is determined to be equal to or less than a predetermined value, the transmission unit of the image processing device transmits screen generation data to the mobile terminal device even when the transmission unit is in a wireless connection state. Processing equipment.
(19) A portable terminal device that is detachably attached to the image processing device, and is connected to the image processing device in a wired manner while being attached to the image processing device, and the image processing device. Wireless connection means that can be wirelessly connected to the image processing apparatus in a state of being removed from the connection, connection switching means for switching between wired connection and wireless connection in accordance with attachment / detachment of the image processing apparatus, display means, One or a plurality of screen generation data for generating a screen on the image processing apparatus side displayed on the display means, which is transmitted from the image processing apparatus at the time of wired connection, or transmitted from the image processing apparatus at the time of wireless connection. Receiving means for receiving screen data for displaying a screen on the image processing apparatus side generated by the image processing apparatus using the screen generation data, and the connection switching means Screen generation means for generating a screen on the image processing apparatus side displayed on the display means using the screen generation data received by the reception means in a state where the connection with the image processing apparatus is switched to a wired connection And in the state where the connection with the image processing apparatus is switched to the wired connection by the connection switching means, the screen on the image processing apparatus side generated by the screen generation means is displayed on the display means, and the image processing apparatus Display control means for displaying on the display means a screen on the image processing apparatus side based on the screen data received by the receiving means in a state in which the connection is switched to a wireless connection. apparatus.
(20) Operation information notifying means for notifying the image processing apparatus of operation information indicating the contents of the operation performed on the screen on the image processing apparatus side displayed on the display means, and generating the screen to the image processing apparatus Requesting means for requesting transmission of the screen data or screen data to its own device, when the screen generation data or screen data is compressed data, decompression means for decompressing the screen generation data or screen data, 20. The portable terminal device according to item 19, further comprising: a reception completion notifying unit that notifies the image processing device of completion of reception of the screen generation data or screen data.
(21) The screen generation data includes a plurality of parts data constituting a part of the screen, and the screen generation means combines the plurality of parts data when wired connection is performed. 21. The portable terminal device according to the item 19 or 20, which generates a side screen.
(22) The screen generation data includes image data to be displayed as a preview image. At the time of wired connection, the screen generation unit generates the preview screen by editing and using the image data. The portable terminal device in any one of 19-21.
(23) Any one of items 19 to 22 above, further comprising operation information notifying means for notifying the image processing apparatus of operation information indicating the contents of an operation performed on the screen on the image processing apparatus side displayed on the display means. The mobile terminal device according to 1.
(24) In a state where the wireless connection is switched, the receiving unit also receives screen generation data transmitted in parallel with the screen data from the image processing apparatus, and the screen generating unit is configured to receive the receiving unit. When the reception of the screen generation data by is completed, the screen generation data received by the receiving means is used to generate a screen to be displayed on the display means, and the display control means is Until the screen is generated, the screen on the image processing apparatus side based on the screen data received by the receiving means is displayed on the display means. After the screen is generated by the screen generating means, the screen generating means generates the screen. 24. The portable terminal device according to any one of items 19 to 23, wherein the screen on the image processing apparatus side is displayed on the display means.
(25) The portable terminal device according to item 24, further comprising reception completion notifying means for notifying the image processing device of completion of reception of screen generation data.
(26) When the user performs an editing operation on the image data displayed on the display unit, the screen generation unit uses the image data already received from the image processing device during wired connection. A screen corresponding to a user editing operation is generated, and the display control unit displays the generated screen on the display unit, and corresponds to the user editing operation generated and transferred by the image processing apparatus when connected wirelessly. Screen data to be displayed on the display means, and after the transfer of the image data transferred from the image processing apparatus in parallel with the screen data is completed, the screen generation means uses the image data to display a screen corresponding to the user's editing operation. 26. The portable terminal device according to any one of 19 to 25 above, which is generated and displayed on a display unit.

  According to the image processing cooperation system according to the present invention, when the connection between the image processing device and the portable terminal device is a wired connection, the image processing device transmits one or a plurality of screen generation data to the portable terminal device. Then, when the connection with the mobile terminal device is switched to the wireless connection, screen data for displaying the screen generated by using the screen generation data by the screen generation means is transmitted to the mobile terminal device. If the connection with the image processing device is a wired connection, the portable terminal device generates a screen using the screen generation data transmitted from the image processing device, displays the screen on the display means, and is switched to a wireless connection In the state, the screen on the image processing apparatus side is displayed on the display means based on the screen data generated and transmitted by the image processing apparatus using the screen generation data.

  In other words, depending on whether the connection between the image processing device and the portable terminal device is a wired connection or a wireless connection, the screen on the image processing device side is switched between the image processing device and the portable terminal device, and image processing is performed accordingly. Since the type of data transmitted from the device to the portable terminal device can also be switched, the time until the screen on the image processing device side is displayed on the portable terminal device when switching to the wireless connection can be shortened. The waiting time until the screen on the image processing device side is displayed on the portable terminal device is shortened.

  In addition, according to the image processing device of the present invention, when the connection with the portable terminal device is switched from the wired connection to the wireless connection, the time until the screen on the image processing device side is displayed on the portable terminal device is shortened. A possible image processing apparatus can be obtained.

  In addition, according to the portable terminal device according to the present invention, when the connection with the image processing device is switched from the wired connection to the wireless connection, the portable terminal device can reduce the time until the screen on the image processing device side is displayed; You can get none.

It is a block diagram of the image processing cooperation system which concerns on one Embodiment of this invention. It is a block diagram which shows the electrical structure of an image processing apparatus and a portable terminal device. It is a block diagram which shows the firmware structure of an image processing apparatus and a portable terminal device. It is a figure which shows the relationship between each software of an image processing apparatus and a portable terminal device. It is a figure for demonstrating the processing method when a portable terminal is mounted | worn with an image processing apparatus and both are wired-connected. (A) is a figure which shows an example of part data, (B) is a figure which shows the screen produced | generated by synthesize | combining part data. It is a figure for demonstrating the processing method when a portable terminal device is removed from an image processing apparatus and both are wirelessly connected. It is a flowchart which shows the flow of a process when a portable terminal device is mounted | worn with the image processing apparatus and it is wired. It is a flowchart which shows the flow of a process when a portable terminal device is removed from the image processing apparatus and connected wirelessly. It is a state transition diagram during communication between an image processing device and a mobile terminal device. FIG. 10 is a diagram for illustrating another embodiment of the present invention and for explaining a method of processing image data at the time of wired connection. Similarly, it is a figure for demonstrating the processing method of the image data at the time of a wireless connection. Similarly, it is a state transition diagram during communication between the image processing device and the portable terminal device.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a configuration diagram of an image processing cooperation system according to an embodiment of the present invention.

  This image processing cooperation system includes an image processing device 1 and a mobile terminal device 2 such as a smartphone or a tablet terminal that is detachably attached to the image processing device 1, and the mobile terminal device 2 is connected to the image processing device 1. In the mounted state, the image processing device 1 and the portable terminal device 2 can be connected by wire via a USB interface or the like, and when the portable terminal device 2 is detached from the image processing device 1, the image processing device 1. The mobile terminal device 2 can be wirelessly connected via the WIFI module, the wireless LAN router 3, and the network 4.

  In this embodiment, the above-described MFP having a copy function, a print function, a scan function, a facsimile function and the like is used as the image processing apparatus 1. Hereinafter, the image processing apparatus is also referred to as an MFP. The mobile terminal device is also referred to as a mobile terminal.

  FIG. 2 is a block diagram showing an electrical configuration of the MFP 1 and the portable terminal 2.

  The MFP 1 includes a CPU 101, a ROM 102, a main memory 103, a scanner unit 104, a print unit 105, a determination unit 106, an operation panel unit 107, a WIFI (Wireless Fidelity) module 109, a LAN (Local Area Network) controller 110, a USB interface (USB I / I). F) 111 and the like.

  The CPU 101 performs overall control of the entire MFP 1 by operating according to an operation program stored in the ROM 102 or the like. For example, in addition to controlling the various functions of the MFP 1, in this embodiment, the attachment / detachment state of the portable terminal 2 to the MFP 1 is detected, and the connection with the portable terminal 2 is wired or wirelessly connected according to the attachment / detachment state. Or the display data of the operation screen of the MFP 1 displayed on the portable terminal 2 is transmitted to the portable terminal 2 based on a request from the portable terminal 2, or when the connection with the portable terminal 2 is a wired connection and wirelessly. Control is performed such as switching the type of display data of the operation screen to be transmitted to the portable terminal 2 depending on the connection, and a detailed description thereof will be described later.

  The ROM 102 is a memory that stores an operation program of the CPU 101 and the like.

  The main memory 103 has a RAM 103a and a storage unit 103b. The RAM 103a is a memory that provides a work area when the CPU 101 operates according to the operation program. The storage unit 103b stores various data, applications, and the like, and a hard disk device or the like is used. In particular, in this embodiment, a large number of parts data constituting a part of the screen is stored as screen generation data that is a component of the operation screen for generating various operation screens displayed on the operation panel unit 107. Has been. Then, part data for generating an operation screen corresponding to the user's operation is called from the storage unit 103a by the CPU 101, and the operation screen is generated by being synthesized by the CPU 101 and displayed on the operation panel unit 107. It is transmitted to the mobile terminal 2. In addition, a plurality of files such as a document file composed of image data read by the scanner unit 104 and a document file transmitted from an external device are stored.

  The scanner unit 104 reads an image of a document set on a platen glass (not shown) or a document set on an automatic document feeder (not shown) and outputs image data as electronic data.

  The print unit 105 prints image data of a document read by the scanner unit 104 or print data transmitted from the portable terminal 2 or the like on a sheet.

  The determination unit 106 compares the communication speed at the time of wireless connection with the communication speed at the time of wired connection, determines whether the communication speed at the time of wireless connection is faster than the communication speed at the time of wired connection, It is determined whether or not the total size of the screen generation data transmitted to the mobile terminal 2 is equal to or smaller than a predetermined value.

  The operation panel unit 107 operates the MFP 1 and displays a message or the like to the user, and includes a key unit 107a having a start key, a numeric keypad, and the like, and a display unit 107b including a touch panel such as a liquid crystal.

  The WIFI module 109 is an interface for wireless connection with the portable terminal 2 when the portable terminal 2 is detached from the MFP 1, and the LAN controller 110 is connected to an external device such as a personal computer or another MFP. It controls communication.

  The USB interface 111 is an interface for wired connection with the portable terminal 2 when the portable terminal 2 is attached to the MFP 1.

  On the other hand, a smart phone, a tablet terminal, electronic paper, etc. are used for the portable terminal 2. FIG. The portable terminal 2 includes a CPU 201, a RAM 202, a storage unit 203, a display unit 204, a WIFI module 205, a LAN controller 206, a USB interface 207, file processing units 208a and 208b, a cooperation unit 209, and the like.

  The CPU 201 controls the entire mobile terminal 2 in an integrated manner, and operates according to an OS (Operating System) that is basic software, a cooperative application, or a general-purpose application described later.

  The RAM 202 is a memory serving as a work area when the CPU 201 operates according to an operation program.

  The storage unit 203 stores various data, applications, and the like, and a hard disk device or the like is used. In this embodiment, programs such as a basic OS, a cooperative application, and a general-purpose application are stored. In the following description, an application is also simply referred to as an application.

  The display unit 204 includes a device such as a liquid crystal provided with a touch panel.

  The WIFI module 205 is an interface for wirelessly connecting to the MFP 1 when the portable terminal 2 is detached from the MFP 1, and the LAN controller 206 controls communication with an external device such as a user terminal. is there. The USB interface 207 is an interface for wired connection with the MFP 1 when the portable terminal 2 is attached to the MFP 1.

  The file processing units 208a and 208b perform processing such as editing, forwarding, or mail delivery on files. Hereinafter, both the file processing units 208a and 208b are referred to as a file processing unit 208.

  The cooperation unit 209 is for enabling the functions of the MFP 1 to be used in order to smoothly perform the function cooperation between the MFP 1 and the portable terminal 2. Specifically, screen display data for reproducing the same operation screen displayed on the operation panel unit 107 of the MFP 1 in order to use the copy function, print function, scan function, facsimile function, etc. of the MFP 1 Received from the MFP 1 and displayed on the display unit 204 as a remote screen, or when the user operates the displayed remote screen, the coordinates of the operation position on the screen are detected and transmitted to the MFP 1. The MFP 1 analyzes the coordinates of the operation position, identifies the user's operation instruction, and executes an operation according to the operation instruction.

  By the way, the function of the file processing unit 208 is realized by the CPU 201 executing a general-purpose application. In other words, the general-purpose application is software for realizing the file processing unit 208 that processes various files. For example, a file editing application, an application having a function of distributing a file by e-mail, and a function of storing a file on the web Various kinds of applications such as an application that the user has can be used.

  The cooperation unit 209 realizes these functions when the CPU 201 executes the cooperation application described above. That is, the cooperation application is software for achieving smooth function cooperation between the MFP 1 and the mobile terminal 2. In addition, file transfer and download can be performed between the cooperative app and the general-purpose app.

  FIG. 3 is a block diagram illustrating firmware configurations of the MFP 1 and the portable terminal 2.

  The MFP 1 includes an operation panel (application) 121 and an open API (Open Application Program Interface) 122 in the application layer, and includes a common API (Common API) 125 and a panel display image generation unit 126 in the common API layer. The hardware-dependent layer includes a panel screen generation unit 123, a panel control unit 124, a portable terminal control unit 127, and the like.

  In addition, the kernel layer includes a panel input control unit 129, a panel LCD control unit 130, a JPEG coder driver 131, and the like, and a device layer includes an operation panel unit 107, a USB interface unit 111, a JPEG compression unit 133, a WIFI module 109, and the like. I have.

  The panel screen generation unit 123 is a screen that is data of an operation screen displayed on the operation panel unit 107 when a cooperative operation is performed from the mobile terminal 2 in a state where the mobile terminal 2 and the MFP 1 are wirelessly connected. Create data.

  The panel control unit 124 controls the communication state with the operation panel unit 107. The panel display image generation unit 126 displays image data read by the scanner unit 104 and other image data such as a preview image when the mobile terminal 2 and the MFP 1 are wirelessly connected. In this case, the image data to be displayed is generated. The generated image data is transmitted to the mobile terminal 2.

  The portable terminal control unit 127 controls an interface or the like for communicating with the portable terminal 2 during the cooperation process with the portable terminal 2.

  The panel input control unit 129 performs control related to operation input of the operation panel unit 107, and the panel LCD control unit 130 performs display control of the display unit 107 b of the operation panel unit 107. The JPEG compression unit 133 compresses data and files for displaying the operation screen transmitted from the mobile terminal control unit 127 via the JPEG coder driver 131 by JPEG or the like. The data is transmitted to the mobile terminal 2 via the interface 111 and the WIFI module 109.

  On the other hand, the mobile terminal 2 includes an image display application 221, a display image generation application 222, a JPEG development application 223, an application layer including various applications such as a browser, a platform layer including various managers 225 and 226, and the like. The OS layer includes a library 227 and a kernel 228, and hardware 229.

  The JPEG decompression application 223 decompresses (decompresses) the compressed data transmitted from the MFP 1. The display image generation application 222 generates an operation screen on the MFP 2 side displayed on the display unit 204 based on the decompressed data. When the user's editing operation such as enlargement, reduction, and page turning is performed on the data displayed on the operation screen on the MFP 2 side, the data transmitted from the MFP 1 is enlarged or reduced. Reduce or change pages.

  The image display application 221 performs drawing processing of screen data on the MFP 1 side generated by the display image generation application 222.

  In this embodiment, the image display application 221, the display image generation application 222, and the JPEG expansion application 223 are configured as a part of functions realized by the cooperation application.

  Next, processing when the MFP 1 and the portable terminal 2 are operated in cooperation will be described.

  A user connects the portable terminal 2 to the MFP 1 wirelessly or by wire, and tries to operate the portable terminal 2 and the MFP 1 in cooperation. The cooperation operation between the portable terminal 2 and the MFP 1 is started when the user starts a cooperation application installed in the portable terminal 2.

  FIG. 4 shows the relationship between the software of the MFP 1 and the portable terminal 2. The left side shows the software on the MFP 1 side and the right side shows the software on the portable terminal 2.

  The cooperative application of the mobile terminal 2 is located between the OS of the mobile terminal 2 and the application software that is general-purpose software, and includes user operation information, transmission / reception data with the mobile terminal 2, and display screen data on the mobile terminal 2 side. Can be acquired, and these data can be transmitted to the MFP 1 as necessary. In other words, the cooperation software of the mobile terminal 2 includes user operation reception processing on the mobile terminal 2 side such as calling of an application, operation screen display data reception / display processing of the MFP 1 (mobile terminal side operation screen processing), and cooperation operation related Execute processing etc. The linked operation related processing includes communication with the MFP 1 of operation information indicating the contents of the operation.

  On the other hand, when the OS on the MFP 1 and the application software for the MFP 1 are all in-house designed / customized and all necessary information can be obtained, the cooperative application of the MFP 1 may not be located between the OS and the application software. Necessary information may be obtained by communicating with the OS and application software while being independent of the OS and application software. The cooperative application of the MFP 1 executes user operation reception processing on the MFP 1 side such as calling of an application, transmission processing of MFP operation screen data to the portable terminal 2 (MFP side operation screen processing), cooperative operation related processing, and the like. As the cooperative operation-related processing, reception / execution processing of operation information from the portable terminal 2 and the like can be cited.

  When the user selects a cooperation operation icon on the operation screen displayed on the display unit 204 of the mobile terminal 2 and instructs MFP cooperation, the cooperation software of the mobile terminal 2 communicates with the registered MFP 1, and the MFP 1. Call the link software and request acquisition of the operation screen. Thereby, the cooperative operation mode is started, and the operation screen on the MFP side is displayed on the display unit 105a of the MFP 1 and the display unit 204 of the portable terminal 2. The process until the operation screen on the MFP side is displayed differs depending on whether the portable terminal 2 and the MFP 1 are wired or wirelessly connected. This will be described later. Note that both the MFP operation screen and the mobile terminal operation screen may be displayed in parallel on the display unit 204 of the mobile terminal 2.

  When the user operates a remote screen, which is an operation screen on the MFP side, displayed on the mobile terminal 2, operation information indicating the content of the operation is transmitted to the MFP 1. If the MFP is an instruction to execute a job or the like, the MFP performs a corresponding operation. On the other hand, when it is necessary to display the next operation screen according to the operation, display data for the next operation screen is transmitted to the mobile terminal 2.

  Next, a processing method for displaying the operation screen on the MFP 1 side on the portable terminal 2 will be described.

  First, a processing method when the portable terminal 2 is attached to the MFP 1 and both are connected by wire will be described with reference to FIG.

  When the portable terminal 2 is attached to the MFP 1, the MFP 1 and the portable terminal 2 are connected by wire via the mutual USB interfaces 111 and 207, and the CPU 101 of the MFP 1 recognizes that the connection is made by wire. .

  When the user operates the screen of the mobile terminal 1, operation information indicating the content of the operation and a part data acquisition request are transmitted from the mobile terminal 2 to the MFP 1. Operation information may also be used as a part data acquisition request.

  When the MFP 1 receives the operation information and part data acquisition request from the portable terminal 2, the CPU 101 analyzes the operation information and calls a plurality of part data for generating a screen according to the operation from the storage unit 103b. These parts data are transmitted together with the arrangement information to the portable terminal 2 via the USB interface 111 as shown by a thick arrow in FIG.

  In the portable terminal 2, after reception of part data or the like is completed via the USB interface 207, the CPU 201 combines the plurality of part data according to the arrangement information, and generates an operation screen on the MFP side. Thus, the convenience of image display can be improved by combining the part data with the mobile terminal 2 to generate the screen.

  FIG. 6A shows an example of part data. In the example of FIG. 6A, the part data includes a base part 301 corresponding to each mode such as a copy mode, and key parts for mode setting such as “character / photo”, “auto color”, “program”, and “application setting”. The CPU 201 generates an operation screen 300 on the MFP side as shown in FIG. 6B by combining these part data.

  After generating the operation screen 300, the CPU 201 displays the generated screen on the display unit 204.

  Next, when the user removes the portable terminal 2 from the MFP 1, the CPU 101 of the MFP 1 detects that the portable terminal 2 has been removed, and switches communication with the portable terminal 2 to a wireless connection. Accordingly, the CPU 201 of the portable terminal 2 also switches communication with the MFP 1 to wireless connection.

  An operation screen generation processing method on the MFP 1 side when the portable terminal 2 and the MFP 1 are wirelessly connected will be described with reference to FIG.

  The communication speed at the time of wireless connection is generally slower than the communication speed at the time of wired connection. For this reason, as in the case of wired connection, when a plurality of parts data is transmitted to the mobile terminal 2 and the mobile terminal 2 generates a screen after the transmission of the part data is completed, it takes time until the transmission of the part data is completed. The display of the operation screen on the MFP 1 side on the display unit 204 is delayed, which increases the waiting time of the user.

  Therefore, in this embodiment, when the wireless connection is switched, the CPU 101 of the MFP 1 generates a screen in the MFP 1 by combining the part data called from the storage unit 103b as shown by a thick solid arrow in FIG. The generated screen display data (screen data) is wirelessly transmitted to the mobile terminal 2 via the WIFI module 109.

  After the mobile terminal 2 receives the transmitted screen data via the WIFI module 205, the CPU 201 renders the received screen data and displays it on the display unit 204.

  As described above, in the wireless connection state, the MFP 1 generates a screen by synthesizing the part data, and transmits the generated screen data until the screen data is generated and received and displayed by the mobile terminal 2. Compared with the case where parts data is wirelessly transmitted to the portable terminal 2 and the screen is created by the portable terminal 2, the time until the screen is generated and displayed is shortened, and the waiting time of the user can be shortened. .

  Further, in this embodiment, as indicated by thick broken line arrows in FIG. 7, a plurality of parts data itself is also transmitted by utilizing a communication idle at the time of transmission of screen data. Then, the portable terminal 2 displays the operation screen on the MFP 1 side on the display unit 204 of the portable terminal 2 based on the screen data transmitted from the MFP 1 until transmission of all the parts data to be transmitted is completed. After the transmission is completed, the transmitted part data is synthesized by the portable terminal 2 to generate a screen and displayed on the display unit 204. Thereby, the operation screen on the MFP 1 side can be displayed on the portable terminal 2 more quickly.

  When the reception of the part data is completed, the portable terminal 2 transmits a completion notification to the MFP 1, and the MFP 1 stops transmitting the screen data even if the transmission of the screen data is not completed.

  FIG. 8 is a flowchart showing the flow of processing when the portable terminal 2 is attached to the MFP 1 and connected by wire.

  When the user operates the touch panel of the display unit 204 of the mobile terminal 2 in step S1, the mobile terminal 2 receives input information from the touch panel driver in step S2, performs gesture determination and coordinate conversion in step S3, and performs the touch position. Is identified.

  In step S4, a touch event (operation information) including the specified touch position and the part data acquisition request is transmitted to the MFP 1.

  In step S5, the MFP 1 performs reception processing such as a touch event from the portable terminal 2, and then performs touch event analysis processing in step S6. In step S7, an operation screen to be displayed based on the analysis result is displayed. The identification process starts a process of transferring a plurality of parts data, which are constituent elements of the operation screen, and their arrangement information to the portable terminal 2. That is, in step S8, the parts data and the like are PNG-compressed, and in step S9, the compressed parts data and the like are transmitted to the portable terminal 2.

  After receiving the part data and the like in step S10, the portable terminal 2 notifies the MFP 1 of the completion of reception when reception of all the part data is completed, and expands (decompresses) the received data in step S11. Next, screen generation processing is executed in step S12. Specifically, a plurality of part data (for example, part A, part B, part C) are combined at a predetermined position, and the screen on the MFP 1 side is reproduced.

  Next, a drawing process is executed in step S13 and displayed on the display unit 204 in step S14.

  FIG. 9 is a flowchart showing a processing flow when the portable terminal 2 is detached from the MFP 1 and both are wirelessly connected.

  When the user operates the touch panel of the display unit 204 of the mobile terminal 2 in step S21, the mobile terminal 2 receives input information from the touch panel driver in step S22, performs gesture determination and coordinate conversion in step S23, and performs the touch position. Is identified.

  In step S24, a touch event (operation information) including the specified touch position and the like and a screen acquisition request are transmitted to the MFP 1.

  In step S25, the MFP 1 performs processing for receiving touch events and the like from the portable terminal 2, and then performs processing for analyzing touch events in step S26. In step S27, the MFP 1 displays an operation screen to be displayed based on the analysis result. Identify and perform screen generation processing. Specifically, a plurality of part data (for example, part A, part B, and part C) that are constituent elements of the operation screen are combined at a predetermined position to reproduce the screen on the MFP 1 side.

  In step S28, the generated screen data (screen data) is JPEG compressed, and in step S29, the compressed screen data is transmitted to the portable terminal 2. Note that the part data itself is also transmitted using the communication space.

  After receiving the screen data in step S30, the portable terminal 2 expands (decompresses) the received screen data in step S31. Next, a drawing process is executed in step S32 and displayed on the display unit 204 in step S33.

  FIG. 10 is a state transition diagram during communication between the MFP 1 and the portable terminal 2. The state transition diagram of the present embodiment is shown on the left side and the conventional state on the left side, respectively. Initially, it is assumed that the portable terminal 2 is attached to the MFP 1 and both are wired.

  In the conventional state transition diagram on the left side, when an operation on the touch panel is detected by the mobile terminal 2 and a touch event or the like is transmitted from the mobile terminal 2, the MFP 1 analyzes the event and corresponding part data (in FIG. Compress parts and start transferring. When the transfer is completed, the mobile terminal 2 executes screen generation and drawing processing.

  Next, assuming that the portable terminal 2 is removed from the MFP 1 at the timing T1, both are switched to wireless connection.

  In this state, even if the touch panel of the display unit 204 of the portable terminal 2 is operated and a touch event or the like is transmitted from the portable terminal 2 to the MFP 1, the MFP 1 and the portable terminal 2 perform the same operation as in the case of wired connection. That is, when the MFP 1 receives the touch event, the MFP 1 analyzes the event, compresses the corresponding part data, and starts transfer. When the reception of the part data is completed, the mobile terminal 2 executes screen generation and drawing processing.

  On the other hand, in the present embodiment on the right side, the process is the same as before until the portable terminal 2 is removed from the MFP 1. At timing T1, when the touch panel is operated while the mobile terminal 2 is detached from the MFP 1 and both are switched to wireless connection, a touch event or the like is transmitted from the mobile terminal 2 to the MFP 1.

  When the MFP 1 receives the touch event, the MFP 1 analyzes the event, calls the corresponding part data from the storage unit 103b, synthesizes them, and generates a screen. Then, the screen data of the generated screen is compressed and transfer is started. At the same time, the transfer of the part data itself starts. The part data is transmitted using the space available when the screen data is transferred.

  When the reception of the screen data is completed, the portable terminal 2 transmits a completion notification to the MFP 1 and then executes a drawing process. When a plurality of screens are changed, screen generation in the MFP 1, screen data compression / transfer, reception by the mobile terminal 2, completion notification transmission, and drawing processing are executed for each screen.

  When the transfer of the part data transferred in parallel at timing T2 is completed, a completion notification is sent from the portable terminal 2 to the MFP 1, the MFP 1 stops the subsequent screen generation and transfer processing, and the portable terminal 2 is transferred. A screen is generated and displayed using the part data.

  As described above, in this embodiment, when the portable terminal 2 is detached from the MFP 1 and the both are switched to the wireless connection, the operation screen generation using the part data is switched from the portable terminal 2 to the MFP 1 and is generated by the MFP 1. Since the screen data is transmitted to the portable terminal 2 and displayed, the operation screen on the MFP 1 side is more portable even after switching to the wireless connection than when the portable terminal 2 generates the screen after the part data transfer is completed. 2 can be shortened and the waiting time of the user can be shortened.

  In addition, when wireless connection is made, part data is also transmitted in parallel with the transmission of the screen data, and after the transmission of the part data is completed, a screen is generated by the portable terminal 2 using the transmitted part data. The time until the operation screen is displayed on the portable terminal 2 can be further shortened.

  Next, another embodiment of the present invention will be described.

  In this embodiment, for example, a preview screen for previewing a file composed of image data of a document read by the scanner unit 104 is displayed on the mobile terminal 2.

  First, a file processing method at the time of wired connection will be described with reference to FIG.

  When the user operates the screen of the mobile terminal 1 to instruct preview display, the mobile terminal 2 transmits operation information indicating the content of the operation and an acquisition request for image data as screen generation data to the MFP 1.

  When the MFP 1 receives the operation information and the image data acquisition request from the portable terminal 2, the CPU 101 analyzes the operation information and reads the original set on the scanner unit 104.

  A file including image data of a document read by the scanner unit 104 is transmitted to the mobile terminal 2 via the main memory 103 and the USB interface 111. When the portable terminal 2 receives a file via the USB interface 207, the CPU 201 creates a preview screen using the received file and displays it on the display unit 204.

  If it is necessary to display part data other than a file on the preview screen, the part data and the arrangement information of the part data and the file are also transmitted from the MFP 1 to the portable terminal 2, and the CPU 201 combines the part data and the file. To generate a preview screen.

  On the other hand, when the portable terminal 2 is removed from the MFP 1 and switched to a wireless connection, a file composed of image data of a document read by the scanner unit 104 passes through the main memory 103 as indicated by a thick solid arrow in FIG. To the CPU 101. The CPU 101 generates a preview screen using a file. In this case, when it is necessary to display part data other than a file on the preview screen, the CPU 101 combines the part data and the file to generate a preview screen.

  After the preview screen is generated, the screen data is wirelessly transmitted to the mobile terminal 2 via the WIFI module 109.

  When the mobile terminal 2 receives the screen data of the preview screen, the CPU 201 displays the preview screen on the display unit 204 based on the received screen data.

  The MFP 1 uses the space when wirelessly communicating the screen data of the preview screen, and as shown by the thick broken line arrow in FIG. 12, in parallel with the screen data, the file itself (raw data) or further necessary part data is sent. Wirelessly transmit to the mobile terminal 2. Until the transmission and transmission of the raw data of the file is completed, a preview screen is displayed on the display unit 204 of the portable terminal 2 based on the screen data transmitted from the MFP 1. The mobile terminal 2 generates a preview screen using the transmitted raw data and the like and displays it on the display unit 204.

  FIG. 13 is a state transition diagram during communication between the MFP 1 and the portable terminal 2 with respect to the preview screen, and shows a conventional state transition diagram on the left side and a state transition diagram of the present embodiment on the right side. Initially, it is assumed that the portable terminal 2 is attached to the MFP 1 and both are wired.

  In the conventional state transition diagram on the left side, when an operation on the touch panel is detected by the portable terminal 2 and a touch event (job start instruction) is transmitted from the portable terminal 2, the MFP 1 analyzes the event and the scanner unit 104 performs one page. Read the original document for the eyes. After reading, the file consisting of image data is compressed and transfer is started. When the transfer is completed, the mobile terminal 2 notifies the MFP 1 of the end of the transfer, and then generates and displays a preview screen. After reading the image data of the first page, the MFP 1 reads the second page, further the third page, and so on.

  Next, assuming that the portable terminal 2 is removed from the MFP 1 at the timing T1 during reading of the second page, both are switched to the wireless connection, but the MFP 1 performs the same operation from the second page onward. That is, after reading the second page, the image data is compressed and transfer is started. When the transfer is completed, the mobile terminal 2 notifies the MFP 1 of the end of the transfer, and then generates and displays a preview screen. The MFP 1 executes the reading of the third page following the reading of the image data of the second page.

  On the other hand, in this embodiment, until the portable terminal 2 is removed from the MFP 1, it is the same as the conventional one. At timing T1, when the portable terminal 2 is removed from the MFP 1 and both are switched to a wireless connection, the MFP 1 generates a preview screen after reading the second page of the document, and compresses the screen data of the preview screen after the generation. And transfer to the portable terminal 2. In parallel, the transfer of the image data itself (raw data) for the second page is also started.

  When the transfer of the screen data is completed, the mobile terminal 2 notifies the MFP 1 of the end of the transfer, and then displays the received preview screen screen data on the display unit 204.

  After reading the second page, the MFP 1 starts reading the third page. After reading, the MFP 1 generates screen data for the third page and transfers it to the portable terminal 2. In parallel, the screen data of the third page itself is also transferred.

  After completing the reading of the designated page, the MFP 1 waits for the transfer end notification of the screen data of the last page from the mobile terminal 2 and transmits a job end notification to the mobile terminal 2.

  As described above, in this embodiment, when the portable terminal 2 is removed from the MFP 1 and the both are switched to the wireless connection, the generation of the preview screen using the image data is switched from the portable terminal 2 to the MFP 1, and is generated by the MFP 1. Since the data of the preview screen is transmitted to the mobile terminal 2 and displayed, the preview screen on the MFP 1 side is displayed even after switching to the wireless connection compared to the case where the mobile terminal 2 generates the preview screen after the transfer of the image data is completed. The time until it is displayed on the mobile terminal 2 can be shortened, and the waiting time of the user can be shortened.

  Further, since the image data itself is also transferred to the portable terminal 2 after switching to the wireless connection, the image data displayed in the preview on the portable terminal 2 in the wireless connection state is enlarged, reduced, and turned. Even when an editing operation is instructed, the image data that has already been sent can be edited on the mobile terminal 2 side, so there is no need to have the MFP 1 send screen data after editing the image data. After the operation instruction, the edited preview screen can be quickly displayed, and the operability for the user is improved.

  Note that the image data (file) displayed in the preview is not limited to that read by the scanner unit 104, and may be a file stored in the storage unit 103b of the MFP 1.

  Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment.

  For example, when the MFP 1 and the portable terminal 2 are switched from wired connection to wireless connection, the MFP 1 is switched to screen generation. However, the determination unit 106 determines that the communication speed at the time of wireless connection is faster than the communication speed at the time of wired connection. If it is determined that the communication speed at the time of wireless connection is faster than the communication speed at the time of wired connection, there is no decrease in the communication speed. Data for screen generation such as a preview file may be transmitted to the mobile terminal 2 and the mobile terminal 2 may generate a screen.

  Further, the determination unit 106 determines whether the total size of the screen generation data such as part data and preview file transmitted to the mobile terminal 2 at the time of wireless connection is equal to or less than a predetermined value, and determines that it is equal to or less than the predetermined value. In such a case, even if the wired connection is switched to the wireless connection, there is no significant increase in communication time, so that the MFP 1 transmits screen generation data to the portable terminal 2 in the same manner as in the wired connection. You may generate a screen with

  In the above embodiment, the case where the wired connection is switched to the wireless connection has been described. However, the same applies to the case where the wireless connection state is switched to the wired connection state.

  When a preview screen by compact PDF (CPDF) is instructed, the background image of the CPDF image generated by the MFP 1 is preferentially transmitted to the portable terminal 2 at the time of wireless connection, and the character image is transmitted following the background image to the portable terminal. 2 is good to send. As a result, the user can grasp the preview image faster.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Portable terminal device 101 CPU
102 ROM
103b Storage unit 104 Scanner unit 105 Print unit 106 Judgment unit 107 Operation panel unit 107b Display unit 109 WIFI module 111 USB interface 201 CPU
202 RAM
203 Storage Unit 204 Display Unit 205 WIFI Module 207 USB Interface 209 Cooperation Unit

Claims (26)

An image processing cooperation system comprising an image processing device and a mobile terminal device detachably attached to the image processing device,
The image processing apparatus includes:
With the mobile terminal device mounted, wired connection means that can be connected to the mobile terminal device by wire,
Wireless connection means that can be wirelessly connected to the mobile terminal device with the mobile terminal device removed;
Connection switching means for switching between a wired connection and a wireless connection in accordance with the attachment / detachment of the portable terminal device;
The screen on the image processing device side displayed on the mobile terminal device using one or a plurality of screen generation data in a state where the connection with the mobile terminal device is switched to the wireless connection by the connection switching means. Screen generation means for generating;
In the state where the connection with the portable terminal device is switched to the wired connection by the connection switching unit, the screen generation data is transmitted to the portable terminal device, and the connection with the portable terminal device is switched to the wireless connection Then, a transmission unit that transmits screen data for displaying the screen generated by the screen generation unit to the mobile terminal device;
With
The portable terminal device
Wired connection means that can be connected to the image processing device in a wired manner while being attached to the image processing device;
Wireless connection means that can be wirelessly connected to the image processing apparatus in a state of being removed from the image processing apparatus;
Receiving means for receiving the screen generation data or screen data transmitted from the image processing device;
A connection switching unit that switches between a wired connection and a wireless connection in accordance with attachment to and detachment from the image processing apparatus;
Display means;
The screen on the image processing apparatus displayed on the display means is displayed using the screen generation data received by the receiving means in a state where the connection with the image processing apparatus is switched to a wired connection by the connection switching means. Screen generation means for generating;
When the connection with the image processing apparatus is switched to a wired connection by the connection switching unit, the screen on the image processing apparatus side generated by the screen generation unit is displayed on the display unit, and the connection with the image processing apparatus is performed. Is switched to wireless connection, display control means for displaying on the display means a screen on the image processing apparatus side based on the screen data received by the receiving means;
An image processing cooperation system characterized by comprising: The portable terminal device
Operation information notifying means for notifying the image processing apparatus of operation information indicating the contents of the operation performed on the screen on the image processing apparatus side displayed on the display means;
Request means for requesting the image processing device to transmit the screen generation data or the screen data to the own device;
Decompression means for decompressing the screen generation data or screen data when the screen generation data or screen data is compressed data;
A reception completion notifying means for notifying the image processing device of completion of reception of the screen generation data or screen data;
The image processing cooperation system according to claim 1, further comprising:   The screen generation data includes a plurality of part data constituting a part of the screen, and at the time of wired connection, the screen generation means of the mobile terminal device performs image processing by combining the plurality of part data. 3. The image according to claim 1, wherein a screen on the apparatus side is generated, and a screen generation unit of the image processing apparatus generates a screen on the image processing apparatus side by combining the plurality of part data when wirelessly connected. Processing linkage system.   The screen generation data includes image data to be displayed as a preview image. When a wired connection is established, the screen generation unit of the mobile terminal device generates a preview screen by editing and using the image data. The image processing cooperation system according to claim 1. The portable terminal device includes an operation information notifying unit that notifies the image processing device of operation information indicating the content of an operation performed on the screen on the image processing device side displayed on the display unit of the own device,
The image processing device includes: an analysis unit that analyzes the operation information notified from the mobile terminal device; a selection unit that selects screen generation data based on an analysis result of the analysis unit; and the selection unit Compression means for compressing screen data for displaying the screen generated by the screen generation means using the generated screen generation data or the selected screen generation data,
The image processing cooperation system according to claim 1, wherein the transmission unit of the image processing device transmits the screen generation data or the screen data compressed by the compression unit to the portable terminal device. In a state where the connection with the portable terminal device is switched to a wireless connection by the connection switching unit,
The transmission means of the image processing device transmits screen generation data to the portable terminal device in parallel with the screen data,
The screen generation means of the portable terminal device generates a screen to be displayed on the display means by using the screen generation data received by the reception means when reception of the screen generation data by the reception means is completed. And
The display control means of the portable terminal device displays the screen on the image processing apparatus side based on the screen data received by the receiving means on the display means until the screen is generated by the screen generating means, and the screen generating means The image processing cooperation system according to claim 1, wherein after the screen is generated, the screen on the image processing device side generated by the screen generation unit of the mobile terminal device is displayed on the display unit. The portable terminal device includes a reception completion notification means for notifying the image processing device of completion of reception of screen generation data,
The image processing cooperation system according to claim 6, wherein the image processing apparatus stops transmission of the screen data when receiving a reception completion notification from the mobile terminal device. The image processing apparatus includes a communication speed determining unit that determines whether a communication speed at the time of the wireless connection is faster than a communication speed at the time of a wired connection,
When the communication speed determining means determines that the communication speed at the time of wireless connection is faster than the communication speed at the time of wired connection, the transmission means of the image processing apparatus can store the screen generation data even if it is in a wireless connection state. The image processing cooperation system according to claim 1, wherein the image processing cooperation system is transmitted to the mobile terminal device. The image processing device includes size determining means for determining whether or not the total size of the screen generation data to be transmitted to the mobile terminal device during the wireless connection is a predetermined value or less.
When the size determining unit determines that the total size of the screen generation data is equal to or less than a predetermined value, the screen generation data is transmitted to the portable terminal device even when the transmission unit of the image processing apparatus is in a wireless connection state. The image processing cooperation system according to claim 1, which transmits the image processing cooperation system. When the user performs an editing operation on the image data displayed on the display unit of the mobile terminal device, the screen generation unit of the mobile terminal device has already received an image from the image processing device during wired connection. A screen corresponding to the editing operation of the user is generated using data, and the display control means of the mobile terminal device displays the generated screen on the display means,
At the time of wireless connection, the image processing device generates a screen corresponding to the editing operation of the user and transfers the screen data to the mobile terminal device, and transfers the image data to the mobile terminal device in parallel with the screen data,
When the mobile terminal device is wirelessly connected, the screen data corresponding to the user's editing operation transferred from the image processing device is displayed on the display means, and the transfer of the image data transferred in parallel with the screen data is completed. Thereafter, the screen generation unit generates a screen corresponding to the user's editing operation using the image data and displays the screen on the display unit. An image processing apparatus in which a mobile terminal device is detachably mounted,
With the mobile terminal device mounted, wired connection means that can be connected to the mobile terminal device by wire,
Wireless connection means that can be wirelessly connected to the mobile terminal device with the mobile terminal device removed;
Connection switching means for switching between a wired connection and a wireless connection in accordance with the attachment / detachment of the portable terminal device;
The screen on the image processing device side displayed on the mobile terminal device using one or a plurality of screen generation data in a state where the connection with the mobile terminal device is switched to the wireless connection by the connection switching means. Screen generation means for generating;
In the state where the connection with the portable terminal device is switched to the wired connection by the connection switching unit, the screen generation data is transmitted to the portable terminal device, and the connection with the portable terminal device is switched to the wireless connection Then, a transmission unit that transmits screen data for displaying the screen generated by the screen generation unit to the mobile terminal device;
An image processing apparatus comprising:   The screen generation data includes a plurality of part data constituting a part of the screen, and transmits the plurality of parts data to the portable terminal device at the time of wired connection, and the screen generation means at the time of wireless connection. The image processing apparatus according to claim 11, wherein the screen on the image processing apparatus side is generated by combining the plurality of part data.   The image processing apparatus according to claim 11, wherein the screen generation data includes image data displayed as a preview image. Analyzing means for analyzing operation information notified from the mobile terminal device and indicating operation content performed on the screen on the image processing device side displayed on the mobile terminal device;
Selection means for selecting data for screen generation based on the analysis result of the analysis means;
Compression means for compressing screen data selected by the selection means or screen data for displaying a screen generated by the screen generation means using the selected screen generation data;
With
The image processing apparatus according to claim 11, wherein the transmission unit transmits the screen generation data or the screen data compressed by the compression unit to the mobile terminal device.   The transmission means also transmits screen generation data to the mobile terminal device in parallel with the screen data in a state where the connection with the mobile terminal device is switched to a wireless connection by the connection switching means. The image processing apparatus in any one of -14.   The image processing device according to claim 15, wherein when receiving a notification of completion of reception of screen generation data from the mobile terminal device, transmission of the screen data is stopped. A communication speed determining means for determining whether the communication speed at the time of wireless connection is faster than the communication speed at the time of wired connection;
When the communication speed determining means determines that the communication speed at the time of wireless connection is faster than the communication speed at the time of wired connection, the transmission means transmits the screen generation data to the portable terminal device even if the transmission means is in a wireless connection state. The image processing apparatus according to any one of claims 11 to 16, wherein the image processing apparatus is transmitted to Size determining means for determining whether or not the total size of the screen generation data to be transmitted to the mobile terminal device during the wireless connection is a predetermined value or less;
When the size determining unit determines that the total size of the screen generation data is equal to or less than a predetermined value, the screen generation data is transmitted to the portable terminal device even when the transmission unit of the image processing apparatus is in a wireless connection state. The image processing apparatus according to claim 11, which transmits the image processing apparatus. A portable terminal device that is detachably attached to an image processing device,
Wired connection means that can be connected to the image processing device in a wired manner while being attached to the image processing device;
Wireless connection means that can be wirelessly connected to the image processing apparatus in a state of being removed from the image processing apparatus;
Connection switching means for switching between a wired connection and a wireless connection in accordance with the attachment / detachment with the image processing device;
Display means;
One or a plurality of screen generation data for generating a screen on the image processing apparatus side displayed on the display means, which is transmitted from the image processing apparatus at the time of wired connection, or transmitted from the image processing apparatus at the time of wireless connection Receiving means for receiving screen data for displaying a screen of the image processing apparatus generated by the image processing apparatus using the screen generation data;
The screen on the image processing apparatus side displayed on the display means using the screen generation data received by the receiving means in a state where the connection with the image processing apparatus is switched to a wired connection by the connection switching means. Screen generation means for generating
When the connection with the image processing apparatus is switched to a wired connection by the connection switching unit, the screen on the image processing apparatus side generated by the screen generation unit is displayed on the display unit, and the connection with the image processing apparatus is performed. Is switched to wireless connection, display control means for displaying on the display means a screen on the image processing apparatus side based on the screen data received by the receiving means;
A portable terminal device comprising: Operation information notifying means for notifying the image processing apparatus of operation information indicating the contents of the operation performed on the screen on the image processing apparatus side displayed on the display means;
Request means for requesting the image processing device to transmit the screen generation data or the screen data to the own device;
Decompression means for decompressing the screen generation data or screen data when the screen generation data or screen data is compressed data;
A reception completion notifying means for notifying the image processing device of completion of reception of the screen generation data or screen data;
The mobile terminal device according to claim 19, comprising:   The screen generation data includes a plurality of part data constituting a part of the screen, and the screen generation unit synthesizes the plurality of part data at the time of wired connection, thereby displaying the screen on the image processing apparatus side. 21. The portable terminal device according to claim 19 or 20, wherein the mobile terminal device is generated.   The screen generation data includes image data to be displayed as a preview image, and the screen generation unit generates a preview screen by editing and using the image data at the time of wired connection. 21. The mobile terminal device according to any one of 21.   The operation information notifying means for notifying the image processing apparatus of operation information indicating the contents of the operation performed on the screen on the image processing apparatus side displayed on the display means. Mobile terminal device. With the wireless connection switched,
The receiving means also receives screen generation data transmitted in parallel with the screen data from the image processing device,
The screen generation unit generates a screen to be displayed on the display unit using the screen generation data received by the reception unit when reception of the screen generation data by the reception unit is completed,
The display control means displays the screen on the image processing apparatus side based on the screen data received by the receiving means on the display means until the screen is generated by the screen generating means, and the screen generating means generates the screen. The portable terminal device according to any one of claims 19 to 23, wherein a screen on the image processing device side generated by the screen generation unit is displayed on the display unit.   25. The portable terminal device according to claim 24, further comprising reception completion notifying means for notifying the image processing device of completion of reception of screen generation data. When the user performs an editing operation on the image data displayed on the display unit, the screen generation unit uses the image data already received from the image processing device when the wired connection is established. A screen corresponding to the operation is generated, and the display control means displays the generated screen on the display means,
When wirelessly connected, the screen data corresponding to the user editing operation generated and transferred by the image processing device is displayed on the display means, and after the transfer of the image data transferred from the image processing device in parallel with the screen data is completed The mobile terminal device according to any one of claims 19 to 25, wherein the screen generation means generates a screen corresponding to a user's editing operation using the image data and displays the screen on the display means.

Images