JP2015133023A - Image processing system, image processor, and portable terminal device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system, or the like for improving the operability with respect to a simple display area in a remote screen in the case where a simple display area of an image processor side, and a simple display area of a portable terminal device side are adjacent to each other, in the remote screen which has the simple display area, works as an operation screen for the image processor, and is displayed in the portable terminal device.SOLUTION: When a first simple display area 300 of an image processor 1 side, and a second simple display area 400 of a portable terminal device 2 side are adjacent to each other in a remote screen D1 displayed by display means 204a of the portable terminal device 2, the image processor 1 or the portable terminal device 2 performs a control to avoid the display where the first simple display area 300 and the second simple display area 400 are adjacent to each other.

Description

  The present invention relates to an image processing system including an image processing device such as an MFP (Multi Function Peripherals) which is a multifunction digital multifunction peripheral, and a mobile terminal device such as a smartphone or a tablet terminal capable of remotely operating the image processing device, The present invention relates to an image processing device and a mobile terminal device used in the system.

  In recent years, portable terminal devices such as smartphones and tablet terminals can store digitized documents, and can browse and edit documents regardless of time and place. In addition, a file edited by the portable terminal device can be transmitted to an image processing apparatus such as the MFP, and the image processing apparatus can execute processing such as printing or saving. Furthermore, a file composed of image data of a document read by the scanner unit of the image processing apparatus can be transmitted to the mobile terminal device, and can be edited by the mobile terminal device.

  However, it is necessary to operate the mobile terminal device and the image processing device separately, and there is a problem that the operation becomes complicated.

  Therefore, recently, an operation screen on the image processing apparatus side for using the functions of the image processing apparatus is displayed as a remote screen on the display unit of the mobile terminal device, so that the image processing device can be operated cooperatively from the mobile terminal device. An image processing system including an image processing device and a portable terminal device has been proposed. In this system, the remote screen on the image processing device side displayed on the display unit of the mobile terminal device is displayed based on the screen data transmitted from the image processing device.

  That is, information indicating that an operation has been performed is transmitted from the mobile terminal device to the image processing device, and the image processing device creates the next screen data based on this information, and transmits this screen data to the mobile terminal device. The mobile terminal device updates and displays the remote screen based on the screen data.

  By the way, on the mobile terminal device, a band-like simple display area for simply displaying the state of the device itself, the state of the application, or the like with icons or the like or displaying a part of the message in a simple manner is displayed on the screen. In some cases, the operation screen of the image processing apparatus may display a simple band-like display area that displays toner, paper information, and the like with icons or the like, or displays a part of a message.

  When the user performs a pull-out operation such as a flick operation on the simple display area on the portable terminal device side or the simple display area on the image processing device side, one end in the width direction of the display area is pulled out and simplified display is performed. The area is expanded and a detailed information display screen is displayed.

  However, as shown in FIG. 27, when the operation screen of the image processing apparatus on which such a simple display area 300 is displayed is displayed on the mobile terminal device 2 as the remote screen D1 indicated by a thick frame, the remote screen D1 is displayed. The displayed band-shaped simple display area 300 and the band-shaped simple display area 400 on the mobile terminal device 2 side may be displayed in an adjacent state.

  In such a display state, when the user performs a flick operation or the like in the direction of arrow A in FIG. 27 on the simple display area 300 on the remote screen D1, the following problem has occurred. In FIG. 27, an arrow B indicates a case where a flick operation is performed on the simple display area 400 on the mobile terminal device 2 side.

JP 2012-204993 A

  That is, the simple display area 300 on the remote screen D1, which is the operation screen of the image processing apparatus, is a narrow area. When the user performs a flick operation or the like on the simple display area 300, the simple display area 300 is moved to the arrow A. If the flick operation is performed across the width direction in the direction of, the simple display area 400 on the adjacent mobile terminal device 2 side is erroneously flicked, and the simple display area 400 on the mobile terminal device 2 side is pulled out. . For this reason, in order to draw out the simple display area 300 on the remote screen D1 accurately, the operation has to be performed starting from a narrow area in the simple display area 300, and the operability is not good.

  Note that if the flick operation is performed after the remote screen D1 is enlarged by pinch operation or the like in order to widen the starting point area of the flick operation, the entire screen cannot be confirmed because the entire screen is enlarged. For this reason, a screen reduction operation and a scroll operation are necessary after that, and the operation becomes troublesome.

  In the above-mentioned patent document 1, a numeric keypad is displayed when an attempt is made to set a frame erase on the operation panel, and even if it is assumed that there is a frame erase setting value below the numeric keypad screen, nothing is set. Since it is not possible to confirm whether the numeric keypad display area has been set in advance so that the numeric keypad screen does not overlap with the lower frame erase setting value, this technique can also solve the above problem. There wasn't.

  On the other hand, there is a technique for displaying content such as an advertisement image designated by a WEB browser such as a smartphone at a designated position on a screen.

  However, using this technology, even if an attempt is made to move the simple display area of the image processing device displayed on the portable terminal device to the designated position, the remote screen reproduces one screen display data transmitted from the image processing device. Therefore, it is difficult to perform another operation on only a part of the remote screen.

  The present invention has been made in view of such circumstances, and includes a simple display area on the image processing device side in a remote screen, which is an operation screen of an image processing device having a simple display area, and a portable terminal device side. When the simple display area is displayed adjacent to the mobile terminal device, the operability with respect to the simple display area in the remote screen is not good, and the simple display area on the mobile terminal device side is erroneously operated. It is an object of the present invention to provide an image processing system, an image processing device, and a mobile terminal device that can solve the problem.

The above problem is solved by the following means.
(1) An image processing system including an image processing device and a portable terminal device capable of remotely operating the image processing device, wherein the image processing device simply displays at least the status and / or message of the device itself. An operation screen of the own apparatus having a first simple display area that is a display area and that pulls out the simple display area and displays detailed information when the user performs a pull-out operation on the simple display area. Display data creation means for creating remote screen display data for display as a remote screen on the mobile terminal device, transmission means for transmitting the created remote screen display data to the mobile terminal device, and the remote screen display Operation information performed by the user on the remote screen displayed on the mobile terminal device based on the data for the mobile terminal device And receiving means for receiving, when the operation information is received by the receiving means, the display data creation means creates the remote screen display data according to the received operation information, and transmits the transmission information. The means transmits the created remote screen display data to the portable terminal device, and the portable terminal device receives the remote screen display data from the image processing device, the display means, A second simple display area for displaying the operation screen of the image processing apparatus as a remote screen on the display means based on the remote screen display data received by the receiving means, and at the same time displaying at least the status and / or message of the apparatus itself. Is displayed together with the remote screen, and the user performs a pull-out operation on the second simple display area. A display control means for pulling out the simple display area and displaying detailed information on the display means, and when a user performs an operation on the remote screen displayed on the display means, operation information corresponding to the operation is displayed. Transmitting means for transmitting to the image processing device, the first simple display area on the remote screen of the image processing device, and the second simple display area displayed on the display means of the portable terminal device, When the images are adjacent to each other, the image processing device or the portable terminal device performs control to avoid adjacent display of the first and second simple display areas.
(2) The image processing device transmits display contents of the first simple display area to the portable terminal device, and creates remote screen display data in which the first simple display area is erased to generate the portable information. The portable terminal device receives the display contents of the first simple display area and displays the contents in the second simple display area, whereby the first and second simple display areas are transmitted. 2. The image processing system according to item 1, wherein adjacent display of the display area is avoided.
(3) The portable terminal device transmits the display content of the second simple display area to the image processing device and hides the second simple display area. The image processing device By receiving display contents of the simple display area, creating remote screen display data for displaying the contents in the first simple display area, and transmitting the data to the portable terminal device, the first and second simple displays 2. The image processing system according to item 1, wherein adjacent display of the display area is avoided.
(4) The image processing system according to (1), wherein the portable terminal device avoids adjacent display of the first and second simple display areas by hiding the second simple display area.
(5) The image processing device transmits the position information of the first simple display area to the portable terminal device, the portable terminal device receives the position information of the first simple display area, and 2. The image processing system according to item 1, wherein the second simple display area is moved to a position away from the first simple display area and displayed, thereby avoiding adjacent display of the first and second simple display areas. .
(6) The portable terminal device transmits position information of the second simple display area to the image processing device, and the image processing device receives the position information of the second simple display area, and By creating remote screen display data in which the first simple display area is moved to a position away from the second simple display area and transmitting the data to the portable terminal device, the first and second simple display areas 2. The image processing system according to item 1, wherein the adjacent display is avoided.
(7) The portable terminal device or the image processing device includes a determination unit that determines whether or not there is an empty area in which the display content of the first simple display area can be displayed in the second simple display area. If it is determined that there is a free area, the mobile terminal device receives the display content of the first simple display area, displays the content in the second simple display area, and there is no free area. If it is determined, the image processing apparatus creates remote screen display data in which the first simple display area is moved to a position away from the second simple display area, and transmits the data to the portable terminal device. The image processing system according to item 2, wherein adjacent display of the first and second simple display areas is avoided.
(8) The portable terminal device or the image processing device includes a determination unit that determines whether or not there is a free area in which the display content of the second simple display area can be displayed in the first simple display area. When it is determined that there is a free area, the image processing apparatus receives the display contents of the second simple display area and displays the contents in its own first simple display area. Data is generated and transmitted to the mobile terminal device, and if it is determined that there is no free area, the mobile terminal device moves the second simple display area away from the first simple display area. 4. The image processing system according to item 3, wherein the display is performed by moving the display to avoid adjacent display of the first and second simple display areas.
(9) An image processing apparatus used in the image processing system according to any one of items 1 to 8.
(10) A mobile terminal device used in the image processing system according to any one of items 1 to 8.

  According to the invention described in (1), the first simple display area on the image processing device side and the second simple display area on the portable terminal device side in the remote screen displayed on the display means of the portable terminal device Are adjacent to each other, the image processing device or the portable terminal device performs control to avoid adjacent display of the first simple display area and the second simple display area, and therefore the first simple display area and the second simple display area The adjacent display of the simple display area is avoided. For this reason, since it is possible to perform a flick operation or the like across the first simple display area of the remote screen in the width direction, it is not necessary to perform a flick operation starting from a narrow area, and operability can be improved. In addition, it is possible to eliminate the inconvenience of erroneously flicking the simple display area on the adjacent mobile terminal device side.

  According to the invention described in (2) above, the display content of the first simple display area is displayed in the second simple display area of the mobile terminal device together with the display content of the mobile terminal device. Therefore, the inconvenience of erroneously operating the second simple display area on the mobile terminal device side in an attempt to operate the first simple display area on the remote screen can be solved. A user who wants to know the detailed contents of the first simple display area may perform a flick operation or the like on the second simple display area. In this case, a flick operation or the like across the second simple display area in the width direction or the like. Therefore, it is not necessary to perform a flick operation starting from a narrow area, and the operability can be improved.

  According to the invention described in (3) above, the display content of the second simple display area on the mobile terminal device side is displayed together with the display content of the image processing device in the first simple display area of the remote screen, Since the second simple display area on the mobile terminal device side is hidden, the second simple display area on the mobile terminal device side is erroneously operated in an attempt to operate the first simple display area on the remote screen. Can be eliminated. A user who wants to know the detailed contents of the second simple display area of the mobile terminal device may perform a flick operation or the like on the first simple display area on the remote screen. In this case, the first simple display area is moved in the width direction. Therefore, it is not necessary to perform the flick operation starting from a narrow area, and the operability can be improved.

  According to the invention described in (4) above, since the mobile terminal device hides the second simple display area, the mobile terminal device side second mobile phone device tries to operate the first simple display area on the remote screen. The inconvenience of operating the simple display area by mistake can be solved.

  According to the invention described in item (5) above, the mobile terminal device receives the position information of the first simple display area from the image processing device, and changes the second simple display area of itself to the first simple display area. Since the display is moved to a position away from the first display area, the adjacent display state between the first simple display area and the second simple display area is avoided. For this reason, since it is possible to perform a flick operation or the like across the first simple display area of the remote screen in the width direction, it is not necessary to perform a flick operation starting from a narrow area, and operability can be improved. In addition, it is possible to eliminate the inconvenience of erroneously flicking the simple display area on the adjacent mobile terminal device side.

  According to the invention described in item (6) above, the image processing apparatus receives the position information of the second simple display area from the portable terminal device, and changes its own first simple display area to the second simple display area. Since the remote screen display data moved to a position away from the remote terminal is created and transmitted to the mobile terminal device, the first simple display area and the second simple display are displayed when the remote screen is displayed on the mobile terminal device. The adjacent display state of the area is avoided. For this reason, since it is possible to perform a flick operation or the like across the first simple display area of the remote screen in the width direction, it is not necessary to perform a flick operation starting from a narrow area, and operability can be improved. In addition, it is possible to eliminate the inconvenience of erroneously flicking the simple display area on the adjacent mobile terminal device side.

  According to the invention described in (7) above, when it is determined by the mobile terminal device or the image processing device that there is a free area in which the display content of the first simple display area can be displayed in the second simple display area In addition, the display contents of the first simple display area are displayed in the second simple display area, and when it is determined that there is no empty area, the image processing apparatus displays the first simple display area in the second simple display area. Remote screen display data moved to a position away from the area is created and transmitted to the portable terminal device. Therefore, in the state where the remote screen is displayed on the mobile terminal device, the adjacent display state of the first simple display area and the second simple display area is avoided, so that it is not necessary to perform a flick operation starting from a narrow area. The operability can be improved, and the inconvenience of erroneously operating the second simple display area when trying to operate the first simple display area of the remote screen can be solved.

  According to the invention described in (8) above, when it is determined by the mobile terminal device or the image processing device that there is a free area in which the display content of the second simple display area can be displayed in the first simple display area In addition, the display content of the second simple display area is displayed in the first simple display area, and when it is determined that there is no empty area, the mobile terminal device uses the second simple display area as the first simple display area. Move to a position away from the display area and display. Therefore, in the state where the remote screen is displayed on the mobile terminal device, the adjacent display state of the first simple display area and the second simple display area is avoided, so that it is not necessary to perform a flick operation starting from a narrow area. The operability can be improved, and the inconvenience of erroneously operating the second simple display area of the mobile terminal device in an attempt to operate the first simple display area of the remote screen can be reliably solved.

  According to the invention described in the preceding item (9), it is not necessary to perform a flick operation starting from a narrow area, the operability can be improved, and a simple display area on the adjacent mobile terminal device side is erroneously flicked. An image processing apparatus that can eliminate inconveniences is obtained.

  According to the invention described in the above item (10), it is not necessary to perform a flick operation starting from a narrow area, the operability can be improved, and a simple display area on the adjacent mobile terminal device side is erroneously flicked. A portable terminal device that can eliminate inconveniences is obtained.

1 is a configuration diagram of an image processing cooperation system in which a mobile terminal device according to an embodiment of the present invention is used. It is a block diagram which shows the electrical structure of an image processing apparatus and a portable terminal device. It is a block diagram which shows the firmware structure of an image processing apparatus and a portable terminal device. It is a figure which shows the relationship between each software of an image processing apparatus and a portable terminal device. It is a figure which shows the screen of the display part of the portable terminal device in the example 1 of control for avoiding the adjacent display of a 1st simple display area and a 2nd simple display area. FIG. 6 is a sequence diagram illustrating operations of the image processing apparatus and the mobile terminal device in the case of Control Example 1 in FIG. 5. 7 is a flowchart showing main operations of the image processing system shown in FIGS. 5 and 6. It is a figure which shows the screen of the display part of the portable terminal device in the control example 2 for avoiding the adjacent display of a 1st simple display area and a 2nd simple display area. It is a sequence diagram which shows operation | movement of the image processing apparatus and portable terminal device in the case of the control example 2 of FIG. 10 is a flowchart showing main operations of the image processing system shown in FIGS. 8 and 9. It is a figure which shows the screen of the display part of the portable terminal device in the example 3 of control for avoiding the adjacent display of a 1st simple display area and a 2nd simple display area. FIG. 12 is a sequence diagram illustrating operations of the image processing apparatus and the mobile terminal device in the control example 3 of FIG. 11. 13 is a flowchart showing main operations of the image processing system shown in FIGS. 11 and 12. It is a figure which shows the screen of the display part of the portable terminal device in the example 4 of control for avoiding the adjacent display of a 1st simple display area and a 2nd simple display area. It is a sequence diagram which shows operation | movement of the image processing apparatus and portable terminal device in the case of the control example 4 of FIG. 16 is a flowchart showing main operations of the image processing system shown in FIGS. 14 and 15. It is a figure which shows the screen of the display part of the portable terminal device in the control example 5 for avoiding the adjacent display of a 1st simple display area and a 2nd simple display area. It is a figure which shows the screen of the display part of the portable terminal device in the example 6 of control for avoiding the adjacent display of a 1st simple display area and a 2nd simple display area. FIG. 19 is a sequence diagram illustrating operations of the image processing device and the mobile terminal device in the control example 6 of FIG. 18. 20 is a flowchart showing main operations of the image processing system shown in FIGS. 18 and 19. It is a figure which shows the screen of the display part of the portable terminal device in the example 7 of control for avoiding the adjacent display of a 1st simple display area and a 2nd simple display area. FIG. 22 is a sequence diagram illustrating operations of the image processing apparatus and the mobile terminal device in the control example 7 of FIG. 21. It is a flowchart which shows the main operations of the image processing system shown in FIG.21 and FIG.22. It is a figure which shows the screen of the display part of the portable terminal device in the example 8 of control for avoiding the adjacent display of a 1st simple display area and a 2nd simple display area. FIG. 25 is a sequence diagram illustrating operations of the image processing device and the mobile terminal device in the control example 8 of FIG. 24. FIG. 26 is a flowchart showing main operations of the image processing system shown in FIGS. 24 and 25. FIG. It is a figure which shows the screen of the display part of the conventional portable terminal device.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a configuration diagram of an image processing system according to an embodiment of the present invention.

  The image processing system includes an image processing device 1 and a mobile terminal device 2, and the image processing device 1 and the mobile terminal device 2 can be connected to each other via a wireless LAN router 3 and a network 4. . Note that the image processing apparatus 1 and the portable terminal apparatus 2 can be connected to each other by a USB interface or the like, but in this embodiment, description will be made assuming that they are connected to each other by wireless communication.

  In this embodiment, the above-described MFP having a copy function, a print function, a scan function, a facsimile function and the like is used as the image processing apparatus 1. Hereinafter, the image processing apparatus is also referred to as an MFP. The mobile terminal device is also referred to as a mobile terminal.

  FIG. 2 is a block diagram showing an electrical configuration of the MFP 1 and the portable terminal 2.

  The MFP 1 includes a CPU 101, ROM 102, RAM 103, scanner unit 104, print unit 105, storage unit 106, operation panel unit 107, WIFI (Wireless Fidelity) module 109, LAN (Local Area Network) controller 110, USB interface (USB I / F). 111 etc. are provided.

  The CPU 101 performs overall control of the entire MFP 1 by operating according to an operation program stored in the ROM 102 or the like. For example, in addition to controlling the various functions of the MFP 1 to be usable, in this embodiment, based on a request from the mobile terminal 2, for remote screen display for displaying the operation screen of the own apparatus as a remote screen on the mobile terminal 2. Data is transmitted to the portable terminal 2, or remote screen display data to be executed or displayed next is transmitted to the portable terminal 2 in accordance with a user operation performed on the portable terminal 2 side.

  The ROM 102 is a memory that stores an operation program of the CPU 101, and the RAM 103 is a memory that provides a work area when the CPU 101 operates according to the operation program.

  The scanner unit 104 reads an image of a document set on a platen glass (not shown) or a document set on an automatic document feeder (not shown) and outputs image data as electronic data.

  The print unit 105 prints image data of a document read by the scanner unit 104 or print data transmitted from the portable terminal 2 or the like on a sheet.

  The storage unit 106 stores various data, applications, and the like, and uses a hard disk device or the like. In particular, in this embodiment, display data for various operation screens displayed on the operation panel unit 107 is stored, and display data for operation screens corresponding to user operations is called from the storage unit 106 by the CPU 101. Displayed on the operation panel unit 107 or transmitted to the portable terminal 2. A plurality of files such as a document file composed of image data read by the scanner unit 104 and a document file transmitted from an external apparatus are also stored.

  The operation panel unit 107 operates the MFP 1 and displays a message or the like to the user. The operation panel unit 107 includes a key unit 107a having mechanical hardware keys such as a start key, a stop key, a reset key, and a numeric keypad, and a liquid crystal display. A display unit 107b including a touch panel is provided.

  The WIFI module 109 is an interface for wireless connection with the portable terminal 2, and the LAN controller 110 controls communication with an external device such as a personal computer or another MFP.

  The USB interface 111 is an interface for wired connection with the portable terminal 2.

  On the other hand, a smart phone, a tablet terminal, electronic paper, etc. are used for the portable terminal 2. FIG. The portable terminal 2 includes a CPU 201, a RAM 202, a storage unit 203, a display unit 204a, an operation unit 204b, a WIFI module 205, a LAN controller 206, a USB interface 207, file processing units 208a and 208b, a cooperation unit 209, and the like.

  The CPU 201 controls the entire mobile terminal 2 in an integrated manner, and operates according to an OS (Operating System) that is basic software, a cooperative application, or a general-purpose application described later.

  The RAM 202 is a memory serving as a work area when the CPU 201 operates according to an operation program.

  The storage unit 203 stores various data, applications, and the like, and a hard disk device or the like is used. In this embodiment, programs such as remote screen display data including a first simple display area (described later) transmitted from the MFP 1, a basic OS, a cooperative application, and a general-purpose application are stored. In the following description, an application is also simply referred to as an application.

  The display unit 204a is a device such as a liquid crystal, and the operation unit 204b is a touch panel provided on the display unit 204a and capable of touch input operation. The touch operation is usually performed by a gesture operation that defines the operation content according to the number of touched fingers and the movement of the fingers. Examples of gesture operations include flick operations for pulling out and scrolling (page advance and page back), pinch-in operations for instructing image scaling (enlargement or reduction), rotation, etc., pinch-out operations, etc. it can.

  The WIFI module 205 is an interface for wirelessly connecting the mobile terminal 2 to the MFP 1, and the LAN controller 206 controls communication with an external device such as a user terminal. The USB interface 207 is an interface for connecting the portable terminal device 2 to the MFP 1 by wire.

  The file processing units 208a and 208b perform processing such as file editing, transfer, or mail delivery. Hereinafter, both the file processing units 208a and 208b are referred to as a file processing unit 208.

  The cooperation unit 209 is for enabling the functions of the MFP 1 to be used in order to smoothly perform the function cooperation between the MFP 1 and the portable terminal 2. Specifically, remote screen display data for reproducing the same operation screen as that displayed on the operation panel unit 107 of the MFP 1 in order to use the copy function, print function, scan function, facsimile function, etc. of the MFP 1 Is received from the MFP 1 and the operation screen is displayed on the display unit 204 as a remote screen or when the user presses an operation button or the like displayed on the remote screen, the coordinates of the operation position on the screen are detected and operated. Information is sent to the MFP 1. The MFP 1 analyzes the coordinates of the operation position, identifies the user's operation instruction, and executes an operation according to the operation instruction.

  The function of the file processing unit 208 is realized by the CPU 201 executing a general-purpose application. In other words, the general-purpose application is software for realizing the file processing unit 208 that processes various files. For example, a file editing application, an application having a function of distributing a file by e-mail, and a function of storing a file on the web Various kinds of applications such as an application that the user has can be used.

  The cooperation unit 209 realizes these functions when the CPU 201 executes the cooperation application described above. That is, the cooperation application is software for achieving smooth function cooperation between the MFP 1 and the mobile terminal 2. In addition, file transfer and download can be performed between the cooperative app and the general-purpose app.

  FIG. 3 is a block diagram illustrating firmware configurations of the MFP 1 and the portable terminal 2.

  The MFP 1 includes an operation panel (application) 121 and an open API (Open Application Program Interface) 122 in the application layer, and includes a common API (Common API) 125 and a panel display image generation unit 126 in the common API layer. The hardware-dependent layer includes a panel screen generation unit 123, a panel control unit 124, a portable terminal control unit 127, and the like.

  In addition, the kernel layer includes a panel input control unit 129, a panel LCD control unit 130, a JPEG coder driver 131, and the like, and a device layer includes an operation panel unit 107, a USB interface unit 111, a JPEG compression unit 133, a WIFI module 109, and the like. I have.

  Panel screen generation unit 123 generates screen data on the MFP 1 side displayed on operation panel unit 107 when a cooperative operation is performed from portable terminal 2.

  The panel control unit 124 controls the communication state with the operation panel unit 107 and the like, and the portable terminal control unit 127 controls an interface and the like for communicating with the portable terminal 2 during the cooperation process with the portable terminal 2. .

  The panel input control unit 129 performs control related to operation input of the operation panel unit 107, and the panel LCD control unit 130 performs display control of the display unit 107 b of the operation panel unit 107. The JPEG compression unit 133 performs JPEG compression on the remote screen display data and the like transmitted from the mobile terminal control unit 127 via the JPEG coder driver 131. The compressed data is stored in the USB interface 111 or the WIFI module 109. Is transmitted to the portable terminal 2 via.

  On the other hand, the mobile terminal 2 includes an application layer including various applications such as an image display application 221, a display image generation application 222, a JPEG expansion application 223, and a browser 224, a platform layer including various managers 225, 226, and the like. An OS layer including various libraries 227 and a kernel 228 and hardware 229 are included.

  The JPEG decompression application 223 decompresses the compressed data transmitted from the MFP 1.

  The display image generation application 222 is a belt-like shape for displaying the state of the mobile terminal 2 with an icon or the like or displaying a part of the message together with the operation screen of the MFP 1 displayed as a remote screen on the display unit 204 of the mobile terminal 2. A simple display area (second simple display area) is displayed. Further, when the operation screen of MFP 1 displayed as a remote screen includes a simple display area (first simple display area) on the MFP 1 side, the first simple display area and the second simple display of the device itself. It is determined whether the areas are adjacent. If it is determined that the area is not adjacent, the screen is displayed as it is. If it is determined that the area is adjacent, display control processing for avoiding adjacent display is performed. . This process will be described later.

  In this embodiment, whether or not they are adjacent to each other is determined by determining whether or not the first simple display area and the second simple display area exist on the same side of the remote screen. .

  The image display application 221 uses the remote screen display data transmitted from the MFP 1 and expanded by the JPEG development application 223 to create an operation screen of the MFP 1 as a remote screen and display it on the display unit 204.

  In this embodiment, the image display application 221, the display image generation application 222, and the JPEG expansion application 223 are configured as a part of functions realized by the cooperation application.

  Next, processing when the MFP 1 and the portable terminal 2 are operated in cooperation will be described.

  A user wirelessly connects the portable terminal 2 to the MFP 1 via a network and tries to operate the portable terminal 2 and the MFP 1 in cooperation. The cooperation operation between the portable terminal 2 and the MFP 1 is started when the user starts a cooperation application installed in the portable terminal 2.

  FIG. 4 shows the relationship between the software of the MFP 1 and the portable terminal 2. The left side shows the software on the MFP 1 side and the right side shows the software on the portable terminal 2.

  The cooperative application of the mobile terminal 2 is located between the OS of the mobile terminal 2 and the application software that is general-purpose software, and receives data from the MFP 1, processes the received data, user operation information, and the mobile terminal 2. Remote screen display data can be acquired, and the acquired data can be transmitted to the MFP 1 as necessary. That is, the cooperation software of the mobile terminal 2 is related to user operation reception processing on the mobile terminal 2 side such as application calling, reception / display processing of remote screen display data of the MFP 1 (mobile terminal side operation screen processing), and cooperation operation related Execute processing etc. The linked operation related processing includes communication with the MFP 1 of operation information indicating the contents of the operation.

  On the other hand, when the OS on the MFP 1 and the application software for the MFP 1 are all in-house designed / customized and all necessary information can be obtained, the cooperative application of the MFP 1 may not be located between the OS and the application software. Necessary information may be obtained by communicating with the OS and application software while being independent of the OS and application software. The cooperative application of the MFP 1 executes user operation reception processing on the MFP 1 side such as calling of an application, transmission processing of MFP side screen data to the portable terminal 2 (MFP side operation screen processing), cooperative operation related processing, and the like. As the cooperative operation-related processing, reception / execution processing of operation information from the portable terminal 2 and the like can be cited.

  When the user selects the cooperation operation icon displayed on the display unit 204 of the mobile terminal 2 and instructs MFP cooperation, the cooperation software of the mobile terminal 2 communicates with the registered MFP 1, and the cooperation software of the MFP 1 is executed. Call and request acquisition of remote screen display data. Thereby, the cooperative operation mode is started, and the operation screen on the MFP side is displayed on the display unit 105a of the MFP 1 and the display unit 204 of the portable terminal 2. Note that the screen display on the display unit 105a of the MFP 1 may not be performed.

  Next, the operation of the image processing cooperation system shown in FIG. 1 will be described.

  Now, as shown in FIG. 27, the operation screen of the MFP 1 created by the MFP 1 is displayed on the display unit 204a of the portable terminal 2 as a remote screen D1. In the upper area of the remote screen D1, a strip-shaped first simple display area 300 that is long on the left and right and has a small vertical width is displayed. The first simple display area 300 is an area for simply displaying the state of the MFP 1 such as out of paper or an error with icons or the like. In the first simple display area 300, not only an icon but also a simple message or a part of the message may be displayed, or only a message or a part of the message may be displayed.

  In the first simple display area 300, when a user performs an operation of pulling down the first simple display area 300 by a flick operation or the like, operation information is transmitted from the portable terminal 2 to the MFP 1, and the MFP 1 Remote screen display data in which detailed information indicating the state of the MFP 1 and the like is displayed is enlarged and transmitted to the portable terminal 2, and the portable terminal 2 receives the first display area 300. The simple display area 300 is expanded downward to display a remote screen displaying detailed information indicating the state of the MFP 1 and the like.

  Further, on the display unit 204a of the mobile terminal 2, a strip-shaped second simple display area 400 that is long on the left and right and has a small vertical width is displayed near the upper portion of the remote screen D1. The second simple display area 400 is an area for simply displaying, for example, an icon or the like with a wireless connection state with an external device. In the second simple display area, not only an icon but also a simple message or a part of the message may be displayed, or only a message or a part of the message may be displayed. Further, when the user performs an operation of pulling down the second simple display area by a flick operation or the like, the portable terminal 2 expands the second simple display area 400 downward and displays detailed information indicating the state of the MFP 1 and the like. The displayed screen is displayed.

  As a result of the remote screen D1 and the second simple display area 400 of the portable terminal 2 being displayed in this way, the first strip-like simple display area 300 and the second simple display area of the portable terminal 2 in the remote screen D1 are displayed. The area 400 is displayed adjacent to the top and bottom. For this reason, when the user performs a click operation or the like in a manner that crosses the first simple display area 300 of the remote screen D1 from the upper side to the lower side of the area, and draws the area 300 downward to display detailed information. In addition, there is a risk that the second simplified display area 400 of the portable terminal 2 will be operated by mistake. In order to prevent this, a flick operation must be performed starting from a narrow area of the first simple display area 300, and the operability is not good.

  Therefore, in this embodiment, the MFP 1 or the portable terminal 2 performs the following control in order to avoid the adjacent display of the first simple display area 300 and the second simple display area 400.

  Unless otherwise specified, after the user activates the cooperative application of the portable terminal 2, the position information of the first simple display area 300 displayed on the remote screen is transmitted from the MFP 1 to the portable terminal 2, and the portable terminal 2 receives the information. The position on the screen of the first simple display area 300 indicated by the position information is compared with the position of the position 400 of the second simple display area of the mobile terminal 2 itself, and both are located on the same side of the remote screen. If they are positioned, it is determined that they are displayed adjacent to each other, and control is performed to avoid adjacent display of the first simple display area 300 and the second simple display area 400. Note that the position information of the second simple display area 400 is transmitted from the portable terminal 2 to the MFP 1, so that the MFP 1 compares the positions of the first and second simple display areas 300 and 400, and the two areas are adjacent to each other. It may be judged whether or not.

  On the other hand, if the first simple display area 300 and the second simple display area 400 are not located on the same side of the remote screen, it is determined that they are not displayed adjacent to each other. In this case, the MFP 1 transmits remote screen display data including the first simple display area 300 to the portable terminal 2 as in the conventional case.

However, in the following control example, when the comparison of the position of the first simple display area 300 and the position of the second simple display area 400 of the mobile terminal 2 itself is mentioned, the comparison is performed at that time.
[Control Example 1]
In this control example 1, as shown in FIG. 5, the first simple display area 300 on the remote screen D1 is erased, and the icon or the like that is the display content of the first simple display area 300 is displayed on the second of the mobile terminal 2. It is displayed in the simple display area 400. In the example of FIG. 5, an icon indicating the state of the mobile terminal 2 is displayed in the left area 401 of the second simplified display area 400, and an icon indicating the state of the MFP 1 is displayed in the right area 402. .

  When the user wants to know detailed information indicating the state of the MFP 1 or the like, the user performs an operation of pulling down the second simple display area 400 by a flick operation or the like, so that the second simple display area 400 is expanded downward and the MFP 1 Detailed information indicating the status and the like is displayed.

  Note that in the drawings after FIG. 5 and the like, the first simple display area may be abbreviated as the first area and the second simple display area may be abbreviated as the second area.

  FIG. 6 shows operations of the MFP 1 and the portable terminal 2 when the icons of the first simple display area 300 as shown in FIG. 5 are displayed in the second simple display area 400 of the portable terminal 2. It is a sequence diagram.

  When the user of the portable terminal 2 activates the cooperative application, remote screen display data corresponding to the initial screen is transmitted from the MFP 1 (S1). The portable terminal 2 receives this by the cooperation application and displays it as a remote screen D1 on the display unit 204a of the portable terminal 2.

  When the user performs a touch operation on the displayed remote screen D1, coordinate information touched from the cooperative app is transmitted to the MFP 1 (S2).

  The MFP 1 displays display contents such as icons displayed in the first simplified display area 300 included in the remote screen display data which is the next operation screen data corresponding to the received coordinate information (hereinafter referred to as “status view information” or simply “ Is transmitted to the portable terminal 2 (S3). Furthermore, remote screen display data in which the first simple display area 300 is deleted is created and transmitted to the mobile terminal 2 (S4). The SV information is transmitted every time it is updated. The remote screen display data from which the first simple display area 300 is deleted is not transmitted unless there is a change.

  When the cooperative application of the mobile terminal 2 receives the SV information of the MFP 1 and the remote screen display data that does not exist in the first simple display area 300, the remote app displays the remote screen D1 on the display unit 204a based on the remote screen display data. At the same time, the SV information is sent to the OS of the portable terminal 2 (S5).

  The OS of the portable terminal 2 additionally integrates the SV information of the MFP 1 into the second simple display area 400 of the MFP 1 and displays the SV information on the display unit 204a in a manner adjacent to the remote screen D1. However, since the first simple display area 300 is not displayed on the remote screen D1, the second simple display area 400 of the mobile terminal 2 is erroneously operated in an attempt to operate the first simple display area 300 of the remote screen D1. Can eliminate the inconvenience.

  In this case, the user who wants to know the detailed contents of the SV information of the MFP 1 performs a flick operation or the like on the second simple display area 400 of the portable terminal 2 as indicated by an arrow A in FIG.

  FIG. 7 is a flowchart showing main operations of the image processing system shown in FIGS.

  In step S11, the MFP 1 determines whether its own SV information has been updated. If not updated (NO in step S11), it waits for the update. When updated (YES in step S11), in step S12, the MFP 1 transmits its own SV information to the cooperative application.

Next, in step S13, after the cooperative application transmits the SV information of MFP1 to the OS of the portable terminal 2, the OS of portable terminal 2 adds the SV information of MFP1 to its second simplified display area 400 in step S14. Combine and display.
[Control Example 2]
In this control example 2, as shown in FIG. 8, the second simple display area 400 of the portable terminal 2 is not displayed, and the icon or the like that is the display content of the second simple display area 400 is displayed on the remote screen D1 of the MFP 1. The first simple display area 300 is combined and displayed. In the example of FIG. 8, an icon or the like indicating the state of the mobile terminal 2 is displayed in the left area 301 of the first simple display area 300, and an icon or the like indicating the state of the MFP 1 is displayed in the right area 302. ing.

  When the user wants to know detailed information indicating the state of the mobile terminal 2, the user performs an operation of pulling down the first simple display area 300 by a flick operation or the like, so that the first simple display area is expanded downward. Detailed information indicating the state and the like of the mobile terminal 2 is displayed.

  FIG. 9 shows the operations of the MFP 1 and the portable terminal 2 when the icons of the second simple display area 400 as shown in FIG. 8 are displayed in the first simple display area 300 of the remote screen D1. It is a sequence diagram.

  When the user of the portable terminal 2 activates the cooperative application, remote screen display data corresponding to the initial screen is transmitted from the MFP 1 (S21). The portable terminal 2 receives this by the cooperation application and displays it as a remote screen on the display unit 204a of the portable terminal 2.

  When the user performs a touch operation on the displayed remote screen, coordinate information touched from the cooperation application is transmitted to the MFP 1 (S22).

  On the other hand, the OS of the mobile terminal 2 transmits display contents such as icons displayed in the second simplified display area 400 (“SV information” of the mobile terminal 2) to the cooperative application (S23). The SV information is transmitted to the MFP 1 (S24). The SV information is transmitted every time it is updated.

  The MFP 1 creates remote screen display data in which the received SV information of the mobile terminal 2 is added to the first simple display area 300 of the MFP 1 (S25), and transmits the data to the mobile terminal 2 (S26).

  When the cooperative application of the mobile terminal 2 receives the remote screen display data from the MFP 1, the mobile terminal 2 displays the remote screen on the display unit 204 a based on the remote screen display data, but the mobile terminal 2 has its second simplified display area. 400 is not displayed on the display unit 204a.

  As described above, since the second simple display area 400 of the mobile terminal 2 is not displayed, the second simple display area of the mobile terminal 2 is erroneously operated in an attempt to operate the first simple display area 300 of the remote screen D1. And the need for flicking operations starting from a narrow area of the first simple display area 300 is eliminated.

  In this case, the user who wants to know the detailed contents of the SV information of the portable terminal 2 performs a flick operation or the like on the first simple display area 300 of the remote screen D1, as indicated by an arrow A in FIG.

  FIG. 10 is a flowchart showing main operations of the image processing system shown in FIGS.

  In step S31, the portable terminal 2 determines whether its own SV information has been updated. If not updated (NO in step S31), it waits for the update. When updated (YES in step S31), in step S32, the OS of the mobile terminal 2 transmits its own SV information to the cooperative application.

Next, in step S <b> 33, the cooperative application transmits the SV information of the mobile terminal 2 to the MFP 1. In step S <b> 34, the MFP 1 creates remote screen display data in which the SV information of the portable terminal 2 is additionally integrated in the first simple display area 300 of the MFP 1, and transmits it to the portable terminal 2. The portable terminal 2 reproduces and displays the remote screen D1 based on the received remote screen display data.
[Control Example 3]
In this control example 3, as shown in FIG. 11, the second simplified display area 400 of the mobile terminal 2 is erased so as not to be displayed.

  FIG. 12 is a sequence diagram showing operations of MFP 1 and portable terminal 2 when second simplified display area 400 of portable terminal 2 as shown in FIG. 11 is not displayed.

  When the user of the portable terminal 2 activates the cooperative application, remote screen display data corresponding to the initial screen is transmitted from the MFP 1 (S41). The portable terminal 2 receives this by the cooperation application and displays it as a remote screen D1 on the display unit 204a of the portable terminal 2.

  When the user performs a touch operation on the displayed remote screen D1, coordinate information touched from the cooperation application is transmitted to the MFP 1 (S42).

  When the MFP 1 has a status display function (SV function), the MFP 1 requests the portable terminal 2 to hide the second simplified display area 400 of the portable terminal 2 (S43), and the first Remote screen display data, which is the next operation screen data corresponding to the coordinate information, is transmitted to the portable terminal 2 (S44).

  The cooperative application of the portable terminal 2 displays the remote screen D1 based on the remote screen display data, and transmits a second simple display area non-display request from the MFP 1 to the OS of the portable terminal 2 (S45). The OS of the mobile terminal 2 hides the second simple display area 400 of the mobile terminal 2 (S46).

  Thus, since the portable terminal 2 hides and erases the display of the second simplified display area 400 of the portable terminal 2, the second portable terminal 2 tries to operate the first simplified display area 300 of the remote screen D1. The inconvenience of erroneously operating the simple display area and the inconvenience of having to perform a flick operation starting from a narrow area of the first simple display area 300 can be solved.

  FIG. 13 is a flowchart showing main operations in the image processing system shown in FIGS.

  In step S51, the MFP 1 confirms the presence or absence of the SV function. If the SV function is present (YES in step S51), in step S52, the MFP 1 transmits a non-display request for the second simple display area 400 of the mobile terminal 2 to the cooperative application of the mobile terminal 2.

In step S53, when the cooperation application transmits a non-display request for the second simple display area 400 from the MFP 1 to the OS of the portable terminal 2, the OS of the portable terminal 2 displays the second simple display of the portable terminal itself in step S54. The area 400 is hidden.
[Control Example 4]
In this control example 4, as shown in FIG. 14, the second simple display area 400 of the mobile terminal 2 is moved from the upper side of the remote screen D1 to another side and displayed, thereby displaying the first simple display. The adjacent display of the area 300 and the second simplified display area 400 is avoided.

  FIG. 15 is a sequence diagram showing operations of the MFP 1 and the portable terminal 2 when the display position of the second simple display area of the portable terminal 2 as shown in FIG. 14 is changed.

  When the user of the portable terminal 2 activates the cooperative application, remote screen display data corresponding to the initial screen is transmitted from the MFP 1 (S61). The portable terminal 2 receives this by the cooperation application and displays it as a remote screen on the display unit 204a of the portable terminal 2.

  When the user performs a touch operation on the displayed remote screen D1, coordinate information touched from the cooperation application is transmitted to the MFP 1 (S62).

  The MFP 1 transmits the position information of the first simple display area 300 of itself to the mobile terminal 2 (S63), and transmits remote screen display data, which is the next operation screen data corresponding to the coordinate information, to the mobile terminal 2. (S64). Note that the coordinate information in the remote screen D1 may be transmitted as the position information of the first simple display area 300, and information on which side of the remote screen D1 the first simple display area 300 exists on. May be sent.

  When the cooperative application of the portable terminal 2 receives the position information of the first simple display area 300 from the MFP 1, is the first simple display area 300 of the MFP 1 and the second simple display area 400 of the portable terminal 2 adjacent to each other? In other words, it is determined whether the first simple display area 300 and the second simple display area 400 are located on the same side of the remote screen D1, and if they are adjacent, the second simple display area 400 is determined. Is transmitted to the OS of the portable terminal 2 (S65).

  The OS of the portable terminal 2 moves its second simplified display area 400 to the other side of the remote screen D1 as shown in FIG. 14 (S66).

  As described above, the mobile terminal 2 receives the position information of the first simple display area 300 from the MFP 1, and the remote screen D <b> 1 so that the second simple display area 400 is separated from the first simple display area 300. Since the display is moved to the other side, the second simple display area of the mobile terminal 2 is erroneously operated in an attempt to operate the first simple display area 300 of the remote screen D1, or the first The inconvenience of having to perform a flick operation starting from a narrow area of the simple display area 300 can be solved.

  FIG. 16 is a flowchart showing main operations in the image processing system shown in FIGS.

  In step S <b> 71, the MFP 1 transmits position information of its own first simple display area 300 included in the remote screen D <b> 1 to the cooperative app of the mobile terminal 2.

  In step S72, the cooperative application that has received the position information from the MFP 1 compares the position of the first simple display area 300 and the position of the second simple display area 400, and both exist on the same side of the remote screen D1. It is determined whether or not (S72). If they are present on the same side (YES in step S72), in step S73, the cooperative app transmits a movement request for the second simple display area 400 to the OS of the portable terminal 2. In step S74, the OS of the mobile terminal 2 moves in the second simple display area 400 in response to a movement request from the cooperation application. In step S75, the cooperative operation is continued.

If the first simple display area 300 and the second simple display area 400 do not exist on the same side of the remote screen D1 in step S72 (NO in step S72), the process proceeds directly to step S75.
[Control Example 5]
In this control example 5, as a result of enlarging the remote screen D1 having the first simple display area, the first simple display area and the second simple display area of the portable terminal 2 are displayed adjacent to each other. Is shown.

  As shown in FIG. 17A, a first simple display area 300 for simply displaying the state of the MFP 1 with icons is displayed in the remote screen D1, and a message is displayed below the first simple display area 300. It is assumed that another belt-like first simple display area 500 for simple display of the part is displayed, and a second simple display area 400 on the portable terminal 2 side is displayed adjacent to the upper part of the remote screen D1.

  When the message portion of the first simple display area 500, which is the message area, is flicked downward as shown in FIGS. 5A and 5B, the area is pulled out and expanded, and a long sentence is displayed. The

  When the user enlarges the remote screen D1, the first simple display area 500, which is a message area, is enlarged and moved upward as shown in FIG. 400 may be displayed adjacent to 400. In this state, the same problem as in the case where the first simple display area 300 and the second simple display area 400 are adjacent to each other occurs.

  Therefore, in this embodiment, when the first simple display area 500 and the second simple display area 400 are displayed adjacently as a result of the remote screen D1 being enlarged, as shown in FIG. The position of the second simple display area 400 is moved to the other side of the remote screen D1 and displayed.

Since the operations of the MFP 1 and the portable terminal 2 after the screen enlargement in this case are the same as the operations shown in FIGS. 15 and 16, the description thereof will be omitted.
[Control Example 6]
In this control example 6, as shown in FIG. 18, the first simple display area 300 of the MFP 1 is displayed by moving from the upper side of the remote screen D1 to another side and displaying it. And the adjacent display of the second simple display area 400 is avoided.

  FIG. 19 is a sequence diagram showing operations of MFP 1 and portable terminal 2 when the display position of first simple display area 300 of MFP 1 as shown in FIG. 18 is changed.

  When the user of the portable terminal 2 activates the cooperative application, remote screen display data corresponding to the initial screen is transmitted from the MFP 1 (S81). The portable terminal 2 receives this by the cooperation application and displays it as a remote screen on the display unit 204a of the portable terminal 2.

  When the user performs a touch operation on the displayed remote screen D1, coordinate information touched from the cooperation application is transmitted to the MFP 1 (S82).

  The MFP 1 transmits the position information of its own first simple display area 300 to the portable terminal 2 (S83). When the cooperation application of the portable terminal 2 receives the position information of the first simple display area 300 from the MFP 1, the cooperation application requests the position information of the second simple display area 400 on the portable terminal 2 side from the OS of the portable terminal 2 (S84). ). The OS transmits the position information of the second simple display area 400 to the cooperative app (S85).

  The cooperative application of the mobile terminal 2 determines whether the first simple display area 300 of the MFP 1 and the second simple display area 400 of the mobile terminal 2 are adjacent to each other, in other words, the first simple display area 300 and the second simple display area 400. It is determined whether or not the simple display area 400 is located on the same side of the remote screen D1, and if it is adjacent, a movement request for the first simple display area 300 is transmitted to the MFP 1 (S86).

  The MFP 1 creates remote screen display data in which the first simple display area 300 of the MFP 1 is moved to the other side of the remote screen D1 as shown in FIG. 18 (S87), and the created remote screen display data is carried around. It transmits to the cooperation application of the terminal 2 (S88).

  As described above, the MFP 1 creates the remote screen display data by moving the first simple display area 300 of the MFP 1 to the other side of the remote screen D1 so as to be separated from the second simple display area 400. Since it is displayed on the display unit 204a of the terminal 2, there is an inconvenience that the second simple display area 400 of the portable terminal 2 is erroneously operated in order to operate the first simple display area 300 of the remote screen D1. The inconvenience of having to perform a flick operation starting from a narrow area of the simple display area 300 can be solved.

  In this case, a user who wants to know detailed information of the first simple display area 300 may perform a flick operation in the width direction (right direction on the page) of the first simple display area 300 as indicated by an arrow A in FIG. .

  FIG. 20 is a flowchart showing main operations in the image processing system shown in FIGS.

  In step S <b> 91, the MFP 1 transmits position information of its own first simple display area 300 included in the remote screen D <b> 1 to the cooperative app of the mobile terminal 2.

  In step S92, the cooperative application that has received the position information from the MFP 1 compares the position of the first simple display area 300 and the position of the second simple display area 400, and both exist on the same side of the remote screen D1. It is determined whether or not (S72). If they exist on the same side (YES in step S92), the cooperative app transmits a request to move the first simple display area 300 to the MFP 1 in step S93. In step S <b> 94, the MFP 1 creates remote screen display data moved in the first simple display area 300 in response to a movement request from the cooperation application, and transmits it to the mobile terminal 2. In step S95, the cooperation operation is continued.

If the first simple display area 300 and the second simple display area 400 do not exist on the same side of the remote screen D1 in step S92 (NO in step S92), the process proceeds directly to step S95.
[Control Example 7]
In this control example 7, when there is an empty area in the second simple display area 400 of the mobile terminal 2, an icon or the like that is the display content of the first simple display area 300 is displayed in the second simple display area 400 of the mobile terminal 2. When there is no free area, the first simple display area 300 is moved away from the second simple display area 400 as shown in FIGS. Change and display.

  FIG. 22 shows the MFP 1 and the portable terminal when the first simple display area 300 is moved to a position away from the second simple display area 400 and displayed when there is no empty area in the second simple display area 400. FIG.

  When the user of the portable terminal 2 activates the cooperative application, remote screen display data corresponding to the initial screen is transmitted from the MFP 1 (S101). The portable terminal 2 receives this by the cooperation application and displays it as a remote screen D1 on the display unit 204a of the portable terminal 2.

  When the user performs a touch operation on the displayed remote screen D1, coordinate information touched from the cooperation application is transmitted to the MFP 1 (S102).

  The MFP 1 transmits its own SV information to the portable terminal 2 (S103). When the cooperative application of the mobile terminal 2 receives the SV information of the MFP 1, it requests the free area size of the second simple display area 400 from the OS of the mobile terminal 2 (S 104), and the OS determines the free area size. (S105).

  The cooperative app determines whether the SV information of the MFP 1 can be displayed in the second simplified display area 400 of the mobile terminal 2 (S106). If the second simple display area 400 has no free space and the SV information of the MFP 1 cannot be displayed, the cooperative application of the mobile terminal 2 requests the MFP 1 to move the first simple display area 300 (S107), and the MFP 1 itself As shown in FIG. 21B, remote screen display data is generated by moving the first simple display area 300 to the other side of the remote screen D1 as shown in FIG. 21B (S108), and is transmitted to the cooperative application (S109).

  As described above, when the mobile terminal 2 determines that there is no free area in the second simple display area 400 where the display content of the first simple display area 300 can be displayed, the MFP 1 performs the first simple display. Remote screen display data in which the area 300 is moved to a position away from the second simplified display area 400 is created and transmitted to the portable terminal 2. Therefore, in the state where the remote screen D1 is displayed on the portable terminal 2, the adjacent display state of the first simple display area 300 and the second simple display area 400 is avoided, and therefore a flick operation is performed starting from a narrow area. This eliminates the need to improve the operability and reliably eliminate the inconvenience of erroneously operating the second simple display area 400 of the portable terminal 2 in an attempt to operate the first simple display area 300 on the remote screen.

  FIG. 23 is a flowchart showing main operations of the image processing system shown in FIGS.

  In step S111, the MFP 1 transmits its own SV information to the cooperative application. In step S112, the cooperative app acquires the size of the empty area of the second simple display area 400 of the mobile terminal 2, and then displays the SV information of the MFP 1 in the second simple display area 400 of the mobile terminal 2 in step S113. Determine if you can.

  If the SV information of the MFP 1 cannot be displayed in the second simple display area 400 (NO in step S113), in step S114, the cooperative application requests the MFP 1 to move the first simple display area 300, and the MFP 1 in step S115. Then, the remote screen display data moved in its own first simple display area 300 is created and transmitted to the portable terminal 2. In step S116, the cooperative operation is continued.

On the other hand, if the SV information of the MFP 1 can be displayed in the second simplified display area 400 of the mobile terminal 2 in step S113 (YES in step S113), the MFP 1 displays the SV information of the MFP 1 in cooperation with the mobile terminal 2 in step 117. In step S118, the cooperative application transmits the SV information of the MFP 1 to the OS of the mobile terminal 2. In step S119, the OS adds the SV information of the MFP 1 to the second simplified display area 400 of its own and displays it, and then proceeds to step S116.
[Control Example 8]
In this control example 8, when there is an empty area in the first simple display area 300 of the MFP 1, an icon or the like that is the display content of the second simple display area 400 of the mobile terminal 2 is displayed in the first simple display area of the MFP 1. When the display is combined with 300 and there is no free space, the second simple display area 400 is separated from the first simple display area 300 as shown in FIGS. It is changed to display.

  FIG. 25 shows the MFP 1 and the portable terminal when the second simple display area 400 is moved to a position away from the first simple display area 300 and displayed when there is no empty area in the first simple display area 300. FIG.

  When the user of the portable terminal 2 activates the cooperative application, remote screen display data corresponding to the initial screen is transmitted from the MFP 1 (S121). The portable terminal 2 receives this by the cooperation application and displays it as a remote screen D1 on the display unit 204a of the portable terminal 2.

  When the user performs a touch operation on the displayed remote screen D1, coordinate information touched from the cooperative app is transmitted to the MFP 1 (S122).

  The OS of the portable terminal 2 transmits its own SV information to the cooperation application (S123). The cooperative application requests the empty area size of the first simple display area from the MFP 1 (S124), and the MFP 1 returns the empty area size to the cooperative application (S125).

  The cooperative app determines whether the SV information of the mobile terminal 2 can be displayed in the first simple display area 300 of the MFP 1 (S126). When there is no free space in the first simple display area 300 and the SV information of the mobile terminal 2 cannot be displayed, the cooperative application of the mobile terminal 2 requests the OS of the mobile terminal 2 to move the second simple display area 400 ( In S127), the OS moves and displays its second simplified display area 400 to the other side of the remote screen D1 as shown in FIG.

  As described above, when it is determined by the mobile terminal 2 that there is no empty area in which the display content of the second simple display area 400 can be displayed in the first simple display area 300, the mobile terminal 2 The simple display area 400 is moved to a position away from the first simple display area 300 and displayed. Therefore, in the state where the remote screen D1 is displayed on the portable terminal 2, the adjacent display state of the first simple display area 300 and the second simple display area 400 is avoided, and therefore a flick operation is performed starting from a narrow area. This eliminates the need to improve the operability and reliably eliminate the inconvenience of erroneously operating the second simple display area 400 of the portable terminal 2 in an attempt to operate the first simple display area 300 on the remote screen.

  FIG. 26 is a flowchart showing main operations of the image processing system shown in FIGS.

  In step S131, the OS of the mobile terminal 2 transmits the SV information of the mobile terminal 2 to the cooperation application. In step S132, after the cooperative application acquires the size of the empty area of the first simple display area 300 of the MFP 1, can the SV information of the portable terminal 2 be displayed in the first simple display area 300 of the MFP 1 in step S133? Judge whether or not.

  When the SV information of the portable terminal 2 cannot be displayed in the first simple display area 300 (NO in step S133), the cooperation application requests the OS of the portable terminal 2 to move the second simple display area 400 in step S134. In step S135, the OS of the portable terminal 2 moves and displays its own second simple display area 400. In step S136, the cooperative operation is continued.

  On the other hand, if the SV information of the portable terminal 2 can be displayed in the first simplified display area 300 of the MFP 1 in step S133 (YES in step S133), the OS of the portable terminal 2 sends its SV information to the linked application in step 137. In step S138, the cooperative app transmits the SV information of the portable terminal 2 to the MFP 1. In step S139, the MFP 1 creates remote screen display data in which the SV information of the portable terminal 2 is added to the first simple display area 300 of the MFP 1 and transmits it to the cooperative application of the portable terminal 2, and then proceeds to step S136.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment. For example, it is determined whether or not the first simple display area 300 of the MFP 1 and the second simple display area 400 of the mobile terminal 2 are displayed adjacent to each other, and the first simple display area 300 and the second simple display. The mobile terminal 2 determines whether or not there is a free area in the area 400, but the MFP 1 may be configured.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Portable terminal device 101 CPU
102 ROM
107b Display unit 201 CPU
202 RAM
203 Storage Unit 204a Display Unit 204b Operation Unit 209 Cooperation Unit 300, 500 First Simple Display Area 400 Second Simple Display Area

Claims (10)

An image processing system comprising an image processing device and a portable terminal device capable of remotely operating the image processing device,
The image processing apparatus includes:
A simple display area for simple display of at least the status of the device and / or a message, wherein when the user performs a pull-out operation on the simple display area, the simple display area is pulled out to display detailed information. Display data creation means for creating remote screen display data for causing the mobile terminal device to display an operation screen of the own device having a simple display area as a remote screen;
Transmitting means for transmitting the created remote screen display data to a mobile terminal device;
Receiving means for receiving, from the mobile terminal device, operation information performed by the user on the remote screen displayed on the mobile terminal device based on the remote screen display data;
When the operation information is received by the receiving means, the display data creating means creates the remote screen display data according to the received operation information, and the sending means is created Transmit remote screen display data to the mobile terminal device,
The portable terminal device
Receiving means for receiving the remote screen display data from the image processing device;
Display means;
Based on the remote screen display data received by the receiving means, the operation screen of the image processing apparatus is displayed as a remote screen on the display means, and at least a second simple display for simply displaying the status and / or message of the own apparatus. Display control means for displaying an area together with the remote screen and for pulling out the simple display area and displaying detailed information on the display means when a user performs a pull-out operation on the second simple display area;
When the user performs an operation on the remote screen displayed on the display unit, a transmission unit that transmits operation information corresponding to the operation to the image processing device;
With
When the first simple display area on the remote screen of the image processing device and the second simple display area displayed on the display means of the portable terminal device are adjacent to each other, the image processing device or the portable terminal device Is an image processing system that performs control to avoid adjacent display of the first and second simple display areas.   The image processing device transmits the display contents of the first simple display area to the mobile terminal device, and creates remote screen display data in which the first simple display area is deleted to the mobile terminal device. And the portable terminal device receives the display contents of the first simple display area and displays the contents in the second simple display area, whereby the first and second simple display areas are displayed. The image processing system according to claim 1, wherein adjacent display is avoided.   The portable terminal device transmits the display content of the second simple display area to the image processing device, hides the second simple display area, and the image processing device displays the second simple display area. Of the first and second simple display areas by generating remote screen display data for displaying the contents in the first simple display area and transmitting the data to the portable terminal device. The image processing system according to claim 1, wherein adjacent display is avoided.   The image processing system according to claim 1, wherein the portable terminal device avoids adjacent display of the first and second simple display areas by hiding the second simple display area.   The image processing device transmits the position information of the first simple display area to the portable terminal device, the portable terminal device receives the position information of the first simple display area, and receives the second information. The image processing system according to claim 1, wherein adjacent display of the first and second simple display areas is avoided by moving the simple display area to a position away from the first simple display area.   The portable terminal device transmits position information of the second simple display area to the image processing device, and the image processing device receives the position information of the second simple display area, and By creating remote screen display data in which the simple display area is moved to a position away from the second simple display area and transmitting it to the portable terminal device, adjacent display of the first and second simple display areas is performed. The image processing system according to claim 1, wherein: The portable terminal device or the image processing device includes a determination unit that determines whether or not there is a free area that can display the display content of the first simple display area in the second simple display area,
When it is determined by the determining means that a free area exists, the mobile terminal device receives the display content of the first simple display area, displays the content in the second simple display area, and When it is determined that there is no image, the image processing device creates remote screen display data by moving the first simple display area to a position away from the second simple display area, and the portable terminal device The image processing system according to claim 2, wherein adjacent display of the first and second simple display areas is avoided by transmitting to. The portable terminal device or the image processing device includes a determination unit that determines whether or not there is a free area in which the display content of the second simple display area can be displayed in the first simple display area,
When it is determined by the determining means that a free area exists, the image processing apparatus receives the display content of the second simple display area and displays the content in the first simple display area of itself. Remote screen display data is created and transmitted to the portable terminal device. When it is determined that there is no free area, the portable terminal device separates the second simple display area from the first simple display area. The image processing system according to claim 3, wherein adjacent display of the first and second simple display areas is avoided by moving the display to a position where the display is performed.   An image processing apparatus used in the image processing system according to claim 1.   The portable terminal device used for the image processing system in any one of Claims 1-8.

Images