JP6418258B2 - Image processing apparatus and remote screen control program - Google Patents

Description

This invention smartphone, an image processing equipment and the remote screen control program such as a MFP (Multi Function Peripherals) is a multifunctional digital image forming apparatus connectable to the portable terminal device such as a tablet terminal by wireless communication.

  In recent years, portable terminal devices such as smartphones and tablet terminals can store digitized documents, and can browse and edit documents regardless of time and place. In addition, a file edited by the portable terminal device can be transmitted to an image processing apparatus such as the MFP, and the image processing apparatus can execute processing such as printing or saving. Furthermore, a file composed of image data of a document read by the scanner unit of the image processing apparatus can be transmitted to the mobile terminal device, and can be edited by the mobile terminal device.

  However, it is necessary to operate the mobile terminal device and the image processing device separately, and there is a problem that the operation becomes complicated.

  Therefore, recently, the operation screen of the image processing apparatus is displayed as a remote screen on the display unit of the portable terminal device, and the image processing apparatus and the portable terminal device can be operated cooperatively from the portable terminal device. A system has been proposed. In this cooperative system, the remote screen on the image processing device side displayed on the display unit of the mobile terminal device is displayed based on the screen data transmitted from the image processing device. The remote screen based on this screen data is essentially the same as the screen displayed on the operation panel of the image processing apparatus.

  That is, information indicating that an operation has been performed is transmitted from the mobile terminal device to the image processing device, and the image processing device generates next screen data based on this information, and transmits this screen data to the mobile terminal device. The mobile terminal device updates and displays the remote screen based on the screen data.

  On the other hand, the mobile terminal device includes a touch panel. When the user performs a touch operation on the display screen, position information (coordinate data) of the touch operation is transmitted to the image processing device, and the image processing device performs the touch operation. In response, the next screen data is generated or the designated job is executed.

  By the way, the image processing apparatus is often provided with an external key composed of hardware such as a hard numeric keypad, a hard keyboard, and a switch.

  In some cases, the operation screen may be a screen having, for example, a set value input area that can be operated using the outer key. In this case, if an operation screen is displayed on the operation panel of the image processing apparatus, a set value can be input using the outer key. In some cases, software keys such as a soft keyboard and a software numeric keypad are displayed on the operation panel, and setting values can be input using the software keys.

  On the other hand, the setting value input in the mobile terminal device is often performed by a method of inputting by a software key displayed when the setting value input display area on the screen is tapped, but the user is in the vicinity of the image processing device. When the mobile terminal device is operated, it is not necessary to display software keys that are displayed so as to overlap the screen, and therefore it may be easier to operate using the physical external key.

  In Patent Document 1, an optimal menu screen is selected and displayed according to the connection state of an optional device, and when the optional device function setting items have a hierarchical structure, the connection is made when the optional device is connected. A technique for changing the setting item of the optional device to the top is disclosed.

  In Patent Document 2, an icon indicating which input means can be used for the input area displayed on the panel is displayed in the small display area, while clarifying which input means should be used. The technique which can change an input means flexibly is disclosed.

  Patent Document 3 discloses a technique for changing display contents according to a connected external device.

JP 2007-83620 A JP 2006-155232 A JP 2005-55492 A

  However, as described above, when the user operates the portable terminal device in the vicinity of the image processing device, even if the user wants to use the outer key, conventionally, the distance between the image processing device and the portable terminal device is not known. Regardless of the distance, only the display that can use the software key is displayed on the portable terminal device, so that the user cannot recognize that the external key of the image processing device can be used, which has a problem in terms of operability. .

  Such a problem could not be solved even by the techniques described in Patent Documents 1 to 3 described above.

The present invention has been made in view of the above technical background, and the operation screen of the image processing apparatus displayed as a remote screen on the mobile terminal device has an external key configured by hardware on the image processing apparatus side. If the operation screen can be used and input, a remote screen that allows the user to recognize that the external key of the image processing device can be used according to the distance between the image processing device and the mobile terminal device is carried. It is an object of the present invention to provide an image processing device that can be displayed on a terminal device, and to provide a remote screen control program.

The above problem is solved by the following means.
(1) Connection means connectable to the mobile terminal device, an external key configured by hardware, distance detection means for detecting a distance between the mobile terminal device connected by the connection means, and the connection means Transmitting means for transmitting data for causing the portable terminal device to display an operation screen of the device as a remote screen in a state where the portable terminal device is connected to the portable terminal device; The device is connected to the mobile terminal device, and receives a request for transmission of screen data corresponding to the external key from the mobile terminal device , and the operation screen can be input using the external key. If the distance to the portable terminal device detected by the distance detecting means is less than a certain value in the case of a simple operation screen, a display prompting the use of the external key is displayed on the portable Transmitting the data for the end device to the portable terminal device, an image processing apparatus characterized by.
(2) When the distance from the mobile terminal device detected by the distance detection unit exceeds a certain value, the transmission unit transmits key arrangement information corresponding to the outer key to the mobile terminal device; The preceding paragraph 1 is provided with receiving means for receiving, from the portable terminal device, key on / off information when a user operates a key corresponding to an external key displayed in correspondence with the arrangement information on the portable terminal device. An image processing apparatus according to 1.
(3) When the distance from the mobile terminal device detected by the distance detection unit exceeds a certain value, the transmission unit displays an operation screen on which a key corresponding to the outer key is arranged on the mobile terminal device The coordinate data of the touch position when the user operates the key corresponding to the external key on the operation screen displayed on the mobile terminal device is transmitted from the mobile terminal device. 2. The image processing apparatus according to item 1, further comprising receiving means for receiving.
(4) The image processing apparatus according to any one of Items 1 to 3, wherein the external key is at least one of a hard numeric keypad, a hard keyboard, and a switch.
(5) The display for prompting the use of the external key when the distance from the mobile terminal device detected by the distance detection means is a predetermined value or less is a display of a message indicating that the external key can be used 1 The image processing apparatus in any one of -4.
(6) The operation screen has at least one set value input area, and the display for prompting the use of the outer key when the distance from the mobile terminal device detected by the distance detection unit is a predetermined value or less is 6. The image processing device according to any one of the preceding items 1 to 5, which is a display indicating that the set value input area is in a selected state.
(7) When the distance from the portable terminal device detected by the distance detection unit exceeds a certain value, the transmission unit displays the operation screen on which the display for prompting the display of the key corresponding to the external key is made. Data to be displayed on the terminal device is transmitted to the mobile terminal device, the operation screen has at least one set value input area, and a display for prompting display of a key corresponding to the external key is the set value input 7. The image processing apparatus according to any one of the preceding items 1 to 6, which is a display indicating that an area is not selected.
(8) The key corresponding to the foreign key is an emulation of at least one of a preview key, a guide key, an enlarged display key, a utility key, and a job mode switching key that exists as the foreign key. The image processing apparatus according to any one of 2 to 7.
(9) Data for causing the mobile terminal device to display a screen when the distance from the mobile terminal device detected by the distance detection means is a predetermined value or less is transmitted to the mobile terminal device, and the mobile terminal device stores the data When the distance from the mobile terminal device detected by the distance detection unit exceeds a certain value when the screen based on the display is displayed, the transmission unit has exceeded the certain value from the mobile terminal device 9. The image processing device according to any one of the preceding items 1 to 8, wherein data for causing the mobile terminal device to display the screen of the case is transmitted again to the mobile terminal device.
(10) Data for causing the mobile terminal device to display a screen when the distance from the mobile terminal device detected by the distance detection unit exceeds a certain value is transmitted to the mobile terminal device, and the mobile terminal device stores the data If the distance from the mobile terminal device detected by the distance detection means is equal to or less than a certain value when the screen based on the screen is displayed, the transmission means 10. The image processing device according to any one of the preceding items 1 to 9, wherein data for displaying the screen of the case on the mobile terminal device is transmitted again to the terminal device.
(11) A distance for detecting a distance from the portable terminal device connected by the connecting means to the computer of the image processing apparatus having a connecting means connectable to the portable terminal device and an external key configured by hardware A detecting step, and a transmitting step of transmitting data for displaying the operation screen of the own device as a remote screen to the mobile terminal device in a state of being connected to the mobile terminal device by the connecting means. In the transmission step, the device is connected to the portable terminal device , and receives a transmission request for screen data corresponding to an external key from the portable terminal device. Is an operation screen that can be input using the outer key, the distance from the mobile terminal device detected by the distance detection step is If it is value or less, the remote screen control program for executing the data for the display prompting the use of the outer key to the portable terminal device processing to be transmitted to the portable terminal device, to the computer.
(12) When the distance from the mobile terminal device detected by the distance detection step exceeds a certain value, the transmission step includes a process of transmitting key arrangement information corresponding to the outer key to the mobile terminal device. A receiving step of receiving from the mobile terminal device key on / off information when the user operates a key corresponding to the external key displayed on the mobile terminal device corresponding to the arrangement information. 12. The remote screen control program according to item 11, wherein the computer is further executed.
(13) When the distance to the mobile terminal device detected by the distance detection step exceeds a certain value, the transmission step displays an operation screen on which a key corresponding to the outer key is arranged on the mobile terminal device. Coordinate data of the touch position when the user operates the key corresponding to the outer key on the operation screen displayed on the mobile terminal device, causing the computer to execute processing for transmitting data to be displayed to the mobile terminal device 12. The remote screen control program according to claim 11, further causing the computer to execute a reception step of receiving the message from the portable terminal device.
(14) The remote screen control program according to any one of items 11 to 13, wherein the external key is at least one of a hard numeric keypad, a hard keyboard, and a switch.
(15) The display for prompting the use of the external key when the distance from the mobile terminal device detected by the distance detection step is equal to or less than a predetermined value is a display of a message indicating that the external key can be used. The remote screen control program according to any one of -14.
(16) The operation screen has at least one set value input area, and the display for prompting the use of the external key when the distance from the mobile terminal device detected by the distance detection step is equal to or less than a predetermined value is 16. The remote screen control program according to any one of the preceding items 11 to 15, which is a display indicating that the set value input area is in a selected state.
(17) When the distance from the portable terminal device detected by the distance detection step exceeds a certain value, the transmission step displays an operation screen on which a display prompting display of a key corresponding to the external key is made. Causing the computer to execute processing for transmitting data to be displayed on the terminal device to the mobile terminal device, and the operation screen has at least one set value input area, and prompts the display of a key corresponding to the external key. 17. The remote screen control program according to any one of the preceding items 11 to 16, which is a display indicating that the set value input area is not selected.
(18) The key corresponding to the foreign key is an emulation of at least one of a preview key, a guide key, an enlarged display key, a utility key, and a job mode switching key that exists as the foreign key. The remote screen control program according to any one of 11 to 17.
(19) Data for causing the mobile terminal device to display a screen when the distance from the mobile terminal device detected by the distance detection step is equal to or less than a certain value is transmitted to the mobile terminal device, and the mobile terminal device stores the data When the distance from the mobile terminal device detected by the distance detection step exceeds a certain value when the screen based on the screen is displayed, the distance from the mobile terminal device exceeds a certain value in the transmission step 19. The remote screen control program according to any one of the preceding items 11 to 18, which causes the computer to execute a process of transmitting data for causing the mobile terminal device to display data for displaying the screen in the case to the mobile terminal device again.
(20) Data for causing the mobile terminal device to display a screen when the distance to the mobile terminal device detected by the distance detection step exceeds a certain value is transmitted to the mobile terminal device, and the mobile terminal device stores the data If the distance from the mobile terminal device detected by the distance detection step is equal to or less than a certain value when the screen based on the screen is displayed, the distance from the portable terminal device is equal to or less than the certain value in the transmission step. 20. The remote screen control program according to any one of the preceding items 11 to 19, which causes the computer to execute a process of retransmitting data for causing the mobile terminal device to display data for displaying the screen in the case to the mobile terminal device.

According to the invention described in (1), the image processing device is connected to the mobile terminal device , and the transmission request for screen data corresponding to the external key is received from the mobile terminal device. In the case where the operation using the external key is possible, if the distance from the mobile terminal device is equal to or less than a certain value, the image processing device displays a display prompting the use of the external key configured by hardware on the mobile terminal device. Since the data to be transmitted is transmitted to the mobile terminal device, the mobile terminal device is displayed to prompt the use of the external key when the distance from the image processing device is short.

Therefore, Ru can encourage the use of external key to the user by the display.

According to the invention as described in the preceding paragraph (2), when the distance between the portable terminal device exceeds a certain value, since the location information of the key corresponding to the outer key is transmitted to the mobile terminal device, portable terminal device A key corresponding to the external key can be displayed based on the arrangement information, and when the user operates the key corresponding to the external key, the on / off information of the key is transmitted from the portable terminal device to the image processing device. Sent.

According to the invention described in (3) above, when the distance to the mobile terminal device exceeds a certain value , data for displaying the operation screen on which the key corresponding to the external key is arranged on the mobile terminal device is portable. Since it is transmitted to the terminal device, the mobile terminal device can display an operation screen on which a key corresponding to the external key is arranged, and when the user operates the key corresponding to the external key, the coordinates of the touch position Data is transmitted from the portable terminal device to the image processing device.

  According to the invention described in (4) above, when the distance between the image processing device and the mobile terminal device is short, the user uses at least one of a hard numeric keypad, a hard keyboard, and a switch to operate the operation screen. Input can be made.

  According to the invention described in (5) above, when the distance between the image processing device and the portable terminal device is equal to or less than a certain value, a message indicating that the external key can be used is displayed as a display prompting the use of the external key.

  According to the invention described in item (6) above, the operation screen has at least one set value input area, and when the distance between the image processing device and the mobile terminal device is equal to or less than a certain value, the use of the outer key is prohibited. As a prompting display, a display indicating that the set value input area is in a selected state is made.

  According to the invention described in (7) above, the operation screen has at least one set value input area, and corresponds to the outer key when the distance between the image processing device and the mobile terminal device exceeds a certain value. As a display prompting the display of the key, a display indicating that the set value input area is not selected is made.

  According to the invention described in item (8) above, when the distance between the image processing device and the portable terminal device exceeds a certain value, a preview key, a guide key, an enlarged display key, a utility as a key corresponding to the outer key A key emulating at least one of the key and the job mode switching key is displayed on the operation screen.

According to the invention described in item (9) above, data for causing the mobile terminal device to display a screen when the distance between the image processing device and the mobile terminal device is equal to or less than a certain value is transmitted to the mobile terminal device. when the remote screen based on 該De over data in device is displayed, when the distance between the portable terminal device exceeds a predetermined value, the remote screen displayed on the mobile terminal device, the distance is a predetermined value Switch to the screen when exceeding.

According to the invention described in (10) above, data for displaying on the mobile terminal device a screen when the distance between the image processing device and the mobile terminal device exceeds a certain value is transmitted to the mobile terminal device. when the remote screen based on 該De over data in device is displayed, when the distance between the portable terminal device becomes a predetermined value or less, the remote screen displayed on the mobile terminal device, the distance constant value Switch to the screen in the following cases.

According to the invention described in the preceding item ( 11 ), when the distance between the image processing device and the mobile terminal device is short , data for causing the mobile terminal device to display to prompt the use of the external key is transmitted to the mobile terminal device. The processing can be executed by a computer of the image processing apparatus.

1 is a configuration diagram of a display processing system in which an image processing apparatus according to an embodiment of the present invention is used. It is a block diagram which shows the electrical structure of an image processing apparatus and a portable terminal device. It is a block diagram which shows the firmware structure of an image processing apparatus and a portable terminal device. It is a figure which shows the relationship between each software of an image processing apparatus and a portable terminal device. It is a top view of an operation panel part. It is a figure which shows the remote screen displayed on the display part of a portable terminal, when the distance of an image processing apparatus and a portable terminal device exceeds the threshold value. It is a figure which shows the remote screen displayed on the display part of a portable terminal, when the distance of an image processing apparatus and a portable terminal device is less than a threshold value. It is a figure which shows the other example of the remote screen displayed on the display part of a portable terminal, when the distance of an image processing apparatus and a portable terminal device is less than a threshold value. It is a figure which shows the other example of the remote screen displayed on the display part of a portable terminal, when the distance of an image processing apparatus and a portable terminal device exceeds a threshold value. FIG. 10 is a diagram illustrating another embodiment of the present invention, which is an example of a remote screen displayed on a display unit of a mobile terminal when a distance between the image processing apparatus and the mobile terminal exceeds a threshold value. Similarly, when the distance between the image processing device and the mobile terminal device falls within the threshold from the distance exceeding the threshold, it is a diagram illustrating an example of a remote screen that is switched and displayed on the display unit of the mobile terminal. It is a sequence diagram of the information communication performed between an image processing apparatus and a portable terminal device. It is a sequence diagram which shows the other example of the information communication performed between an image processing apparatus and a portable terminal device. It is a flowchart which shows operation | movement of an image processing apparatus. It is a flowchart which shows operation | movement of a portable terminal device.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a configuration diagram of a display processing system using an image processing apparatus according to an embodiment of the present invention.

  The display processing system includes an image forming apparatus 1 as an image processing apparatus, a mobile terminal device 2 including a smartphone or a tablet terminal, and the like. The image forming apparatus 1 and the mobile terminal device 2 include a wireless LAN router 3, They can be connected to each other via the network 4. Note that the image forming apparatus 1 and the portable terminal device 2 can be connected to each other by a USB interface or the like, but in this embodiment, they are connected to each other by wireless communication.

  In this embodiment, the above-described MFP having a copy function, a print function, a scan function, a facsimile function and the like is used as the image forming apparatus 1. Hereinafter, the image forming apparatus is also referred to as an MFP. The mobile terminal device is also referred to as a mobile terminal.

  FIG. 2 is a block diagram showing an electrical configuration of the MFP 1 and the portable terminal 2.

The MFP 1 includes a CPU 101, ROM 102, RAM 103, scanner unit 104, print unit 105, storage unit 106, operation panel unit 107, WIFI (Wireless Fidelity) module 109, LAN (Local Area Network) controller 110, USB interface (USB I / F). 111 etc. are provided.

  The CPU 101 performs overall control of the entire MFP 1 by operating according to an operation program stored in the ROM 102 or the like. For example, in addition to controlling various functions of the MFP 1, in this embodiment, the own device and the portable terminal are compared by a method such as comparing the position information of the portable terminal 2 acquired from the portable terminal 2 with the position information of the own device. 2 is detected, whether the detected distance is equal to or less than a preset threshold value, or based on a request from the mobile terminal 2, an operation screen of the MFP 1 displayed as a remote screen on the mobile terminal 2 Screen data is created and transmitted to the mobile terminal 2, or next screen data corresponding to the operation information transmitted from the mobile terminal 2 is generated and transmitted to the mobile terminal 2.

At this time, when the operation screen can be input using an external key (described later) configured by hardware, if the distance from the mobile terminal 2 is equal to or less than a threshold value, a display prompting the use of the external key is made. Screen data to be displayed on the mobile terminal 2 as a remote screen, and when the distance from the mobile terminal 2 exceeds a threshold, the operation screen on which a key corresponding to the outside key is displayed, or Screen data for causing the portable terminal 2 to display the operation screen on which the display of prompting the key is displayed as a remote screen is generated.

  The ROM 102 is a memory that stores an operation program of the CPU 101, and the RAM 103 is a memory that provides a work area when the CPU 101 operates according to the operation program.

  The scanner unit 104 reads an image of a document set on a platen glass (not shown) or a document set on an automatic document feeder (not shown) and outputs image data as electronic data.

  The print unit 105 prints image data of a document read by the scanner unit 104 or print data transmitted from the portable terminal 2 or the like on a sheet.

  The storage unit 106 stores various data, applications, and the like, and uses a hard disk device or the like. In particular, in this embodiment, display data (screen data) for various operation screens displayed on the operation panel unit 107, arrangement information of external keys, and the like are stored, and display of operation screens according to user operations is stored. Data is called from the storage unit 106 by the CPU 101 and displayed on the operation panel unit 107 or transmitted to the mobile terminal 2. Further, a plurality of files such as a document file composed of image data read by the scanner unit 104, a document file transmitted from an external device, and position information of the installation location of the own device are stored.

  The operation panel unit 107 operates the MFP 1 and displays a message or the like to the user. As shown in FIG. 5, a key unit 107a having a start key 121 as an external key, a hard numeric key 122, a switch, etc. A display unit 107b including a touch panel such as a liquid crystal is provided. The external key is not limited to a start key, a hard numeric keypad, and a switch, and may be a hard keyboard, a preview key, a guide key, an enlarged display key, a utility key, a job mode switching key, or the like.

  The WIFI module 109 is an interface for wireless connection with the portable terminal 2, and the LAN controller 110 controls communication with an external device such as a personal computer or another MFP.

  The USB interface 111 is an interface for wired connection with the portable terminal 2.

  On the other hand, a smart phone, a tablet terminal, electronic paper, etc. are used for the portable terminal 2. FIG. The portable terminal 2 includes a CPU 201, a RAM 202, a storage unit 203, a display unit 204a, an operation unit 204b, a WIFI module 205, a LAN controller 206, a USB interface 207, file processing units 208a and 208b, a cooperation unit 209, a position information acquisition unit 210, and the like. It has.

  The CPU 201 controls the entire mobile terminal 2 in an integrated manner, and operates according to an OS (Operating System) that is basic software, a cooperative application, or a general-purpose application described later.

  The RAM 202 is a memory serving as a work area when the CPU 201 operates according to an operation program.

  The storage unit 203 stores various data, applications, and the like, and a hard disk device or the like is used. In this embodiment, screen data transmitted from the MFP 1 and programs such as a basic OS, a cooperative application, and a general-purpose application are stored. In the following description, an application is also simply referred to as an application.

  The display unit 204a is a device such as a liquid crystal, and the operation unit 204b is a touch panel provided on the display unit 204a and capable of touch operation input. The touch operation is usually performed by a gesture operation that defines the operation content according to the number of touched fingers and the movement of the fingers. As an example of the gesture operation, a pinch-in operation or a pinch-out operation for instructing an editing process such as scaling (enlargement or reduction) or rotation with respect to an image displayed on the screen can be exemplified.

  The WIFI module 205 is an interface for wirelessly connecting the mobile terminal 2 to the MFP 1, and the LAN controller 206 controls communication with an external device such as a user terminal. The USB interface 207 is an interface for connecting the portable terminal device 2 to the MFP 1 by wire.

  The file processing units 208a and 208b perform processing such as editing, forwarding, or mail delivery on files. Hereinafter, both the file processing units 208a and 208b are referred to as a file processing unit 208.

  The cooperation unit 209 is for enabling the functions of the MFP 1 to be used in order to smoothly perform the function cooperation between the MFP 1 and the portable terminal 2. Specifically, screen data for reproducing the same screen as the screen displayed on the operation panel unit 107 of the MFP 1 in order to use the copy function, print function, scan function, facsimile function, etc. of the MFP 1 is sent from the MFP 1. Received and displayed on the display unit 204a as a remote screen, or when the user operates the displayed remote screen, the operation unit 204b detects the coordinates of the operation position on the screen (touch panel) and transmits it to the MFP 1 as operation information. To do. The MFP 1 analyzes the coordinates of the operation position, identifies the user's operation instruction, and executes an operation according to the operation instruction.

  The function of the file processing unit 208 is realized by the CPU 201 executing a general-purpose application. In other words, the general-purpose application is software for realizing the file processing unit 208 that processes various files. For example, a file editing application, an application having a function of distributing a file by e-mail, and a function of storing a file on the web Various kinds of applications such as an application that the user has can be used.

  The cooperation unit 209 realizes these functions when the CPU 201 executes the cooperation application described above. That is, the cooperation application is software for achieving smooth function cooperation between the MFP 1 and the mobile terminal 2. In addition, file transfer and download can be performed between the cooperative app and the general-purpose app.

  The position information acquisition unit 210 acquires the current position information of the mobile terminal 2 by the GPS function. The acquired position information is transmitted from the portable terminal 2 to the MFP 1 via the cooperation unit 209. In this embodiment, the position information is acquired periodically or when requested by the MFP 1 at any other timing and transmitted to the MFP 1.

  FIG. 3 is a block diagram illustrating firmware configurations of the MFP 1 and the portable terminal 2.

The MFP 1 includes an operation panel (application) 121 and an open API (Open Appli) in the application layer.
cation Program Interface) 122 and the like, the common API layer includes the common API (Common API) 125, the panel display image generation unit 126, and the like, and the hardware-dependent layer includes the panel screen generation unit 123, the panel control unit 124, and the mobile phone. A terminal control unit 127 and the like are provided.

  In addition, the kernel layer includes a panel input control unit 129, a panel LCD control unit 130, a JPEG coder driver 131, and the like, and a device layer includes an operation panel unit 107, a USB interface unit 111, a JPEG compression unit 133, a WIFI module 109, and the like. I have.

  The panel screen generation unit 123 generates screen data on the MFP 1 side displayed on the operation panel unit 107 when a cooperative operation is performed from the mobile terminal 2.

  The panel control unit 124 controls the communication state with the operation panel unit 107. Further, the panel display image generation unit 126 generates next screen data to be displayed on the mobile terminal 2 based on operation information from the mobile terminal 2.

  The portable terminal control unit 127 controls an interface or the like for communicating with the portable terminal 2 during the cooperation process with the portable terminal 2.

  The panel input control unit 129 performs control related to operation input of the operation panel unit 107, and the panel LCD control unit 130 performs display control of the display unit 107 b of the operation panel unit 107. The JPEG compression unit 133 performs JPEG compression on the screen data transmitted from the mobile terminal control unit 127 via the JPEG coder driver 131, and the compressed data is transmitted via the USB interface 111 or the WIFI module 109. Is transmitted to the portable terminal 2.

  On the other hand, the mobile terminal 2 includes an image display application 221, a display image generation application 222, a JPEG development application 223, an application layer including various applications such as a browser, a platform layer including various managers 225 and 226, and the like. The OS layer includes a library 227 and a kernel 228, and hardware 229.

  The JPEG decompression application 223 decompresses the compressed data transmitted from the MFP 1.

  The display image generation application 222 is currently displayed and stored in the storage unit 203 when the user performs an image enlargement / reduction operation or the like on the remote screen displayed on the display unit 204 a of the mobile terminal 2. Using the screen data, the editing processing is performed by the own device and displayed on the display unit 204.

  The image display application 221 generates the screen of the MFP 1 as a remote screen using the screen data transmitted from the MFP 1 and expanded by the JPEG development application 223 and displays the screen on the display unit 204.

  In this embodiment, the image display application 221, the display image generation application 222, and the JPEG expansion application 223 are configured as a part of functions realized by the cooperation application.

  Next, processing when the MFP 1 and the portable terminal 2 are operated in cooperation will be described.

A user wirelessly connects the portable terminal 2 to the MFP 1 via a network and tries to remotely operate the MFP 1 from the portable terminal 2. The cooperation operation between the portable terminal 2 and the MFP 1 is started when the user starts a cooperation application installed in the portable terminal 2.

  FIG. 4 shows the relationship between the software of the MFP 1 and the portable terminal 2. The left side shows the software on the MFP 1 side and the right side shows the software on the portable terminal 2.

  The cooperative application of the mobile terminal 2 is located between the OS of the mobile terminal 2 and application software that is general-purpose software, and includes user operation information, transmission / reception data with the mobile terminal 2, screen data on the mobile terminal 2 side, The position information obtained by the position information acquisition unit 210 can be acquired, and these data can be transmitted to the MFP 1 as necessary. In other words, the cooperation software of the mobile terminal 2 includes user operation reception processing on the mobile terminal 2 side, such as calling an application, reception / display processing of screen data for displaying the operation screen of the MFP 1 (mobile terminal side operation screen processing), cooperation Perform operation-related processing. The linked operation related processing includes communication with the MFP 1 of operation information indicating the content of the operation performed by the user via the operation unit 204b.

  On the other hand, when the OS on the MFP 1 and the application software for the MFP 1 are all in-house designed / customized and all necessary information can be obtained, the cooperative application of the MFP 1 may not be located between the OS and the application software. Necessary information may be obtained by communicating with the OS and application software while being independent of the OS and application software. The cooperative application of the MFP 1 executes user operation reception processing on the MFP 1 side such as calling of an application, transmission processing of MFP side screen data to the portable terminal 2 (MFP side operation screen processing), cooperative operation related processing, and the like. As the cooperative operation-related processing, reception / execution processing of operation information from the portable terminal 2 and the like can be cited.

  When the user selects a remote operation icon on the operation screen displayed on the display unit 204 of the mobile terminal 2 and instructs MFP cooperation, the cooperation software of the mobile terminal 2 communicates with the registered MFP 1, and MFP 1 Call the link software and request the acquisition of screen data. As a result, the remote operation mode is started, and the screen data of the initial screen is transmitted from the MFP 1 to the portable terminal 2 in a state compressed with JPEG or the like in order to improve transfer efficiency.

  The portable terminal 2 displays the received screen data on the MFP side as a remote screen on the display unit 204a. When the screen size of the display unit 204a of the mobile terminal 2 is different from the screen size of the display unit 107b of the MFP 1, the mobile terminal 2 adjusts and displays the screen data in accordance with the screen size of the display unit 204a of itself.

  The same screen may or may not be displayed on the display unit 105a of the MFP 1 simultaneously with the screen display on the portable terminal 2.

  When the user performs a touch operation on the remote screen, which is the screen on the MFP 1 side, displayed on the mobile terminal 2, it is determined whether the touch operation is a gesture operation for instructing enlargement / reduction of screen data or a normal button operation. Is done. Then, the determination result is transmitted as touch information to the MFP 1 together with the coordinate data of the touch position.

  On the other hand, if it is determined that the operation is a normal button operation, the portable terminal 2 transmits coordinate data of the touch position on the operation unit 204b to the MFP 1.

  The MFP 1 analyzes the received operation information including coordinate data and touch information, determines which button on the screen is pressed, generates next screen data corresponding to the button press, and transmits it to the portable terminal 2. Or, job processing corresponding to the button press is executed. The portable terminal 2 displays the received next screen data on the display unit 204b and updates the screen.

  In this way, the MFP 1 can be remotely operated by the portable terminal 2.

  In this embodiment, the mobile terminal 2 receives position information indicating the current position of the mobile terminal 2 at the time of acquisition request from the MFP 1, transmission of coordinate data of the touch position, or periodically or at an arbitrary timing. The MFP 1 detects the distance between the own device and the portable terminal 2 from the current position of the portable device 2 based on the received position information and the installation position of the own device registered in advance. . In addition, the detection of the distance with the portable terminal 2 is not limited to this method. The content of the screen data to be generated is changed according to the detected distance.

  That is, when the remote screen displayed on the mobile terminal 2 is an operation screen that can be input using the external key of the key unit 107a, the distance between the own device and the mobile terminal 2 is compared with a preset threshold value. When the distance exceeds the threshold value, in other words, when the mobile terminal 2 is located far from the MFP 2, the operation screen on which the key corresponding to the external key is displayed is displayed on the mobile terminal 2 as a remote screen. Screen data to be generated is transmitted to the mobile terminal 2.

  The mobile terminal 2 that has received the screen data displays a remote screen based on the screen data on the display unit 204a.

  FIG. 6 is a diagram illustrating a remote screen displayed on the display unit 204a of the mobile terminal 2 when the distance from the mobile terminal 2 exceeds the threshold. On this remote screen, in addition to the normal operation screen D1, a key display portion D2 having a software key 300 corresponding to the external key is displayed. In this embodiment, the key 300 corresponding to the external key is an emulator that emulates at least one of a preview key, a guide key, an enlarged display key, a utility key, and a job mode switching key that exist as external keys. .

  The user can perform an input operation on the operation screen using the software key 300 displayed in the key display area D2.

  The key display area D2 may be combined with the image data of the operation screen D1 as a part of an image constituting the remote screen to generate screen data. In this case, when the user touches any key 300 in the key display area D2 displayed on the mobile terminal 2, the coordinate data of the touch position and the touch state (for example, the type of gesture operation when touched) are shown. Touch information is transmitted from the portable terminal 2 to the MFP 1, and the MFP 1 analyzes the coordinate data to identify the operated key and touch state, and generates next screen data or performs corresponding processing.

  Further, the MFP 1 transmits the arrangement information of the key 300 included in the key display unit D2 to the portable terminal 2 as screen data together with the screen data of the operation screen D1, and the portable terminal 2 transmits each key 300 based on the arrangement information. You may display on the display part 204a. In this case, when the user touches any key 300 of the key display portion D2 displayed on the mobile terminal 2, the mobile terminal 2 uses the key 300 on / off information (if the key 300 is a button, only the on information is toggled). On / off information in the case of a switch) is transmitted to the MFP 1. The MFP 1 analyzes the on / off information to identify the operated key, and generates next screen data or performs a corresponding process.

  As described above, when the distance from the mobile terminal 2 is long, the mobile terminal 2 displays a remote screen on which the key 300 corresponding to the external key is displayed. Therefore, the user presses the key displayed on the remote screen. The input to the operation screen can be performed.

On the other hand, when the distance from the mobile terminal 2 is within the threshold, a remote screen as shown in FIG. 7 is displayed on the display unit 204a of the mobile terminal 2. In this remote screen, the normal operation screen D1
In addition, a message D3 indicating that the external key of the MFP 1 can be used is displayed.

  Accordingly, the message display can prompt the user to use the external key, and the user can use the external key of the MFP 1 without displaying the software key on the portable terminal 2, so that the operability is good. Become.

  FIG. 8 is a diagram illustrating another example of a remote screen displayed on the mobile terminal 2 when the distance from the mobile terminal 2 is within a threshold.

  In this example, the operation screen D1 has two set value input areas D11 and D12 at the top and bottom. A display indicating that the upper set value input area D11 is selected is displayed. In FIG. 8, the selected state is indicated by hatching. Which of the set value input areas D11 and D12 is to be selected is set in advance.

  Further, a message D4 indicating that the outside key of the MFP 1 can be used is displayed in the remote screen. Note that the message D4 is not necessarily displayed.

  The user can input a setting value in the setting value input area D11 in the selected state by operating the outside key of the key unit 107a of the MFP1.

  As described above, when the distance from the mobile terminal 2 is within the threshold value, one of the set value input areas D11 and D12 is in the selected state. The user can be prompted to use the external key, and the user can use the external key of the MFP 1 without displaying the software key on the portable terminal 2, which improves operability.

  On the other hand, when the distance from the portable terminal 2 exceeds the threshold, the remote screen is displayed in a state where all the setting value input areas D11 and D12 are not selected, as shown in FIG. Thereby, it is possible to prompt the user to display the key corresponding to the external key with such display of the selected state.

  When the user touches any set value input area, the set value input area is switched to a selected state, and a software numeric keypad or the like is displayed on the display unit 204a of the portable terminal 2 based on the screen data from the MFP 1. The user can input a set value using the software key.

  10 and 11 show another embodiment of the present invention. In this embodiment, when the distance to the mobile terminal 2 exceeds the threshold, the display unit 204a of the mobile terminal 2 has an operation having a set value input area as shown in FIG. 10 based on the screen data from the MFP 1. A software numeric keypad 301 is displayed as a remote screen together with the screen D5.

  In this state, it is assumed that the user holding the mobile terminal 2 moves toward the MFP 1 and as a result, the distance from the mobile terminal 2 is within the threshold. Then, the MFP 1 transmits new screen data to the portable terminal 2, and a message that the display of the software numeric keypad disappears and the external key of the MFP 1 can be used on the display unit 204 a of the portable terminal 2 as shown in FIG. The screen switches to the remote screen on which D6 is displayed. That is, the screen is switched to a screen prompting the user to use the external key of the MFP 1. As a result, even when the user who has the portable terminal 2 approaches the MFP 1, the inconvenience that the software key is continuously displayed and the operability is hindered can be solved.

Although not shown in the figure, when the user holding the mobile terminal 2 moves away from the MFP 1 and the distance from the mobile terminal 2 becomes within the threshold value and exceeds the threshold value, the remote screen is displayed as shown in FIG. 11 is switched to the display of FIG.

  FIG. 12 is a sequence diagram of information communication processing executed between the MFP 1 and the portable terminal 2.

  When the portable terminal 2 transmits a connection request to the MFP 1 (circle numeral 1), the MFP 1 responds with a connection permission (circle numeral 2). Next, when the portable terminal 2 requests the MFP 2 to acquire screen data (circle numeral 3), the MFP 1 transmits the screen data of the operation screen to the portable terminal 2 (circle numeral 4).

  Next, the portable terminal 2 requests screen data of a key corresponding to the outer key (circle numeral 5). The MFP 1 acquires the position information of the mobile terminal 2 and detects the distance from the mobile terminal 2.

  When the distance is equal to or smaller than the threshold value, that is, when the distance to the mobile terminal 2 is close, the MFP 1 generates screen data including a message that prompts the user to use the external key of the MFP 1 without displaying the key corresponding to the external key. Send (round numeral 6). Based on the received screen data, the portable terminal 2 displays a remote screen including a message prompting the use of the external key of the MFP 1.

  On the other hand, when the distance exceeds the threshold value, that is, when the distance from the mobile terminal 2 is far, the MFP 1 generates screen data including software key arrangement information corresponding to the external key and transmits the screen data to the mobile terminal 2 (circle numerals). 7). Based on the received screen data, the portable terminal 2 displays a remote screen on which the software key is arranged according to the arrangement information of the software key corresponding to the external key.

  When the user presses one of the software keys 300 corresponding to the external key on the remote screen displayed on the mobile terminal 2, the mobile terminal 2 displays on / off information of the pressed key 300 (the key 300 is a button In the case of ON / OFF information in the case of the toggle switch, the ON / OFF information in the case of the toggle switch) is transmitted to the MFP 1 (circle numeral 8). The MFP 1 analyzes the received on / off information and the like, executes corresponding job processing and the like, or generates next screen data.

  FIG. 13 is another sequence diagram of information communication executed between the MFP 1 and the portable terminal 2.

  When the portable terminal 2 transmits a connection request to the MFP 1 (circle numeral 1), the MFP 1 responds with a connection permission (circle numeral 2). Next, when the portable terminal 2 requests the MFP 2 to acquire screen data (circle numeral 3), the MFP 1 transmits the screen data of the operation screen to the portable terminal 2 (circle numeral 4).

  Next, the portable terminal 2 requests screen data of a key corresponding to the outer key (circle numeral 5). The MFP 1 acquires the position information of the mobile terminal 2 and detects the distance from the mobile terminal 2.

  When the distance is equal to or smaller than the threshold value, that is, when the distance to the mobile terminal 2 is short, the MFP 1 generates screen data including a message prompting the use of the external key of the MFP 1 without displaying the key corresponding to the external key. (Circle number 6). Based on the received screen data, the portable terminal 2 displays a remote screen including a message prompting the use of the external key of the MFP 1.

  On the other hand, when the distance exceeds the threshold value, that is, when the distance from the mobile terminal 2 is far, the MFP 1 generates screen data including image data of the software key corresponding to the external key and transmits the screen data to the mobile terminal 2 (circle numerals). 9). Based on the received screen data, the portable terminal 2 displays a remote screen having a software key corresponding to the external key.

  When the user presses any one of the software keys 300 corresponding to the external key on the remote screen displayed on the mobile terminal 2, the mobile terminal 2 displays coordinate data and touch information indicating the position of the pressed key 300. Is transmitted to the MFP 1 (circle numeral 10). The MFP 1 analyzes the received coordinate data and the like, executes corresponding job processing or the like, or generates next screen data.

  FIG. 14 is a flowchart showing the operation of the MFP 1. This operation is executed by the CPU 101 of the MFP 1 operating according to an operation program recorded on a recording medium such as the ROM 102.

  In step S01, a connection request from the portable terminal 2 is waited. If there is a connection request (YES in step S01), a connection permission is returned in step S02.

  Next, in step S03, the mobile terminal 2 waits for an acquisition request for screen data. If there is an acquisition request (YES in step S03), the operation screen data of its own device is transmitted to the mobile terminal 2 in step S04.

  In step S05, the distance to the mobile terminal 2 is detected, and in step S06, it is determined whether the distance to the mobile terminal 2 exceeds a threshold value. If the threshold value is not exceeded (NO in step S06), in step S09, after generating screen data having a message corresponding to the external key of MFP 1 hidden and a message prompting the use of the external key of MFP 1 is generated, Proceed to S10. If the threshold value is exceeded (YES in step S06), a key image corresponding to the external key is generated in step S07, and screen data is generated by combining the generated image with the operation screen image in step S08. Thereafter, the process proceeds to step S10.

  In step S10, the generated screen data is transmitted to the portable terminal 2, and in step S11, it is determined whether there is a change in the screen content. If there is a change (YES in step S11), the process returns to step S05. If there is no change (NO in step S11), it is determined in step S12 whether operation information such as coordinate data of the touch position has been received from the portable terminal 2. If not received (NO in step S12), the process returns to step S11. If received (YES in step S12), the process proceeds to step S13.

  In step S13, coordinate data and touch information included in the received operation information are analyzed, and corresponding processing such as generation of next screen data and execution of a job is performed.

  Next, in step S14, it is checked whether or not the portable terminal 2 is connected. If it is connected (YES in step S14), the process returns to step S05. If not connected (NO in step S14), the process is terminated.

  FIG. 15 is a block diagram showing the operation of the mobile terminal 2.

  In step S21, the portable terminal 2 transmits a connection request to the MFP 2. In step S22, the portable terminal 2 waits for a response of connection permission from the MFP 2. If there is a connection permission response (YES in step S22), a screen data acquisition request is transmitted to MFP 2 in step S23.

  Next, in step S24, it waits for reception of screen data. When received (YES in step S24), in step S25, a remote screen based on the received screen data is displayed on the display unit 204a, and in step S26, an acquisition request for foreign key corresponding screen data is transmitted to the MFP1.

  In step S27, screen data is received, and in step S28, a remote screen is displayed based on the screen data. This screen is a screen on which a key corresponding to the external key is displayed when the distance between the mobile terminal 2 and the MFP 1 exceeds the threshold, and the external key when the distance between the mobile terminal 2 and the MFP 1 is within the threshold. This is a screen on which a key corresponding to is not displayed and a message prompting the use of the external key is displayed.

  Next, in step S29, it is determined whether or not a touch operation has been performed by the user on the remote screen. If not (NO in step S29), the process returns to step S28. When a touch operation is performed (YES in step S29), it is determined in step S30 whether the operation is a connection end operation. If it is a connection termination operation (YES in step S30), the process is terminated. If the operation is not a connection termination operation (YES in step S30), in step S31, the coordinate data and touch information of the touch position are notified to the MFP 1, and the process returns to step S24.

DESCRIPTION OF SYMBOLS 1 Image forming apparatus 2 Portable terminal device 101 CPU
102 ROM
104 Scanner unit 105 Print unit 106 Storage unit 107 Operation panel unit 107b Display unit 109 WIFI module 111 USB interface 120, 121 External key 201 CPU
202 RAM
203 Storage Unit 204a Display Unit 204b Operation Unit 205 WIFI Module 207 USB Interface 209 Cooperation Unit 300 Keys corresponding to External Keys D11, D12 Numerical Input Area

Claims (20)

A connection means connectable to the mobile terminal device;
An external key configured by hardware,
Distance detection means for detecting a distance from the mobile terminal device connected by the connection means;
Transmitting means for transmitting, to the mobile terminal device, data for causing the mobile terminal device to display the operation screen as a remote screen in a state where the mobile device is connected to the mobile terminal device by the connecting means;
With
The transmission means is in a state where the own device is connected to the mobile terminal device, and receives a transmission request for screen data corresponding to an external key from the mobile terminal device , and the operation screen uses the external key. In the case of the operation screen that can be input, if the distance from the mobile terminal device detected by the distance detection means is equal to or less than a certain value, the mobile terminal device displays a display prompting the use of the external key. Data for transmitting to the portable terminal device.   When the distance from the portable terminal device detected by the distance detecting unit exceeds a certain value, the transmitting unit transmits key arrangement information corresponding to the outer key to the portable terminal device, and the portable terminal device 2. The apparatus according to claim 1, further comprising: a receiving unit configured to receive, from the portable terminal device, key on / off information when a user operates a key corresponding to the external key displayed corresponding to the arrangement information. Image processing apparatus.   When the distance from the mobile terminal device detected by the distance detection unit exceeds a certain value, the transmission unit causes the mobile terminal device to display an operation screen on which a key corresponding to the outer key is arranged. Receive data to be transmitted to the mobile terminal device and coordinate data of a touch position when the user operates a key corresponding to an external key on the operation screen displayed on the mobile terminal device from the mobile terminal device The image processing apparatus according to claim 1, comprising means.   The image processing apparatus according to claim 1, wherein the external key is at least one of a hard numeric keypad, a hard keyboard, and a switch.   The display for prompting the use of the external key when the distance from the mobile terminal device detected by the distance detection unit is a predetermined value or less is a message indicating that the external key can be used. An image processing apparatus according to any one of the above.   The operation screen has at least one set value input area, and a display for prompting the use of the external key when the distance from the mobile terminal device detected by the distance detection unit is a predetermined value or less is the set value input The image processing apparatus according to claim 1, wherein the image processing apparatus is a display indicating that the area is in a selected state. When the distance from the mobile terminal device detected by the distance detection unit exceeds a certain value, the transmission unit displays an operation screen on the mobile terminal device with a display prompting display of a key corresponding to the external key. Transmitting data to be displayed to the portable terminal device;
The operation screen has at least one set value input area, and the display for prompting the display of the key corresponding to the outside key is a display indicating that the set value input area is not selected. The image processing apparatus according to any one of 1 to 6.   The key corresponding to the external key is an emulation of at least one of a preview key, a guide key, an enlarged display key, a utility key, and a job mode switching key that exists as an external key. 8. The image processing apparatus according to any one of items 7.   Data for causing the mobile terminal device to display a screen when the distance from the mobile terminal device detected by the distance detection means is a predetermined value or less is transmitted to the mobile terminal device, and the screen based on the data is displayed on the mobile terminal device. When the distance from the mobile terminal device detected by the distance detection means exceeds a certain value when being displayed, the transmission means displays a screen when the distance from the mobile terminal device exceeds a certain value The image processing apparatus according to claim 1, wherein data for causing the mobile terminal apparatus to display data is transmitted again to the mobile terminal apparatus.   Data for causing the mobile terminal device to display a screen when the distance from the mobile terminal device detected by the distance detection unit exceeds a certain value is transmitted to the mobile terminal device, and the screen based on the data is transmitted to the mobile terminal device. When the distance from the mobile terminal device detected by the distance detection means is equal to or less than a certain value when displayed, the transmission means displays a screen when the distance from the mobile terminal device is equal to or less than a certain value. The image processing apparatus according to claim 1, wherein data for causing the mobile terminal apparatus to display data is transmitted again to the mobile terminal apparatus. In a computer of an image processing apparatus provided with connection means connectable to a portable terminal device and an external key configured by hardware,
A distance detection step of detecting a distance from the mobile terminal device connected by the connection means;
A transmission step of transmitting data for displaying the operation screen of the device as a remote screen to the mobile terminal device in a state of being connected to the mobile terminal device by the connection means;
And execute
In the transmission step, the device itself is connected to the portable terminal device, and a transmission request for screen data corresponding to the foreign key is received from the portable terminal device , and the operation screen uses the foreign key. In the case of the operation screen that can be input, when the distance from the mobile terminal device detected by the distance detection step is a predetermined value or less, the mobile terminal device is caused to display a display prompting the use of the external key. The remote screen control program for making the said computer perform the process which transmits the data for this to the said portable terminal device.   When the distance from the mobile terminal device detected by the distance detection step exceeds a certain value, the transmission step sends a process of transmitting key arrangement information corresponding to the external key to the mobile terminal device. A receiving step of receiving, from the portable terminal device, key on / off information when a user operates a key corresponding to the external key displayed corresponding to the arrangement information on the portable terminal device. The remote screen control program according to claim 11, further executed.   When the distance from the mobile terminal device detected by the distance detection step exceeds a certain value, the transmission step displays an operation screen on which a key corresponding to the outer key is arranged on the mobile terminal device. Causing the computer to execute a process of transmitting data to the mobile terminal device, and coordinates data of a touch position when a user operates a key corresponding to an external key on an operation screen displayed on the mobile terminal device. The remote screen control program according to claim 11, further causing the computer to execute a reception step of receiving from a terminal device.   The remote screen control program according to claim 11, wherein the external key is at least one of a hard numeric keypad, a hard keyboard, and a switch.   15. The display prompting the use of the external key when the distance from the mobile terminal device detected by the distance detection step is a predetermined value or less is a message indicating that the external key can be used. The remote screen control program according to any one of the above.   The operation screen has at least one set value input area, and a display prompting the use of the external key when the distance from the mobile terminal device detected by the distance detecting step is a predetermined value or less is the set value input The remote screen control program according to any one of claims 11 to 15, which is a display indicating that an area is in a selected state. When the distance to the mobile terminal device detected by the distance detection step exceeds a certain value, the transmission step displays an operation screen on the mobile terminal device that is displayed to prompt the display of the key corresponding to the external key. Causing the computer to execute a process of transmitting data for display to the portable terminal device;
The operation screen has at least one set value input area, and the display for prompting the display of the key corresponding to the outside key is a display indicating that the set value input area is not selected. The remote screen control program according to any one of 11 to 16.   13. The key corresponding to the external key is an emulation of at least one of a preview key, a guide key, an enlarged display key, a utility key, and a job mode switching key that exists as an external key. The remote screen control program according to any one of 17.   Data for causing the mobile terminal device to display a screen when the distance from the mobile terminal device detected by the distance detecting step is a predetermined value or less is transmitted to the mobile terminal device, and the screen based on the data is displayed on the mobile terminal device. When the distance from the mobile terminal device detected by the distance detection step exceeds a certain value when being displayed, the transmission step displays a screen when the distance from the mobile terminal device exceeds a certain value The remote screen control program according to any one of claims 11 to 18, which causes the computer to execute a process of transmitting data for causing the mobile terminal device to display data again to the mobile terminal device.   Data for causing the mobile terminal device to display a screen when the distance to the mobile terminal device detected by the distance detection step exceeds a certain value is transmitted to the mobile terminal device, and the screen based on the data is displayed on the mobile terminal device. When the distance from the mobile terminal device detected by the distance detection step is equal to or less than a certain value when being displayed, the transmission step displays a screen when the distance from the mobile terminal device is equal to or less than a certain value The remote screen control program according to any one of claims 11 to 19, which causes the computer to execute a process of transmitting data for displaying data on the mobile terminal device to the mobile terminal device again.

Images