JP2015069420A - Operation support program, terminal device, processing execution device, and processing execution program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation support program allowing a processing execution device to be remotely controlled suitably, a terminal device, a processing execution device, and a processing execution program.SOLUTION: By the operation support program, a display object image based on image data received from the processing execution device is displayed on a first display unit, and operation position information corresponding to a position on the display object image designated by designating operation is transmitted to the processing execution device as the transmission source of the image data corresponding to display object image data. Therefore, the processing execution device is capable of recognizing which position has been indicated on the display object image. Since the display object image comprises images of a plurality of screens constituting a series of screen groups, even operators included in screens not displayed on the processing execution device may be taken as operation objects. Thus, the processing execution device can be remotely controlled suitably by the terminal device.

Description

  The present invention relates to an operation support program, a terminal device, a process execution device, and a process execution program.

  Patent Document 1 proposes a technique for remotely operating a multifunction peripheral from a portable terminal having a touch panel superimposed on an LCD. Specifically, the captured image of the operation unit of the multifunction device is displayed on the LCD of the mobile terminal, the displayed operation unit is operated by the user via the touch panel, and an instruction according to the operated operation unit is issued. To the multifunction machine.

JP2013-8183A

  However, in the technique of Patent Literature 1, only the operation element included in the image captured by the mobile terminal is targeted. The multifunction device may display a screen including a soft key as an operation unit on a display unit of the multifunction device. When the displayed screen is not a screen including a desired soft key, the user needs to take a picture to display the screen on the LCD of the mobile terminal after switching to the screen including the desired soft key, It was complicated.

  The present invention has been made in view of the above-described circumstances, and an object thereof is to provide an operation support program, a terminal device, a process execution device, and a process execution program that can suitably operate a process execution device remotely.

  In order to achieve this object, an operation support program according to the present invention includes a first display unit, an input unit that detects an input operation on the first display unit, a communication unit that communicates with a process execution device, a control unit, Is a program that can be executed by the control unit and that operates the processing execution device based on an input operation detected by the input unit, and constitutes a series of screen groups. A display target image, which is image data for a plurality of screens and includes at least images of a plurality of indicators each assigned a process performed by the processing execution device and included in the series of screen groups, is displayed on the first display unit. Image receiving means for receiving image data for processing from the processing execution device via the communication unit, image display means for displaying the display target image based on the image data received by the image receiving means, Operation position information corresponding to the position on the display target image designated by the designation operation for designating the position on the display target image displayed on the first display unit, which is the input operation detected by the input unit. The control unit is caused to function as information transmission means that transmits to the processing execution apparatus that is the transmission source of the image data via the communication unit.

  In addition, the process execution device of the present invention includes a first display unit, an input unit that detects an input operation on the first display unit, a communication unit that communicates with the process execution device, and a control unit. The unit is image data for a plurality of screens that form a series of screen groups, and includes at least a plurality of indicators that are included in the series of screen groups and each assigned with a process performed by the processing execution device Image receiving means for receiving image data for displaying the target image on the first display section from the processing execution device via the communication section, and the display target image based on the image data received by the image receiving means. An image display means to display and a position on the display target image designated by a designation operation for designating a position on the display target image displayed on the first display unit, which is an input operation detected by the input unit. The operation position information corresponding to, via said communication unit, and a, and information transmitting means for transmitting to the processing execution unit that is the sender of the image data.

  The operation support program of the present invention can be configured in various modes such as a terminal device, a control device that controls the terminal device, an operation support system, an operation support method, and a recording medium that records the operation support program. The process execution apparatus of the present invention can be configured in various modes such as a control apparatus that controls the process execution apparatus, a process execution system, a process execution method, a process execution program, and a recording medium that records the process execution program. .

  According to the operation support program of the first aspect, the display target image based on the image data received from the process execution device is displayed on the first display unit. The operation position information corresponding to the position on the display target image specified by the specifying operation is transmitted to the processing execution apparatus that is the transmission source of the image data corresponding to the display target image data. Thereby, the process execution apparatus can grasp which position on the display target image is designated. Since the display target image is an image for a plurality of screens constituting a series of screen groups, an operator included in a screen that is not displayed in the processing execution apparatus can be an operation target. Therefore, remote operation of the processing execution device from the terminal device can be suitably performed.

  According to the operation support program of claim 2, in addition to the effect of claim 1, the following effect is obtained. The terminal device generates a display target image based on an element image that is an image of each of a plurality of indicators or a group image that includes a plurality of element images and element image position information that specifies an arrangement of the images, Can be displayed.

  According to the operation support program of claim 3, in addition to the effect of claim 2, the following effect is obtained. Since the background image is arranged as the background of the indicator image displayed on the second display unit of the processing execution device among the images of the indicators arranged in accordance with the element image position information, the user can process The indicator currently displayed on the 2nd display part of an execution apparatus can be grasped | ascertained.

  According to the operation support program of the fourth aspect, in addition to the effect produced by any one of the first to third aspects, the following effect is produced. The position on the display target image specified by the specifying operation specifies which position of which screen in the series of screen groups is specified, and information indicating the specified screen and position is transmitted to the processing execution device. The Therefore, the process execution device can grasp the position instructed on the terminal device even if the screen is not displayed on the second display unit.

  According to the operation support program of the fifth aspect, in addition to the effect produced by any one of the first to fourth aspects, the following effect is obtained. Even when the display target image is enlarged or reduced, the designated position can be transmitted to the process execution device as a position on the display target image at the same magnification. Therefore, the process execution device is instructed by the terminal device. You can grasp the position appropriately.

  According to the operation support program of the sixth aspect, in addition to the effect of any one of the first to fifth aspects, when the input operation is performed outside the display target image, the input operation is performed according to the input operation. Can be executed.

  According to the process execution device of claim 8, a display target image including at least a plurality of indicator images each assigned a process performed by the process execution device, included in a series of screen groups including a plurality of screens. Image data to be displayed on the display unit of the terminal device is transmitted to the terminal device. When the operation position information indicating the position on the display target image corresponding to the image data transmitted from the terminal device indicates the position on the indicator image, the process assigned to the indicator image Is done. Therefore, an operator included in a screen that is not displayed on the process execution device can be operated by the terminal device as an operation target. Thereby, the process execution apparatus can be suitably remotely operated from the terminal apparatus.

  According to the process execution device of claim 9, in addition to the effect of claim 8, the following effect is obtained. When the operation position information is received from the terminal device, the position on the display target image specified by the operation position information specifies which position of which screen in the series of screen groups is specified. Therefore, even if the screen is not displayed on the display unit, the position instructed on the terminal device can be grasped.

  According to the process execution device of claim 10, in addition to the effect of claim 8 or 9, when the screen displayed on the display unit is switched, the background image to be displayed on the terminal device is displayed with a plurality of indicators. Can be displayed on the background of the indicator image displayed on the display unit of the processing execution apparatus. A user who operates the terminal device can grasp the indicator being displayed on the process execution device.

  According to the process execution device of the eleventh aspect, in addition to the effect produced by any one of the eighth to tenth aspects, the following effect is produced. If there is a mismatch between the operation position information received from the terminal device and the screen group displayed on the process execution device at that timing, the process assigned to the operator indicated by the operation position information is not executed. Therefore, it is possible to maintain consistency between the screen displayed on the process execution device and the remote operation from the terminal device.

  According to the terminal device of the seventh aspect, the same effect as the operation support program according to the first aspect is obtained. Further, according to the process execution program of the twelfth aspect, the same effect as the process execution apparatus of the first aspect is obtained.

It is a block diagram which shows the electrical structure of a multifunction peripheral device and a portable terminal. It is a figure which illustrates the outline of the present invention typically. 6 is a flowchart illustrating MFP-side processing during remote operation. It is a flowchart which shows the terminal side process at the time of remote operation. It is a flowchart which shows the terminal side process at the time of remote operation of 2nd Embodiment.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. First, a first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing an electrical configuration of a multifunction peripheral device (hereinafter referred to as “MFP”) 10 and a terminal 200 that can communicate with the MFP 10. Note that the MFP 10 and the terminal 200 shown in FIG. 1 are embodiments of the processing execution device and the terminal device of the present invention, respectively.

  The MFP 10 has various functions such as a print function, a scan function, a copy function, and a facsimile function. Although details will be described later, the MFP 10 is configured to be able to execute various processes performed in the MFP 10 by remote operation from the terminal 200 on which the operation support application (hereinafter referred to as “operation support application”) 212b is mounted.

  The MFP 10 is mainly provided with a CPU 11, flash memory 12, RAM 13, operation keys 15, LCD 16, touch panel 17, scanner unit 18, printer unit 19, NCU 20, modem 21, and wireless communication unit 23. These units are connected to each other via the input / output port 24. The CPU 11 controls various functions of the MFP 10 according to fixed values and programs stored in the flash memory 12, data stored in the RAM 13, or various signals transmitted / received via the NCU 20, and input / output Each unit connected to the port 24 is controlled.

  The flash memory 12 is a nonvolatile memory, and stores a control program 12a for controlling the operation of the MFP 10 and the like. Each process shown in the flowchart of FIG. 3 to be described later is a process executed by the CPU 11 in accordance with the control program 12a. Further, the flash memory 12 is prepared with image data of a part image and a background image for constituting each screen displayed on the LCD 16 for all displayable screens. The RAM 13 is a rewritable volatile memory having a temporary area for temporarily storing various data when the CPU 11 executes the control program 12a.

  The operation key 15 is a mechanical key provided on the casing of the MFP 10 and accepts various setting values and instructions from the user. The LCD 16 is a liquid crystal display device and displays various screens. A touch panel 17 is provided on the LCD 16 so as to overlap the LCD 16. For example, when an indicator such as a finger or a stick touches or approaches the screen of the LCD 16, the touch panel 17 detects the touched or approached position and inputs the detected position to the MFP 10. The scanner unit 18 reads a document and converts it into image data. The printer unit 19 prints an image based on the image data on a recording sheet. The modem 21 modulates image data to be transmitted at the time of FAX transmission into a signal that can be transmitted to a telephone line network (not shown) and transmits the signal via the NCU 20 or is input from the telephone line network via the NCU 20. The received signal is demodulated into image data. The NCU 20 connects a telephone line network (not shown) and the MFP 10, and controls the connection state with the telephone line network by closing or disconnecting the line according to an instruction from the modem 21.

  The wireless communication unit 23 is an interface for performing wireless communication using a wireless LAN. In the present embodiment, the communication performed by the wireless communication unit 23 is wireless communication based on a wireless LAN compliant with the IEEE802.11b / g / n standard. In the present embodiment, the wireless communication unit 23 communicates with the terminal 200 based on the Wi-Fi (registered trademark) standard in the infrastructure mode via the access point (AP) 50 that is a relay device (hereinafter, referred to as “wireless communication”). This communication is referred to as “Wi-Fi communication”).

  The terminal 200 of this embodiment is a tablet terminal. The terminal 200 is mainly provided with a CPU 211, a flash memory 212, a RAM 213, an operation key 215, an LCD 216, a touch panel 217, a memory card interface (memory card I / F) 218, and a wireless communication unit 223. These units are connected to each other via an input / output port 224. The CPU 211 controls each unit connected to the bus line 224 according to a fixed value, a program, or the like stored in the ROM 212 or the like.

  The flash memory 212 is a rewritable nonvolatile memory. The flash memory 212 stores an operating system 212a and an operation support application 212b. Hereinafter, the “operating system” may be referred to as “OS”. In the following description, the CPU 211 that executes a program such as an application or an operating system may be simply described by a program name. For example, the description “application” may mean “CPU 211 executing the application”. The OS 212a is basic software for realizing the standard functions of the terminal 200. In this embodiment, an Android (registered trademark) OS is adopted as an example of the OS 212a.

  The operation support application 212b is an application program for remotely operating a device such as the MFP 10. The operation support application 212b is installed in the terminal 200 by the user, and enables remote operation of the corresponding device from the terminal 200. Each process shown in the flowchart of FIG. 4 to be described later is a process executed by the CPU 211 in accordance with the operation support application 212b.

  The operation key 215 is a mechanical key provided on the casing of the terminal 200, and accepts various setting values and instructions from the user. LCD 216, touch panel 217, memory card I / F 218, and wireless communication unit 223 are the same as LCD 16, touch panel 17, memory card I / F 18, and wireless communication unit 23, respectively.

  The outline of the present invention will be described with reference to FIG. FIG. 2A shows a function menu screen 70 as an example of a screen displayed on the LCD 16 of the MFP 10. The function menu screen 70 is composed of two pages of screens, that is, a screen 70a for the first page and a screen 70b for the second page. The user can appropriately switch the screen displayed on the LCD 16 to the screen 70a or the screen 70b by appropriately pressing the button images 91c and 91d displayed on the function menu screen 70 (so-called “tap operation”). .

  The screen displayed on the LCD 16 is composed of a background image and component images arranged on the background image. The function menu screen 70 shown in FIG. 2A includes a background image 91 and a button image 81 as a component image arranged on the background image 91. The button image 81 is an operator for selecting a function to be executed by the MFP 10. In this embodiment, the button image 81 includes five button images 81a to 81e. When the user performs a pressing operation on any of the button images 81a to 81e, the MFP 10 performs processing corresponding to the pressed button image 81, specifically, a function corresponding to the pressed button image 81. The display process of a screen (not shown) related to the execution of is executed.

  The five button images 81a to 81e are displayed on two screens instead of being collected on one screen. Specifically, the button images 81a to 81c are displayed on the screen 70a of the first page, and the button images 81d and 81e are displayed on the screen 70b of the second page. Therefore, the user needs to switch and display the screen 70a or the screen 70b as appropriate according to the button image to be operated among the button images 81a to 81e.

  The background image 91 includes a page number 91a and date / time information 91b, which are text. The background image 91 includes button images 91c and 91d that are operators for switching the screen displayed on the LCD 16 to the previous page or the next page.

  Each screen displayed on the LCD 16, such as the function menu screen 70, is configured by overlaying a background image layer such as the background image 91 and a component image layer such as the button image 81. When displaying a screen on the LCD 16, the MFP 10 generates a layer of a background image and a layer in which component images are arranged in a predetermined arrangement, and displays each layer on the LCD 16 in a superimposed manner.

  When the display target screen is a series of screen groups including a plurality of screens, such as the function menu screen 70, the MFP 10 sets button images 81 for all the screens constituting the series of screen groups. A layer in which component images such as are arranged is generated and stored in the RAM 13. That is, when the function menu screen 70 is a display target, the MFP 10 previously generates a layer of the background image 91, a layer of the component image for the screen 70a, and a layer of the component image for the screen 70b. When displaying the screen 70 a on the LCD 16, the MFP 10 superimposes the layer of the component image for the screen 70 a and the layer of the background image 91 on the LCD 16. On the other hand, when displaying the screen 70b on the LCD 16, the MFP 10 superimposes the layer of the component image for the screen 70b and the layer of the background image 91 on the LCD 16. Note that image data corresponding to the button image 81 and the background image 91 is prepared in the flash memory 12 in advance.

  Although details will be described later, when the MFP 10 is remotely operated from the terminal 200 on which the operation support application 212b is mounted, the MFP 10 provides the terminal 200 with the background image and the image data of the component image constituting the screen being displayed on the LCD 16. Send. As a result, the same image as that displayed on the LCD 16 of the MFP 10 is displayed on the LCD 216 of the terminal 200 as an image for remote operation. The user can remotely operate the MFP 10 based on the remote operation image. In particular, when the screen displayed on the LCD 16 is a series of screen groups composed of a plurality of screens such as the function menu screen 70, not only one screen being displayed but also images for a plurality of screens can be remotely controlled. Is displayed on the LCD 216 as an image.

  2B and 2C show an image 170 that is one of the images for remote operation. The image 170 is a remote operation image corresponding to the function menu screen 70. The image 170 includes two screen images constituting the function menu screen 70, more specifically, button images 81 (81a to 81e) displayed on the screens 70a and 70b and a background image 91. The background image 91 includes a page number 91a, date / time information 91b, and button images 91c and 91d. The terminal 200 receives from the MFP 10 position information of the images 81 and 91 together with image data corresponding to the button images 81 (81a to 81e) and the background image 91, respectively. The terminal 200 arranges the images 81 and 91 based on the received position information, whereby an image 170 as shown in FIGS. 2B and 2C is displayed on the LCD 216.

  The position information of the button image 81 includes bitmap information of the button images 81a to 81e and coordinates indicating the positions of the button images 81a to 81e. On the other hand, the position information of the background image 91 includes bitmap information of the background image 91 and coordinates indicating the position of the background image 91. In the present embodiment, the coordinates included in the position information are coordinates indicating the upper left position of each image. Therefore, coordinates P1 to P5 indicating the upper left position of each of the button images 81a to 81e and coordinates Q1 indicating the upper left position of the background image 91 are included as position information of the button image 81 or the background image 91.

  The display content of the image 170 differs depending on whether the screen displayed on the LCD 16 of the MFP 10 is the screen 70a or the screen 70b. That is, FIG. 2B shows an image 170a when the screen being displayed on the LCD 16 of the MFP 10 is the screen 70a. On the other hand, FIG. 2C shows an image 170b when the screen displayed on the LCD 16 of the MFP 10 is the screen 70b. The display position of the background image 91 is different between the image 170a and the image 170b. This is based on the fact that the value of the coordinate Q1 received from the MFP 10 is different.

  In the image 170a, a background image 91 is displayed as the background of the button images 81a to 81c. On the other hand, in the image 170b, the background image 91 is displayed as the background of the button images 81d and 81e. That is, among the button images 81 included in the function menu screen 70, the background image 91 is displayed as the background of the button image being displayed on the LCD 16 of the MFP 10. Therefore, the user can grasp the screen displayed at that time on the MFP 10 side from the position of the background image in the remote operation image displayed on the LCD 216.

  The user can cause the MFP 10 to execute processing corresponding to the pressed button image 81 by performing a pressing operation on any of the button images 81a to 81e which are the operators in the image 170. At this time, it does not matter whether or not the button images 81a to 81e to be operated are displayed on the LCD 16 of the MFP 10. That is, according to the operation support application 212b, even the button image included in the screen not displayed on the LCD 16 among the button images 81a to 81 can be set as the operation target.

  Further, according to the operation support application 212b, when an operation such as a drag operation is performed on an image for remote operation displayed on the LCD 216 such as the image 170, the MFP 10 is operated as if the operation was performed on the MFP 10. Can execute the process. On the other hand, when various operations are performed on the area 216a on the LCD 216 where the image for remote operation is not displayed, the terminal 200 can be caused to execute processing corresponding to the operation. For example, when a pinch-out operation is performed on the area 216a, the displayed remote operation image can be enlarged from the display size at that time.

  2 illustrates an example in which the button images 81 that are component images are arranged in the longitudinal direction of the LCD 16 and the LCD 216, that is, in the X-axis direction. However, the arrangement direction of the component images is the short direction of the LCD 16 and the LCD 216. That is, it may be in the Y-axis direction. That is, when the component images are arranged in the short direction of the LCD 16, the remote operation image displayed on the LCD 216 is an image in which the component images are arranged in the short direction of the LCD 216.

  FIG. 3 is a flowchart showing MFP-side processing during remote operation, which is executed by the CPU 11 of the MFP 10. This process is a process for remotely operating the MFP 10 from the terminal 200. This process is started on the condition that the preliminary process for performing the remote operation is completed between the MFP 10 and the terminal 200.

  Prior to the description of FIG. 3, the pre-processing will be described. The pre-processing starts when the user operates the MFP 10 and sets the remote operation to be valid. Next, the user operates the terminal 200 to activate the operation support application 212b, and causes the terminal 200 on which the operation support application 212b is activated to search for the MFP 10 in which the remote operation is enabled. The terminal 200 registers the searched MFP 10 as a remote operation target in the terminal 200, and transmits registration information indicating registration, the IP address of the terminal 200, and the like to the registered MFP 10 by Wi-Fi communication. Receiving these, the MFP 10 establishes a communication link with the terminal 200. The pre-processing is completed when the communication link is established. The MFP 10 can be remotely operated from the terminal 200.

  As shown in FIG. 3, the CPU 11 obtains the background image, the displayed component image, the image data of the component image hidden for screen transition, the position information of each image, and the screen ID of the displayed screen. And transmitted to the terminal 200 by Wi-Fi communication (S301). The screen ID transmitted by the CPU 11 in S301 is identification information for specifying the screen displayed on the LCD 16 and is prepared in the flash memory 12 in advance.

  The position information of each image is composed of bitmap information of each image and coordinate information of each image. By executing the processing of S301, the image data transmitted from the MFP 10 and the image for remote operation corresponding to the position information, for example, the image 170 shown in FIG. 2B are displayed on the LCD 216 of the terminal 200.

  In the present embodiment, the coordinate information of each image is the upper left coordinate of each image. In the case of a series of screen groups composed of a plurality of screens such as the function menu screen 70, the coordinate P of the component image included in the first page screen such as the screen 70a is the component image of the component image. The actual coordinates on the screen including are used as they are. On the other hand, as the coordinates P of the component images included in the second and subsequent screens such as the screen 70b, the total coordinates obtained by virtually arranging the screens constituting the series of screen groups are used.

  Specifically, among the total coordinates, the value of the virtual screen alignment direction (hereinafter, simply referred to as “screen alignment direction”) in the series of screen groups is the value on the screen including the target component image. Of the actual coordinates of the component image, the value is calculated from the value A1 of the screen alignment direction, the page number N of the screen including the component image, and the length L of the LCD 16 in the screen alignment direction. On the other hand, among the total coordinates, the value A2 of the screen arrangement direction is used as it is among the actual coordinates of the component image on the screen including the target component image. That is, when the screen alignment direction is the X direction, the total coordinates of the component images included in the second and subsequent screens are (A1 + (N × L + 1), A2). On the other hand, when the screen alignment direction is the Y direction, the total coordinates of the component images included in the second and subsequent screens are (A2, A1 + (N × L + 1)).

  The “virtual screen arrangement direction in a series of screen groups” is equal to the arrangement direction of component images included in the series of screen groups. That is, in the function menu screen 70 in which the button images 81 are arranged in the longitudinal direction of the LCD 16, the virtual screen arrangement direction in the series of screen groups is the longitudinal direction of the LCD 16. Therefore, for example, when the length of the LCD 16 in the longitudinal direction is 480 dots, the length L is 480. On the other hand, in the screen in which the component images are arranged in the short direction of the LCD 16, the virtual screen arrangement direction in the series of screen groups is the short direction of the LCD 16. Therefore, for example, when the length of the LCD 16 in the short direction is 320 dots, the length L is 320.

  Next, the coordinates of the background image will be described. In the case of a series of screen groups composed of a plurality of screens, such as the function menu screen 70, the coordinate Q of the background image such as the background image 91 indicates which page the screen currently displayed on the LCD 16 is. It becomes the coordinates according to. Specifically, when the screen displayed on the LCD 16 is the screen of the first page, the upper left coordinate Q of the background image is (0, 0). When the screen being displayed on the LCD 16 is the second or subsequent page, the value E of the screen alignment direction among the coordinates at the upper left of the background image is expressed by the equation E = (length L + 1 of the screen alignment direction). Calculate based on Therefore, for example, when the screen arrangement direction is the X direction, the coordinate Q of the background image is (E, 0). On the other hand, when the screen arrangement direction is the Y direction, the coordinate Q of the background image is (0, E).

  After the process of S301, the CPU 11 determines whether operation information has been received from the terminal 200 (S302). When the CPU 11 determines that no operation information has been received from the terminal 200 (S302: No), the CPU 11 determines whether it is a transmission timing for periodically transmitting image data, position information, and the like to the terminal 200 ( S313). In the present embodiment, the transmission timing is every 50 msec. If the CPU 11 determines that it is the transmission timing (S313: Yes), the CPU 11 proceeds to S301.

  When the CPU 11 determines that it is not the transmission timing in S313 (S313: No), the CPU 11 determines whether various events have occurred in the MFP 10 based on the operation of the operation element such as the button image 81 (S314). . When the CPU 11 determines that no event has occurred (S314: No), the CPU 11 shifts the processing to S302. On the other hand, when the CPU 11 determines that an event has occurred (S314: Yes), the CPU 11 executes processing according to the event that has occurred (S315), and shifts the processing to S301. Note that this event includes a case where the button image 91c or 91d is pressed in the MPF 10 and the display on the LCD 16 is switched from the screen 70a to the screen 70b, or the screen 70b is switched to the screen 70a.

  In S302, when the CPU 11 determines that operation information has been received from the terminal 200 (S302: Yes). The CPU 11 determines whether the screen ID received from the terminal 200 together with the operation information is appropriate (S303). Specifically, when the received screen ID is a screen ID corresponding to the screen being displayed on the LCD 16 or a screen ID corresponding to another screen that constitutes a series of screen groups together with the screen being displayed. In S303, the CPU 11 determines that the received screen ID is appropriate.

  If the CPU 11 determines that the received screen ID is inappropriate in S303 (S303: No), the CPU 11 proceeds to S302. In such a case, the CPU 11 does not perform the processes of S304 to S312 described later, that is, the process according to the operation information received by the MFP 10 from the terminal 200. When the received screen ID is inappropriate, it means that the remote operation image operated on the terminal 200 side does not correspond to the screen currently displayed on the LCD 16 or a series of screen groups including the screen. . Therefore, according to the process of S303, consistency between the screen displayed on the LCD 16 and the remote operation from the terminal 200 can be achieved.

  On the other hand, when the CPU 11 determines that the received screen ID is appropriate in S303 (S303: Yes), the CPU 11 determines what the received operation information is (S304). When the CPU 11 determines that the received operation information is drag information indicating that a drag operation has been performed on the remote operation image displayed on the terminal 200 (S304: drag information), the CPU 11 A drag process according to the received drag information, more specifically, a drag process corresponding to the drag operation on the remote operation image is executed (S310). When the CPU 11 determines that the received operation information is flick information indicating that a flick operation has been performed on the remote operation image displayed on the terminal 200 (S304: flick information), the CPU 11 Performs the flick process according to the received flick information, more specifically, the flick process corresponding to the flick operation on the remote operation image (S311).

  When the CPU 11 determines that the received operation information is pressing information indicating that a pressing operation has been performed on the remote operation image displayed on the terminal 200 (S304: pressing information), the CPU 11 It is determined whether or not the pressed coordinates included in the received pressed information fit within the screen of the LCD 16 (S305). Specifically, for example, when the screen size of the LCD 16 is 480 × 320 dots, the received pressed coordinates are coordinates located within a rectangular area from (0, 0) to (480, 320). If there is, the CPU 11 determines that the pressed coordinates are within the LCD 16.

  When the CPU 11 determines that the received pressed coordinates are within the screen of the LCD 16 (S305: Yes), the CPU 11 determines the position indicated by the pressed coordinates on the screen of the first page as the pressed position (S306). On the other hand, when the CPU 11 determines that the received pressed coordinates do not fit within the screen of the LCD 16 (S305: No), the CPU 11 determines which coordinates on the screen of which page the received pressed coordinates are set as the pressed position. (S312).

  Specifically, in S312, the CPU 11 first calculates a quotient obtained by dividing X and Y of the received pressed coordinates (X, Y) by the length Lx in the X direction of the LCD 16 and the length Ly in the Y direction, respectively. calculate. Next, the CPU 11 determines from the value Na of the integer part of the quotient that is 1 or more of the quotient (X / Lx) or the quotient (Y / Ly) whether the screen of the Na page has been pressed. The CPU 11 determines the pressed coordinates on the screen of the Nath page based on the remainder Ma of the quotient (X / Lx) or quotient (Y / Ly) having a quotient of 1 or more. That is, when the quotient (X / Lx) is 1 or more, the coordinate (Ma, Y) on the screen of the Na page is determined as the pressed position. On the other hand, when the quotient (Y / Ly) is 1 or more, the coordinate (X, Ma) on the screen of the Na page is determined as the pressed position.

  The CPU 11 determines whether the pressed position determined in S306 or S312 is located on various operators for inputting instructions to the MFP 10, such as the button image 81 (S307). When the CPU 11 determines that the determined pressing position is not located on the operator (S307: No), the pressing operation performed on the remote operation image displayed on the terminal 200 is performed on the operator. Indicates that it is not broken. Therefore, in such a case, the CPU 11 proceeds to S302. On the other hand, when the CPU 11 determines that the determined pressing position is located on the operation element (S307: Yes), the CPU 11 executes a pressing process on the operation object to be pressed (S308).

  For example, when the CPU 11 determines that the pressing operation performed on the remote operation image displayed on the terminal 200 is a pressing operation on any of the button images 81a to 81e, the CPU 11 performs the pressing process. Then, processing corresponding to the pressed button image 81 is executed. Specifically, the CPU 11 executes processing for displaying a screen (not shown) related to execution of a function corresponding to the pressed button images 81a to 81e. As a result, the screen displayed on the LCD 16 is switched from the function menu screen 79 to a screen related to execution of the function corresponding to the pressed button images 81a to 81e.

  Alternatively, when the CPU 11 determines that the pressing operation performed on the remote operation image displayed on the terminal 200 is the pressing operation on the button image 91c or the button image 91d, the CPU 11 performs the pressing process as A process of switching the screen displayed on the LCD 16 to the screen of the previous page or the next page with respect to the currently displayed screen is executed. When the CPU 11 determines that the pressing operation performed on the remote operation image displayed on the terminal 200 is a pressing operation on an operator that instructs execution of various functions such as a copy function, the CPU 11 Performs a process using a function to be instructed as a pressing process.

  According to the processing of S308, S310, and S311, the MFP 10 is remotely operated from the terminal 200 on which the operation support application 212b is installed. Then, after the processes of S308, S310, and S311, the CPU 11 transmits execution result information indicating the execution result of each process to the terminal 200 by Wi-Fi communication (S309). The execution result information includes information indicating whether the display screen of the LCD 16 has changed as a result of the processing. After the process of S309, the CPU 11 shifts the process to S301.

  FIG. 4 is a flowchart showing terminal-side processing at the time of remote operation, which is executed by the CPU 211 of the terminal 200. This process is a process for remotely operating the MFP 10. This process is started on the condition that the above-described preliminary process is completed, as in FIG.

  The CPU 211 waits for reception of image data or the like transmitted by the MFP 10 in S301 or reception of an operation on the touch panel 217 (S401: No, S404: No). When receiving image data or the like from the MFP 10 (S401: Yes), the CPU 211 stores the screen ID received together with the image data in the RAM 213 (S402). Next, the CPU 211 configures an image for remote operation based on the positional information of each image received together with the image data, and displays it on the LCD 216 (S403). As a result of the process of S403, for example, the image 170a in FIG. 2B or the image 170b in FIG. 2C is displayed on the LCD 216. After the process of S403, the CPU 211 moves the process to S404.

  When the CPU 211 receives an operation on the touch panel 217 (S404: Yes), the CPU 211 determines the type of the received operation (S405). When the CPU 211 determines that the pressing operation has been received (S405: pressing), the CPU 211 acquires the enlargement / reduction ratio Z of the image for remote operation displayed on the LCD 216 with respect to the same size image (S406). The enlargement / reduction ratio Z is a size ratio of the current image to the equal-size image. In the case of the same size image, the enlargement / reduction ratio Z is 1. When the enlargement / reduction ratio Z is a value smaller than 1, it indicates that the current image is reduced from the same-size image. On the other hand, when the enlargement / reduction ratio Z is a value larger than 1, it indicates that the current image is enlarged from the same-size image. The enlargement / reduction ratio Z is stored in the RAM 216 and is updated whenever the image displayed on the LCD 216 is enlarged or reduced. Note that the same size image is an image that has not been enlarged or reduced. In the present embodiment, an image based on the image data received from the MFP 10 in S401 is assumed to be a normal size image.

  Based on the enlargement / reduction ratio Z acquired in S406, the CPU 211 converts the pressed coordinates received as the pressed operation into pressed coordinates for the same-size image (S407). Specifically, when the received pressed coordinates are (X1, Y1), the CPU 211 divides X1 and Y1 by the enlargement / reduction ratio Z, respectively. That is, the pressed coordinates after conversion are (X1 / Z, Y1 / Z). Next, the CPU 211 transmits the pressing information and the screen ID stored in the RAM 213 to the MFP 10 by Wi-Fi communication (S408). The pressing information includes information indicating that the pressing operation has been performed, and the pressed coordinates after conversion in S407. As a result of the process of S408, the CPU 11 of the MFP 10 executes the processes of S305 to S308 and S312 described above on condition that the image ID is appropriate.

  As described above, even if the image for remote operation displayed on the LCD 216 is enlarged or reduced, the coordinates pressed by the user are transmitted to the MFP 10 as the pressed coordinates of the equal-size image. Thus, it is possible to appropriately execute the process according to the operated operator. In addition, even when the remote operation image is enlarged or reduced in this manner, the MFP 10 can execute processing according to the user's intention, so that the user has the desired size of the remote operation image. It can be used by enlarging or reducing it, which is convenient. When the enlargement / reduction ratio Z is 1, that is, when the current image is an equal-magnification image, the received pressed coordinates may be transmitted without performing the conversion in S407.

  On the other hand, when the CPU 211 determines that the drag operation has been received (S405: drag), the CPU 211 transmits the drag information and the screen ID stored in the RAM 213 to the MFP 10 by Wi-Fi communication (S410). The drag information includes information indicating that the drag operation has been performed, and start and end point coordinates of the accepted drag operation. On the other hand, when the CPU 211 determines that the flick operation has been accepted (S405: flick), the CPU 211 transmits the flick information and the screen ID stored in the RAM 213 to the MFP 10 by Wi-Fi communication (S411). The flick information includes information indicating that the flick operation has been performed, and the acceleration direction and acceleration of the accepted flick operation. As a result of the processing of S410 or S411, the CPU 11 of the MFP 10 executes the above-described processing of S310 or S311 on condition that the image ID is appropriate.

  After the processing of S408, S410, and S411, the CPU 211 waits to receive execution result information from the MFP 10 (S409: No, 413: No). The execution result information is information that the CPU 11 of the MFP 10 transmits in the above-described S309. When the CPU 211 receives the execution result information from the MFP 10 (S409: Yes), the CPU 211 shifts the processing to S401. In S <b> 408, S <b> 410, and S <b> 411, if the CPU 211 transmits information corresponding to the operation on the touch panel 217 to the MFP 10, and the screen displayed on the LCD 16 is changed by the MFP 10, Information indicating that is included. Therefore, since image data or the like corresponding to the changed screen is transmitted from the MFP 10, the CPU 211 executes the process of S403. As a result, the LCD 216 of the terminal 200 displays the changed screen according to the changed screen. A remote operation image is displayed.

  When the CPU 211 receives no execution result information from the MFP 10 and a predetermined time, for example, 5 seconds elapses (S413: Yes), the CPU 211 displays the operated remote operation image on the MFP 10. The fact that it is different from the screen is displayed on the LCD 216 (S414), and the process proceeds to S401. The display by the process of S414 is displayed when the CPU 11 of the MFP 10 determines that the screen ID transmitted in S408 is inappropriate and the execution result information is not transmitted. Therefore, the user can grasp that the screen of the MFP 10 has been changed from the image for remote operation and cannot be operated based on the display.

  On the other hand, when the CPU 211 determines in S405 that an operation for an area other than the image for remote operation on the LCD 216, that is, the area 216a is accepted (S405: outside image operation), the CPU 211 performs processing according to the accepted operation. Is executed (S412), and the process proceeds to S401. For example, when the CPU 211 receives a pinch-out operation on the area 216a, the CPU 211 enlarges the displayed remote operation image in accordance with the pinch-out operation amount. On the other hand, when the CPU 211 receives a pinch-in operation on the area 216a, the CPU 211 reduces the displayed remote operation image according to the pinch-in operation amount. When the CPU 211 receives a drag operation or flick operation on the area 216, the CPU 211 scrolls the displayed remote operation image in accordance with the drag operation or flick operation. As described above, according to the operation support application 212b, it is possible to effectively use the area 216 where the image for remote operation is not displayed. Further, since the size and display part of the image for remote operation can be changed according to the user's desire, the operability and convenience of the image for remote operation are good.

  According to the first embodiment, the MFP 10 can be remotely operated from the terminal 200 on which the operation support application 212b is installed. In particular, in the case of a series of screen groups composed of a plurality of screens, such as the function menu screen 70, even an operator included in a screen that is not displayed on the LCD 16 of the MFP 10 is subject to remote operation from the terminal 200. Therefore, it is excellent in operability and convenience.

  Next, a second embodiment of the present invention will be described with reference to FIG. In the first embodiment described above, on the MFP 10 side, the pressing coordinate of the pressing operation on the touch panel 217 of the terminal 200 is a pressing position on a screen of which page among a series of image groups composed of a plurality of screens. It was set as the structure which determines whether there exists. In contrast, in the second embodiment, the determination is performed on the terminal 200 side. In the second embodiment, the same parts as those in the first embodiment described above are denoted by the same reference numerals, and the description thereof is omitted.

  Hereinafter, the difference from the first embodiment will be mainly described. The MFP 11 processing in the second embodiment at the time of remote operation is executed by the CPU 11 according to the control program 12a of the second embodiment. In the present embodiment, since the above determination is not performed on the MFP 10 side, in the present embodiment executed by the CPU 11 of the MFP 10, the MFP-side processing at the time of remote operation is the same processing in the first embodiment described above (FIG. 3). ) To S305, S306, and S312 are omitted.

  FIG. 5 is a flowchart illustrating terminal-side processing during remote operation according to the second embodiment. This process is executed by the CPU 211 according to the operation support application 212b of the second embodiment. This process differs from the first embodiment when the operation received by the CPU 211 is a pressing operation. Specifically, the CPU 11 acquires the screen size of the MFP 10 after the process of S407 (S501). As a method of acquiring the screen size, there are a method of inquiring the MFP 10 for the screen size of the LCD 16 and acquiring it as a reply, a method of storing the screen size of the LCD 16 in the flash memory 212 and reading it out. As a method of storing the screen size in the flash memory 212, for example, a method of storing the screen size received as a result of an inquiry to the MFP 10, a method of storing the screen size acquired from the MFP 10 in the pre-processing, There is a method of previously storing a table in which the screen size is stored.

  Next, the CPU 211 determines whether the pressed coordinates after conversion fit within the screen of the LCD 16 of the MFP 10 as in the process of S305 (S502). When the CPU 211 determines that the converted pressed coordinates fall within the screen of the LCD 16 (S502: Yes), the CPU 211 determines the converted pressed coordinates as the pressed coordinates on the screen of the first page (S503). On the other hand, when the CPU 211 determines that the pressed coordinates after conversion do not fit within the screen of the LCD 16 (S502: No), the CPU 211 determines which coordinates on the screen of which page the pressed coordinates after conversion are pressed. (S504). The determination in S504 can be performed in the same manner as in S312.

  The CPU 211 executes the processing of S408 after the processing of S503 or S504. In the second embodiment, in step S <b> 408, the CPU 211 transmits, as pressing information, information indicating that a pressing operation has been performed and information indicating which coordinate on which screen is pressed to the MFP 10. The MFP 10 does not perform each process of S305, S306, and S312 and presses which position on which page of a series of image groups composed of a plurality of screens based on the pressing information received from the terminal 200. You can recognize it.

  According to the second embodiment, on the terminal 200 side, the pressing coordinate of the pressing operation on the touch panel 217 is a position on the screen of which page among a series of image groups configured by a plurality of screens. Can be determined. That is, on the terminal 200 side, the position of the pressing operation can be specified even on a screen that is not displayed on the LCD 16 of the MFP 10.

In the above embodiment, the operation support application 212b is an example of an operation support program. The terminal 200 is an example of a terminal device. The MFP 10 is an example of a process execution device.
The control program 12a is an example of a process execution program. The LCD 216 is an example of a first display unit. The touch panel 217 is an example of an input unit. The wireless communication unit 223 is an example of a communication unit of the terminal device. The CPU 211 is an example of a control unit of the terminal device. The LCD 16 is an example of a second display unit or a display unit. The wireless communication unit 23 is an example of a communication unit of the process execution device. The CPU 11 is an example of a control unit of the process execution device. The flash memory 12 is an example of a storage unit. The function menu screen 70 is an example of a series of screen groups. The button images 81, 91c, and 91d are examples of the operator image. Each of the button images 81a to 81e is an example of an element image. The remote operation image 170 is an example of a display target image.

  The CPU 211 that executes the process of S401 is an example of an image receiving unit. The CPU 211 that executes the process of S403 is an example of an image display unit. The CPU 211 that executes the process of S408 is an example of an information transmission unit. The CPU 211 that executes the process of S501 is an example of an acquisition unit. The CPU 211 that executes the processes of S502 to S504 is an example of a specifying unit. The CPU 211 that executes the process of S407 is an example of a conversion unit. The CPU 211 that executes the process of S412 is an example of a process execution unit. CPU11 which performs the process of S301 is an example of an image transmission means. The CPU 11 that executes the process of S308 is an example of a process execution unit. The CPU 11 that executes the processes of S305, S306, and S312 is an example of the specifying unit of claim 9. The CPU 11 that executes the process of S303 is an example of a determination unit.

  Although the present invention has been described based on the embodiments, the present invention is not limited to the above-described embodiments, and various improvements and modifications can be easily made without departing from the spirit of the present invention. Can be inferred.

  For example, in the above-described embodiment, the terminal 200 that is a tablet terminal is illustrated as the terminal device of the present invention. However, as long as it is a device that can communicate with a processing execution device such as the MFP 10 and has a screen such as an LCD, Even a personal computer, a smartphone, a portable terminal, a digital camera, a portable music playback device, or the like can be an example of the terminal device of the present invention. In the above embodiment, the Android OS is exemplified as the OS 12a, but another OS may be adopted. In the above embodiment, the touch panel 217 detects a pressing operation on a remote operation image. However, the present invention may be applied to a configuration in which an operation for a remote operation image is performed using a pointing device such as a mouse. Can be applied.

  In the above embodiment, the MFP 10 which is a multifunction peripheral device is exemplified as the processing execution device of the present invention. However, a device having a single function such as a scanner device or a printer device, a device for reproducing images or music, and the like. Even so, it can be an example of the processing execution apparatus of the present invention.

  In the above embodiment, the MFP 10 is configured to transmit the image data and the position information to each of the button images 81a to 81e to the terminal 200. However, the image of the collective image in which a plurality of button images are arranged in advance. The data and the position information of the aggregate image may be transmitted to the terminal 200. Even if the collective image is prepared in the MFP 10 in advance, the collective image may be generated from component images included in each screen for each screen, for example, before transmission in S301. It should be noted that the “collective image” of this modification is an example of the “collective image” in the claims.

  In the above embodiment, the image for remote operation is an image composed of a component image such as the button image 81 and a background image such as the background image 91, but the image for remote operation is composed only of the component image. May be. That is, the MFP 10 may be configured to transmit only the image data and position information of the component image. Further, not only the screen being displayed on the LCD but also a background image of a screen that is not displayed may be included in the image for remote operation. In the above embodiment, instead of the MFP 10 transmitting the image data of the background image and the component image and the position information of each image to the terminal 200, the MFP 10 generates an image for remote operation. Alternatively, it may be configured to transmit to the terminal 200.

  In the above embodiment, the function menu screen 70 composed of two screens is exemplified as a series of screen groups. However, the present invention can be applied even to screen groups composed of three or more screens. In the above embodiment, the upper left coordinate of the screen of the first page in the series of screen groups is set to (0, 0). However, the screen used as a reference for the position is not limited to the first page, and is set as appropriate. it can. Alternatively, the upper left coordinate of the screen being displayed on the LCD 16 may be (0, 0).

  In the embodiment, the communication between the MFP 10 and the terminal 200 is exemplified by the Wi-Fi communication in the infrastructure mode via the AP 50. However, even in the Wi-Fi communication in the ad hoc mode that is a direct connection, The present invention can be applied. Alternatively, the present invention can be applied to other communication methods such as Bluetooth (registered trademark).

10: MFP. 200: terminal, 212b: operation support application

Claims (12)

The control unit of the terminal device including the first display unit, the input unit that detects an input operation on the first display unit, the communication unit that communicates with the processing execution device, and the control unit is executable. And an operation support program for operating the processing execution device based on the input operation detected by the input unit,
A display target image, which is image data for a plurality of screens constituting a series of screen groups and includes at least a plurality of indicator images each assigned a process performed by the processing execution device, included in the series of screen groups. Image receiving means for receiving image data to be displayed on the first display unit from the processing execution device via the communication unit;
Image display means for displaying the display target image based on the image data received by the image receiving means on the first display section;
Operation position information corresponding to the position on the display target image designated by the designation operation for designating the position on the display target image displayed on the first display unit, which is the input operation detected by the input unit. As an information transmission means for transmitting to the processing execution apparatus that is the transmission source of the image data via the communication unit,
An operation support program for causing the control unit to function. The image receiving means receives each of the plurality of indicator images as element images, receives image data for each element image or for each group image including a plurality of element images, and for each element image or each group image. Receiving element image position information specifying the arrangement of the element image or the group image,
The image display means displays the display target image in which images of the plurality of indicators are arranged based on image data and element image position information received by the image receiving means. Item 1. The operation support program according to Item 1. The process execution device includes a second display unit,
The image receiving means is provided for each element image or each group image transmitted by the process execution device on condition that one screen of the series of screen groups is displayed on the second display unit. Receiving image data, image data of a background image displayed on the second display unit, the element image position information, and background image position information indicating the position of the background image;
The image display means arranges each of the images of the plurality of indicators based on the image data, the element image position information, and the background image position information received by the image receiving means, and the images of the plurality of indicators. The operation support program according to claim 2, wherein the display target image in which the background image is arranged as a background of an indicator image displayed on the second display unit is displayed. The process execution device includes a second display unit,
Obtaining means for obtaining a length of the display area of the second display unit along an arrangement direction of the series of screen groups;
A position specified by a specifying operation for specifying a position on a display target image displayed on the first display unit, which is an input operation detected by the input unit based on the length acquired by the acquiring unit As a specifying means for specifying which position of which screen in the series of screen groups is specified,
Make the control function,
4. The information transmission unit according to claim 1, wherein the information transmission unit transmits information indicating the screen and the position specified by the specifying unit as the operation position information to the processing execution device. 5. Operation support program. When the display target image displayed on the first display unit is displayed enlarged or reduced at the same magnification, a position on the display target image that is an input operation detected by the input unit is specified. Conversion means for converting the position designated by the designation operation into a position in the display target image of the same magnification, based on a scaling factor of display with respect to the same magnification;
And let the control unit function,
In the case where the display target image is displayed enlarged or reduced from the same magnification, the information transmission unit uses the information indicating the position obtained by conversion by the conversion unit as the operation position information, as the processing execution device. The operation support program according to any one of claims 1 to 4, wherein the operation support program is transmitted to. When a predetermined input operation is detected by the input unit, if the input operation is performed outside the display target image, a process execution unit that executes a process according to the input operation ,
Make the control function,
When a predetermined input operation is detected by the input unit and the input operation is performed on the display target image, the information transmission unit displays information corresponding to the input operation. 6. The operation support program according to claim 1, wherein the operation support program is transmitted. A first display unit;
An input unit for detecting an input operation on the first display unit;
A communication unit that communicates with the processing execution device;
A control unit,
The controller is
A display target image, which is image data for a plurality of screens constituting a series of screen groups and includes at least a plurality of indicator images each assigned a process performed by the processing execution device, included in the series of screen groups. Image receiving means for receiving image data to be displayed on the first display unit from the processing execution device via the communication unit;
Image display means for displaying the display target image based on the image data received by the image receiving means;
Operation position information corresponding to the position on the display target image designated by the designation operation for designating the position on the display target image displayed on the first display unit, which is the input operation detected by the input unit. Information transmitting means for transmitting to the processing execution apparatus which is the transmission source of the image data via the communication unit
A terminal device comprising: A display unit;
A communication unit that communicates with the terminal device;
A control unit,
The controller is
A display target image that is image data for a plurality of screens constituting a series of screen groups, and includes at least a plurality of indicator images assigned to the processes performed by the processing execution device, which are included in the series of screen groups. Image transmission means for transmitting image data to be displayed on the display unit of the terminal device to the terminal device via the communication unit;
The operation position information received from the terminal device based on the transmission of the image data to the terminal device by the image transmission means is the operation position information indicating the position on the display target image specified in the terminal device. In this case, when the operation position information indicates the position of the indicator on the image, a process execution means for executing a process assigned to the indicator image;
A process execution device comprising: When the operation position information is received from the terminal device, the display target image specified by the operation position information based on the length of the display area of the display unit along the arrangement direction of the series of screen groups A specifying means for specifying which position of which screen in the series of screen groups designates which position;
9. The process execution means, when the image of the indicator is located at the screen and position specified by the specification means, the process assigned to the image of the indicator is executed. Process execution device.   The image transmitting means is configured such that each of the plurality of indicator images is an element image on the condition that one screen of the series of screen groups is displayed on the display unit. Image data for each group image including the element image, image data of a background image of the screen being displayed on the display unit, element image position information specifying the arrangement of the element image or the group image, and the series of Background image position information for specifying the position of the background image relative to the position of the images of the plurality of indicators, which is calculated based on which screen of the screen group is displayed on the display unit. The processing execution apparatus according to claim 8 or 9, wherein the transmission is performed. A storage unit for storing screen information for identifying each screen;
The image transmission means includes the image data and the screen information stored in the storage unit on the condition that one screen of the series of screen groups is displayed on the display unit. Screen information corresponding to the screen displayed on the display unit is transmitted to the terminal device,
When the operation position information is received from the terminal device based on the transmission of the image data and the screen information to the terminal device by the image transmission means, the display unit at the reception timing of the operation position information A determination means for determining whether the screen displayed on the screen includes a screen specified by the screen information received from the terminal device together with the operation position information, the screen being included in the series of screen groups;
The process execution means is included in the series of screen groups in which the screen displayed on the display unit at the reception timing of the operation position information includes a screen specified by the received screen information by the determination means. If the operation position information received from the terminal device indicates the position of the indicator on the image, on the condition that the screen is determined to be a screen to be displayed, the process assigned to the indicator image is executed. The process execution device according to claim 8, wherein A display unit;
A communication unit that communicates with the terminal device;
A processing execution program that can be executed by the control unit.
A display target image that is image data for a plurality of screens constituting a series of screen groups, and includes at least a plurality of indicator images assigned to the processes performed by the processing execution device, which are included in the series of screen groups. Image transmission means for transmitting image data to be displayed on the display unit of the terminal device to the terminal device via the communication unit;
The operation position information received from the terminal device based on the transmission of the image data to the terminal device by the image transmission means is the operation position information indicating the position on the display target image specified in the terminal device. In this case, when the operation position information indicates the position of the indicator on the image, as a process execution means for executing the process assigned to the indicator image,
A processing execution program for causing the control unit to function.

Images