JP2013186773A - User terminal equipment, image processor, operator terminal equipment, information processing system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent a multi-touch operation to be performed by an operator from being guided to an inoperable state by a user.SOLUTION: When two touch position coordinates showing that two points have been simultaneously touched by an operator are included in operation information acquired from operator terminal equipment 2, an image processor determines positions where almost the same distance as a distance L1 between the two points actually touched by the operator is set or a position where the distance between the two points is set to a preliminarily set distance as two display positions where an operation guidance image G1 is displayed for guiding a touch operation to those two points to a user on an operation panel 28.

Description

  The present invention relates to a user terminal device, an image processing device, an operator terminal device, an information processing system, and a program, and in particular, a technique for guiding an operation performed on the operator terminal device to a user using the user terminal device or the image processing device. About.

  Generally, when the same image data is displayed on a plurality of display devices having different screen sizes, the size of the image data displayed on each display device is displayed differently according to the screen size ratio. As a conventional technique for preventing this, when a distance between any two points on the image data is designated, the image data is enlarged or reduced so that the distance between the two points becomes the designated distance on one display device. In other display devices, a technique has been proposed in which display magnification is determined and displayed so as to be the same size as the display size of one display device (for example, Patent Document 1). This prior art has a configuration in which the display magnification of the image data in each display device is individually determined and displayed so that the distance between any two points designated on the image data is the actual size at the time of image display. is there. Thus, for example, when a plurality of users confirm the specific finish of a document on their respective display devices, documents of the same size can be displayed on each display device. It is possible to prevent image discrepancy.

Japanese Patent Publication No. 2005-148450

  By the way, in recent years, mobile terminal devices such as smartphones have been widely used. This type of portable terminal device occupies a large area of the display screen and the touch panel that can be operated by the user with respect to the entire device, and is configured to detect not only a single touch operation by the user but also a multi-touch operation. . The single touch operation is an operation of touching one point on the display screen, and the multi-touch operation is an operation of touching two or more points on the display screen.

  For example, the user activates the application and uses the mobile terminal device by single-touching the display position of the icon image corresponding to the desired application from among a plurality of icon images displayed on the display screen. can do. Also, when enlarging or reducing the image displayed on the display screen, a pinch-out operation is performed to increase the distance between the two points while multi-touching any two points on the image, or By performing a pinch-in operation that reduces the distance between two points, an image can be displayed in a size desired by the user. In particular, the multi-touch operation as described above has an advantage that it is not necessary to display an enlarge button, a reduce button, or the like on the display screen, and the operation image is easily understood by the user.

  On the other hand, in recent years, an image processing apparatus such as an MFP (Multifunction Peripheral) that can detect not only a single touch operation but also a multi-touch operation on an operation panel serving as a user interface is becoming widespread. However, the image processing device is not a user-specific machine such as a portable terminal device, but is installed in an office or the like and is shared by many users. Therefore, it takes time for each user to become familiar with the operation. The fact is that it is not possible. Therefore, for example, while an image preview is displayed, an enlargement button or a reduction button can be displayed by a multi-touch operation. There is a case where an inquiry is made to a support center provided by a manufacturer and distributor without knowing what operation should be performed.

  However, every time the support center receives an inquiry about operation, dispatching an operator to the site has a problem of increasing costs. On the other hand, it is difficult to accurately convey the operation on the operation panel only by the operator and the user exchanging by telephone.

  Therefore, for example, for a mobile terminal device used by an operator (hereinafter referred to as an “operator terminal device”), an application capable of mutually transmitting and receiving information related to the display screen and screen operation in cooperation with the image processing device is installed in advance. When there is an inquiry from the user, it is conceivable that the operator starts the application and performs the operation guidance so that the operation is displayed on the operation panel of the image processing apparatus by operating the operator terminal device.

  However, in this case, the guidance image of the touch operation cannot be displayed at the coordinate position as it is on the operation panel of the image processing apparatus based on the coordinate position where the operator performed the touch operation on the operator terminal device. Because, when the display resolutions of the operator terminal device and the operation panel in the image processing device are different from each other, the same screen in which a plurality of icon images are arranged on each of the operator terminal device and the operation panel is displayed in full screen. This is because even in the state, the coordinate position of the icon image displayed on the operator terminal device is different from the coordinate position of the icon image displayed on the operation panel. In such a case, if the guide image is displayed as it is at the coordinate position touched by the operator terminal device on the operation panel of the image processing device, the operation on the icon image different from the icon image on which the operator performed the touch operation is performed. Therefore, it is impossible to guide accurate operation.

  Therefore, on the operation panel of the image processing apparatus, when the guide image is displayed based on the coordinate position where the operator performs the touch operation, the display position of the guide image is subjected to coordinate conversion based on the screen size ratio or the resolution ratio. It is necessary to decide. By performing such coordinate conversion, for example, when one point at the center of the screen is touch-operated in the operator terminal device, an image for guiding the touch operation is also displayed in the center of the screen on the operation panel of the image processing apparatus. Can do. When one point at the screen corner is touch-operated on the operator terminal device, an image for guiding the touch operation to the one point at the screen corner can be similarly displayed on the operation panel of the image processing apparatus. .

  However, when a multi-touch operation is performed on the operator terminal device, if a guide image is displayed by performing coordinate conversion in the same manner as described above, the user who operates the operation panel of the image processing device can There is a possibility that an image for guiding the touch operation is displayed. For example, when the operator performs a multi-touch operation on two screen corners on a diagonal line using a smartphone, the two guide images displayed on the operation panel of the image processing apparatus are two screen corners on the diagonal line of the operation panel. Is displayed. However, when the screen size of the operation panel is several times larger than the screen size of the smartphone, the user can simultaneously perform touch operations with one hand on the two guide images displayed in the screen corner of the operation panel. Can not be. In such a case, it is difficult for the user to accurately grasp what the operation performed by the operator is. Therefore, in the case of multi-touch operation, there is a problem that the operation of the operator cannot be accurately transmitted to the user.

  As described above, the multi-touch operation involves a pinch-out operation that increases the distance between the two points and a pinch-in operation that decreases the distance between the two points, as described above. If the distance between the two points to be displayed is only displayed at a predetermined distance, it may not be possible to guide the user to perform a pinch-out operation or a pinch-in operation performed by the operator with one-hand operation. Is similar to the above in that it cannot be accurately communicated to the user.

  The present invention has been made to solve the above-described problems, and prevents the multi-touch operation performed by the operator from being guided to a state where the user cannot operate, so that the user can correctly perform the multi-touch operation by the operator. The object is to provide a user terminal device, an image processing device, an operator terminal device, an information processing system, and a program that can be learned.

  In order to achieve the above object, the invention according to claim 1 is a user terminal device capable of communicating with an operator terminal device including a display panel capable of multi-touch operation, and display means having a screen size different from that of the display panel; Operation detection means for detecting an operation on the display means, and operation information including touch position coordinates on the display panel is acquired from the operator terminal device when an operation on the display panel is detected in the operator terminal device. And determining the display position on the display unit corresponding to the touched position on the display panel by converting the touch position coordinates included in the operation information according to the resolution of the display unit. Display position determining means to operate, and an operation plan for guiding operations performed on the display panel Display control means for displaying an image at a display position determined by the display position determination means, wherein the display position determination means includes two touch positions when two places are touched simultaneously on the display panel. When coordinates are included in the operation information, the 2 in the display panel is based on the position on the display unit corresponding to one of the two points or any one point between the two points. The display position of the two points on the display means is determined at a position that is substantially the same as the physical distance between the points or at a preset distance.

  According to a second aspect of the present invention, in the user terminal device according to the first aspect, the display control means includes movement information relating to movement of the two points starting from the touch position coordinates of the two points in the operation information. In this case, the touch operation guide image is displayed at the two display positions determined by the display position determining means, and two points from the display position of the touch operation guide image are displayed based on the movement information included in the operation information. A moving operation guidance image for guiding the moving direction is displayed.

  According to a third aspect of the present invention, in the user terminal device according to the first or second aspect, the display position determining means maintains a relative positional relationship between the two points based on the touch position coordinates of the two points. Thus, the display position of the two points on the display means is determined.

  According to a fourth aspect of the present invention, in the user terminal device according to the second aspect, when the display position determining means determines the display position of the two points on the display means at a position that is a preset distance, If the movement information included in the operation information is information that causes the two points to approach each other, the display position of the two points is determined at a position that is a first distance set in advance, and is included in the operation information If the movement information to be moved is information that separates the two points from each other, the display position of the two points is determined at a position that becomes a second distance that is preset in a distance closer to the first distance. It is the structure characterized by these.

  According to a fifth aspect of the present invention, in the user terminal device according to the first, second, or fourth aspect, when the left-handed information is preset, the display position determining unit is based on the touch position coordinates of the two points. The relative position relationship between the two points is reversed left and right, and the display position of the two points on the display means is determined.

  The invention according to claim 6 is an image processing apparatus capable of communicating with an operator terminal device having a display panel capable of multi-touch operation, and has a display means having a larger screen size than the display panel, and an operation for the display means Operation detection means for detecting the screen, and screen data transmission means for displaying the same display screen on the display panel of the operator terminal device by transmitting screen information relating to the display screen of the display means to the operator terminal device, The touch position coordinates on the display panel are detected from the operator terminal device when an operation on the display panel is detected while the same display screen as the display unit is displayed on the display panel of the operator terminal device. Operation information acquisition means for acquiring operation information including the operation information, and included in the operation information Display position determining means for determining a display position on the display means corresponding to the touched position on the display panel by converting the touch position coordinates to be displayed according to the resolution of the display means, and displaying on the display means Display control means for further displaying an operation guide image for guiding operations performed on the display panel at a display position on the display means determined by the display position determining means on the display screen. The display position determination means, when the operation information includes two touch position coordinates associated with two touches on the display panel at the same time, With the position on the display means corresponding to any one point between two points as a reference, substantially the same as the physical distance between the two points on the display panel; Alternatively, the display position of the two points on the display means is determined at a position having a preset distance.

  The invention according to claim 7 is an operator terminal device capable of communicating with a user terminal device including display means for displaying various screens, wherein the display panel has a screen size different from that of the display means, and an operation on the display panel is detected. Detecting the touch position coordinates with respect to the display panel as the operation on the display panel is detected by the operation detecting means, the storage detecting means for storing resolution information relating to the resolution of the display means, By converting the touch position coordinates based on the resolution information, the display position determining means for determining the display position on the display means corresponding to the touched position on the display panel, and the display position determining means Transmitting means for transmitting display position information based on the display position to the user terminal device; And the display position determining means, when two touch position coordinates are detected by the operation detecting means when two places are touched simultaneously on the display panel, one of the two points, Alternatively, a physical distance between the two points touched on the display panel is substantially the same as a position on the display unit corresponding to an arbitrary point between the two points, or a preset distance. The display position of the two points on the display means is determined at a position where

  According to an eighth aspect of the present invention, in the operator terminal device according to the seventh aspect, when the movement of the two touch position coordinates is detected by the operation detection unit, the transmission unit moves the two points. Is transmitted to the user terminal device.

  The invention according to claim 9 is an information processing system in which an operator terminal device having a display panel that can be operated by an operator and a user terminal device having a display means that can be operated by a user can communicate with each other, An operation information generating unit that is provided in the operator terminal device and generates operation information including touch position coordinates on the display panel in response to detection of an operator operation on the display panel, and the operator terminal device or the In the display means corresponding to the touched position on the display panel by converting the touch position coordinates included in the operation information according to the resolution of the display means in the user terminal apparatus provided in the user terminal device Display position determining means for determining a display position; and the user Display control means provided on a terminal device for displaying an operation guidance image for guiding operations performed on the display panel at a display position determined by the display position determining means, and the display position determining means When the operation information includes two touch position coordinates associated with touching two places on the display panel, one of the two points or an arbitrary point between the two points Using the position on the display means corresponding to one point as a reference, the physical distance between the two points on the display panel is approximately the same as the physical distance between the two points, or the two points on the display means are located at a preset distance. The display position is determined.

  The invention according to claim 10 is a communication unit that communicates with an operator terminal device including a display panel capable of multi-touch operation, a display unit having a screen size different from that of the display panel, and an operation detection unit that detects an operation on the display unit. A program executed in a user terminal device comprising: an operation for the display panel detected in the operator terminal device from the operator terminal device to the display panel. Operation information acquisition means for acquiring operation information including touch position coordinates, and by converting the touch position coordinates included in the operation information according to the resolution of the display means, the position corresponding to the touched position on the display panel Display position determining means for determining a display position in the display means; And a display control means for displaying an operation guidance image for guiding an operation performed on the display panel at the display position determined by the display position determination means, and the display position determination means When the operation information includes two touch position coordinates resulting from two touches simultaneously on the display panel, one of the two points or an arbitrary point between the two points With reference to the corresponding position on the display means, the display position of the two points on the display means is set to a position that is substantially the same as the physical distance between the two points on the display panel or a preset distance. It is the structure characterized by determining.

  According to an eleventh aspect of the present invention, there is provided communication means for communicating with a user terminal device including display means for displaying various screens, a display panel having a screen size different from that of the display means, and operation detection for detecting an operation on the display panel. And a storage means for storing resolution information relating to the resolution of the display means, wherein the operator terminal device is detected by the operation detection means when an operation on the display panel is detected. Accordingly, a display position determining unit that determines a display position on the display unit corresponding to a position touched on the display panel by converting touch position coordinates on the display panel based on the resolution information. And based on the display position determined by the display position determining means. The display position determination means functions as transmission means for transmitting display position information to the user terminal device, and the display position determination means is configured to detect two touch positions by the operation detection means when two places are touched simultaneously on the display panel. When the coordinates are detected, the physical position between the two points on the display panel is based on the position on the display unit corresponding to one of the two points or one arbitrary point between the two points. The display position of the two points on the display means is determined at a position that is substantially the same as the distance or at a preset distance.

  According to the present invention, even when a multi-touch operation in which two places are simultaneously touched on the operator terminal device is performed, the display unit of the user terminal device or the image processing device has a range in which the user can simultaneously touch with one hand operation. It is possible to display an operation guidance image inside. Therefore, it is possible to satisfactorily prevent the multi-touch operation performed by the operator with one hand operation from being displayed with guidance in a state where the user cannot perform the one hand operation.

It is a figure showing an example of 1 composition of an information processing system. It is a block diagram which shows an example of the hardware constitutions and mechanism structure of an operator terminal device. It is a block diagram which shows an example of the hardware constitutions and functional structure of an image processing apparatus. It is a figure which shows the communication sequence of an operator terminal device and an image processing apparatus. It is a figure which shows an example of the state as which the same screen as the screen currently displayed on the display part of the operation panel was displayed in the display part of an operator terminal device. It is a figure explaining the display position determination process in an image processing apparatus when operation by an operator is single touch operation. It is a figure which shows the concept which performs the coordinate transformation similar to a single touch operation in the case of multi-touch operation. It is a figure explaining the concept of the display position determination process in an image processing device when an operator performs pinch-in operation by multi-touch operation. It is a figure explaining the concept of the display position determination process in an image processing apparatus when an operator performs pinch out operation by multi-touch operation. It is a figure explaining the concept of the specific process performed when determining the display position of 2 points | pieces in the case of multi-touch operation. It is a figure explaining the processing concept which determines the display position of 2 points | pieces by the method different from FIG. It is a figure explaining the processing concept which determines the display position of 2 points | pieces on the basis of a midpoint. It is a flowchart which shows an example of the process sequence performed in an operator terminal device. It is a flowchart which shows an example of the process sequence performed in an image processing apparatus. It is a flowchart which shows an example of the detailed process sequence of the display position determination process in 1st Embodiment. It is a figure which shows the example of a screen displayed on the display part of an operation panel, when the user of an image processing apparatus performs the setting regarding own touch width. It is a figure which shows the information hold | maintained previously in the image processing apparatus of 2nd Embodiment. It is a flowchart which shows an example of the detailed process sequence of the display position determination process in 2nd Embodiment.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the embodiments described below, members that are common to each other are denoted by the same reference numerals, and redundant descriptions thereof are omitted.

(First embodiment)
FIG. 1 is a diagram illustrating a configuration example of an information processing system 1 according to the present embodiment. The information processing system 1 is configured such that an operator terminal device 2 used by an operator and an image processing device 3 used by a user such as an MFP (Multifunction Peripherals) are connected to each other via a network 4 so as to communicate with each other. The network 4 is a wide area communication network including the Internet and a public telephone network, for example.

  The image processing apparatus 3 has a plurality of functions such as a copy function, a print function, a scan function, a FAX function, a BOX function, and a network communication function, for example. Various jobs related to image processing are executed based on the instructions. The image processing apparatus 3 is installed in, for example, an office and is used by many users. An operation panel 28 serving as a user interface when used by a user is provided on the front side of the apparatus main body of the image processing apparatus 3. The operation panel 28 functions as the user terminal device 26 in the present embodiment, displays various operation screens for the user, and accepts user operations performed on such operation screens.

  On the other hand, the operator terminal device 2 is configured by a smartphone, a tablet terminal device, or the like, and is, for example, a device that is occupied by an operator of a support center provided by a manufacturer or dealer of the image processing device 3. The operator terminal device 2 includes a display panel 11 that can be multi-touched by an operator. An application program for communicating with the image processing device 3 is installed in the operator terminal device 2 in advance. The operator terminal device 2 starts data communication with the image processing device 3 designated by the operator via the network 4 by starting the application program.

  When the user who uses the image processing apparatus 3 does not know the operation on the operation panel 28, for example, he calls the support center and receives instruction from the operator about the operation of the operation panel 28 while talking to the operator. At this time, the information processing system 1 according to the present embodiment is displayed on the operation panel 28 of the image processing device 3 by the operator starting an application program in the operator terminal device 2 and establishing a communication state with the image processing device 3. The display screen to be displayed can be displayed on the display panel 11 of the operator terminal device 2. When the operator performs an operation desired by the user on the display panel 11 in a state where the same display screen as that of the image processing device 3 is displayed on the display panel 11 of the operator terminal device 2, the operation information is converted into image processing. An operation guidance image for prompting the user to perform the same operation as the operation performed by the operator on the operation panel 28 of the image processing apparatus 3 is displayed. Therefore, the user can learn the operation performed by the operator by performing an operation on the operation panel 28 in accordance with the operation guide image displayed on the operation panel 28. Hereinafter, the information processing system 1 will be described in more detail.

  FIG. 2 is a block diagram illustrating an example of a hardware configuration and a mechanism configuration of the operator terminal device 2. The operator terminal device 2 includes a CPU 10 that performs arithmetic processing based on a program, a display panel 11 that displays various types of information to the operator and receives operation inputs from the operator, a ROM (Read Only Memory), and an SRAM (Static Random). (Access Memory), a battery-backed NVRAM (nonvolatile RAM), and the like, and a communication control unit 13 that performs data communication via the network 4.

  The display panel 11 has a predetermined screen size and display resolution, and includes a display unit 14 including a color liquid crystal display, and displays various screens to the operator. The display panel 11 has an operation unit 15 that receives an operation by an operator. The operation unit 15 is configured by, for example, a touch panel sensor arranged on the display screen of the display unit 14. Further, as described above, the application program 12a for communicating with the image processing apparatus 3 is installed in the storage unit 12 in advance.

  When detecting an application activation instruction by the operator, the CPU 10 reads the application program 12a from the storage unit 12 and executes it. As shown in FIG. 2, the CPU 10 functions as a communication state establishment unit 16, a screen information acquisition unit 17, an operation detection unit 18, and an operation information generation unit 19.

  The communication state establishing unit 16 is a processing unit that establishes a communication state with the image processing apparatus 3 via the communication control unit 13 based on an instruction from the operator. The communication state establishing unit 16 transmits a connection request to a connection destination input by an operator and receives a response signal permitting connection from the connection destination, thereby negotiating with the connection destination for data communication and performing communication. Establish state. When the communication state establishing unit 16 establishes a communication state with the image processing device 3, the communication state establishing unit 16 transmits panel information related to the screen size and display resolution of the display unit 14 mounted on the display panel 11 to the image processing device 3 in advance. Configured as follows.

  The screen information acquisition unit 17 is a processing unit that acquires the screen information transmitted from the image processing device 3 and displays it on the display unit 14 after the communication state establishment unit 16 establishes the communication state with the image processing device 3. is there. As a result, the same screen as that displayed on the operation panel 28 of the image processing apparatus 3 is displayed on the display unit 14 of the display panel 11. Each time the screen information acquisition unit 17 receives screen information from the image processing device 3, the screen information acquisition unit 17 updates the display screen in the display unit 14 based on the screen information.

  The operation detection unit 18 is a processing unit that detects an operation performed on the operation unit 15 in a state where the same display screen as the operation panel 28 of the image processing apparatus 3 is displayed on the display unit 14. The operation detection unit 18 can detect not only a single touch operation by an operator but also a multi-touch operation. The single touch operation is an operation in which the operator touches one point on the display screen of the display unit 14 with one fingertip. The multi-touch operation is an operation in which the operator touches two or more points on the display screen in the display unit 14 with two or more fingertips in one hand.

  The operation information generation unit 19 is a processing unit that generates operation information related to an operation performed by the operator based on an operation detected by the operation detection unit 18. The operation information generation unit 19 maps the touch position on the operation unit 15 on which the touch operation has been performed by the operator onto the pixel on the display unit 14 and generates the touch position coordinates on the display screen. The operation information generation unit 19 generates operation information including the touch position coordinates, and transmits the operation information to the image processing device 3 via the communication control unit 13. That is, in this embodiment, the communication control unit 13 functions as a transmission unit that transmits various types of information to the image processing apparatus 3.

  For example, when the operator performs a single touch operation with one fingertip, the operation information generation unit 19 generates operation information including the touch position coordinates of one point on the display screen of the display unit 14 single-touched by the operator. To the image processing apparatus 3. In the case of a single touch operation, an operator may touch a point on the display screen and then perform a flick operation to move the fingertip from the touch position. In such a case, the operation information generation unit 19 generates operation information including the touch position coordinates first touched by the operator and information indicating the movement direction of the fingertip from the touch position coordinates, and sends the operation information to the image processing apparatus 3. Send.

  When the operator performs a multi-touch operation with two fingertips, the operation information generation unit 19 generates operation information including the touch position coordinates of two points on the display screen of the display unit 14 multi-touched by the operator. To the image processing apparatus 3. In the case of multi-touch operation, with the operator touching two points on the display screen, a pinch-out operation is performed to increase the distance between the two points, or the distance between the two points is reduced. A pinch-in operation may be performed. In such a case, the operation information generation unit 19 obtains operation information including two touch position coordinates touched first by the operator and information indicating a movement direction in which each fingertip has moved from the two touch position coordinates. It is generated and transmitted to the image processing apparatus 3.

  With the configuration as described above, when the application program 12a is activated, the operator terminal device 2 transmits operation information indicating the operation content to the image processing device 3 every time an operation is performed by the operator.

  FIG. 3 is a block diagram illustrating an example of a hardware configuration and a functional configuration of the image processing apparatus 3. The image processing apparatus 3 includes a system control unit 20 that controls the overall operation of the apparatus, a panel control unit 27 that controls the operation panel 28, and a network interface 21 that is connected to the network 4 and communicates with other devices. A scanner unit 22 that reads a document, a printer unit 23 that performs print output, a FAX unit 24 that transmits and receives FAX data via a public telephone line, and an image data storage unit for storing image data 25 is connected to a bridge 29 for data exchange. Note that the user terminal device 26 in the present embodiment includes a panel control unit 27 and an operation panel 28.

  The system control unit 20 includes a CPU 20a, a RAM 20b, and a ROM 20c. When the CPU 20a reads and executes a program stored in advance in the ROM 20c when the image processing apparatus 3 is powered on, the system control unit 20 Control the overall operation. The RAM 20b stores temporary data generated when the CPU 20a executes a program. The system control unit 20 is configured to acquire information input to the operation panel 28 by the user using the image processing apparatus 3 from the panel control unit 27 and execute various processes. For example, when the system control unit 20 acquires user information such as a user ID and a password from the panel control unit 27, the system control unit 20 performs user authentication based on the user information. As a result, if it can be specified that the user operating the operation panel 28 is a registered regular user, the operation state of the image processing apparatus 3 is shifted from the logout state to the login state where the regular user has logged in. Further, when the system control unit 20 acquires information input by the user in the login state from the panel control unit 27, the network interface 21, the scanner unit 22, the printer unit 23, the FAX unit 24, and the image data storage are based on the information. Each part of the part 25 is controlled to operate a function designated by the user. For example, in the case of the copy function, the scanner unit 22 and the printer unit 23 are driven to reflect the copy job settings input by the user. In the case of the BOX function, the image data storage unit 25 is accessed to read out image data stored in a BOX (storage area) designated by the user. When a job execution instruction is issued by the login user, the system control unit 20 controls the execution of the job specified by the user by driving the above units as necessary.

  The panel control unit 27 controls the operation panel 28 serving as a user interface as described above. The panel control unit 27 includes a CPU 32, a RAM 33, and a ROM 34. The ROM 34 stores in advance various screen information for the user to perform various operations, and stores a program executed by the CPU 32 in advance. The CPU 32 functions as an operation information acquisition unit 35, a display position determination unit 36, a display control unit 37, and an operation detection unit 38 by reading and executing the program. The RAM 33 is for storing temporary data and the like when the CPU 32 performs various processes described later.

  The operation panel 28 includes a display unit 30 configured by a color liquid crystal display and the like, and an operation unit 31 configured by a touch panel key disposed on the display screen of the display unit 30. 3 is configured to display various operation screens to a user who uses the terminal 3 and to accept an operation input from the user. The display unit 30 in the present embodiment has a screen size and display resolution different from those of the display panel 11 in the operator terminal device 2. In particular, when the operator terminal device 2 is configured by a smartphone, the display unit 30 of the operation panel 28 has a larger screen size and display resolution than the operator terminal device 2. Further, the operation unit 31 can sense not only a single touch operation by a user but also a multi-touch operation.

  In the panel control unit 27, the display control unit 37 and the operation detection unit 38 always function in a normal state where the image processing apparatus 3 is powered on. At this time, the display control unit 37 causes the display unit 30 to display a screen operable by the user by reading screen information from the ROM 34 and outputting the screen information to the operation panel 28. The operation detection unit 38 detects a user operation performed on the operation unit 31 in a state where such a screen is displayed. When the operation detection unit 38 detects an operation by the user, the operation detection unit 38 outputs the detection information to the display control unit 37. Thereby, the display control part 37 changes the screen displayed on the display part 30 to another screen based on operation performed by the user. Further, when the operation detection unit 38 detects an operation by the user, the operation detection unit 38 outputs the detection information to the system control unit 20. Thereby, the system control unit 20 reflects the operation performed by the user on each unit of the image processing apparatus 3 or starts execution of the job.

  In addition, when the display control unit 37 displays a screen on which the user can perform a multi-touch operation on the display unit 30, the operation detection unit 38 detects the operation as a multi-touch operation when the user performs a multi-touch operation. . However, even in a state where a screen capable of multi-touch operation is displayed on the display unit 30, the user can perform a single touch operation. Therefore, when the user performs a single touch operation, the user's operation is detected as a single touch operation.

  On the other hand, when a screen on which the user cannot perform the multi-touch operation is displayed on the display unit 30 by the display control unit 37, the operation detection unit 38 does not detect the multi-touch operation by the user, and is all single-touch. Configured to detect as an operation. Therefore, the user who uses the image processing apparatus 3 may not know what multi-touch operation is possible in what screen state. You will receive instructions about the operation while making a call.

  The panel control unit 27 constantly monitors whether or not a connection request from the operator terminal device 2 has been received via the network interface 21 in a normal state where the image processing apparatus 3 is powered on. When the connection request is received, the panel control unit 27 transmits a response signal to the operator terminal device 2 to establish a communication state with the operator terminal device 2, and then the operation information acquisition unit 35 described above in the CPU 32. The display position determination unit 36, the display control unit 37, and the operation detection unit 38 are caused to function so as to perform operations in cooperation with each other.

  When the communication state with the operator terminal device 2 is established, the display control unit 37 outputs the screen information currently displayed on the display unit 30 to the network interface 21, and the screen information is transmitted to the operator terminal via the network interface 21. Transmit to device 2. That is, in this embodiment, the network interface 21 has a function of transmitting screen information to the image processing apparatus 3 in cooperation with the display control unit 37. As a result, the same screen as the full screen displayed on the operation panel 28 is displayed on the operator terminal device 2 in full screen. In addition, each time the display control unit 37 updates the screen of the display unit 30 thereafter, the display control unit 37 transmits screen information regarding the updated screen to the operator terminal device 2 via the network interface 21.

  The operation information acquisition unit 35 is a processing unit that acquires operation information transmitted from the operator terminal device 2 when the operator performs an operation on the display screen of the operator terminal device 2. The operation information acquired by the operation information acquisition unit 35 includes operation information related to a single touch operation or operation information related to a multi-touch operation. When the operation information acquisition unit 35 acquires the operation information from the operator terminal device 2, the operation information acquisition unit 35 outputs the operation information to the display position determination unit 36.

  The display position determination unit 36 analyzes the operation information acquired by the operation information acquisition unit 35 and determines the display position of the operation guidance image for prompting the user to perform the same operation as the operation performed by the operator on the operator terminal device 2. Part. As described above, the operation information transmitted from the operator terminal device 2 includes touch position coordinates that are positions on the screen where the operator performs a touch operation. On the other hand, since the display unit 30 of the operation panel 28 has a different screen size and resolution from the display panel 11 of the operator terminal device 2, the display unit 30 maintains the coordinate position as it is based on the touch position coordinates included in the operation information. If the operation guidance image is displayed, there is a problem that the operation guidance image that prompts the user to perform the touch operation is displayed at a position different from the position on the screen where the operator performs the touch operation. Therefore, the display position determining unit 36 performs coordinate conversion on the touch position coordinates included in the operation information based on the screen size ratio or the resolution ratio between the display panel 11 and the operation panel 28, so that the operation guidance is displayed on the display unit 30. The display position where the image is to be displayed is determined. A specific method for determining the display position will be described in detail later.

  When the display position determination unit 36 determines the display position of the operation guide image, the display control unit 37 displays the operation guide image at the display position on the display unit 30 and determines the position where the user should perform a touch operation. invite. At this time, the display control unit 37 displays the operation guidance image superimposed on the front side of the screen displayed on the full screen in the display unit 30. In this state, when the user touches the position where the operation guide image is displayed and performs the same operation as the operation performed by the operator, the operation detection unit 38 detects the operation, so that the image processing apparatus 3 is desired by the user. The operation result is reflected.

  In addition, when the operation information received from the operator terminal device 2 includes movement information indicating the movement direction, the display control unit 37 guides the direction in which the fingertip is moved while performing a touch operation on the operation panel 28. The operation guidance image for displaying is displayed on the display unit 30. Therefore, when the operator performs a pinch-in operation, a pinch-out operation, a flick operation, or the like starting from the first touch position, the same operation can be guided to the user.

  FIG. 4 is a diagram showing a communication sequence between the operator terminal device 2 and the image processing device 3. First, based on an inquiry from the user, the operator operates the operator terminal device 2 to start the application program 12a in the operator terminal device 2 (process P1). When the activation of the application is completed, the operator terminal device 2 transmits a connection request to the image processing device 3 based on an instruction from the operator, thereby establishing a communication state with the image processing device 3 being operated by the user. (Process P2). When the communication state with the image processing device 3 is established, the operator terminal device 2 reads information on the screen size and display resolution of the display panel 11, and transmits the information to the image processing device 3 as panel information D1 (process P3). ). The image processing apparatus 3 can specify the screen size and display resolution in the operator terminal apparatus 2 by acquiring and storing the panel information D1 in advance.

  When the advance preparation up to the process P3 is completed, the operator terminal device 2 and the image processing device 3 cooperate with each other, and a process of guiding the operation performed by the operator to the user is started. That is, the image processing apparatus 3 transmits screen information D2 displayed on the operation panel 28 to the operator terminal apparatus 2 (process P4), and the operator terminal apparatus 2 displays a screen based on the screen information D2 received from the image processing apparatus 3. Display is performed (process P5). As a result, the same screen as that displayed on the operation panel 28 of the image processing apparatus 3 is displayed on the operator terminal device 2 operated by the operator.

  FIG. 5 is a diagram illustrating an example of a state in which the same screen as the screen displayed on the display unit 30 of the operation panel 28 is displayed on the display unit 14 of the operator terminal device 2. As shown in FIG. 5A, when screen information D2 is transmitted from the image processing device 3 to the operator terminal device 2 in a state where a screen related to the BOX operation is displayed on the operation panel 28 of the image processing device 3, On the display unit 14 of the operator terminal device 2, the same screen as the operation panel 28 is displayed on the full screen as shown in FIG. At this time, since the screen size and resolution of the operator terminal device 2 are different from those of the operation panel 28, when the operator terminal device 2 receives the screen information D2, the screen reduction processing or enlargement processing or pixel thinning is performed as necessary. Processing, interpolation processing, and the like are performed, and the same screen as the operation panel 28 is displayed in full screen.

  However, the operator terminal device 2 does not necessarily have to display the same screen as the operation panel 28 in full screen, and may display the same screen as the operation panel 28 on a part of the display unit 14. For example, when the screen size and the display resolution of the operator terminal device 2 are larger than those of the operation panel 28, it is possible to perform display in accordance with the display size on the operation panel 28. Therefore, it is necessary to perform full screen display. Absent. When the screen size and display resolution of the operator terminal device 2 are smaller than those of the operation panel 28, it is preferable to display the same screen as that of the operation panel 28 as described above.

  Returning to FIG. 4, when the operator terminal device 2 displays the same screen as the operation panel 28 of the image processing device 3, the operator terminal device 2 detects an operation performed by the operator while the screen is displayed (process P <b> 6). When the operation by the operator is detected, the operator terminal device 2 generates operation information D3 and transmits it to the image processing device 3 (process P7).

  When receiving the operation information D2 from the operator terminal device 2, the image processing device 3 determines the display position of the operation guidance image for performing the operation guidance to the user (process P8), and on the display unit 30 of the operation panel 28. An operation guidance image is displayed at the display position (process P9). Thereafter, when an operation is performed by the user, the image processing apparatus 3 updates the screen displayed on the operation panel 28 based on the operation. Therefore, after the process P9, the processes of the processes P4 to P9 are repeatedly performed, and an operation guidance image based on the operation is displayed on the operation panel 28 every time the operation is performed by the operator.

  FIG. 6 is a diagram for explaining display position determination processing in the image processing apparatus 3 when the operation by the operator is a single touch operation. 6 illustrates a case where the coordinate origin of the touch position is set at the upper left corner of the screen in both the operator terminal device 2 and the operation panel 28. This coordinate origin is determined by the operator terminal device 2 and the operation panel 28. As long as it is set at a position common to each other on the panel 28, it is not necessarily limited to the upper left corner of the screen. FIG. 6 illustrates the case where the X axis is set in the vertical direction of the screen and the Y axis is set in the horizontal direction. The same applies to other drawings shown below.

  First, as shown in FIG. 6A, when the operator single-touches the position where the icon image C1 is displayed on the operator terminal device 2, the operator terminal device 2 touches the display screen of the single-touched position Pa. The position coordinates (Xa, Ya) are detected. Then, the operator terminal device 2 generates operation information including the touch position coordinates and transmits the operation information to the image processing device 3.

  When acquiring the operation information, the image processing device 3 determines a display position Pb for displaying the operation guide image G1. At this time, if the touch position coordinates (Xa, Ya) touched by the operator in the operator terminal device 2 are mapped as they are to the pixels in the display unit 30 of the operation panel 28, the screen size and resolution of the operation panel 28 are determined by the operator terminal. Since it is different from the device 2, the mapping is performed on the pixel at the position PD shown in FIG. However, since the pixel position PD is different from the pixel position at which the icon image C1 is displayed on the operation panel 28, if the operation guidance image G1 for guiding the user's touch operation is displayed at the pixel position PD, the operator can The user cannot accurately teach the user the operation of touching the icon image C1. Therefore, the image processing apparatus 3 performs the coordinate conversion of the touch position coordinates (Xa, Ya) based on the resolution ratio between the operator terminal apparatus 2 and the operation panel 28 in the display position determination unit 36 described above, thereby the icon touched by the operator. The coordinates (Xb, Yb) of the display position Pb for displaying the image C1 are calculated.

For example, the number of pixels in the X direction in the operator terminal device 2 is N (x), the number of pixels in the Y direction is N (y), and the number of pixels in the X direction on the display unit 30 of the operation panel 28 is M (x), Y When the number of pixels in the direction is M (y), the coordinates (Xb, Yb) of the display position Pb are
(Xb, Yb) = (Xa · M (x) / N (x), Ya · M (y) / N (y))
Is calculated by The same applies to the calculation based on the screen sizes in the X and Y directions.

  The display position determination unit 36 determines the display position Pb by performing the above calculation. In the image processing apparatus 3, the display control unit 37 functions to display the operation guidance image G1 at the display position Pb determined as described above. When displaying the operation guidance image G1, the display control unit 37 does not display with only one pixel at the display position Pb, but operates with a predetermined image size centered on the pixel at the display position Pb, for example. A guide image G1 is displayed. Therefore, as shown in FIG. 6B, the operation guidance image G1 is displayed so as to be superimposed on the position of the icon image C1 where the touch operation is performed by the operator. The user can perform the same operation as the single touch operation performed by the operator by touching the position where the operation guide image G1 is displayed.

  Next, a case where the operation by the operator is a multi-touch operation will be described. FIG. 7 is a diagram illustrating a concept of performing coordinate transformation similar to a single touch operation in the case of a multi-touch operation. In the present embodiment, for example, when preview display of an image stored in the image data storage unit 25 is being performed in the image processing apparatus 3, an operation for enlarging or reducing the image displayed in the preview is performed. In addition, multi-touch operation by the user becomes possible. In the example of FIG. 7, a case where the operator guides the user to such a multi-touch operation for the enlargement operation or reduction operation is shown.

  As shown in FIG. 7A, in the state where the preview display of the image data is being performed on the operator terminal device 2, the operator performs a multi-touch operation on the upper right and lower left of the screen in the display unit 14, and then 2 When a pinch-in operation that reduces the distance between the points is performed, the operator terminal device 2 detects such an operation by the operator. That is, the operator terminal device 2 detects the touch position coordinates (Xa1, Ya1), (Xa2, Ya2) on the display screen of the two initial positions Pa1 and Pa2 that are initially multi-touched, and the initial position. The moving direction in which two fingertips have moved from Pa1 and Pa2 is detected. Then, the operator terminal device 2 generates operation information including two touch position coordinates and information regarding the two movement directions, and transmits the operation information to the image processing device 3.

  When the image processing apparatus 3 acquires the operation information related to the multi-touch operation, the image processing device 3 determines display positions Pb1 and Pb2 for displaying the two operation guide images G1 for allowing the user to perform the multi-touch operation. At this time, if each of the two touch position coordinates included in the operation information is coordinate-transformed as in the case of the single touch operation, a display for guiding the multi-touch operation of two points as shown in FIG. 7B. The distance L between the positions Pb1 and Pb2 increases on the display unit 30 of the operation panel 28, and the operation guidance image may be displayed in a range that the user cannot reach with two fingertips of one hand.

  Therefore, in the present embodiment, when the operation information includes two points of touch position information associated with the multi-touch operation by the operator, the display position determination unit 36 determines a single display position when determining the display position of these two points. The display position is determined in a manner different from that in the case of the touch operation. 8 and 9 are diagrams for explaining the concept of display position determination processing in the image processing apparatus 3 when the operation by the operator is a multi-touch operation. FIG. 8 shows a state in which the operator is performing the multi-touch operation. FIG. 9 shows a case where a pinch-in operation is performed, and FIG. 9 shows a case where a pinch-out operation is performed while the operator is performing a multi-touch operation.

  First, as shown in FIG. 8A, in the state where the preview display of the image data is being performed on the operator terminal device 2, the operator performs a multi-touch operation on the upper right and lower left two points Pa1 and Pa2 on the display unit 14. Thereafter, when a pinch-in operation is performed to reduce the distance between the two points, the operator terminal device 2 transmits operation information based on such an operator operation to the image processing device 3.

  When acquiring such operation information, the image processing apparatus 3 calculates a physical distance L1 between the two points Pa1 and Pa2 multi-touched by the operator. That is, the display position determination unit 36 determines the two points Pa1 based on the touch position coordinates (Xa1, Ya1), (Xa2, Ya2) of the two points Pa1 and Pa2 included in the operation information and the pixel size in the operator terminal device 2. , Pa2 physical distance L1 is calculated. Then, as shown in FIG. 8B, the display position determination unit 36 sets two points at a position on the operation panel 28 that is substantially the same distance L2 as the physical distance L1 between the two points Pa1 and Pa2 multi-touched by the operator. Display positions Pb1 and Pb2 are determined. Based on the above determination, the image processing apparatus 3 displays the touch operation guidance image G1 that prompts the user to perform a touch operation on the two display positions Pb1 and Pb2 on the operation panel 28, and relates to the moving direction included in the operation information. Based on the information, a moving operation guidance image G2 for guiding the user to move in the same direction as the pinch-in operation performed by the operator is displayed. In the example shown in the figure, the movement operation guide image G2 is displayed as an arrow image indicating the movement direction, but the image is not limited to such an image.

  Next, as shown in FIG. 9A, in the state where the preview display of the image data is being performed on the operator terminal device 2, the operator performs multi-touch operation on the screen center two points Pa1 and Pa2 on the display unit 14, and thereafter When a pinch-out operation that increases the distance between two points is performed, the operator terminal device 2 transmits operation information based on such an operator operation to the image processing device 3.

  Even when such operation information is acquired, the image processing apparatus 3 calculates the physical distance L1 between the two points Pa1 and Pa2 multi-touched by the operator as described above. Then, as shown in FIG. 9B, the image processing apparatus 3 has two points at a position on the operation panel 28 that is substantially the same distance L2 as the physical distance L1 of the two points Pa1 and Pa2 multi-touched by the operator. Display positions Pb1 and Pb2 are determined. Further, based on the determination, the image processing apparatus 3 displays a touch operation guide image G1 that prompts the user to perform a touch operation on the two display positions Pb1 and Pb2 on the operation panel 28, and also includes a moving direction included in the operation information. On the basis of the information regarding the movement operation guidance image G2 for guiding the movement operation in the same direction as the pinch-out operation performed by the operator to the user.

  Thus, in the case of multi-touch operation, the user should first touch two points on the operation panel 28 at a distance L2 that is substantially the same distance L1 between the two points that the operator actually touched first. By displaying this touch operation guide image G1, it is possible to prevent the two touch operation guide images G1 from being displayed in a range where the user operating the operation panel 28 cannot operate with one hand. As a result, the user can perform the same operation as the operation performed by the operator with one hand, and can learn the operation on the operation panel 28 accurately.

  FIG. 10 is a diagram for explaining the concept of specific processing performed when the display position determination unit 36 determines the two display positions Pb1 and Pb2 in the case of the multi-touch operation. Here, as an example, as shown in FIG. 10A, a case where the operator touches two points Pa1 and Pa2 in the operator terminal device 2 will be described. The distance between the two points Pa1 and Pa2 touched by the operator in the operator terminal device 2 is L1. In this case, the display position determination unit 36 sets a reference point on the operation panel 28 based on the two touch position coordinates (Xa1, Ya1) and (Xa2, Ya2) included in the operation information acquired from the operator terminal device 2. Set one. This reference point is one of the two touch position coordinates (Xa1, Ya1), (Xa2, Ya2), or between the two touch position coordinates (Xa1, Ya1), (Xa2, Ya2). An arbitrary point located on the line segment is assumed to be a point obtained by coordinate conversion similar to that in the case of the single touch operation described above.

  In FIG. 10B, the two touch position coordinates (Xa1, Ya1) and (Xa2, Ya2) included in the operation information and the midpoint of these two points are coordinate-converted into coordinate positions on the operation panel 28. This is an example showing points Pb1 ′, Pb2 ′, and PC. Among these points, the point PC is a midpoint on the line segment L connecting the points Pb1 ′ and Pb ′. In the present embodiment, for example, when the midpoint PC is set as the reference point, the display position determining unit 36 is on the line segment L connecting the points Pb1 ′ and Pb ′ as shown in FIG. , Two coordinate positions b1 and b2 that are the same distance as the physical distance L1 between the two points Pa1 and Pa2 touched by the operator are calculated. However, the coordinate positions b1 and b2 may be out of the pixel positions on the display unit 30 of the operation panel 28. Therefore, the display position determination unit 36 maps each of the two coordinate positions b1 and b2 calculated as described above to the nearest pixel in the display unit 30, and determines the two display positions Pb1 and Pb2. . With such mapping, the interval between the two display positions Pb1 and Pb2 may not match the distance L1, but the operation panel 28 is operated in order to maintain the distance L2 that is substantially the same as the distance L1. It has no effect on the user.

  Further, as described above, the display position determination unit 36 has the same distance as the physical distance L1 between the two points Pa1 and Pa2 touched by the operator on the line segment L connecting the points Pb1 ′ and Pb2 ′. By calculating the two coordinate positions b1 and b2, the two display positions Pb1 and Pb2 on the operation panel 28 can be determined while maintaining the relative positional relationship between the two points touched by the operator. It is like that.

  In FIG. 10, when the display position determination unit 36 determines the two display positions Pb1 and Pb2, a process for determining a position substantially the same as the distance L1 between the two points actually touched by the operator. And is not necessarily limited to the processing described above.

  FIG. 11 is a diagram illustrating a processing concept for determining two display positions Pb1 and Pb2 by a method different from that in FIG. Here, as an example, as shown in FIG. 11A, a case where the operator touches two points Pa1 and Pa2 in the operator terminal device 2 will be described. In the operator terminal device 2, the physical distance between the two points Pa1 and Pa2 touched by the operator is L1.

  The display position determination unit 36 maps the two touch position coordinates (Xa1, Ya1), (Xa2, Ya2) included in the operation information acquired from the operator terminal device 2 to the display coordinates of the operation panel 28 as they are. As shown in FIG. 11B, two points Pb1 ′ and Pb2 ′ on the operation panel 28 are determined. At this time, since the pixel size and the pixel pitch in the display unit 30 of the operation panel 28 are different from the pixel size and the pixel pitch in the operator terminal device 2, the distance between the two points Pb1 ′ and Pb2 ′ is substantially the same as the distance L1. L2. The display position determination unit 36 selects one of the two points Pb1 ′ and Pb2 ′, and performs coordinate conversion on the coordinate value of the selected one point in the same manner as in the case of the single touch operation.

  FIG. 11B illustrates a case where one of the two points Pb1 ′ and Pb2 ′ is selected, and one display position Pb1 is determined by coordinate conversion of the point Pb1 ′. Then, the display position determination unit 36 calculates an offset amount ΔP = (ΔX, ΔY) based on the difference between the display position Pb1 and the point Pb1 ′. Next, the display position determination unit 36 determines the other display position Pb2 by adding the offset amount ΔP to the coordinate value of the other point Pb2 ′ that has not undergone coordinate conversion. That is, the display position Pb2 corresponding to the other point Pb2 ′ is the same as the amount of movement from the one point Pb1 ′ to the display position Pb1, with the display position Pb1 obtained by converting the coordinate of the one point Pb1 ′ as a reference. Only by translating the other point Pb2 ′. As a result, the two display positions Pb1 and Pb2 are determined to be positions that maintain a distance L2 that is substantially the same as the distance L1.

  On the other hand, FIG. 11C illustrates the case where one of the two points Pb1 ′ and Pb2 ′ is selected, and one display position Pb2 is determined by coordinate conversion of the point Pb2 ′. To do. Also in this case, the display position determination unit 36 calculates the offset amount ΔP = (ΔX, ΔY) from the difference between the display position Pb2 and the point Pb2 ′, as described above. Next, the display position determination unit 36 determines the other display position Pb1 by adding the offset amount ΔP to the coordinate value of the other point Pb1 ′ that has not been subjected to coordinate conversion. As a result, the two display positions Pb1 and Pb2 are determined to be positions that maintain a distance L2 that is substantially the same as the distance L1.

  11B and 11C, one of the two points Pb1 ′ and Pb2 ′ is selected to perform coordinate conversion, and the other point is set with the point obtained by the coordinate conversion as a reference point. The display position corresponding to is determined as a position having a distance L2 substantially the same as the distance L1. However, as shown in FIGS. 11B and 11C, when one of the two points Pb1 ′ and Pb2 ′ is selected and the reference point is set, the user is guided to the multi-touch operation. There is a possibility that the operation guidance image G1 will be displayed biased to the screen edge of the operation panel 28. In order to prevent this, as shown in FIG. 12, it is preferable to set the midpoint between the two points Pb1 ′ and Pb2 ′ as a reference point.

  FIG. 12 is a diagram illustrating a processing concept for determining the display positions Pb1 and Pb2 with reference to the midpoint of the two points. Here, as an example, as shown in FIG. 12A, a case where the operator touches two points Pa1 and Pa2 in the operator terminal device 2 will be described. In the operator terminal device 2, the physical distance between the two points Pa1 and Pa2 touched by the operator is L1.

  The display position determination unit 36 maps the two touch position coordinates (Xa1, Ya1), (Xa2, Ya2) included in the operation information acquired from the operator terminal device 2 to the display coordinates of the operation panel 28 as they are. As shown in FIG. 12B, the coordinate value of the midpoint PC ′ of these two points Pb1 ′ and Pb2 ′ is calculated. Then, the display position determination unit 36 converts the coordinate value of the midpoint PC ′ as in the case of the single touch operation, and obtains the coordinate value of the reference point PC. The display position determination unit 36 calculates the offset amount ΔP = (ΔX, ΔY) from the difference between the midpoint PC ′ and the reference point PC, and the offset amount ΔP for each of the coordinate values of the two points Pb1 ′ and Pb2 ′. Are added to determine two display positions Pb1 and Pb2.

  As shown in FIG. 12, by setting the coordinates of the midpoint as the reference point and determining the two display positions Pb1 and Pb2, the two display positions Pb1 and Pb2 are determined approximately at the center of the screen on the operation panel 28. The Therefore, the operation guidance image G1 for guiding the multi-touch operation to the user can be prevented from being biased and displayed on the screen end portion of the operation panel 28, which is a more preferable display mode.

  Next, specific processing procedures performed in each of the operator terminal device 2 and the image processing device 3 will be described.

  FIG. 13 is a flowchart illustrating an example of a processing procedure performed in the operator terminal device 2. This process is started when the application program 12a is activated in the operator terminal device 2. When the application is started, the operator terminal device 2 transmits a connection request to the image processing device 3 designated as a connection destination by the operator (step S10). When the operator terminal device 2 receives the response signal from the image processing device 3 and confirms that the communication state has been established (YES in step S11), the operator terminal device 2 transmits the panel information D1 in the operator terminal device 2 to the image processing device 3. (Step S12).

  Thereafter, when the operator terminal device 2 receives the screen information D2 from the image processing device 3 (YES in step S13), the operator terminal device 2 performs screen display on the display panel 11 based on the screen information D2 (step S14). If the screen information D2 has not been received (NO in step S13), the process in step S14 is skipped.

  Subsequently, when the operator terminal device 2 detects that the operation by the operator is started (YES in step S15), the operator terminal device 2 records the operation content by the operator until the operation is completed (steps S16 and S17). When the operator removes the fingertip from the display panel 11 and detects that the touch operation is completed (YES in step S17), the operator terminal device 2 operates the operation information D3 indicating the operation performed by the operator based on the recorded information. (Step S18), and the operation information D3 is transmitted to the image processing apparatus 3 (step S19). If no operation by the operator is detected in step S15 (NO in step S15), the processes in steps S16 to S19 are skipped.

  Then, it is determined whether or not an end instruction is issued by the operator (step S20). If there is no end instruction (NO in step S20), the process returns to step S13 and the above-described processing is repeated. Thereby, whenever the operator terminal device 2 receives the screen information D2 from the image processing device 3, the operator terminal device 2 updates the display screen of the display panel 11 based on the screen information D2. The operator terminal device 2 performs processing for generating operation information D3 and transmitting it to the image processing device 3 every time an operation by the operator is detected.

  On the other hand, when an end instruction is detected by the operator (YES in step S20), the operator terminal device 2 transmits an end command to the image processing device 3 (step S21), and disconnects communication with the image processing device 3. To finish the process.

  Next, FIG. 14 and FIG. 15 are flowcharts showing an example of processing procedures performed in the image processing apparatus 3. This process is started when the image processing apparatus 3 receives a connection request transmitted from the operator terminal apparatus 2. When receiving the connection request transmitted from the operator terminal device 2, the image processing device 3 performs a response process on the operator terminal device 2 and establishes a communication state with the operator terminal device 2 (step S30). And it waits until the panel information D1 transmitted from the operator terminal device 2 is received (step S31). When the panel information D1 is received, the panel information D1 is temporarily stored.

  Thereafter, the image processing device 3 transmits the screen information D2 displayed on the operation panel 28 to the operator terminal device 2 (step S32).

  Subsequently, when receiving the operation information D3 from the operator terminal device 2 (YES in step S33), the image processing device 3 performs display position determination processing for displaying the touch operation guidance image G1 (step S34), and touch operation guidance. Each of the image G1 and the moving operation guide image G2 is generated as necessary (step S35), and the generated operation guide images G1 and G2 are displayed at the determined display positions on the operation panel 28 (step S36). . As a result, the operation guidance images G1 and G2 based on the operation performed by the operator are displayed superimposed on the display screen on the operation panel 28, and the operation to be performed by the user on the current display screen is guided. Will come to be. If the operation information D3 has not been received from the operator terminal device 2 (NO in step S33), the processes in steps S34 to S36 are skipped.

  In addition, when detecting an operation on the operation panel 28 by the user (YES in step S37), the image processing apparatus 3 updates the screen displayed on the operation panel 28 based on the operation (step S38). If no user operation is detected (NO in step S37), the process in step S38 is skipped.

  Further, the image processing apparatus 3 determines whether or not an end command has been received from the operator terminal apparatus 2 (step S39). If no end command has been received, whether the screen of the operation panel 28 has been further updated. It is determined whether or not (step S40). As a result, if the end command has not been received and the screen has been updated (YES in step S40), the process returns to step S32 and the above-described processing is repeated. On the other hand, if the end command has not been received and the screen has not been updated (NO in step S40), the process returns to step S33 and the above-described processing is repeated. Therefore, the image processing device 3 transmits the screen information D2 to the operator terminal device 2 every time the display screen of the operation panel 28 is updated based on the operation by the user. Further, each time the operation information D3 is received from the operator terminal device 2, the image processing device 3 determines the display position for displaying the operation guide images G1 and G2, and the operation guide image G1 is within a range that the user can operate with one hand. , G2 are displayed.

  On the other hand, when receiving an end command from the operator terminal device 2 (YES in step S39), the image processing device 3 disconnects communication with the operator terminal device 2 (step S41) and displays the operation guidance images G1 and G2. To complete the process.

  FIG. 15 is a flowchart illustrating an example of a detailed processing procedure of the display position determination process (step S34). When starting this processing, the image processing apparatus 3 extracts touch position coordinates included in the operation information D3 (step S50), and performs coordinate conversion of the touch position coordinates (step S51). Then, the image processing apparatus 3 determines whether or not the operation performed by the operator is a multi-touch operation (step S52). This determination is made by determining whether the touch position coordinates included in the operation information D3 are one point or two points or more, and if it is one point, it becomes a single touch operation. Touch operation. When the operation by the operator is a single touch operation (NO in step S52), the image processing device 3 determines a coordinate value obtained by coordinate conversion of the touch position coordinates as a display position (step S53), and a display position determination process Exit.

  On the other hand, when the operation by the operator is a multi-touch operation (YES in step S52), the image processing apparatus 3 calculates a physical distance L1 between two points on the display panel 11 actually touched by the operator (step S54). ), An arbitrary one point is selected as a reference point from the two line segments transformed in step S51 (step S55). Then, the image processing apparatus 3 calculates the coordinate values of the two points that are the same distance as the distance L1 on the two line segments after the coordinate conversion based on the selected reference point (step S56), and the coordinate values of these two points Is mapped to the nearest display pixel on the operation panel 28 to determine two display positions (step S57). Therefore, in the case of a multi-touch operation, two display positions that are substantially the same distance L2 as the distance L1 between the two points actually touched by the operator are determined.

  As described above, the image processing apparatus 3 of the present embodiment includes the user terminal device 26 that guides the user to the operation performed on the operator terminal device 2 by communicating with the operator terminal device 2. Yes. The user terminal device 26 includes a display unit 30 having a different screen size and display resolution from the display panel 11 provided in the operator terminal device 2, and operation information including touch position coordinates from the operator terminal device 2 to the display panel 11. The operation information acquisition unit 35 for acquiring D3 and the display unit 30 corresponding to the position touched on the display panel 11 by converting the touch position coordinates included in the operation information D3 according to the resolution of the display unit 30. Display position determining unit 36 for determining the display position of the display, and display control for displaying operation guide images G1 and G2 for guiding operations performed on the display panel 11 at the display position determined by the display position determining unit 36. And a unit 37. In particular, in the present embodiment, as described above, the display position determination unit 36, when the operation information D3 includes two touch position coordinates associated with two touches on the display panel 11 at the same time, A physical distance L1 between two points actually touched on the display panel 11 on the basis of a position on the display unit 30 corresponding to one of the two points or one arbitrary point between the two points. The display positions of these two points are determined at substantially the same position.

  Therefore, even when the operator performs a multi-touch operation using the operator terminal device 2 having a small screen size, such as a smartphone, the display unit 30 of the user terminal device 26 operated by the user has two points touched simultaneously by the operator. The operation guide images G1 and G2 are displayed at substantially the same distance as the distance L1. That is, in the present embodiment, it is possible to satisfactorily prevent the multi-touch operation performed by the operator from guiding the operation in a range where the user cannot operate with one hand, and therefore, for the user, the same as the multi-touch operation performed by the operator. The operation can be performed normally, and the operation of the user terminal device 26 can be learned efficiently.

  Further, in the present embodiment, as the operator performs a pinch-in operation or a pinch-out operation while performing a multi-touch operation, the operation information D3 received by the user terminal device 26 has 2 touch position coordinates as the start point. When the movement information regarding the movement of the points is included, when the display control unit 37 displays the operation guidance images G1 and G2, the touch operation guidance image G1 is displayed at the two display positions determined by the display position determination unit 36. In addition to displaying, based on the movement information included in the operation information D3, a movement operation guidance image G2 for guiding the movement directions of two points from the display position of the touch operation guidance image G1 is displayed. Therefore, when the operator performs a pinch-in operation or a focus-out operation, an operation similar to the operation performed by the operator within a range that can be operated by the user can be guided to the user.

  In the present embodiment, in the case of a multi-touch operation, when the display position determination unit 36 determines two display positions based on the two touch position coordinates, the relative positional relationship between the two points is maintained. The display positions of the two points on the display unit 30 are determined. Therefore, the operation guide images G1 and G2 can be displayed faithfully with respect to the operation performed by the operator, and there is an advantage that the user can accurately grasp the operation performed by the operator.

  In the present embodiment, the panel information D1 of the display panel 11 is transmitted in advance from the operator terminal device 2 to the image processing device 3 at the timing when the communication state between the operator terminal device 2 and the image processing device 3 is established. In addition, the configuration has been described in which when the image processing device 3 receives the operation information D3 from the operator terminal device 2, the coordinate conversion is performed based on the panel information D1 acquired in advance from the operator terminal device 2, and the display position is determined. However, the present invention is not limited to this, and the processing for determining the display position for displaying the operation guidance images G1 and G2 on the operation panel 28 of the image processing device 3 may be performed by the operator terminal device 2. In this case, for example, at the timing when the communication state between the operator terminal device 2 and the image processing device 3 is established, the panel information of the operation panel 28 is transmitted in advance from the image processing device 3 to the operator terminal device 2 contrary to the above. Keep it. When the operator terminal device 2 detects the operation of the operator, the operator terminal device 2 displays the operation guidance images G1 and G2 on the image processing device 3 based on the panel information acquired in advance from the image processing device 3. The display position may be determined, and the display position may be transmitted to the image processing apparatus 3 as the operation information D3. That is, in this case, the display position determination unit 36 described above is provided in the operator terminal device 2.

(Second Embodiment)
Next, a second embodiment will be described. In the first embodiment described above, when the operation performed by the operator is a multi-touch operation, the operation guide image G1, so that the distance L2 is substantially the same as the distance L1 between the two points actually touched by the operator. An example in which the display positions of two points for displaying G2 are determined is illustrated. In the present embodiment, each user who uses the image processing apparatus 3 makes settings related to his / her touch width in advance with respect to the image processing apparatus 3, and when the operator performs a multi-touch operation, the operation panel 28. 3 illustrates a form in which the distance L2 between two points for displaying the touch operation guidance image G1 is displayed at a position where the touch width is set in advance by the user. Also in this embodiment, the detailed configurations of the information processing system 1, the operator terminal device 2, and the image processing device 3 are the same as those in the first embodiment.

  FIG. 16 is a diagram illustrating an example of a screen displayed on the display unit 30 of the operation panel 28 when the user of the image processing apparatus 3 performs settings related to his / her touch width. In this screen example, there are two setting items, touch width setting and dominant arm setting. In the touch width setting, as selection items that can be alternatively selected by the user, there are a plurality of items of the same distance, an adult male, an adult female, a numerical value input, and a touch input. The touch width setting may include other selection items such as children. Also, in the dominant arm setting, there are two items, right-handed and left-handed, as selection items that can be alternatively selected by the user. For example, in the touch width setting, the default value is a state in which the same distance is selected in advance, and in the dominant arm setting, the right handed is a default value in a state where the distance is preselected. When the touch width setting is set to the same distance, when the operator performs a multi-touch operation, as described in the first embodiment, the distance approximately the same as the distance L1 between the two points actually touched by the operator Two display positions are determined so as to be L2, and operation guide images G1 and G2 are displayed at the two display positions.

  Each user can change one or both of the touch width setting and the dominant arm setting to a desired setting by logging in to the image processing apparatus 3 and displaying a screen as shown in FIG. 16 on the operation panel 28. it can. In the example of FIG. 16, a user A who is a login user changes the touch width setting from the default value and selects an adult male. When the user selects an adult man or an adult woman, when the operator performs a multi-touch operation in a state where the image processing device 3 communicates with the operator terminal device 2 as in the first embodiment, Depending on whether the subsequent operation is a pinch-in operation or a pinch-out operation, the touch operation guide image G1 is displayed at a position having a touch width set in advance for an adult man or an adult woman.

  Each user can select a numerical value input and set the desired touch width as a numerical value in the numerical value input field on the right side when setting the desired touch width. At this time, the user can distinguish and input the touch width when performing the pinch-in operation and the touch width when performing the pinch-out operation. Furthermore, when each user actually performs a multi-touch operation and sets the touch width, the user selects a touch input and directly sets the desired touch width by performing a multi-touch operation on the screen. Is also possible. Also at this time, the user can distinguish and set the touch width when performing a pinch-in operation and the touch width when performing a pinch-out operation.

  Furthermore, when each user is left-handed, the user can change the right-handed setting, which is the default value, to the left-handed setting by selecting left-handed in the dominant-hand setting. When the user selects left-handed, when the right-handed operator performs a multi-touch operation while the image processing device 3 is communicating with the operator terminal device 2, an operation guidance for guiding the multi-touch operation The images G1 and G2 are displayed on the operation panel 28 at positions that are horizontally reversed.

  FIG. 17 is a diagram showing information stored in advance in the image processing apparatus 3 (more strictly speaking, the user terminal device 26) of the present embodiment. FIG. 17A shows an example of touch width setting information 41 in which the touch width set by each user is registered. The touch width setting information 41 is information in which each user, the touch width setting set by each user, and the dominant arm setting set by each user are associated with each other. Therefore, when each user logs in to the image processing apparatus 3, the touch width setting and dominant arm setting associated with the logged-in user can be read.

  On the other hand, FIG. 17B shows the touch width reference information 42 registered in advance for an adult male or an adult female. In the touch width reference information 42, a setting value selected by a user such as an adult male or an adult female, the first touch width in a pinch-in operation, and the first touch width in a pinch-out operation are associated with each other. Information. Therefore, when the user who has logged into the image processing apparatus 3 has previously set an adult male, an adult female, or the like as the touch width setting, the operation panel 28 can be used during multi-touch operation by referring to the touch width reference information 42. The operation guide images G1 and G2 can be displayed by determining the distance between the two points to be displayed.

  The touch width setting information 41 and the touch width reference information 42 as described above are stored and saved in advance in, for example, the RAM 33 provided in the panel control unit 27 or a non-illustrated nonvolatile RAM.

  In this embodiment, when the operator terminal device 2 and the image processing device 3 communicate with each other, and the image processing device 3 receives the operation information D3 related to the multi-touch operation from the operator terminal device 2, the display position determination unit 36 Is configured to determine a touch width corresponding to the logged-in user based on the touch width setting information 41 and the touch width reference information 42 as described above, and to determine a display position for displaying the operation guide images G1 and G2. Is done. In this embodiment, the main processing procedure performed in the image processing apparatus 3 is the same as that shown in FIG. 14, but the details of the display position determination process (step S34) are different from those in the first embodiment. It will be a thing.

  FIG. 18 is a flowchart illustrating an example of a detailed processing procedure of the display position determination process performed by the image processing apparatus 3 in the present embodiment. Note that steps S60 to S63 in this flowchart are the same as steps S50 to S53 in the flowchart shown in FIG.

  When the operation information D3 received from the operator terminal device 2 is information related to the multi-touch operation (YES in step S62), the image processing device 3 reads the touch width setting information 41 and the touch width reference information 42 described above from the RAM 33, The touch width corresponding to the login user is determined (step S64). At this time, if the login user sets the touch width setting to the default value, the physical distance L1 between the two points actually touched by the operator is determined as the touch width as in the first embodiment. The On the other hand, if the login user has changed the touch width setting from the default value, the touch width is determined based on the contents of the setting change. In this case, if the operator is performing a pinch-in operation, the touch width corresponding to the pinch-in operation is determined, and if the operator is performing a pinch-out operation, the touch width corresponding to the pinch-out operation is determined.

  The image processing apparatus 3 selects one arbitrary point as a reference point from the two line segments whose coordinates are converted in step S61 (step S65), and the two line segments whose coordinates are converted based on the selected reference point. The coordinate values of the two points having the same distance as the touch width are calculated (step S66). Next, the image processing apparatus 3 determines whether or not the login user is left-handed (step S67). If the logged-in user is left-handed, the left and right coordinate values calculated in step S66 are reversed left and right (step S68). . As a result of this horizontal reversal, the coordinate values of the two points calculated in step S66 are coordinate-converted to positions that are horizontally reversed on the display unit 30 of the operation panel 28. If the logged-in user is not left-handed, the process skips without performing the left-right reversing process in step S68. The image processing apparatus 3 maps the two coordinate values obtained as described above to the nearest display pixel on the display unit 30 of the operation panel 28, and displays the two points for displaying the touch operation guide image G1. The display position is determined (step S69).

  Therefore, in this embodiment, the operation information D3 received by the image processing device 3 from the operator terminal device 2 includes two touch position coordinates indicating that the operator touched two locations on the display panel 11 at the same time. In this case, the distance between the two points on the operation panel 28 is set in advance with reference to the position on the display unit 30 corresponding to one of the two points or one arbitrary point between the two points. Two display positions for displaying the touch operation guide image G1 are determined at the position (touch width). The present embodiment is the same as the first embodiment except for the points described above.

  As described above, in the image processing device 3 of the present embodiment, the operation information D3 includes two touch position coordinates indicating that the operator touched two locations on the display panel 11 at the operator terminal device 2 at the same time. When the two operation guidance images G1 for guiding the two touch positions to the user are displayed on the operation panel 28, the distance between the two operation guidance images G1 is set to a distance set in advance by the user. The display position of one operation guidance image G1 is determined.

  Therefore, when the operator performs a multi-touch operation using the operator terminal device 2, two operation guidance images are displayed on the display unit 30 of the operation panel 28 operated by the user at intervals set in advance by the user. G1 and G2 are displayed. As a result, in this embodiment, as in the first embodiment, the multi-touch operation performed by the operator can be well prevented from being guided by the user in a range where the user cannot operate with one hand. Can normally perform the same operation as the multi-touch operation performed by the operator, and can learn the operation on the operation panel 28 efficiently.

  Further, in this embodiment, when the display position of two points is determined at a position having a preset distance (touch width) in the image processing apparatus 3, the movement information included in the operation information D3 brings the two points close to each other. If it is information (information related to the pinch-in operation), two display positions are determined at positions corresponding to the first distance corresponding to the pinch-in operation preset by the user. In addition, if the movement information included in the operation information D3 is information that separates two points from each other (information related to pinch out), the movement information is set to a position that is a second distance corresponding to the pinch out operation preset by the user. Determine the display position of the point. This second distance is set in advance as a distance shorter than the first distance.

  Therefore, when the operator performs a pinch-in operation or a pinch-out operation by a multi-touch operation on the operator terminal device 2, the user can also perform a pinch-in operation by a multi-touch operation on the display unit 30 of the operation panel 28 operated by the user. Two operation guidance images G1 and G2 are displayed at intervals at which a pinch-out operation can be performed.

  In this embodiment, when left-handed information is set in advance in association with the user who operates the operation panel 28, the image processing apparatus 3 determines the two points based on the two touch position coordinates included in the operation information D3. The display position of the point on the operation panel 28 is determined by reversing the relative positional relationship between the left and right. Therefore, if the user is left-handed, the operation guidance images G1 and G2 for guiding the multi-touch operation performed by the operator can be displayed in the left-right reversed position for left-handed users. Has an advantage that the operation becomes easier.

  In the above description, the example in which the display position of the operation guidance images G1 and G2 is determined in the image processing device 3 when the image processing device 3 receives the operation information D3 from the operator terminal device 2 has been described. However, the present invention is not limited to such a configuration example, and the present embodiment may be configured to determine the display positions of the operation guidance images G1 and G2 in the operator terminal device 2 as in the first embodiment. In this case, the operator terminal device 2 acquires information on the touch width preset by the user from the image processing device 3, and determines two display positions based on the acquired touch width.

(Modification)
As mentioned above, although embodiment regarding this invention was described, this invention is not restricted to the content mentioned above, A various modification is applicable.

  For example, in the above, the case where the user terminal device 26 is mounted on the image processing device 3 is illustrated. However, since the user terminal device 26 is only required to be able to communicate with the operator terminal device 2 and to detect a multi-touch operation, the user terminal device 26 is not mounted on the image processing device 3 and is independent of the other. Electronic equipment may be used. For example, the user terminal device 26 may be a smartphone or a doublet terminal.

  In the above, when the operator performs a pinch-in operation or a pinch-out operation on the operator terminal device 2, the same pinch-in operation or pinch-out operation is guided to the user on the display unit 30 of the operation panel 28. Therefore, an example in which the movement operation guide image G2 is displayed in an aspect such as an arrow indicating the movement direction has been described. Such a movement operation guidance image G2 may be displayed in a different manner depending on, for example, the movement amount or movement speed of the fingertip moved by the operator.

DESCRIPTION OF SYMBOLS 1 Information processing system 2 Operator terminal device 3 Image processing apparatus 11 Display panel 26 User terminal device 28 Operation panel 30 Display part (display means)
35 Operation information acquisition unit (operation information acquisition means)
36 Display position determining unit (display position determining means)
37 Display control unit (display control means)
38 Operation detection unit (operation detection means)

Claims (11)

A user terminal device capable of communicating with an operator terminal device having a display panel capable of multi-touch operation,
Display means having a screen size different from that of the display panel;
Operation detecting means for detecting an operation on the display means;
Operation information acquisition means for acquiring operation information including touch position coordinates on the display panel from the operator terminal device, as an operation on the display panel is detected in the operator terminal device;
Display position determination means for determining a display position on the display means corresponding to the touched position on the display panel by converting touch position coordinates included in the operation information according to the resolution of the display means;
Display control means for displaying an operation guidance image for guiding operations performed on the display panel at a display position determined by the display position determination means;
With
When the operation information includes two touch position coordinates associated with touching two places on the display panel at the same time, the display position determining means may select one of the two points or the two With the position on the display means corresponding to an arbitrary point between the points as a reference, the display is displayed at a position that is substantially the same as the physical distance between the two points on the display panel or at a preset distance. A user terminal device characterized by determining display positions of the two points in the means.   The display control means touches the two display positions determined by the display position determination means when the operation information includes movement information regarding the movement of the two points starting from the two touch position coordinates. An operation guidance image is displayed, and a movement operation guidance image for guiding a moving direction of two points from a display position of the touch operation guidance image is displayed based on movement information included in the operation information. Item 4. The user terminal device according to Item 1.   2. The display position determination means determines two display positions on the display means based on the touch position coordinates of the two points while maintaining a relative positional relationship between the two points. Or the user terminal device of 2.   When the display position determining means determines the display position of the two points on the display means at a position that is a preset distance, the movement information included in the operation information causes the two points to approach each other. If so, the display position of the two points is determined at a position that becomes a preset first distance, and if the movement information included in the operation information is information that separates the two points from each other, 3. The user terminal device according to claim 2, wherein the display positions of the two points are determined at a position that becomes a second distance preset in advance near the first distance.   When the left-handed information is set in advance, the display position determining means reverses the relative position relationship between the two points based on the touch position coordinates of the two points, and determines the display positions of the two points on the display means. The user terminal apparatus according to claim 1, wherein the user terminal apparatus is determined. An image processing apparatus capable of communicating with an operator terminal device including a display panel capable of multi-touch operation,
Display means having a larger screen size than the display panel;
Operation detecting means for detecting an operation on the display means;
Screen data transmission means for displaying the same display screen on the display panel of the operator terminal device by transmitting screen information relating to the display screen of the display means to the operator terminal device;
The touch position coordinates on the display panel are detected from the operator terminal device when an operation on the display panel is detected while the same display screen as the display unit is displayed on the display panel of the operator terminal device. Operation information acquisition means for acquiring operation information including:
Display position determination means for determining a display position on the display means corresponding to the touched position on the display panel by converting the touch position coordinates included in the operation information according to the resolution of the display means;
On the display screen displayed by the display means, a display further displaying an operation guidance image for guiding operations performed on the display panel at the display position determined by the display position determining means on the display means. Control means;
With
When the operation information includes two touch position coordinates associated with touching two places on the display panel at the same time, the display position determining means may select one of the two points or the two With the position on the display unit corresponding to an arbitrary point between the points as a reference, the display unit is located at a position that is substantially the same as the physical distance between the two points on the display panel or a preset distance. An image processing apparatus for determining a display position of the two points. An operator terminal device capable of communicating with a user terminal device comprising display means for displaying various screens,
A display panel having a screen size different from that of the display means;
Operation detecting means for detecting an operation on the display panel;
Storage means for storing resolution information relating to the resolution of the display means;
As the operation on the display panel is detected by the operation detection means, touch position coordinates on the display panel are detected, and the touch position coordinates are converted on the basis of the resolution information. Display position determining means for determining a display position on the display means corresponding to the position made;
Transmitting means for transmitting display position information based on the display position determined by the display position determining means to the user terminal device;
With
The display position determining means, when two touch position coordinates are detected by the operation detecting means when two places on the display panel are touched simultaneously, one of the two points, or the point A position that is substantially the same as the physical distance between the two touched points on the display panel or a preset distance with reference to the position on the display means corresponding to any one point between the two points And determining the display positions of the two points on the display means.   The transmission means transmits movement information indicating a movement direction of the two points to the user terminal device when movement of the two touch position coordinates is detected by the operation detection means. The operator terminal device according to 7. An operator terminal device having a display panel that can be operated by an operator and a user terminal device having a display means that can be operated by a user are information processing systems that can communicate with each other,
Operation information generating means provided in the operator terminal device, and generating operation information including touch position coordinates on the display panel when an operation of the operator on the display panel is detected;
The touch position coordinate provided in the operator terminal device or the user terminal device and included in the operation information is converted in accordance with the resolution of the display means in the user terminal device, so that the touched position on the display panel is obtained. Display position determining means for determining a display position in the corresponding display means;
Display control means provided in the user terminal device, for displaying an operation guidance image for guiding operations performed on the display panel at a display position determined by the display position determination means;
With
When the operation information includes two touch position coordinates associated with touching two places on the display panel at the same time, the display position determining means may select one of the two points or the two With the position on the display unit corresponding to an arbitrary point between the points as a reference, the display unit is located at a position that is substantially the same as the physical distance between the two points on the display panel or a preset distance. An information processing system characterized by determining display positions of the two points. In a user terminal device comprising: a communication unit that communicates with an operator terminal device including a display panel capable of multi-touch operation; a display unit having a screen size different from that of the display panel; and an operation detection unit that detects an operation on the display unit. A program to be executed,
The user terminal device,
Operation information acquisition means for acquiring operation information including touch position coordinates on the display panel from the operator terminal device, as an operation on the display panel is detected in the operator terminal device.
A display position determining unit that determines a display position on the display unit corresponding to a position touched on the display panel by converting the touch position coordinates included in the operation information according to the resolution of the display unit; and
Display control means for displaying an operation guidance image for guiding an operation performed on the display panel at a display position determined by the display position determination means;
Function as
When the operation information includes two touch position coordinates associated with touching two places on the display panel at the same time, the display position determining means may select one of the two points or the two With the position on the display unit corresponding to an arbitrary point between the points as a reference, the display unit is located at a position that is substantially the same as the physical distance between the two points on the display panel or a preset distance. A program for determining a display position of the two points. Communication means for communicating with a user terminal device including display means for displaying various screens, a display panel having a screen size different from that of the display means, an operation detection means for detecting an operation on the display panel, and a resolution of the display means A storage unit that stores resolution information regarding the program executed in an operator terminal device,
The operator terminal device,
The display corresponding to the position touched on the display panel by converting the touch position coordinates on the display panel based on the resolution information as the operation detecting unit detects an operation on the display panel. Display position determining means for determining a display position in the means, and
Transmitting means for transmitting display position information based on the display position determined by the display position determining means to the user terminal device;
Function as
The display position determining means, when two touch position coordinates are detected by the operation detecting means when two places are touched simultaneously on the display panel, one of the two points, or the point Using the position on the display means corresponding to an arbitrary point between two points as a reference, the display is displayed at a position that is substantially the same as the physical distance between the two points on the display panel or a preset distance. A program for determining a display position of the two points in the means.

Images