JP2016130978A - Operation display device and operation display system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation display device and an operation display system that display, with a screen being displayed on a portable terminal as a reference, a screen obtained by expanding the screen in an arbitrary direction on an operation panel.SOLUTION: A part of a predetermined image is displayed on a portable terminal 40. An operation display device acquires the position of the portable terminal and the predetermined image overlapped on a display surface of a display part 31 and information on the position of the part of the predetermined image, and performs expansion display control of displaying, on the display part, an image so that the part being displayed on the portable terminal is located at a position projected on the display surface of the display part 31.SELECTED DRAWING: Figure 6

Description

  The present invention relates to an operation display device that uses a portable terminal together, and an operation display system thereof.

  Patent Document 1 discloses a method of using an operation panel included in an image processing apparatus such as a multifunction peripheral and a user's mobile terminal in a coordinated operation. In Patent Document 1, a part of an image is displayed on an operation panel, and a part that is a continuation of the part is displayed on a mobile terminal juxtaposed on the operation panel. The part can be visually recognized.

  For example, FIG. 11 shows a case where the mobile terminal is arranged on the right side of the screen of the operation panel provided in the image processing apparatus.

  In addition to Patent Document 1, there is a technique for causing the display device B to inherit the entire display screen of the display device A when the display device A and the display device B are brought into contact with each other.

JP2013-088506A

  In the method disclosed in Patent Document 1, the operation panel of the image processing apparatus displays a part of one image, and a part of the image is displayed on the portable terminal to extend the display range. It is also assumed that a part of one image is displayed on the terminal and a subsequent part of the image is displayed on the operation panel.

  For example, a user who has a portable terminal displays a part of the operation screen of the image processing apparatus on the portable terminal, approaches the image processing apparatus while performing a setting operation, and arrives in front of the image processing apparatus In this case, it is desirable to continue the setting operation that has been performed on the mobile terminal in cooperation with the operation panel. At this time, it is convenient if a screen expanded in any direction on the screen displayed on the mobile terminal (the screen that has been operated so far) is displayed on the operation panel and can be operated in cooperation. For example, if you bring your mobile device closer to the right side of the operation panel, the screen that follows the left side of the screen currently displayed on the mobile device will be displayed on the operation panel, and if you bring your mobile device closer to the left side of the operation panel, If a screen that continues to the right of the currently displayed screen is displayed on the operation panel, the setting operation can be continued by expanding the screen in the direction desired by the user.

  However, since many operation panels have a hard key installed next to the screen, depending on the installation position, the mobile terminal may overlap the hard key on the operation panel, and the user presses the hard key. Can not be. In FIG. 11, the portable terminal overlaps with a hard key on the right side of the screen of the operation panel, and the user cannot press the hard key.

  In addition, the method of bringing the mobile device close to either the top, bottom, left, or right of the operation panel can expand the screen to only one of the top, bottom, left, and right. In addition, it is not possible to respond to requests such as expanding the screen to the entire periphery of the top, bottom, left and right.

  The present invention is intended to solve the above-described problem, and an operation display device capable of displaying on the operation panel a screen obtained by expanding the screen in an arbitrary direction on the basis of the screen displayed on the mobile terminal. The purpose is to provide an operation display system.

  The gist of the present invention for achieving the object lies in the inventions of the following items.

[1] An operation display device including an operation unit and a display unit;
A mobile device that displays a portion of the predetermined image;
A position detection unit for detecting the position of the mobile terminal superimposed on the display surface of the display unit;
With
The operation display device acquires the part of the predetermined image, the information on the image, and the position of the mobile terminal detected by the position detection unit, and the part displayed on the mobile terminal An extended display control is performed to display the image on the display unit so that the part is located at a position projected onto the display surface.

  In the invention described in [1] or [5] below, the display unit of the operation display device is used so that the user can visually recognize a predetermined image at a time as compared with the case of displaying only on the mobile terminal. Can be extended. In addition, since the display range is expanded and displayed based on a part of the display on the mobile terminal, a portion of the predetermined image displayed on the display unit of the operation display device depends on a position where the mobile terminal is overlapped on the display surface. Be changed. Therefore, the user can display a desired portion of the predetermined image on the display unit by changing the position where the portable terminal is overlapped.

[2] When the position detection unit detects a change in the position of the portable terminal superimposed on the display surface after performing the extended display control, the part displayed on the portable terminal is not changed. The display system according to [1], wherein the extended display control is re-executed in accordance with the position of the mobile terminal after the change.

  In the invention described in [2] or [6] below, when the position of the mobile terminal is moved after the mobile terminal is overlaid on the display surface of the display unit, the display content of the mobile terminal is not changed. Corresponding to the position after movement, a portion of the predetermined image displayed on the display unit of the operation display device is changed. Accordingly, the user can easily display a desired portion of the predetermined image on the display unit by appropriately moving the position of the mobile terminal after being superimposed on the display surface.

[3] The display system according to [1] or [2], wherein the operation display device displays the image on the display unit at the same scale as the portable terminal.

  In the invention described in [3] or [7] below, the operation display device displays a continuation of a portion displayed by the mobile terminal in a predetermined image at the same scale as the mobile terminal. Thereby, from a user, the part which the portable terminal is displaying and the part which the display part displays are connected in a predetermined image.

[4] The operation display device includes a normal operation mode and a power saving mode that consumes less power than the normal operation mode as the operation mode of the device, and the extended display during operation in the power saving mode. In the control, the display system according to any one of [1] to [3], wherein display of a portion of the display surface on which the portable terminal is overlapped is turned off.

  In the invention described in [4] or [8] below, in the power saving mode, power consumption is reduced by turning off the display of a portion where the portable terminal is overlapped and cannot be seen by the user. Even if the display of the above-described part is turned off, the same content is displayed on the mobile terminal that is overlaid on this part, so the range that the user can view at a time in a predetermined image does not decrease.

[5] an operation unit;
A display unit;
A position detection unit that detects a position of the portable terminal superimposed on the display surface of the display unit when a portable terminal that displays a part of the predetermined image is superimposed on the display surface of the display unit;
The part of the predetermined image, the information on the screen, and the position of the mobile terminal detected by the position detection unit are acquired, and the part displayed on the mobile terminal is displayed on the display surface. A control unit that performs extended display control to display the image on the display unit so that the part is positioned at the projected position;
An operation display device comprising:

[6] When the position detection unit detects a change in the position of the portable terminal superimposed on the display surface after performing the extended display control, the control unit displays the one displayed on the portable terminal. The operation display device according to [5], wherein the extended display control is re-executed according to the position of the mobile terminal after the change without changing the unit.

[7] The operation display device according to [5] or [6], wherein the control unit displays the image on the display unit at the same scale as the mobile terminal.

[8] The control unit includes a normal operation mode and a power saving mode that consumes less power than the normal operation mode as the operation mode of the device, and the extended display control during the operation in the power saving mode The operation display device according to any one of [5] to [7], wherein display of a portion of the display surface on which the mobile terminal is overlapped is turned off.

  According to the operation display device and the operation display system according to the present invention, a screen obtained by expanding the screen in an arbitrary direction can be displayed on the operation panel based on the screen displayed on the mobile terminal.

It is a figure which shows the structural example of the operation display system which concerns on embodiment of this invention. It is a block diagram which shows schematic structure of an image processing apparatus. It is a block diagram which shows schematic structure of a portable terminal. It is a figure which shows the example of an original image and a portable display location. It is a figure which shows the mode of a portable terminal and an operation panel before overlapping a portable terminal. It is a figure which shows the example of the display content when a portable terminal is piled up on the display surface of a display part. It is a figure which shows the example of the display content when a portable terminal is installed in the position different from FIG. It is a figure which shows a mode when the position of a portable terminal is moved during execution of extended display control. It is a figure which shows the example of the content displayed on a display part when extended display control is performed during a power saving mode. 6 is a flowchart illustrating processing performed by the image processing apparatus when extended display control is performed. It is a figure which shows the example at the time of installing a portable terminal in the position which can overlook both the screen of an operation panel, and the screen of a portable terminal.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows a configuration example of an operation display system 5 that serves as an operation display system of the present invention. The operation display system 5 includes an image processing apparatus 10 serving as an operation display apparatus of the present invention, and a portable terminal 40 capable of interlocking display contents with the image processing apparatus 10 through wireless communication. The portable terminal 40 is connected to the image processing apparatus 10 through a network such as a LAN (Local Area Network) so as to be capable of wireless communication.

  The image processing apparatus 10 includes a copy function that optically reads a document and prints a duplicate image on a recording sheet, a scan function that stores image data of the read document as a file, and transmits the file to an external terminal via a network, A so-called MFP (MFP) having a printer function for forming an image related to print data received from a PC (Personal Computer) or the like over a network on a recording sheet and printing it out, a facsimile function for transmitting and receiving image data according to a facsimile procedure, and the like. ). In the embodiment of the present invention, the image processing apparatus 10 includes an operation panel 30 (see FIG. 2) including a display unit 31 and an operation unit 32, and setting operations and the like related to printing are performed on the operation panel 30. Accept.

  The portable terminal 40 is a portable information processing device such as a tablet.

  In the embodiment of the present invention, when the mobile terminal 40 displaying a part of a predetermined image (original image) is overlaid on the display surface of the display unit 31, the image processing apparatus 10 The position of the terminal 40 is detected, and the original image and the position information (the position of the part described above) of the portion being displayed by the portable terminal 40 (hereinafter referred to as a portable display location) in the original image are acquired. The original image is displayed on the display unit 31 such that the same part as the portable display location is located at a position where the image (portable display location) displayed on the mobile terminal 40 is projected on the display surface of the display unit 31. Perform extended display control.

  In this way, by linking the display contents of the display unit 31 and the portable terminal 40, it is possible to expand the range that the user can view at a time in the original image. Moreover, the part displayed on the display part 31 among original images is changed by the position which the portable terminal 40 overlaps on a display surface. Therefore, the user can cause the display unit 31 to display a desired portion of the original image by changing the position where the portable terminal 40 is overlapped.

  FIG. 2 is a block diagram illustrating a schematic configuration of the image processing apparatus 10. The image processing apparatus 10 includes a CPU (Central Processing Unit) 11 as a control unit that comprehensively controls the operation of the image processing apparatus 10. The CPU 11 has a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a nonvolatile memory 14, a hard disk device 15, an image reading unit 16, an automatic document feeder (ADF) 17, an image processing unit via a bus. 18, a printer unit 19, a network communication unit 21, a facsimile communication unit 22, an operation panel 30 and the like are connected.

  The CPU 11 is based on the OS program, and executes middleware, application programs, and the like. Various programs are stored in the ROM 12, and each function of the image processing apparatus 10 is realized by the CPU 11 executing various processes according to these programs. The CPU 11 serves as a control unit of the present invention.

  The RAM 13 is used as a work memory for temporarily storing various data when the CPU 11 executes processing based on a program, an image memory for storing image data, and the like.

  The nonvolatile memory 14 is a memory (flash memory) whose stored contents are not destroyed even when the power is turned off, and is used for storing various setting information.

  The hard disk device 15 is a large-capacity non-volatile storage device, and stores various programs and data in addition to print data and display screen data. In the embodiment of the present invention, a dedicated application for establishing communication with the portable terminal 40 and performing extended display control is stored.

  The image reading unit 16 functions to optically read a document and acquire image data. The image reading unit 16 sequentially, for example, a light source that irradiates light on a document, a line image sensor that receives the reflected light for one line in the width direction, and a line-by-line reading position in the length direction of the document. An optical path composed of a moving unit for moving, a lens, a mirror, and the like for guiding reflected light from the document to the line image sensor to form an image, and a conversion unit for converting an analog image signal output from the line image sensor into digital image data And so on.

  The automatic document feeder 17 feeds the document set on the platen one by one from the top one by one in order, passes the reading position of the image reading unit 16 and discharges it to a predetermined discharge position. Fulfill. The image reading unit 16 has a function of reading a document placed on the platen glass and a function of sequentially reading a document conveyed by the automatic document conveyance unit 17.

  The image processing unit 18 performs rasterization processing for converting print data into image data, image data compression and expansion processing, in addition to processing such as image enlargement / reduction and rotation.

  The printer unit 19 has a function of forming an image corresponding to image data on a recording sheet. Here, it has a recording paper transport device, a photosensitive drum, a charging device, a laser unit, a developing device, a transfer separation device, a cleaning device, and a fixing device, and forms an image by an electrophotographic process. It is configured as a so-called laser printer. Other methods may be used for image formation.

  The network communication unit 21 functions to communicate with the mobile terminal 40, an external PC, a server, and the like through a network such as a LAN. Note that the method of communicating with the portable terminal 40 is not limited to the method described above, and other methods may be used, for example, infrared communication or Bluetooth (registered trademark).

  The facsimile communication unit 22 performs a function of transmitting / receiving image data to / from an external device having a facsimile function through a telephone line.

  The operation panel 30 includes a display unit 31, an operation unit 32, and a camera 33. The operation panel 30 is controlled by the CPU 11 or a CPU provided separately in the operation panel 30.

  The display unit 31 functions to display various operation screens, setting screens, and the like. The display unit 31 includes a liquid crystal display (LCD ... Liquid Crystal Display) and its driver. The operation unit 32 functions to receive various operations from the user. The operation unit 32 includes various hard switches (hard keys 36) such as a start button and a numeric keypad, and a touch panel 35 provided on the display surface of the display unit 31. The touch panel 35 detects a coordinate position where the display surface of the display unit 31 is touched with a touch pen or a finger.

  The camera 33 captures a predetermined range around the image processing apparatus 10. Then, the CPU 11 analyzes the image obtained by the photographing, and recognizes the presence or absence of an object (here, the portable terminal 40) within the range. For example, the camera 33 images the front side (front) of the operation panel 30 in a range of about 150 degrees in the horizontal direction. In the embodiment of the present invention, the CPU 11 analyzes the image obtained by the camera 33 to detect the position of the mobile terminal 40 superimposed on the display surface of the display unit 31. The camera 33 and the CPU 11 serve as a position detection unit of the present invention.

  Note that the distance that the image processing apparatus 10 can recognize that the mobile terminal 40 is located in front of the operation panel 30 is limited, and the distance between the operation panel 30 and the mobile terminal 40 is limited. If it exceeds, the image processing apparatus 10 recognizes that the mobile terminal 40 is not located in front of the front of the operation panel 30.

  In the embodiment of the present invention, the image processing apparatus 10 can switch between the normal operation mode and the power saving mode that consumes less power than the normal operation mode as the operation mode of the image processing apparatus 10 itself. For example, when a certain period of time has elapsed without receiving an operation from the user or without executing a job, the normal operation mode is shifted to the power saving mode to reduce power consumption. When an operation is received from the user or job execution is started in the power saving mode, the normal operation mode is restored from the power saving mode.

  FIG. 3 is a block diagram illustrating a schematic configuration of the mobile terminal 40. The mobile terminal 40 has a CPU 41 that performs overall control of the operation of the mobile terminal 40. A ROM 42, RAM 43, nonvolatile memory 44, camera 45, distance measurement unit 46, operation unit 47, display unit 48, network communication unit 49, and direction sensor unit 51 are connected to the CPU 41 through a bus. Yes.

  The CPU 41 is based on an OS (Operating System) program and executes middleware, application programs, and the like. Various programs are stored in the ROM 42, and the function of the portable terminal 40 is realized by the CPU 41 executing various processes in accordance with these programs.

  The RAM 43 is used as a work memory for temporarily storing various data when the CPU 41 executes processing based on a program or for storing display data.

  The nonvolatile memory 44 is a memory (flash memory) whose stored contents are not destroyed even when the power is turned off, and is used for storing various setting information. In the embodiment of the present invention, a dedicated application for establishing communication with the image processing apparatus 10 and linking display contents is stored.

  The display unit 48 functions to display various operation screens and setting screens. The display unit 48 includes a liquid crystal display and its driver. The operation unit 47 functions to receive various operations from the operator. The operation unit 47 includes a small number of hard keys 47 b and a touch panel 47 a provided on the display surface of the display unit 48.

  The camera 45 captures the direction of the line of sight of the user who views the display unit 48 of the mobile terminal 40, that is, the direction in which the back surface of the display unit 48 faces. The portable terminal 40 has a thin plate shape, and almost the entire surface side is the display surface of the display unit 48. The camera 45 is provided outward on the back side (back side) of the mobile terminal 40 and photographs the direction in which the back side of the mobile terminal 40 faces.

  The network communication unit 49 functions to communicate with the image processing apparatus 10, an external PC, a server, and the like. For example, the user launches a dedicated application on the portable terminal 40, the application issues a communication establishment request to the image processing apparatus 10, and communication is established with the image processing apparatus 10 that has received the request. When communication is established between the portable terminal 40 and the image processing apparatus 10 in this way, the portable terminal 40 and the image processing apparatus 10 are in a state where a cooperative operation is possible.

  The direction sensor unit 51 is configured by combining a plurality of geomagnetic sensors and a plurality of acceleration sensors. The direction sensor unit 51 detects the posture and angle of the mobile terminal 40. The orientation sensor unit 51 detects the direction and speed of the displacement when the mobile terminal 40 is tilted or moved, and notifies the CPU 41 of the detection result.

  Next, examples of display contents displayed by the image forming apparatus 10 and the portable terminal 40 when the image processing apparatus 10 performs extended display control will be described with reference to FIGS.

  FIG. 4 shows an entire image 80 that is an example of an original image and a partial image 81 that is an example of a mobile display location displayed on the mobile terminal 40. In the entire image 80, a tree image is arranged on the left side, and twelve display items from setting 1 to setting 12 are arranged on the right side in 4 rows and 4 columns. In FIG. 4, the mobile terminal 40 is a portion including display items (setting 3 and setting 4) arranged in the third row and the fourth row of the first column among 12 display items as mobile display locations. A partial image 81 is displayed.

  FIG. 5 shows a state before the portable terminal 40 displaying the partial image 81 is brought close to the operation panel 30. In FIG. 5, since the portable terminal 40 is not located in front of the display surface of the display unit 31, the extended display control is not performed.

  FIG. 6 shows an example where the portable terminal 40 is superimposed on the display surface of the display unit 31. In FIG. 6, the mobile terminal 40 is superimposed on the lower left portion of the display surface of the display unit 31, and the display unit 31 displays the entire image so that the partial image 81 displayed on the mobile terminal 40 is positioned at the lower left portion of the display surface. 80 is displayed. That is, the entire image 80 is displayed on the display unit 31 such that the same image as the partial image 81 displayed on the mobile terminal 40 is displayed on the display unit 31 directly below the mobile terminal 40. The

  Specifically, in FIG. 6, a portion including 12 display items (setting 1 to setting 12) in the entire image 80 is displayed on the display unit 31, and the user includes the mobile terminal 40 and the display unit. By looking over the part where the portable terminal 40 is not overlaid in 31, all 12 display items (setting 1 to setting 12) in the entire image 80 can be viewed at a time. The image processing apparatus 10 displays the entire image 80 on the display unit 31 at the same scale as the partial image 81 displayed by the mobile terminal 40.

  FIG. 7 shows an example in which the portable terminal 40 is overlapped at a position different from that in FIG. In FIG. 7, the portable terminal 40 is superimposed on the lower center part of the display surface of the display unit 31, and the display unit 31 is arranged such that the partial image 81 displayed by the portable terminal 40 is positioned at the lower center part of the display surface. A whole image 80 is displayed.

  Specifically, in FIG. 7, the display unit 31 displays a tree image in the entire image 80 and a portion including eight display items (setting 1 to setting 8). Then, by overlooking the portion of the display unit 31 where the portable terminal 40 is not overlaid, the tree image and the eight display items (setting 1 to setting 8) in the entire image 80 can be viewed at a time. 7, the image processing apparatus 10 displays the entire image 80 on the display unit 31 at the same scale as the partial image 81 displayed on the mobile terminal 40.

  Thus, when the portable terminal 40 is overlapped on the display surface of the display unit 31, the portable display location displayed on the display surface of the display unit 31 and the portable display location displayed by the portable terminal 40 are The original image is displayed on the display unit 31 so that the same portion is located. Thereby, the range which a user can visually recognize at a time can be expanded in an original image.

  Moreover, the part displayed on the display part 31 among original images is changed by the position which the portable terminal 40 overlaps on a display surface. Therefore, the user can cause the display unit 31 to display a desired portion of the original image by adjusting the position where the portable terminal 40 is overlapped.

  6 and 7, the image processing apparatus 10 displays the continuation of the portion (portable display location) displayed by the mobile terminal 40 in the original image at the same scale as the mobile terminal 40. Thereby, from a user, the part which the portable terminal 40 is displaying and the part which the display part 31 is displaying appear to be connected in a predetermined image.

  Further, in the embodiment of the present invention, the content of the setting operation received on the portable terminal 40 before the start of the extended display control is reflected in the image processing device 10 at the start of the extended display control, and during the execution of the extended display control. The operation content received by the portable terminal 40 is immediately reflected in the image processing apparatus 10.

  Next, a case where the position of the mobile terminal 40 is moved during the execution of the extended display control will be described with a specific example. When the image processing apparatus 10 detects a change in the position of the mobile terminal 40 superimposed on the display surface after performing the extended display control, the display content displayed on the mobile terminal 40 is not changed, but after the change. The extended display control is re-executed according to the position of the mobile terminal 40. That is, when the position of the portable terminal 40 is moved after the portable terminal 40 is overlaid on the display surface of the display unit 31, the display content of the portable terminal 40 is not changed, corresponding to the position after the movement, The part displayed on the display part 31 among original images is changed.

  FIG. 8 shows a case where the portable terminal 40 is moved from the state of FIG. 6 to the same position as the position of the portable terminal 40 in FIG. In the figure, the position of the mobile terminal 40 before moving is indicated by a broken line. The portable display location (partial image 81) does not change before and after the movement.

  Before the movement of the portable terminal 40, as shown in FIG. 6, the display unit 31 displayed a portion including 12 display items in the entire image 80. After the movement, as in FIG. A portion including the tree image at 80 and eight display items (setting 1 to setting 8) is displayed on the display unit 31.

  As described above, the portion displayed on the display unit 31 in the original image can be changed by changing the position of the mobile terminal 40 on the display unit 31 during the execution of the extended display control. Therefore, the user can easily display a desired portion of the original image on the display unit 31 by appropriately moving the position of the mobile terminal 40 after being superimposed on the display unit 31.

  Next, a case where extended display control is performed while the image processing apparatus 10 is in the power saving mode will be described. In the power saving mode, the image processing apparatus 10 turns off the display of the portion of the display surface on which the portable terminal 40 is superimposed. Note that the display may be turned off only for the region where the mobile display location displayed on the mobile terminal 40 is projected onto the display surface of the display unit 31.

  FIG. 9 shows an example of display contents displayed on the display unit 31 when the extended display control is performed in the power saving mode. In FIG. 9, the portable terminal 40 is located at the same position as in FIG. And the display of the part (it is set as the hidden range 90) on which the portable terminal 40 is overlapped is turned off. In the figure, the hidden range 90 is shown in gray. In FIG. 9, the display of the hidden range 90 is turned off, but since the mobile terminal 40 displaying the partial image 81 is located at this position, it appears to the user as shown in FIG.

  As described above, when the extended display control is performed in a state where the image processing apparatus 10 is in the power saving mode, the display of the portion where the portable terminal 40 overlaps (the hidden range 90) on the display surface of the display unit 31 is turned off. Even if the display of this part is turned off, the mobile terminal 40 displays the screen corresponding to this part. Therefore, the power consumption corresponding to the display turned off is reduced without reducing the range that the user can visually recognize in the original image. Can be reduced.

  FIG. 10 is a flowchart illustrating processing performed when the image processing apparatus 10 performs extended display control. The image processing apparatus 10 implements the processing shown in the flowchart of FIG. 10 by the CPU 11 executing a program stored in the ROM 12 or the hard disk device 15.

  First, when communication with the mobile terminal 40 is established (step S101), the position information of the original image and the portion displayed on the mobile terminal 40 in the original image is acquired (step S102).

  Next, the operation panel 30 is photographed by the camera 33 (step S103), and the image obtained by the photographing is analyzed (step S104). As a result of the analysis in step S104, when it is determined that the portable terminal 40 does not overlap the display surface of the display unit 31 (step S105; No), the process returns to step S102 and the processing is continued.

  When it is determined that the mobile terminal 40 is overlapped with the display surface of the display unit 31 (step S105; Yes), the position on the display surface where the mobile terminal 40 is overlapped is calculated (step S106).

  The original image is displayed in a region other than the calculated position so that the portion (portable display location) of the original image being displayed on the portable terminal 40 is positioned at the position calculated in step S106 (step S106). S107).

  If the image processing apparatus 10 is in the power saving mode (step S108; Yes), this process is terminated. If the image processing apparatus 10 is in the normal operation mode (step S108; No), the same image as the portable display location being displayed on the portable terminal 40 is displayed at the position calculated in step S106 (step S109). Exit.

  Note that the process in FIG. 10 is repeated while the mobile terminal 40 and the image processing apparatus 10 establish communication. In step S102 after the second time, the original image and the position information of the portion of the original image displayed by the mobile terminal 40 may be reacquired every time. Only the changed part may be reacquired.

  The embodiment of the present invention has been described with reference to the drawings. However, the specific configuration is not limited to that shown in the embodiment, and there are changes and additions within the scope of the present invention. Are also included in the present invention.

  In the embodiment of the present invention, in FIGS. 4, 6, 7, and 8, the mobile terminal 40 has the upper 3/4 portion superimposed on the display surface of the display unit 31. The entire mobile terminal 40 may be overlaid on the display surface of the display unit 31 instead of the unit.

  In the embodiment of the present invention, the position of the mobile terminal 40 superimposed on the display surface of the display unit 31 is detected by analyzing an image captured by the camera. Not exclusively. For example, the position of the portable terminal 40 may be detected by a sensor or the like. If the touch panel 35 is a resistive film type touch panel, the position of the portable terminal 40 installed on the touch panel 35 can be detected.

  In the embodiment of the present invention, the camera 33 of the image processing apparatus 10 detects the position of the portable terminal 40. However, the portable terminal 40 detects the position of the own terminal by the camera 45, and the image processing apparatus 10 You may be notified. The position of the portable terminal 40 may be detected by a device provided outside.

  In the embodiment of the present invention, the image processing device 10 is an example of the operation display device of the present invention. However, the operation display device of the present invention is not limited to this, and an arbitrary one having an operation unit and a display unit. It can be a device. Further, the present invention includes a program that causes an information processing apparatus including an operation unit and a display unit to function like the image processing apparatus 10 in the embodiment.

  In the embodiment of the present invention, when executing the extended display control, the image processing apparatus 10 displays a portion subsequent to the portion displayed by the mobile terminal 40 in the original image at the same scale as the mobile terminal 40. However, the scale when displaying the original image is not limited to this. You may display on a different scale from the portable terminal 40. FIG.

  After overlapping the portable terminal 40 and confirming the display of the operation panel 30 (the confirmation operation is not moved at the same position for a while or receives an operation for confirmation), the portable terminal 40 is removed from the display surface. The operation may be continued only with the operation panel 30 (for example, in the operator's pocket). In that case, the same image as the image (portable display location) displayed on the mobile terminal 40 after the mobile terminal 40 is removed is displayed in the portion where the mobile terminal 40 overlapped.

5 ... Operation display system 10 ... Image processing device 11 ... CPU
12 ... ROM
13 ... RAM
DESCRIPTION OF SYMBOLS 14 ... Nonvolatile memory 15 ... Hard disk apparatus 16 ... Image reading part 17 ... Automatic document conveyance part 18 ... Image processing part 19 ... Printer part 21 ... Network communication part 22 ... Facsimile communication part 30 ... Operation panel 31 ... Display part 32 ... Operation part 33 ... Camera 35 ... Touch panel 36 ... Hard key 40 ... Portable terminal 41 ... CPU
42 ... ROM
43 ... RAM
44 ... Non-volatile memory 45 ... Camera 46 ... Distance measurement unit 47 ... Operation unit 47a ... Touch panel 47b ... Hard key 48 ... Display unit 49 ... Network communication unit 51 ... Direction sensor unit 80 ... Whole image 81 ... Partial image 90 ... Hidden range

Claims (8)

An operation display device including an operation unit and a display unit;
A mobile device that displays a portion of the predetermined image;
A position detection unit for detecting the position of the mobile terminal superimposed on the display surface of the display unit;
With
The operation display device acquires the part of the predetermined image, the information on the image, and the position of the mobile terminal detected by the position detection unit, and the part displayed on the mobile terminal An extended display control is performed to display the image on the display unit so that the part is located at a position projected onto the display surface. When the position detection unit detects a change in the position of the mobile terminal overlaid on the display surface after performing the extended display control, the part displayed on the mobile terminal is not changed, after the change. The display system according to claim 1, wherein the extended display control is re-executed according to a position of the mobile terminal. The display system according to claim 1, wherein the operation display device displays the image on the display unit at the same scale as the portable terminal. The operation display device has a normal operation mode and a power saving mode with less power consumption than the normal operation mode as the operation mode of its own device. The display system according to any one of claims 1 to 3, wherein display of a portion of the display surface on which the mobile terminal is overlapped is turned off. An operation unit;
A display unit;
A position detection unit that detects a position of the portable terminal superimposed on the display surface of the display unit when a portable terminal that displays a part of the predetermined image is superimposed on the display surface of the display unit;
The part of the predetermined image, the information on the screen, and the position of the mobile terminal detected by the position detection unit are acquired, and the part displayed on the mobile terminal is displayed on the display surface. A control unit that performs extended display control to display the image on the display unit so that the part is positioned at the projected position;
An operation display device comprising: When the position detection unit detects a change in the position of the mobile terminal superimposed on the display surface after performing the extended display control, the part displayed on the mobile terminal changes. The operation display device according to claim 5, wherein the extended display control is re-executed according to the position of the mobile terminal after the change. The operation display device according to claim 5, wherein the control unit displays the image on the display unit at the same scale as the mobile terminal. The control unit includes a normal operation mode and a power saving mode that consumes less power than the normal operation mode as the operation mode of the device, and during the operation in the power saving mode, in the extended display control, The operation display device according to any one of claims 5 to 7, wherein display of a portion of the display surface on which the portable terminal is overlapped is turned off.