JP6369243B2 - Operation display device, program, operation display system - Google Patents

Description

The present invention relates to an operation display device , a program, and an operation display system that use a portable terminal together.

  As a technique for determining whether the pressing is an effective operation or an erroneous operation when the operation button is pressed, and determining that the pressing is an erroneous operation, for example, Patent Document 1 or Patent There is a technique described in Document 2.

  In Patent Document 1, when an operation button displayed on a touch panel is pressed, an image obtained by imaging the pressed surface is analyzed to determine whether or not the touch panel is pressed by a finger. A technique is disclosed in which when the touch panel is pressed by the user, the pressing operation is an effective operation, and when the touch panel is pressed by a finger other than the finger, the pressing operation is an invalid operation (see Patent Document 1).

  In Patent Document 2, in a mobile phone configured by rotatingly attaching a first housing and a second housing, when the second housing starts to rotate, the rotation is finished. In the meantime, a technique for disabling an erroneous operation that occurs when the rotating second casing touches the hard key by disabling the acceptance of the operation on the hard key installed in the first casing is disclosed. (See Patent Document 2).

JP 2011-118708 A JP 2005-277627 A

  By the way, there is a method in which an operation panel provided in an image forming apparatus such as a multifunction peripheral and a user's portable terminal are used in a coordinated operation. For example, it is possible to display one large image by displaying the continuation of the image displayed on the operation panel on a mobile terminal juxtaposed on the operation panel.

  In this method, when the operation panel and the portable terminal are used in a coordinated operation, it is desirable to install the portable terminal at a position where the user can overlook both the operation panel screen and the portable terminal screen. For example, if the screen of the portable terminal is located next to the screen of the operation panel, the user can overlook both screens.

  However, since many operation panels have a hard key installed next to the screen, depending on the installation position, an operation error may occur when the mobile terminal or its handle touches the hard key. For example, if you want to collaborate with mobile devices juxtaposed on the right side of the operation panel screen, if a hard key is installed on the right side of the operation panel screen, they are juxtaposed on the right side of the operation panel screen. There is a risk that an erroneous operation may occur when the hard key is pressed by the portable terminal or its handle. Note that the above-described erroneous operation occurs not only when the hard key is pressed, but also when the mobile terminal or its handle touches the operation button displayed on the touch panel. There is a request to prevent this erroneous operation.

  In the method described in Patent Document 1, it is determined whether the received operation is valid or invalid based on whether the touch panel is pressed by hand. Therefore, in this method, an erroneous operation that occurs when the hand of the mobile terminal accidentally touches the touch panel is processed as an effective operation, and thus the above-described erroneous operation cannot be prevented.

  In the method described in Patent Document 2, the operation of the operation panel and the portable terminal are linked with each other only when the second casing starts rotating and the operation of the hard key is prohibited only during the period until the rotation ends. However, this does not correspond to the above-described adverse effects that occur when used, and an erroneous operation may occur when the mobile terminal or its handle is touched when it is not rotating.

The present invention is intended to solve the above-described problem, and an operation display that can suppress an erroneous operation that occurs when a mobile terminal or its handle contacts the operation unit when the mobile terminal is used close to the operation unit. device, and its object is to provide a program and an operation display systems.

  The gist of the present invention for achieving the object lies in the inventions of the following items.

[1] an operation unit;
A display unit;
A detection unit for detecting a position of a predetermined mobile terminal;
A linkage control unit for controlling the execution of a linkage operation in which the display of the screen related to the acceptance of the operation and the acceptance of the operation are performed by using the mobile terminal and the own device in cooperation with each other;
Operation control for partially disabling operation reception in the operation unit when the position of the portable terminal detected by the detection unit is in front of the operation unit during execution of the cooperative operation An operation display device comprising: a unit.

In the inventions according to [1] and [ 9] below, when the mobile terminal and the operation display device are operating in cooperation , and the mobile terminal is located in front of the front of the operation unit, the operation is accepted in the operation unit. Is partially disabled. In the linked operation, for example, display contents are linked, or an operation received by one is reflected on the other. When the mobile terminal is positioned in front of the front of the operation unit, the operation unit (such as an operation button) is hidden by the mobile terminal when viewed from the user of the mobile terminal, so that the user cannot operate the operation unit. In addition, an erroneous operation may occur when the handle of the touch panel touches an operation button (for example, a hard key or a soft key). Therefore, when the portable terminal is positioned in front of the front of the operation unit, the acceptance of the operation in the operation unit is partially invalidated. Thereby, when using a portable terminal close to an operation part, generation | occurrence | production of the erroneous operation by a portable terminal or its handle touching an operation part can be suppressed. The front of the operation unit is, for example, a surface on which operation buttons that can be operated by the user are arranged.

  Note that even if the mobile terminal in front of the operation unit does not touch the operation button, the handle of the mobile terminal may touch the operation button. By invalidating the acceptance of the operation, it is possible to suppress an erroneous operation caused especially by the hand of the portable terminal touching the operation button.

[2] The operation control unit partially disables operation reception in the operation unit when the mobile terminal is positioned in front of the operation unit in front of the operation unit with the back side facing the operation unit. The operation display device according to [1], wherein

In the invention according to the above [2] and [ 10] below, the reception of the operation in the operation unit is partially invalidated only when the back side of the portable terminal faces the operation unit. That is, when the operation unit is hidden behind the mobile terminal as viewed from the user operating the mobile terminal, the reception of the operation at the operation unit is partially invalidated. When the front side of the portable terminal is facing the operation unit, the acceptance of the operation in the operation unit is valid.

[ 3 ] The operation control unit invalidates reception of an operation in an area hidden by the mobile terminal in the operation unit as the operation to be partially invalidated. [1] or The operation display device according to [2] .

In the invention according to the above [3] and [ 11] below, the acceptance of operations in the area where the mobile terminals overlap (area hidden by the mobile terminals) in the operation unit is invalidated. For example, when the portable terminal overlaps with the operation unit and a part of the operation unit cannot be seen by the user, the acceptance of the operation at the part of the position where the mobile terminal has disappeared is invalidated.

[ 4 ] The operation control unit invalidates the acceptance of the operation on the operation button located in front of the front of the plurality of operation buttons of the operation unit as the operation to be partially invalidated. The operation display device according to [1] or [2], characterized in that:

In the invention according to the above [ 4 ] and the following [ 12], among the plurality of operation buttons included in the operation unit, the reception of the operation on the operation button located in front of the front of the portable terminal is invalidated. For example, when the portable terminal overlaps with the operation unit and the user cannot see a part of the operation unit, the acceptance of the operation with the operation button at the part of the position where the mobile terminal is not visible is invalidated.

[ 5 ] The operation control unit divides the operation buttons of the operation unit into a plurality of groups, and when the portable terminal is in front of some operation buttons in one group, The operation display device according to [1] or [2], wherein reception of an operation is invalidated in all of the operation buttons belonging to one group .

The [5] and in the invention according to the following [13], if there is a mobile terminal in front of the front part of the operation button among the plurality of operation buttons belonging to one group, the operation belonging to the first group Disables acceptance of operations on all buttons. For example, when the portable terminal overlaps with the operation unit and a part of a predetermined operation button group (for example, a numeric keypad) cannot be seen by the user, the acceptance of the operation is invalidated in all the operation button groups.

[ 6 ] The operation control unit partially disables operation reception in the operation unit when the position of the mobile terminal detected by the detection unit is within a predetermined distance in front of the operation unit. The operation display device according to any one of [1] to [ 5 ].

In the inventions according to [ 6 ] and [ 14] below, when the position of the mobile terminal detected by the detection unit is within a predetermined distance in front of the front of the operation unit, the operation reception in the operation unit is partially invalidated To. Even if the operation unit and the mobile terminal overlap, if the two are separated from each other, no erroneous operation will occur due to the touch of the mobile terminal or its handle, and the user will be in the space between the operation unit and the mobile terminal. Since the operation unit can be operated with a hand in hand, the acceptance of the operation in the operation unit is validated.

[ 7 ] The predetermined distance can be set for each operation button of the operation unit ,
When the distance between the portable terminal located in front of the front of the operation unit and the front of the operation unit is less than the predetermined distance set for one operation button, the operation control unit performs the one operation. operation display device according to you wherein [6] to invalidate the acceptance of the operation in buttons.

In the invention according to [ 7 ] and [ 15] below, a threshold (predetermined distance) can be set for each operation button, and the distance between the mobile terminal located in front of the front of the operation unit and the front of the operation unit is When it is less than the threshold value set for one operation button, the acceptance of the operation on the operation button is invalidated. For example, a serious problem due to an erroneous operation (for example, a problem in the case of an operation that cannot be recovered, specifically, a job execution button such as a send button, a memory erase button, an operation setting cancel button, etc. For an operation button that causes a problem when the button is operated, a predetermined distance is made longer than an operation button that is less affected by an erroneous operation.

[ 8 ] The operation control unit, in the case of at least one of the case where the reception of the operation in the operation unit is disabled from the valid, or the case where the operation reception in the operation unit is disabled from the valid, The operation display device according to any one of [1] to [ 7 ], wherein the display unit displays the fact.

In the invention according to [ 8 ] and [ 16] below, when the operation acceptance is switched from valid to invalid, or when the operation acceptance is validated from invalid to valid, The effect is displayed on the display unit. Note that at least one of the operation display device and the mobile terminal may be displayed.

[ 9 ] An operation display system having a portable terminal and an operation display device including an operation unit and a display unit,
A detection unit for detecting a position of the mobile terminal with respect to the operation display device;
A linkage control unit that controls execution of a linkage operation in which the portable terminal and the manipulation display device are used together to display a screen related to the acceptance of the operation and accept the operation;
Operation control that partially disables acceptance of operation in the operation unit when the position of the mobile terminal detected by the detection unit is in front of the operation unit during execution of the cooperative operation An operation display system comprising:

[ 10 ] The operation control unit partially disables operation reception in the operation unit when the portable terminal is positioned in front of the operation unit in front of the back side of the operation terminal. [ 9 ] The operation display system according to [ 9 ].

[ 11 ] The operation control unit invalidates reception of an operation in an area hidden by the mobile terminal in the operation unit as the operation to be partially invalidated. [9] The operation display system according to [10] .

[ 12 ] The operation control unit invalidates the acceptance of the operation on the operation button located in front of the front of the plurality of operation buttons of the operation unit as the operation to be partially invalidated. The operation display system according to [9] or [10] , characterized in that :

[ 13 ] The operation control unit divides the operation buttons of the operation unit into a plurality of groups, and when the mobile terminal is in front of some operation buttons in one group, The operation display system according to [9] or [10] , wherein reception of an operation is invalidated in all operation buttons belonging to one group .

[ 14 ] The operation control unit partially disables operation reception in the operation unit when the position of the mobile terminal detected by the detection unit is within a predetermined distance in front of the operation unit. The operation display system according to any one of [ 9 ] to [ 13 ].

[ 15 ] The predetermined distance can be set for each operation button of the operation unit ,
When the distance between the portable terminal located in front of the front of the operation unit and the front of the operation unit is less than the predetermined distance set for one operation button, the operation control unit performs the one operation. The operation display system according to [14], wherein reception of an operation on a button is invalidated.

[ 16 ] In the case where at least one of the operation control unit when the operation reception at the operation unit is disabled from the valid state or the operation reception at the operation unit is disabled from the valid state, The operation display system according to any one of [ 9 ] to [ 15 ], wherein the display unit displays the fact.

[17] A program that causes an information processing device to function as the operation display device according to any one of [1] to [ 8 ].

Operation display device according to the present invention, according to the program and the operation display systems, when used in close the portable terminal the operation unit of the operation display device, it is possible to prevent the occurrence of erroneous operation on the operation panel.

It is a figure which shows the structural example of the operation display system which concerns on embodiment of this invention. It is a block diagram which shows schematic structure of an image processing apparatus. It is a block diagram which shows schematic structure of a portable terminal. It is a figure which shows a mode that the operation panel was seen from the front. It is a figure which shows a mode just before superimposing a portable terminal on an operation panel. It is a figure which shows a mode when a portable terminal is piled up on the operation panel. It is a figure which shows the mode of the operation panel also including the part hidden by the portable terminal. It is a figure which shows an example at the time of dividing the operation button which comprises a hard key into several groups. It is a figure which shows a mode when the operation panel and the portable terminal are separated. It is a figure which shows a mode when a portable terminal is located near the operation panel. It is a flowchart which shows the process in the case of invalidating reception of operation with an operation panel. It is a figure which shows a mode when a portable terminal is piled up on the half of the screen of the display part.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 shows a configuration example of an operation display system 5 according to an embodiment of the present invention. The operation display system 5 includes an image processing apparatus 10 serving as an operation display apparatus of the present invention, and a portable terminal 40 capable of interlocking display contents with the image processing apparatus 10 through wireless communication. The portable terminal 40 is connected to the image processing apparatus 10 so as to be capable of wireless communication by a network such as a LAN (Local Area Network) or short-range wireless communication.

  The image processing apparatus 10 includes a copy function that optically reads a document and prints a duplicate image on a recording sheet, a scan function that stores image data of the read document as a file, and transmits the file to an external terminal via a network, A so-called MFP (MFP) having a printer function for forming an image related to print data received from a PC (Personal Computer) or the like over a network on a recording sheet and printing it out, a facsimile function for transmitting and receiving image data according to a facsimile procedure, and the like. ).

  The portable terminal 40 is a portable information processing device such as a tablet. In the embodiment of the present invention, the mobile terminal 40 performs a cooperative operation with the image processing apparatus 10. In the embodiment of the present invention, display contents of a display unit 48 of the portable terminal 40 to be described later and display contents of a display unit 31 (see FIG. 2) of the operation display unit 30 are linked, and linked display for displaying one large image is linked. Let it be an example of operation.

  When performing linked display, the user confirms the image by bringing the mobile terminal 40 close to the operation display unit 30 so that the display unit 31 and the display unit 48 can be seen. At this time, there is a possibility that the portable terminal 40 or its handle touches the operation unit 32 (see FIG. 2) and an erroneous operation occurs.

Therefore, in the operation display system 5, when the user brings the mobile terminal 40 close to the operation display unit 30 so as to be in front of the front of the operation display unit 30, the operation of the operation unit 32 installed on the operation display unit 30 is performed. Disable acceptance. Thereby, generation | occurrence | production of the above-mentioned erroneous operation can be suppressed. Here, the front of the operation display unit 30 is a surface on which the operation unit 32 that can be operated by the user is arranged. In the state of the image processing apparatus 10 shown in FIG. It is a look.

  FIG. 2 is a block diagram illustrating a schematic configuration of the image processing apparatus 10. The image processing apparatus 10 includes a CPU (Central Processing Unit) 11 as a control unit that comprehensively controls the operation of the image processing apparatus 10. The CPU 11 has a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a nonvolatile memory 14, a hard disk device 15, an image reading unit 16, an automatic document feeder (ADF) 17, an image processing unit via a bus. 18, a printer unit 19, a network communication unit 21, a facsimile communication unit 22, a short-range wireless communication unit 23, an operation display unit 30, and the like are connected.

  The CPU 11 is based on an OS (Operating System) program, and executes middleware, application programs, and the like. Various programs are stored in the ROM 12, and each function of the image processing apparatus 10 is realized by the CPU 11 executing various processes according to these programs. The CPU 11 serves as an operation control unit in the present invention.

  The RAM 13 is used as a work memory for temporarily storing various data when the CPU 11 executes processing based on a program, an image memory for storing image data, and the like.

  The nonvolatile memory 14 is a memory (flash memory) whose stored contents are not destroyed even when the power is turned off, and is used for storing various setting information.

  The hard disk device 15 is a large-capacity non-volatile storage device, and stores various programs and data in addition to print data and display screen data. In the embodiment of the present invention, a management table to be described later is stored.

  The image reading unit 16 functions to optically read a document and acquire image data. The image reading unit 16 sequentially, for example, a light source that irradiates light on a document, a line image sensor that receives the reflected light for one line in the width direction, and a line-by-line reading position in the length direction of the document. An optical path composed of a moving unit for moving, a lens, a mirror, and the like for guiding reflected light from the document to the line image sensor to form an image, and a conversion unit for converting an analog image signal output from the line image sensor into digital image data And so on.

  The automatic document feeder 17 feeds the document set on the platen one by one from the top one by one in order, passes the reading position of the image reading unit 16 and discharges it to a predetermined discharge position. Fulfill. The image reading unit 16 has a function of reading a document placed on the platen glass and a function of sequentially reading a document conveyed by the automatic document conveyance unit 17.

  The image processing unit 18 performs rasterization processing for converting print data into image data, image data compression and expansion processing, in addition to processing such as image enlargement / reduction and rotation.

  The printer unit 19 has a function of forming an image corresponding to image data on a recording sheet. Here, it has a recording paper transport device, a photosensitive drum, a charging device, a laser unit, a developing device, a transfer separation device, a cleaning device, and a fixing device, and forms an image by an electrophotographic process. It is configured as a so-called laser printer. Other methods may be used for image formation.

  The network communication unit 21 functions to communicate with the mobile terminal 40, an external PC, a server, and the like through a network such as a LAN.

  The facsimile communication unit 22 performs a function of transmitting / receiving image data to / from an external device having a facsimile function through a telephone line.

  The near field communication unit 23 performs near field communication with an external device such as the mobile terminal 40. The communicable distance is, for example, within 10 cm. The short-range wireless communication unit 23 is provided on the operation display unit 30 and can communicate only with a device within 10 cm from the operation display unit 30.

  The operation display unit 30 includes a display unit 31, an operation unit 32, a distance measurement unit 33, and a camera 34. The operation display unit 30 is controlled by the CPU 11 or a CPU provided separately in the operation display unit 30.

  The display unit 31 functions to display various operation screens, setting screens, and the like. The display unit 31 includes a liquid crystal display (LCD ... Liquid Crystal Display) and its driver. The operation unit 32 functions to receive various operations from the user. The operation unit 32 includes various hard switches (hard keys 36) such as a start button and a numeric keypad, and soft keys 35 using a touch panel provided on the display surface of the display unit 31. The touch panel detects a coordinate position where the display surface of the display unit 31 is touched with a touch pen or a finger. When the detected coordinate position is on the soft key 35 displayed on the display unit 31, the CPU 11 recognizes that the soft key 35 is pressed.

  The distance measuring unit 33 measures the distance from the portable terminal 40 existing within a predetermined distance from the operation display unit 30. The method for measuring the distance may be any known method.

  The camera 34 analyzes an image obtained by photographing a predetermined range around the image processing apparatus 10 and recognizes the presence or absence of an object (here, the mobile terminal 40) within the range. For example, the camera 34 captures a range of about 150 degrees in the horizontal direction in the direction (front) in which the front of the operation display unit 30 faces.

  Note that the distance that the image processing apparatus 10 can recognize that the portable terminal 40 is located in front of the front of the operation display unit 30 is limited, and the distance between the operation display unit 30 and the portable terminal 40 is When the limit is exceeded, the image processing apparatus 10 recognizes that the mobile terminal 40 is not located in front of the operation display unit 30.

  FIG. 3 is a block diagram illustrating a schematic configuration of the mobile terminal 40. The mobile terminal 40 has a CPU 41 that performs overall control of the operation of the mobile terminal 40. The CPU includes a ROM 42, a RAM 43, a nonvolatile memory 44, a camera 45, a distance measuring unit 46, an operation unit 47, a display unit 48, a network communication unit 49, a short-range wireless communication unit 51, and a CPU. An orientation sensor unit 52 is connected.

  The CPU 41 is based on an OS (Operating System) program and executes middleware, application programs, and the like. Various programs are stored in the ROM 42, and the function of the portable terminal 40 is realized by the CPU 41 executing various processes in accordance with these programs.

  The RAM 43 is used as a work memory for temporarily storing various data when the CPU 41 executes processing based on a program or for storing display data.

  The nonvolatile memory 44 is a memory (flash memory) whose stored contents are not destroyed even when the power is turned off, and is used for storing various setting information.

  The display unit 48 functions to display various operation screens and setting screens. The display unit 48 includes a liquid crystal display and its driver. The operation unit 47 functions to receive various operations from the operator. The operation unit 47 includes a small number of hard switches and a touch panel 47 a provided on the display surface of the display unit 48.

  The camera 45 captures the direction of the line of sight of the user who views the display unit 48 of the mobile terminal 40, that is, the direction in which the back surface of the display unit 48 faces. The portable terminal 40 has a thin plate shape, and almost the entire surface side is the display surface of the display unit 48. The camera 45 is provided outward on the back side (back side) of the mobile terminal 40 and photographs the direction in which the back side of the mobile terminal 40 faces. Here, the back surface side of the mobile terminal 40 is a surface on the back side of the mobile terminal 40 shown in FIG. 1 with the side on which the display unit 48 is disposed as the front surface side. That is, it is the surface opposite to the side on which the user operates.

  The distance measuring unit 46 measures the distance from the image processing apparatus 10 existing within a predetermined distance from the portable terminal 40. The method for measuring the distance may be any known method. Here, the distance to the operation display unit 30 existing in the direction in which the camera 45 captures the image, that is, the direction in which the back surface of the mobile terminal 40 faces is measured.

  The network communication unit 49 functions to communicate with the image processing apparatus 10, an external PC, a server, and the like. The near field communication unit 51 performs near field communication with an external device such as the image processing apparatus 10. The communicable distance is, for example, within 10 cm.

  The direction sensor unit 52 is configured by combining a plurality of geomagnetic sensors and a plurality of acceleration sensors. The direction sensor unit 52 detects the posture and angle of the mobile terminal 40. Further, the orientation sensor unit 52 detects the direction and speed of the displacement when the mobile terminal 40 is tilted or moved, and notifies the CPU 41 of the detection result.

  Next, the arrangement of the display unit 31 and the operation unit 32 on the operation display unit 30 and a specific example when an erroneous operation occurs will be described.

  FIG. 4 shows a state in which the operation display unit 30 is viewed from the front side of the operation display unit 30. In FIG. 4, a display unit 31 (touch panel) is arranged on the left side as viewed from the user, and a hard key 36 is arranged on the right side in front of the operation display unit 30.

  In the embodiment of the present invention, the linked display described above is performed as an example of the linked operation. When performing the interlocking display, for example, the user arranges the display unit 48 of the portable terminal 40 on the right side of the display unit 31 so that both the display unit 31 and the display unit 48 can be seen. However, since the hard key 36 is installed on the right side of the display unit 31, when the mobile terminal 40 is brought close to the right side of the display unit 31, that is, near the hard key 36, the mobile terminal 40 and its handle are There is a possibility that the hard key 36 is touched and erroneous operation occurs.

  Therefore, in the operation display system 5, in order to suppress the occurrence of the erroneous operation, an erroneous operation suppression process is performed to invalidate the acceptance of the operation on the hard key 36 when the portable terminal 40 is brought close to the operation display unit 30. In the embodiment of the present invention, it is assumed that the erroneous operation suppression processing is activated during the cooperative operation and is not activated in a state where the cooperative operation is not performed.

  Note that the holder of the portable terminal 40 is often near the hard key 36 compared to the portable terminal 40. Then, it is desired to prevent an erroneous operation by the hand of the mobile terminal 40 while accepting an operation with a normal finger. Therefore, in the present embodiment, the validity / invalidity of the operation is determined based on the distance between the portable terminal 40 and the hard key 36, and the portable terminal 40 approaches the hard key 36 within a predetermined distance in order to suppress erroneous operation by the handle. Disable the operation at a later stage. If a finger without the portable terminal 40 approaches the hard key 36, the operation is not invalidated.

  In the following, four examples of the case where the acceptance of the operation in the operation unit 32 is invalidated are shown.

<Example 1>
In the aspect example 1, only the portion of the hard key 36 where the portable terminal 40 overlaps and cannot be seen by the user is invalidated. 5, 6, and 7 show the operation display unit 30 and the portable terminal 40 when the interlocking display is performed.

  FIG. 5 shows a state where the mobile terminal 40 is not located on the front side of the operation display unit 30. Of the plurality of operation buttons constituting the hard key 36, an operation button for which operation reception is enabled is referred to as an effective button 36A. In FIG. 5, all the operation buttons are effective buttons 36A.

  In the interlocking display of FIG. 5, a list in which a plurality of user names are arranged in the vertical order is displayed on the display unit 31 of the operation display unit 30. The display unit 48 of the portable terminal 40 displays a list in which telephone numbers corresponding to the user names displayed on the display unit 31 are arranged in the vertical order in the same order as the user names. When the display unit 31 and the display unit 48 are separated, it is difficult for the user to intuitively understand the telephone number corresponding to each user name.

  FIG. 6 shows a state where the portable terminal 40 is positioned in front of the operation display unit 30 (front side). In FIG. 6, among the operation buttons constituting the hard key 36, some operation buttons are hidden behind the mobile terminal 40 and are not visible to the user. Among the operation buttons, a portion of the operation button that is not hidden by the mobile terminal 40 positioned on the front side thereof, that is, an operation button where the mobile terminal 40 is not positioned in front of the front is the effective button 36A.

  In FIG. 6, the display unit 48 of the portable terminal 40 is brought close to the display unit 31 so that the corresponding telephone number is positioned in the horizontal direction of each user name. In this state, it is easy for the user to intuitively understand the telephone number corresponding to each user name.

  FIG. 7 shows a state of the operation display unit 30 including a portion hidden by the mobile terminal 40 in the state of FIG. In FIG. 7, the position where the portable terminal 40 exists is indicated by a dotted line.

  Of the plurality of operation buttons constituting the hard key 36, an operation button for which operation acceptance is disabled is referred to as an invalid button 36B. In FIG. 7, an operation button hidden by the mobile terminal 40, that is, an operation button in which the mobile terminal 40 is located in front of the front is an invalid button 36B. An operation button that is not hidden by the portable terminal 40 located on the front side of the operation display unit 30, that is, an operation button where the portable terminal 40 is not located in front of the front is an effective button 36A.

  Thus, in the aspect example 1, the operation of the operation button in which the mobile terminal 40 is located in front of the front is invalidated. For example, when the user wants to input a number from the numeric keypad, the portable terminal 40 is moved slightly from the front front side of the numeric keypad to the side to operate the numeric keypad so that the numeric keypad can be seen. When the position of the portable terminal 40 after the movement is in front of the front of the start button, it is desired to input only to the numeric keypad, so that the input to the start button is invalidated while inputting to the numeric keypad. Situation. Therefore, only the operation button to be input can be operated by invalidating the reception of the operation only for the operation button (in the above example, the start button) in which the portable terminal 40 exists in front of the front. In addition, regarding the operation buttons within a certain distance in the circumferential direction (for example, within 3 cm) with respect to the operation buttons positioned in front of the front of the mobile terminal 40, even if the mobile terminal 40 is not positioned in front of the front Input acceptance may be disabled (even if at least a portion is not hidden).

<Example 2>
In the aspect example 2, when a plurality of operation buttons constituting the hard key 36 are divided into a plurality of groups and the mobile terminal 40 is in front of some operation buttons in one group, the operation of the group is operated. Disables acceptance of operations on all buttons.

  FIG. 8 shows a state where the hard keys 36 are divided into three groups. In FIG. 8, there are three groups: a group 37A composed of numeric keys, a group 37B composed of three operation buttons for selecting a function to be used, and a group 37C composed of a start button.

  For example, when the portable terminal 40 is in front of some of the operation buttons of the group 37A, the reception of the operation is invalidated in all the operation buttons of the group 37A. In addition, when the mobile terminal 40 is in front of some (one or two) operation buttons among the three operation buttons constituting the group 37B, the operation is performed on all the operation buttons of the group 37B. Disable acceptance.

  As described above, in the aspect example 2, when the mobile terminal 40 is in front of some operation buttons in a certain group (a plurality of predetermined operation buttons), the operation is accepted in all the operation buttons of the group. Disable. In the aspect example 2, since the acceptance / invalidation of the acceptance of the operation is switched not for each operation button but for each group, the processing load is reduced as compared with the case where the operation is switched for each operation button.

  The grouping of the operation buttons is not limited to the example of FIG. 8, but it is desirable to make the operation button group in which each operation button is installed in the vicinity as one group as possible. Even if only the handle having the side of the mobile terminal 40 is an operation button in front, if the mobile terminal 40 is located in front of another operation button belonging to the same group as this operation button, this group Since the acceptance of operations for all the operation buttons belonging to is invalid, it is possible to suppress erroneous operations by the handle.

<Example 3>
In the aspect example 3, when the mobile terminal 40 is in front of some of the operation buttons constituting the hard key 36, the reception of the operation is invalid for all the operation buttons of the hard key 36. To do.

  For example, when the portable terminal 40 is in front of some operation buttons of the numeric keypad, or some (one or two) of the three operation buttons constituting the group 37B in FIG. When the portable terminal 40 is in front of the operation buttons, the reception of operations is invalidated in all the operation buttons. The acceptance of operations may be disabled not only by the hard keys 36 but also by the entire operation unit 32 including the operation buttons (soft keys 35) displayed on the touch panel.

  As described above, in the aspect example 3, when the mobile terminal 40 is in front of some of the plurality of operation buttons constituting the hard key 36, the operation of all the operation buttons of the hard key 36 is performed. Disable acceptance. As a result, the acceptance / invalidation of the operation acceptance is switched by all the operation buttons at a time, so that the processing load is reduced as compared with the case of switching for each operation button.

  Even if only the handle having the side surface of the mobile terminal 40 is an operation button in front, if the mobile terminal 40 is positioned in front of another operation button, the acceptance of the operation becomes invalid. , It can suppress erroneous operation by the handle.

<Example 4>
In the aspect example 4, a case will be described in which the distance from the operation display unit 30 to the portable terminal 40 is calculated, and the acceptance of the operation on the hard key 36 is invalidated only when the distance is less than a predetermined threshold.

In the operation display system 5, it is determined whether or not the distance between the operation display unit 30 and the portable terminal 40 is equal to or greater than a predetermined value. For example, it is determined whether or not the distance is less than a predetermined threshold by the following method or the like.
Method 1, if communication by the short-range wireless communication unit 23 and the short-range wireless communication unit 51 is possible, it is determined that the distance is less than the threshold, and if not, the distance is greater than or equal to the threshold. to decide. In addition, when this method is adopted, the shortest distance incapable of communication between the short-range wireless communication unit 23 and the short-range wireless communication unit 51 is set as a threshold value.
Method 2, the image captured by the camera 34 is analyzed, the distance measuring unit 33 calculates the distance from the operation display unit 30 to the portable terminal 40, and determines whether or not the calculated distance is less than the threshold value.
The determination method may be a method other than the above method 1.2.

  9 and 10 are examples in the case where the portable terminal 40 is located on the front side of the operation display unit 30, and FIG. 9 is a diagram in which the distance from the operation display unit 30 to the portable terminal 40 is 10 cm or more. 10 indicates a case where the distance from the operation display unit 30 to the portable terminal 40 is less than 10 cm.

  When the threshold value is 10 cm, in FIG. 9, the distance between the operation display unit 30 and the portable terminal 40 is 10 cm or more, that is, the threshold value or more, so the portable terminal 40 is located on the front side of the operation display unit 30. The acceptance of the operation on the hard key 36 is valid.

  When the threshold value is 10 cm, in FIG. 10, the distance between the operation display unit 30 and the portable terminal 40 is less than 10 cm, that is, less than the threshold value, and thus the portable terminal 40 is located on the front side of the operation display unit 30. In this case, the acceptance of the operation on the hard key 36 becomes invalid.

  Note that a common predetermined value may be provided for all the operation buttons constituting the hard key 36, but in the embodiment of the present invention, a different value (threshold value) is provided for each operation button, and the operation display unit 30 Only the operation buttons whose distance between the portable terminals 40 is less than the threshold are invalidated to accept the operation.

  For example, important buttons such as an operation button for starting printing (such as a job execution button), an erase button for deleting a job or memory, a reset button, a cancel button for operation setting, etc. If only a threshold value is set higher than other operation buttons, even if the mobile terminal 40 or its handle touches the operation display unit 30 and an erroneous operation occurs, unintended printing starts. Not. The occurrence rate of erroneous operations can be suppressed only for specific operation buttons without reducing the convenience of the user as much as possible.

  As another example, different threshold values may be provided depending on the type of object in front of the operation button. For example, it is assumed that the threshold for the hand of the mobile terminal 40 is set to be less than 10 cm and the threshold of the mobile terminal 40 (main body) is set to be less than 5 cm for one operation button. When the object in front of the operation button is determined and it is determined that it is a handle, if the distance between the handle and the operation button is less than 10 cm, the operation reception on the operation button is invalidated. . When it is determined that the object in front of the operation button is the mobile terminal 40 (main body), if the distance between the mobile terminal 40 and the operation button is less than 5 cm, the operation reception on the operation button is invalidated. .

  The threshold value may be a preset value or may be changeable by the user. The greater the value set as the threshold (the longer the distance), the lower the risk of erroneous operation by the hand of the mobile terminal 40.

  The threshold value of each operation button is registered in the management table in the hard disk device 15, and the image processing apparatus 10 refers to the threshold value registered in the management table when determining whether the calculated distance is less than the threshold value. To do.

  Further, the distance from the predetermined point on the operation display unit 30 to the portable terminal 40 may be calculated to determine whether or not the distance is less than the threshold value. The distance to the terminal 40 may be calculated and it may be determined whether the distance is less than a threshold value.

  Thus, in the aspect example 4, even when the mobile terminal 40 is located on the front side of the operation display unit 30, depending on whether the distance between the operation display unit 30 and the mobile terminal 40 is less than a predetermined value, The acceptance / invalidation of the acceptance of the operation in the operation unit 32 is switched. If the portable terminal 40 or its handle is so far away that they do not touch the operation display unit 30, the acceptance of the operation in the operation unit 32 is not invalidated, so if the portable terminal 40 is separated from the operation display unit 30, the operation unit 32 can be used, and the convenience of the user is improved.

  Next, in the operation display system 5, a process in which the image processing apparatus 10 determines whether or not to accept the operation of the operation button of the operation display unit 30 will be described with reference to FIG. 11. In FIG. 11, an erroneous operation suppression process in the case where the aspect example 1 and the aspect example 4 are combined will be described.

  First, the image processing apparatus 10 checks whether or not a cooperative operation is being performed with the mobile terminal 40 (step S101). In the embodiment of the present invention, it is checked whether or not interlocking display is performed. If linked display is not performed, this process is terminated. When interlocking display is performed, it is confirmed whether an object exists in front of the front of each operation button based on an image captured by the camera 34 (step S102).

  Next, if there is no object in front of any of the operation buttons (step S103; No), the process returns to step S102 and the process is continued. If there is an object in front of any of the operation buttons (step S103; Yes), the distance from the operation button to the object is measured (calculated) (step S104).

  Referring to the management table stored in the hard disk device 15, a threshold value for switching between valid / invalid of accepting operation with each operation button is confirmed (step S105). In the operation button whose distance has been measured, it is confirmed whether or not the distance is less than a threshold value (step S106).

  If there is an operation button whose measured distance is less than the threshold value (step S107; Yes), if the operation reception is valid for the operation button, the operation reception is switched to invalid (step S108).

  Thereafter, in step S108, information on the operation button whose operation has been accepted is switched from valid to invalid is displayed on the display unit 31 (step S109), and the process is terminated.

  If there is no operation button whose measured distance is less than the threshold (step S107; No), it is checked whether there is an operation button for which the operation reception is invalid (step S110).

  The process of FIG. 11 is repeatedly executed. When there is an operation button in which the acceptance of the operation is invalid due to the processing executed previously (step S110; Yes), among the operation buttons, for the operation buttons whose measured distance is equal to or greater than the threshold value, The operation acceptance is switched from invalid to valid (step S111).

  After that, in step S111, information on the operation button whose operation acceptance has been switched from invalid to valid is displayed on the display unit 31 (step S112), and this process is terminated.

  If there is no operation button in which the operation acceptance is invalid (step S110; No), this process is terminated. For example, when the operation reception is enabled for all the operation buttons at the first execution time of the process of FIG. 11 or in the previous process, there is no operation button in which the operation reception is disabled.

  In step S109 and step S112, not the display unit 31 but the display unit 48 of the portable terminal 40 may display the information of the operation button whose operation acceptance has been invalidated, or both display units (31, 48). ) May be displayed. Information about each of the effective operation buttons and the invalid operation buttons may be displayed.

  In FIG. 11, although the object recognized as being located in front of the operation button in front of the operation button based on the image taken by the camera 34 is other than the portable terminal 40, the acceptance of the operation is invalidated. The operation acceptance may be invalidated only when is a portable terminal 40. Further, only when the mobile terminal 40 is positioned with the back side facing the operation display unit 30, the acceptance of the operation may be invalidated.

  The embodiment of the present invention has been described with reference to the drawings. However, the specific configuration is not limited to that shown in the embodiment, and there are changes and additions within the scope of the present invention. Are also included in the present invention.

  In the embodiment of the present invention, only the operation acceptance of the hard key 36 is invalidated. However, the acceptance of the operation may be invalidated also on the operation button (soft key 35) displayed on the touch panel. For example, depending on the image displayed on the display unit 31, the user may perform an operation while overlapping the portable terminal 40 so that half of the screen is hidden. FIG. 12 shows a state where the mobile terminal 40 is positioned in front of the right half portion of the display unit 31.

  At this time, since the mobile terminal 40 or its handle may touch the operation button (soft key 35) on the touch panel, an erroneous operation may occur, so that the operation button displayed on the touch panel is similar to the hard key 36. For the operation buttons in which the mobile terminal 40 is located in front of the (soft key 35), the acceptance of operations is invalidated.

  In the embodiment of the present invention, the linked display is described as an example of the linked operation, but the linked operation may be other than the linked display. For example, when the display unit 48 of the mobile terminal 40 has a large screen, a cooperative operation is assumed in which the display is performed only by the mobile terminal 40 and the operation is received by the hard key 36 of the operation display unit 30. There is a case where the acceptance of the operation with the operation button (soft key 35) displayed on the display unit 31 is invalidated to suppress an erroneous operation.

  In the embodiment of the present invention, the operation display unit 30 invalidates the operation only when the cooperative operation is performed. However, the operation display unit 30 may invalidate the operation reception even if the cooperative operation is not performed. Good. For example, even if the cooperative operation is not being performed, when the input operation is performed on the operation display unit 30 with reference to the content displayed on the mobile terminal 40, it is assumed that the mobile terminal 40 is brought closer to the operation display unit 30. .

  In the embodiment of the present invention, the distance to the portable terminal 40 is calculated for each operation button, and it is determined whether the distance is less than the threshold set for the operation button. Of the calculated distances, only the shortest distance may be used to determine whether or not the distance is less than a threshold value for all operation buttons.

  In the embodiment of the present invention, the image processing device 10 is an example of the operation display device of the present invention. However, the operation display device of the present invention is not limited to this, and an arbitrary one having an operation unit and a display unit. It can be a device. Further, the present invention includes a program that causes an information processing apparatus including an operation unit and a display unit to function like the image processing apparatus 10 in the embodiment.

  In the embodiment of the present invention, the camera 34 of the image processing apparatus 10 detects the position of the portable terminal 40, but the portable terminal 40 may detect the position of the own terminal. An external device may be provided to detect the position of the mobile terminal 40.

  In the embodiment of the present invention, the image processing apparatus 10 determines whether or not the mobile terminal 40 is positioned in a direction in which the front of the operation display unit 30 faces, and the operation display unit 30 is enabled to accept the operation. The mobile terminal 40 determines whether or not the mobile terminal 40 is located on the front side of the operation display unit 30 by analyzing an image obtained by the camera 45, and the operation display unit. An instruction for switching between valid / invalid of accepting the operation at 30 may be transmitted to the image processing apparatus 10. For example, the mobile terminal 40 may execute the processing from step S101 to step S107 in FIG. 11 and transmit an instruction to switch the validity / invalidity of accepting the operation to the image processing apparatus 10 with each operation button. An external device may be provided to transmit the aforementioned determination and instruction. The mobile terminal that performs the above-described determination and instruction and the program thereof are also included in the present invention (CPU 41 in the embodiment serves as an operation control unit in the mobile terminal of the present invention).

5 ... Operation display system 10 ... Image processing device 11 ... CPU
12 ... ROM
13 ... RAM
DESCRIPTION OF SYMBOLS 14 ... Nonvolatile memory 15 ... Hard disk device 16 ... Image reading part 17 ... Automatic document conveyance part 18 ... Image processing part 19 ... Printer part 21 ... Network communication part 22 ... Facsimile communication part 23 ... Short-distance wireless communication part 30 ... Operation display part DESCRIPTION OF SYMBOLS 31 ... Display part 32 ... Operation part 33 ... Distance measurement part 34 ... Camera 35 ... Soft key 36 ... Hard key 36A ... Valid button 36B ... Invalid button 37 (A, B, C) ... Group 40 ... Portable terminal 41 ... CPU
42 ... ROM
43 ... RAM
44 ... Non-volatile memory 45 ... Camera 46 ... Distance measuring part 47 ... Operation part 47a ... Touch panel 48 ... Display part 49 ... Network communication part 51 ... Near field communication part 52 ... Direction sensor part

Claims (17)

An operation unit;
A display unit;
A detection unit for detecting a position of a predetermined mobile terminal;
A linkage control unit for controlling the execution of a linkage operation in which the display of the screen related to the acceptance of the operation and the acceptance of the operation are performed by using the mobile terminal and the own device in cooperation with each other;
Operation control for partially disabling operation reception in the operation unit when the position of the portable terminal detected by the detection unit is in front of the operation unit during execution of the cooperative operation An operation display device comprising: a unit. The operation control unit partially disables operation reception in the operation unit when the portable terminal is positioned in front of the operation unit in front of the operation unit with a back side facing the operation unit. The operation display device according to claim 1 . The operation control unit as an operation in which the partially disabled, among the operation unit, according to claim 1 or 2, characterized in that to disable reception of the operation in the area hidden by the mobile terminal Operation display device. The operation control unit invalidates the acceptance of an operation on an operation button located in front of the front of the plurality of operation buttons of the operation unit as the operation to be partially invalidated. operation display device according to claim 1 or 2, characterized. The operation control unit divides the operation buttons of the operation unit into a plurality of groups, and when the portable terminal is in front of some operation buttons of one group , the one group operation display device according to claim 1 or 2, characterized in that to disable reception of the operation in all belong operation button. The operation control unit partially disables operation reception in the operation unit when the position of the mobile terminal detected by the detection unit is within a predetermined distance in front of the operation unit. The operation display device according to any one of claims 1 to 5 . The predetermined distance can be set for each operation button of the operation unit ,
When the distance between the portable terminal located in front of the front of the operation unit and the front of the operation unit is less than the predetermined distance set for one operation button, the operation control unit performs the one operation. The operation display device according to claim 6 , wherein reception of an operation on a button is invalidated. The operation control unit is configured to display the display unit in at least one of a case where reception of an operation in the operation unit is disabled from valid or a case where operation reception in the operation unit is disabled from valid. The operation display device according to any one of claims 1 to 7 , wherein the fact is displayed on the screen. An operation display system having a portable terminal and an operation display device including an operation unit and a display unit,
A detection unit for detecting a position of the mobile terminal with respect to the operation display device;
A linkage control unit that controls execution of a linkage operation in which the portable terminal and the manipulation display device are used together to display a screen related to the acceptance of the operation and accept the operation;
Operation control that partially disables acceptance of operation in the operation unit when the position of the mobile terminal detected by the detection unit is in front of the operation unit during execution of the cooperative operation An operation display system comprising: The operation control unit partially disables operation reception in the operation unit when the portable terminal is positioned in front of the operation unit in front of the operation unit with a back side facing the operation unit. The operation display system according to claim 9 . The operation control unit as an operation in which the partially disabled, among the operation unit, according to claim 9 or 10, characterized in that to disable reception of the operation in the area hidden by the mobile terminal Operation display system. The operation control unit invalidates the acceptance of an operation on an operation button located in front of the front of the plurality of operation buttons of the operation unit as the operation to be partially invalidated. The operation display system according to claim 9 or 10 , characterized in that The operation control unit divides the operation buttons of the operation unit into a plurality of groups, and when the portable terminal is in front of some operation buttons of one group , the one group The operation display system according to claim 9 or 10 , wherein reception of an operation is invalidated in all of the operation buttons belonging to . The operation control unit partially disables operation reception in the operation unit when the position of the mobile terminal detected by the detection unit is within a predetermined distance in front of the operation unit. The operation display system according to any one of claims 9 to 13 . The predetermined distance can be set for each operation button of the operation unit ,
When the distance between the portable terminal located in front of the front of the operation unit and the front of the operation unit is less than the predetermined distance set for one operation button, the operation control unit performs the one operation. The operation display system according to claim 14 , wherein reception of an operation on a button is invalidated. The operation control unit is configured to display the display unit in at least one of a case where reception of an operation in the operation unit is disabled from valid or a case where operation reception in the operation unit is disabled from valid. The operation display system according to any one of claims 9 to 15, wherein a message to that effect is displayed. A program for causing an information processing device to function as the operation display device according to any one of claims 1 to 8 .

Images