JP2016062410A - Image forming apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology of determining a non-touch operation of a user more properly.SOLUTION: An image forming apparatus detects a three-dimensional position of a finger of a user in a detection area, and divides the detection area into an area R1 and an area R2. In first determination processing on a position where the finger of the user is located, when the finger of the user is determined to be located in the area R2, the user operation is determined to be a gesture operation (in principle). When second determination processing determines that the finger of the user is located in the area R1 (a movement path P21) even if the first determination processing determines that the finger of the user is located in the area R2, the user operation is determined not to be a gesture operation (but to be a touch operation).SELECTED DRAWING: Figure 7

Description

  The present invention relates to an image forming apparatus and related technology.

  Although not a technique related to the image forming apparatus, there is a technique for giving an instruction to the apparatus (on-vehicle apparatus) without causing a contact operation to the display apparatus.

  For example, Patent Document 1 describes a technique that allows a user to give an instruction (volume adjustment, etc.) to an in-vehicle device without touching a liquid crystal panel of the in-vehicle device (such as a car audio device). Yes. Specifically, when the user holds the palm over the front side of the LCD panel of the in-vehicle device and tilts the palm so that the right finger is down (LCD panel side) (right tilt operation), the volume increases. Become. On the other hand, if the operation of tilting the palm so that the left finger is down (the liquid crystal panel side) (left tilt operation), the volume is reduced. In this way, the user can give an instruction (volume adjustment or the like) to the in-vehicle device by “gesture operation” without touching the liquid crystal panel.

JP 2008-83946 A

  By the way, in an image forming apparatus such as an MFP (Multi-Functional Peripheral), a desired instruction is given to the image forming apparatus by pressing (touching) a button or the like in the operation screen. “Touch operation” is generally adopted.

  Here, the inventors of the present application have devised that the image forming apparatus adopts a technique for receiving an instruction by a non-contact operation as described above. A plurality of operations including “touch operation” and “gesture operation” can be performed on the image forming apparatus. In this case, the image forming apparatus can distinguish these operations as follows. For example, when the user's finger touches the operation screen, the image forming apparatus determines the user's operation as a touch operation, and when the user's finger does not touch the operation screen, the image forming apparatus determines the user's operation as a gesture operation. As described above, when “touch operation” and “gesture operation” are used, the user operation is either “touch operation” or “gesture operation” depending on whether or not the user's finger touches the operation screen. It is conceivable that it is determined whether or not.

  However, when the “touch operation” and the “gesture operation” are distinguished based only on the presence / absence of contact with the operation screen, a non-contact operation (a “search operation” described later) is performed prior to the touch operation. A non-contact operation, etc.) may be determined (erroneous determination) as a gesture operation. For example, when searching for a desired button from among the buttons on the operation screen, some of the users are not touching the operation screen while touching (pressing) the button with their fingers closer to the operation screen. There may be a user who moves his / her finger in the left-right direction or the like in the vicinity of the operation screen in the state (front side). Note that such an operation (an operation of moving a finger without touching the operation screen in order to search for a desired button or the like during a touch operation) is also referred to as a “search operation”.

  The search operation and the gesture operation are both performed without touching the operation screen. Therefore, the image forming apparatus may not be able to distinguish between a gesture operation and a search operation. In this case, an operation including a search operation performed by the user is determined as a gesture operation (erroneous determination), and the image forming apparatus cannot correctly grasp the original intention of the user.

  As described above, an operation other than the gesture operation is determined (erroneously determined) by the image forming apparatus as the gesture operation, and thus the user's original intention may not be transmitted correctly.

  Then, this invention makes it a subject to provide the technique which can determine more accurately the non-contact operation by a user.

  In order to solve the above-mentioned problem, the invention of claim 1 is an image forming apparatus, wherein a detection means for detecting a three-dimensional position of a user's finger in a detection area near the operation screen of the image forming apparatus, and the detection area Control means for determining which of a plurality of types of user operation is, the plurality of types of operations have a first type of operation and a second type of operation, The first type of operation is an operation for touching any one of the plurality of operation members in the operation screen, and when the user searches for a desired operation member from among the plurality of operation members, The second type of operation is an operation other than the first type of operation, including a search operation that moves a finger in the proximity region of the operation screen without touching the operation screen. , Touch the operation screen The control means divides the detection area into a plurality of areas having a first area close to the operation screen and a second area other than the first area, When it is determined that the user's finger is present in the second area in the first determination process regarding the presence position of the user's finger, it is determined that the user operation is the second type of operation. However, even if it is determined in the first determination process that the user's finger is present in the second area, the user's finger that is executed after the first determination process is determined. When it is determined that the user's finger is present in the first area in the second determination process relating to the presence position, it is determined that the user operation is not the second type of operation. Do

  According to a second aspect of the present invention, in the image forming apparatus according to the first aspect of the invention, the control unit determines in the first determination process that the user's finger is present in the second region, and When it is determined in the second determination process that the user's finger exists in the first area, it is determined that the user operation is the first type of operation. And

  According to a third aspect of the present invention, in the image forming apparatus according to the first or second aspect of the present invention, in the first determination process, the control unit causes the object to remain in the second region over a first period. In the second determination process, an object is detected over the second period in the second determination process. If so, it is determined that the finger of the user exists in the first area.

  A fourth aspect of the present invention is the image forming apparatus according to any one of the first to third aspects, wherein the plurality of regions are different from both the first region and the second region. And the control means determines that the user operation is the second type of operation when it is determined in the first determination process that the user's finger exists in the second area. However, even if it is determined in the first determination process that the user's finger is present in the second area, the object remains in the third area for a certain period of time. In the second determination process, it is determined that the user operation is not the second type of operation.

  According to a fifth aspect of the present invention, in the image forming apparatus according to the fourth aspect of the invention, the plurality of types of operations are different from both the first type of operations and the second type of operations. The control means determines in the first determination process that the user's finger is present in the second area, and the user's finger is in the third area. In the second determination process, it is determined that the user operation is the third type of operation.

  According to a sixth aspect of the present invention, in the image forming apparatus according to the fourth or fifth aspect, the third type of operation is performed by the user accessing a peripheral member provided around the operation screen. The third area includes an area arranged in a portion through which the user's finger passes when the user accesses the peripheral member.

  According to a seventh aspect of the present invention, in the image forming apparatus according to any one of the fourth to sixth aspects, the detection unit is configured such that the volume of the third area is a predetermined ratio with respect to the volume of the detection area. Is larger, the detection sensitivity of the detection means is increased to enlarge the detection area and the volume of the second area.

  According to an eighth aspect of the present invention, in the image forming apparatus according to any one of the fourth to seventh aspects, the control unit includes the user over a period longer than a predetermined period longer than the predetermined period. When the first finger is present in the detection area, the third area is invalidated, the invalidated third area is regarded as the second area, and the first determination process and the first At least one of the two determination processes is executed.

  According to a ninth aspect of the present invention, in the image forming apparatus according to any one of the fourth to seventh aspects, the control means is an operation panel having the operation screen and is detachable from the image forming apparatus. When the control panel is removed from the image forming apparatus, the third area is invalidated, the invalidated third area is regarded as the second area, and the first determination process and the At least one of the second determination processes is executed.

  A tenth aspect of the present invention is the image forming apparatus according to any one of the first to third aspects, wherein the plurality of areas are different from both the first area and the second area. And the control means determines that the user operation is the second type of operation when it is determined in the first determination process that the user's finger exists in the second area. However, even if it is determined in the first determination process that the user's finger is present in the second area, the object remains in the third area for a certain period of time. In the second determination process, it is determined that the user operation has not been performed.

  According to an eleventh aspect of the present invention, in the image forming apparatus according to the tenth aspect, the third area is an area arranged in an overlapping area between the passage space on the front side of the image forming apparatus and the detection area. It is characterized by being.

  According to a twelfth aspect of the present invention, in the image forming apparatus according to the eleventh aspect of the present invention, the control means is an operation panel having the operation screen, the operation panel being capable of changing a rotation angle with respect to a horizontal direction. If a part of the second area in the detection area protrudes further to the front side than the vertical plane in contact with the foremost part of the image forming apparatus according to the change in the rotation angle, the part is included. An area is set as the third area.

  According to a thirteenth aspect of the present invention, a computer detects a three-dimensional position of a user's finger in a detection region near the operation screen of the image forming apparatus, and the detection region is a first region close to the operation screen. And a second region other than the first region, and executing a first determination process regarding which of the plurality of regions the user's fingers are present; b ) After the step a), again detecting the three-dimensional position of the user's finger and executing a second determination process regarding which of the plurality of regions the user's finger exists; c) determining whether a user operation in the detection area is a plurality of types of operations, wherein the plurality of types of operations include: A first type of operation and a second type of operation, wherein the first type of operation is an operation for touching any one of the plurality of operation members in the operation screen, An operation including a search operation for moving a user's finger in a close region of the operation screen without contacting the operation screen when searching for a desired operation member from a plurality of operation members, The type of operation is an operation other than the operation of the first type and is performed without touching the operation screen. In the step c), the user's operation is performed in the first determination process. Although it is determined that the user operation is the second type of operation when it is determined that a finger is present in the second area, the user's finger is determined to be in the first determination process. Exists in the second region Even if it is determined that the user's finger is present in the first area in the second determination process, the user operation is not the second type of operation. It is characterized in that it is determined.

  According to the first to thirteenth aspects of the invention, the image forming apparatus detects the three-dimensional position of the user's finger in the detection area, and divides the detection area into the first area and the second area. To do. The image forming apparatus notifies that the user operation is the second type of operation when it is determined in the first determination process regarding the presence position of the user's finger that the user's finger exists in the second area. However, even if it is determined in the first determination process that the user's finger exists in the second area, the user's finger exists in the first area in the second determination process. When it is determined, the user operation is determined not to be the second type operation. Therefore, it is possible to avoid a non-contact operation other than the second type operation being determined (erroneously determined) as the second type operation. Therefore, it is possible to more accurately determine the non-contact operation by the user.

1 is a diagram illustrating an appearance of an image forming apparatus (MFP). It is a figure which shows the functional block of an image forming apparatus. It is a figure which shows the mode of touch operation. It is a figure which shows the mode of gesture operation. It is a figure which shows the mode of gesture operation with respect to an operation screen. It is a figure which shows the mode of gesture operation with respect to an operation screen. It is a figure which shows the movement path | route of a user's finger. It is a figure which shows the determination result of the operation kind for every movement path | route. 6 is a flowchart illustrating an operation type determination process executed in the image forming apparatus. 6 is a flowchart illustrating an operation type determination process executed in the image forming apparatus. It is a figure which shows several user operation. It is a conceptual diagram which shows the arrangement position of a 3rd area | region. It is a figure which shows the determination result of the operation kind for every movement path | route which concerns on 2nd Embodiment. It is a figure which shows the example of arrangement | positioning of an accessory apparatus. It is a conceptual diagram which shows the arrangement position of the 3rd area | region which concerns on the modification of 2nd Embodiment. It is a figure which shows the example of arrangement | positioning of an accessory apparatus. It is a conceptual diagram which shows the arrangement position of the 3rd area | region which concerns on 3rd Embodiment. It is a conceptual diagram which shows the 2nd area | region after the expansion which concerns on 3rd Embodiment. It is a conceptual diagram which shows the arrangement position of the 3rd area | region which concerns on the modification of 3rd Embodiment. It is a conceptual diagram which shows the 2nd area | region after the expansion which concerns on the modification of 3rd Embodiment. It is a conceptual diagram which shows the arrangement position of the 3rd area | region which concerns on 4th Embodiment. It is a conceptual diagram which shows the arrangement position of the 3rd area | region which concerns on the modification of 4th Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<1. First Embodiment>
<1-1. Outline of configuration>
FIG. 1 is a diagram illustrating an appearance of the image forming apparatus 10, and FIG. 2 is a diagram illustrating functional blocks of the image forming apparatus 10. Here, an MFP (Multi-Functional Peripheral) is exemplified as the image forming apparatus 10.

  The MFP 10 is a device (also referred to as a multi-function device) having a scan function, a copy function, a facsimile function, a box storage function, and the like. Specifically, as shown in the functional block diagram of FIG. 2, the MFP 10 includes an image reading unit 2, a print output unit 3, a communication unit 4, a storage unit 5, an operation unit 6, a detection unit 13, a controller 9, and the like. Various functions are realized by operating these parts in a complex manner.

  An image reading unit (also referred to as a scan unit) 2 optically reads (that is, scans) a document placed at a predetermined position of the MFP 10, and image data of the document (both a document image or a scan image). A processing unit for generating

  The print output unit 3 is an output unit that prints out an image on various media such as paper based on data related to a print target.

  The communication unit 4 is a processing unit capable of performing facsimile communication via a public line or the like. Furthermore, the communication unit 4 can also perform network communication via a network. In this network communication, for example, various protocols such as TCP / IP (Transmission Control Protocol / Internet Protocol) are used. By using the network communication, the MFP 10 can exchange various data with a desired destination. The communication unit 4 includes a transmission unit 4a that transmits various data and a reception unit 4b that receives various data.

  The storage unit 5 includes a storage device such as a hard disk drive (HDD). The storage unit 5 stores image data of each operation screen.

  The operation unit 6 includes an operation input unit 6a that receives an operation input to the MFP 10 and a display unit 6b that displays and outputs various types of information.

  The MFP 10 is provided with a substantially plate-like operation panel unit 6c (see FIG. 1). Further, the operation panel unit 6c has a touch panel 25 (see FIG. 1) on the front side thereof. The touch panel 25 functions as a part of the operation input unit 6a and also functions as a part of the display unit 6b. The touch panel 25 is configured by embedding a piezoelectric sensor or the like in a liquid crystal display panel, and can display various kinds of information and accept an operation input (such as a button pressing operation) by an operator (user). The display unit 6b selectively switches and displays the operation screen displayed on the touch panel 25.

  For example, on the touch panel 25, various operation screens (including button images) are displayed. The operator can set various operation contents of the MFP 10 and give an operation instruction by pressing an operation member (button or the like) virtually arranged in the operation screen of the touch panel 25.

  In addition, a plurality of electrodes (electrodes in the X direction and the Y direction) arranged in a grid pattern are embedded in the liquid crystal display panel of the touch panel 25. The X-direction electrode and the Y-direction electrode are sequentially scanned, and the capacitance value of an object (such as an operator's finger) in a predetermined area (detection area (described later)) on the front surface side of the touch panel 25. Is measured (detected). The MFP 10 specifies the position of the object in the X direction and the Y direction based on the distribution of capacitance values. Further, the MFP 10 specifies the position of the object in the Z direction (distance between the object and the touch panel 25) according to the size of the capacitance. The capacitance changes according to the distance between the object and the touch panel 25, and the distance (position in the Z direction) between the object and the touch panel 25 is specified according to the changed capacitance. Specifically, the capacitance increases when the operator's finger approaches the touch panel 25, and conversely, the capacitance decreases when the operator's finger moves away from the touch panel 25. Then, the position of the operator's finger in the Z direction is specified according to the magnitude of the increased or decreased capacitance. Note that the method for specifying the position of an object (such as an operator's finger) is not limited to a method using a capacitance method, and may be a method using infrared rays or the like. For example, the position of the operator's finger is specified (detected) by the angle of the infrared ray (infrared reflected to the operator's finger) incident on the light receiving unit disposed near the periphery of the touch panel 25 or the like. You may do it.

  In the detection region (three-dimensional region (space region) where the user's finger can be detected) near the operation screen 200, the detection unit 13 detects the user's finger (here, the portion of the user's finger closest to the operation screen 200). Detect 3D position.

  The controller (control unit) 9 is a control device that is built in the MFP 10 and controls the MFP 10 in an integrated manner. The controller 9 is configured as a computer system including a CPU and various semiconductor memories (RAM and ROM). The controller 9 implements various processing units by executing predetermined software programs (hereinafter also simply referred to as programs) stored in a ROM (for example, EEPROM) in the CPU. Note that the program (specifically, a program module group) may be installed in the MFP 10 via a portable recording medium such as a USB memory or a network.

  Specifically, as illustrated in FIG. 2, the controller (control unit) 9 realizes various processing units including an input control unit 11, a display control unit 12, and a determination unit 14 by executing the program.

  The input control unit 11 is a control unit that controls an operation input operation to the operation input unit 6a (the touch panel 25 or the like). For example, the input control unit 11 controls an operation for receiving an operation input on the operation screen 200 displayed on the touch panel 25.

  The display control unit 12 is a processing unit that controls a display operation in the display unit 6b (the touch panel 25 or the like). The display control unit 12 controls operations for displaying various types of information on the operation screen 200.

  As will be described later, the determination unit 14 divides the detection region into a plurality of (here, two) regions, and determines the presence position of the user's finger in the first determination process and the second determination process. . Further, the determination unit tentatively determines which of a plurality of types of operations (touch purpose operation and gesture operation) is performed as a user operation (operation by the user) based on the first determination processing result. Then, based on the second determination processing result, it is determined which of a plurality of types of operations is performed as a user operation.

<1-2. Operation>
In the first embodiment, the detection region is divided into two regions (region R1 and region R2). In the first embodiment, the presence position of the user's finger is determined in the first determination process, and the type of user operation is provisionally determined based on the first determination process result. Then, the presence position of the user's finger is determined again in the second determination process, and the type of user operation is determined based on the second determination process result.

<Detection area classification>
As shown below, the determination unit 14 sets the detection area to 2 to determine whether the touch purpose operation or the gesture operation is performed (whether the user operation is the touch purpose operation or the gesture operation). Divide into two areas.

  The touch purpose operation is an operation for touching any of a plurality of operation members (buttons or the like (for example, the document setting button 401 (FIG. 5))) in the operation screen 200. The touch purpose operation is an operation including a search operation. The touch target operation may include the touch operation itself or may not include the touch operation. Here, the touch operation is an operation for touching the operation screen 200. As shown in FIG. 3, the user presses (touches) an operation member (button or the like (for example, the document setting button 401 (FIG. 5))) virtually displayed on the operation screen 200 (touch operation). ), A desired instruction can be given to the MFP 10. The search operation is a non-contact operation that is performed prior to the touch operation. The search operation is an operation of moving the user's finger in the proximity region of the operation screen 200 without touching the operation screen 200 when searching for a desired operation member from the plurality of operation members (buttons or the like). is there. For example, the user can search for a desired button while performing a search operation in the proximity region of the operation screen 200 prior to the touch operation.

  The gesture operation is an operation other than the touch purpose operation, and is an operation performed without touching the operation screen 200 as shown in FIG. The gesture operation is an operation (non-contact operation) for giving an instruction to the MFP 10 by a gesture (hand movement). For example, the user can give a desired instruction to the MFP 10 by an operation (gesture operation) of moving his / her finger in the front side space (upper space) of the operation screen 200. The gesture operation will be specifically described with reference to FIGS. 5 and 6. The operation screen 200 (210) of FIGS. 5 and 6 includes some buttons (original setting button 401, book original button 402,...) Among a plurality of buttons virtually arranged in the horizontal direction (lateral direction). ..) is displayed as the display target button. When the user performs a gesture operation to move his / her finger from a certain position in the front side space of the operation screen 210 toward the left side without touching the operation screen 210 in FIG. 5, the MFP 10 displays the button to be displayed. Change and execute the scroll operation of the buttons. Specifically, the display control unit 12 moves the plurality of buttons from the original position (FIG. 5) to the left side according to the movement of the finger, and displays the operation screen 210 of FIG. Note that the gesture operation is performed at a position relatively distant from the operation screen 200 (compared to the proximity region where the search operation is performed).

  Both the search operation and the gesture operation are operations (non-contact operation) in which a finger is moved in a non-contact manner in the front side space (upper space) of the operation screen 200. Therefore, a desired button on the operation screen 200 is displayed. There is a possibility that the search operation performed when searching for the image is determined (erroneously determined) by the MFP 10 as a gesture operation.

  For this reason, in order to distinguish whether the touch purpose operation (operation including the search operation) or the gesture operation is performed by the user, the determination unit 14 sets the detection area to two areas (area R1 and area R2). ) (See FIG. 7).

  As shown in FIG. 7, the region R1 (also referred to as area R1) is a region that is close to the operation screen 200 out of the two regions R1 and R2, and is also referred to as “proximity region”. When the user's finger is detected in the area R1, in principle, the determination unit 14 determines that “the user operation is a touch purpose operation”.

  The region R2 (also referred to as area R2) is a predetermined region other than the area R1 in the detection region. The area R2 is a region that is located farther than the area R1 out of the two regions R1 and R2, and is also referred to as a “separated region”. When the user's finger is detected in the area R2, in principle, the determination unit 14 determines that “the user operation is a gesture operation”.

  As described above, the detection area is divided into two areas R1 and R2, and it is determined whether the user's finger exists in the areas R1 and R2, thereby determining whether the user operation is a touch-purpose operation or not. It is possible to determine whether there is.

  However, if the type of user operation is simply determined based only on the first presence position of the user's finger (first detection position in the detection area), the following erroneous determination may occur. Specifically, with the intention of the user performing a touch target operation (including a search operation), his / her fingers are moved from the area R2 to the detection area, and then moved from the area R2 to the area R1 ( (See movement path P21 in FIG. 7). In this case, if the type of the user operation is determined based only on the first entry into the area R2, the user operation (which is originally a touch purpose operation) may be erroneously determined as a gesture operation.

  Therefore, in this embodiment, a mode in which the type of the user operation is determined based on the movement state of the user's finger between the regions R1 and R2 is further illustrated. According to this, the kind of non-contact operation by the user can be determined more accurately.

<Movement path of user's fingers>
Before describing the determination of the type of user operation, the movement path of the user's finger will be described.

  FIG. 7 is a diagram illustrating an example of movement of the user's finger. As shown in FIG. 7, there are two routes that the user's finger enters into the detection area (route P2 that enters from area R2 (movement routes P21 and P22) and route P1 that enters from area R1 (movement route). P11, P12)).

  A route P2 in which the user's finger enters the detection area from the area R2 is a route in which the user's finger continues to exist in the area R2 (movement route P22), and a user's finger from the area R2 to the area R1. And a moving route (moving route P21).

  The movement path P22 is a path where the user's finger enters the detection area from the area R2 and continues to exist in the area R2. This movement route P22 includes a route in which the user's fingers continue to exist in the area R2 and then move out of the detection area.

  The movement path P21 is a path along which the user's finger moves from the area R2 to the area R1. For example, when the user brings his / her finger close to the operation screen 200 in order to perform the “touch target operation”, the user's finger enters the detection area from the area R2, and then moves from the area R2 to the area R1. The user may perform a search operation. In such a situation, the user's finger follows the movement path P21.

  In the movement route P21, not only the movement operation of the finger after entering the area R1, but also the movement operation of the finger in the area R2 can be included in the “touch purpose operation”. That is, the “touch target operation” includes not only an operation performed by the user in the area R1 (search operation) but also an operation performed by the user in the area R2 immediately before entering the area R1 (preceding operation (for example, an operation screen)). A proximity operation to 200))) may also be included.

  In addition, the route P1 in which the user's finger enters the detection region from the area R1 with respect to the detection region is a route (movement route P11) in which the user's finger continues to exist in the area R1, and the user's finger is the area. The route is further classified into a route (movement route P12) moving from R1 to area R2.

  The movement path P11 is a path where the user's finger enters the detection area from the area R1 and continues to exist in the area R1. The movement route P11 includes a route in which the user's finger continues to exist in the area R1 and then moves out of the detection area.

  The movement path P12 is a path along which the user's finger moves from the area R1 to the area R2. For example, the user has performed a search operation in the area R1, but cannot find a desired button, and performs a gesture operation in the area R2 to move (slide) the buttons in the operation screen 200. There is. In such a situation, the user's finger follows the movement path P12.

  As described above, the movement paths of the user's fingers are roughly classified into four movement paths P22, P21, P12, and P11 (see FIG. 7).

<Determination of user operation type>
Next, processing (operation type determination processing) for determining the type of user operation (non-contact operation) will be described with reference to FIGS. 9 and 10 are flowcharts showing the operation type determination process executed in the MFP 10. In this operation type determination process, the type of user operation (non-contact operation) (“touch”) is based on which of the above-described four movement paths P22, P21, P12, and P11 the user's finger movement path corresponds to. It is determined whether it is “target operation” or “gesture operation”.

  A process for recognizing the specific content of the gesture operation (non-contact operation) (also referred to as an operation content recognition process) and a process for changing the operation screen based on the processing result of the operation content recognition process (also referred to as an operation screen change process). ) Is not shown in FIGS. The operation content recognition process and the operation screen change process (both not shown) are executed separately from the operation type determination process (FIGS. 9 and 10) and in parallel with the operation type determination process.

  In the operation content recognition process, for example, in order to move (scroll) the button in the operation screen 200, an operation of sliding the finger from the left side to the right side (also referred to as “slide operation for rightward scrolling”) is performed. Or a reverse operation (also referred to as a “left-side scroll slide operation”) is recognized. The operation content recognition processing (recognition processing of the specific content of the gesture operation) requires a certain amount of time (for example, 1 second).

  In the operation screen change process, a corresponding operation is executed based on the operation content recognized in the operation content recognition process. For example, when the operation content recognition process recognizes that the “slide operation for rightward scrolling” has been performed, the buttons in the operation screen 200 are scrolled to the right in the operation screen change process.

  However, as will be described later, the processing content (recognition result) recognized in the operation content recognition processing is invalidated when it is determined in the operation type determination processing that the user operation is not a “gesture operation”. In other words, the recognition result in the operation content recognition process is determined to be valid on condition that the user operation is determined to be a “gesture operation” in the operation type determination process. Then, based on the valid recognition result, the operation screen change process is executed. Here, a waiting time (for example, 1 second) for determining the validity of the recognition result after completion of the operation content recognition process is provided, but the present invention is not limited thereto, and the waiting time is not provided. You may do it.

  In the following, the operation type determination process that is performed when the user's finger enters the detection area from the area R2 (movement paths P22, P21) will be described with reference to the flowcharts of FIGS.

  In steps S11 to S17 in FIG. 9, first determination processing (first determination) regarding the presence position of the user's finger is performed, and the type of user operation is tentatively determined.

  In step S <b> 11, the determination unit 14 determines whether or not the user's finger is detected in the detection area. When it is determined that the user's finger has been detected by the detection unit 13 within the detection area, the determination unit 14 further acquires the detection position (three-dimensional position) of the user's finger, and the process proceeds to step S12.

  In step S12, the determination unit 14 determines whether the detection position of the user's finger is within the area R1 or the area R2. Here, it is determined that the user's finger has been detected in the area R2.

  In step S <b> 13, the determination unit 14 determines whether or not the timer 301 (FIG. 2) is executing (timekeeping). If the timer 301 does not perform the timing operation, the process proceeds to step S14, and the timer 301 starts the timing operation. If the timekeeping operation of the timer 301 is being executed, the process proceeds to step S15 (without performing the process of step S14).

  In step S <b> 15, the determination unit 14 determines whether or not a predetermined period T <b> 12 (e.g., 0.5 seconds) has elapsed since the time measurement operation of the timer 301 was started. In other words, it is determined whether or not an object (here, the user's finger) has been detected over the predetermined period T12 in the area R2. The period T12 is a threshold value for determining that the user's finger exists in the area R2 in the first determination process.

  If the elapsed time (timed time) of the timer 301 has not reached the predetermined period T12, the process returns to step S11, and it is determined again whether or not the user's finger has been detected in the detection area.

  Thus, each process of step S11-S13, S15 is performed repeatedly (however, suppose that the detection area | region in step S12 does not change (here it is the area R2)).

  When the elapsed time of the timer 301 reaches the predetermined period T12 in such repetitive processing, the determination unit 14 stops the time counting operation of the timer 301 (step S16), and the process proceeds to step S17.

  Before the elapsed time of the timer 301 reaches the predetermined period T12, the user's finger moves from the area R2 to the outside of the detection region (“NO” is determined in step S15 and then “NO” is determined in step S11). ), It is determined that the user operation is neither a touch purpose operation nor a gesture operation, and the operation type determination process ends. That is, when the detection period in area R2 (or area R1) is extremely short, it is determined that no intentional operation has been performed.

  In step S17, the determination unit 14 determines the presence position of the user's finger. Thereby, the process result (determination result) of the first determination process is obtained. Specifically, when an object is detected in the area R2 over a predetermined period T12 (for example, 0.5 seconds), the determination unit 14 performs the first determination that the user's finger exists in the area R2. It is determined as a processing result.

  In step S <b> 17, the determination unit 14 further includes a plurality of types of user operations in the detection area (here, gesture operation and touch purpose operation) based on the processing result of the first determination process related to the position of the user's finger. Is tentatively determined (determined). Here, it is tentatively determined that the user operation is a “gesture operation” based on the determination that the presence position of the user's finger is the area R2 (first determination processing result).

  In steps S18 to S26 in FIG. 10, a second determination process related to the position of the user's finger is performed, and the type of user operation is determined.

  Specifically, also in the second determination process, the type of user operation is determined depending on whether or not the user's finger continues to exist in the area related to the first determination process result (for example, area R2) (FIG. 8). FIG. 8 is a diagram illustrating the determination result of the operation type for each movement route.

  Specifically, first, processing based on the current (new) detection position of the user's finger is executed (see steps S18 and S19 in FIG. 10).

  (1) If the detected position of the user's finger is the same as the position related to the first determination process result, the process of step S18 is repeatedly executed. If the user's finger is not detected in the detection area, the process proceeds to step S26 via step S19. In these cases, the same region as the region related to the first determination processing result (for example, “area R2”) is determined as the second determination processing result regarding the presence position of the user's finger. When the area related to the first determination processing result is the area R2 and the area related to the second determination processing result is also the area R2, the user's finger is moved along the movement route P22 (see FIG. 7). It is determined. Further, regarding the type of user operation, the same content as the operation type determination result (for example, “gesture operation”) based on the first determination processing result is determined as the final determination result (see the path P22 column in FIG. 8). ).

  On the other hand, (2) when the user's finger is detected in the detection region and the detected position of the user's finger is different from the position related to the first determination processing result, the process proceeds to step S20. When it is confirmed in steps S20 to S24 that the user's finger is present at a new region position over the period T2, the user's finger is separated from the region related to the first determination processing result (for example, “area R2”). It is determined that the user has moved to the area (for example, “area R1”) (step S25). When the area related to the first determination processing result is the area R2 and the area related to the second determination processing result is the area R1, the user's finger is moved along the movement route P21 (see FIG. 7). It is determined. Furthermore, it is determined that the user operation is a different type of operation (“touch target operation”) from the determination result based on the first determination processing result (for example, “gesture operation”) (see the column of path P21 in FIG. 8). ).

  First, (1) the operation type determination process corresponding to the movement route P22 (see FIG. 7) will be described in more detail.

  In step S18, the determination unit 14 acquires the three-dimensional position of the user's finger again, and is the same as the region related to the first determination processing result (the region determined to have the user's finger (here, the area R2)). It is determined whether or not the user's finger is detected by the detection unit 13 in the area.

  If it is determined in step S18 that the user's finger has been continuously detected in the area R2, the determination unit 14 uses the first determination processing result (that the user's finger is located in the area R2) as it is. This is maintained as the second determination processing result. The determination unit 14 also maintains a provisional determination result regarding the type of user operation (specifically, the fact that the user operation is a gesture operation). Thereafter, the process of step S18 is executed again. As a result, it is continuously determined that the user operation is a gesture operation in a period in which the user's finger is continuously detected in the area R2. As described above, the specific content of the user operation can be recognized simultaneously and in parallel by a processing routine (not shown) of the operation content recognition process operating separately and in parallel.

  On the other hand, when the user's finger moves out of the area R2 and the user's finger is not detected in the area R2, the process proceeds from step S18 to step S19.

  In step S <b> 19, it is determined whether or not the finger that has moved out of the region related to the first determination processing result (here, area R <b> 2) has been detected in the detection region. If it is determined that the user's finger is not detected in the detection area, the process proceeds to step S26. When it is determined that the user's finger is detected by the detection unit 13 within the detection area, the process proceeds to step S20 (described later).

  In step S26, the determination unit 14 determines the first determination processing result as the second determination processing result. Specifically, when the user's finger moves out of the detection area after the process of step S18 is repeated, the determination unit 14 displays a provisional result indicating that the position of the user's finger is the area R2. Confirmed as the second determination processing result. Also, in step S26, the determination unit 14 determines that the user operation is a gesture operation as a determination result of the type of user operation based on the second determination processing result.

  Even when the user's finger moves out of the detection area without repeating the process of step S18, the first determination processing result is regarded as the second determination processing result (taken over as it is), and the user's finger is moved. It is determined that the location of the user is area R2 and that the user operation is a gesture operation.

  As described above, when the user's finger continues to exist in the area R2 after entering the detection area from the area R2 (and then goes out of the detection area), the user operation is a gesture operation. It is determined (refer to the column of path P22 in FIG. 8).

  Next, (2) the operation type determination process corresponding to the movement route P21 (see FIG. 7) will be described in more detail.

  When the movement path P21 is moved and the user's finger moves from the area R2 to the area R1, “NO” is determined in step S18 and “YES” is determined in step S19 based on the current detection position regarding the user's finger. Then, it is determined that the user's finger has been detected in the area R1 (step S20). That is, it is determined that the user's finger has moved from area R2 to area R1.

  In the next step S <b> 21, the determination unit 14 determines whether or not the timer 301 is executing (timekeeping). If the timer 301 does not perform the timekeeping operation, the process proceeds to step S22, and the determination unit 14 starts the timer 301 timekeeping operation. If the time measurement operation of the timer 301 is being executed, the process proceeds to step S23 (without performing the process of step S22).

  In step S <b> 23, the determination unit 14 determines whether or not a predetermined period T <b> 21 (e.g., 0.5 seconds) has elapsed since the time measurement operation of the timer 301 was started. In other words, it is determined whether or not an object (here, the user's finger) has been detected in the area R1 over the predetermined period T21. The period T21 is a threshold for determining that the user's finger is present in the area R1 in the second determination process. Here, it is assumed that the predetermined period T21 has the same length (for example, 0.5 seconds) as the predetermined period T12.

  When the elapsed time (timed time) of the timer 301 reaches the predetermined period T21, the determination unit 14 stops the timed operation of the timer 301 (step S24), and the process proceeds to step S25.

  If it is determined that the user's finger has been detected again in the area R2 before the elapsed time of the timer 301 reaches the predetermined period T21, the timer 301 is reset and the process of step S18 is executed again. . In this case, the second determination process result is tentatively determined to be the same result as the first determination process result, and the determination result indicating that the type of user operation is “gesture operation” is also maintained.

  In step S25, the second determination processing result is finally determined. Specifically, content different from the first determination processing result is determined as the second determination processing result. For example, when it is determined in the first determination process that the user's finger is present in the area R2, when the user's finger is detected in the area R1 over a predetermined period T21 in steps S20 to S23. The fact that the position of the user's finger is in the area R1 is finally determined as the second determination processing result. In other words, it is determined as the second determination processing result that the position of the user's finger is changed from the area R2 to the area R1.

  In step S25, the operation type of the user operation is further determined based on the second determination processing result. Specifically, when it is determined in the first determination process that the user's finger is present in the area R2 and in the second determination process that the user's finger is present in the area R1. The determination unit 14 finally determines that the user operation is not a gesture operation (specifically, the touch target operation is performed by the user) (see the column of the path P21 in FIG. 8). In other words, it is determined that the user operation is not a gesture operation as a processing result (final processing result) of the operation type determination process.

  If it is finally determined in step S25 that the user operation is not a gesture operation, the operation screen change process corresponding to the user operation is not performed. That is, when it is determined that the user operation is not a “gesture operation” in the operation type determination process, the recognition result of the operation content recognition process is invalidated, and the operation screen change process is not performed.

  Here, the user operation to be determined regarding the operation type is determined that the user's finger is present in the area R1 in the second determination process among the plurality of user operations C1 to C3 (see below) (see FIG. 11). The operations (series of operations (preceding operation and search operation)) C2 and C3 (described below) started by the user before the time T0 is performed.

  FIG. 11 is a diagram illustrating user operations C1 to C3. The user operation is classified into three user operations C1 to C3 based on the progress status of the operation content recognition process or the like at the time T0 when it is determined in the second determination process that the user's finger is present in the area R1. Is done.

  The user operation C1 is an operation in which both the operation content recognition processing and the operation screen change processing corresponding to the processing result of the operation content recognition processing have already been completed at the time T0 (that is, “screen changed” operation). It is.

  The user operation C2 has completed the operation content recognition process at time T0, but has not yet started the operation screen change process corresponding to the process result of the operation content recognition process (ie, “standby” operation). It is.

  The user operation C3 is an operation that is in the middle of the operation content recognition process at the time T0 and is determined to be the touch target operation in the second determination process (that is, an operation of “recognizing”). The user operation C3 is also an operation for which the corresponding operation screen change process has not yet been started.

  Among the plurality of user operations C1 to C3, in particular, it is determined that the user operations C2 and C3 for which the corresponding operation screen change processing has not yet been started are gesture operations, and the recognition result of the operation content recognition processing (rightward direction) Scroll operation for scrolling etc.) is invalidated, and the operation screen changing process is not performed. That is, the user operations C2 and C3 do not change the operation screen 200.

  In this embodiment, since the operation screen change process corresponding to the user operation C1 has already been completed, the user operation C1 is not a determination target user operation related to the operation type. However, the present invention is not limited to this, and the user operation C1 may be treated as a determination target related to the operation type, and the user operation C1 may be invalidated. Specifically, the operation screen 200 may be returned to the state before the operation screen change process that has already been performed.

  When the process of step S25 (or step S26) ends, the processing routine of the operation type determination process (FIGS. 9 and 10) ends. Then, after the minute period has elapsed, the processing routine (steps S11 to S11) is executed again.

  In the operation as described above, when it is determined in the first determination process that the user's finger exists in the area R2, it is determined in principle that the user operation is a gesture operation. However, even when it is determined in the first determination process that the user's finger exists in the area R2, when it is determined in the second determination process that the user's finger exists in the area R1, It is determined that the user operation is not a gesture operation (step S25). Therefore, it is possible to avoid a non-contact operation (touch target operation) other than the gesture operation being determined as the gesture operation. Therefore, it is possible to more accurately determine the non-contact operation by the user.

  Next, the operation type determination process performed when the user's finger enters the detection area from the area R1 (movement paths P11 and P12) will be described with reference to the flowcharts of FIGS.

  Specifically, it is determined that the user's finger has been detected in the area R1 (steps S11 and S12), and an object (user's finger) in the area R1 for a predetermined period T11 (for example, 0.5 seconds). Is detected (steps S13 to S16), it is determined as a first determination processing result that the user's finger is present in the area R1 (step S17). Then, it is determined that the user operation is a touch purpose operation (step S17). The period T11 is a threshold value for determining that the user's finger exists in the area R1 in the first determination process.

  Thereafter, when the user's finger continues to exist in area R1 (step S18) (movement path P11 in FIG. 7) and moves out of the detection area (step S19), the user's finger exists in area R1. Is determined as the second determination processing result (step S26). That is, it is determined that the user operation on the movement route P11 has been performed. Further, it is determined that the user operation is a touch purpose operation (see the path P11 column in FIG. 8).

  Further, when the user's finger moves from area R1 to area R2 (movement path P12 in FIG. 7) (steps S18 to S20), an object (user) over a predetermined period T22 (for example, 0.5 seconds) in area R2. It is determined whether or not a finger is detected (steps S21 to S24). The period T22 is a threshold for determining that the user's finger is present in the area R2 in the second determination process.

  When it is determined that the object has been detected in the area R2 over the predetermined period T22, the content different from the first determination processing result (that the user's finger exists in the area R2) is the second determination. The processing result is determined (step S25). That is, it is determined that a user operation on the movement route P12 has been performed. Then, it is determined that the user operation is a gesture operation (see the column of path P12 in FIG. 8). In other words, when it is determined in the first determination process that the user's finger exists in the area R1, and in the second determination process, it is determined that the user's finger exists in the area R2, the user operation Is determined to be a gesture operation.

  Thus, when it is determined in the first determination process that the user's finger is present in the area R1, it is determined in principle that the user operation is a touch-purpose operation. However, even if it is determined in the first determination process that the user's finger is present in the area R1, if it is determined in the second determination process that the user's finger is present in the area R2, the user It is determined that the operation is not a touch purpose operation (step S25). Therefore, it is possible to avoid the non-contact operation (gesture operation) by the user being determined as the touch purpose operation. Therefore, it is possible to more accurately determine the non-contact operation by the user.

<2. Second Embodiment>
The second embodiment is a modification of the first embodiment. Below, it demonstrates centering on difference with 1st Embodiment.

  In the first embodiment, the aspect in which the area R1 and the area R2 are provided in the detection region is illustrated.

  In the second embodiment, an example in which an area R3 (described later) is further provided in the detection region in addition to the areas R1 and R2 is illustrated.

  In the second embodiment, a determination process of the type of non-contact operation when the user's finger enters the area R3 will be described below.

  When a user uses a peripheral member (such as an accessory device of the MFP 10) provided around the operation screen 200, it is conceivable that the user's finger passes through the detection area (area R2). Specifically, as shown in the movement path P23 of FIG. 12, when a user places a document on a document table (ADF: Auto Document Feeder (ADF)) 30 of the MFP 10, the user's finger is detected in the detection area. If it passes through (area R2), the movement of the user's finger may be determined as a gesture operation. Such an operation (operation in which the user accesses the peripheral member) is also referred to as “peripheral access operation”. This “peripheral access operation” is an operation different from both the gesture operation and the touch purpose operation.

  From such a situation, in the second embodiment, as shown in FIG. 12, in addition to the areas R1 and R2, a region R3 (also referred to as area R3) is further provided in the detection region. The area R3 is a predetermined area that is different from both the area R1 and the area R2, and is provided in a part of an area that is relatively far away from the operation screen 200 (than the area R1). In addition, here, the area R3 is arranged in a portion through which the user's finger passes when the user accesses the peripheral member. For example, as shown in FIG. 12, the detection area is located above the upper surface (specifically, the plane including the upper surface) 80 of the accessory device (original table 30) that is higher than the touch panel 25. The area is set as area R3. Note that this area R3 is an area in which the movement of the finger is not recognized as an operation on the operation screen 200 (in principle) even if the user's finger is detected by the detection unit 13; Is also referred to.

  Also in the second embodiment, each processing of FIG. 9 and FIG. 10 is performed similarly to the first embodiment.

  First, the operation type determination process for the user operation (non-contact operation) of the movement paths P22 and P23 (the path P2 where the user's finger enters the detection area from the area R2) in FIG. 12 is a flowchart in FIGS. This will be described with reference to FIG.

  Specifically, when it is determined that the user's finger has been detected in the detection region (step S11), the three-dimensional position of the user's finger is acquired, and the user's finger is in areas R1, R2, and area. It is determined in which region of R3 the detection has been made (step S12). Here, it is determined that the user's finger has been detected in area R2.

  When the user's finger is detected in the area R2 for a predetermined period T12 (for example, 0.5 seconds) (steps S13 to S16), the first determination is that the user's finger is present in the area R2. It is determined as a processing result (step S17). If it is determined that the position of the user's finger is in the area R2, it is provisionally determined that the user operation is a gesture operation (step S17).

  Thereafter, when the user's finger continues to exist in the area R2 (step S18) (movement path P22 in FIG. 12) and moves out of the detection area (step S19), the first determination processing result is obtained. It is determined as the second determination processing result as it is (step S26). Specifically, it is determined as a second determination processing result that the user's finger is present in the area R2, and based on the second determination processing result, it is determined that the user operation is a gesture operation ( (Refer to the column of path P22 in FIG. 13). FIG. 13 is a diagram illustrating a part of the operation type determination result for each movement route according to the second embodiment.

  Further, when the user's finger moves from the area R2 to the area R3 (movement path P23 in FIG. 12) (steps S18 to S20), the user's finger moves in the area R3 over a certain period T23 (for example, 0.5 seconds). It is determined whether or not a finger has been detected in area R3 (steps S21 to S24). The period T23 is a threshold for determining that the user's finger is present in the area R3 in the second determination process.

  When it is determined that the object is detected in the area R3 over the predetermined period T23, the content different from the first determination processing result (that the user's finger exists in the area R3) is the second determination. The processing result is determined (step S25). That is, it is determined that the user operation on the movement route P23 has been performed. The user operation (specifically, the operation started by the user before the time point when it is determined in the second determination process that the object exists in the area R3 for a certain period T23) is a gesture operation. Is determined (see the column of path P23 in FIG. 13). In other words, when it is determined in the first determination process that the user's finger exists in the area R2, and in the second determination process, it is determined that the user's finger exists in the area R3, the user operation Is determined to be a peripheral access operation.

  In the operation as described above, when it is determined in the first determination process that the user's finger exists in the area R2, it is determined in principle that the user operation is a gesture operation. However, even when it is determined in the first determination process that the user's finger exists in the area R2, when it is determined in the second determination process that the user's finger exists in the area R3, It is determined that the user operation is not a gesture operation. Therefore, it is possible to avoid a non-contact operation (peripheral access operation) other than a gesture operation being determined as a gesture operation. Therefore, it is possible to more accurately determine the non-contact operation by the user.

  Next, for the operation type determination processing for the user operation (non-contact operation) of the movement routes P33 and P32 (the route P3 where the user's finger enters the detection region from the area R3), refer to the flowcharts of FIGS. While explaining.

  Specifically, it is determined that the user's finger has been detected in the area R3 (steps S11 and S12), and an object (user's finger) over a predetermined period T13 (for example, 0.5 seconds) in the area R3. Is detected (steps S13 to S16), it is determined as a first determination processing result that the user's finger is present in the area R3 (step S17). Based on the first determination processing result, it is determined that the user operation is a peripheral access operation (step S17). The period T13 is a threshold value for determining that the user's finger exists in the area R3 in the first determination process.

  Thereafter, when the user's fingers continue to exist in the area R3 and then move out of the detection region (movement path P33 in FIG. 12), it is determined as the second determination processing result that the user's fingers are present in the area R3. (Steps S18 to S26). That is, it is determined that the user operation on the movement route P33 has been performed. In this case, it is determined that the user operation is a peripheral access operation (step S26) (see the column of path P33 in FIG. 13).

  Further, the user's fingers move from area R3 to area R2 (movement path P32 in FIG. 12) (steps S18 to S20), and the user's fingers move in area R2 over a predetermined period T22 (for example, 0.5 seconds). When detected (steps S21 to S24), contents different from the first determination process (that the user's finger exists in the area R2) are determined as the second determination process result (step S25). That is, it is determined that the user operation on the movement route P32 has been performed. Then, it is determined that the user operation is a gesture operation (see the column of path P32 in FIG. 13). In other words, when it is determined in the first determination process that the user's finger is present in the area R3, and in the second determination process, it is determined that the user's finger is present in the area R2, It is determined that the user operation is a gesture operation.

  As described above, when it is determined in the first determination process that the user's finger exists in the area R3, it is determined in principle that the user operation is the peripheral access operation. However, even if it is determined in the first determination process that the user's finger is present in the area R3, when it is determined in the second determination process that the user's finger is present in the area R2, It is determined that the user operation is not a peripheral access operation (step S25). Therefore, it is possible to avoid the non-contact operation (gesture operation) by the user being determined as the peripheral access operation. Therefore, it is possible to more accurately determine the non-contact operation by the user.

  For convenience of illustration, only the movement paths P22, P23, P32, and P33 are shown in FIG. 12, but in the second embodiment, the user's finger enters the detection area from the area R1, and then There may also be a moving path P13 or the like where the finger moves from the area R1 to the area R3. There may also be a movement path P31 in which the user's finger enters the detection area from the area R3 and then moves from the area R3 to the area R1. The same processing as the determination processing of the type of non-contact operation is also performed on these movement paths (see the flowcharts of FIGS. 9 and 10).

  Specifically, it is determined in the first determination process (steps S11 to S17) that the user's finger is present in the area R1, and in the second determination process (steps S18 to S26), the user's finger is present in the area R3. If it is determined that the user exists, it is determined that a user operation on the movement route P13 has been performed. It is determined that the user operation on the movement route P13 is a peripheral access operation.

  Further, it is determined in the first determination process (steps S11 to S17) that the user's finger is present in the area R3, and in the second determination process (steps S18 to S26), the user's finger is present in the area R1. Is determined, it is determined that a user operation on the movement route P31 has been performed. It is determined that the user operation on the movement path P31 is a touch purpose operation.

  In the second embodiment, an example in which the accessory device (for example, the document table 30) exists at a position higher than the touch panel 25 is illustrated, but the present invention is not limited to this. For example, the idea of the second embodiment may be applied even when the accessory device is present on the side of the touch panel 25.

  FIG. 14 is a diagram (plan view) illustrating a situation (an example of arrangement of peripheral devices in a modification of the second embodiment) in which an accessory device (here, the authentication device 40) is arranged on the side (here, the left side) of the touch panel 25. Figure). Here, for simplification, it is assumed that the angle θ (FIG. 12) with respect to the horizontal plane of the operation panel unit 6 c is 0 degree (0 °).

  FIG. 15 is a conceptual diagram showing the arrangement of the area R3 in the situation where the authentication device 40 is arranged on the left side of the touch panel 25 (FIG. 14).

  When the accessory device is present on the side of the touch panel 25 (see FIG. 14), the area R3 is disposed in a portion through which the user's finger passes when accessing the accessory device disposed on the side of the touch panel 25 ( The operation type determination process (FIGS. 9 and 10) for the user operation may be executed.

<3. Third Embodiment>
The third embodiment is a modification of the second embodiment. Below, it demonstrates centering on difference with 2nd Embodiment.

  In the third embodiment, when the volume of the area R3 is larger than a predetermined ratio with respect to the volume of the entire detection area, the detection sensitivity is increased to expand the detection area and the volume of the area R2 is expanded. An embodiment is illustrated. In the third embodiment, it is assumed that the accessory device is present on the side of the touch panel 25 or the like.

  FIG. 16 is a diagram illustrating an arrangement example of peripheral devices in the third embodiment, and is a diagram illustrating an arrangement example (peripheral device arrangement example) different from FIG. 14. FIG. 16 shows a situation where the authentication device 40 is arranged on the left side of the touch panel 25 and the keyboard 60 is arranged on the right side of the touch panel 25. FIG. 17 is a conceptual diagram showing the arrangement position of the area R3 (R3a, R3b) in a situation (FIG. 16) in which the accessory devices are arranged on both sides of the touch panel 25. Note that the volume of the area R3 (R3a, R3b) related to the two accessory devices is set to an appropriate value (for example, 30% of the volume of the entire detection region).

  Here, due to the presence of the area R3 in the detection area (especially at both ends in the horizontal direction), the volume of the area R2 becomes small, and the user performs a gesture operation (especially a gesture for moving a finger in the horizontal direction). It may be difficult to perform (operation).

  For this reason, in the third embodiment, when the volume of the area R3 is larger than a predetermined ratio (for example, 50%) with respect to the volume of the detection region, the detection unit 13 increases the detection sensitivity. The detection area is enlarged and the volume of the area R2 is enlarged.

  When two accessory devices (authentication device 40 and keyboard 60) are arranged on the sides (both sides) of touch panel 25 as shown in FIG. 16, the entire volume of area R3 (the total volume of areas R3a and R3b) Is 60% of the volume of the entire detection region, and the volume of the area R3 is larger than a predetermined ratio (for example, 50%) with respect to the volume of the entire detection region. When it is determined that the volume of the area R3 is larger than a predetermined ratio with respect to the volume of the entire detection area based on the configuration (number, etc.) of the accessory devices arranged near the operation screen 200, the detection unit 13 Increases the detection sensitivity and expands the volume of the detection region, and expands the volume of the area R2 (see FIG. 18). Specifically, the detection unit 13 expands the detection area in the normal direction (upward) of the operation screen 200 (specifically, the height of the detection area is further increased by a predetermined value (for example, about 5 centimeters)), Increase the volume of the detection area. And the detection part 13 expands area R2 with expansion of the volume of a detection area | region. Specifically, the position and size of the area R3 (R3a, R3b) are maintained as they are, and all of the detection region portion (increased portion) enlarged in accordance with the increase in detection sensitivity is set in the area R2. Specifically, the area R2 is enlarged (the height of the area R2 is increased by a predetermined value (for example, about 5 centimeters)) in the normal direction (upward) of the operation screen 200, and the area R2 is also provided on the area R3. To enlarge the area R2.

  When one accessory device (only the authentication device 40) is attached to the side of the touch panel 25 (see FIG. 14), the volume of the area R3 is 30% of the volume of the detection region, and the volume of the area R3 is It is determined that the volume is smaller than a predetermined ratio (50%) with respect to the volume of the detection area. In this case, the detection unit 13 does not change the detection sensitivity, and does not increase the volume of the detection region and the volume of the area R2. By not increasing the detection sensitivity, it is possible to avoid an increase in power consumption.

  As described above, according to the operation according to the third embodiment, when the volume of the area R3 is larger than the predetermined ratio with respect to the volume of the detection area, the volumes of the detection area and the area R2 are expanded. Therefore, it is possible to avoid that it is difficult for the user to perform a gesture operation due to the presence of the area R3 (the volume of the area R2 is small) by increasing the volume of the area R2.

  In the third embodiment, the mode in which the area R3 is set on one side or both sides in the horizontal direction in the detection region is exemplified, but the present invention is not limited to this. For example, when the area R3 is set on the upper side in the vertical direction in the detection area, the detection sensitivity and the area R2 may be enlarged by increasing the detection sensitivity in the same manner as described above.

  FIG. 19 is a conceptual diagram showing the arrangement of the area R3 according to a modification of the third embodiment. In FIG. 19, an area R3 (R3c) relating to the document table 30 is also set on the upper side in the detection region.

  In the arrangement example as shown in FIG. 19, when it is determined that the volume of the area R3 is larger than a predetermined ratio with respect to the volume of the detection region, the detection unit 13 increases the detection sensitivity, and the volume of the detection region. The volume of the area R2 may be expanded in the normal direction (upward) (see FIG. 20). In this case, the area R3 may be enlarged as shown in FIG.

<4. Fourth Embodiment>
The fourth embodiment is a modification of the second embodiment. Below, it demonstrates centering on difference with 2nd Embodiment.

  In 2nd Embodiment, area R3 illustrated the aspect arrange | positioned in the part through which a user's finger passes at the time of access to an attachment apparatus by a user.

  The fourth embodiment exemplifies a mode in which the area R3 is arranged in an overlapping area between the passage space on the front side of the MFP 10 and the detection area.

  In the second embodiment, even if it is determined in the first determination process that the user's finger exists in the area R2, the second determination process indicates that the user's finger exists in the area R3. In the case of determination, an example in which it is determined that “the user operation is a peripheral access operation” is illustrated.

  In the fourth embodiment, even if it is determined in the first determination process that the user's finger exists in the area R2, it is determined in the second determination process that the user's finger exists in the area R3. In this case, an example in which it is determined that “user operation has not been performed” is illustrated.

  For example, when a user who is not operating the MFP 10 (non-operating user) passes the front side of the MFP 10, an object (a part of the body (such as an arm) of the non-operating user) is a passage space on the front side of the MFP 10. (Accidentally). In such a case, if the object is detected in the detection area, the movement of the object may be determined (erroneously determined) as a user operation (gesture operation).

  For this reason, in the fourth embodiment, an area R3 is arranged in an overlapping area between the passage space on the front side of the MFP 10 and the detection area.

  Specifically, as shown in FIG. 21, when a part of the area R2 in the detection region protrudes further to the front side than the vertical surface 90 in contact with the foremost part F1 of the MFP 10, a part of the area R2 Is arranged as area R3.

  For example, when it is determined that the object is detected only in the area R3, the determination unit 14 determines that no user operation is performed.

  In addition, among non-operating users, there may be a user who causes an object (an arm or the like) to enter the area R2 (accidentally). Even in such a case, it is determined whether or not a user operation has been performed by the operation type determination process (FIGS. 9 and 10).

  Specifically, first, in the first determination process (steps S11 to S17), it is determined that an object (such as a non-operating user's arm) is present in the area R2, and it is provisional that the user operation is a gesture operation. To be determined.

  Thereafter, the object moves from the area R2 to the area R3 (steps S18 to S20), and it is determined that the object exists in the area R3 for a certain period T23 (for example, 0.5 seconds) (steps S21 to S24). ) And content different from the first determination processing result (that the object exists in the area R3) are determined as the second determination processing result (step S25).

  In step S25 of the fourth embodiment, when it is determined in the second determination process that an object exists in the area R3 for a certain period T23, the user operation (specifically, the time of the determination) It is determined that the operation (started by the user from before) has not been performed. In other words, it is determined that the movement of the object is not performed on the MFP 10.

  As described above, according to the operation according to the fourth embodiment, when a part of the area R2 protrudes further to the front side than the vertical surface 90 in contact with the forefront part F1 of the MFP 10, a part of the area R2 is Since it is set as the area R3, it is possible to avoid the movement of an object (such as a non-operating user's arm) in the passage space on the front side of the MFP 10 being determined as a user operation.

  When the operation panel 6c can change the rotation angle θ (see FIGS. 21 and 22) with respect to the horizontal direction, the area can be changed according to the change of the rotation angle θ of the operation panel 6c. R3 may be changed.

  FIG. 22 is a diagram illustrating an arrangement position of the area R3 when the angle θ of the operation panel unit 6c is relatively small. For example, when the rotation angle θ of the operation panel unit 6c in FIG. 22 is changed to the rotation angle θ of the operation panel unit 6c in FIG. 21, the front surface of the area R2 is higher than the vertical surface 90 in contact with the foremost surface portion F1 of the MFP 10. A protruding portion protruding to the side (a protruding portion after changing the rotation angle θ) may be set (changed) as the area R3.

  Thus, since the area R3 is changed according to the rotation angle θ of the operation panel unit 6c, it is possible to set an appropriate area R3 according to the rotation angle θ of the operation panel unit 6c.

<5. Modified example>
Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described content.

  For example, in the first embodiment, the mode in which the user operation on the movement route P12 is determined to be the “gesture operation” is illustrated. However, the present invention is not limited to this. May be determined as “.”

  In the second embodiment, the mode in which the user operation on the movement route P32 is determined to be the “gesture operation” is exemplified. However, the present invention is not limited to this, and the user operation on the movement route P32 is “periphery access operation”. May be determined as “.”

  Moreover, in the said 2nd and 3rd embodiment, although the aspect in which area R3 always exists was illustrated, it is not limited to this, Area R3 was invalidated when the predetermined condition was satisfied, and it was invalidated The area R3 may be regarded as the area R2.

  For example, when the user's finger exists in the detection area for a period longer than a predetermined period T10 longer than the threshold T23, the area R3 is invalidated, and the invalidated area R3 is defined as the area R2. It may be considered. Specifically, the determination unit 14 determines that a predetermined period T10 (for example, 2 seconds) has elapsed since the user's finger was detected in the area R3 (that is, the threshold T23 (for example, 0.5 seconds) has elapsed). When 1.5 seconds elapses, the area R3 is invalidated. Then, the determination unit 14 regards the invalidated area R3 as the area R2, and executes at least one of the first determination process and the second determination process. In other words, when the user's finger is present in the detection area (area R3) for a relatively long period, it is considered that the user's finger has entered the detection area for an operation (gesture operation) on the operation screen 200. It is. Then, the area R3 is invalidated, the area R3 is regarded as the area R2, and it is determined which of the touch purpose operation and the gesture operation is performed.

  Alternatively, when the operation panel unit 6c is detachable from the MFP 10 and the operation panel unit 6c is detached from the MFP 10, the area R3 is invalidated and the invalidated area R3 is regarded as the area R2. You may do it. Specifically, when the operation panel unit 6c is detached from the MFP 10, it is determined that the accessory device of the MFP 10 does not exist in the vicinity of the operation panel unit 6c, and the determination unit 14 invalidates the area R3. Then, the determination unit 14 regards the invalidated area R3 as the area R2, and executes at least one of the first determination process and the second determination process. On the other hand, when the operation panel unit 6c is attached to the MFP 10, it is determined that the peripheral members of the MFP 10 exist near the operation panel unit 6c, and the determination unit 14 does not invalidate the area R3.

  When the predetermined condition is satisfied, the area R3 is invalidated and the area R3 is regarded as the area R2. Therefore, the area R2 is enlarged, so that the MFP 10 intends to perform the gesture operation. Can be determined more accurately.

  In each of the above embodiments, the thresholds (predetermined periods) T12, T21, T11, T22, T23, and T13 for determining the position of the user's finger are set to the same value (for example, 0.5 seconds). The embodiment to be illustrated was illustrated.

  However, the present invention is not limited to this, and the threshold values T12, T21, T11, T22, T23, and T13 for determining the presence position of the user's finger may be set to individual values.

6c Operation panel unit 10 MFP (image forming apparatus)
25 Touch panel 200, 210 Operation screen P11, P12, P21, P22, P23, P32, P33 Movement route R1-R3 area

Claims (13)

An image forming apparatus,
Detecting means for detecting a three-dimensional position of a user's finger in a detection region near the operation screen of the image forming apparatus;
Control means for determining which of a plurality of types of user operations in the detection area;
With
The plurality of types of operations include a first type of operation and a second type of operation,
The first type of operation is an operation for touching any one of the plurality of operation members in the operation screen, and when the user searches for a desired operation member from among the plurality of operation members, An operation including a search operation for moving a finger in a proximity region of the operation screen without touching the operation screen;
The operation of the second type is an operation other than the operation of the first type and is performed without touching the operation screen.
The control means divides the detection area into a plurality of areas having a first area close to the operation screen and a second area other than the first area, and the presence position of the user's finger In the first determination process relating to, when it is determined that the user's finger is present in the second area, it is determined that the user operation is the second type of operation. Even if it is determined that the user's finger is present in the second region in the determination process, the second determination regarding the presence position of the user's finger executed after the first determination process An image forming apparatus comprising: determining that the user operation is not the second type operation when it is determined in the processing that the user's finger is present in the first area. The image forming apparatus according to claim 1.
The control means determines in the first determination process that the user's finger is present in the second region, and indicates that the user's finger is present in the first region. When the determination process is performed, it is determined that the user operation is the first type of operation. The image forming apparatus according to claim 1, wherein:
The control means includes
In the first determination process, when an object is detected in the second region over a first period, it is determined that the user's finger exists in the second region,
In the second determination process, when an object is detected in the first region over a second period, it is determined that the user's finger exists in the first region. An image forming apparatus. The image forming apparatus according to any one of claims 1 to 3,
The plurality of regions also include a third region that is different from both the first region and the second region;
The control means determines that the user operation is the second type operation when it is determined in the first determination process that the user's finger is present in the second area. Even if it is determined in the first determination process that the user's finger is present in the second region, the fact that the object is present in the third region for a certain period of time may be present. An image forming apparatus, wherein when the determination is made in the second determination processing, it is determined that the user operation is not the second type of operation. The image forming apparatus according to claim 4.
The plurality of types of operations also include a third type of operation that is different from both the first type of operation and the second type of operation,
The control means determines in the first determination process that the user's finger is present in the second area, and indicates that the user's finger is present in the third area. When the determination processing is performed, it is determined that the user operation is the third type of operation. The image forming apparatus according to claim 4 or 5, wherein:
The third type of operation includes an operation in which the user accesses a peripheral member provided around the operation screen,
The image forming apparatus according to claim 3, wherein the third area includes an area arranged in a portion through which the user's finger passes when the user accesses the peripheral member. The image forming apparatus according to any one of claims 4 to 6,
When the volume of the third area is larger than a predetermined ratio with respect to the volume of the detection area, the detection means increases the detection sensitivity by the detection means and expands the detection area. 2. An image forming apparatus, wherein the volume of the area 2 is enlarged. The image forming apparatus according to claim 4, wherein:
The control means invalidates the third area when the user's finger exists in the detection area for a period longer than the predetermined period longer than the predetermined period. An image forming apparatus, wherein the third area is regarded as the second area and at least one of the first determination process and the second determination process is executed. The image forming apparatus according to claim 4, wherein:
The control unit invalidates and invalidates the third area when an operation panel having the operation screen and detachable from the image forming apparatus is detached from the image forming apparatus. The image forming apparatus, wherein the third area is regarded as the second area and at least one of the first determination process and the second determination process is executed. The image forming apparatus according to any one of claims 1 to 3,
The plurality of regions also include a third region that is different from both the first region and the second region;
The control means determines that the user operation is the second type operation when it is determined in the first determination process that the user's finger is present in the second area. Even if it is determined in the first determination process that the user's finger is present in the second region, the fact that the object is present in the third region for a certain period of time may be present. An image forming apparatus, wherein when it is determined in the second determination process, it is determined that the user operation has not been performed. The image forming apparatus according to claim 10.
The image forming apparatus, wherein the third area is an area arranged in an overlapping area between a passage space on the front side of the image forming apparatus and the detection area. The image forming apparatus according to claim 11.
The control means is an operation panel having the operation screen and can change the rotation angle of the operation panel that can change the rotation angle with respect to the horizontal direction. The image forming apparatus according to claim 1, wherein a part including the part is set as the third area when a part protrudes further to the front side than a vertical surface contacting the foremost part of the image forming apparatus. On the computer,
a) detecting a three-dimensional position of a user's finger in a detection area near the operation screen of the image forming apparatus, and using the detection area as a first area adjacent to the operation screen and a second area other than the first area; A first determination process relating to which of the plurality of regions the user's fingers are present, and
b) after the step a), again detecting the three-dimensional position of the user's finger and executing a second determination process regarding which of the plurality of regions the user's finger exists; ,
c) determining which one of a plurality of types of user operations in the detection area;
A program for executing
The plurality of types of operations include a first type of operation and a second type of operation,
The first type of operation is an operation for touching any one of the plurality of operation members in the operation screen, and when the user searches for a desired operation member from among the plurality of operation members, An operation including a search operation for moving a finger in a proximity region of the operation screen without touching the operation screen;
The operation of the second type is an operation other than the operation of the first type and is performed without touching the operation screen.
In step c), when it is determined in the first determination process that the user's finger is present in the second region, it is determined that the user operation is the second type of operation. However, even if it is determined in the first determination process that the user's finger is present in the second area, the user's finger is not detected in the second determination process. When it is determined that the user operation is present in the area, it is determined that the user operation is not the second type operation.

Images