JP2016173616A - Image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device that attains both of improvement in operability and reduction in manufacturing costs by detecting a three-dimensional gesture of users and other detection object thereof by means of a common configuration.SOLUTION: A detection unit (21) is configured to generate an electromagnetic field (EFL) in spaces (RGA, RGB and RGC) around an operation panel (160) with an inclination variable, and detect a distortion of the electromagnetic field deriving from existence of objects (HND and 801) in the space. A measurement unit (22) is configured to measure a three-dimensional coordinate of the object from the distortion, and a conductivity material (801) is mounted to a portion of displaceable MFP (100) relatively with respect to the operation panel. On the basis of the coordinate of the object measured by the measurement unit, a discrimination unit (23) is configured to discriminate that the object is any of a hand (HND) of a user and the conductivity member. When the object is the hand of the user, an interpretation unit (434) is configured to interpret an input operation indexing a gesture of the user, and when the object is the conductivity member, an estimation unit (435) is configured to estimate an inclination angle (θ) of the operation panel.SELECTED DRAWING: Figure 8

Description

  The present invention relates to an image processing apparatus, and more particularly to a technique for causing the apparatus to interpret a user's gesture as an input operation.

  In recent years, in the development of image processing apparatuses such as printers, copiers, scanners, facsimile machines (FAX), etc., the review from the viewpoint of universal design (UD) is mainly focused on input devices such as operation panels (user interface: UI). Is being promoted. An image processing apparatus is often shared by a plurality of users as seen in office use. Therefore, in order to further improve the operability, it is important for UD to be easy to operate regardless of whether it is a healthy person or a disabled person.

  An operation panel having a variable inclination angle with respect to the horizontal direction is an example of a UD employed in an image processing apparatus (see, for example, Patent Document 1). In general, the display on the operation panel is most clearly visible from the direction perpendicular to the screen (the normal direction of the screen). Therefore, a tall user can easily see the display by tilting the panel horizontally. On the other hand, a short user may simply stand the panel vertically. In this way, by changing the tilt angle of the operation panel and allowing the user to adjust the tilt of the panel according to his / her height, any user can use the panel regardless of its height. Can be displayed easily.

  A three-dimensional (3D) gesture sensor is one of the promising technologies for realizing UD in the UI of an image processing apparatus (see, for example, Patent Document 2-4 and Non-Patent Document 1). The “3D gesture sensor” refers to a detector that detects a three-dimensional movement of a user's finger, hand, etc. and identifies the user's three-dimensional gesture from the movement, or a detector having the function. The principle of this sensor is diverse. For example, it uses an acceleration sensor mounted on the user's finger etc. to calculate its three-dimensional trajectory from the acceleration of the finger etc., and is mounted on the user's finger etc. Detecting electromagnetic waves emitted from a transmitter and calculating the three-dimensional coordinates of the finger, etc., photographing the user's finger etc. using infrared light, laser light, etc. Those that calculate three-dimensional coordinates, and those that detect (quasi-static) electric field distortion caused by a user's finger or the like and calculate the three-dimensional coordinates of the finger or the like (hereinafter referred to as “electric field type”). Including. If gestures such as flick and slide (swipe) are used, operations on the image processing apparatus can be expressed intuitively. Therefore, it is easy for anyone to understand the operations regardless of gender, nationality, language, culture, and so on. In this sense, the 3D gesture sensor is expected to further improve the operability of the image processing apparatus.

JP 11-119498 A Japanese Patent Application Laid-Open No. 07-104922 Special table 2013-519933 gazette Special table 2015-500545 gazette JP 2002-135369 A

"Microchip Gest IC (registered trademark) design guide", Microchip, 2013-2015 ("MICROCHIP GestIC (registered trademark) Design Guide," Microchip Technology, Inc., 2013-2015)

In addition to further improvement in operability, image processing apparatuses are required to further reduce manufacturing costs. These two requirements are essentially in a trade-off relationship. In fact, it is effective to install a new sensor such as a 3D gesture sensor to further improve the operability. However, reducing the number of sensors is effective for reducing the manufacturing cost.
As one of contrivances for achieving both improvement in operability and reduction in manufacturing cost, a technique is known in which an existing sensor detects another target in addition to the original detection target. For example, the mobile phone disclosed in Patent Literature 5 determines whether or not a call is in progress by causing the touch panel to detect the user's contact with the receiver.

  In order to mount a 3D gesture sensor in an image processing apparatus for the purpose of improving operability, it is desirable to make this sensor detect a detection target of an existing sensor in addition to a user's gesture. If this contrivance is realized, the manufacturing cost of the image processing apparatus can be reduced by reducing the number of existing sensors. However, nothing is known about such a device yet.

  An object of the present invention is to solve the above-described problems, and in particular, by detecting a user's three-dimensional gesture and other detection objects with a common configuration, improving operability and reducing manufacturing costs. An object of the present invention is to provide an image processing apparatus capable of achieving both.

  An image processing apparatus according to one aspect of the present invention includes an operation panel, generates an electric field in a space extending from a surface of the operation panel to a predetermined range, and sets it as a monitoring target space. A detection unit that detects the distortion of the electric field caused by the detection unit, a measurement unit that measures the three-dimensional coordinates of the object in the space to be monitored from the distortion of the electric field detected by the detection unit, and the operation panel of the image processing apparatus. And an object based on the three-dimensional coordinates measured by the measuring unit, and a conductive member that is provided at a position that can be displaced in a movable manner and that changes the distortion of the electric field according to the relative position with the operation panel. A discriminating unit that discriminates between an indicator used for user input operation and a conductive member, and a tertiary measured by the measuring unit when the discrimination result by the discriminating unit is an indicator. Interpreting the coordinate movement as a user's gesture, interpreting the input operation indexed by the gesture, and if the determination result is a conductive member, the operation panel and this image processing from the three-dimensional coordinates measured by the measuring unit An estimation unit that estimates a relative position of the apparatus with respect to the above-described part, and a control unit that determines an operation state of the image processing apparatus based on the position estimated by the estimation unit and controls processing according to the operation state And.

  The detection unit includes a transmission electrode arranged in a planar shape, four reception electrodes arranged separately on each side of a rectangular surface extending in parallel with a predetermined distance from a plane including the transmission electrode, A transmission unit that transmits a transmission voltage signal to the transmission electrode; and a reception unit that detects a reception voltage signal received by each of the four reception electrodes in response to the transmission voltage signal, the distortion of the electric field described above with respect to the transmission voltage signal You may detect as a distortion of the waveform of a received voltage signal. In this case, the electrical potential is such that the overall potential can be regarded as substantially uniform as compared with the level of the transmission voltage signal, and the potential can be regarded as substantially equal to the ground potential as compared with the level of the transmission voltage signal. The detection unit detects a change appearing in a drop in the level of the reception voltage signal with respect to the transmission voltage signal depending on whether or not the maintained object exists in the space to be monitored, and the conductive member has a potential substantially equal to the ground potential. The measurement unit may measure the three-dimensional coordinates of the object from the change appearing in the drop of the level.

The discriminating unit discriminates that the object located at the three-dimensional coordinate is an indicator when the three-dimensional coordinate measured by the measuring unit is located at a distance equal to or less than the threshold from the detecting unit, and the three-dimensional coordinate is detected. When the object is located at a distance exceeding the threshold from the part, the object located at the three-dimensional coordinate may be determined to be a conductive member.
The discriminating unit discriminates that the object located at the three-dimensional coordinate is an indicator when the three-dimensional coordinate measured by the measuring unit is located at a distance equal to or less than the threshold from the detecting unit, and the three-dimensional coordinate is detected. When the distance from the portion exceeds the threshold, the temporal change of the three-dimensional coordinate is further observed. If the width of the temporal change is equal to or less than a predetermined value, the object positioned at the three-dimensional coordinate is the conductive member. It may be determined that

  The operation panel has a variable surface inclination with respect to the horizontal plane, and the conductive member may be arranged so that the trajectory with respect to the surface passes through the space to be monitored when the surface of the operation panel is inclined from the horizontal plane. In this case, the operation panel includes a display unit for displaying an operation screen, the estimation unit estimates the inclination of the operation panel from the three-dimensional coordinates measured by the measurement unit, and the control unit displays according to the inclination. The display attribute may be changed by the part. The interpretation unit may estimate the coordinates on the operation screen targeted by the user's gesture based on the tilt of the operation panel.

  The image processing apparatus may further include a scanner that reads an image from a sheet placed on the document table, and an upper lid that is attached to the scanner so as to be openable and closable so as to cover the document table. In this case, the operation panel is installed on the edge of the document table, and the conductive member is arranged so that the locus drawn on the surface of the operation panel as the upper lid is opened and closed passes through the space to be monitored. The operating state of the image processing apparatus may be determined based on the position of the upper lid estimated by the estimation unit.

  This image processing apparatus is attached to an image forming unit that forms an image on a sheet with toner or ink, and can be opened and closed at a portion of the front surface of the image processing apparatus located below the operation panel. Or you may provide further the front door which enables replenishment of an ink or taking in and out of a sheet | seat. In this case, the conductive member is arranged so that the locus drawn on the surface of the operation panel as the front door is opened and closed passes through the space to be monitored, and the control unit is positioned at the position of the front door estimated by the estimation unit. The operating state of the image processing apparatus may be determined based on the result.

  In the image processing apparatus according to the present invention, first, the detection unit detects distortion of the electric field due to the presence of the object in the space to be monitored. Next, the determination unit measures the three-dimensional coordinates of the object from the distortion of the electric field, and determines whether the object is the indicator or the conductive member based on the three-dimensional coordinates. When the determination result is an indicator, the interpretation unit interprets an input operation by a user's gesture from the movement of the indicator, and when it is a conductive member, the estimation unit includes an operation panel and a part provided with the conductive member. Guess the relative position of. In this way, this image processing apparatus detects a user's three-dimensional gesture and a three-dimensional displacement of a predetermined part with a common configuration. Thereby, this image processing apparatus can achieve both improvement in operability and reduction in manufacturing cost.

(A) is a perspective view which shows the external appearance of the image processing apparatus by embodiment of this invention, (b) is a side view of the operation panel mounted in this image processing apparatus. (A), (b) is a schematic diagram which respectively shows the relationship between the inclination angle of the operation panel shown in FIG. 1, and a tall user and a short user. It is a block diagram which shows the structure of the electronic control system of the image processing apparatus shown in FIG. It is a block diagram which shows the electronic control system of the operation panel shown in FIG. (A) is a top view of the electrode group of the electric field type | mold 3D gesture sensor shown in FIG. 4, (b) is an enlarged view of the cross section along the straight line bb which (a) shows, (c). These are the equivalent circuit diagrams of the transmission electrode 4T, the 4th reception electrode 4E, a transmission part, and a receiving part which (a) shows. (A)-(c) shows the electric field generated between the transmission electrode 4T and the substrate 4G when a positive voltage with respect to the ground potential is applied to the transmission electrode 4T shown in FIG. FIG. 5 is a side view showing a case where a ground conductor is not present in the vicinity, a user's hand is present, and a grounded conductor is present. (D) is a graph which shows the difference of the distortion of the waveform of the received voltage signal according to the presence or absence of a ground conductor for every receiving electrode. (A) is a side view which shows the range of the three-dimensional coordinate of the grounding conductor which can be detected with the electric field type | mold 3D gesture sensor shown in FIG. (B) is a side view showing a virtual range of the three-dimensional coordinates of the ground conductor that can be identified by detection by the 3D gesture sensor. (C) is a side view showing two different ground conductors D1 and D2 located at two points of the front side space URG shown in (a), and (d) is located at another one point of the front side space URG. It is a side view which shows one grounding conductor D3 to do. (A) is a schematic diagram which shows the space of the monitoring object set in the range which (a) of FIG. 7 shows. (B) is a schematic diagram showing the relationship between the space to be monitored and the virtual range shown in (b) of FIG. (C) is a schematic diagram showing a space to be monitored when the surface of the touch panel is inclined with respect to the horizontal plane by swinging the operation panel. (D) is a schematic diagram showing the correspondence between the space to be monitored shown in (c) and the virtual range shown in (b) of FIG. (A), (b) is a graph which respectively shows an example of the time change of the Z-axis component of the three-dimensional coordinate measured from the distortion of the waveform of the received voltage signal RVS. It is a flowchart of the process by the discrimination method (A). It is a flowchart of the process by the discrimination method (B). (A) is a correspondence table between the display attributes of the LCD shown in FIG. 4, the detection characteristics by the 3D gesture sensor, and the tilt angle θ of the operation panel. (B) and (c) show characters displayed on the screen when it is estimated that the tilt angle of the operation panel belongs to the first section (θ <θB) and the second section (θ ≧ θB), respectively. It is a schematic diagram. (A), (b), and (c) are respectively the first section (θ ≦ θ1), the second section (θ1 <θ <θ2), and the third section (θ2). It is a schematic diagram showing a user's line of sight directed to the screen when belonging to ≦ θ). (D) is a table | surface which shows the correction value of the object coordinate according to this inclination angle. (A) is a schematic diagram showing the positional relationship between the upper lid of the document table of the scanner shown in FIG. 1 and the space to be monitored by the electric field type 3D gesture sensor shown in FIG. (B) is a schematic diagram showing the correspondence between the space to be monitored shown in (a) and the virtual range of the three-dimensional coordinates of the ground conductor that can be identified by detection by this 3D gesture sensor. (C) is a schematic diagram showing the space to be monitored when the upper lid shown in (a) is opened. (D) is a schematic diagram showing the correspondence between the space to be monitored shown in (c) and the virtual range of the three-dimensional coordinates of the ground conductor that can be identified by detection by a 3D gesture sensor. (A) is a schematic diagram which shows the positional relationship of the front door of the image processing apparatus shown in FIG. 1, and the space of the monitoring object by the electric field type | mold 3D gesture sensor shown in FIG. (B) is a schematic diagram showing the correspondence between the space to be monitored shown in (a) and the virtual range of the three-dimensional coordinates of the ground conductor that can be identified by detection by the 3D gesture sensor. (C) is a schematic diagram showing the space to be monitored when the front door shown in (a) is opened. (D) is a schematic diagram showing the correspondence between the space to be monitored shown in (c) and the virtual range of the three-dimensional coordinates of the ground conductor that can be identified by detection by a 3D gesture sensor. (A) and (b) are respectively measured by the determination unit when the inclination angle of the operation panel shown in FIG. 1 is 0 ° and is a positive value, and the user's hand is positioned at a fixed point in the first area 3. It is a schematic diagram which shows a dimensional coordinate. (C) is a correspondence table of the three-dimensional coordinates measured by the determination unit, the tilt angle of the operation panel, and the gesture coordinates.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Appearance of image processing device]
FIG. 1A is a perspective view showing an appearance of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus 100 is a multi-function peripheral (MFP) that has a function of a scanner, a color copier, and a color laser printer. Referring to FIG. 1A, an automatic document feeder (ADF) 110 is mounted on the top surface of the MFP 100 so as to be openable and closable. A scanner 120 is built in the upper part of the casing located directly under the ADF 110, a printer 130 is built in the lower part, and a plurality of stages of paper feed cassettes 140 are attached to the bottom of the scanner 130 so that they can be pulled out. A gap GAP is opened between the scanner 120 and the printer 130, and a paper discharge tray 150 is disposed therein. A paper discharge port 131 is installed in the back of the gap GAP, and a sheet is discharged from the paper discharge tray 150 to the paper discharge port 131. An operation panel 160 is attached to the front portion of the housing located beside the gap GAP. A touch panel 161 is embedded in the front surface of the operation panel 160, and various mechanical push buttons 162 are arranged around the touch panel 161. On the touch panel 161, an electrode of an electric field type 3D gesture sensor is stacked in addition to a touch pad and a thin film display.

[Operation panel with variable inclination with respect to the horizontal plane]
FIG. 1B is a side view of the operation panel 160. Referring to FIG. 1B, the operation panel 160 is variable in inclination with respect to the horizontal plane. Specifically, the back surface 163 of the operation panel 160 is swingably connected to a hinge portion 102 that protrudes forward from the casing 101 of the MFP 100. The back surface 163 includes a lever 164. By causing the user to pull the lever 164, the operation panel 160 moves vertically (around the virtual axis in FIG. 1) around a virtual axis passing through the hinge portion 102 in the horizontal direction (the normal direction of the paper surface in FIG. 1B). In (b), it swings in the vertical direction of the paper surface. Along with this swinging, the touch panel 161 embedded in the front surface of the operation panel 160 has an angle θ with respect to the horizontal plane because the screen has a substantially horizontal posture ((b) in FIG. 1 is indicated by a solid line). It changes to a posture inclined by (only (b) in FIG. 1 is indicated by a one-dot chain line). This angle, that is, the inclination angle θ can be adjusted within a range of about 0 ° to about 90 °, for example.

  FIG. 2 is a schematic diagram showing the relationship between the inclination angle θ of the operation panel 160 and the height of the user's back. Referring to FIG. 2A, a tall user generally adjusts the tilt angle θ of the operation panel 160 to near 0 ° and lays the screen of the touch panel 161 near horizontal. On the other hand, a short user generally adjusts the inclination angle θ to about 90 ° and raises the screen of the touch panel 161 to near vertical.

  These reasons are mainly because the inclination of the touch panel 161 with which the screen display is easy to see is related to the height of the user. In fact, a thin film display including the touch panel 161, such as a liquid crystal display (LCD), generally has the highest contrast near the center of the viewing angle, so that they are most clearly visible from the normal direction of the screen. As shown in FIG. 2A, a relatively tall user such as an adult has a high eye position. Therefore, if the inclination angle θ of the operation panel 160 is adjusted to around 0 °, the line of sight is made normal to the screen. Easy to match the direction. On the other hand, as shown in FIG. 2 (b), a relatively short user such as a wheelchair user, an elderly person, or a child is the opposite, and if the inclination angle θ is adjusted to around 90 °, the line of sight is displayed. Easy to align with the normal direction.

As described above, since the tilt angle of the operation panel 160 is variable, the screen display can be easily seen for any user regardless of the height of the user.
[Electronic control system of image processing device]
FIG. 3 is a block diagram showing the configuration of the electronic control system of MFP 100. Referring to FIG. 3, in this system, the image forming unit 10, the operation unit 20, the image input unit 26, and the main control unit 30 are connected to each other through a bus 40 so as to communicate with each other.

-Image forming section-
The image forming unit 10 is an element that forms an image on a sheet based on image data, and includes a feeding unit 11, an image forming unit 12, a fixing unit 13, and a paper discharge unit 14. The feeding unit 11 feeds sheets one by one from the sheet feeding cassette 140 to the image forming unit 12 using a transport roller group. The image forming unit 12 forms a toner image on the sheet sent from the feeding unit 11. The fixing unit 13 thermally fixes the toner image on the sheet sent from the image forming unit 12. The paper discharge unit 14 discharges the sheet on which the toner image is fixed to the paper discharge tray 150.

-Operation part-
The operation unit 20 is an input device (UI) installed in the MFP 100, interprets a user operation from a user gesture or the like, and notifies the main control unit 30. The operation unit 20 includes an operation panel 160. The operation panel 160 includes a detection unit 21, a measurement unit 22, a determination unit 23, a control unit 24, and a display unit 25.

  The detection unit 21 uses the touch pad to detect a touch on the touch pad by an object (hereinafter referred to as “indicator”) used by the user for an input operation on the MFP 100 such as a user's finger or a stylus. The detection unit 21 also generates an electric field in the space around the operation panel 160 using the electrode group of the electric field type 3D gesture sensor, and detects the distortion of the electric field due to the presence of the object in this space. This object includes, for example, a conductive indicator that is substantially grounded through the user's body, such as the user's hand, a metal stylus held in the hand. The measurement unit 22 measures the three-dimensional coordinates of the object from the electric field distortion detected by the detection unit 21. Based on the three-dimensional coordinates, the determination unit 23 determines whether the motion of the object corresponds to the user's gesture or the swing of the operation panel 160. Based on the detection result of the detection unit 21 and the determination result of the determination unit 23, the control unit 24 interprets the user's input operation using the touch pad or the push button or the input operation indicated by the user's three-dimensional gesture. , Information related to these input operations (hereinafter referred to as “operation information”) is transmitted to the main control unit 30. In response to an instruction from the main control unit 30, the display unit 25 displays a graphic user interface (GUI) screen such as an operation screen and an input screen for various information on the touch panel 161.

-Image input section-
In addition to the scanner 120, the image input unit 26 includes a memory interface (I / F) 27 and a network (LAN) I / F 28. The scanner 120 uses an optical device to irradiate a document captured by the ADF 110 or a document placed on a document table below the scanner 120, and reads characters, designs, or photographs from the intensity distribution of the reflected light. Convert to image data. The memory I / F 27 reads image data to be printed from an external storage device such as a USB memory or a hard disk drive (HDD) through a video input terminal such as a USB port or a memory card slot, or scans the image data into these storage devices. The image data captured at 120 is written out. The LAN I / F 28 is connected to an external LAN by wire or wirelessly, acquires image data to be printed from an electronic device on the LAN, or sends image data captured by the scanner 120 to the electronic device.

−Main control unit−
Main control unit 30 is an electronic circuit mounted on one printed circuit board installed inside MFP 100. Referring to FIG. 3, the main control unit 30 includes a CPU 31, a RAM 32, and a ROM 33. The CPU 31 implements various functions as a control subject for the other elements 10 and 20 by executing various firmware. For example, the CPU 31 displays a GUI screen such as an operation screen on the operation unit 20 to accept a user input operation. In response to this input operation, the CPU 31 determines an operation mode of the MFP 100 such as an operation mode, a standby mode, a sleep mode, and instructs the elements 10 and 20 to perform processing corresponding to the operation mode. The RAM 32 provides the CPU 31 with a work area when the CPU 31 executes firmware, and stores image data to be printed received by the operation unit 20. The ROM 33 includes a non-writable semiconductor memory device and a rewritable semiconductor memory device such as an EEPROM or an HDD. The former stores firmware, and the latter provides the CPU 31 with a storage area for environment variables and the like.

[Operation panel structure]
FIG. 4 is a block diagram showing an electronic control system of the operation panel 160. Referring to FIG. 4, the detection unit 21 includes a touch panel 161, a timing generation unit 411, a transmission unit 412, and a reception unit 413. The touch panel 161 includes a touch pad 4P, an LCD 4D, and electrode groups 4T, 4N, 4S, 4E, and 4W. These are stacked on a printed circuit board (PCB) 4G. The timing generation unit 411, the transmission unit 412, and the reception unit 413 are electronic circuits mounted on a printed circuit board built in the operation panel 160. Further, an AD conversion unit 42, a measurement unit 22, a determination unit 23, a control unit 24, and a communication unit 44 are mounted on this board. In particular, each of the measurement unit 22, the determination unit 23, and the control unit 24 is configured as one module included in the signal processing circuit. The control unit 24 further includes a touch monitoring unit 431, a display control unit 432, a button monitoring unit 433, a gesture interpretation unit 434, and an inclination angle estimation unit 435 as submodules.

  The touch pad 4P and the touch monitoring unit 431 configure, for example, a resistive film type touch sensor. The touch pad 4P includes a structure in which two upper and lower substrates are bonded to each other with a spacer in between. A plurality of thin strip-shaped transparent electrodes are arranged in parallel on each substrate. Since the longitudinal directions of these transparent electrodes are orthogonal to each other on the upper and lower substrates, these intersecting portions are distributed in a matrix on the entire touch pad 4P. When the indicator touches the touch pad 4P, the upper substrate is recessed at the contact portion, and the intersection of the upper and lower transparent electrodes is short-circuited. The touch monitoring unit 431 detects which pair of the upper and lower transparent electrodes is short-circuited, thereby detecting which crossing portion the indicator is in contact with. The touch monitoring unit 431 further identifies the type of a two-dimensional gesture such as tap or flick from the detected temporal displacement of the intersecting portion, collates the position of the intersecting portion with the screen display of the touch panel 161, and determines the virtual Interpret the input operation target indicated by the gesture, such as buttons and menu items. In this way, the touch monitoring unit 431 generates operation information and notifies the main control unit 30 of the operation information.

The LCD 4D and the display control unit 432 constitute the display unit 25 shown in FIG. The display control unit 432 generates image data representing a GUI screen such as an operation screen in accordance with an instruction from the main control unit 30 and sends it to the LCD 4D. The LCD 4D adjusts the luminance of each pixel based on this image data. As a result, a GUI screen is displayed on the screen of the touch panel 161.
The button monitoring unit 433 monitors whether various push buttons 162 on the operation panel 160 shown in FIG. 1 are pressed. When any one of the push buttons 162 is pressed, the button monitoring unit 433 identifies the button, interprets processing associated with the button, such as start / stop of printing, and generates operation information related thereto. Notify the main control unit 30.

  Electrode groups 4T, 4N,..., 4W included in touch panel 161, timing generation unit 411, transmission unit 412, reception unit 413, AD conversion unit 42, measurement unit 22, determination unit 23, gesture interpretation unit 434, and inclination angle estimation unit Reference numeral 435 constitutes an electric field type 3D gesture sensor 410. Specifically, the electrode groups 4T,..., 4W, the timing generation unit 411, the transmission unit 412, and the reception unit 413 generate a quasi-static electric field in the space around the touch panel 161, and the object such as the user's hand. The distortion of the electric field due to the presence is detected. The AD converter 42 evaluates this distortion with a digital value. The measuring unit 22 measures the three-dimensional coordinates of the object from this digital value. The determination unit 23 selects either the gesture interpretation unit 434 or the inclination angle estimation unit 435 as an output destination based on the three-dimensional coordinates. The gesture interpretation unit 434 identifies the type of three-dimensional gesture such as tap, flick, slide, rotation, and the like based on the temporal change of the three-dimensional coordinates received from the determination unit 23. The gesture interpretation unit 434 also collates the three-dimensional coordinates with the screen display of the touch panel 161 based on the tilt angle of the operation panel 160, and estimates GUI parts targeted by the user's gesture such as virtual buttons and menu items. . The gesture interpretation unit 434 further generates an operation information by interpreting an input operation indicated by the user's gesture based on the identified gesture type and the estimated GUI component, and notifies the main control unit 30 of the operation information. The tilt angle estimation unit 435 estimates the tilt angle θ of the operation panel 160 based on the three-dimensional coordinates received from the determination unit 23.

The communication unit 44 communicatively connects the control unit 24 to the bus 40 shown in FIG. 3, receives instructions, image data, and the like from the main control unit 30 through the bus 40, passes them to the control unit 24, and generates the control unit 24. Operation information or the like is transmitted to the main control unit 30 via the bus 40.
[Structure and Principle of Electric Field Type 3D Gesture Sensor]
5A is a plan view of the electrode groups 4T, 4N, 4S, 4E, and 4W of the electric field type 3D gesture sensor 410, and FIG. 5B is a cross-sectional view taken along a line bb shown in FIG. It is a partial enlarged view. Referring to FIGS. 5A and 5B, the PCB 4G has a rectangular shape, and has a first insulating layer 51, a transmission electrode 4T, a second insulating layer 52, and four reception electrodes 4N, 4S on its surface. 4E and 4W are laminated in order from the bottom. The first insulating layer 51, the transmission electrode 4T, and the second insulating layer 52 spread on the surface of the PCB 4G in the same planar shape. The first insulating layer 51 electrically separates the transmitting electrode 4T from the PCB 4G, and the second insulating layer 52 electrically separates the receiving electrodes 4N,..., 4W from the transmitting electrode 4T. The receiving electrodes 4N,..., 4W are arranged separately on each side of a virtual rectangular area extending on the surface of the second insulating layer 52. Each of the receiving electrodes 4N,... Has a thin strip shape and extends along the side of the rectangular area.

  Hereinafter, for convenience of explanation, when the inclination angle θ of the operation panel 160 shown in FIG. 1B is 0 °, that is, when the surface of the touch panel 161 is substantially horizontal, reception closest to the casing 101 of the MFP 100 is performed. The electrode 4N is referred to as a “first receiving electrode”, and the farthest receiving electrode 4S is referred to as a “second receiving electrode”. Furthermore, when the touch panel 161 is viewed from the front of the MFP 100, the reception electrode 4W located on the left side is referred to as a “third reception electrode”, and the reception electrode 4E located on the right side is referred to as a “fourth reception electrode”. Further, an XYZ orthogonal coordinate system fixed on the surface of the touch panel 161 is assumed as follows. The longitudinal direction of the first receiving electrode 4N and the second receiving electrode 4S is the X axis, the longitudinal direction of the third receiving electrode 4W and the fourth receiving electrode 4E is the Y axis, and the normal direction of the surface of the touch panel 161 is Z. Axis.

  FIG. 5C is an equivalent circuit diagram of the transmission electrode 4T, the fourth reception electrode 4E, the transmission unit 412, and the reception unit 413. The other receiving electrodes 4N, 4S, and 4W are also included in a similar equivalent circuit. Referring to FIG. 5C, the PCB 4G is connected to a grounded conductive member (hereinafter referred to as “ground member”), such as the chassis of the MFP 100, and the transmission electrode 4T is connected to the transmission unit 412. The fourth receiving electrode 4E is connected to the receiving unit 413. The transmission unit 412 sends a transmission voltage signal TVS to the transmission electrode 4T. Transmission voltage signal TVS is, for example, a rectangular pulse wave. The receiving unit 413 detects the reception voltage signal RVS received by the fourth reception electrode 4E according to the transmission voltage signal TVS. Transmission of the transmission voltage signal TVS by the transmission unit 412 and detection of the reception voltage signal RVS by the reception unit 413 are synchronized by a clock signal generated by the timing generation unit 411 shown in FIG.

  The waveform of the reception voltage signal RVS is generally distorted with respect to the waveform of the transmission voltage signal TVS. For example, when the transmission voltage signal TVS is a rectangular pulse, the rising and falling edges of the reception voltage signal RVS are both dull. Such waveform distortion is mainly caused by the capacitance CTG between the transmission electrode 4T and the substrate 4G (or ground potential), the capacitance CET between the fourth reception electrode 4E and the transmission electrode 4T, and the fourth. This is caused by the capacitance CEG between the reception electrode 4E and the substrate 4G (or ground potential).

  When a certain object approaches the fourth reception electrode 4E, a new capacitance CH is added between the fourth reception electrode 4E and the ground potential. This object, such as a user's hand, a grounded metal member, etc., has electrical conductivity such that the overall potential can be considered substantially uniform compared to the level of the transmission voltage signal TVS, and the level of the transmission voltage signal TVS. Is an object that is maintained at a potential that can be regarded as substantially equal to the ground potential (hereinafter collectively referred to as “ground conductor”). Since the electrostatic capacitance CH resulting from the presence of the ground conductor changes in accordance with the relative position of the ground conductor and the fourth reception electrode 4E, the waveform distortion of the reception voltage signal RVS also changes in accordance with this position. To do. In particular, the closer the ground conductor is to the fourth reception electrode 4E, the larger the drop in the level of the reception voltage signal RVS with respect to the transmission voltage signal TVS.

  6A to 6C are generated between the transmission electrode 4T and the substrate 4G (or ground potential) when a positive voltage with respect to the ground potential is applied to the transmission electrode 4T. It is a side view which shows the electric field EFL. 6A shows a case where no ground conductor exists near the touch panel 161. FIGS. 6B and 6C show an indicator HND such as a user's hand and other ground conductors MTL in the vicinity of the touch panel 161, respectively. Indicates the presence of.

  Referring to FIG. 6A, the electric field EFL generated in the space around the touch panel 161 is symmetric with respect to the surface of the touch panel 161. Here, when the frequency of the transmission voltage signal TVS is set to, for example, about several tens of kHz to several hundreds of kHz, the wavelength reaches about several km, and is larger than the size of the electrode groups 4T, 4E,. Is big enough. Therefore, the electric field generated in the space around the touch panel 161 when the transmission voltage signal TVS is applied to the transmission electrode 4T is quasi-static and may be regarded as being equivalent to the electric field EFL shown in FIG.

  6B and 6C, unlike FIG. 6A, the electric field EFL is distorted around the indicator HND, the ground conductor MTL, and the like. In particular, since the indicator HND and the conductor MTL are substantially at the ground potential, the electric lines of force accompanying the electric field EFL are drawn into the ground conductors HND and MTL and escape to the ground electrode through them. As a result, the distortion of the electric field EFL increases as the ground conductors HND and MTL are approached. If the size of the ground conductors HND and MTL is sufficiently smaller than the size of the substrate 4G, the distortion of the electric field EFL caused by the ground conductors HND and MTL remains locally with respect to the size of the substrate 4G. Therefore, there is a difference in potential difference between the four receiving electrodes 4N, 4S, 4E, and 4W from the case of FIG. 6A according to the difference in distance from the ground conductors HND and MTL. In FIGS. 6B and 6C, since the third receiving electrode 4W is closer to the ground conductors HND and MTL than the fourth receiving electrode 4E, the potential difference from the case of FIG. 6A is the third receiving electrode 4W. Is larger than the fourth receiving electrode 4E.

  This potential difference, that is, the potential difference according to the presence or absence of the ground conductor is different for each of the receiving electrodes 4N,..., 4W, in the equivalent circuit shown in FIG. Appears as a difference in CH. As a result, the difference in waveform distortion of the reception voltage signal RVS depending on the presence or absence of the capacitance CH, that is, the presence or absence of the ground conductor, differs for each of the reception electrodes 4N,. In particular, the difference in distortion is larger as the receiving electrode is closer to the ground conductor.

  FIG. 6D is a graph showing the difference in distortion of the waveform of the reception voltage signal RVS for each reception electrode 4N,. The vertical axis of this graph represents the amount of change in the drop in the level of the reception voltage signal RVS with respect to the transmission voltage signal TVS, for example, as a relative difference, and the horizontal axis represents the representative ground conductor from the third reception electrode 4W. The horizontal distance X of the point, for example, the center of gravity is represented. Further, in this graph, the first receiving electrode 4N and the second receiving electrode 4N are connected to the fourth receiving electrode 4E from the position just above the third receiving electrode 4W while maintaining the center of gravity of the ground conductor at a constant distance from the surface of the touch panel 161. It is assumed that the electrode moves parallel to the receiving electrode 4S.

  Referring to FIG. 6D, when the center of gravity of the ground conductor is located directly above the third receiving electrode 4W, that is, at the horizontal distance X = 0, the third receiving electrode 4W is closest to the center of gravity. The fourth receiving electrode 4E is located farthest. Therefore, the amount of change in the drop of the level of the reception voltage signal RVS is the largest at the third reception electrode 4W and the smallest at the fourth reception electrode 4E. Since the center of gravity of the ground conductor is also closer to the first reception electrode 4N than to the second reception electrode 4S, the amount of change in the drop in the level of the reception voltage signal RVS is larger in the first reception electrode 4N than in the second reception electrode 4S.

  The center of gravity of the ground conductor moves away from the third reception electrode 4W as the horizontal distance X from the third reception electrode 4W increases, but approaches the fourth reception electrode 4E. Accordingly, the amount of change in the drop in the level of the reception voltage signal RVS decreases at the third reception electrode 4W, but increases at the fourth reception electrode 4E. Since the center of gravity of the ground conductor maintains the same distance with respect to the first reception electrode 4N and the second reception electrode 4S, the amount of change in the drop of the level of the reception voltage signal RVS is always higher than that of the second reception electrode 4S. Large at the receiving electrode 4N. Further, the closer the center of gravity of the ground conductor is to the intermediate point (horizontal distance X = WDT / 2) between the third receiving electrode 4W and the fourth receiving electrode 4E, the more the first receiving electrode 4N and the second receiving electrode 4S are penetrated. Since many electric lines of force are drawn into the ground conductor, the amount of change in the drop in the level of the received voltage signal RVS increases in any of the electrodes 4N and 4S. As shown in FIG. 5A, the arrangement of the electrode groups 4T, 4N,..., 4W is symmetrical with respect to the central axis passing through the substrate 4G in the Y-axis direction. Is also symmetric with respect to the intermediate point X = WDT / 2.

  As shown in FIG. 6D, the amount of change in the drop in the level of the reception voltage signal RVS differs between the reception electrodes 4N,..., 4W depending on the distance from the center of gravity of the ground conductor. Accordingly, a table or function indicating this correspondence relationship is obtained by associating the three-dimensional coordinates of the center of gravity of the ground conductor with each combination of the amounts of change between the four receiving electrodes 4N,. It is incorporated in the determination unit 23. Using this table or function, the determination unit 23 specifies the corresponding three-dimensional coordinates of the center of gravity of the ground conductor from the combination of the amounts of change detected from each receiving electrode 4N.

[Limit of detection by electric field type 3D gesture sensor]
FIG. 7A is a side view showing the range of the three-dimensional coordinates of the ground conductor that can be detected by the electric field type 3D gesture sensor 410. Referring to (a) of FIG. 7, this range is not only a space URG facing the screen of the touch panel 161 (hereinafter referred to as “front side space”), but also a space extending on the opposite side across the touch panel 161. It extends to LRG (hereinafter referred to as “back space”). This is because the electric field EFL generated by the transmission electrode 4T spreads not only in the front space URG but also in the back space LRG as shown in FIG.

However, the 3D gesture sensor 410 cannot determine whether the ground conductor is located in the front side space URG or the back side space LRG. Furthermore, it is not easy to simultaneously identify a plurality of ground conductors. These are due to the following reasons.
FIG. 7B is a side view showing a virtual range of the three-dimensional coordinates of the ground conductor that can be identified by detection by the 3D gesture sensor 410. Referring to FIG. 7B, any one point in this virtual range includes one point (X0, Y0, Z0) of the front side space URG shown in FIG. The two points correspond to the symmetry points (X0, Y0, -Z0) of the back side space LRG with respect to the surface. This means that these two points are indistinguishable.

  Actually, as shown in FIG. 6A, the distribution of the electric field EFL generated by the transmission electrode 4T is symmetric with respect to the surface of the touch panel 161 unless the ground conductor enters. In this case, when the ground conductor is located at one point (X0, Y0, Z0) of the front space URG and when it is located at the symmetrical point (X0, Y0, -Z0), reception due to distortion of the electric field EFL is received. There is no difference in the distortion of the waveform of the voltage signal RVS in any of the receiving electrodes 4N. Therefore, these two points cannot be distinguished only from the waveform distortion of the reception voltage signal RVS.

  (C) of FIG. 7 is a side view showing two different ground conductors D1 and D2 located at two points (X1, Y1, Z1) and (X2, Y2, Z2) of the front side space URG shown in (a). (D) is a side view showing one ground conductor D3 located at another point (X3, Y3, Z3) of the front side space URG. Referring to FIG. 7C, the electric field EFL generated by the transmission electrode 4T is distorted in the vicinity of each of the two ground conductors D1 and D2. However, in this case, in the reception voltage signal RVS detected by each reception electrode 4N,..., The waveform distortion accompanying the distortion of the electric field EFL in the vicinity of each ground conductor D1, D2 is synthesized. This combined distortion may be indistinguishable from the distortion of the waveform of the reception voltage signal RVS accompanying the distortion of the electric field EFL in the vicinity of the ground conductor D3 shown in FIG. In this case, whether or not different ground conductors D1 and D2 are located at two points (X1, Y1, Z1) and (X2, Y2, Z2) only from the waveform distortion of the reception voltage signal RVS, or another point ( It cannot be distinguished whether one ground conductor D3 is located at X3, Y3, Z3).

The detection by the electric field type 3D gesture sensor 410 has these limitations. Therefore, in order to use this sensor, a device for avoiding these limitations is required.
[Distinction between user gestures and swinging operation panel by discrimination unit]
FIG. 8A is a schematic diagram showing a space to be monitored set within the range of the three-dimensional coordinates of the ground conductor that can be detected by the electric field type 3D gesture sensor 410. Referring to (a) of FIG. 8, the monitoring target space includes a first area RGA, a second area RGB, and a third area RGC. The first region RGA is a region closer to the surface (Z = 0) of the touch panel 161 than the virtual plane BNA (Z = ZB) in the front side space URG shown in FIG. 7A, and the third region RGC is This is a region farther from the surface of the touch panel 161 than the plane BNA. The second area RGB is an area farther from the surface of the touch panel 161 than the virtual plane BNB (Z = −ZB) in the back space LRG. These two virtual planes BNA and BNB are symmetric with respect to the surface (Z = 0) of the touch panel 161.

  Further referring to FIG. 8A, a conductive member 801 is attached to the casing 101 of the MFP 100. The housing 101 is made of an insulating resin, whereas the conductive member 801 is made of a conductive material, for example, a rectangular metal plate. The conductive member 801 is further maintained at a ground potential by a ground member such as a chassis of the MFP 100. When the inclination angle θ of the operation panel 160 is 0 °, the conductive member 801 is located in the second region RGB.

  FIG. 8B is a schematic diagram showing a virtual range of the three-dimensional coordinates of the ground conductor that can be identified by detection by the 3D gesture sensor 410. Referring to (b) of FIG. 8, similarly to (b) of FIG. 7, one point (X0, Y0, Z0) of the front side space URG and the back side space LRG are included in any one point of this virtual range. The two points correspond to the symmetry points (X0, Y0, -Z0). Therefore, in this virtual range, each point in the region (Z <ZB) closer to the surface (Z = 0) of the touch panel 161 than the virtual plane BND (Z = ZB; hereinafter referred to as “boundary surface”). Corresponds to only one point of the first region RGA in the space to be monitored shown in FIG. On the other hand, at each point of the region (Z> ZB) farther from the surface of the touch panel 161 than the boundary surface BND in the virtual range, the third region RGC in the space to be monitored shown in FIG. Correspond to two points of the second region RGB and the symmetry point of the second region RGB.

  Using this correspondence, the determination unit 23 determines whether the distortion of the electric field EFL detected by the detection unit 21 is caused by the indicator HND or the conductive member 801. Specifically, first, the measurement unit 22 measures the three-dimensional coordinates of the center of gravity of the ground conductor from the waveform distortion of the reception voltage signal RVS detected by the detection unit 21 with the reception electrodes 4N,. If the three-dimensional coordinates are located on the boundary surface BND or closer to the surface of the touch panel 161 than the three-dimensional coordinates, the determination unit 23 connects the ground conductor located at the three-dimensional coordinates to the indicator HND located in the first region RGA. Determine.

  Here, as shown in FIGS. 8A and 8B, the conductive member 801 already exists in the second region RGB. However, the second region RGB is farther from the receiving electrodes 4N,... Than the first region RGA. Therefore, the distortion of the electric field EFL caused by the conductive member 801 located in the second region RGB is negligible in the contribution to the distortion of the waveform of the reception voltage signal RVS compared to that caused by the indicator HND located in the first region RGA. As small as possible. That is, if the three-dimensional coordinates measured by the measuring unit 22 are located on the boundary surface BND or closer to the surface of the touch panel 161 than that, the coordinates are regarded as indicating the actual position of the indicator HND within the allowable error range. It's okay.

  If the three-dimensional coordinate measured by the measurement unit 22 is farther from the surface of the touch panel 161 than the boundary surface BND, the determination unit 23 (A) conducts the ground conductor located at the three-dimensional coordinate to the conductive member 801 located in the second region RGB. Or (B) further observing the temporal change of the three-dimensional coordinate, and based on the width of the temporal change, the grounding conductor positioned at the three-dimensional coordinate is a conductive member positioned in the second region RGB. It is discriminated between 801 and an indicator located in the third region RGC. Details of these determination methods (A) and (B) will be described later.

  FIG. 8C is a schematic diagram showing the monitoring target spaces RGA, RGB, and RGC when the surface of the touch panel 161 is inclined with respect to the horizontal plane due to the swing of the operation panel 160. Referring to FIG. 8C, the back surface of the operation panel 160 approaches the conductive member 801 as the inclination angle θ of the operation panel 160 increases from 0 °. As a result, the second region RGB moves relative to the conductive member 801.

  FIG. 8D is a schematic diagram showing a correspondence relationship between the space to be monitored shown in FIG. 8C and the virtual range of the three-dimensional coordinates of the ground conductor that can be identified by detection by the 3D gesture sensor 410. Referring to FIG. 8D, in this virtual range, the conductive member 801 has a surface of the touch panel 161 (Z = Z) rather than the boundary surface BND (Z = ZB) with a change in the inclination angle θ of the operation panel 160. A constant trajectory TRC is drawn in a region far from 0). If the indicator HND does not exist in the monitoring target space, the three-dimensional coordinates measured by the measuring unit 22 are located on the trajectory TRC. Further, since each point on the trajectory TRC corresponds to a different value of the inclination angle θ of the operation panel 160, if this correspondence is used, the inclination angle θ of the operation panel 160 is calculated from the three-dimensional coordinates measured by the measurement unit 22. Can be estimated.

-Discrimination method (A)-
When the three-dimensional coordinate measured from the distortion of the waveform of the received voltage signal RVS is farther from the surface of the touch panel 161 than the boundary surface BND, the ground conductor positioned at the three-dimensional coordinate is the conductive member 801 positioned in the second region RGB. Determined.
FIG. 9A is a graph showing an example of a temporal change in the Z-axis component of the three-dimensional coordinate measured from the waveform distortion of the reception voltage signal RVS. The vertical axis of this graph represents the Z-axis component, that is, the distance from the surface of the touch panel 161, and the horizontal axis represents time. Referring to FIG. 9A, in the period from the first time t0 to the second time t1, the Z-axis component is smaller than the Z coordinate “ZB” of the boundary surface. Therefore, the determination unit 23 determines the ground conductor located at the three-dimensional coordinates measured in the period t0-t1 as the indicator HND located in the first region RGA. At the second time t1, the Z-axis component exceeds the Z coordinate “ZB” of the boundary surface, and remains larger than the Z coordinate “ZB” of the boundary surface even after a predetermined time ΔTA has elapsed from the second time t1. Therefore, the determination unit 23 determines the ground conductor located at the three-dimensional coordinates measured in the period t1−t1 + ΔTA as the conductive member 801 located in the second region RGB. Thereafter, since the Z-axis component again falls below the Z coordinate “ZB” of the boundary surface at the third time t2, the determination unit 23 determines the ground conductor located at the three-dimensional coordinate measured after the third time t2 in the first region. Discriminate from the indicator HND located in the RGA.

  In the discrimination method (A), as described above, first, the Z coordinate “ZB” of the boundary surface is set as a threshold for the Z-axis component of the measured three-dimensional coordinate. If the measured Z-axis component of the three-dimensional coordinates is equal to or smaller than the threshold value ZB, the three-dimensional coordinates are immediately determined as the position of the indicator HND. On the other hand, if the Z-axis component continues to exceed the predetermined time ΔTA and the threshold value ZB, the three-dimensional coordinate is determined as the position of the conductive member 801. When the time during which the Z-axis component exceeds the threshold value ZB is shorter than the predetermined time ΔTA, the three-dimensional coordinates are excluded from the determination target.

  The reason for adding “continuation of the predetermined time ΔTA” to the latter condition is to avoid the risk of misidentifying the indicator HND that has entered the third region RGC as the conductive member 801. As shown in FIGS. 8B and 8D, the region farther from the surface of the touch panel 161 than the boundary surface BND in the virtual range of the three-dimensional coordinates of the ground conductor that can be identified by detection by the 3D gesture sensor 410. (Z> ZB) corresponds to the third region RGC as well as the second region RGB. There is a possibility that the indicator HND enters the third region RGC. However, if the third region RGC is sufficiently away from the surface of the touch panel 161, the indicator HND enters the region through the passage for the purpose of moving to the first region RGA or withdrawing from the first region RGA, or the gesture. It is limited to those that protrude from the first region RGA unintentionally by the user. Therefore, since the duration of this intrusion is generally short, if the predetermined time ΔTA is set sufficiently long, the risk of misidentifying the indicator HND as the conductive member 801 can be sufficiently reduced.

-Discrimination method (B)-
When the three-dimensional coordinate measured from the waveform distortion of the received voltage signal RVS is farther from the surface of the touch panel 161 than the boundary surface BND, the determination unit 23 further observes the temporal change of the three-dimensional coordinate, and the temporal change thereof. Based on the width of the grounding conductor, the grounding conductor located at the three-dimensional coordinate is determined as either the conductive member 801 located in the second region RGB or the indicator located in the third region RGC.

  FIG. 9B is a graph showing another example of the temporal change of the Z-axis component of the three-dimensional coordinate measured from the waveform distortion of the reception voltage signal RVS. The vertical axis of this graph represents the Z-axis component, that is, the distance from the surface of the touch panel 161, and the horizontal axis represents time. Referring to FIG. 9B, in the period from the first time t0 to the second time t1, the Z-axis component is smaller than the Z coordinate “ZB” of the boundary surface. Therefore, the determination unit 23 determines the ground conductor located at the three-dimensional coordinates measured in the period t0-t1 as the indicator HND located in the first region RGA. Since the Z-axis component exceeds the Z coordinate “ZB” of the boundary surface at the second time t1, the determination unit 23 uses the three-dimensional coordinates measured by the measurement unit 22 during the period until the predetermined time ΔTB elapses from the second time t1. Observe the time change of the monitor, especially monitor the width of the change. Since this width is smaller than the predetermined value ΔFR, the determination unit 23 determines the ground conductor located at the three-dimensional coordinates measured in the period t1−t1 + ΔTB as the conductive member 801 located in the second region RGB. Thereafter, at the third time t2, the Z-axis component starts to fluctuate in a range larger than the Z coordinate “ZB” of the boundary surface. Therefore, during the period from the third time t2 until the predetermined time ΔTB elapses, the determination unit 23 performs measurement. The temporal change of the three-dimensional coordinates measured by the unit 22 is observed. In this period t2-t2 + ΔTB, the change width of the three-dimensional coordinate exceeds the predetermined value ΔFR, and therefore the determination unit 23 determines the ground conductor located at the three-dimensional coordinate as the indicator HND located in the third region RGC. .

  In the discrimination method (B), as in the discrimination method (A), if the Z-axis component of the measured three-dimensional coordinates is equal to or less than the threshold, that is, the Z coordinate “ZB” of the boundary surface, the three-dimensional coordinates are immediately indicated by the indicator HND. It is determined that On the other hand, in the discrimination method (B), unlike the discrimination method (A), when the Z-axis component exceeds the threshold value ZB, the change in the three-dimensional coordinates measured from the time point until the predetermined time ΔTB elapses. If the width is less than or equal to a predetermined value ΔFR, the three-dimensional coordinate is determined as the position of the conductive member 801, and if it exceeds the predetermined value ΔFR, it is determined as the position of the indicator HND.

In FIG. 9B, the width of change in the Z-axis direction is compared with a predetermined value ΔFR for convenience of explanation. However, the measured three-dimensional coordinates are determined as the position of the indicator HND if the change width exceeds the predetermined value ΔFR in any direction, not limited to the Z-axis direction.
The discrimination method (B) is different from the discrimination method (A). For example, the third region RGC cannot be sufficiently separated from the surface of the touch panel 161. As a result, the indicator HND in the gesture frequently enters the region RGC. Even when it is obtained, it is possible to distinguish between the conductive member 801 located in the second region RGB and the indicator HND located in the third region RGC. Certainly, the second region RGB and the third region RGC have the same distance from the receiving electrode 4N,..., And therefore the contribution to the waveform distortion of the received voltage signal RVS is caused by the conductive member 801 located in the second region RGB. The distortion of the electric field EFL caused by the distortion is approximately the same as that caused by the indicator HND located in the third region RGC. However, since the conductive member 801 is fixed to the casing of the MFP 100, it remains substantially stationary, whereas the indicator HND continues to move by an operation such as a gesture. Therefore, the temporal change of the three-dimensional coordinates measured when the indicator HND enters the third region RGC is superimposed with a fluctuation component having a short period and a large amplitude due to the movement of the indicator HND. The On the other hand, the conductive member 801 does not greatly deviate from the constant trajectory TRC shown in FIG. In consideration of these, if the predetermined time ΔTB is sufficiently short and the predetermined value ΔFR is set sufficiently large, the conductive member 801 located in the second region RGB from the temporal change of the three-dimensional coordinates to be measured. And the indicator HND located in the third region RGC.

-Flow of processing by discrimination method (A)-
FIG. 10 is a flowchart of processing by the determination method (A). This process is started when the detection unit 21 detects the reception voltage signal RVS.
In step S101, the measurement unit 22 reads the digitized reception voltage signal RVS from the AD conversion unit 42, analyzes the distortion of the waveform, and the distortion is a ground conductor to the monitored space RGA, RGB, RGC. It is confirmed whether or not it includes changes caused by the intrusion. If this change is included, the process proceeds to step S102; otherwise, the process ends.

In step S102, the measurement unit 22 measures the three-dimensional coordinate POS of the center of gravity of the ground conductor from the waveform distortion of the reception voltage signal RVS. Thereafter, the process proceeds to step S103.
In step S103, the determination unit 23 checks whether the three-dimensional coordinate POS measured by the measurement unit 22 is farther from the surface of the touch panel 161 than the boundary surface BND. If so, the process proceeds to step S104. If not, the process proceeds to step S106.

In step S104, the three-dimensional coordinate POS measured by the measurement unit 22 is farther from the surface of the touch panel 161 than the boundary surface BND. The determination unit 23 confirms whether or not the predetermined time ΔTA has elapsed since the time when this was first confirmed. If the predetermined time ΔTA has elapsed, the process proceeds to step S105, and if not, the process is repeated from step S101.
In step S105, even when the predetermined time ΔTA has elapsed, the three-dimensional coordinate POS measured by the measurement unit 22 is farther from the surface of the touch panel 161 than the boundary surface BND. Therefore, the determination unit 23 determines the three-dimensional coordinate POS as the coordinates of the conductive member 801 located in the second region RGB. Thereafter, the process ends.

In step S106, the three-dimensional coordinate POS measured by the measurement unit 22 is closer to the surface of the touch panel 161 than the boundary surface BND. Therefore, the determination unit 23 determines the three-dimensional coordinates POS as the coordinates of the indicator HND located in the first region RGA. Thereafter, the process ends.
-Flow of processing by discrimination method (B)-
FIG. 11 is a flowchart of processing by the determination method (B). This process is started when the detection unit 21 detects the reception voltage signal RVS.

In step S111, the determination unit 23 sets the flag FL to “0”. Thereafter, the process proceeds to step S112.
In step S112, the measurement unit 22 reads the digitized reception voltage signal RVS from the AD conversion unit 42, analyzes the distortion of the waveform, and the distortion is grounded to the monitored spaces RGA, RGB, and RGC. It is confirmed whether or not it includes changes caused by the intrusion. If this change is included, the process proceeds to step S113; otherwise, the process ends.

In step S113, the measurement unit 22 measures the three-dimensional coordinate POS of the center of gravity of the ground conductor from the waveform distortion of the reception voltage signal RVS. Thereafter, the process proceeds to step S114.
In step S114, the determination unit 23 checks whether the three-dimensional coordinate POS measured by the measurement unit 22 is farther from the surface of the touch panel 161 than the boundary surface BND. If far away, the process proceeds to step S115, and if not far, the process proceeds to step S121.

In step S115, the three-dimensional coordinate POS measured by the measurement unit 22 is farther from the surface of the touch panel 161 than the boundary surface BND. At this time, the determination unit 23 checks whether or not the flag FL is “0”. If "0", the process proceeds to step S116, and if not "0", the process proceeds to step S119.
In step S116, since the flag FL is “0”, the three-dimensional coordinate POS is a value initially measured by the measurement unit 22 after the processing is started. The determination unit 23 sets the value POS to the initial value PPS and changes the flag FL to “1”. Thereafter, the process proceeds to step S117.

In step S117, the determination unit 23 checks whether or not a predetermined time ΔTB has elapsed since the change in the waveform distortion confirmed in step S112 occurred at the start of processing. If the predetermined time ΔTB has elapsed, the process proceeds to step S118, and if not, the process is repeated from step S112.
In step S118, the three-dimensional coordinate POS measured by the measurement unit 22 is more in touch with the touch panel 161 than the boundary surface BND even when the predetermined time ΔTB has elapsed since the change in waveform distortion confirmed in step S112 occurred at the start of processing. It is far from the surface. Therefore, the determination unit 23 determines the three-dimensional coordinate POS from the coordinates of the conductive member 801 located in the second region RGB. Thereafter, the process ends.

In step S119, since the flag FL is not "0", a value has already been set to the initial value PPS. Therefore, the determination unit 23 calculates the change width MOV = | POS−PPS | between the three-dimensional coordinate POS measured in step S113 and the initial value PPS. Thereafter, the process proceeds to step S120.
In step S120, the determination unit 23 checks whether the change width MOV calculated in step S119 is equal to or less than a predetermined value ΔFR. If it is equal to or smaller than the predetermined value ΔFR, the process proceeds to step S117, and if it exceeds the predetermined value ΔFR, the process proceeds to step S121.

In step S121, the three-dimensional coordinate POS measured in step S113 is closer to the surface of the touch panel 161 than the boundary surface BND. Further, the change width MOV calculated in step S119 exceeds the predetermined value ΔFR. Therefore, the determination unit 23 determines the three-dimensional coordinates POS measured in step S113 from the coordinates of the indicator HND. Thereafter, the process ends.
[Processing according to the tilt angle of the operation panel]
When the three-dimensional coordinates measured by the measurement unit 22 are determined as the coordinates of the conductive member 801, the determination unit 23 passes the three-dimensional coordinates to the inclination angle estimation unit 435 shown in FIG. Based on the three-dimensional coordinates, the inclination angle estimation unit 435 estimates the inclination angle θ of the operation panel 160. Specifically, it is estimated which point on the locus TRC of the conductive member 801 shown in FIG. 8D corresponds to this three-dimensional coordinate, and the inclination angle θ of the operation panel 160 corresponding to this point is determined. Then, a search is made from a correspondence table between the position on the trajectory TRC and the inclination angle of the operation panel.

  The control unit 24 controls, for example, the following two types of processing according to the inclination angle θ of the operation panel 160 estimated by the inclination angle estimation unit 435. (1) The display unit 25 is caused to change the display attribute of the LCD 4D according to the tilt angle θ of the operation panel. (2) The gesture interpretation unit 434 estimates coordinates on the operation screen targeted by the user's gesture (hereinafter referred to as “target coordinates”) based on the inclination angle θ of the operation panel 160.

-Changing display attributes-
12A is a correspondence table between the display attributes of the LCD 4D, the detection characteristics of the 3D gesture sensor 410, and the inclination angle θ of the operation panel 160. FIG. This table is stored, for example, in a memory element built in the control unit 24. Referring to FIG. 12A, first, the range in which the inclination angle θ of the operation panel 160 can be taken is a first section (θ <θB) where the range is less than the threshold θB and a second section (θ ≧ θB) beyond that. It is divided in two. Next, in the first section, the screen display size of the LCD 4D and the speed of the gesture that can be detected by the 3D gesture sensor 410 are both set to “standard”, and the types of gestures that can be detected are “flick”, “ “Slide” and “Rotation” are both set to “Yes”. On the other hand, in the second section, the screen display size is set to “enlarged”, the speed of the detectable gesture is set to “slow”, and among the types of detectable gestures, “flick” is changed to “slide” "Is set.

  When the tilt angle estimation unit 435 estimates the tilt angle θ of the operation panel 160, the control unit 24 first determines whether the estimated value belongs to the first section (θ <θB) or the second section (θ ≧ θB). Is determined. Next, the control unit 24 retrieves the display attribute of the LCD 4D corresponding to the determination result and the characteristics detected by the 3D gesture sensor 410 from the above correspondence table, and sets them in the display unit 25 and the gesture interpretation unit 434.

  FIGS. 12B and 12C show the operation panel 160 when the inclination angle θ of the operation panel 160 is estimated to belong to the first section (θ <θB) and the second section (θ ≧ θB), respectively. It is a schematic diagram which shows the character displayed on a screen. Referring to (b) and (c) of FIG. 12, when it is estimated that the inclination angle θ belongs to the second section rather than the character displayed on the screen when it is estimated to belong to the first section. The characters are enlarged in size. This corresponds to the fact that the screen display size of the LCD 4D is set to “standard” for the first section and “enlarged” for the second section in the correspondence table of FIG. By the processing of the display unit 25.

  In the correspondence table of FIG. 12A, the speed of the gesture that can be detected by the 3D gesture sensor 410 is set to “standard” for the first section and “slow” for the second section. In response to this, the gesture interpretation unit 434 sets the lower limit of the speed of the indicator assumed when interpreting the temporal change of the three-dimensional coordinates received from the determination unit 23 as the user's gesture, and the inclination angle θ is set to the first interval. It is set lower when it is estimated that it belongs to the second section than when it is estimated that it belongs.

  In the correspondence table of FIG. 12A, “flick” among the types of gestures detectable by the 3D gesture sensor 410 is set to “change to slide” for the second section. In response to this, the gesture interpretation unit 434 is estimated that the inclination angle θ belongs to the second section even when the temporal change of the three-dimensional coordinates received from the determination unit 23 can be interpreted as “flick”. Sometimes interpreted as “slide”.

  As described above, the control unit 24 changes the display attribute of the LCD 4D and the detection characteristic of the 3D gesture sensor 410 according to the tilt angle θ of the operation panel. Thereby, those attributes can be automatically adapted to individual users. Actually, the inclination angle θ of the operation panel 160 is set to the first section in many cases, as shown in FIG. 2A, by a relatively tall user. On the other hand, the inclination angle θ of the operation panel 160 is set to the second section in many cases as shown in FIG. 2B by a relatively short user such as a wheelchair user. Therefore, especially when it is estimated that the inclination angle θ belongs to the second section, the screen display of the operation panel 160 is changed to a setting that can be easily seen by a wheelchair user, an elderly person, a child, and the like, and is detected by the 3D gesture sensor 410. You can adjust the characteristics of the hand to a relatively slow hand movement. Specifically, as described above, when the inclination angle θ is estimated to belong to the second section, the size of the character displayed on the screen is enlarged, and the speed of the indicator assumed when interpreting as a gesture is increased. The lower limit is set low, and a gesture that can be interpreted as “flick” may be interpreted as “slide”.

-Estimation of gesture target-
FIG. 13D is a table showing target coordinate correction values according to the inclination angle θ of the operation panel 160. This table is stored, for example, in a memory element built in the control unit 24. Referring to (d) of FIG. 13, first, the range in which the inclination angle θ of the operation panel 160 can be taken is a first interval (θ ≦ θ1) equal to or less than the first threshold θ1, larger than the first threshold θ1, and the second threshold. It is divided into a second section (θ1 <θ <θ2) smaller than θ2 and a third section (θ2 ≦ θ) equal to or greater than the second threshold value θ2. Next, the correction value (ΔX, ΔY) of the target coordinate is set to (0, + Δξ) in the first section, (0, 0) in the second section, and (0, −Δη) in the third section. Is set to

  When the tilt angle estimation unit 435 estimates the tilt angle θ of the operation panel 160, the control unit 24 first determines whether the estimated value belongs to the first section, the second section, or the third section. Next, the control unit 24 retrieves the correction value of the target coordinate corresponding to the determination result from the table of FIG. 13D and sets it in the gesture interpretation unit 434. In response to this, the gesture interpretation unit 434 first estimates the three-dimensional coordinates (X, Y, Z) of the fingertip from the detection result by the 3D gesture sensor 410. Next, the gesture interpretation unit 434 adds the correction value (ΔX, ΔY) to the coordinates (X, Y) of the point that the fingertip is projected perpendicularly to the screen, and uses the obtained coordinates (X + ΔX, Y + ΔY) as the target coordinates. Estimate as The gesture interpretation unit 434 interprets the GUI part of the operation screen displayed at the target coordinates as a gesture target.

In this way, the gesture interpretation unit 434 estimates target coordinates based on the tilt angle θ of the operation panel 160. As a result, as illustrated below, the accuracy of identifying the GUI component that is the target of the gesture is improved, so that the input operation indicated by the gesture can be accurately interpreted.
In FIGS. 13A, 13B, and 13C, the inclination angle θ of the operation panel 160 is set to the first interval (θ ≦ θ1), the second interval (θ1 <θ <θ2), and the third interval ( It is a schematic diagram showing the user's lines of sight LS1, LS2, and LS3 directed to the screen of the operation panel 160 when belonging to (θ2 ≦ θ).

  Referring to FIG. 13A, when it is estimated that the inclination angle θ belongs to the first section, the screen of the operation panel 160 is close to the horizontal plane, and therefore the line of sight LS1 directed toward the user's fingertip FP1 is the screen. Tilt diagonally from the normal direction. As a result, the point TP1 on the screen where the line of sight LS1 intersects deviates from the point MP1 obtained by projecting the user's fingertip FP1 perpendicularly to the screen by a distance Δξ in the positive direction of the Y axis. On the other hand, the target coordinates estimated by the gesture interpreter 434 are the coordinates (X, Y) of the projection point MP1 projected from the fingertip FP1 to the screen obtained from the detection result by the 3D gesture sensor 410, as shown in the table of (d) of FIG. (X, Y + Δξ) obtained by adding the correction value (0, + Δξ) retrieved from Therefore, the target coordinates exactly coincide with the point TP1 on the screen where the line of sight LS1 intersects.

  Referring to FIG. 13B, when it is estimated that the inclination angle θ belongs to the second section, the screen of the operation panel 160 is likely to be adjusted to an inclination that is easy for the user to see. That is, it can be considered that the line of sight LS2 directed toward the user's fingertip FP2 matches the normal direction of the screen of the operation panel 160. At this time, the coordinate TP2 on the screen at which the line of sight LS2 intersects is equal to the coordinate of the point MP2 obtained by projecting the fingertip FP2 estimated from the detection result by the 3D gesture sensor 410 perpendicular to the screen. On the other hand, since the correction value retrieved from the table of FIG. 13D is (0, 0), the target coordinates estimated by the gesture interpretation unit 434 are the fingertip FP2 obtained from the detection result by the 3D gesture sensor 410. The coordinates (X, Y) of the projection point MP2 onto the screen. Therefore, the target coordinates exactly coincide with the point TP2 on the screen where the line of sight LS2 intersects.

  Referring to FIG. 13C, when it is estimated that the inclination angle θ belongs to the third section, the screen of the operation panel 160 is close to the vertical plane, and thus the line of sight LS3 directed to the user's fingertip FP3 is From the normal direction of the screen, it is inclined obliquely in the opposite direction to the inclination shown in FIG. As a result, the point TP3 on the screen where the line of sight LS3 intersects deviates from the point MP3 obtained by projecting the user's fingertip FP3 perpendicularly to the screen by a distance Δη in the negative direction of the Y axis. On the other hand, the target coordinates estimated by the gesture interpreter 434 are the coordinates (X, Y) of the projection point MP3 projected from the fingertip FP3 to the screen obtained from the detection result by the 3D gesture sensor 410, as shown in the table of FIG. (X, Y−Δη) obtained by adding the correction value (0, −Δη) retrieved from Therefore, the target coordinates exactly coincide with the point TP3 on the screen where the line of sight LS3 intersects.

[Advantages of the embodiment]
In MFP 100 according to the embodiment of the present invention, as described above, first, detection unit 21 detects distortion of electric field EFL due to the presence of a ground conductor in space RGA-RGC to be monitored. Next, the measurement unit 22 measures the three-dimensional coordinates of the ground conductor from the distortion of the electric field EFL, and the determination unit 23 determines that the ground conductor is based on the three-dimensional coordinates. It is determined whether it is. Specifically, first, the measurement unit 22 measures the three-dimensional coordinates of the center of gravity of the ground conductor from the distortion of the waveform of the reception voltage signal RVS detected by the detection unit 21 with the reception electrodes 4N,. If the three-dimensional coordinates are located on the boundary surface BND shown in FIGS. 8B and 8D, or closer to the surface of the touch panel 161 than that, the determination unit 23 determines the ground conductor located at the three-dimensional coordinates. And the indicator HND located in the first region RGA. On the other hand, if the three-dimensional coordinates are farther from the surface of the touch panel 161 than the boundary surface BND, the determination unit 23 determines that the ground conductor located at the three-dimensional coordinates is (A) the conductive member 801 located in the second region RGB. Or (B) based on the width of the temporal change of the three-dimensional coordinates, it is determined whether the conductive member 801 is located in the second region RGB or the indicator located in the third region RGC.

  As described above, the MFP 100 detects the user's three-dimensional gesture and the swing of the operation panel 160 with the same 3D gesture sensor 410. Accordingly, it is not necessary to add a sensor for detecting the tilt angle θ of the operation panel 160 separately from the 3D gesture sensor 410. Thus, the MFP 100 can achieve both improvement in operability and reduction in manufacturing cost.

  Further, in this MFP 100, when the determination unit 23 determines that it is an indicator, the gesture interpretation unit 434 passes the movement of the indicator and interprets the input operation indicated by the user's gesture. On the other hand, when the determination unit 23 determines that the conductive member 801 is used, the inclination angle estimation unit 435 estimates the inclination angle θ of the operation panel 160 from the position of the conductive member 801, and the control 24 responds to the inclination angle θ. The display attribute of the LCD 4D and the characteristics of detection by the 3D gesture sensor 410 are changed. Thereby, those attributes can be automatically adapted to individual users. Furthermore, the gesture interpretation unit 434 estimates target coordinates based on the inclination angle θ of the operation panel 160. As a result, the accuracy of identifying the target of the gesture is improved, so that the input operation indicated by the gesture can be accurately interpreted.

[Modification]
(A) The image processing apparatus 100 shown in FIG. 1 is an MFP. The image processing apparatus according to the embodiment of the present invention may be any other printer such as a laser printer or an inkjet, a copier, a scanner, or a FAX.
(B) The measuring unit 22 measures the three-dimensional coordinates of the center of gravity as a representative point of the ground conductor. In addition, the representative point of the ground conductor may be a characteristic point of the conductor, such as a tip of the conductor, a symmetrical point of the shape of the outer surface, or the like.

  (C) If the three-dimensional coordinates measured by the measurement unit 22 are farther from the surface of the touch panel 161 than the boundary surface BND, the determination unit 23 employs one of the determination methods (A) and (B). In addition, the determination unit 23 adopts the determination method (A) if the inclination angle θ of the operation panel 160 is equal to or smaller than a specific threshold, and adopts the determination method (B) if the inclination exceeds the threshold. Both methods may be switched according to the angle θ.

  (D) In the above embodiment, the inclination angle θ of the operation panel 160 is estimated from the displacement of the touch panel 161 with respect to the conductive member 801 fixed to the casing of the MFP 100. In addition, the position of the movable member that moves in the vicinity of the touch panel 161, such as opening / closing of the upper cover of the document table of the scanner 120, opening / closing of the front door of the housing, or the height of the placement surface of the paper discharge tray 150, It is possible to guess it as follows.

-Top cover of document platen-
FIG. 14A is a schematic diagram showing the positional relationship between the upper lid of the document table of the scanner 120 and the space to be monitored by the electric field type 3D gesture sensor 410. Referring to FIG. 14A, this monitored space is composed of only the front side space URG shown in FIG. 7A, that is, the first area RGA and the third area RGC shown in FIG. Is done. A part of the third region RGC overlaps the peripheral edge of the ADF 110. Since the ADF 110 is mounted on the upper cover of the document table of the scanner 120, it can be opened and closed together with the upper cover. FIG. 14A shows a state in which the ADF 110 is closed. A conductive member 802 is attached to the peripheral edge portion of the ADF 110 that overlaps the third region RGC. While the peripheral edge of the ADF 110 is made of an insulating resin, the conductive member 802 is made of a conductive material, for example, a rectangular metal plate. The conductive member 802 is further maintained at a ground potential by a ground member such as a chassis of the MFP 100.

  FIG. 14B shows a correspondence relationship between the space RGA and RGC to be monitored shown in FIG. 14A and the virtual range of the three-dimensional coordinates of the ground conductor that can be identified by the detection by the 3D gesture sensor 410. It is a schematic diagram. Referring to FIG. 14B, each point of the virtual range corresponds to only one point in the front space URG, so that this virtual range can be identified with the front space URG. In this case, a virtual plane BNA (Z = ZB) that divides the first region RGA and the third region RGC is the boundary surface BND as it is.

  Using this correspondence, the determination unit 23 determines whether the distortion of the electric field EFL detected by the detection unit 21 is the indicator HND or the conductive member 802. Specifically, first, the measurement unit 22 measures the three-dimensional coordinates of the center of gravity of the ground conductor from the waveform distortion of the reception voltage signal RVS detected by the detection unit 21 with the reception electrodes 4N,. If the three-dimensional coordinates are located on the boundary surface BND or closer to the surface of the touch panel 161 than the three-dimensional coordinates, the determination unit 23 connects the ground conductor located at the three-dimensional coordinates to the indicator HND located in the first region RGA. Determine. Since the first region RGA is closer to the receiving electrodes 4N,... Than the third region RGC, the conductive member located in the third region RGC as compared with the distortion of the electric field EFL caused by the indicator HND located in the first region RGA. Anything due to 802 may be ignored. On the other hand, if the three-dimensional coordinate measured by the measurement unit 22 is farther from the surface of the touch panel 161 than the boundary surface BND, the determination unit 23 stores the three-dimensional coordinate in the third region RGC according to the determination method (A) or (B). It is discriminated at any position between the existing conductive member 802 and the indicator.

FIG. 14C is a schematic diagram showing the monitoring target spaces RGA and RGC when the ADF 110 is opened. Referring to (c) of FIG. 14, the conductive member 802 is detached from the third region RGC with the opening of the ADF 110.
FIG. 14D is a schematic diagram showing a correspondence relationship between the space RGA, RGC to be monitored shown in FIG. 14C and the virtual range of the three-dimensional coordinates of the ground conductor that can be identified by detection by the 3D gesture sensor 410. It is. Referring to FIG. 14D, as the ADF 110 opens, the conductive member 802 departs from this virtual range while drawing a constant trajectory TRC in the third region RGC. Therefore, when the indicator HND is not present in the monitored spaces RGA and RGC, if the three-dimensional coordinates of the ground conductor located in the third region RGC are measured, it is determined that the ADF 110 is in a closed state. If the three-dimensional coordinates are not measured, the determination unit 23 can determine that the ADF 110 is in an open state.

As described above, the MFP 100 can also detect the user's three-dimensional gesture and the opening / closing of the ADF 110 with the same 3D gesture sensor 410. As a result, the number of existing sensors for detecting the opening / closing of the ADF 110 can be reduced, so that the MFP 100 can achieve both improved operability and reduced manufacturing costs.
-Front door of housing-
FIG. 15A is a schematic diagram showing the positional relationship between the front door of the housing of the MDF 100 and the space to be monitored by the electric field type 3D gesture sensor 410. Referring to (a) of FIG. 15, the space to be monitored is composed of a first area RGA and a second area RGB shown in (a) of FIG. A part of the second region RGB overlaps with the upper end portion of the front door 151. FIG. 15A shows a state where the front door 151 is closed. A conductive member 803 is attached to the upper end of the front door 151 that overlaps the second region RGB. Whereas the front door 151 is made of an insulating resin, the conductive member 803 is made of a conductive material, for example, a rectangular metal plate. The conductive member 803 is further maintained at a ground potential by a ground member such as a chassis of the MFP 100.

  FIG. 15B shows a correspondence relationship between the space RGA, RGB of the monitoring target shown in FIG. 15A and the virtual range of the three-dimensional coordinates of the ground conductor that can be identified by detection by the 3D gesture sensor 410. It is a schematic diagram. Referring to FIG. 15B, each point in the region (Z <ZB) closer to the surface (Z = 0) of the touch panel 161 than the boundary surface BND (Z = ZB) in the virtual range is shown. One point in the first region RGA corresponds to one point in the second region RGB corresponding to each point in the region (Z> ZB) farther from the surface of the touch panel 161 than the boundary surface BND.

  Using this correspondence, the determination unit 23 determines whether the distortion of the electric field EFL detected by the detection unit 21 is the indicator HND or the conductive member 803. Specifically, first, the measurement unit 22 measures the three-dimensional coordinates of the center of gravity of the ground conductor from the waveform distortion of the reception voltage signal RVS detected by the detection unit 21 with the reception electrodes 4N,. If the three-dimensional coordinates are located on the boundary surface BND or closer to the surface of the touch panel 161 than the three-dimensional coordinates, the determination unit 23 connects the ground conductor located at the three-dimensional coordinates to the indicator HND located in the first region RGA. Determine. Since the first region RGA is closer to the receiving electrodes 4N,... Than the second region RGB, the conductive member located in the second region RGB compared to the distortion of the electric field EFL caused by the indicator HND located in the first region RGA. What is caused by 803 may be ignored. On the other hand, if the three-dimensional coordinates measured by the measurement unit 22 are farther from the surface of the touch panel 161 than the boundary surface BND, the determination unit 23 converts the three-dimensional coordinates into the second region RGB according to the determination method (A) or (B). The position of the existing conductive member 803 is determined.

FIG. 15C is a schematic diagram illustrating the monitoring target spaces RGA and RGB when the front door 151 is opened. Referring to FIG. 15C, the conductive member 803 is detached from the second region RGB as the front door 151 is opened.
FIG. 15D is a schematic diagram showing a correspondence relationship between the space RGA, RGB of the monitoring target shown in FIG. 15C and the virtual range of the three-dimensional coordinates of the ground conductor that can be identified by detection by the 3D gesture sensor 410. It is. Referring to FIG. 15D, as the front door 151 opens, the conductive member 803 departs from this virtual range while drawing a constant trajectory TRC in the second region RGB. Therefore, when the indicator HND is not present in the first region RGA, the determination unit 23 determines that the front door 151 is in a closed state if the three-dimensional coordinates of the ground conductors located in the second region RGB are measured. If it is determined and the three-dimensional coordinates are not measured, the determination unit 23 can determine that the front door 151 is in an open state. The front door 151 is thus opened, for example, when the image forming unit 10 is replenished with toner or when a sheet is pulled out from the image forming unit 10 when a paper jam occurs. Therefore, when the determination unit 23 determines that the front door 151 is in the open state, the control unit 24 requests the main control unit 30 to suspend processing by the MFP 100, for example.

As described above, the MFP 100 can also detect the user's three-dimensional gesture and the opening / closing of the front door 151 with the same 3D gesture sensor 410. As a result, it is possible to reduce the number of existing sensors for detecting the opening / closing of the front door 151. Thus, the MFP 100 can achieve both improvement in operability and reduction in manufacturing cost.
-Supplement-
The swinging of the operation panel 160 and the opening / closing of the ADF 110 and the front door 151 are both manual. In addition, the position of the movable member that moves automatically can be detected by the 3D gesture sensor 410. For example, if the placement surface of the paper discharge tray 150 automatically falls in the gap GAP between the scanner 120 and the printer 130 as the number of sheets stacked thereon increases, it is mounted on the placement surface. The 3D gesture sensor 410 may detect the position of the conductive member.

  (E) In the above embodiment, the conductive member 801 is disposed in the second region RGB farther from the receiving electrode 4N than the first region RGA, so that the contribution to the distortion of the waveform of the received voltage signal RVS is conducted. The distortion of the electric field EFL caused by the member 801 is ignored with respect to the distortion caused by the indicator HND located in the first region RGA. In this case, the three-dimensional coordinates of the conductive member 801 are measured when the indicator HND does not exist in the space to be monitored.

  In addition, even when the indicator HND is present in the first region RGA, it is possible to detect a change in the distortion of the waveform of the reception voltage signal RVS due to the relative displacement between the reception electrodes 4N,. In this case, it is possible to specify at least the inclination angle θ of the operation panel 160 together with the three-dimensional coordinates (hereinafter referred to as “gesture coordinates”) of the indicator HND from the change as follows.

  16A and 16B, the inclination angle θ of the operation panel 160 is 0 ° and a positive value, respectively, and the indicator HND is positioned at the fixed point (Xj, Yj, Zj) of the first region RGA. It is a schematic diagram which shows the three-dimensional coordinate (X, Y, Z) and (X, Y, Z) which the measurement part 22 measures when doing. Referring to FIGS. 16A and 16B, when the conductive member 801 is positioned at the three-dimensional coordinates (Xc, Yc, Zc), (Xc, Yc, Zc), the measurement unit 22 measures 3 Strictly speaking, the dimensional coordinates (X, Y, Z) and (X, Y, Z) are different from the fixed point (Xj, Yj, Zj). Furthermore, strictly speaking, due to the difference between the three-dimensional coordinates (Xc, Yc, Zc) and (Xc, Yc, Zc) of the conductive member 801, the three-dimensional coordinates (X, Y, Z) measured by the measuring unit 22 ( Differences also arise from X, Y, Z). Here, the trajectory TRC drawn by the conductive member 801 is constant. Therefore, in a state where the indicator HND is fixed to each point (Xj, Yj, Zj) of the first region RGA, the position (Xc, Yc, Zc) of the conductive member 801 and the three-dimensional coordinates (X , Y, Z) is obtained in advance. If collated with this correspondence, the three-dimensional coordinates (Xc, Yc, Zc) of the conductive member 801 and one point (Xj) of the first region RGA are calculated from the three-dimensional coordinates (X, Y, Z) measured by the measuring unit 22. , Yj, Zj), that is, a combination with gesture coordinates can be estimated. In particular, the inclination angle θ of the operation panel 160 can be specified from the three-dimensional coordinates (Xc, Yc, Zc) of the conductive member 801.

  FIG. 16C is a correspondence table of the three-dimensional coordinates (X, Y, Z) measured by the measurement unit 22, the tilt angle θ of the operation panel 160, and the gesture coordinates (Xj, Yj, Zj). This table shows an example of the above correspondence. Referring to (c) of FIG. 16, the inclination angle θ of the operation panel 160 is less than the threshold θB for each small rectangular parallelepiped space to which the three-dimensional coordinates (X, Y, Z) measured by the measurement unit 22 belong. Whether or not and corresponding gesture coordinates are defined. In particular, even if the corresponding gesture coordinates are the same, the three-dimensional coordinates measured by the measurement unit 22 differ depending on whether the inclination angle θ of the operation panel 160 is less than the threshold θB.

  For example, when the indicator HND is not present in the first region RGA, if the inclination angle θ of the operation panel 160 is less than the threshold θB, the three-dimensional coordinates measured by the measurement unit 22 are small spaces {(X, Y, Z) If | X0 <X ≦ X1, Y0 <Y ≦ Y1, Z0 <Z ≦ Z1} and the inclination angle θ is greater than or equal to the threshold θB, this three-dimensional coordinate is a small space {(X, Y, Z) | X0 <X ≦ X1, Y0 <Y ≦ Y1, Z0 <Z ≦ Z1}. When the indicator HND is located at the gesture coordinates (Xj0, Yj0, Zj0), if the inclination angle θ of the operation panel 160 is less than the threshold θB, the three-dimensional coordinates measured by the measurement unit 22 are small spaces {(X, Y , Z) | X2 <X ≦ X3, Y2 <Y ≦ Y3, Z2 <Z ≦ Z3}, and if the inclination angle θ is equal to or greater than the threshold θB, the three-dimensional coordinate is represented by a small space {(X, Y, Z ) | X2 <X ≦ X3, Y2 <Y ≦ Y3, Z2 <Z ≦ Z3}.

  If this correspondence is used, it is estimated from the three-dimensional coordinates (X, Y, Z) measured by the measurement unit 22 whether or not the inclination angle θ of the operation panel 160 is less than the threshold θB together with the corresponding gesture coordinates. Is possible. For example, if the three-dimensional coordinates are located in the small space {(X, Y, Z) | X0 <X ≦ X1, Y0 <Y ≦ Y1, Z0 <Z ≦ Z1}, the inclination angle θ of the operation panel 160 is the threshold value. It is estimated that it is less than θB and the indicator HND is not present in the first region RGA. If the three-dimensional coordinates are located in the small space {(X, Y, Z) | X2 <X ≦ X3, Y2 <Y ≦ Y3, Z2 <Z ≦ Z3}, the inclination angle θ is equal to or greater than the threshold θB. In addition, it is estimated that the indicator HND is located at the gesture coordinates (Xj0, Yj0, Zj0).

The size of the small space defined by (c) in FIG. 16 is mainly determined by the detection accuracy of the 3D gesture sensor 410. If this accuracy is further increased, it is possible to divide the range of the inclination angle θ of the operation panel 160 corresponding to each small space more finely than two types of whether or not it is less than the threshold value θB.
A correspondence table similar to the correspondence table shown in (c) of FIG. 16 is replaced with the conductive member 801 with respect to the existing conductive members fixed to the casing of the MFP 100, such as the hinge portion 102 shown in (b) of FIG. It may be set. Even using this correspondence table, it is possible to estimate the inclination angle θ and the gesture coordinates of the operation panel 160 from the three-dimensional coordinates measured by the measurement unit 22.

  (F) In the embodiment described above, the 3D gesture sensor 410 mounted on the operation panel 160 of the MFP 100 is used, the inclination angle θ of the operation panel 160, the ADF 110, the front door 151 of the housing 101, and the like. And the position of the movable member that moves around. In addition, if the electronic device uses an operation panel equipped with a 3D gesture sensor as an input device, the 3D gesture sensor can be used to detect the position of a movable member that moves around the operation panel. is there.

  The present invention relates to an image processing apparatus, and as described above, uses a 3D gesture sensor not only for detecting a user's gesture but also for estimating a position of a part that can be displaced relative to an operation panel. Thus, the present invention is clearly industrially applicable.

100 MFP
101 MFP housing 160 Operation panel 161 Touch panel 801 Conductive member URG Front side space LRG Back side space RGA First area of the monitoring target space RGB Second area of the monitoring target space RGC Third area of the monitoring target space BNA First area A virtual plane that separates the third area from the third area BNB A virtual plane that separates the second area from other areas in the back space BND boundary surface θ Inclination angle of the operation panel

Claims (10)

An image processing apparatus having an operation panel;
A detection unit configured to generate an electric field in a space extending from the surface of the operation panel to a predetermined range and set as a monitoring target space, and detect distortion of the electric field due to the presence of an object in the space;
A measurement unit that measures the three-dimensional coordinates of the object in the space to be monitored from the distortion of the electric field detected by the detection unit;
A conductive member provided in a portion of the image processing apparatus that is displaceable relative to the operation panel, and changing a distortion of the electric field according to a relative position with the operation panel;
A determination unit for determining whether the object is an indicator used for a user input operation on the image processing apparatus or the conductive member based on the three-dimensional coordinates measured by the measurement unit;
When the determination result by the determination unit is the indicator, the interpretation unit interprets the input operation indicated by the gesture by regarding the movement of the three-dimensional coordinates measured by the measurement unit as a user's gesture;
When the determination result is the conductive member, an estimation unit that estimates a relative position between the operation panel and the part of the image processing device from the three-dimensional coordinates measured by the measurement unit;
A control unit that determines an operation state of the image processing device based on the position estimated by the estimation unit, and controls processing according to the operation state;
An image processing apparatus. The detector is
Transmitting electrodes arranged in a planar shape;
Four receiving electrodes arranged separately on each side of a rectangular surface extending in parallel with a predetermined distance from a plane including the transmitting electrodes;
A transmitter for transmitting a transmission voltage signal to the transmission electrode;
A receiving unit for detecting a reception voltage signal received by each of the four reception electrodes according to the transmission voltage signal;
Including
The image processing apparatus according to claim 1, wherein the distortion of the electric field is detected as a distortion of a waveform of the reception voltage signal with respect to the transmission voltage signal. Compared to the level of the transmission voltage signal, the electric potential is such that the overall potential can be regarded as substantially uniform, and the potential is considered to be substantially equal to the ground potential when compared with the level of the transmission voltage signal. The detection unit detects a change that appears in a drop in the level of the reception voltage signal with respect to the transmission voltage signal, depending on whether or not the detected object exists in the space to be monitored;
The conductive member is maintained at a potential substantially equal to a ground potential;
The image processing apparatus according to claim 2, wherein the measurement unit measures a three-dimensional coordinate of the object from a change that appears in the drop of the level. The discrimination unit
When the three-dimensional coordinate measured by the measurement unit is located at a distance below the threshold from the detection unit, it is determined that the object located at the three-dimensional coordinate is the indicator,
2. The apparatus according to claim 1, wherein when the three-dimensional coordinate is located at a distance exceeding the threshold from the detection unit, it is determined that an object located at the three-dimensional coordinate is the conductive member. 4. The image processing device according to any one of 3 to 3. The discrimination unit
When the three-dimensional coordinate measured by the measurement unit is located at a distance below the threshold from the detection unit, it is determined that the object located at the three-dimensional coordinate is the indicator,
When the three-dimensional coordinate is located at a distance exceeding the threshold value from the detection unit, further observe the temporal change of the three-dimensional coordinate, and if the width of the temporal change is equal to or less than a predetermined value, the three-dimensional The image processing apparatus according to claim 1, wherein an object located at coordinates is determined as the conductive member. The operation panel has a variable surface inclination with respect to a horizontal plane,
The conductive member is arranged such that when the surface of the operation panel is inclined from a horizontal plane, a trajectory with respect to the surface passes through the space to be monitored. The image processing apparatus according to any one of the above. The operation panel includes a display unit for displaying an operation screen,
The estimation unit estimates the inclination of the operation panel from the three-dimensional coordinates measured by the measurement unit,
The image processing apparatus according to claim 6, wherein the control unit causes the display unit to change a display attribute according to the tilt. The operation panel includes a display unit for displaying an operation screen,
The estimation unit estimates the inclination of the operation panel from the three-dimensional coordinates measured by the measurement unit,
The image processing apparatus according to claim 6, wherein the interpretation unit estimates coordinates on the operation screen targeted by a user's gesture based on an inclination of the operation panel. A scanner that reads an image from a sheet placed on the platen;
An upper lid attached to the scanner so as to be openable and closable so as to cover the platen;
Further comprising
The operation panel is installed on an edge of the document table,
The conductive member is arranged such that a locus drawn on the surface of the operation panel as the upper lid is opened and closed passes through the space to be monitored.
The image processing apparatus according to claim 1, wherein the control unit determines an operation state of the image processing apparatus based on the position of the upper lid estimated by the estimation unit. . An image forming unit that forms an image on a sheet with toner or ink;
A front door that is detachably attached to a portion of the front surface of the image processing apparatus located below the operation panel, and that allows toner or ink to be replenished to the image forming unit, or a sheet to be taken in and out,
Further comprising
The conductive member is arranged such that a locus drawn on the surface of the operation panel as the front door opens and closes passes through the space to be monitored.
The image processing according to claim 1, wherein the control unit determines an operation state of the image processing device based on the position of the front door estimated by the estimation unit. apparatus.

Images