JP2011175543A - Indicator detection device and touch panel - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an indicator detection device easily and rapidly detecting the distance between an indicator and a detection surface, and a touch panel having the indicator detection device. <P>SOLUTION: The indicator detection device 1 includes a right imaging part 11R and a left imaging part 11L for imaging the surface of a display area 22 roughly parallel to the surface of the display area 22, and an indicator distance calculation part 15 for calculating the distance between the indicator and the surface of the display area 22 according to the position of the indicator in the image picked up by at least one of the right imaging part 11R and the left imaging part 11L. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a pointer detection device that detects a pointer on a detection surface, and a touch panel including the pointer detection device.

  A so-called touch panel that includes a display device that displays an image and an indicator detection device that detects an indicator such as a user's finger or stylus and detects the indicator on the display device is widely used. The touch panel is operated by an intuitive and simple operation in which a user who visually recognizes an image displayed on the display device touches a predetermined position on the display device (on the image) using the image as a clue.

  As a method of the indicator detection device provided in the touch panel, for example, there is a capacitance method. The capacitance method is a method in which a conductive film is provided on a display device and a change in capacitance caused by a finger touching or approaching the conductive film is detected as a current change through a current. It detects the close position.

  However, in the capacitance method, since it is necessary to provide a conductive film or the like on the display device, light transmittance on the display device is lowered, and visibility of an image is deteriorated outdoors.

  Therefore, as a method for solving this problem, for example, an infrared matrix method (a method in which a plurality of light emitting units and light receiving units are spread at the end of the detection surface, and an indicator is detected from signals generated by the respective light receiving units) The method described in Patent Document 1 (infrared light emitting units and image sensors are provided at two corners of the detection surface, and an infrared reflector is provided at the end of the detection surface. A method for detecting a touched position) has been proposed.

  Further, in Patent Documents 2 and 3, the distance between the indicator and the detection surface is detected by comparing the indicator in the image obtained by imaging with the model of the indicator recorded in advance. A pointer detection apparatus has been proposed.

Japanese Patent No. 4229964 JP-A-8-328735 JP 2009-259110 A

  In the case of an indicator detection device using an infrared ray as proposed in Patent Document 1, it is not necessary to provide an additional member such as a conductive film on the display device. It becomes possible to prevent deterioration of visibility in the outdoors. However, since the detection accuracy deteriorates due to the influence of infrared rays radiated from other than the light emitting unit (for example, the sun) outdoors, it becomes a problem. Furthermore, not only infrared rays but also the influence of disturbance (variation of detection environment) is great outdoors. Therefore, even if it is a pointer detection apparatus which implement | achieves the detection accuracy which does not cause trouble in indoor use, it may become unbearable for use outdoors.

  Further, with the pointer detection device proposed in Patent Documents 2 and 3, it is possible to detect the distance between the pointer and the detection surface. However, there is a problem that the detection accuracy deteriorates due to the influence of disturbance as described above. In addition, since it is necessary to record a model of the indicator in advance, the type and usage of the indicator are restricted, and a device for recording the model is required, which complicates the device. This is a problem because it leads to an increase in size. Furthermore, when a distance is detected by comparing an indicator in an image with a model, a complicated calculation is required, so that the calculation amount becomes enormous and it becomes difficult to operate quickly.

  Then, this invention aims at providing the indicator detection apparatus which enabled it to detect the distance of an indicator and a detection surface simply and rapidly, and the touchscreen provided with the said indicator detection apparatus. To do.

  In order to achieve the above object, a pointer detection apparatus according to the present invention is a pointer detection apparatus that detects a pointer on a detection surface, and images the image on the detection surface substantially parallel to the detection surface. And at least one imaging unit that generates a distance detection unit that detects a distance between the indicator and the detection surface based on a position of the indicator in an image generated by imaging of the imaging unit. It is characterized by providing.

  In the indicator detection device having the above configuration, the indication on the detection surface is based on the position of the indicator in an image generated by two or more imaging units that capture the detection surface from different locations. A position detector that detects the position of the body may be further provided.

  With this configuration, the position detection unit and the distance detection unit both determine the position of the indicator on the detection surface and the distance between the indicator and the detection surface based on the position in the image of the indicator recognized from the image. Will be detected. Therefore, it is possible to share the detection process. Therefore, it is possible to simplify and speed up the processing of the pointer detection apparatus.

  Further, in the pointer detection apparatus having the above configuration, the distance detection unit may determine a distance between the pointer and the detection surface based on the position of the pointer in an image generated by any one of the imaging units. May be detected.

  If comprised in this way, it will become possible to reduce the calculation amount required for the detection of the distance of a pointer and a detection surface.

  Moreover, in the pointer detection apparatus having the above configuration, the imaging unit may generate an image having a plurality of signal values for one pixel by imaging visible light.

  With this configuration, for example, when the indicator detection device is used outdoors, it is possible to accurately distinguish between various disturbances generated by the indicator entering the detection surface and the indicator. . In particular, it is possible to make it less susceptible to disturbance than to recognize an indicator in an image in which one pixel has only a single signal value. Is possible.

  Moreover, in the pointer detection apparatus having the above-described configuration, the optical axis of the imaging unit may be present in substantially the same plane as the detection surface.

  If comprised in this way, in the image produced | generated by an imaging part, it will become possible to shorten the length of the predetermined direction (direction substantially parallel to an optical axis) of a detection surface. For this reason, it is possible to suppress the position of the detection surface immediately below the indicator in the image from fluctuating due to the movement of the indicator (movement in a direction substantially parallel to the optical axis). Therefore, it is possible to easily calculate the length between the indicator and the detection surface in the image.

  Further, in the pointer detection apparatus having the above configuration, the distance detection unit stores a background image storage unit that stores a background image that is an image generated by imaging of the imaging unit, and an image generated by imaging of the imaging unit. The difference between the target image and the background image stored in the background image storage unit is obtained, and the position of the indicator in the target image is obtained from the position of the region where the difference is a predetermined size or more. An image processing unit and an indicator distance calculating unit that calculates a distance between the indicator and the detection surface from the position of the indicator obtained by the image processing unit may be provided.

  If comprised in this way, it will become possible to detect the distance of a pointer and a detection surface by simple processing. In particular, the distance between the indicator and the detection surface can be detected without requiring analysis of the size and shape of the indicator in the target image.

  In addition, a touch panel according to the present invention includes a display device that displays an image in a display area, and the above indicator detection device that uses the display area as a detection surface.

  With the configuration of the present invention, it is possible to detect the distance between the indicator and the detection surface by recognizing the indicator from at least one of the images captured by the imaging unit. In particular, it is possible to detect the distance between the indicator and the detection surface simply and quickly without the need to compare the recognized indicator with a model or the like.

  The significance or effect of the present invention will be further clarified by the following description of embodiments. However, the following embodiment is merely one of the embodiments of the present invention, and the meaning of the terminology of the present invention or each constituent element is limited to those described in the following embodiments. is not.

These are the top view and side view which show an example of a structure of the touchscreen which is one Embodiment of this invention. These are block diagrams which show an example of a structure of a pointer detection apparatus. These are figures explaining an example of the detection method of the position on the detection surface of a pointer. These are figures explaining an example of the detection method of the position on the detection surface of a pointer. These are the figures explaining an example of the detection method of the distance of a pointer and a detection surface. These are figures explaining an example of operation of a pointer detection device. These are the schematic diagrams of the left side image explaining an example of the setting method of a processing range. These are flowcharts which show an example of the operation | movement which determines the presence or absence of the fluctuation | variation of an imaging environment. These are the top view and side view which show another example of a structure of the touchscreen which is one Embodiment of this invention. FIG. 9 is a schematic diagram illustrating an example of a left image and a right image generated by the left imaging unit and the right imaging unit illustrated in FIG. 8.

<Overall configuration>
A touch panel according to an embodiment of the present invention will be described below with reference to the drawings. 1A and 1B are a plan view and a side view illustrating an example of a configuration of a touch panel according to an embodiment of the present invention. FIG. 1A is a plan view showing a surface on which an image of the touch panel is displayed. On the other hand, FIG.1 (b) is a side view which shows the left side surface of a touchscreen when the surface shown to Fig.1 (a) is turned to the right. FIG. 2 is a block diagram illustrating an example of the configuration of the pointer detection apparatus.

  As shown in FIG. 1, the touch panel 1 of this example includes a display device 20 that displays an image in a display area 22 surrounded by a non-display area 21. In addition, the touch panel 1 includes a left imaging unit 11L that images the display area 22 surface from the left side of the touch panel 1 and a right imaging unit 11R that images the display area 22 surface from the right side of the touch panel 1. The apparatus 10 (refer FIG. 2) is provided. Further, the non-display area 21 surface and the display area 22 surface exist in substantially the same plane. Note that a part of the left imaging unit 11L and the right imaging unit 11R (for example, a part of an imaging element, a lens, or the like) may be arranged to be embedded in the non-display area 21.

  In FIG. 1A and FIG. 1B, the directions in which the left imaging unit 11L and the right imaging unit 11R image are indicated by arrows. As illustrated in FIG. 1B, the left imaging unit 11L and the right imaging unit 11R perform imaging so as to be substantially parallel to the display area 22 surface as a detection surface. At this time, for example, the optical axes of the left imaging unit 11L and the right imaging unit 11R are substantially parallel to the display area 22 surface. Of the left imaging unit 11L and the right imaging unit 11R, at least one of the imaging units 11L and 11R involved in detection of the distance between the indicator and the display region 22 surface (details will be described later) is the display region 22 surface. As long as the image is taken so as to be substantially parallel to the line.

  As shown in FIG. 2, the pointer detection apparatus 10 includes a left imaging unit 11L and a right imaging unit 11R, and a left image processing unit 12L that processes a left image generated by imaging of the left imaging unit 11L. A left background image storage unit 13L that temporarily stores a left background image used for processing in the left image processing unit 12L, and a right image processing unit 12R that processes a right image generated by imaging of the right imaging unit 11R. The right background image storage unit 13R that temporarily stores the right background image used for the processing in the right image processing unit 12R, the processing result of the left image by the left image processing unit 12L, and the right image by the right image processing unit 12R. A pointer position calculation unit 14 that calculates the position of the pointer on the display area 22 surface (on the detection surface) based on the processing result and outputs the pointer position information, and a left image processing unit The distance between the indicator and the display area 22 surface (detection surface) is calculated based on at least one of the processing result of the left image by 2L and the processing result of the right image by the right image processing unit 12R, and is output as indicator distance information. And a pointer distance calculation unit 15. The indicator is used by the user to indicate an arbitrary position on the detection surface such as the user's finger or stylus. In the following description, a finger is exemplified as an indicator for the purpose of concrete description. In addition, although it is illustrated that the processing result of the left image processing unit 12L and the processing result of the right image processing unit 12R are input to the indicator distance calculation unit 15, a configuration in which at least one processing result is input. If it is.

  The left imaging unit 11L is disposed at the upper left of the rectangular display area 22. The right imaging unit 11R is disposed at the upper right of the display area 22. That is, the left imaging unit 11L and the right imaging unit 11R are disposed at both ends of the upper side of the display area 22. Further, the left image generated by the imaging of the left imaging unit 11L and the right image generated by the imaging of the right imaging unit 11R each have an angle of view of 90 ° or more, and the entire display area 22 is within the angle of view. It is preferable that it fits. Furthermore, the left imaging unit 11L and the right imaging unit 11R include imaging elements that can generate a color image with a light receiving element that detects visible light, such as a single-plate Bayer array image sensor or a three-plate image sensor. A thing is preferable.

<Example of method for detecting the position of the indicator on the detection surface>
Next, an example of a method for detecting the position of the pointer on the display area 22 surface (on the detection surface) by the pointer detection apparatus 10 will be described with reference to FIGS. 3 and 4. 3 and 4 are diagrams for explaining an example of a method for detecting the position of the indicator on the detection surface. 3 is a diagram showing the left imaging unit 11L, and FIG. 4 is a diagram showing the right imaging unit 11R.

  FIG. 3A is a plan view showing the angle of view of the left image generated by the imaging of the left imaging unit 11L, and corresponds to FIG. 1A showing the plane of the touch panel. FIG. 3B is a schematic diagram showing an example of the left side image, and shows an image shown on the image sensor of the left side imaging unit 11L shown in FIG. Similarly, FIG. 4A is a plan view showing the angle of view of the right image generated by imaging of the right imaging unit 11R, and corresponds to FIG. 1A showing the plane of the touch panel. FIG. 4B is a schematic diagram illustrating an example of the right image, and illustrates an image that is reflected on the imaging element of the right imaging unit 11R illustrated in FIG. 3B and 4B, the left side imaged when the optical axes of the left imaging unit 11L and the right imaging unit 11R are in substantially the same plane as the display area 22 plane. An image and a right image are illustrated.

  As shown in FIG. 3A and FIG. 3B, the left imaging unit 11L generates a left image including the upper side to the left side of the display area 22 around itself. Here, considering an angle centered on the upper left of the display area 22 where the left imaging unit 11L is arranged, if the upper side is defined as 0 ° and the left side is defined as 90 °, the direction in which the indicator D within the angle of view is It is expressed by the clockwise angle α. In the left image of this example, the lower right corner RB of the non-display area 21 and the display area 22 is reflected. In addition, when a part of the left imaging unit 11L is embedded in the non-display area 21 as described above, a cross section or a side surface of the non-display area 21 in the left image (lower side of the display area 22 in FIG. 3B). (Part)) may be reflected.

  The same applies to the right imaging unit 11R. As shown in FIG. 4A and FIG. 4B, the right imaging unit 11R generates a right image including the upper side to the right side of the display area 22 around itself. Here, considering the angle centered on the upper right of the display area 22 where the right imaging unit 11R is arranged, if the upper side is defined as 0 ° and the right side is defined as 90 °, the direction in which the indicator D within the angle of view is It is represented by an angle β counterclockwise (counterclockwise). In the right image of this example, the lower left corner LB of the non-display area 21 and the display area 22 is reflected. In addition, when a part of the right imaging unit 11R is embedded in the non-display area 21 as described above, the cross-section or side surface of the non-display area 21 in the right image (the lower side of the display area 22 in FIG. 4B). (Part)) may be reflected.

  The angles α and β can be calculated from the horizontal position of the indicator D in the left image and the right image, as shown in FIGS. 3B and 4B. However, in order to actually calculate the angle α and the angle β, it is necessary to recognize the indicator D from each of the left image and the right image. In this example, the left image processing unit 12L calculates the angle α by recognizing the indicator D in the left image, and the right image processing unit 12R recognizes the indicator D in the right image to thereby obtain the angle β. Is calculated. The details of the method of recognizing the indicator D in the left image and the right image by the left image processing unit 12L and the right image processing unit 12R will be described later.

The indicator position calculation unit 14 performs, for example, triangulation using the angle α and the angle β calculated by the left image processing unit 12L and the right image processing unit 12R and the known length k of the upper side of the display area 22. Then, the position of the indicator D on the surface of the display area 22 is calculated. Specifically, an arbitrary position on the display area 22 is expressed by coordinates (x, y), the upper left is (0, 0), the value of the x coordinate increases toward the right, and the value of the y coordinate increases toward the bottom. In this case, the coordinates (x ₁ , y ₁ ) on the display region 22 surface of the indicator D can be obtained by solving the following simultaneous equations.

x ₁ × tan α = y ₁
(K−x ₁ ) × tan β = y ₁

  The display device 20 performs various control operations such as switching an image to be displayed on the display area 22 based on the indicator position information (that is, the operation content of the touch panel 1 by the user) output from the indicator position calculation unit 14. Do.

<Example of detection method of distance between indicator and detection surface>
Next, an example of a method for detecting the distance between the indicator and the display area 22 surface (detection surface) by the indicator detection device 10 will be described with reference to FIG. FIG. 5 is a diagram illustrating an example of a method for detecting the distance between the indicator and the detection surface. Fig.5 (a) is a side view which shows the left side surface of a touch panel, and is equivalent to FIG.1 (b). FIG. 5B is a schematic diagram illustrating an example of the left side image, which is generated by imaging of the left imaging unit 11L illustrated in FIG. 5A, and corresponds to FIG. 3B. It is. For simplification of description, the case where the distance is detected based on only the left image will be specifically described here.

  In the example shown in FIGS. 5A and 5B, the distance (the double arrow in the figure) between the tip of the indicator D and the surface of the display area 22 is detected. At this time, similarly to the above-described method for detecting the position of the indicator D on the detection surface, the left image processing unit 12L recognizes the indicator D in the left image (details of the recognition method will be described later). Then, the indicator distance calculating unit 15 exists at a predetermined position in the left image from the tip of the indicator D based on the vertical position of the indicator D in the left image recognized by the left image processing unit 12L. The distance between the indicator D and the display area 22 surface is calculated by obtaining the length in the image up to the display area 22 surface.

  The display device 20 acquires the indicator distance information output from the indicator distance calculation unit 15, for example, detects (guesses) the timing at which the user touches the surface of the display area 22, and reacts to the user's operation. To speed up. Further, for example, the image displayed on the display area 22 is deformed according to the distance between the indicator and the display area 22 surface (specifically, the indicator is displayed as the distance between the indicator and the display area 22 decreases). The operability and convenience are improved by enlarging the image of the buttons that are present and widening the interval.

  In addition, although the method of detecting the distance between the indicator D and the display area 22 surface based on the left image has been illustrated, it may be based on the right image or both. Note that when the distance is detected based on both the left image and the right image, the distance obtained from each may be comprehensively determined (for example, an average value may be obtained). In addition, the distance between the indicator D and the surface of the display area 22 may be detected from either one of the images (for example, an image in which the indicator D can be recognized with high accuracy).

  Further, when the indicator distance calculation unit 15 calculates the distance between the indicator D and the surface of the display area 22, the indicator position information may be used. For example, the pointer position information may be used when calculating the distance in the image between the indicator D and the display area 22 surface obtained from the left image or the right image as the distance. Further, for example, in order to obtain the position of the display area 22 surface immediately below the indicator D in the left image or the right image with high accuracy, the indicator position information may be used.

<Operation example>
Next, an example of operation | movement of the indicator detection apparatus 10 is demonstrated with reference to drawings. FIG. 6 is a diagram for explaining an example of the operation of the pointer detection apparatus. In this example, the left imaging unit 11L and the right imaging unit 11R, the left image processing unit 12L and the right image processing unit 12R, the left background image storage unit 13L, and the right background image storage unit 13R perform the same operation. Therefore, for simplification of description, the operations of the left imaging unit 11L, the left image processing unit 12L, and the left background image storage unit 13L will be described as a representative, and the right imaging unit 11R and the right image processing unit 12R will be described. The operations of the right-side background image storage unit 13R are the same, and the description thereof is omitted.

  As shown in FIG. 6, when the pointer detection apparatus 10 starts operating, first, the left image processing unit 12L acquires the left image generated by the imaging of the left imaging unit 11L (STEP 1). Note that the left image processing unit 12L may convert the type of the signal value of the left image acquired from the left imaging unit 11L as necessary. The left image processed by the left image processing unit 12L may be, for example, one pixel having signal values of R (red), G (green), and B (blue), or Y ( Luminance) U and V (color difference) may be provided with each type of signal value, and H (hue) S (saturation) B (luminance) may be provided with each type of signal value. It doesn't matter.

  The left image processing unit 12L determines whether the acquired left image is the first frame (that is, the first left image acquired after the start of the operation) or whether the display area 22 in the acquired left image has been acquired previously. It is determined whether or not the display area 22 in the left image (for example, the previous frame) has shifted (STEP 2).

  When the left image processing unit 12L determines that the acquired left image is the first frame, or when it is determined that the display area 22 in the acquired left image has shifted (STEP 2, YES), the left image processing unit 12L displays the left image. The region 22 is detected (STEP 3). At this time, for example, the position of the display area 22 may be confirmed by detecting a predetermined mark or the like, or the position of the display area 22 may be confirmed based on an image (light) displayed in the display area 22. It doesn't matter.

  The left image processing unit 12L sets a range to be processed in the left image (hereinafter referred to as a processing range) based on the detected display area 22 (STEP 4). The processing range will be described with reference to FIG. FIG. 7 is a schematic diagram of the left image for explaining an example of the method for setting the processing range, and corresponds to FIG. FIG. 7 shows an example in which the range on the display area 22 surface in the left image is set as the processing range A.

  When the left image processing unit 12L sets the processing range A (STEP 4) or determines that the acquired left image is not in the first frame and the display area 22 is not shifted (STEP 2, NO), the imaging environment is next set. It is determined whether or not (for example, the light intensity in the entire angle of view) has changed (STEP 5). Details of the operation of STEP 5 will be described with reference to FIG. FIG. 8 is a flowchart illustrating an example of an operation for determining whether there is a change in the imaging environment.

  As shown in FIG. 8, the left image processing unit 12L first calculates a left difference image (STEP 51). For example, the left image processing unit 12L obtains a difference value of the signal value for each pixel at a corresponding position between the left background image stored in the left background image storage unit 13L and the left image acquired in the most recent STEP1. A left difference image is calculated using the difference value as the signal value of each pixel. In addition, when each pixel of the left background image and the left image includes a plurality of types of signal values, it is preferable to calculate a difference value for each type of signal value. Further, the difference value may be calculated for all the pixels included in the left background image and the left image. However, it is preferable to limit the calculated pixels to the processing range A because the amount of calculation can be reduced.

  By the way, the left background image may be stored in advance in the left background image storage unit 13L before the operation illustrated in FIG. 6 is started, for example. For example, the left image processing unit 12L monitors a plurality of consecutively acquired images for a predetermined time, and uses the left background image storage unit 13L as a left background image with a frame having a small signal value variation (difference value). You may memorize. For example, the left image of the first frame may be stored as the left background image. Note that it is preferable to limit the left background image stored in the left background image storage unit 13L to an image within the processing range A because the storage amount can be reduced.

  Next, with respect to the left side difference image calculated in STEP 51, it is determined whether or not there are a predetermined number or more (a predetermined size or more) of pixels within the processing range A that have a difference value equal to or greater than a threshold value. (STEP 52). That is, it is determined whether there is a change in the spatial direction of the imaging environment.

  At this time, when each pixel of the left difference image has a difference value of a plurality of types of signal values, it may be compared with each threshold value for each type of signal value, or the difference values of a plurality of types of signal values are added together. You may compare a comprehensive difference value and a threshold value. The “threshold value” is set to a value large enough to assume that the imaging environment has changed. In addition, this “predetermined number” is such that the entire processing range A is changed (that is, not the local change of the image due to the indicator D entering the angle of view but the imaging environment is changed). It is a number.

  When the number of pixels whose difference value is equal to or greater than the threshold value is less than the predetermined number (STEP 52, NO), it is determined that the imaging environment has not changed (STEP 5, NO). On the other hand, if the number of pixels whose difference value is greater than or equal to the threshold value is greater than or equal to a predetermined number (STEP 52, YES), then whether the number of frames that satisfy the condition of STEP 52 (STEP 52 is YES) continues to be greater than or equal to the predetermined number It is determined whether or not (STEP 53). That is, it is determined whether there is a change in the imaging environment in the time direction. Note that the “predetermined number” is a number that allows the imaging environment to be continuously changed and regarded as not transient.

  When the number of frames satisfying the condition of STEP 52 is less than the predetermined number, it is determined that the imaging environment has not changed (STEP 5, NO). On the other hand, when the number of frames satisfying the condition of STEP 52 exceeds a predetermined number, it is determined that the imaging environment has changed (STEP 5, YES). If the left image processing unit 12L determines that the imaging environment has changed, the left image processing unit 12L updates the left background image stored in the left background image storage unit 13L (STEP 6). For example, the left image acquired in the latest STEP 1 is stored again as the background image.

  When the left background image storage unit 13L updates the left background image (STEP 6) or the left image processing unit 12L determines that the imaging environment has not changed (STEP 5, NO), the left image processing unit 12L A left difference image is calculated from the left background image stored in the image storage unit 13L and the left image acquired in the most recent STEP 1 (STEP 7). Note that the method for calculating the left difference image may be the same as the method described in STEP 51. If it is determined in STEP 5 that the imaging environment has not changed, STEP 7 may be omitted by applying the left side difference image calculated in STEP 51 to subsequent operations.

  Then, the left image processing unit 12L, from each pixel of the generated left difference image, a pixel that displays the indicator D (hereinafter referred to as an indicator display pixel) and a pixel that displays the background (hereinafter, background display pixel). (STEP 8). For example, by the same method as in STEP 52, a pixel whose difference value is greater than or equal to a threshold is set as an indicator display pixel, and a pixel whose difference value is smaller than the threshold is set as a background display pixel. The “threshold value” is set to a value large enough to accurately determine that the indicator is the indicator D. Further, the left image processing unit 12L may generate an image (so-called binarized image) in which the indicator display pixel and the background display pixel of the left background image are expressed with different signal values.

  Further, the left image processing unit 12L determines whether or not there are a predetermined number or more of indicator display pixels in the left difference image (including the binarized image) within the detection range (STEP 9). The detection range is at least a range within the processing range A, and is preferably a range on the surface of the display area 22 into which the indicator D can enter. Further, the “predetermined number” is a number that can be accurately determined as the indicator D, and is, for example, a number that can eliminate a pixel-unit variation caused by noise or the like.

  When the left image processing unit 12L determines that there are a predetermined number or more of the indicator display pixels in the left difference image (or binarized image) within the detection range (STEP 9, YES), the left image processing unit 12L The direction in which the indicator D exists is specified (STEP 10). For example, the angle α is calculated by the method described above (see FIG. 3).

  The indicator position calculation unit 14 calculates the position of the indicator D on the surface of the display region 22 based on the angle α and the angle β obtained in STEP 10 by, for example, the method described above (STEP 11). Then, the indicator position calculation unit 14 outputs the calculated position of the indicator D to the display device 20 as indicator position information.

  The indicator distance calculation unit 15 is also the same, and calculates the distance between the indicator D and the surface of the display area 22 by the above-described method (STEP 12). Then, the indicator distance calculation unit 15 outputs the calculated distance between the indicator D and the display area 22 surface to the display device 20 as indicator distance information. This STEP 12 may be performed before STEP 10 and STEP 11, or may be performed in parallel.

  After the indicator distance calculation unit 15 calculates the distance between the indicator D and the surface of the display area 22 (STEP 12), or the left image processing unit 12L determines that the indicator display pixels are less than the predetermined number (STEP 9, NO ) When the operation of the pointer detection apparatus 10 is to be terminated (STEP 13, YES), the operation is terminated. On the other hand, when the operation is continued without ending (STEP 13, NO), the process returns to STEP 1.

  By the way, “Left”, “L”, and “α” in the descriptions of STEP 1 to STEP 10 described above are appropriately replaced with “Right”, “R”, and “β”, so that the right imaging unit 11R and the right image processing are performed. The specific operations of the unit 12R and the right background image storage unit 13R can be grasped.

  If comprised as mentioned above, it will become possible to detect the distance of the indicator D and the display area 22 surface by recognizing the indicator D from at least one of a left side image and a right side image. In particular, it is possible to detect the distance between the indicator D and the surface of the display area 22 simply and quickly without requiring the recognized indicator D to be compared with a model or the like.

  Furthermore, it is possible to detect the distance between the indicator D and the display area 22 surface from any one of the left image and the right image. That is, it is possible to reduce the amount of calculation required for detecting the distance.

  Further, the indicator position detector 14 and the indicator distance detector 15 both indicate the position of the indicator D on the surface of the display area 22 based on the position in the image of the indicator D recognized from the left image and the right image. And the distance of the indicator D and the display area 22 surface is detected. Therefore, it is possible to share the detection process. Therefore, it is possible to simplify and speed up the processing of the pointer detection apparatus 10.

  In addition, since the left image and the right image are generated by capturing visible light, it is possible to detect the distance between the indicator D and the surface of the display region 22 with high accuracy. In particular, in the touch panel 1 arranged outdoors, various disturbances occur when the indicator D enters the surface of the display area 22 that is a detection surface, and thus the detection accuracy deteriorates. However, by using at least one of the left image and the right image generated by imaging visible light as described above, it is possible to accurately distinguish the disturbance from the indicator D.

  Further, by providing the left image and the right image with a plurality of types of signal values for one pixel, the indicator D is recognized from the left image and the right image based on the plurality of indices. Therefore, unlike the above-described conventional technique, one pixel is less susceptible to the disturbance than recognizing the indicator D in the image having only a single signal value (signal value indicating the intensity of infrared rays). It becomes possible. Therefore, the indicator D can be recognized with high accuracy from the left image and the right image.

  Further, by detecting the distance between the indicator D and the surface of the display area 22 based on the position in the image of the indicator D recognized from at least one of the left image and the right image, the instruction can be performed by simple processing. It becomes possible to detect the distance between the body D and the display area 22 surface. In particular, it is possible to detect the distance between the indicator D and the surface of the display area 22 without analyzing the size and shape of the indicator D in the left and right images.

  Further, when at least one optical axis of the left imaging unit 11L and the right imaging unit 11R is configured to be in the same plane as the display area 22, the display is performed in the generated left image or right image. This is preferable because the length of the region 22 surface in a predetermined direction (a direction substantially parallel to the optical axis) can be shortened. With this configuration, the position of the surface of the display region 22 that is directly below the indicator D in the left image or the right image varies depending on the movement of the indicator D (movement in a direction substantially parallel to the optical axis). It becomes possible to suppress. Therefore, it is possible to easily calculate the length between the indicator D and the display area 22 surface in the left image or the right image.

  In addition, the left imaging unit 11L and the right imaging unit 11R are provided at both ends of one end of the rectangular display region 22 serving as a detection surface, so that an image from a position close to the display region 22 can be obtained on the display region 22 surface. It is possible to generate a left image and a right image in which the entire image is effectively contained within the angle of view. Therefore, it is possible to generate a left side image and a right side image that are not easily affected by disturbance and have few blind spots.

  Further, when the left image processing unit 12L (or the right image processing unit 12R) determines in STEP 5 that the imaging environment has changed, the left background image storage unit 13L (or the right background image storage unit 13R) records it. Update the left background image (or right background image). Therefore, it is possible to update the background image in response to changes in the imaging environment. Therefore, it is possible to recognize the indicator D in the left image and the right image with high accuracy while suppressing the influence of disturbance.

  The definition of the angle α and the angle β shown in FIGS. 3 and 4 and the image capturing method (for example, a lens to be used) by the left imaging unit 11L and the right imaging unit 11R are merely examples, and the display area 22 plane Any other definition and imaging method may be applied as long as the position of the upper indicator D can be detected. For example, the angle α and the angle β may be defined with the upper side of the non-display area 21 as a reference, or an imaging method in which an image in which the edge of the display area 22 is curved may be applied.

  Moreover, although illustrated about the structure provided with two imaging parts, 11L of left side imaging parts, and 11R of right side imaging parts, it is good also as a structure provided with three or more imaging parts. The same applies to the image processing unit and the background image storage unit.

  In addition, the left image processing unit 12L, the right image processing unit 12R, the left background image storage unit 13L, and the right background image storage unit 13R are displayed as separate bodies on the block diagram shown in FIG. It doesn't matter.

  Further, the left image processing unit 12L calculates a left difference image from the left image and the left background image, and the right image processing unit 12R calculates a right difference image from the right image and the right background image, whereby the left image and the right side image are calculated. Although the method of recognizing the indicator D from each of the images has been illustrated, the method of recognizing the indicator D is not limited to this. For example, the indicator D is recognized by comparing at least two left images generated by continuous imaging (for example, calculating the difference image as described above) and comparing at least two right images in the same manner. It doesn't matter.

  In FIG. 6, when the processing range A of STEP 4 is set for at least one of the left image and the right image, or when STEP 6 is updated for at least one of the left background image and the right background image. Returning to STEP 1, the left image processing unit 12L may re-acquire the left image newly captured by the left imaging unit 11L, and the right image processing unit 12R may re-acquire the right image newly captured by the right imaging unit 11R. Absent. In addition, when one of the left image processing unit 12L and the right image processing unit 12R sets the processing range A of STEP 4 or updates the left background image or the right background image of STEP 6, the other performs the same operation. It doesn't matter.

  Further, in STEP 9 of FIG. 6, the detection range of the pointer position calculation unit 14 and the pointer distance calculation unit 15 is made common, but may be different. For example, the indicator position calculation unit 14 may use a very close area on the display area 22 as the detection range. Further, for example, the indicator distance calculation unit 15 may set a detection range from a very close area on the display area 22 to a certain distance.

  Further, the touch panel may be configured as shown in FIGS. FIG. 9 is a plan view and a side view showing another example of the configuration of the touch panel according to the embodiment of the present invention. FIG. 9A is a plan view showing a surface on which an image of the touch panel is displayed, and corresponds to FIG. On the other hand, FIG. 9B is a side view showing the left side surface of the touch panel with the surface shown in FIG. 9A on the right side, and corresponds to FIG. FIG. 10 is a schematic diagram illustrating an example of an image generated by the left imaging unit or the right imaging unit, and corresponds to FIG. 3B or 4B.

  As illustrated in FIG. 9A, the touch panel 1 a according to the present embodiment includes a frame F that is disposed on the non-display area 21. The frame body F is arranged around the display area 22 which is a detection surface of the pointer detection apparatus 10. FIG. 9A shows an example in which the frame F is arranged on the left side, the right side, and the lower side of the display area 22. You may arrange as follows. For example, the frame body F may be arranged so as to surround the entire circumference of the display area 22.

  The frame F may be any color. However, a color with little light reflection (for example, black or gray) or a color opposite to the indicator D (that is, a complementary color, for example, the frame F and the indicator D are displayed in the left image and the right image, respectively. It is preferable that the difference value of the signal values of RGB, YUV, HSB, etc. of the pixel is large.

  If comprised in this way, the fluctuation | variation of the imaging environment on the display area 22 surface will be suppressed by the frame F. FIG. Therefore, it is possible to accurately recognize the indicator D in the left image and the right image.

  In addition, by changing the color of the frame F to a color with little light reflection (for example, black or gray), it is possible to further suppress fluctuations in the imaging environment.

  On the other hand, by making the color of the frame F complementary to that of the indicator D, the frame F (that is, the background) and the indicator D in the left image and the right image generated by imaging visible light are made clearer. (For example, the signal value (difference value) of the indicator display pixel in the difference image can be made larger). Therefore, the indicator D in the left image and the right image can be recognized with higher accuracy.

  By the way, when it is set as the structure shown in FIG.9 and FIG.10, you may read "the display area" of STEP2 and STEP3 of FIG. Furthermore, in STEP4, the range in which the frame F is displayed in the left image and the right image may be set as the processing range. Further, STEP 5 (determining whether there is a change in the imaging environment) and STEP 6 (updating the left background image and right background image) may not be performed.

<Modification>
About the indicator detection apparatus 10 in embodiment of this invention, it is good also as control apparatuses, such as a microcomputer, performing part or all operation | movement. Further, all or part of the functions realized by such a control device is described as a program, and the program is executed on a program execution device (for example, a computer) to realize all or part of the functions. It doesn't matter if you do.

  In addition to the above-described case, the touch panels 1 and 1a illustrated in FIGS. 1 and 9 and the pointer detection apparatus 10 illustrated in FIG. 2 can be realized by hardware or a combination of hardware and software. When a part of the touch panel 1 or 1a or the indicator detection device 10 is configured using software, a block for a part realized by software represents a functional block of the part.

  As mentioned above, although embodiment in this invention was described, the range of this invention is not limited to this, It can add and implement various changes in the range which does not deviate from the main point of invention.

  INDUSTRIAL APPLICABILITY The present invention can be used for a pointer detection device that detects a pointer on a detection surface, and a touch panel that includes the pointer detection device.

DESCRIPTION OF SYMBOLS 1,1a Touch panel 10 Indicator detection apparatus 11L Left imaging part 11R Right imaging part 12L Left image processing part 12R Right image processing part 13L Left background image storage part 13R Right background image storage part 14 Indicator position calculation part 15 Pointer distance calculation Part 20 Display device 21 Non-display area 22 Display area D Indicator F Frame

Claims (6)

An indicator detection device for detecting an indicator on a detection surface,
At least one imaging unit that captures an image on the detection surface substantially parallel to the detection surface to generate an image;
A distance detection unit that detects a distance between the indicator and the detection surface based on a position of the indicator in an image generated by imaging of the imaging unit;
An indicator detection apparatus comprising: A position detection unit that detects the position of the indicator on the detection surface based on the position of the indicator in an image generated by two or more imaging units that capture the detection surface from different locations,
The pointer detection apparatus according to claim 1, further comprising:   The distance detection unit detects a distance between the indicator and the detection surface based on a position of the indicator in an image generated by any one of the imaging units. Or the indicator detection apparatus of Claim 2.   The pointer detection apparatus according to claim 1, wherein the imaging unit generates an image having a plurality of signal values in one pixel by imaging visible light.   5. The pointer detection apparatus according to claim 1, wherein an optical axis of the imaging unit exists in a plane substantially the same as the detection surface. A display device for displaying an image in a display area;
The pointer detection apparatus according to any one of claims 1 to 5, wherein the display area is a detection surface;
A touch panel comprising:

Images