JP2011118465A - Position detecting device, imaging apparatus, method and program for detecting position, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect the position of an imaging object contacting an imaging surface even when the imaging object moves on an optical differential touch panel. <P>SOLUTION: A touch position detecting device 1 includes: an image adjustment part 2 which obtains lighting up images and lighting off images; first and second smoothing image generation parts 4a, 4b which generate smoothing images in which the lighting off images obtained by the image adjustment part 2 are more smoothed than the lighting up images; first and second differential image generation parts 5a, 5b which generate differential images indicating the differences between the lighting up images and the smoothing images; and first and second touch position detection parts 6a, 6b which detect the positions of the images of the imaging objects contacting the imaging panel included in the differential images. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a position detection device that detects the position of a pointing object that instructs an imaging panel, an imaging device including the position detection device, and a position detection method.

  For example, Patent Document 1 discloses a touch panel that allows an input operation by touching a display screen (imaging surface) of a display device that can receive light (capture) in addition to display with a finger or the like.

  Further, in the display device described in Patent Document 2, in order to remove the influence of external light, as shown in FIGS. 23 and 24, in the state where the backlight is turned on and the backlight is turned off. An image is acquired by alternately repeating the imaging of the time, and a difference image is generated by taking a difference between an image in a lighting state (lighted image) and an image in a light-off state (light-off image). FIG. 23 is a diagram showing a difference image generated in a dark place. FIG. 24 is a diagram showing a difference image generated in a bright place.

  As a result, a portion where the backlight is reflected on the imaging target such as a finger touching or close to the display screen remains as a bright image, and a portion corresponding to the background becomes dark. By using such a difference image, it is possible to determine whether or not an imaging target such as a finger has touched the display screen without being affected by external light.

  However, in the method described in Patent Document 2, as shown in FIG. 25B, when the imaging target moves, there is a time difference between imaging in the lit state and imaging in the unlit state. The position of the image to be imaged is shifted. If the difference is taken with the position shifted, differential noise is generated at the position shifted position. FIGS. 25A and 25B are diagrams for explaining a problem when the touched finger moves. In FIGS. 2A and 2B, the lit image is referred to as “on image”, and the unlit image is referred to as “off image”.

  Such differential noise also occurs when the finger is moved while the finger is not in contact with the display screen, as shown in FIG. FIGS. 26A and 26B are diagrams for explaining a problem when a finger in a non-touch state moves.

  That is, in the difference image, a brighter part and a darker part than the actual part are generated, and such a part may erroneously determine that an imaging target that is not in contact with the display screen is in contact.

  In view of this, in the display imaging device of Patent Document 3, an interpolation image is generated using two unlit images, and the difference between the lit image and the interpolated image captured during the imaging timing of the two unlit images is calculated. Thus, the differential noise is reduced.

JP 2008-250949 A (published October 16, 2008) JP 2006-276223 A (released on October 12, 2006) JP 2009-205423 A (published September 10, 2009)

  In the invention of Patent Document 3, in order to solve the problem due to the difference in imaging timing, for example, images of different timings are taken along the time axis in the order of the first extinguished image, the illuminated image, and the second extinguished image. A new extinguished image interpolated from the extinguished image is generated and aligned with the illuminated image, so that noise generated after the difference is reduced and the position of the object can be detected stably.

  However, this method requires one extra extinguished image to generate the difference image, and also requires interpolation processing of the extinguished image, which increases the processing cost. In addition, if the object does not perform a constant velocity linear motion, differential noise is generated in the differential image.

  An object of the present invention is to generate a differential image with reduced differential noise even in the case where the imaging target moves other than the constant-velocity linear motion in the position detection device that takes the difference between the lit image and the unlit image, An object of the present invention is to provide a position detection device capable of accurately detecting the position of an imaging target in contact with an imaging surface and an imaging device including the position detection device.

  In order to solve the above problems, a position detection device according to the present invention is a position detection device that detects the position of an image to be picked up in a picked-up image. A lighting image that is an image obtained by imaging the imaging target with the imaging panel in a state where irradiation is performed, and an image obtained by imaging the imaging target with the imaging panel in a state where light irradiation is not performed, the lighting image and Is an acquisition unit that acquires an extinguished image captured with a predetermined time difference, and a smoothed image generation unit that generates a smoothed image obtained by smoothing the extinguished image acquired by the acquiring unit than the lighting image, A difference image generation unit that generates a difference image indicating a difference between the lighting image and the smoothed image generated by the smoothed image generation unit, and the imaging image included in the difference image generated by the difference image generation unit. It is characterized in that it comprises a position detector for detecting a position of the image of the imaging target in contact with Le.

  In order to solve the above problems, a position detection method according to the present invention is a position detection method in a position detection device that detects the position of an image to be captured in a captured image, and the imaging target from the side where the imaging panel is located A lighting image that is an image obtained by imaging the imaging target with the imaging panel in a state where light irradiation is performed on the image, and an image obtained by imaging the imaging target with the imaging panel in a state where the light irradiation is not performed, An acquisition step of acquiring a light-off image captured with a predetermined time difference from the lighting image, and a smoothed image that generates a smoothed image obtained by smoothing the light-off image acquired in the acquisition step from the lighted image A difference image generation step for generating a difference image indicating a difference between the generation step, the lighting image and the smoothed image generated in the smoothed image generation step, and the difference image generation Contained in the difference image generated in degree, it is characterized in that it comprises a position detection step of detecting a position of the image of the imaging target in contact with said imaging panel.

  According to said structure, an acquisition part acquires a lighting image and a light extinction image. The lighting image is an image obtained by imaging the imaging target with the imaging panel in a state where light irradiation is performed on the imaging target from the side where the imaging panel is arranged. The extinguished image is an image obtained by imaging the imaging target with the imaging panel in a state where light irradiation is not performed. Further, the lit image and the unlit image are captured with a predetermined time difference.

  The smoothed image generation unit generates a smoothed image obtained by smoothing the extinguished image acquired by the acquisition unit more than the lit image. The difference image generation unit generates a difference image indicating a difference between the lighting image and the smoothed image generated by the smoothed image generation unit. The position detection unit detects the position of the image to be imaged that is in contact with the imaging panel and is included in the difference image generated by the difference image generation unit.

  By taking the difference between the lit image and the unlit image, it is possible to subtract the brightness of the background of the imaging target and extract the image of the imaging target irradiated with light. However, in this case, when there is a difference in the imaging timing of the lit image and the unlit image, and the imaging target moves during that time, the position is between the image of the imaging target in the lit image and the image in the unlit image. Deviation occurs.

  By smoothing the extinguished image, the contour of the image to be imaged included in the extinguished image is blurred. For this reason, when the difference between the lit image and the smoothed unlit image (smoothed image) is taken, the difference in pixel values in the portion where the positional deviation occurs (outer edge portion of the image) is reduced. As a result, it is possible to reduce the change in the pixel value in the portion where the pixel value increases due to the positional deviation and the portion where the pixel value decreases.

  Therefore, differential noise caused by the movement of the imaging target can be reduced, and the position of the imaging target in contact with the imaging panel can be detected with high accuracy. Further, even when the imaging target moves other than the uniform linear motion, the differential noise can be reduced.

  The difference image generation unit preferably replaces the pixel value of a pixel having a pixel value smaller than 0 with 0 in the generated difference image.

  According to said structure, the pixel value of the pixel which has a pixel value smaller than 0 among the pixels contained in a difference image is substituted by 0. Even when the pixel value is negative, when the contour (edge) of the image to be imaged is detected by the gradient of the pixel value, the edge is detected by the pixel having the negative pixel value. Such an edge derived from a negative pixel value causes the image to be imaged to be erroneously detected.

  Therefore, with the above configuration, there is no pixel having a negative pixel value, so that the possibility of erroneous detection of an image to be captured can be reduced.

  Moreover, it is preferable that the said smoothed image production | generation part smoothes the said light extinction image using the spatial filter as a smoothing filter.

  With the above configuration, the unlit image is smoothed by the spatial filter. This spatial filter is, for example, an average value filter or a median filter. Therefore, by appropriately selecting the spatial filter, it is possible to set a preferable manner and degree of smoothing the extinguished image.

  Moreover, it is preferable that the said smoothed image production | generation part smoothes the said light extinction image using the smoothing filter smaller than the magnitude | size of the image of the contact part with the said imaging panel of the said imaging target assumed beforehand. .

  According to said structure, the smoothing filter used in order to smooth a light extinction image is a thing smaller than the magnitude | size of the image of the part in which an imaging target contacts an imaging panel. The size of the image of the portion where the imaging target contacts the imaging panel is assumed in advance, and the size of the smoothing filter is set in advance based on the size.

  When a smoothing filter larger than the size of the image of the contact portion is used, the entire image is significantly smoothed, and the pixel value becomes small. Therefore, the possibility that the image of the imaging target that is in contact with the imaging panel is not normally detected increases.

  Therefore, the above-mentioned side effects caused by smoothing the extinguished image by making the smoothing filter used for smoothing the extinguished image smaller than the size of the image of the portion where the imaging target is in contact with the imaging panel. Can be reduced.

  Moreover, it is preferable that the said smoothed image production | generation part produces | generates the said smoothed image using the smoothing filter of the magnitude | size of several steps corresponding to the multiple types of imaging target assumed beforehand.

  According to said structure, when smoothing a light extinction image, the smoothing filter of the magnitude | size of the several step corresponding to the multiple types of imaging target assumed beforehand is used. Therefore, smoothing at a level suitable for each imaging target can be performed.

  In addition, the smoothed image generation unit generates a plurality of smoothed images from one extinguished image using smoothing filters of a plurality of stages corresponding to a plurality of types of imaging targets assumed in advance, The difference image generation unit generates a plurality of difference images respectively indicating differences between the lighting image and the plurality of smoothed images generated by the smoothed image generation unit, and the position detection unit includes the difference image generation unit. For each of the plurality of difference images generated by the above, a pattern for detecting the image of the imaging target corresponding to the size of the smoothing filter used to generate the smoothed image that is the basis of the difference image It is preferable to detect the position of the image to be imaged included in each difference image.

  According to said structure, the smoothing filter of the magnitude | size corresponding to several steps is preset for every several types of imaging target. The smoothed image generation unit generates a plurality of smoothed images corresponding to the number of smoothing filters from one extinguished image using each of such smoothing filters. The difference image generation unit performs a process of obtaining a difference between the lighting image and one of the plurality of smoothed images generated by the smoothed image generation unit for each of the plurality of smoothed images.

  The smoothing filter used by the smoothed image generation unit and the pattern (detection pattern) for detecting the image to be imaged used by the position detection unit are associated with each other. For each of the plurality of generated difference images, the position detection unit uses each detection pattern corresponding to the size of the smoothing filter used to generate the smoothed image that is the basis of the difference image. The position of the image to be imaged included in the image is detected. Since an image that does not match the detection pattern is not detected as an image to be picked up, the position detecting unit finally detects the position of one image to be picked up if there is one kind of image pickup target.

  With this configuration, position detection processing corresponding to each of a plurality of types of imaging targets can be performed in parallel even in a situation where it is not known which imaging target is to be imaged among a plurality of types of imaging targets assumed in advance. Can do. Therefore, it is possible to reduce the amount of processing for position detection rather than performing the position detection processing for each imaging target in series.

  In addition, the smoothed image generation unit acquires external light intensity information indicating external light intensity that is the intensity of light around the imaging target at the time when the lighting image or the extinguishing image is captured, and the external light It is preferable to smooth the extinguished image only when the external light intensity indicated by the intensity information exceeds a predetermined reference value.

  When the external light intensity is low, the unlit image is entirely dark and the image to be imaged is not clearly visible. Therefore, even if the imaging target moves, the difference noise generated by taking the difference between the lit image and the unlit image does not occur remarkably.

  According to the above configuration, the smoothed image generating unit smoothes the extinguished image only when the external light intensity at the time when the illuminated image or the extinguished image is captured exceeds a predetermined reference value. In other words, the smoothed image generation unit does not smooth the extinguished image when the external light intensity is equal to or less than the predetermined reference value.

  Therefore, it is possible to reduce the amount of processing for position detection in a situation where the smoothing of the extinguished image is not necessarily required and the external light intensity is low.

  In addition, the smoothed image generation unit acquires external light intensity information indicating external light intensity that is the intensity of light around the imaging target at the time when the lighting image or the extinguishing image is captured, and the external light It is preferable to smooth the extinguished image using a smoothing filter having a size corresponding to the intensity of external light indicated by the intensity information.

  As the external light intensity increases, the difference noise generated by taking the difference between the lit image and the unlit image becomes more prominent. For this reason, it is preferable to use a smoothing filter having a larger size and a greater smoothing effect as the external light intensity increases.

  According to said structure, a smoothed image generation part smoothes a light extinction image using the smoothing filter of the size according to external light intensity | strength. For example, the smoothed image generation unit increases the size of the smoothing filter to be used in a stepwise manner as the external light intensity increases.

  Therefore, it is possible to smooth the unlit image suitable for the external light intensity (that is, suitable for the level of differential noise).

  Further, the position detection device generates a second difference image indicating a difference between the lit image and the unlit image acquired by the acquisition unit, and in the generated second difference image, an image of the imaging target assumed in advance is generated. A difference image determination unit that determines whether or not the number of pixels having a pixel value larger than the pixel value is equal to or greater than a predetermined number; and the smoothed image generation unit includes the difference image determination unit, It is preferable to smooth the unlit image only when it is determined that the number of images is equal to or greater than a predetermined number.

  According to said structure, a difference image determination part produces | generates the 2nd difference image which shows the difference of the lighting image and light extinction image which the acquisition part acquired, and in this 2nd difference image, the imaging target assumed beforehand It is determined whether or not the number of pixels having a pixel value larger than the pixel value of the image is equal to or greater than a predetermined number. If the number of pixels is equal to or greater than a predetermined number, there is a high possibility that differential noise is occurring at a predetermined level or higher. The smoothed image generating unit smoothes the extinguished image only in that case.

  Therefore, the smoothing process of the unlit image can be omitted in a situation where the smoothing of the unlit image is not necessarily required and the generation level of the differential noise is low.

  Further, the position detection unit extracts pixels having a pixel value greater than or equal to a predetermined value from the difference image generated by the difference image generation unit, and contacts the position of the image formed by the extracted pixels with the imaging panel. It is preferable to detect it as the position of the image to be imaged.

  According to said structure, the pixel which has a pixel value more than a predetermined value is extracted from a difference image. Since the image of the imaging target in contact with the imaging panel included in the difference image is brighter than the background portion, the image of the imaging target can be extracted by extracting pixels having pixel values greater than or equal to a predetermined value. The predetermined value is, for example, a value assumed in advance as a lower limit value of a pixel value of a pixel that forms an image to be imaged in contact with the imaging panel in the difference image. Then, the position of the image formed by the extracted pixels is detected as the position of the image to be imaged.

  Therefore, with the above configuration, it is possible to appropriately detect the position of the image to be imaged in contact with the imaging panel.

  Further, the position detection unit binarizes the difference image generated by the difference image generation unit with a predetermined value as a boundary, and the position of the image included in the binarized difference image is captured in contact with the imaging panel. It is preferable to detect the position of the target image.

  According to said structure, the image of the imaging target which contacted the imaging panel is extracted by binarizing a difference image on the basis of a predetermined value. The predetermined value is, for example, a value assumed in advance as a lower limit value of a pixel value of a pixel that forms an image to be imaged in contact with the imaging panel in the difference image. Then, the position of the image included in the binarized difference image is detected as the position of the image to be imaged.

  Therefore, with the above configuration, it is possible to appropriately detect the position of the image to be imaged in contact with the imaging panel.

  In addition, the position detection unit extracts pixels having edge strengths greater than or equal to a predetermined value from the difference image generated by the difference image generation unit, and detects the image of the imaging target with respect to the set of extracted pixels. It is preferable to detect the image of the imaging target in contact with the imaging panel by performing pattern matching with the pattern for the purpose.

  According to said structure, the pixel which has edge intensity more than predetermined value is extracted from a difference image. The image of the imaging target in contact with the imaging panel has a more prominent edge than the image of the imaging target that is not in contact with the imaging panel. Therefore, by extracting a pixel having an edge intensity equal to or greater than a predetermined value, an image of the imaging target that is in contact with the imaging panel is extracted. The predetermined value is, for example, a value assumed in advance as a lower limit value of the edge strength of the image to be imaged that is in contact with the imaging panel.

  Then, pattern matching with the detection pattern is performed on the extracted set of pixels. An image that matches the detection pattern is recognized as an image to be imaged in contact with the imaging panel.

  Therefore, with the above configuration, it is possible to appropriately detect the image of the imaging target in contact with the imaging panel.

  An imaging device according to the present invention is an imaging device including the position detection device and an imaging panel, and the imaging panel includes an illumination unit that performs light irradiation on the imaging target at predetermined time intervals, and the illumination. An imaging sensor that captures the lighting image and the extinguishing image in synchronization with the light irradiation timing of the imaging unit, and the acquisition unit acquires the lighting image and the extinguishing image captured by the imaging sensor. It is a feature.

  According to said structure, the illumination part with which the imaging panel was equipped performs light irradiation to an imaging target at a predetermined time interval. That is, the illumination unit repeats turning on and off alternately along the time axis. The imaging sensor captures a lit image and an unlit image in synchronization with the light irradiation timing of the illumination unit. That is, imaging is performed when the illumination unit is turned on and imaging is performed when the illumination unit is turned off. The acquisition unit of the position detection device acquires a lit image and an unlit image captured by the imaging sensor. Thereafter, processing in the smoothed image generation unit, the difference image generation unit, and the position detection unit is performed, and the position of the image to be imaged in contact with the imaging panel is detected.

  Therefore, even when the imaging target moves, the possibility of erroneously determining that the imaging target that is not in contact with the imaging panel is in contact can be reduced, and the position of the imaging target in contact with the imaging panel can be detected with high accuracy. .

  The imaging panel preferably includes a display backlight, and the illumination unit is preferably provided separately from the display backlight.

  According to the above configuration, the imaging panel includes the display backlight. When the display backlight is turned on / off to capture a lit image and an unlit image, if the display screen is dark, sufficient illumination light is not emitted even in the lit state, and the lit image becomes dark. As a result, the difference image may become completely dark. Further, when the display backlight is turned on / off, there is a problem that the display screen flickers. This problem can be solved by providing the illumination unit separately from the display backlight.

  Moreover, it is preferable that the said illumination part radiate | emits the light of the wavelength of a near infrared region or a near ultraviolet region.

  Light having a wavelength in the near-infrared or near-ultraviolet region has little influence on the display and is efficiently reflected on an imaging target such as a finger. Therefore, by making the illumination unit emit light having a wavelength in the near-infrared or near-ultraviolet region, the display can be easily seen and the detection accuracy of the imaging target can be increased.

  The imaging sensor preferably receives light having a wavelength in the near infrared or near ultraviolet region.

  With the above configuration, light having a wavelength in the near infrared or near ultraviolet region can be received, and a lighted image and a lighted image can be formed.

  The imaging sensor is preferably set to a sensitivity that does not saturate under sunlight irradiation.

  When the imaging sensor is saturated in an environment where the external light intensity is strong, complicated processing such as changing the shutter speed (exposure time) is required.

  With the above configuration, the imaging sensor does not saturate under sunlight irradiation (about 100,000 lux), so that complicated processing such as changing the shutter speed (exposure time) can be omitted.

  Further, a position detection program for operating the position detection device, a position detection program for causing a computer to function as each of the above-described units, and a computer-readable recording medium on which the position detection program is recorded are also within the technical scope of the present invention. included.

  As described above, the position detection device according to the present invention is a lighting image that is an image obtained by imaging the imaging target with the imaging panel in a state where light irradiation is performed on the imaging target from the side on which the imaging panel is located; An acquisition unit that acquires an image of the imaging target captured by the imaging panel in a state where the light irradiation is not performed, and an unlit image captured with a predetermined time difference from the lighting image, and the acquisition unit A difference image indicating a difference between a smoothed image generation unit that generates a smoothed image obtained by smoothing the acquired extinguishing image than the lighting image, and a smoothed image generated by the lighting image and the smoothed image generation unit. And a position detection unit that detects a position of an image to be imaged that is in contact with the imaging panel and is included in the difference image generated by the difference image generation unit.

Further, the position detection method according to the present invention includes a lighting image that is an image obtained by imaging the imaging target with the imaging panel in a state where the imaging target is irradiated with light from the side where the imaging panel is located, and the light irradiation. An acquisition process of acquiring an image of the imaging target captured by the imaging panel in a state in which the image is not captured, and an extinguished image captured with a predetermined time difference from the lighting image, and the extinguishing acquired in the acquisition process A smoothed image generating step for generating a smoothed image obtained by smoothing an image from the lighting image and a difference image indicating a difference between the lighting image and the smoothed image generated in the smoothed image generating step are generated. Difference image generation step;
And a position detection step of detecting the position of the image of the imaging target in contact with the imaging panel, which is included in the differential image generated in the differential image generation step.

  Therefore, it is possible to reduce differential noise caused by the movement of the imaging target, and to obtain an effect that the position of the imaging target in contact with the imaging panel can be detected with high accuracy.

It is a block diagram which shows the structure of the touch position detection apparatus which concerns on one Embodiment of this invention. It is sectional drawing which shows the structure of LCD with a built-in optical sensor. It is a graph which shows the relationship between the change of external light intensity, and the size of a smoothing filter. It is a graph which shows another relationship between the change of external light intensity, and the size of a smoothing filter. (A) is a figure which shows the production | generation process of the difference image when the finger of a touch state is stationary, (b) is a figure which shows the production | generation process of the difference image when the finger of a touch state moves. is there. (A) is a figure which shows the production | generation process of a difference image when the finger | toe of a non-touch state is stationary, (b) shows the production | generation process of the difference image when the finger | toe of a non-touch state moves. FIG. A graph showing a detection result when an image of a finger pad is detected by generating a differential image without smoothing an extinguished image obtained by capturing a finger in a stationary state and extracting pixels having pixel values equal to or greater than a predetermined threshold It is. A graph showing a detection result when an image of a finger pad is detected by generating a differential image without smoothing an extinguished image obtained by capturing a moving finger and extracting a pixel having a pixel value equal to or greater than a predetermined threshold It is. It is a graph which shows the detection result at the time of detecting the image of a finger pad by smoothing the light-off image which imaged the finger of a stationary state, generating a difference picture, and extracting the pixel which has a pixel value more than a predetermined threshold. . It is a graph which shows the detection result at the time of detecting the image of a finger pad by smoothing the light extinction image which imaged the finger of a movement state, generating a difference image, and extracting the pixel which has a pixel value more than a predetermined threshold. . It is a figure for demonstrating the graph shown in FIGS. It is a graph which shows the detection result at the time of detecting the image of a finger pad by producing | generating a difference image, without smoothing the light extinction image which image | photographed the finger | toe in a stationary state, and extracting an edge feature. It is a graph which shows the detection result at the time of detecting the image of a finger pad by producing | generating a difference image, without smoothing the light extinction image which imaged the finger of the movement state, and extracting an edge feature. It is a graph which shows the detection result at the time of detecting the image of a finger pad by smoothing the light-off image which image | photographed the finger | toe of a stationary state, producing | generating a difference image, and extracting an edge feature. It is a graph which shows the detection result at the time of detecting the image of a finger pad by smoothing the light extinction image which imaged the finger of the movement state, generating a difference image, and extracting an edge feature. It is a figure which shows the example which smooth | blunted the light-off image using the smoothing filter of the same size as the size of a contact partial image. It is a figure for demonstrating the significance of using the smoothing filter of the size corresponding to a pointing object. (A) is a figure for demonstrating the effect of an average value filter, (b) is a figure for demonstrating the effect of an intermediate value filter. It is a figure for demonstrating the reason for clipping the pixel value smaller than 0 in a difference image. It is a flowchart which shows an example of the flow of a process in the said touch position detection apparatus. It is the schematic which shows the structure of the touch position detection apparatus which concerns on another embodiment of this invention. It is a flowchart which shows an example of the flow of a process in the said touch position detection apparatus. It is a figure which shows the difference image produced | generated in the dark place. It is a figure which shows the difference image produced | generated in the bright place. (A) And (b) is a figure for demonstrating a problem when the finger of a touch state moves. (A) And (b) is a figure for demonstrating a problem when the finger | toe of a non-touch state moves.

[Embodiment 1]
The following describes one embodiment of the present invention with reference to FIGS. Here, as an example of the present invention, an image of an indication member (generally referred to as an indication object) such as a user's finger or a touch pen that designates an arbitrary position on the touch panel is captured, and the image is analyzed to be applied to the touch panel. A touch position detection device (optical difference type touch panel) 1 that detects a contact position of a pointing object that has touched the touch panel will be described. The touch position detection device 1 can also be expressed as a display device, an imaging device, a display imaging device, an input device, an information device, or an electronic device.

(Configuration of Touch Position Detection Device 1)
FIG. 1 is a block diagram illustrating a configuration of a touch position detection device 1 according to the present embodiment. As shown in FIG. 1, the touch position detection device (position detection device) 1 includes a touch panel unit (imaging panel) 10, a main control unit 20, a storage unit 30, and a primary storage unit 31.

(Configuration of touch panel unit 10)
The touch panel unit 10 includes an optical sensor built-in LCD (liquid crystal panel / display) 11, an AD (analog / digital) converter 15, a timing control unit 16, and a sensitivity adjustment unit 17.

  Since the optical sensor built-in LCD 11 has a built-in optical sensor (imaging sensor) 12, it can capture not only a display but also an image. Therefore, the optical sensor built-in LCD 11 functions as an imaging screen that captures an image (hereinafter referred to as a captured image) including an image of a pointing object such as a finger that is in contact with or close to the display screen of the optical sensor built-in LCD 11 as a touch panel. .

  The AD converter 15 digitizes the signal acquired by the optical sensor 12.

  The timing control unit 16 controls the timing for turning on / off the backlight 14 for the touch panel and the imaging timing by the optical sensor 12 to synchronize both.

  The optical sensor 12 includes a lighting image that is an image of the pointing object in a state in which the pointing object is irradiated with light from the touch panel backlight 14 and an image in which the pointing object is captured in a state where no light irradiation is performed. It is a light receiving element that captures a certain unlit image. At that time, the optical sensor 12 alternately captures the lit image and the unlit image according to the synchronization signal output from the timing control unit 16 at predetermined time intervals. That is, the unlit image is taken with a predetermined time difference from the lit image.

  For example, for each frame period of image display in the optical sensor built-in LCD 11, the touch panel backlight 14 is turned on in the first half of the frame period, and the touch panel backlight 14 is turned off in the second half of the frame period. In this case, the optical sensor 12 outputs the image captured in the first half period as a lit image to the image adjustment unit 2 via the AD converter 15, and adjusts the image captured in the second half period as a extinguished image via the AD converter 15. Output to part 2. The lit image and the unlit image captured in the same frame period are associated as a pair of captured images for generating a difference image. That is, a difference image is generated from a noticed lighting image among the lighting images and the lighting images captured alternately and the lighting image captured immediately after the captured image.

  The sensitivity adjustment unit 17 adjusts the sensitivity of each optical sensor 12 according to a user instruction or external light intensity. When the optical sensor 12 is saturated in an environment where the external light intensity is strong, complicated processing such as changing the shutter speed (exposure time) is required. The sensitivity adjustment unit 17 adjusts the sensitivity of the optical sensor 12 so that the optical sensor 12 is not saturated even under sunlight irradiation (about 100,000 lux). The sensitivity of the sensitivity adjustment unit 17 may be set in advance. The external light intensity is the intensity of light around the pointing object (imaging target).

(Details of LCD 11 with built-in optical sensor)
FIG. 2 is a cross-sectional view showing the configuration of the optical sensor built-in LCD 11. As shown in FIGS. 1 and 2, the LCD 11 with a built-in photosensor includes a plurality of photosensors 12, a display backlight 13, and a touch panel backlight (illumination unit) 14.

  For example, the photosensors 12 are arranged in a matrix on the active matrix substrate of the photosensor built-in LCD 11, and one photosensor 12 is provided for each pixel of the photosensor built-in LCD 11. The arrangement pattern and the number of the optical sensors 12 are not limited to those described above, and may be changed as appropriate.

  The optical sensor 12 mainly receives light having a wavelength in the near infrared (wavelength: about 750 to 2500 nm) or near ultraviolet (wavelength: about 200 to 380 nm) region. The reason is that, as will be described later, the touch panel backlight 14 emits light having a wavelength in the near-infrared or near-ultraviolet region, so that the light emitted from the touch panel backlight 14 can be mainly received. It has become. The optical sensor 12 does not react only to near-infrared light or near-ultraviolet light, and may react somewhat to light of other wavelengths.

  The signal acquired by the optical sensor 12 is digitized by the AD converter 15 and then output to the image adjustment unit 2.

  The optical sensor built-in LCD 11 includes a liquid crystal panel (not shown) including a color filter having R (red), G (green), and B (blue) picture elements. The display backlight 13 is a light source that irradiates light to the liquid crystal panel from the back surface of the liquid crystal panel. One pixel is formed by three picture elements of R, G, and B.

  The touch panel backlight 14 is an illuminating device that irradiates light to the pointing object, and repeats lighting / extinguishing at predetermined time intervals in accordance with a synchronization signal output from the timing control unit 16. That is, the touch panel backlight 14 irradiates the pointing object with light at predetermined time intervals.

  The touch panel backlight 14 emits light having a wavelength in the near infrared or near ultraviolet region. Light in the near-infrared or near-ultraviolet region has little effect on the display of a liquid crystal panel that uses visible light (wavelength: about 380 nm to 750 nm), and is efficient for pointing objects such as fingers as well as visible light. reflect. Therefore, by emitting light having a wavelength in the near-infrared or near-ultraviolet region, the display on the liquid crystal panel can be easily seen and the detection accuracy of the pointing object can be increased. Further, by using near-infrared light or near-ultraviolet light, the optical sensor 12 is preferably used in a bright environment such as a fluorescent lamp (visible light) or sunlight (including all of near-ultraviolet light, visible light, and near-infrared light). It can be made to withstand without saturating.

  The configuration and arrangement of the display backlight 13 and the touch panel backlight 14 may be known ones. For example, a cold cathode tube or an LED (Light Emitting Diode) can be used as the light source of the display backlight 13 and the touch panel backlight 14. You may use the edge light system which arrange | positions these light sources on the side surface of a light-guide plate, and radiate | emits light as backlight light from the upper surface of a light-guide plate. Further, a direct type system in which a light source is disposed on the back side of the light guide plate and light is emitted as backlight light from the upper surface of the light guide plate may be used.

  The display backlight 13 and the touch panel backlight 14 may be formed separately, and these backlights may be stacked in the emission direction of the backlight light.

  Further, the light guide plates of the display backlight 13 and the touch panel backlight 14 may be shared. For example, white LEDs are arranged along the side of the light guide plate as a light source for the display backlight 13, and near infrared LEDs are two-dimensionally arranged on the back side of the light guide plate as a light source for the backlight 14 for the touch panel. May be. Conversely, a white LED may be provided on the back side of the light guide plate, and a near infrared LED may be provided on the side surface side of the light guide plate.

(Configuration of main control unit 20)
The main control unit 20 includes an image adjustment unit 2, an external light intensity calculation unit 3, a first smoothed image generation unit 4a, a second smoothed image generation unit 4b, a first difference image generation unit 5a, and a second difference image generation unit. 5b, a first touch position detection unit 6a, a second touch position detection unit 6b, and an application execution unit 7.

  The image adjustment unit 2 performs a calibration process for adjusting the gain and offset of the captured images (lighted image and extinguished light image) captured by the touch panel unit 10, and outputs the adjusted captured image to the external light intensity calculation unit 3. To do. In the following description, it is assumed that a grayscale image is output with an accuracy of 8 bits and 256 gradations. The image adjustment unit 2 also functions as an acquisition unit that acquires a lighting image and a light-off image from the touch panel unit 10.

  The external light intensity calculation unit 3 uses the lighting image or the extinguishing light image output from the image adjustment unit 2 to calculate the external light intensity that is the light intensity around the pointing object when the lighting image or the extinguishing image is captured. calculate. For example, the external light intensity calculation unit 3 has a predetermined order when at least a part of output values (pixel values) output from the optical sensor 12 and indicating the amount of received light are arranged in descending order of the output values. The output value at is used as the external light intensity.

  That is, the external light intensity calculation unit 3 calculates the pixel value and the number of pixels having the pixel value when the pixel values of the pixels of the captured image are arranged in descending order with respect to the captured image acquired by the optical sensor 12. Create a histogram showing the relationship. This histogram is preferably created from the pixel values of all the pixels of the captured image. However, from the viewpoint of cost and processing speed, instead of obtaining from all the pixels constituting the captured image, pixels selected at regular intervals from the pixel values of the captured image may be used.

  Further, the external light intensity calculation unit 3 limits the area where the sample point is taken in the captured image, obtains the average value of the pixel values of the pixels in each limited area, and sets the average value to the highest value. You may make it employ | adopt as light intensity. As described above, any method may be used as a method for calculating the external light intensity from the captured image.

  Note that the illuminance sensor may be provided inside or outside the LCD 11 with the optical sensor to measure the intensity of the external light. However, in order to accurately calculate the intensity of the light incident on the optical sensor 12, the optical sensor 12 may be used. It is preferable to calculate from the acquired image. The illuminance sensor may measure the intensity of visible light or may measure the intensity of infrared light. That is, the external light intensity may be the intensity of visible light or the intensity of infrared light.

  However, since it is preferable to measure the intensity of external light that affects the captured image, light having the same wavelength region as the light emitted from the touch panel backlight 14 (in other words, the light received by the optical sensor 12). It is preferable to measure the intensity as the external light intensity. That is, it is preferable to measure light having a wavelength in the near infrared or near ultraviolet region as the external light intensity.

  The first smoothed image generation unit 4a and the second smoothed image generation unit 4b are smoothed by smoothing the extinguished image output from the image adjustment unit 2 by using a spatial filter as a smoothing filter, rather than the lighting image. Generate a digitized image. Although the lit image and the unlit image may be smoothed for image adjustment, the unlit image is smoothed separately from such smoothing. Therefore, the unlit image is smoothed more significantly than the lit image.

  At this time, the first and second smoothed image generation units 4a and 4b generate a smoothed image using a plurality of stages of smoothing filters corresponding to a plurality of types of pointing objects assumed in advance. Each smoothing filter is smaller than the size of an image of a contact portion of the pointing object with the optical sensor built-in LCD 11 (hereinafter referred to as a contact portion image) that is assumed in advance. The reason for this will be described later.

  The first and second smoothed image generation units 4a and 4b start the smoothing process from one of the four corners of the unlit image, and smooth the entire unlit image while shifting the smoothing filter by one pixel. To do.

  The first smoothed image generating unit 4a and the second smoothed image generating unit 4b are different from each other in the size of the smoothing filter used for smoothing. Other than that, it has the same function. The first smoothed image generating unit 4a smoothes the extinguished image using a smoothing filter smaller than the image of the finger pad touching the optical sensor built-in LCD 11 under the assumption that the pointing object is a finger. . Based on the assumption that the pointing object is a pen, the second smoothed image generating unit 4b smoothes the extinguished image using a smoothing filter smaller than the image of the pen tip in contact with the LCD 11 with a built-in optical sensor. .

  That is, when the first smoothed image generating unit 4a and the second smoothed image generating unit 4b are regarded as one smoothed image generating unit, the smoothed image generating unit includes a plurality of types assumed in advance. Using the smoothing filters of a plurality of stages corresponding to each of the pointing objects, a number of smoothed images corresponding to the number of pointing objects assumed in advance are generated from one unlit image.

  Further, the first and second smoothed image generation units 4a and 4b acquire the external light intensity information indicating the external light intensity from the external light intensity calculation unit 3, and according to the external light intensity indicated by the external light intensity information The extinguished image is smoothed using a size smoothing filter. FIG. 3 is a graph showing the relationship between the change in external light intensity (infrared intensity) and the size of the smoothing filter. For example, as shown in FIG. 3, when a finger is assumed as the pointing object, if smoothing is not counted as one step, 0, 3 × 3, 5 × 5, 7 × 7 4 A level smoothing filter is set in advance. Then, the first smoothed image generation unit 4a increases the size of the smoothing filter to be used as the external light intensity increases. The reason for this will be described later.

  FIG. 4 is a graph showing another relationship between the change in external light intensity (infrared intensity) and the size of the smoothing filter. As shown in FIG. 4, the first and second smoothed image generating units 4a and 4b determine whether or not the external light intensity exceeds a predetermined reference value. The extinguishing image may be smoothed using a smoothing filter. This figure shows an example in which a 7 × 7 smoothing filter is used when the intensity of infrared light exceeds a medium level.

  Similarly, a four-stage smoothing filter is set in advance when a pen is assumed as the pointing object. However, since each smoothing filter is smaller than the size of the contact partial image of the pointing object assumed in advance, the size of the smoothing filter is set so as to satisfy this condition. That is, the largest smoothing filter among the smoothing filters in a plurality of stages is smaller than the size of the contact partial image of the pointing object that is assumed in advance. It is assumed that the size of the contact part image of the finger is 9 × 9, and the size of the contact part image of the pen is 3 × 3.

  The first and second smoothed image generation units 4a and 4b attach information indicating the correspondence between the smoothed unlit image and the lit image to the generated smoothed image, and the smoothed image is displayed as the first and second smoothed images. It outputs to the 2nd difference image generation part 5a * 5b.

  The 1st difference image generation part 5a and the 2nd difference image generation part 5b generate | occur | produce the difference image which shows the difference of a lighting image and the smoothed image which the 1st, 2nd smoothed image generation part 4a * 4b produced | generated. Specifically, the first difference image generation unit 5a uses the pixel value of each pixel of the lighting image output from the image adjustment unit 2 to generate a smoothed image (first smoothing image generation) associated with the lighting image. In the smoothed image smoothed by the unit 4a), a difference image having a pixel value obtained by subtracting the pixel value of the pixel corresponding to each pixel of the lighting image as the pixel value of each pixel is generated. Similarly, the second difference image generation unit 5b generates a difference image from the lighting image output from the image adjustment unit 2 and the smoothed image smoothed by the second smoothed image generation unit 4b.

  At this time, the first and second difference image generation units 5a and 5b replace the pixel value of the pixel having a pixel value smaller than 0 with 0 in the generated difference image. The reason will be described later.

  When the first difference image generation unit 5a and the second difference image generation unit 5b are regarded as one functional block, the difference image generation unit includes a lighting image and a plurality of smoothed images generated by the smoothed image generation unit. It can be said that a plurality of difference images each indicating a difference from the digitized image are generated.

  The first touch position detection unit 6a detects the position (touch position) of the contact partial image included in the difference image generated by the first difference image generation unit 5a. The first smoothed image generating unit 4a smoothes the extinguished image using a smoothing filter smaller than the image of the finger pad touching the optical sensor built-in LCD 11 under the assumption that the pointing object is a finger. Therefore, the first touch position detection unit 6a performs processing for detecting an image of the finger pad touching the optical sensor built-in LCD 11.

  Similarly, the second touch position detection unit 6b detects the position of the contact partial image included in the difference image generated by the second difference image generation unit 5b. Based on the assumption that the pointing object is a pen, the second smoothed image generating unit 4b smoothes the extinguished image using a smoothing filter smaller than the image of the pen tip in contact with the LCD 11 with a built-in optical sensor. Therefore, the second touch position detection unit 6b performs a process of detecting the image of the pen tip that is in contact with the optical sensor built-in LCD 11.

  Thus, the 1st touch position detection part 6a and the 2nd touch position detection part 6b have the same function except the point from which the object to detect differs. When there is only one type of pointing object that touches the optical sensor built-in LCD 11, either the first touch position detection unit 6a or the second touch position detection unit 6b detects the image. Therefore, the first touch position detection unit 6a and the second touch position detection unit 6b output coordinates indicating the position of the detected image to the application execution unit 7, but the coordinates actually output to the application execution unit 7 are , Usually one. However, when the optical sensor built-in LCD 11 is touched with a plurality of fingers, a plurality of coordinates indicating the positions of the images of the plurality of finger pads are output to the application execution unit 7.

  A known method may be used as a touch position detection method in the first touch position detection unit 6a and the second touch position detection unit 6b. Specific examples thereof will be described later.

  When the first touch position detection unit 6a and the second touch position detection unit 6b are regarded as one functional block, the touch position detection unit performs the processing for each of the plurality of difference images generated by the difference image generation unit. Using the pattern for detecting the contact partial image of the pointing object corresponding to the smoothing filter used to generate the smoothed image that is the basis of the difference image, the contact partial image included in each difference image It can be said that the position is detected.

  The application execution unit 7 executes an application corresponding to the coordinates using the coordinates output from the first touch position detection unit 6a or the second touch position detection unit 6b, or in a specific application, Perform the corresponding process. Any application executed in the application execution unit 7 may be used.

  In the case of the configuration that accepts information indicating whether the optical sensor built-LCD11 is touched by the advance which type of pointing object from the user, or, in the case of not performing the position detection in accordance with the type of pointing object is smoothed One image generation unit, one difference image generation unit, and one touch position detection unit may be provided.

  Further, even if the light sensor-containing LCD11 in advance by any type of pointing object does not acquire information indicating whether the touch, smoothed image generating unit, one by one difference image generation unit and the touch position detection unit is provided, these Each unit may sequentially perform processing for detecting images of all types of pointing objects. However, in this case, since processing time is required, it is preferable to perform a plurality of processes according to the type of the pointing object in parallel as in the present embodiment.

(Details of storage unit)
The storage unit 30 records (1) a control program for each unit, (2) an OS program, (3) an application program, and (4) various data to be read when the program is executed by the main control unit 20. It is. The storage unit 30 is configured by a nonvolatile storage device such as a hard disk or a flash memory.

  The primary storage unit 31 is configured by a volatile storage device such as a RAM (Random Access Memory), and is used for temporarily holding data in the course of the main control unit 20 executing the various programs described above. Used as a work area.

(Example of touch position detection method)
Three specific examples of the touch position detection processing in the first and second touch position detection units 6a and 6b will be described.

(First example)
In the first example, as described in Patent Document 1, the first and second touch position detection units 6a and 6b have edge strengths equal to or greater than a predetermined value from the difference image generated by the difference image generation unit. Pixels are extracted, and a feature of the extracted set of pixels is compared with a feature indicated by the model pattern, thereby detecting the position of the image of the imaging target in contact with the imaging panel.

  Specifically, the first and second touch position detection units 6a and 6b are configured to detect the vertical gradient amount indicating the gradient between the pixel value of the pixel adjacent to the target pixel in the vertical direction and the pixel value of the target pixel and the target pixel. A process of calculating a horizontal gradient amount indicating the gradient between the pixel value of the pixel adjacent in the horizontal direction and the pixel value of the target pixel is performed for each pixel of the difference image. Then, by using the vertical gradient amount and the horizontal gradient amount, an edge pixel that is a portion where the brightness changes rapidly is specified, and a set of vectors indicating the gradient of the pixel value in the edge pixel is extracted. This set of vectors corresponds to the above-mentioned “features of a set of pixels having edge strengths equal to or greater than a predetermined value”.

  Each model pattern includes vector information (predetermined features) indicating the gradient of the pixel value, and the first and second touch position detection units 6a and 6b include a set of vectors extracted from the difference image, Pattern matching is performed with a set of vectors included in the model pattern, and it is determined whether or not an image to be detected exists based on the degree of coincidence. Specifically, a predetermined model pattern composed of a plurality of pixels indicating the gradient direction of the pixel value and a pattern in the gradient direction indicated by the feature amount extracted from the difference image are pattern-matched, and the gradient included in the model pattern A region where the number of pixels matching the direction reaches a predetermined number is detected as a contact partial image.

  By this detection processing, whether the image included in the difference image is the image of the pointing object that has touched the optical sensor built-in LCD 11 (touch image) or the image of the pointing object that has not touched (non-touch image) Judgment has been made.

  Each model pattern has a shape and size corresponding to the size and shape of the pointing object to be detected. When the pointing object is a finger and a pen, a model pattern for finger pad detection and a model pattern for pen tip detection are stored in the storage unit 30 in advance.

  The first touch position detection unit 6a performs pattern matching using a model pattern for finger pad detection and detects a finger pad image. On the other hand, the second touch position detection unit 6b performs pattern matching using a model pattern for detecting the pen tip, and detects an image of the pen tip.

  Then, the first and second touch position detection units 6a and 6b calculate barycentric coordinates (or center coordinates) of the detected images, and output the coordinates to the application execution unit 7 as touch positions.

(Second example)
In the second example, the first and second touch position detection units 6a and 6b extract pixels having pixel values greater than or equal to a predetermined value from the difference image, and contact the positions of the images formed by the extracted pixels. It is detected as the position of the partial image. Specifically, the first and second touch position detection units 6a and 6b extract pixels having a pixel value equal to or greater than a predetermined threshold among the pixels included in the difference image. Since the contact partial image in the difference image has a larger pixel value than the background, the contact partial image is extracted by setting the minimum pixel value of the pixels constituting the average contact partial image as the predetermined threshold value. it can.

  By this extraction processing, whether the image included in the difference image is the image of the pointing object that has touched the LCD 11 with the optical sensor (touch image) or the image of the pointing object that has not touched (non-touch image) Judgment has been made.

  Then, the first and second touch position detection units 6a and 6b calculate the barycentric coordinates (or center coordinates) of the image formed by the extracted pixels, and output the coordinates to the application execution unit 7 as touch positions.

(Third example)
In the third example, as described in Patent Document 2, the first and second touch position detection units 6a and 6b binarize a difference image with a predetermined value as a boundary, and binarize the difference image. Is detected as the position of the contact partial image. The threshold for binarization may be the same as the threshold in the second example. The binarization means, for example, that a pixel having a pixel value equal to or greater than a predetermined threshold is white and a pixel having a pixel value less than the predetermined threshold is black.

  By this binarization processing, the image included in the difference image is an image of the pointing object that is in contact with the optical sensor built-in LCD 11 (touch image) or an image of the pointing object that is not in contact (non-touch image). It is determined whether or not.

  The first and second touch position detection units 6a and 6b specify a white area in the binarized difference image, calculate barycentric coordinates (or center coordinates) of the specified white area, and use the coordinates as the touch position. To the application execution unit 7.

  In the second and third examples, the type of the pointing object cannot be determined. Image detection using a model pattern may be performed in order to determine the type of the pointing object. Specifically, the first touch position detection unit 6a performs pattern matching on the image formed by the pixels extracted from the difference image using a model pattern for finger pad detection, and detects the finger pad image. To do. On the other hand, the second touch position detection unit 6b performs pattern matching using a model pattern for detecting the pen tip, and detects an image of the pen tip. Unlike the first example, the model pattern used at this time does not include vector information and indicates the shape and size of the image.

(Reason for smoothing the difference image)
Next, the reason for smoothing the difference image will be described. FIG. 5A is a diagram showing a difference image generation process when the touched finger is stationary, and FIG. 5B is a difference image generation process when the touched finger is moved. FIG. FIG. 6A is a diagram illustrating a difference image generation process when the finger in the non-touch state is stationary, and FIG. 6B is a diagram illustrating the difference image when the finger in the non-touch state is moved. It is a figure which shows a production | generation process.

  When the finger moves while the finger is in contact with the optical sensor built-in LCD 11 (touched state), differential noise is generated in the differential image. However, as shown in FIG. 5B, the extinguished image (off image) is displayed. When the difference image is generated after smoothing, the difference noise is reduced.

  Even when the finger moves in a state where the finger is not in contact with the optical sensor built-in LCD 11 (non-touch state), a difference noise may be generated in the difference image. However, as shown in FIG. When the difference image is generated after smoothing the image (off image), the difference noise is reduced.

  Here, the effect of reducing this differential noise will be described quantitatively. FIG. 7 shows the detection when the finger pad image is detected by generating a difference image without smoothing the extinguished image obtained by capturing the finger in a stationary state and extracting pixels having pixel values equal to or greater than a predetermined threshold. It is a graph which shows a result. FIG. 8 shows the detection when the image of the finger pad is detected by generating a differential image without smoothing the extinguished image obtained by capturing the moving finger and extracting pixels having pixel values equal to or greater than a predetermined threshold. It is a graph which shows a result. FIG. 9 shows a detection result when the image of the finger pad is detected by smoothing the extinguishing image obtained by capturing a finger in a stationary state and generating a differential image and extracting pixels having pixel values equal to or greater than a predetermined threshold. It is a graph to show. FIG. 10 shows a detection result when the image of the finger pad is detected by smoothing the extinguished image obtained by capturing the moving finger and generating a differential image and extracting pixels having pixel values equal to or greater than a predetermined threshold. It is a graph to show.

  In these figures, a result (touch) in a state where the finger is in contact with the optical sensor built-in LCD 11 and a result (non-touch) in a state where the finger is not in contact with the optical sensor built-in LCD 11 are shown side by side. In the result of touch and non-touch, “0lx” is a result when a lit image and an unlit image are captured in the absence of external light. “Low illuminance” is a result when the external light intensity is ½ of the intensity of the reflected light reflected by the finger pad on the touch panel backlight 14. “Medium illuminance” is the result when the external light intensity is equal to the intensity of the reflected light. “High illuminance” is the result when the external light intensity is twice the intensity of the reflected light.

  FIG. 11 is a diagram for explaining the graphs shown in FIGS. 7 to 10. In each graph shown in FIGS. 7 to 10, the horizontal axis indicates the position of each pixel in the horizontal row of pixels of the difference image including the image of the finger pad as shown in FIG. 11. . The vertical axis indicates the pixel value of each pixel. Further, the pixel value “100” is set as a threshold value for determining whether touch or non-touch. That is, a pixel having a pixel value of 100 or more is recognized as a pixel that forms a contact partial image. This touch position detection method corresponds to the second example described above.

  The size of the contact partial image included in the unlit image is 9 × 9, and the size of the smoothing filter used for smoothing the unlit image is 3 × 3. That is, a smoothing filter having a size smaller than the size of the contact partial image is used. The smoothing filter used is an average value filter.

  As shown in FIGS. 7 and 9, when the finger is stationary, touch / non-touch identification is performed without any problem regardless of the external light intensity. However, as shown in FIG. 8, when the finger moves, a pixel (high luminance pixel) having a pixel value exceeding the threshold value is generated, although it is actually non-touch in the case of high illuminance. In this case, an image of a finger not in contact with the optical sensor built-in LCD 11 is recognized as an image of a finger in contact.

  On the other hand, as shown in FIG. 10, if the extinguished image is smoothed, the number of high-luminance pixels generated when the finger moves is suppressed as compared with the case where the finger is not smoothed. In the result shown in FIG. 10, the high-luminance pixel in the case of non-touch is not set to 0, but this does not deny the effect of the present invention. This is because, even if high-luminance pixels are detected, if the number is small, an image composed of the high-luminance pixels is not an object of the center-of-gravity coordinate calculation process, so that the touch position is not finally calculated.

  A similar effect can be obtained when the first example of the touch position detection method is adopted. FIG. 12 is a graph showing a detection result when a finger pad image is detected by generating a difference image without smoothing the extinguishing image obtained by capturing a finger in a stationary state and extracting an edge feature. FIG. 13 is a graph showing a detection result when a finger pad image is detected by generating a difference image without smoothing an extinguishing image obtained by capturing a moving finger and extracting an edge feature. FIG. 14 is a graph showing a detection result when a finger pad image is detected by smoothing an extinguished image obtained by capturing a finger in a stationary state, generating a difference image, and extracting an edge feature. FIG. 15 is a graph showing a detection result when a finger pad image is detected by smoothing an extinguished image obtained by capturing a moving finger and generating a difference image and extracting an edge feature.

  The imaging conditions and smoothing conditions that resulted in the results shown in FIGS. 12 to 15 are the same as the imaging conditions and smoothing conditions that resulted in the results shown in FIGS.

  In the results shown in FIGS. 12 to 15, the feature quantity indicating the outer edge portion of the finger pad image is extracted. Therefore, when extracting a row of pixels across the finger pad, two peaks are detected. The vertical axis of the graphs shown in FIGS. 12 to 15 represents edge strength (a value indicating the gradient between the pixel value of the target pixel and the pixel value of a pixel adjacent to the target pixel).

  As shown in FIGS. 12 and 14, when the finger is stationary, touch / non-touch identification is performed without any problem regardless of the external light intensity. However, as shown in FIG. 13, when the finger moves, in the case of medium illuminance and high illuminance, pixels that are actually non-touched but have edge strength exceeding the threshold (edge pixels) are generated. . In this case, an image of a finger not in contact with the optical sensor built-in LCD 11 is recognized as an image of a finger in contact.

  On the other hand, as shown in FIG. 15, if the extinguished image is smoothed, the number of edge pixels generated when the finger moves is suppressed as compared with the case where the finger is not smoothed. In the result shown in FIG. 15, the edge pixel in the case of non-touch is not set to 0, but this does not deny the effect of the present invention. Even if edge pixels are detected, if the number of edge pixels is small, the degree of coincidence with the model pattern is low, so that it is not recognized as a contact partial image. Therefore, touch / non-touch discrimination accuracy can be increased by reducing the number of edge pixels in the case of non-touch.

  As can be seen from the results of FIGS. 8 and 13, when the external light intensity is small, there is a low possibility that differential noise exceeding the threshold value is generated even if the finger moves. Therefore, the process of smoothing the extinguished image may be performed after the external light intensity becomes equal to or higher than the medium illuminance.

  Further, since the occurrence of differential noise becomes conspicuous as the external light intensity increases, it is preferable to increase the size of the smoothing filter to be used as the external light intensity increases. This is because the smoothing effect increases as the size of the smoothing filter increases. However, as will be described later, the size of the smoothing filter is preferably smaller than the size of the contact partial image.

(Reason for making the smoothing filter smaller than the contact partial image)
FIG. 16 is a diagram illustrating an example in which the extinguished image is smoothed using a smoothing filter having the same size as the size of the contact partial image. In the figure, a smoothing filter is applied to the difference image generated without smoothing the unlit image including the image of the touch pointing object (contact partial image), and the unlit image including the contact partial image. Difference image generated after smoothing using, non-touch pointed object image included non-touch pointed difference image generated without smoothing, non-touch pointed object image included Difference images generated after the image is smoothed are shown in order.

  The size of the image of the pointing object (contact partial image) is 9 × 9, and the size of the smoothing filter used is 9 × 9. Moreover, the result of the same figure is a result at the time of image | photographing the moved finger | toe in a high illumination intensity external light environment. Below each difference image, the result of performing the touch position detection process on the difference image is shown.

  Smoothing the extinguished image has the effect of reducing differential noise, but has the side effect of reducing the pixel value of the contact partial image. Therefore, it is preferable to set the degree of smoothing the unlit image (that is, the size of the smoothing filter) so that the touch / non-touch determination accuracy is improved.

  As shown in FIG. 16, when the touch-off image is smoothed using a smoothing filter having the same size as the size of the contact partial image, the number of edge pixels and high-luminance pixels is almost zero. It becomes difficult to detect the contact partial image.

  On the other hand, when the unlit image containing the image of the non-touch pointing object is smoothed with a smoothing filter of the same size, the number of both edge pixels and high luminance pixels is significantly reduced. The number is not zero. Therefore, in this example, a touch image is not detected, and a non-touch image is detected as a touch image.

  As described above, when the unlit image is smoothed using the smoothing filter having the same size as the size of the contact partial image, the touch / non-touch determination accuracy may be lowered. Therefore, it is preferable to smooth the extinguished image using a smoothing filter having a size smaller than the size of the contact partial image.

  Therefore, when a plurality of pointing objects to be detected are assumed, it is preferable to prepare a smoothing filter having a size corresponding to each pointing object. FIG. 17 is a diagram for explaining the significance of using a smoothing filter having a size corresponding to the pointing object. The figure shows a difference image generated without smoothing the extinguishment image including the contact part image of the finger pad and the contact part image of the pen tip, and the extinguishing image containing the contact part image. The difference image generated after smoothing, the difference image generated without smoothing the unlit image containing the image of the non-touch finger pad and the image of the non-touch pen tip, the non-touch image The difference images generated after smoothing the extinguished image including the image are shown in order.

  The size of the finger pad image is 9 × 9, the size of the pen tip image is 3 × 3, and the size of the smoothing filter used is 3 × 3. Moreover, the result of the same figure is a result at the time of image | photographing the moved finger | toe in a high illumination intensity external light environment. Below each difference image, the result of performing the touch position detection process on the difference image is shown.

  As shown in the figure, if the pen tip image is smoothed with a smoothing filter of the same size as the image, there is a possibility that the touch image may not be detected, and the non-touch image is detected as the touch image. The possibility of being increased. Therefore, when detecting images of a plurality of pointing objects, it is preferable to use a smoothing filter having a size suitable for each pointing object.

  However, the above description is a case where a smoothing filter having a substantially constant weight in the smoothing filter, such as an average value filter and a median filter, is used. When a smoothing filter having an extremely large weight at the center portion is used, the degree of smoothing is low, and therefore the possibility of the above side effects is low even if the size is larger than the size of the contact portion image. In this case, the effect of reducing the differential noise is also reduced. Therefore, the type and size of the smoothing filter may be set in consideration of the balance between effects and side effects.

(Example of smoothing filter)
FIG. 18A is a diagram for explaining the effect of the average value filter, and FIG. 18B is a diagram for explaining the effect of the intermediate value filter. The smoothing filter (spatial filter) used by the first and second smoothed image generation units 4a and 4b is, for example, an average value filter or an intermediate value (median value) filter. As shown in FIG. 18A, the average value filter calculates the average value of the pixel values of the pixels in the filter, and replaces the pixel value of each pixel in the filter with the calculated average value. As shown in FIG. 18B, the intermediate value filter calculates the intermediate value of the pixel values of the pixels in the filter, and replaces the pixel value of each pixel in the filter with the calculated intermediate value.

  The smoothing filters used by the first and second smoothed image generation units 4a and 4b may be other smoothing filters and are not limited to the above.

(Reason for clipping pixel values less than 0 in the difference image)
FIG. 19 is a diagram for explaining the reason for clipping a pixel value smaller than 0 in the difference image. In the figure, in the result (“edge”) when using the first example of the touch position detection method and the result (“pixel value”) when using the second example of the touch position detection method, The result when the clipping is performed and the result when the clipping is not performed are shown. Each graph is viewed in the same way as the graphs in FIGS. In the example of FIG. 19, the finger pad in a moving state is imaged under a high illuminance outside light environment, and the unlit image is not smoothed.

  The effect of clipping a pixel value smaller than 0 in the difference image is obtained when the first example of the touch position detection method is used. In the first example, the gradient of the pixel value is used as a feature indicating an edge. Therefore, when a pixel value smaller than 0 exists, a gradient of the pixel value is formed by the pixel. Since a pixel having a negative pixel value generated by subtracting the unlit image from the lit image does not form a contact partial image, the pixel having a negative pixel value becomes differential noise in the first example.

  In the result of non-touch “edge” of “no clipping”, pixels with high edge strength are detected. However, “0 clipping” indicates that pixels with high edge strength are reduced by 0 clipping. It can be seen from the result of non-touch “edge”.

(Processing flow in the touch position detection apparatus 1)
FIG. 20 is a flowchart illustrating an example of a process flow in the touch position detection apparatus 1. First, the optical sensor 12 of the optical sensor built-in LCD 11 indicates the pointing object at the first imaging timing when the touch panel backlight 14 is lit among temporally continuous imaging timings under the control of the timing control unit 16. And the captured image (that is, the lighting image) is output to the image adjustment unit 2 (S1).

  At the second imaging timing which is the imaging timing next to the first imaging timing, the touch panel backlight 14 is turned off. In this state, the optical sensor 12 images the pointing object and outputs the captured image (that is, the extinguished image) to the image adjustment unit 2 (S2).

  The image adjustment unit 2 adjusts the received lit image and unlit image, and stores the adjusted captured images in the primary storage unit 31 in association with each other (S3).

  Next, the external light intensity calculation unit 3 calculates the external light intensity from the lighting image or the extinguishing image stored in the primary storage unit 31 as described above, and first sets the external light intensity information indicating the calculated external light intensity. It outputs to the smoothed image generation part 4a and the 2nd smoothed image generation part 4b (S4). When acquiring ambient light intensity information from a sensor that measures ambient light intensity, ambient light intensity information at this time may be acquired, or ambient light intensity information at the time of capturing a captured image may be acquired. .

  The first and second smoothed image generation units 4a and 4b compare the received external light intensity information with a predetermined plurality of threshold levels, and determine a smoothing filter according to the external light intensity (S5a and 5b). In other words, the first and second smoothed image generation units 4a and 4b select an algorithm for smoothing the extinguished image according to the external light intensity.

  At this time, the first smoothed image generating unit 4a determines a smoothing filter having a size suitable for the case where the pointing object is a finger, and the second smoothed image generating unit 4b is suitable for the case where the pointing object is a pen. A smoothing filter of a certain size is determined.

  The first and second smoothed image generating units 4a and 4b smooth the extinguished image stored in the primary storage unit 31 using the determined smoothing filter. The first smoothed image generation unit 4a outputs the generated smoothed image to the first difference image generation unit 5a, and the second smoothed image generation unit 4b outputs the generated smoothed image to the second difference image generation unit 5b. (S6a, 6b).

  The first and second difference image generation units 5a and 5b obtain a lighting image corresponding to the received smoothed image from the primary storage unit 31, and obtain a difference between the lighting image and the received smoothed image. A difference image is generated (S7a, 7b). At this time, the first and second difference image generation units 5a and 5b set the pixel value of a pixel having a pixel value smaller than 0 in the generated difference image to 0. The first difference image generation unit 5a outputs the generated difference image to the first touch position detection unit 6a, and the second difference image generation unit 5b outputs the generated difference image to the second touch position detection unit 6b.

  The first touch position detection unit 6a detects a finger contact partial image included in the received difference image, performs a process for calculating the position (S8a), and uses the coordinates indicating the calculated position as an application execution unit. 7 (S9a).

  In addition, the second touch position detection unit 6b detects a contact partial image of the pen tip included in the received difference image, performs a process for calculating the position (S8b), and calculates coordinates indicating the calculated position. It outputs to the application execution part 7 (S9b).

  The application execution unit 7 performs processing corresponding to the coordinates based on the coordinates received from the first touch position detection unit 6a or the second touch position detection unit 6b (S10).

(Effect of the touch position detection device 1)
As described above, the touch position detection apparatus 1 can reduce the differential noise caused by the movement of the pointing object by smoothing the extinguished image, and can accurately detect the pointing object in contact with the LCD 11 with the built-in optical sensor. In other words, touch / non-touch determination can be performed with high accuracy.

[Embodiment 2]
The following will describe another embodiment of the present invention with reference to FIGS. In addition, about the member similar to Embodiment 1, the same code | symbol is attached | subjected and the description is abbreviate | omitted. FIG. 21 is a schematic diagram illustrating a configuration of the touch position detection device 40 according to the present embodiment. As shown in the figure, the main control unit 21 of the touch position detection device 40 is different from the touch position detection device 1 having the external light intensity calculation unit 3 in that a differential noise determination unit (differential image determination unit). 8 is provided.

  The difference noise determination unit 8 generates a difference image (referred to as a determination difference image) from the lit image and the unlit image output from the image adjustment unit 2, and in the generated determination difference image (second difference image), It is determined whether or not the number of pixels (referred to as high luminance pixels) having a pixel value larger than the pixel value of the image of the pointing object assumed in advance is equal to or greater than a predetermined number. The high luminance pixel is highly likely to be differential noise generated by taking the difference between the lit image and the unlit image. Therefore, when a predetermined number or more of high-luminance pixels are present in the determination difference image, it is determined that differential noise has occurred.

  The first smoothed image generation unit 4a and the second smoothed image generation unit 4b smooth the unlit image only when the difference noise determination unit 8 determines that the number of pixels is equal to or greater than a predetermined number. With this configuration, it is possible to reduce the possibility of performing a useless process of smoothing the extinguished image when no differential noise occurs.

  The first smoothed image generation unit 4a and the second smoothed image generation unit 4b may change the size of the smoothing filter to be used according to the number of high luminance pixels. More specifically, the size of the smoothing filter to be used may be increased stepwise as the number of high luminance pixels in the difference image increases.

  Further, the external light intensity calculation unit 3 is also provided in the touch position detection device 40, and the external light intensity calculation unit 3 calculates when the difference noise determination unit 8 determines that the number of pixels is equal to or greater than a predetermined number. The size of the smoothing filter to be used may be selected according to the outside light intensity.

(Processing flow in the touch position detection device 40)
Next, an example of a processing flow in the touch position detection device 40 will be described. FIG. 22 is a flowchart illustrating an example of a process flow in the touch position detection device 40. Since steps S11 to S12 are the same as steps S1 to S3 described above, description thereof is omitted.

  In step S13, the image adjustment unit 2 adjusts the lit image and the unlit image, and transmits the adjusted lit image and unlit image to the differential noise determination unit 8 and stores them in the primary storage unit 31 (S13).

  When the difference noise determination unit 8 receives the lit image and the unlit image, the difference noise determination unit 8 generates a determination difference image by taking the difference between these images, and stores it in the primary storage unit 31 (S14).

  And the difference noise determination part 8 determines whether the number of high-intensity pixels is more than predetermined number in the produced | generated difference image for a determination (S15).

  If it is determined that the number of high-luminance pixels is equal to or greater than the predetermined number (YES in S15), difference noise determination unit 8 performs first smoothing on a smoothed image generation command that instructs to generate a smoothed image. It outputs to the image generation part 4a and the 2nd smoothed image generation part 4b.

  When the smoothed image generation command is received, the first smoothed image generating unit 4a and the second smoothed image generating unit 4b each acquire the extinguished image from the primary storage unit 31, and smoothing from the extinguished image as described above An image is generated (S16).

  Subsequent steps S17 to S20 are the same as steps S7a · b to S10 described above.

  On the other hand, if it is determined that the number of high-luminance pixels is smaller than the predetermined number (NO in S15), difference noise determination unit 8 issues a touch position detection command that instructs to detect the touch position to the first touch position. It outputs to the detection part 6a and the 2nd touch position detection part 6b.

  When the touch position detection command is received, the first touch position detection unit 6a and the second touch position detection unit 6b each obtain a determination difference image from the primary storage unit 31, and the position of the contact partial image in the determination difference image Is detected (S21).

  Of the first touch position detection unit 6a and the second touch position detection unit 6b, the touch position detection unit that detects the contact partial image executes the application of coordinates indicating the position of the detected contact partial image in the determination difference image. It outputs to the part 7 (S22).

(Effect of the touch position detection device 40)
As described above, the touch position detection device 40 generates a difference image from the lit image and the unlit image before performing the smoothing process, and the pixel value of the image of the pointing object assumed in advance in the difference image. It is determined whether the number of pixels having a larger pixel value is equal to or greater than a predetermined number. Therefore, the smoothing process of the unlit image can be omitted in a situation where the smoothing of the unlit image is not necessarily required and the generation level of the differential noise is low.

(Example of change)
The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  For example, the touch panel unit 10 may output all of the lit image and the unlit image captured by the optical sensor 12 to the image adjustment unit 2, and determines whether or not a change has occurred in the lit image or unlit image. When a change larger than the threshold value occurs, the lit image and the unlit image may be output to the image adjustment unit 2.

  Further, an optical sensor (visible light sensor) that receives visible light (wavelength: about 380 nm to 750 nm) is provided as the optical sensor 12, and the display backlight 13 that emits visible light is turned on / off, and the visible light sensor is used. You may image a lighting image and a light extinction image as mentioned above.

  In addition, each block of the touch position detection devices 1 and 40 described above, in particular, the main control units 20 and 21 may be configured by hardware logic, or may be realized by software using a CPU as follows. .

  That is, the touch position detection devices 1 and 40 include a central processing unit (CPU) that executes instructions of a control program that realizes each function, a read only memory (ROM) that stores the program, and a RAM (random) that expands the program. access memory), a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is to provide a computer with program codes (execution format program, intermediate code program, source program) of a control program (position detection program) for the touch position detection devices 1 and 40 that are software for realizing the functions described above. This can also be achieved by supplying a recording medium recorded in a readable manner to the touch position detecting devices 1 and 40, and the computer (or CPU or MPU) reading and executing the program code recorded on the recording medium. is there.

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the touch position detection devices 1 and 40 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR (high data rate), mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention can also be expressed as follows.

  That is, the display device of the present invention is a display device that performs image display and light reception simultaneously or alternately, and alternately repeats light emission and extinction of light emitting elements arranged on the back surface of the display surface in time. An image received when the light emitting element arranged in the light source is not emitted is acquired, and an image received when the light emitting element is not emitted is smoothed from an image received when the light is emitted, and smoother than an image received when the light is emitted. And a detection unit that generates a difference image by subtracting the smoothed image from an image received when the light is emitted, and detects a state and a position in contact with or close to a display surface from the difference image. It is a feature.

  By reducing the influence of differential noise, a stable pointer operation and touch operation can be performed without being affected by the external light environment, and therefore, it can be applied to a touch panel, an input device, and the like. In particular, the present invention can be suitably applied to display devices with an input function such as a liquid crystal with a touch pad function, and information devices such as a mobile phone, a personal computer, and a PDA equipped with the display device.

DESCRIPTION OF SYMBOLS 1 Touch position detection apparatus 2 Image adjustment part (acquisition part)
4a 1st smoothed image generation part 4b 2nd smoothed image generation part 5a 1st difference image generation part 5b 2nd difference image generation part 6a 1st touch position detection part 6b 2nd touch position detection part 8 Difference noise determination part ( Difference image determination unit)
10 Touch panel (imaging panel)
12 Optical sensor (image sensor)
13 Display backlight 14 Touch panel backlight (illumination unit)
40 Touch position detection device

Claims (20)

A position detection device for detecting a position of an image to be imaged in a captured image,
A lighting image that is an image obtained by imaging the imaging object with the imaging panel in a state where light irradiation is performed on the imaging object from a side where the imaging panel is located, and the imaging object that is captured without the light irradiation. An image captured by a panel, and an acquisition unit that acquires a light-off image captured with a predetermined time difference from the lighting image;
A smoothed image generating unit that generates a smoothed image obtained by smoothing the extinguishing image acquired by the acquiring unit from the lighting image;
A difference image generation unit that generates a difference image indicating a difference between the lighting image and the smoothed image generated by the smoothed image generation unit;
A position detection device comprising: a position detection unit that detects a position of an image to be imaged that is in contact with the imaging panel and is included in the difference image generated by the difference image generation unit.   The position detection device according to claim 1, wherein the difference image generation unit replaces a pixel value of a pixel having a pixel value smaller than 0 with 0 in the generated difference image.   The position detection device according to claim 1, wherein the smoothed image generation unit smoothes the unlit image using a spatial filter as a smoothing filter.   The smoothed image generation unit smoothes the extinguished image using a smoothing filter that is smaller than a size of an image of a contact portion with the imaging panel of the imaging target that is assumed in advance. The position detection apparatus of any one of Claims 1-3.   5. The smoothed image generation unit generates the smoothed image using a plurality of stages of smoothing filters corresponding to a plurality of types of imaging targets assumed in advance. The position detection device according to any one of the above. The smoothed image generation unit generates a plurality of smoothed images from one extinguished image using smoothing filters of a plurality of stages corresponding to a plurality of types of imaging targets assumed in advance,
The difference image generation unit generates a plurality of difference images each indicating a difference between the lighting image and the plurality of smoothed images generated by the smoothed image generation unit,
For each of the plurality of difference images generated by the difference image generation unit, the position detection unit corresponds to the size of the smoothing filter used to generate the smoothed image that is the basis of the difference image. The position detection apparatus according to claim 5, wherein the position of the image of the imaging target included in each difference image is detected using a pattern for detecting the target image.   The smoothed image generation unit acquires external light intensity information indicating external light intensity that is the intensity of light around the imaging target at the time when the lighting image or the extinguishing image is captured, and the external light intensity information 6. The position detection device according to claim 1, wherein the extinguishing image is smoothed only when the external light intensity indicated by exceeds a predetermined reference value.   The smoothed image generation unit acquires external light intensity information indicating external light intensity that is the intensity of light around the imaging target at the time when the lighting image or the extinguishing image is captured, and the external light intensity information The position detection device according to claim 1, wherein the extinguishing image is smoothed by using a smoothing filter having a size corresponding to the intensity of external light indicated by. A second difference image indicating a difference between the lit image and the unlit image acquired by the acquisition unit is generated. In the generated second difference image, a pixel value larger than a pixel value of an image of the imaging target that is assumed in advance is set. The image processing apparatus further includes a difference image determination unit that determines whether or not the number of pixels having the number is equal to or greater than a predetermined number,
The smoothed image generation unit smoothes the unlit image only when the difference image determination unit determines that the number of pixels is equal to or greater than a predetermined number. The position detection apparatus of any one of Claims.   The position detection unit extracts a pixel having a pixel value equal to or greater than a predetermined value from the difference image generated by the difference image generation unit, and captures the position of an image formed by the extracted pixel in contact with the imaging panel. The position detection apparatus according to claim 1, wherein the position detection apparatus detects the position of the target image.   The position detection unit binarizes the difference image generated by the difference image generation unit with a predetermined value as a boundary, and the position of the image included in the binarized difference image of the imaging target in contact with the imaging panel The position detection apparatus according to claim 1, wherein the position detection apparatus detects the position as an image position.   The position detection unit extracts pixels having an edge intensity equal to or greater than a predetermined value from the difference image generated by the difference image generation unit, and detects an image of the imaging target with respect to the set of extracted pixels. The position detection apparatus according to claim 1, wherein an image of an imaging target in contact with the imaging panel is detected by performing pattern matching with a pattern. The position detection device according to any one of claims 1 to 12,
An imaging device comprising an imaging panel,
The imaging panel
An illumination unit that performs light irradiation on the imaging target at a predetermined time interval;
In synchronization with the light irradiation timing of the illuminating unit, an imaging sensor that captures the lighting image and the extinguishing image,
The said acquisition part acquires the said lighting image and the said light extinction image which the said imaging sensor imaged, The imaging device characterized by the above-mentioned. The imaging panel includes a display backlight,
The imaging apparatus according to claim 13, wherein the illumination unit is provided separately from the display backlight.   The imaging device according to claim 13, wherein the illumination unit emits light having a wavelength in a near infrared region or a near ultraviolet region.   The imaging device according to claim 15, wherein the imaging sensor receives light having a wavelength in a near infrared region or a near ultraviolet region.   The imaging device according to any one of claims 13 to 16, wherein the imaging sensor is set to a sensitivity that does not saturate under sunlight irradiation.   A position detection program for operating the position detection device according to any one of claims 1 to 12, wherein the position detection program causes a computer to function as each unit.   The computer-readable recording medium which recorded the position detection program of Claim 18. A position detection method in a position detection device for detecting the position of an image to be imaged in a captured image,
A lighting image that is an image obtained by imaging the imaging object with the imaging panel in a state where light irradiation is performed on the imaging object from a side where the imaging panel is located, and the imaging object that is captured without the light irradiation. An acquisition step of acquiring an image captured by a panel and acquiring an unlit image captured with a predetermined time difference from the lighting image;
A smoothed image generating step for generating a smoothed image obtained by smoothing the unlit image acquired in the acquiring step from the lighted image;
A difference image generation step for generating a difference image indicating a difference between the lighting image and the smoothed image generated in the smoothed image generation step;
A position detection method comprising: a position detection step of detecting a position of an image to be imaged in contact with the imaging panel, which is included in the difference image generated in the difference image generation step.