US20120327035A1

US20120327035A1 - Optical touch system and image processing method thereof

Info

Publication number: US20120327035A1
Application number: US13/495,712
Authority: US
Inventors: Tzung Min Su; Ren-Hau Gu; Chih Hsin LIN; Cheng Nan Tsai
Original assignee: Pixart Imaging Inc
Current assignee: Pixart Imaging Inc
Priority date: 2011-06-21
Filing date: 2012-06-13
Publication date: 2012-12-27
Also published as: TW201301102A; US20170052647A1; US10282036B2; TWI441062B

Abstract

An optical touch system includes an image sensor module and a processor. The image sensor module includes a plurality of image sensing elements that can be independently controlled to achieve different exposure times. The plurality of image sensor elements can produce a picture including a plurality of pixel groups. The processor is configured to extract an intensity value of each pixel group and to select a set of successive pixel groups as an object image according to the intensity values of the pixel groups.

Description

CROSS-REFERENCE TO RELATED APPLICATION

The present application is based on, and claims priority from, Taiwan Patent Application Serial Number 100121547, filed on Jun. 21, 2011, the disclosure of which is hereby incorporated by reference herein in its entirety.

BACKGROUND

1. Technical Field
The present invention relates to an optical touch system and an image processing method thereof.
2. Related Art
An optical touch system generally comprises an imaging device, an illuminating device, and a computing device for determining the touch position of an object. Typically, the object may be a finger, a stylus, or the like. The imaging device is configured to view a touch zone above a touch surface. The illuminating device is configured such that when an object is in the touch zone, the illuminating device can make the object generate an identifiable contrast image on a picture produced by the imaging device. The computing device is configured to calculate the coordinates of the object according to the brightness variation of the picture produced by the imaging device.
The optical touch system can be designed as a system in which the object entering into the touch zone blocks the light projected from the illuminating device so as to form a dark area on the imaging device, or as a system in which the object reflects the light projected from the illuminating device to form a bright area. Regardless of which system is employed, the current coordinate calculating method requires two pictures, a background picture and a picture taken when there is an object in the touch zone. Normally, the background picture is generated and stored before operation. The optical touch system can identify the region having obviously different intensity by comparing the captured picture and the background picture. When the region having obviously different intensity is used as an image formed by an object, the optical system can calculate the coordinates of the object according to some features of the region having obviously different intensity.
In addition to the region having obviously different intensity, the intensity levels of a portion of the background area of the picture may be changed due to the change of the light path or the manner of light reflection of the touch surface caused by the object such that a difference may occur between a background area of the picture and the corresponding portion of the background picture. Such difference may make it impossible for the region having obviously different intensity to be correctly calculated or properly identified. As a result, the coordinates of the object cannot be accurately calculated.

SUMMARY

One embodiment provides an image processing method and an optical touch system using the same. The image processing method can use a single picture to determine the coordinates of an object such that incorrect coordinates will not be obtained when there is a difference between a background area of a picture and the corresponding portion of a background picture.
In one embodiment, an optical touch system comprises an image sensor module and a processor. The image sensor module comprises a plurality of image sensing elements. The image sensing elements are configured to be independently controlled to achieve different exposure times. The plurality of image sensing elements may generate a picture comprising a plurality of pixel groups. The processor is configured to extract an intensity value representing each pixel group. The processor is also configured to select a portion of the pixel groups as an object image according to the intensity values of the pixel groups.
In one embodiment, an image processing method of an optical touch system comprises obtaining a picture comprising a plurality of first pixel groups, determining a plurality of first difference values each determined by subtracting intensity values of two of the plurality of first pixel groups, and selecting a set of successive pixel groups as an object image according to the first difference values.
To better understand the above-described objectives, characteristics and advantages of the present invention, embodiments, with reference to the drawings, are provided for detailed explanations.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described according to the appended drawings in which:

FIG. 1 is a schematic view showing an optical touch system according to one embodiment of the present invention;

FIG. 2 is an illustration of a picture according to one embodiment of the present invention;

FIG. 3 schematically depicts an intensity fluctuating pattern according to one embodiment of the present invention;

FIG. 4 schematically depicts a portion of sensing elements and the circuit of an image sensor module according to one embodiment of the present invention;

FIG. 5 is a flow chart related to an image processing method according to one embodiment of the present invention;

FIG. 6 schematically depicts an intensity fluctuating pattern according to another embodiment of the present invention;

FIG. 7 schematically depicts a difference fluctuating pattern according to one embodiment of the present invention;

FIG. 8 is a flow chart related to an image processing method according to another embodiment of the present invention;

FIG. 9 schematically depicts an intensity fluctuating pattern according to another embodiment of the present invention;

FIG. 10 schematically depicts a difference fluctuating pattern according to another embodiment of the present invention;

FIG. 11 is a flow chart related to an image processing method according to another embodiment of the present invention;

FIG. 12 schematically depicts an intensity fluctuating pattern according to another embodiment of the present invention;

FIG. 13 schematically depicts a difference fluctuating pattern according to another embodiment of the present invention;

FIG. 14 is a flow chart related to an image processing method according to another embodiment of the present invention;

FIG. 15 schematically depicts an intensity fluctuating pattern according to another embodiment of the present invention;

FIG. 16 schematically depicts a difference fluctuating pattern according to another embodiment of the present invention;

FIG. 17 schematically depicts an intensity fluctuating pattern according to another embodiment of the present invention; and

FIG. 18 schematically depicts a difference fluctuating pattern according to another embodiment of the present invention.

DETAILED DESCRIPTION OF DISCLOSED EMBODIMENTS

The following description is presented to enable any person skilled in the art to make and use the disclosed embodiments, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the disclosed embodiments. Thus, the disclosed embodiments are not limited to the embodiments shown, but are to be accorded the widest scope consistent with the principles and features disclosed herein.
FIG. 1 is a schematic view showing an optical touch system 1 according to one embodiment of the present invention. One embodiment of the present invention is related to an image processing method that can calculate the coordinate data using a single picture. The image processing method is applicable to the optical touch system 1 shown in FIG. 1. Referring to FIG. 1, the optical touch system 1 comprises an image sensor module 11 and a processor 12. The processor 12 is coupled with the image sensor module 11 to analyze an object image from the picture generated by the image sensor module 11. The image sensor module 11 is configured to monitor a touch area 13. The illuminating devices 14 and 15 are disposed adjacent to the touch area 13 to provide illumination such that the object on the touch area 13 can generate an identifiable image on a picture. The illuminating device 14 or 15 can be an active light source including, for example, a light tube, a plurality of light emitting diodes, or a combination of a light emitting diode and a light guide member. The illuminating device 14 or 15 may also be a passive light source such as a mirror.
The image processing method of one embodiment of the present invention can be applied using an illumination-compensated picture whose image intensity variation is compensated for and with a normally captured picture whose image intensity variation is not compensated for. The description below begins by describing application of the image processing method of one embodiment of the present invention to an illumination-compensated picture.
The method of compensating for image intensity variation can be implemented as software or hardware means and is applicable to a picture to increase the intensity of the portion that originally has lower intensity and to reduce the intensity of the portion that originally has higher intensity, so as to obtain a new picture with uniform intensity. For example, as shown in FIG. 2, the image sensor module 11 may produce a picture 2. The picture 2 may comprise a plurality of pixel groups 21 that may be arranged along a direction. Each pixel group 21 may comprise a plurality of pixels 211. In one embodiment, each pixel group 21 may be a pixel column, and the plurality of pixel groups 21 are arranged along a row direction. In another embodiment, the pixel group 21 may be a pixel row, and the plurality of pixel groups 21 are arranged along a column direction. The processor 12 is configured to extract an intensity value representing each pixel group 21 from the captured picture 2. In one embodiment, the intensity value representing a pixel group 21 can be a sum of the intensity values of the pixels 211 of the pixel group 21. In another embodiment, the intensity value representing a pixel group 21 can be an average of the intensity values of the pixels 211 of the pixel group 21. After the calculation of the intensity values of all pixel groups 21 is completed, an intensity fluctuating pattern 3 as shown in FIG. 3 can be obtained.
As illustrated in FIG. 3, the intensity fluctuating pattern 3 exhibits significant variation. The intensity fluctuating pattern 3 may be compensated for in advance. In one embodiment, an adjustment value for each pixel group 21 is determined. Next, each adjustment value is multiplied by the intensity value of the corresponding pixel group 21 to obtain a new intensity fluctuating pattern 4 exhibiting less variation. In one embodiment, the adjusted intensity fluctuating pattern 4 can vary within an intensity range 5.
The adjustment value is used to compensate for the variation of an intensity fluctuating pattern. The adjustment value can be determined through many methods, one of which is illustratively demonstrated herein. In one embodiment, an adjustment value can be obtained by the following steps: A background picture is generated using a fixed exposure time. Next, an intensity value (I_n) of each pixel group is determined. Finally, a ratio of a target intensity value (I_T) to the intensity value (I_p) for each pixel group is calculated, wherein the ratio (I_p/I_T) can be used as an adjustment value.
The compensation for the intensity variation of the picture can be achieved through hardware. As illustrated in FIG. 4, the image sensor module 11 comprises a plurality of image sensing elements 41 a and 41 b each comprising an electronic shutter 411 and a photo detector 412. The electronic shutter 411 is coupled with the photo detector 412 to control the exposure time of the photo detector 412. The photo detector 412 generates charge in response to received light. The transistor 413 controls the transferring of the charge from the photo detector 412 to a floating diffusion (FD) output node. The transistor 414 and the constant current source 415 form as a source follower, which can amplify the photovoltaic voltage produced by the photo detector 412. The transistor 416 can be activated when the signal WL1 is at a high level, and at this moment, data can be output to the bit line 417 that is coupled to a readout circuit 220. When signals RST1 and TG1 go high, the transistors 418 and 413 are activated such that the voltage supply VDDAY can reset the photo detector 412 to a photo-electric conversion initiation state. When the transistor 413 is turned on, the charge flows from the photo-detector 412 to the FD output node. When the signal RST1 goes high, the voltage source VDDAY resets the FD output node.
Referring to FIG. 4, the electronic shutters 411 of the image sensing elements 41 a and 41 b are respectively coupled to different shutter control lines 419 a and 419 b. As such, different signals AB1 and AB2 can be applied to the electronic shutters 411 of the image sensing elements 41 a and 41 b to operate the image sensing elements 41 a and 41 b for different exposure times such that the intensity values of the corresponding pixels of a picture can be independently manipulated. During operation, the image sensing element 41 a or 41 b, which is exposed to stronger light intensity, is assigned shorter exposure time and the image sensing element 41 a or 41 b which is exposed to weaker light intensity is assigned longer exposure time. As a result, a captured picture can exhibit a more uniform background intensity level.
The exposure time for controlling the electronic shutter 411 of each image sensing element 41 a or 41 b can be obtained using the following method; however, the present invention is not limited to such method. The method obtains a background picture by a fixed exposure time. Next, the intensity value of each pixel of the background picture is extracted. Thereafter, the exposure time that is needed for operating the corresponding electronic shutter 411 of an image sensing element 41 a or 41 b and can make the corresponding pixel achieve a target intensity level is computed using the target intensity level and the intensity value of the pixel.
FIG. 5 is a flow chart related to an image processing method according to one embodiment of the present invention. Referring to FIG. 5, at Step S51, a picture is captured, wherein the picture comprises a plurality of pixel groups that can be arranged along a direction. At Step S52, the intensity value I(p_i) of each pixel group of the picture is computed to obtain an intensity fluctuating pattern I(p) as shown in FIG. 6, where p_irepresents the i-th pixel group. The intensity value of each pixel group can be the sum of the intensities of pixels or the average of the intensities of pixels of the pixel group. At Step S53, a target intensity value (T1) is decided. In one embodiment, the target intensity value (T1) can be the average of the intensity fluctuating pattern I(p) or the intensity values I(p_i). In another embodiment, the target intensity value (T1) can be a predetermined value. At Step S54, a difference fluctuating pattern D1(p) formed by a plurality of difference values D1(p _i) each calculated, as shown in equation (1), by subtracting the target intensity value (T1) from the corresponding intensity value I(p_i) on the intensity fluctuating pattern I(p) as shown in FIG. 7 is obtained.
D1(p _i)=I(p _i)−T1 (1)
At Step S55, a threshold Th1 is used to determine a section R1 of the intensity fluctuating pattern I(p) that includes pixel groups having ratios of difference values D1(p _i) to the target intensity value T1 less than or greater than the threshold Th1 as an object image.
FIG. 8 is a flow chart related to an image processing method according to another embodiment of the present invention. Referring to FIG. 8, as Step S81, a picture is obtained. The picture comprises a plurality of pixel groups that may be arranged along a direction. At Step S82, the intensity value I(p_i) representing each pixel group (p_i) of the picture is extracted to obtain an intensity fluctuating pattern I(p) as shown in FIG. 9, where p_irepresents the i-th pixel group. The intensity value representing each pixel group can be either the sum of the intensity values of pixels of the pixel group or an average of the intensity values of pixels of the pixel group. At Step S83, a difference fluctuating pattern ICD(p) as shown in FIG. 10 is determined. The difference fluctuating pattern ICD(p) has a plurality of difference values ICD(p_i) sequentially calculated along the arrangement direction of the pixel groups and each ICD(p_i) determined by subtracting the intensity values of two different pixel groups (p_iand p_i+1). At Step S84, on the difference fluctuating pattern ICD(p), the point P_t1whose difference value is less than a threshold Th2 and the point P_t2whose difference value is greater than a threshold Th3 are determined. One of the pixel groups corresponding to the point P_t1, is used as a left boundary pixel group, and one of the pixel groups corresponding to the point P_t2is used as a right boundary pixel group. Accordingly, the section of the intensity fluctuating pattern constituted by a set of successive pixel groups between the left and right boundary pixel groups is obtained, and the section can then be selected as an object image.
FIG. 11 is a flow chart related to an image processing method according to another embodiment of the present invention. Referring to FIG. 11, at Step S111, a picture comprising a plurality of pixel groups arranged along a direction is generated. At Step S112, the intensity value I(p_i) representing each pixel group (p_i) of the picture is extracted to obtain an intensity fluctuating pattern I(p) as shown in FIG. 12, where p_irepresents the i-th pixel group. The intensity value representing each pixel group can be either the sum of the intensity values of pixels of the pixel group or the average of the intensity values of pixels of the pixel group. At Step S113, sequential calculation of the difference value ICD(p_i) between a pixel group p_iand a pixel group p_ithat are spaced at an interval of a predetermined number q is performed one after another along the arrangement of the pixel group p_i, to the pixel groups of the picture to obtain a difference fluctuating pattern ICD(p) as shown in FIG. 13. At Step S114, on the fluctuating pattern ICD(p), the point P_owhose difference value is less than a threshold Th4 and the point P_t4whose difference value is greater than a threshold Th5 are determined. One of the pixel groups corresponding to the point P_ois used as a left boundary pixel group, and one of the pixel groups corresponding to the point P_t4is used as a right boundary pixel group. Accordingly, the section constituted by a set of successive pixel groups between the left and right boundary pixel group is obtained, and the section can be selected as an object image 121 as shown in FIG. 12.
In particular, compared with the method of calculating the difference value of two adjacent pixel groups (p_iand p_i+1), the method of calculating the difference value between a pixel group p_iand a pixel group p_i+q+1that are spaced at a predetermined number q can obtain larger difference values at the edges of the object image 121. As shown in FIG. 12, the difference value between two adjacent points 1212 and 1213 on the left side edge 1211 of the object image 121 is about 50, while the difference value between two separated points 1212 and 1213 is about 90. Moreover, due to the existence of noise, the method of calculating the difference value between a pixel group p_iand a pixel group p_i+q+1that are spaced at a predetermined number q is not easily affected by noise.
The predetermined number q can be a user set number, or a value determined by the modulation transfer function (MTF) of the image sensor module 11, wherein the MTF is a measure of the transfer of modulation (or contrast) from an object to an image. An MTF value can be a point on a MTF curve or an average of points on an MTF curve. From a background picture, a pixel group is selected and an initial intensity value M_orepresenting the pixel group is extracted. The following equation (2), an MTF value, and the initial intensity value M_oare used to determine the to number of iterations needed when a last iterated result is less than a predetermined value, wherein the number of iterations can be used as the number q.
$\begin{matrix} M_{i + 1} = \frac{1 - MTF}{1 + MTF} M_{i} & (2) \end{matrix}$
The description below is related to an image processing method that is applicable to a normally captured picture whose image intensity variation is not compensated for.
FIG. 14 is a flow chart related to an image processing method according to another embodiment of the present invention. Referring to FIG. 14, at Step 141, a background picture comprising a plurality of pixel groups arranged along a direction is generated. At Step S142, intensity values IB(p₁) of the pixel groups of the background picture are extracted to obtain an intensity fluctuating pattern IB(p) as shown in FIG. 15, where p_irepresents the i-th pixel group. The intensity value IB(p_i) of the pixel group can be the sum or the average of the intensity values of the pixels of the pixel group. At Step S143, the sequential calculation of the difference value BSD(p₁) between a pixel group p₁and a pixel group p_i+q+1that are spaced at an interval of a predetermined number q is performed one after another along the arrangement of the pixel group p_i, to the pixel groups of the background picture to obtain a difference fluctuating pattern BSD(p) as shown in FIG. 16. In another embodiment, the sequential calculation of the difference value between two adjacent pixel groups (p_iand p_i+1) is performed one after another along the arrangement of the pixel group p_i, to the pixel groups of the background picture to obtain a difference fluctuating pattern BSD(p).
At Step S144, a minimum of the difference fluctuating pattern BSD(p) is then determined as a threshold Th6, and a maximum of the difference fluctuating pattern BSD(p) is determined as a threshold Th7. A Step S145, a picture comprises a plurality of pixel groups arranged along a direction obtained. At Step S146, an intensity value I(p_i) representing each pixel group of the picture is extracted to obtain an intensity fluctuating pattern I(p) as shown in FIG. 17, where p_irepresents the i-th pixel group. The intensity value I(p_i) of the pixel group can be the sum or the average of the intensity values of the pixels of the pixel group. At Step S147, the sequential calculation of the difference value ISD(p_i) between a pixel group p_iand a pixel group p_i+1that are spaced at an interval of a predetermined number q is performed one after another along the arrangement of the pixel group p_i, to the pixel groups of the picture to obtain a difference fluctuating pattern ISD(p) as shown in FIG. 18. In another embodiment, the sequential calculation of the difference value between two adjacent pixel groups (p_iand p_i+1) is performed one after another along the arrangement of the pixel group p_i, to the pixel groups of the picture to obtain a difference fluctuating pattern ISD(p). At Step S418, on the difference fluctuating pattern ISD(p), the point 811 whose difference value is less than the threshold Th6 and the point 812 whose difference value is greater than a threshold Th7 are determined. One of the pixel groups corresponding to the point 811 is used as a left boundary pixel group, and one of the pixel groups corresponding to the point 812 is used as a right boundary pixel group. Accordingly, the section R2 constituted by a set of successive pixel groups between the left and right boundary pixel groups can be obtained. The section R2 can then be selected as an object image.
The data structures and code described in this detailed description are typically stored on a non-transitory computer-readable storage medium, which may be any device or medium that can store code and/or data for use by a computer system. The non-transitory computer-readable storage medium includes, but is not limited to, volatile memory, non-volatile memory, magnetic and optical storage devices such as disk drives, magnetic tape, CDs (compact discs), DVDs (digital versatile discs or digital video discs), or other media capable of storing code and/or data now known or later developed.
The methods and processes described in the detailed description section can be embodied as code and/or data, which can be stored in a non-transitory computer-readable storage medium as described above. When a computer system reads and executes the code and/or data stored on the non-transitory computer-readable storage medium, the computer system performs the methods and processes embodied as data structures and code and stored within the non-transitory computer-readable storage medium. Furthermore, the methods and processes described below can be included in hardware modules. For example, the hardware modules can include, but are not limited to, application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs), and other programmable-logic devices now known or later developed. When the hardware modules are activated, the hardware modules perform the methods and processes included within the hardware modules.
It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed embodiments. It is intended that the specification and examples be considered as exemplary only, with the true scope of the disclosure being indicated by the following claims and their equivalent.

Claims

1. An optical touch system comprising:

an image sensor module comprising a plurality of image sensing elements configured to be independently controlled to achieve different exposure times, the plurality of image sensing elements generating a picture comprising a plurality of pixel groups; and

a processor configured to extract an intensity value representing each pixel group and select a portion of the pixel groups as an object image according to the intensity values of the pixel groups.

2. The optical touch system of claim 1, wherein the picture is an illumination-compensated picture.

3. The optical touch system of claim 1, wherein each pixel group comprises a plurality of pixels, wherein the intensity value representing each pixel group is a sum or an average of intensity values of the pixels of the pixel group.

4. The optical touch system of claim 1, wherein each pixel group is a line of pixels.

5. The optical touch system of claim 1, wherein the processor is configured to subtract a threshold value from the intensity value representing each pixel group to obtain a plurality of difference values and to select the portion of the pixel groups according to ratios of the difference values to the threshold value.

6. The optical touch system of claim 1, wherein the portion of the pixel groups is greater or smaller than a threshold value.

7. The optical touch system of claim 5, wherein the threshold value is an average of the intensity values representing the pixel groups or a predetermined value.

8. The optical touch system of claim 1, wherein the image sensor module is configured to generate a background picture using a fixed exposure time and use an intensity value of the background picture corresponding to each pixel group to adjust the exposure times of corresponding image sensing elements.

9. The optical touch system of claim 1, wherein the image sensor module is configured to adjust the exposure time for each image sensing element to allow the image sensor module to generate a new background picture comprising a plurality of pixel groups having intensity values within an intensity range.

10. An image processing method of an optical touch system, comprising the steps of:

obtaining a picture comprising a plurality of first pixel groups;

is determining a plurality of first difference values each determined by subtracting intensity values of two of the plurality of first pixel groups; and

selecting a set of successive pixel groups as an object image according to the first difference values.

11. The method of claim 10, wherein the picture is an illumination-compensated picture.

12. The method of claim 10, wherein the step of determining a plurality of first difference values comprises a step of determining a first difference value between the intensity values of each two adjacent ones of the plurality of first pixel groups.

13. The method of claim 10, wherein the step of determining a plurality of first difference values comprises a step of determining a first difference value between the intensity values of a pair of first pixel groups separated by an interval of a predetermined number.

14. The method of claim 13, wherein the predetermined number is determined by the steps of:

obtaining an MTF value of an image sensor;

selecting the intensity value of one of the first pixel groups as an initial intensity value;

using the following equation, the MTF value and the initial intensity value to determine a number of iterations needed when an iterated result is less than a predetermined value:

M_{i + 1} = \frac{1 - MTF}{1 + MTF} M_{i}

where M_irepresents an intensity value on the i-th iteration and K_i+1represents an iterated result on the i-th iteration; and

selecting the number of iterations as the predetermined number.

15. The method of claim 10, further comprising the steps of:

determining, from the plurality of first pixel groups, a first boundary pixel group corresponding to the first difference value less than a first threshold value;

determining, from the plurality of first pixel groups, a second boundary pixel group corresponding to the first difference value greater than a second threshold value; and

selecting a plurality of ones of the pixel groups between the first and second boundary pixel groups as the object image.

16. The method of claim 15, further comprising the steps of:

obtaining a background picture that comprises a plurality of second pixel groups;

determining a plurality of second difference values each determined by subtracting intensity values of two of the plurality of second pixel groups;

selecting a minimum from the second difference values as the first threshold value; and

selecting a maximum from the second difference values as the second threshold value.

17. The method of claim 10, wherein each first pixel group comprises a plurality of pixels, and the intensity value representing each pixel group is a sum of intensity values of the pixels of the pixel group.

18. The method of claim 10, wherein the pixel group is a line of pixels.