JP4685054B2 - Image processing apparatus, image processing method, program, and computer-readable recording medium storing the program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method for detecting a touch position on a touch panel.

  2. Description of the Related Art Conventionally, touch panels that provide a sheet on a display surface and detect contact with a finger, a pen, or the like by a resistance film method, a pressure sensitive method, an electrostatic method, or the like have been widely used. In recent years, in order to reduce the manufacturing cost and reduce the size of the display device, instead of providing a sheet on the display surface, it is possible to detect the shadow of a finger or the like projected on the display screen. A display device that recognizes which part of the screen is touched by image processing is used (for example, Patent Document 1).

  Further, Patent Document 1 discloses a configuration in which, when light is incident on the display screen from an oblique direction, the touch position is corrected based on the light incident direction.

  FIGS. 16A to 16C show the display device 101 described in Patent Document 1. FIG. As shown in FIG. 16A, the display device 101 has a panel surface 102 and a display screen 103, and the display screen 103 is provided in the lower part of the panel surface 102. The panel surface 102 includes a transparent portion 102a and a shielding portion 102b, and a light detection sensor for detecting the position of the shadow of the finger 110 is built in the display screen 103.

  The display device 101 detects which part of the transparent portion 102a the finger 110 touches as follows.

When the ambient light 120 is incident in an oblique direction with the finger 110 in contact with the transparent portion 102a, the shadow 111 on the panel surface 102 shown in FIG. 16B and the display screen shown in FIG. The shadow 113 detected at 103 shifts. On the other hand, the shielding surface 102 b is provided on the panel surface 102, and thereby, a shadow 121 of the shielding portion 102 b is formed on the display screen 103. Therefore, using the width Δx of the shadow 121 as a correction value, the center 114 of the tip portion of the shadow 113 is corrected by a distance Δx in the direction opposite to the shifted direction. Thereby, the touch position 112 of the finger 110 is calculated.
JP 2006-244446 A (published September 14, 2006)

  However, since the display device 101 shown in FIG. 16 is premised on detecting a shadow caused by the ambient light 120 that is parallel light, the display device 101 is used under the environment of the room light 130 as shown in FIG. In this case, the following problems occur.

That is, since the room light 130 is non-parallel light that spreads radially from a light source that is not so far away, the incident angle θ _{1 of} light incident on the shield E of the shielding portion 102b and the transparent portion that is located away from the shield E This is different from the incident angle θ _{2 of the} light incident on the point P on 102a. For this reason, the distance ΔP between the shadow P _{1 of the} point P detected on the display screen 103 and the point P ₂ corresponding to the point P ₂ directly below the display screen 103 is different from the distance Δx. Therefore, even if the shadow P ₁ is corrected by the distance Δx, the corrected shadow P ₁ does not coincide with the point P ₂ , so that an accurate touch position is detected on the display device 101 under the environment of the indoor light 130. I can't.

  Also, when touching the panel surface with a finger, the finger's cross section is generally an elliptical shape that swells in the horizontal direction, so the detected shadow edge of the finger is not the contact point between the finger and the panel surface, but the panel Corresponds to the side of the finger away from the surface.

  This will be specifically described with reference to FIG. Assume that the display device 101 is used in the ambient light 120. When ambient light 120 is incident while the finger 110 is in contact with the transparent portion 102a, the edge of the shadow 113 detected on the display screen 103 corresponds to the side portion of the finger 110. Here, the side portion of the finger 110 is considerably separated from the transparent portion 102a. For this reason, the distance ΔF between the center 114 calculated from the shadow 113 and the point 115 corresponding directly below the touch position 112 of the finger 110 is different from the distance Δx described above. Therefore, even if the center 114 is corrected by the distance Δx, the corrected shadow 114 does not coincide with the point 115. For this reason, when the panel surface is touched with a finger, the display device 101 cannot detect an accurate touch position.

  A method of enlarging the operation button displayed on the operation screen is also conceivable on the assumption that the detected touch position and the actual touch position are slightly shifted. However, when the operation button is enlarged, it is difficult to display an operation screen with detailed contents, particularly in the case of a small display device such as a mobile phone.

  The present invention has been made in view of the above-described problems, and an object thereof is to realize an image processing apparatus and an image processing method that can detect a contact position more accurately.

  In order to solve the above problems, an image processing apparatus according to the present invention contacts the display panel based on a signal from a photosensor in a sensor circuit layer that includes a photosensor that is disposed in a lower layer of the display panel and senses light. An image processing apparatus comprising position information acquisition means for acquiring position information of a shadow projected on the sensor circuit layer of an object to be moved, immediately below the sensor circuit layer of each light shielding portion provided at a plurality of locations on the display panel Displacement detecting means for detecting a displacement between the position corresponding to the position of the shadow and the position of the shadow projected on the sensor circuit layer of the light shielding unit, the displacement, the position information, and the light shielding unit A correction value calculation means for calculating a correction value for the position information from the position of the shadow projected on the sensor circuit layer, and a position information correction method for correcting the position information based on the correction value. When, from the position information corrected by the position information correcting unit is characterized in that it comprises a contact position detecting means for detecting a contact position between the object and the display panel.

  According to said structure, the shadow of the object which contacts a display panel, and the shadow of a light shielding part are projected on a sensor circuit layer. Thereby, a positional information acquisition means acquires the positional information on the shadow of the said object and a light shielding part. When light is incident on the display panel from an oblique direction, the shadow of the object and the light shielding part projected on the sensor circuit layer is incident from the position corresponding to the object and the light shielding part immediately below the sensor circuit layer. The amount of displacement depends on the angle. The displacement detection means detects the displacement, and the correction value calculation means calculates a correction value for the position information from the displacement, the shadow position information of the light shielding unit, and the shadow position of the object. The position information correction unit corrects the position information based on the correction value, and the contact position detection unit detects the contact position between the object and the display panel from the corrected position information.

  Here, when the image processing apparatus is used in an environment of room light, since the room light is non-parallel light, the angle at which the light enters the display panel varies depending on the incident position. Therefore, in the above configuration, a plurality of light shielding portions are provided, and displacements in each light shielding portion are detected. The displacement between the shadow of the object and the position corresponding to the position immediately below the object is determined by the displacement at each light shield and the positional relationship between each light shield and the object. The displacement from the position corresponding to can be calculated as a correction value.

  Therefore, since the position information can be accurately corrected even in an environment where non-parallel light is incident on the display panel, it is possible to realize an image processing apparatus that can detect the contact position more accurately. .

In the image processing apparatus according to the present invention, the light shielding unit includes a first light shielding unit and a second light shielding unit, the horizontal direction of the display panel is the x direction, and the vertical direction of the display panel is the y direction. The coordinates of the first position, which is the position corresponding to the position immediately below the first light shielding portion in the sensor circuit layer, is (x ₀ , y ₀ ), and the shadow projected on the sensor circuit layer of the first light shielding portion is The displacement of the second position, which is the position, from the first position is (Δx ₁ , Δy ₁ ), and the coordinates of the third position, which is the position corresponding to the position immediately below the second light shielding portion in the sensor circuit layer, are ( X ₀ , Y ₀ ), the amount of displacement from the third position of the fourth position, which is the position of the shadow projected on the sensor circuit layer of the second light shielding part, from the third position (Δx ₂ , Δy ₂ ), and the object The coordinates of the shadow projected onto the sensor circuit layer are (P, Q), and the correction value is corrected in the x direction. The Delta] p, [Delta] q correction value in the y direction of the correction value, and when, in the x-direction correction value and the correction value of the y-direction,
Δp = (Δx ₁ X ₀ −Δx ₂ x ₀ + P (Δx ₂ −Δx ₁ )) / (X ₀ + Δx ₂ −x ₀ −Δx ₁ )
Δq = (Δy ₁ Y ₀ −Δy ₂ y ₀ + Q (Δy ₂ −Δy ₁ )) / (Y ₀ + Δy ₂ −y ₀ −Δy ₁ )
It is preferable to obtain by the following formula.

  According to said structure, the value of the variable contained in the right side of each said formula can be detected by the sensor position information acquisition means and the displacement detection means. Therefore, by detecting the value of each variable, the correction value of the position information of the shadow of the object can be obtained.

  In the image processing apparatus according to the present invention, the value obtained by multiplying the correction value in the x direction by a predetermined magnification is a first correction value, and the value obtained by multiplying the correction value in the y direction by the magnification is the second correction value. Preferably, the position information correction unit corrects the position information based on the first correction value and the second correction value.

  If the object in contact with the display panel is an object having a considerable thickness such as a finger, the edge portion of the finger shadow is the side of the finger away from the display panel. The displacement is larger than the displacement of the shadow of the light shielding part. Therefore, according to the above configuration, the position information correction unit corrects the position information using a correction value obtained by multiplying each correction value in the x direction and the y direction by a predetermined magnification. Therefore, the object that touches the display panel By setting a predetermined magnification in accordance with the thickness, a correction value considering the thickness can be calculated.

  In the image processing apparatus according to the present invention, the object is a finger, a distance between the display panel and the sensor circuit layer is d (mm), and a thickness of a portion of the finger in contact with the display panel is 2 t (mm). ), The magnification is preferably 1 + t / d.

  According to the above configuration, the height of the widest side portion of the finger can be approximated to be about half of the thickness of the finger. Therefore, it can be approximated that the distance between the finger side corresponding to the edge portion of the finger shadow and the display panel is also ½ of the finger thickness. Therefore, more accurate correction is possible by setting the predetermined magnification to 1 + t / d.

  The program according to the present invention causes a computer to function as the displacement detection unit, the correction value calculation unit, the position information correction unit, and the contact position detection unit of the image processing apparatus. The program is recorded on a computer-readable recording medium.

  In order to solve the above-described problems, an image processing method according to the present invention contacts the display panel based on a signal from a photosensor in a sensor circuit layer including a photosensor that is disposed in a lower layer of the display panel and senses light. A position information acquisition step of acquiring position information of a shadow projected on the sensor circuit layer of an object to be performed, and a position corresponding to a position directly below the sensor circuit layer of each light shielding portion provided at a plurality of locations on the display panel; A displacement detection step for detecting a displacement of the light shielding portion with respect to the position of the shadow projected on the sensor circuit layer; the displacement, the position information; and the projection on the sensor circuit layer of each light shielding portion A correction value calculation step for calculating a correction value for the position information from the position of the shadow to be performed, a position information correction step for correcting the position information based on the correction value, From the corrected location information in location information correction step, it is characterized by having a contact position detection step of detecting a contact position between the object and the display panel.

  According to the above configuration, in the position information acquisition step, the shadow of the object that touches the display panel and the shadow of the light shielding unit are projected onto the sensor circuit layer, and the position information of the shadow of the object and the light shielding unit is acquired. To do. When light is incident on the display panel from an oblique direction, the shadow of the object and the light shielding part projected on the sensor circuit layer is incident from the position corresponding to the object and the light shielding part immediately below the sensor circuit layer. The amount of displacement depends on the angle. In the displacement detection step, the displacement is detected, and in the correction value calculation step, a correction value for the position information is calculated from the displacement, the position information of the shadow of the light shielding unit, and the position of the shadow of the object. In the position information correction step, the position information is corrected based on the correction value, and in the contact position detection step, the contact position between the object and the display panel is detected from the corrected position information.

  Here, when the image processing apparatus is used in an environment of room light, since the room light is non-parallel light, the angle at which the light enters the display panel varies depending on the incident position. Therefore, in the above configuration, a plurality of light shielding portions are provided, and displacements in each light shielding portion are detected. The displacement between the shadow of the object and the position corresponding to the position immediately below the object is determined by the displacement at each light shield and the positional relationship between each light shield and the object. The displacement from the position corresponding to can be calculated as a correction value.

  Therefore, even in an environment in which non-parallel light is incident on the display panel, the position information can be accurately corrected, so that it is possible to realize an image processing method that can detect the contact position more accurately. .

  As described above, the image processing apparatus according to the present invention has a position corresponding to a position immediately below the sensor circuit layer of each light shielding portion provided at a plurality of locations on the display panel and the sensor circuit layer of the light shielding portion. From the displacement detection means for detecting a displacement with respect to the position of the projected shadow, the position information from the displacement, the position information, and the position of the shadow projected on the sensor circuit layer of each light shielding unit Correction value calculation means for calculating a correction value for the position, position information correction means for correcting the position information based on the correction value, and the object and the display panel from the position information corrected by the position information correction means. Contact position detecting means for detecting the contact position.

  In addition, as described above, the image processing method according to the present invention contacts the display panel based on a signal from a photosensor in a sensor circuit layer including a photosensor that is disposed in a lower layer of the display panel and senses light. A position information acquisition step of acquiring position information of a shadow projected on the sensor circuit layer of an object, a position corresponding to a position directly below the sensor circuit layer of each light shielding portion provided at a plurality of positions on the display panel, and A displacement detection step for detecting a displacement from the shadow position projected on the sensor circuit layer of the light shielding unit, the displacement, the position information, and the projection on the sensor circuit layer of each light shielding unit. A correction value calculating step for calculating a correction value for the position information from the position of the shadow, a position information correction step for correcting the position information based on the correction value, and the position information From the corrected position data in the correction step, and a contact position detecting step of detecting a contact position between the object and the display panel.

  Therefore, there is an effect that an image processing apparatus and an image processing method that can detect the contact position more accurately can be realized.

[Embodiment 1]
An embodiment of the present invention will be described below with reference to FIGS.

  FIG. 1 is a block diagram illustrating a configuration of a display device 1 according to the present embodiment. The display device 1 includes a liquid crystal panel 2, an image processing unit 3, a display control unit 4, and a liquid crystal driving unit 5. The liquid crystal panel 2 includes the optical sensor 2a. The image processing unit 3 includes a position information acquisition unit 31, a displacement detection unit 32, a correction value calculation unit 33, a position information correction unit 34, and a touch position detection unit 35. It is composed of An operation screen is displayed on the liquid crystal panel 2, and operation buttons such as icons are arranged in the operation screen. The user can operate the display device 1 by selecting an operation button with a touch pen or a finger. In the present embodiment, it is assumed that the user operates the display device 1 with a touch pen.

  FIG. 2 is a cross-sectional view showing the configuration of the liquid crystal panel 2. The liquid crystal panel 2 includes a protective cover 21, a glass 22, a sensor circuit layer 23, a liquid crystal layer 24, and a glass 25, and a backlight 26 is provided on the back side. The photosensors 2a are provided in a matrix in the sensor circuit layer 23, one for each pixel. The optical sensor 2a is composed of, for example, a photodiode and a comparator, and outputs a high level signal when the amount of light incident on the optical sensor 2a is greater than or equal to a predetermined value, and the amount of light incident on the optical sensor 2a is less than the predetermined value. In this case, a low level signal is output. Thereby, it can also be set as the structure which the optical sensor 2a detects a shadow, only when the shadow projected on the sensor circuit layer 23 reaches a certain density. Although not shown, a light shield is provided below each optical sensor 2a so that the optical sensor 2a does not detect light from the backlight 26. When the display device 1 is a reflective liquid crystal display device, the backlight 26 is not provided.

  FIG. 3A is a perspective view of the protective cover 21 of the liquid crystal panel 2 and shows a state where the tip P of the touch pen T is in contact with the protective cover 21. Here, minute marks A and B are provided near the two opposite vertices of the protective cover 21. The marks A and B are provided, for example, by sticking a seal, and are provided at positions slightly apart from the end of the protective cover 21, respectively. In the following description in FIGS. 3 and 4, the direction from the right to the left is the x direction, and the direction from the back to the front is the y direction.

FIG. 3B is a plan view of the sensor circuit layer 23 of the liquid crystal panel 2. As shown in FIG. 3A, when ambient light 120, which is parallel light, is incident from directly above the liquid crystal panel 2, the shadows of the marks A and B appear as shown in FIG. Projected to the points A ₁ and B ₁ corresponding to the circuit layer 23 immediately below the marks A and B. Here, the coordinate information of the point A ₁ and the point B ₁ is stored in advance in the displacement detection unit 32 shown in FIG. 1, and the point A ₁ coordinate is (x ₀ , y ₀ ) and the point B ₁ coordinate is ( X ₀ , Y ₀ ).

When the ambient light 120 is incident from directly above the liquid crystal panel 2, the shadow of the tip P of the touch pen T corresponds to a position immediately below the tip P in the sensor circuit layer 23 as shown in FIG. Projected to point P ₁ . Here, assuming that the coordinates of the point P ₁ are (p, q), the point P ₁ corresponds to a position immediately below the tip P in the sensor circuit layer 23, and therefore the coordinates (p, q) are the touch position. In FIG. 3B, shadows of portions other than the tip P of the touch pen T are omitted.

  4A is a perspective view of the protective cover 21 of the liquid crystal panel 2, and shows a state where the tip P of the touch pen T is in contact with the protective cover 21. FIG. The touch position of the tip P of the touch pen T is the same as the touch position of the tip P of the touch pen T shown in FIG. In the configuration shown in FIG. 4A, the room light 130 which is non-parallel light is incident on the liquid crystal panel 2 from an oblique direction.

Here, as shown in FIG. 2, since the optical sensor 2a is built in at a predetermined depth from the surface of the protective cover 21, it is projected onto the sensor circuit layer 23 as shown in FIG. 4B. position of the shadow _{P 2} of the front end P being does not coincide with _the point _{P 1} shown in FIG. 3 (b). Here, assuming that the shadow P ₂ is shifted by Δp in the x direction and Δq in the y direction from the point P ₁ , in order to accurately detect the touch position of the tip P, only this displacement (Δp, Δq) is required. , it is necessary to correct the position of the shadow P _2.

  Hereinafter, a method for calculating the displacement (Δp, Δq) will be described.

As described above, the marks A and B are provided on the protective cover 21. Accordingly, as shown in FIG. 4B, the shadows A ₂ and B _{2 of} the marks A and B are projected onto the sensor circuit layer 23 by the room light 130. Here, if the coordinates of the shadow A ₂ are (x ₀ + Δx ₁ , y ₀ + Δy ₁ ), and the coordinates of the shadow B ₂ are (X ₀ + Δx ₂ , Y ₀ + Δy ₂ ), the marks A and B in the sensor circuit layer 23 Since the coordinates (x ₀ , y ₀ ) · (X ₀ , Y ₀ ) of the point A ₁ and the point B ₁ corresponding directly below are stored in advance, the point A _{1 of the} shadow A ₂ · B ₂ Each displacement (Δx ₁ , Δy ₁ ) and (Δx ₂ , Δy ₂ ) from B ₁ can be calculated.

  Δp, which is the x component of the displacement, is calculated as follows.

FIG. 5 is a side view showing the liquid crystal panel 2, and the direction from right to left in the figure is the x direction. Since the protective cover 21 and the sensor circuit layer 23 are parallel, the ratio of the distance AP in the x direction to the distance PB in the x direction is equal to the distance A ₂ P _{2 in} the x direction and the distance P _2. equal to the ratio of the x-direction distance B _2. Therefore, if p + Δp = P,
(X ₀ −p) :( p−x ₀ ) = ((X ₀ + Δx ₂ ) −P) :( P− (x ₀ + Δx ₁ ))
p = ((Δx ₂ −P) x ₀ + (P−Δx ₁ ) X ₀ ) / (X ₀ + Δx ₂ −x ₀ −Δx ₁ )
Therefore, Δp = P−p
= (Δx ₁ X ₀ −Δx ₂ x ₀ + P (Δx ₂ −Δx ₁ )) / (X ₀ + Δx ₂ −x ₀ −Δx ₁ )
... Formula (1)
It becomes. Thereby, since each variable is already known, Δp which is an x component of the displacement is calculated.

  Δq, which is the y component of the displacement, is calculated in substantially the same manner as the calculation of Δp.

FIG. 6 is a side view showing the liquid crystal panel 2, and the direction from right to left in the figure is the y direction. Since the protective cover 21 and the sensor circuit layer 23 are parallel, the ratio between the distance in the y direction of the distance AP and the distance in the y direction of the distance PB is the distance in the y direction of the distance A ₂ P ₂ and the distance P _2. equal to the ratio of the y-direction distance B _2. Therefore, when q + Δq = Q,
(Y ₀ −q): (q−y ₀ ) = ((Y ₀ + Δy ₂ ) −Q): (Q− (y ₀ + Δy ₁ )
q = ((Δy ₂ −Q) y ₀ + (Q−Δy ₁ ) Y ₀ ) / (Y ₀ + Δy ₂ −y ₀ −Δy ₁ )
Therefore, Δq = Q−q
= (Δy ₁ Y ₀ −Δy ₂ y ₀ + Q (Δy ₂ −Δy ₁ )) / (Y ₀ + Δy ₂ −y ₀ −Δy ₁ )
It becomes. Thereby, Δq which is the y component of the displacement can be calculated.

As described above, the displacement amount (Delta] p, [Delta] q) is calculated, the displacement amount (Delta] p, [Delta] q) as a correction value, corrects the coordinates shadow P _2. Thus, the coordinates of the corrected coincides with the coordinates of the point P ₁ shown in FIG. 3 (a). Thus, it is possible to detect the coordinates of the point P ₁ as the touch position of the tip P of the touch pen T.

  The above processing is performed by the image processing unit 3 shown in FIG. Next, the processing of the image processing unit 3 will be described with reference to FIG.

When the leading end P of the touch pen T is in contact with the liquid crystal panel 2, the position information acquisition unit 31 acquires the position of the shadow P ₂ of the tip P (step S1), the shadow A Mark A · B provided on the protective cover 21 The position of ₂ · B ₂ is acquired (step S2). The displacement detection unit 32 determines whether or not the positions of the shadows A ₂ and B ₂ coincide with the predetermined positions A ₁ and B ₁ (step S3). If they match ( "YES" in step S3), and the touch position detecting unit 35 detects the coordinates of the shadow _{P 2} as the touch position (step S4). If they do not match (“NO” in step S3), the displacement detection unit 32 detects the respective displacements (Δx ₁ , Δy ₁ ) of the shadows A ₂ and B _{2 from} the predetermined positions A ₁ and B ₁ and ( (Δx ₂ , Δy ₂ ) are detected (step S5). Subsequently, the correction value calculation unit 33 calculates correction values (Δp, Δq) for the shadow P ₂ by the calculation method in FIGS. 5 and 6 based on the displacements (Δx ₁ , Δy ₁ ) · (Δx ₂ , Δy ₂ ). ) Is calculated (step S6). Subsequently, the position information correction section 34 corrects the position of the shadow _{P 2,} calculates the position _{P 1} (step S7), and the touch position detecting unit 35 detects the position _{P 1} as the touch position (Step S8) .

  When the touch position detection unit 35 detects the touch position (step S4 or step S8), the touch position detection unit 35 outputs position information of the touch position to the display control unit 4 (step S9). The display control unit 4 outputs a control signal to the liquid crystal drive unit 5 according to the touch position (step S10), and the operation screen is switched (step S11). Note that “YES” in step S3 is limited to the case where parallel light such as ambient light 120 enters the liquid crystal panel 2 from directly above.

  As described above, the display device 1 can accurately detect the touch position even when non-parallel light is incident on the liquid crystal panel 2. Therefore, it is possible to make the operation screen finer by reducing the operation button.

In the above description, as shown in FIGS. 3 and 4, by providing the marks A and B on the protective cover 21, the displacements (Δx ₁ , Δy ₁ ) · for calculating the correction values (Δp, Δq) are obtained. Although (Δx ₂ , Δy ₂ ) is detected, the present invention is not limited to this. Instead of providing the marks A and B, a guide for blocking light may be provided on the outer periphery of the protective cover 21.

  FIG. 8 is a perspective view of the protective cover 21 of the liquid crystal panel 2 and shows a state where the tip P of the touch pen T is in contact with the protective cover 21. A guide G that blocks light is provided on the outer periphery of the protective cover 21, and a rectangular frame that surrounds the display screen is formed by the guide G. In the following description of FIGS. 8 and 9, as in the description of FIGS. 3 and 4, the direction from the right to the left is the x direction, and the direction from the back to the front is the y direction.

FIG. 8B is a plan view of the sensor circuit layer 23 of the liquid crystal panel 2. As shown in FIG. 8A, when ambient light 120, which is parallel light, is incident from directly above the liquid crystal panel 2, the shadow of the guide G causes a sensor circuit layer as shown in FIG. It is projected to the region G ₁ corresponding to the right under the guide G at 23. Here, the position information of the region G ₁ is stored in the displacement amount detector 32 shown previously in FIG.

When the ambient light 120 is incident from directly above the liquid crystal panel 2, the shadow of the tip P of the touch pen T corresponds to the position immediately below the tip P in the sensor circuit layer 23 as shown in FIG. It is projected to a point _{P 1,} similar to FIG. 3 (b), the coordinates of the point _{P 1} and (p, q).

Here, as shown in FIG. 9A, when the indoor light 130 which is non-parallel light is incident on the liquid crystal panel 2 from an oblique direction, as shown in FIG. 9B, in the sensor circuit layer 23, shadows of the guide G is projected to the region G _2. When the room light 130 is incident from the same direction as in FIG. 4A, the region G ₂ has a right side Δx _{1 in} the x direction and a left side Δx in the x direction, compared to the region G _{1. 2} , the upper side is shifted by Δy _{1 in} the y direction, and the base is shifted by Δy _{2 in the} y direction. As described above, the position information of the region G _1, since stored in the displacement amount detector 32 shown previously in FIG. 1, by comparing the area G ₂ and the region G _1, the displacement amount ([Delta] x ₁ , Δx ₂ , Δy ₁ , Δy ₂ ) can be detected.

Here, the shadow P ₂ of the tip P of the touch pen T is, if deviated [Delta] q from the point P ₁ Delta] p in the x direction, the y-direction, the correction value for the shadow P2 (Δp, Δq), like the above calculations , And can be calculated based on each displacement (Δx ₁ , Δx ₂ , Δy ₁ , Δy ₂ ).

FIG. 10 is a side view showing the liquid crystal panel 2, and the direction from right to left in the figure is the x direction. Compared with FIG. 5, in FIG. 5, the displacements (Δx ₁ , Δx ₂ ) are detected by comparing the shadows A ₂ and B _{2 of the} marks A and B and the predetermined positions A ₁ and B _1. and What hand, in FIG. 10, by comparing the shade G ₂ of the guide G and a predetermined region G _1, detects the displacement amount. Δp, which is the x component of the displacement, is calculated by the above equation (1). Also, Δq, which is the y component of the displacement, is calculated in substantially the same manner as Δp.

  As described above, in the present embodiment, a mark or a guide for blocking light is provided on the liquid crystal panel 2, and the position of the shadow at the tip of the touch pen is corrected by detecting the displacement of the shadow of the mark or guide. . In order to block light, for example, a plurality of bars D may be provided on the protective cover 21 as shown in FIG. By detecting the displacement between the position corresponding to the position immediately below the bar D in the sensor circuit layer 23 and the shadow of the bar D, a correction value for the position of the shadow of the tip of the touch pen can be calculated.

  Furthermore, as shown in FIG. 11 (b), the rod D may be embedded in the protective cover 21. Thereby, the troublesomeness of the operation due to the stick D protruding can be eliminated.

  Moreover, when providing a mark in the protective cover 21, in this embodiment, although the structure which provides a mark in two places was demonstrated, you may provide not only in this but in three or more places. However, in order not to obstruct the screen display, it is desirable to provide two places on the diagonal of the protective cover 21 as shown in FIG. As a result, detection of the displacement in the x direction and detection of the displacement in the y direction are most efficiently performed.

  In the present embodiment, a display device having a liquid crystal panel has been described. However, the present invention is not limited to this, and a display device having a CRT monitor may be used.

[Embodiment 2]
Another embodiment of the present invention will be described below with reference to FIGS. In the first embodiment, the configuration for detecting the touch position in the operation with the touch pen has been described. In the present embodiment, the configuration for detecting the touch position in the operation with the finger will be described.

  FIG. 12A is a perspective view of the protective cover 21 of the liquid crystal panel 2 and shows a state where the finger F is in contact with the protective cover 21. Minute marks A and B are provided in the vicinity of two opposing vertices of the protective cover 21, and the room light 130 is incident on the liquid crystal panel from an oblique direction. The configuration of the protective cover 21, the marks A and B, and the indoor light 130 shown in FIG. 12A is the configuration of the protective cover 21, the marks A and B, and the indoor light 130 shown in FIG. Is substantially the same.

FIG. 12B is a plan view of the sensor circuit layer 23 of the liquid crystal panel 2. The sensor circuit layer 23, the shadow _A 2 · _{B 2} shadows _{F 1} and Mark A · B of the finger F by room light 130 is projected. Note that the coordinates of the shadows A ₂ and B ₂ are respectively changed from the points A ₁ and B ₁ (not shown) corresponding to the positions immediately below the marks A and B in the sensor circuit layer 23 by displacements (Δx ₁ , Δx ₂ ). It is assumed that (x ₀ + Δx ₁ , y ₀ + Δy ₁ ) and (X ₀ + Δx ₂ , Y ₀ + Δy ₂ ) shifted by (Δy ₁ , Δy ₂ ). Also in the following description of FIG. 12, the direction from the right to the left is the x direction and the direction from the back to the front is the y direction, as in the description of FIGS. Here, the intersections of the straight line Lx parallel to the x direction and the edge of the shadow F1 are respectively S ₁ (s _x , s _y ) and R ₁ (r _x , r _y ).

FIG. 13 shows a cross section of the finger F along the straight line Lx and the side surface of the liquid crystal panel 2, and the direction from right to left in the figure is the x direction. Since the finger F has a considerable thickness, the edge of the shadow of the finger F due to the light from the light source L is not the contact point between the finger F and the protective cover 21 but the shadow of the side of the finger F. Correspond. For example, the point R ₁ corresponds to the projected portion of the left side portion R toward the finger F, and the point S ₁ corresponds to the projected portion of the right side portion S toward the finger F. Here, the side R and S, since apart from the protective cover 21, the same correction method in the first embodiment, it is impossible to accurately correct the shadow F _1.

Specifically, in the configuration of the first embodiment, a portion where light passing through the side portion R enters the protective cover 21 is a point R ₀ , and a portion corresponding to the point immediately below the point R ₀ in the sensor circuit layer 23 is a point R _2. Then, the point R ₁ is corrected to the point R ₂ . Similarly, in the configuration of the first embodiment, a portion where light passing through the side portion S enters the protective cover 21 is a point S ₀ , and a portion corresponding to the point immediately below the point S ₀ in the sensor circuit layer 23 is a point S _2. , it corrects the points _{S 1} to a point _{S 2.}

However, in order to more accurately detect the touch position of the finger F, the points S ₁ and R ₁ correspond to points R ₃ and S _{3 that} correspond almost directly below the widest portion of the sides of the finger F. It is necessary to correct it. Here, the side portions R and S are located in the vicinity of the widest portion of the side portion of the finger F. Therefore, the distance between the protective cover 21 and the sensor circuit layer 23 is d, the distance between the widest portion of the side portions of the finger F and the protective cover 21 is t, and the point RS ₁ and the point R ₂ in the x direction are in the x direction. The distance is Δr ₁ , the distance between the points R ₁ and R ₃ in the x direction is Δr ₂ , the distance between the points S ₁ and S ₂ in the x direction is Δs ₁ , and the x direction between the points S ₁ and S ₃ If the distance of Δs ₂ is
Δr ₂ ≈ (1 + t / d) · Δr ₁
Δs ₂ ≈ (1 + t / d) · Δs ₁
It becomes. Each value of Δr ₁ and Δs ₁ is calculated from the shadow displacement (Δx ₁ , Δx ₂ ) of the marks A and B by the calculation method described in the first embodiment (see FIG. 5). Each value of Δr ₁ and Δs ₁ is also calculated by the following equation. By using the calculated (Δr ₁ , Δs ₁ ) as a correction value, the point S ₁ · R ₁ can be corrected to the point R ₃ · S ₃ .

  Note that the size of the finger F cannot be specified because there is a difference between individuals. For example, when the finger F is used only by a specific person, a predetermined size can be obtained by registering the finger size in advance. It is possible to specify the magnification 1 + t / d. The thickness of the portion of the finger F that contacts the protective cover 21 may be approximated to 2t. For example, the value of t may be set within a predetermined range such as 2 mm to 10 mm.

  Although the calculation of the correction value in the x direction has been described with reference to FIG. 13, the calculation of the correction value in the y direction is substantially the same.

Figure 14 is a plan view of the sensor circuit layer 23, a shadow F ₁ before area surrounded correction by the dotted line, the area enclosed by the solid line is an area F ₂ after correction. Points R ₃ and S ₃ are located at the edge of the region F ₂ . For even other portions of the shadow F _1, by performing the same correction, it is possible to correct the entire shadow F ₁ in the area F _2. Subsequently, by calculating the center of the tip portion of the area F _2, it is possible to detect the touch position H of the finger F.

  The above processing is performed by the image processing unit 3 shown in FIG. Next, processing in the image processing unit 3 will be described with reference to FIG.

When the finger F comes into contact with the liquid crystal panel 2, the position information acquisition unit 31 acquires the position of the shadow F ₁ of the finger F (step S 12), and the shadows A ₂ and B of the marks A and B provided on the protective cover 21. ₂ is acquired (step S13). The displacement detection unit 32 determines whether or not the positions of the shadows A ₂ and B ₂ match the predetermined positions A ₁ and B ₁ (step S14). If it matches ( "YES" in step S14), and the touch position detecting unit 35 detects the coordinates of the shadow _{F 1} as the touch position (step S15). If they do not match (“NO” in step S14), the displacement detector 32 detects the displacements (Δx ₁ , Δy ₁ ) of the shadows A ₂ and B _{2 from} the predetermined positions A ₁ and B ₁ (Δx ₁ , Δy ₁ ) and ( (Δx ₂ , Δy ₂ ) are detected (step S16). Subsequently, the correction value calculation unit 33 calculates correction values (for example, dot points) for each part of the shadow F ₁ by the calculation method in FIG. 13 based on the displacements (Δx ₁ , Δy ₁ ) · (Δx ₂ , Δy ₂ ). for R _1, it calculates the [Delta] r ₂₎ in x-direction (step S17). Subsequently, the position information correction unit 34 corrects the area of the shadow F ₁ and calculates the area F ₂ (step S18). Thus, the touch position detection unit 35, from the region _{F 2,} to detect the touch position H (step S19).

  When the touch position detection unit 35 detects the touch position (step S15 or step S19), the touch position detection unit 35 outputs position information of the touch position to the display control unit 4 (step S20). The display control unit 4 outputs a control signal to the liquid crystal drive unit 5 according to the touch position (step S21), and the operation screen is switched (step S22).

  In the present embodiment, the configuration in which the marks A and B are provided on the protective cover 21 has been described. However, as shown in FIGS. 8 and 9, the guide G may be provided on the outer periphery of the protective cover 21. Moreover, as shown in FIG. 11, the structure which provides the rod D in the protective cover 21 may be sufficient.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  In addition, each block of the image processing unit 3, in particular, the displacement detection unit 32, the correction value calculation unit 33, the position information correction unit 34, and the touch position detection unit 35 may be configured by hardware logic, as follows. Alternatively, it may be realized by software using a CPU.

  That is, the image processing unit 3 includes a CPU (central processing unit) that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, and a RAM (random access memory) that expands the program. And a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is a recording medium on which a program code (execution format program, intermediate code program, source program) of a control program of the image processing unit 3 which is software for realizing the above-described functions is recorded so as to be readable by a computer. This can also be achieved by supplying the image processing unit 3 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the image processing unit 3 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention can be suitably applied to a display device including a display panel for touch operation.

It is a block diagram which shows the structure of the display apparatus which concerns on this invention. It is sectional drawing which shows the structure of the liquid crystal panel which the said display apparatus has. (A) is a perspective view of the protective cover of the said liquid crystal panel, (b) is a top view of the sensor circuit layer of the said liquid crystal panel. (A) is a perspective view of the protective cover of the said liquid crystal panel, (b) is a top view of the sensor circuit layer of the said liquid crystal panel. It is a side view which shows the said liquid crystal panel at the time of seeing from ay direction. It is a side view which shows the said liquid crystal panel at the time of seeing from x direction. It is a flowchart which shows the process sequence in the image process part of the said display apparatus. (A) is a perspective view of the protective cover of the said liquid crystal panel, (b) is a top view of the sensor circuit layer of the said liquid crystal panel. (A) is a perspective view of the protective cover of the said liquid crystal panel, (b) is a top view of the sensor circuit layer of the said liquid crystal panel. It is a side view which shows the modification of this invention, and shows the said liquid crystal panel when it sees from ay direction. (A), (b) shows the other modification of this invention, and is a side view which shows a part of said liquid crystal panel. (A) is a perspective view of the protective cover of the said liquid crystal panel, (b) is a top view of the sensor circuit layer of the said liquid crystal panel. It is a side view of the said liquid crystal panel shown in FIG. 12 when it sees from ay direction. It is a top view of the sensor circuit layer of the said liquid crystal panel. It is a flowchart which shows the other process sequence in the image process part of the said display apparatus. (A) is sectional drawing which shows the conventional display apparatus, (b) is a top view which shows the panel surface of the said display apparatus, (c) is a top view which shows the display screen of the said display apparatus. is there. It is sectional drawing of the display apparatus shown in FIG. It is sectional drawing of the display apparatus shown in FIG.

Explanation of symbols

1 Display device 2 Liquid crystal panel (display panel)
2a Optical sensor 3 Image processing unit (image processing means)
21 Protective cover (display panel)
23 sensor circuit layer 31 position information acquisition part (position information acquisition means)
32 Displacement detection unit (displacement detection means)
33 Correction value calculation unit (correction value calculation means)
34 position information correction unit (position information correction means)
35 Touch position detection unit (contact position detection means)
A / B mark (light shielding part)
D Bar (light shielding part)
F Finger (object)
G Guide (light shielding part)
P Tip (object)
H Touch position (contact position)

Claims (7)

Based on a signal from a photo sensor in a sensor circuit layer provided with a photo sensor for sensing light disposed under the display panel, position information of a shadow projected on the sensor circuit layer of an object in contact with the display panel is obtained. In an image processing apparatus comprising position information acquisition means for
A displacement for detecting a displacement between a position corresponding to a position directly below the sensor circuit layer of each light shielding portion provided at a plurality of locations on the display panel and a position of a shadow projected on the sensor circuit layer of the light shielding portion. Minute detection means;
Correction value calculating means for calculating a correction value for the position information from the displacement, the position information, and the position of the shadow projected on the sensor circuit layer of each light shielding unit;
Position information correction means for correcting the position information based on the correction value;
An image processing apparatus comprising: a contact position detection unit that detects a contact position between the object and the display panel from the position information corrected by the position information correction unit. The light shielding part includes a first light shielding part and a second light shielding part,
The horizontal direction of the display panel is the x direction, the vertical direction of the display panel is the y direction,
The coordinates of the first position, which is the position corresponding to the position immediately below the first light shielding portion in the sensor circuit layer, are (x ₀ , y ₀ ),
The amount of displacement from the first position of the second position, which is the position of the shadow projected on the sensor circuit layer of the first light shielding part, is (Δx ₁ , Δy ₁ ),
The coordinates of the third position, which is a position corresponding to the position immediately below the second light shielding portion in the sensor circuit layer, are (X ₀ , Y ₀ ),
The displacement from the third position of the fourth position, which is the position of the shadow projected on the sensor circuit layer of the second light shielding portion, is (Δx ₂ , Δy ₂ ),
The coordinates of the shadow projected on the sensor circuit layer of the object are (P, Q),
The correction value in the x direction of the correction value is Δp,
The correction value in the y direction of the correction value is Δq,
Then, the correction value in the x direction and the correction value in the y direction are
Δp = (Δx ₁ X ₀ −Δx ₂ x ₀ + P (Δx ₂ −Δx ₁ )) / (X ₀ + Δx ₂ −x ₀ −Δx ₁ )
Δq = (Δy ₁ Y ₀ −Δy ₂ y ₀ + Q (Δy ₂ −Δy ₁ )) / (Y ₀ + Δy ₂ −y ₀ −Δy ₁ )
The image processing apparatus according to claim 1, wherein the image processing apparatus is obtained by: A value obtained by multiplying the correction value in the x direction by a predetermined magnification is a first correction value, and a value obtained by multiplying the correction value in the y direction by the magnification is a second correction value.
The image processing apparatus according to claim 2, wherein the position information correction unit corrects the position information based on the first correction value and the second correction value. The object is a finger,
The distance between the display panel and the sensor circuit layer is d (mm),
When the thickness of the finger in contact with the display panel is 2t (mm),
The image processing apparatus according to claim 3, wherein the magnification is 1 + t / d.   A program for causing a computer to function as the displacement detection unit, the correction value calculation unit, the position information correction unit, and the contact position detection unit of the image processing apparatus according to any one of claims 1 to 4.   A computer-readable recording medium on which the program according to claim 5 is recorded. Based on a signal from a photo sensor in a sensor circuit layer provided with a photo sensor for sensing light disposed under the display panel, position information of a shadow projected on the sensor circuit layer of an object in contact with the display panel is obtained. A location information acquisition step,
A displacement for detecting a displacement between a position corresponding to a position directly below the sensor circuit layer of each light shielding portion provided at a plurality of locations on the display panel and a position of a shadow projected on the sensor circuit layer of the light shielding portion. A minute detection step;
A correction value calculating step for calculating a correction value for the position information from the displacement, the position information, and a position of a shadow projected on the sensor circuit layer of each light shielding unit;
A position information correction step for correcting the position information based on the correction value;
An image processing method comprising: a contact position detecting step of detecting a contact position between the object and the display panel from the position information corrected in the position information correcting step.

Images