JP2016500458A - Touch screen system and method based on touch position and force - Google Patents

Abstract

A touch screen system and method based on touch location and touch force is disclosed. The touch screen system includes an optical force sensing system interfaced with a capacitive touch sensing system so that both touch position and touch force information can be obtained. A display using a touch screen system is also disclosed.

Description

Explanation of related applications

  This application claims the benefit of priority under 35 USC 119 of US Provisional Patent Application No. 61/738047, filed Dec. 17, 2012. The present specification depends on the content of the specification of the provisional patent application, and the content of the specification of the provisional patent application is incorporated herein by reference in its entirety.

  The present disclosure relates to touch screens, and more particularly, to touch screen systems and methods based on touch position and Dutch force. All publications, papers, patent specifications, patent application publication specifications, etc. mentioned in this specification, including the specifications of US provisional patent applications 61/564003 and 61/564024, are all in their entirety. Included herein by reference.

  The market for displays and other devices (eg, keyboards) with non-mechanical touch functionality is growing rapidly. As a result, touch sensing technology has been developed to provide touch functionality to displays and other devices. The touch sensing function has been widely used for mobile device applications such as smartphones, electronic book readers, laptop computers, and tablet computers.

  The touch-sensing surface has become a preferred way for a user to interact with a portable electronic device. To this end, touch systems in the form of touch screens have been developed that respond to various types of touch, such as single touch, multi-touch and swipe. Some of these systems rely on light scattering and / or light attenuation based on optical contact with the touch screen surface that remains fixed relative to the support frame. An example of such a touch screen system is described in US Pat.

Consumer touch-based devices such as smartphones currently detect user interaction as the presence of an object (eg, a finger, stylus) on or near the device display. This is considered user input,
(1) Determine whether an interaction has occurred,
(2) calculate the XY position of the interaction; and (3) determine the length of the interaction.
Can be quantified.

  Touch screen devices are limited in that they can only collect position and timing data during user input. There is a need for another intuitive input that allows efficient operation and is not cumbersome for the user. By using touch events and input gestures, the user is no longer required to categorize menus, which saves time and extends battery life. Application programming interfaces (APIs) have been developed that consider user input in the form of touch, swipe and flick as gestures that are then used to generate event objects in the software. However, the more user input that can be included in the API, the more robust the performance of the touch screen device.

US Patent Application Publication No. 2011/0122091

  The present disclosure is directed to a touch screen device that uses both position input and force input from a user during a touch event. Force measurements are quantified by cover glass deflection during user interaction. Therefore, in order to generate an event object in the software, further force input parameters can be used in the API. The purpose of this disclosure is the use of force information from touch events in conjunction with projected capacitive touch (PCT) data for the same touch event to generate software based events in a human controlled interface. is there.

  Force touch sensing is an optical monitoring system and method, such as the systems and methods described in US Provisional Patent Application Nos. 61/640605, 61/651136, and 61/74431. Can be achieved.

  Many types of touch-sensitive devices, such as analog resistance, projected capacitive, surface capacitive, surface acoustic wave (SAW), infrared, camera-based optics, and several other methods Exists. The present disclosure will be described with capacitive-capable devices, such as projected capacitive touch (PCT) devices, that have the advantage of allowing multi-touch detection and being very sensitive and durable. The combination of position sensing and force sensing in the touch screen system disclosed herein allows the user to provide unique force related inputs (gestures). Thus, gestures such as pinching gestures can be replaced with gestures that push the touch screen with different magnitudes of force.

  Touch screen devices that utilize a combination of force sensing and position sensing have a number of advantages. The main advantage of using force monitoring is the intuitive interaction that nurtures the user experience. This allows the user to press a single position and adjust the object properties (eg, change the graphic image, change the volume of the audio output, etc.). Conventional attempts at single finger events use long press gestures, such as swiping or prolonged contact with a touch screen. By using force data, a fast response time that eliminates the need for long press gestures is possible. Long press gestures can be operated using a pre-determined equation for response speed (ie, long press gestures can scroll the page at a set speed or rapidly increasing speed), but with force-based sensing, The user can actively change the response time in real-time interaction. Thus, the user can change the scroll, for example, simply by changing the applied touch force. This provides the user with an experience that is more interactive and more efficient in operation.

  In addition, the use of force sensing combined with position sensing allows for a wide range of new touch screen functions (APIs), as described below.

  Additional features and advantages of the disclosure will be set forth in the detailed description which follows, and in part will be readily apparent to those skilled in the art from the description, or the following detailed description, the claims and the accompanying drawings will be described. Including, will be appreciated by practice of the present disclosure as described herein.

  The claims and summary are incorporated into the detailed description set forth below, and form a part of the detailed description.

FIG. 1A illustrates a touch screen system that can measure a touch position using a capacitive touch screen and can also measure an applied force at the touch position using an optical force sensing system in accordance with the present disclosure. FIG. FIG. 1B is a schematic diagram of a display system using the touch screen system of FIG. 1A. 2A is an exploded side view of an example of a display system that uses the touch screen system of FIG. 1A. 2B is a side view of the assembled display system of FIG. 2A. 2C is a top view of the example display system of FIG. 2B, but without a transparent cover sheet. FIG. 2D is a top view of the display system of FIG. 2B including a transparent cover sheet. FIG. 3A is a macroscopic view of an example proximity sensor shown for an example light deflection element and electrically connected to a microcontroller. FIG. 3B shows the top surface of the proximity sensor showing how the deflected light covers different areas of the photodetector as the light deflection element approaches or moves away from and / or rotates relative to the proximity sensor. FIG. FIG. 3C illustrates the proximity sensor top view showing how the deflected light covers different areas of the photodetector as the light deflection element approaches or moves away from and / or rotates relative to the proximity sensor. FIG. FIG. 4A is an enlarged side view of the edge region of the display system of FIG. 2B showing a transparent cover sheet and an adjacent capacitive touch screen. 4B is similar to FIG. 4A, but illustrates how the proximity sensor measures the deflection of the cover sheet caused by the touch force applied to the cover sheet at the touch position. FIG. 4C shows an enlarged side view of the edge region of another embodiment of the display system, where the proximity sensor is positioned proximate to the cover sheet. FIG. 4D is similar to FIG. 4C, but illustrates another method of how the proximity sensor measures the deflection of the cover sheet caused by the touch force applied to the cover sheet at the touch position. FIG. 5A shows an example of a zoom function of a graphic image displayed on the display system with the image before zooming, where zooming is achieved by application of touch force at the touch position. FIG. 5B shows an example of the zoom function of the graphic image displayed on the display system in the image after zooming, where zooming is achieved by applying touch force at the touch position. FIG. 6A shows an example of a page turning function of a graphic image in the form of a book page, and the page turning is achieved by applying a touch force at a touch position. FIG. 6B shows an example of a page turning function of a graphic image in the form of a book page, and the page turning is achieved by applying a touch force at a touch position. FIG. 7 shows an example of a menu selection function that is achieved by applying a touch force at the touch position. FIG. 8A shows an example of the scroll function and shows an increase in touch force. FIG. 8B shows an example of the scroll function, which shows that the scroll rate (speed) can be increased by increasing the touch force. FIG. 9A is similar to FIG. 8A and shows how the scroll function can be jumped from one position to the next by discretizing the force versus scroll bar position function. FIG. 9B is a graph showing a change in position based on the threshold value of the applied force. FIG. 10A shows an example of how a graphic image in the form of a line can be changed by a swipe combined with the application of a selected amount of touch force. FIG. 10B shows an example of how a graphic image in the form of a line can be changed by a swipe combined with the application of a selected amount of touch force. FIG. 11 shows an example of how a display image can be magnified or panned across the field of view using the application of a selected amount of touch force. FIG. 12 shows an example of how a graphic image in the form of a rotating rack of objects can be manipulated using the application of a selected amount of touch force. FIG. 13 shows how repetitive application of touch force (pumping or pulsing) within a short time can be used rather than application of increasing touch force.

  Although Cartesian coordinates are shown in some drawings for reference, there is no limiting purpose for direction or orientation.

  The present disclosure can be understood more readily by reference to the following detailed description, drawings, examples, and claims, and their respective previous and following description. However, prior to the disclosure and description of current compositions, articles, devices and methods, the present disclosure is not limited to particular compositions, articles, devices and methods, unless otherwise specified, and of course Of course it can change. It is understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

  The following description of the present disclosure is provided as a viable teaching of the present disclosure in currently known embodiments. For this purpose, those skilled in the art will recognize that many changes may be made to the various aspects of the disclosure described herein, but still the beneficial results of the disclosure can be obtained, I will accept it. It will also be apparent that some of the desired beneficial results of the present disclosure can be obtained by selecting some of the features of the present disclosure without utilizing other features. Accordingly, those skilled in the art will recognize that many modifications and adaptations to the present disclosure are possible and may even be desirable in some circumstances and are part of the present disclosure. That is, the following description is provided as an illustration of the principles of the present disclosure, not a limitation of the present disclosure.

  Materials, compounds, compositions and components that can be used for, can be used with, prepared for, or can be used for the disclosed methods and compositions Disclosed. Where these and other materials are disclosed herein and combinations, subsets, interactions, groups, etc. of such materials are disclosed, various individual and generic combinations and substitutions of such compounds are disclosed. It is to be understood that each specific reference to each is specifically contemplated and described herein, even if it may not be explicitly disclosed.

  That is, a group of replacement elements A, B, and C, and a group of replacement elements D, E, and F are also disclosed, and if examples of combination embodiments AD are disclosed, each is individually and It is generally considered. That is, in this example, each of the combinations AE, AF, BD, BE, BF, CD, CE, and CF is specifically considered. The disclosure of A, B and / or C, D, E and / or F and combination examples AD should be considered as disclosed. Similarly, any subset or combination of these is specifically contemplated and disclosed. That is, for example, the subgroup consisting of AE, BF, and CE is concretely disclosed by disclosure of A, B, and / or C, D, E, and / or F, and combination examples AD. Should be considered as being disclosed. This concept applies to all aspects of this disclosure including, but not limited to, any components and steps of the composition in the methods of making and using the disclosed composition. That is, if there are various additional steps that can be performed, each such additional step can be performed with any particular embodiment or combination of embodiments of the disclosed method, It is appreciated that each such combination is specifically contemplated and should be considered disclosed.

  FIG. 1A is a schematic illustration of a touch screen system 10 according to the present disclosure. The touch screen system 10 can be used in various consumer electronics products such as mobile phones, keyboards, touch screens and other electronic devices such as devices capable of wireless communication, music players, notebook computers, mobile devices, game controllers, computers. It can be used with a display for a “mouse”, e-book reader, etc.

  Touch screen system 10 includes a conventional capacitive touch screen system 12, such as a PCT touch screen. Examples of the capacitive touch screen system 12 are disclosed in, for example, U.S. Pat. Touch screen system 10 also includes an optical force sensing system 14 that is operatively interfaced with or otherwise operatively coupled to capacitive touch screen system 12. Both capacitive touch screen system 12 and optical force sensing system 14 are electrically connected to a microcontroller 16 that is configured to control the operation of touch screen system 10 as described below. Is done.

  In one example, the microcontroller 16 is provided with a capacitive touch screen system 12 (ie, forms part of the touch screen system) and connects directly to the force sensing system 14 (eg, via a 12C bus), Reconfigured to receive and process the force signal SF from the optical force sensing system 14 (eg, reprogrammed). The microcontroller 16 can also be connected to a multiplexer (not shown) to allow attachment of multiple sensors.

FIG. 1A shows a touch event TE that occurs at the force sensing system 14 at a touch position TL by a touch tool 20, such as a finger shown as an example. Other types of touch device 20, such as a stylus, the tip of a writing instrument, etc. can be used. In response, optical type force sensing system 14 represents a touch force F _T accompanying the touch event TE, a generating a force sensing signal (force signal) SF. Similarly, the capacitive touch screen 12 generates a position sensing signal (position signal) SL that represents a touch position associated with the touch event TE. The force signal SF and the position signal SL are sent to the microcontroller 16. The microcontroller 16 processes these signals, (e.g., API by) constituted an event object based on any of the touch position TL and touch event force F _T to generate the controller software. In one example, the microcontroller adjusts at least one feature of the display image 200 (discussed and introduced below) in response to at least one of the force signal SF and the position signal SL.

  In one example, the optical force sensing system 14 is configured to be retrofitted to provide a conventional capacitive touch screen system 12 with both position sensing and force sensing functions. In one example, the optical force sensing system 14 is configured as an adapter that is added to the capacitive touch screen system 12. In one example, the optical force sensing system 14 is interfaced with the microcontroller 16 as needed to condition the force signal before the force signal SF is applied to the microcontroller 16 (in a dashed box in FIG. 1A). It has a microcontroller 15 of the optical force sensing system itself (shown).

  FIG. 1B is similar to FIG. 1A and is a schematic diagram of an example display system 11 that utilizes the touch screen system 10 of FIG. 1A. The display system 11 has a display assembly 13 configured to generate a display image 200 that can be viewed by a viewer 100 through the touch screen 10.

  2A is an exploded side view of an example display system 11 utilizing the touch screen system 10, and FIG. 2B is an assembled side view of the example display system of FIG. 2A. The display system 11 has a frame 30 having a side wall 32 with a top end 33 and a bottom wall 34. The side wall 32 and the bottom wall 34 define an open inner chamber 36. The display system 11 also includes the aforementioned microcontroller 16 of the touch screen system 10, and in one example, the microcontroller is within the frame compartment 36 adjacent to the bottom wall 34 along with other display system components, such as at least one battery 18. It is in.

  The display system 11 also has a flex circuit 50 that is above the microcontroller 16 and battery 18 in the frame chamber 36. The flex circuit 50 has an upper surface 52 and an edge 53. A plurality of proximity sensor heads 54H are mounted on the upper surface 52 near the edge 53 of the flex circuit 50 in an operable manner. Referring to FIG. 3A, each proximity sensor head 54H includes a light source (eg, LED) 54L and a photodetector (eg, photodiode) 54D. The flex circuit 50 has electrical wiring (connection) 56 that connects different proximity sensor heads 54 </ b> H to the microcontroller 16. In one example, the wiring 56 constitutes a bus (for example, a 12C bus). The electrical wiring 56 transmits a force signal SF generated by the proximity sensor 54.

  Referring back to FIGS. 2A and 2B, the display system 11 further includes a display 60 disposed on the top surface 52 of the flex circuit 50. The display 60 has an upper surface 62, a lower surface 64, and an outer end 65. One or more spacing elements (spacers) 66 are provided on the upper surface 62 adjacent to the outer end 65. Display 60 includes a display controller 61 configured to control the operation of the display, such as the generation of display image 200. Display controller 61 is shown adjacent to touch screen microcontroller 16 and is operatively connected to touch screen microcontroller 16. In one example, only a single microcontroller is used rather than the individual microcontrollers 16 and 61.

  The display system 11 also has a capacitive touch screen 70 adjacent to the display top surface 62 and separated from the display top surface 62 by a spacer 66 to define a gap 67. The capacitive touch screen 70 has an upper surface 72 and a lower surface 74. The capacitive touch screen 70 is electrically connected to the microcontroller 16 by electrical wiring (connection) 76 that constitutes a bus (for example, a 12C bus) in one example. The electrical wiring 76 transmits a position signal SL generated by the capacitive touch screen.

The display system 11 also has a transparent cover sheet 80 having an upper surface 82 and a lower surface 84 and an outer end 85. The transparent cover sheet 80 is supported by the frame on the lower surface 84 of the transparent cover sheet at or near the outer end 85 in contact with the upper end 33 of the frame 30. The lower surface of the cover glass 80 adjacent to the outer end 85 and inside the outer end 85 so that the one or more light deflection elements 86 are optically aligned with the corresponding one or more proximity sensor heads 54H. 84 is supported. In one example, the light deflection element 86 is a plane mirror. The light deflection element 86 is a tilted shape that is used to provide better directional light transmission between the light source 54L of the proximity sensor 54 and the photodetector 54D, as will be described in more detail below. (For example, a wedge shape). In one example, the light deflection element has a curved shape. In another example, the light deflection element has a diffraction grating or a scattering surface. Each proximity sensor head 54H and corresponding optical deflection element 86, the proximity sensor 54 for detecting the displacement of the transparent cover sheet to determine the amount of the touch force F _T that is applied to the transparent cover sheet 80 by a touch event TE Determine.

In one example embodiment, the transparent cover sheet 80 is adjacent to and in close proximity to the capacitive touch screen 70. That is, the lower surface 84 of the transparent cover sheet 80 is in contact with the upper surface 72 of the capacitive touch screen 70. This contact can be facilitated by a thin transparent adhesive layer. By disposing the transparent cover sheet 80 and the capacitive touch screen 70 in contact, as will be discussed below, the transparent cover sheet 80 and the capacitive touch screen 70 are applied when a touch force _FT is applied. 70 can be deflected together.

  Here, it should be noted that the optical force sensing system 14 of FIG. 1 includes a transparent cover sheet 80, a light deflection element 86, a plurality of proximity sensors 54, a flex circuit 50, and electrical wiring 56 in the flex circuit 50. . The capacitive touch screen system 12 includes a capacitive touch screen 70 and electrical wiring 76. The display system 13 is composed of the remaining components, including in particular the display 60 and the display controller 61.

  Continuing with reference to FIG. 2B, the display 60 emits light 68 that travels through the gap 67, the capacitive touch screen 70 (transparent to the light 68) and the transparent cover sheet 80. . The light 68 is visible to the viewer 100 as a display image 200, which can be, for example, a graphic image, drawing, icon, symbol or anything that can be displayed. In one example embodiment, the display system 11 is configured to change at least one characteristic (or characteristic, or attribute, etc.) of the display image 200 based on the force signal SF and the position signal SL. Features of the display image can include dimensions, shape, magnification, position, motion, color, orientation, and the like.

  2C is a top view of the display system 11 of FIG. 2B, but without the transparent cover sheet 80, and FIG. 2D is the same top view, but including the transparent cover sheet. The transparent cover sheet 80 can be made of glass, ceramic or glass-ceramic that is transparent at the visible wavelength of the light 68. An example glass for the transparent cover sheet 80 is Gorilla glass from Corning Inc., Corning, NY, USA. The transparent cover sheet 80 shields the light deflection element 86 and any other components near the outer edge 85 of the display system 11 under the transparent glass sheet to obscure it to the user 100 (FIG. 2B). There may be an opaque cover (bezel) 88 adjacent to the outer end 85. Only a portion of the opaque cover 88 is shown in FIG. 2D for ease of illustration. In one example, the opaque cover 88 serves to block at least visible light and is configured to keep some areas of the display system 11 visible to the user 100, any type of light blocking member, bezel. , Film, paint, glass, component, material, surface pattern, structure, etc.

  FIG. 3A is an enlarged macro view of an example of a proximity sensor 54 having a sensor head 54H having a light source 54L and a photodetector 54D as discussed above. Each proximity sensor head 54H of the system 10 is electrically connected to the microcontroller 16 by electrical wiring 56, such that it is supported at least in part by the flex circuit 50. Examples of the light source 54L include an LED, a laser diode, an optical fiber base laser, a surface light source, and a point light source. The photodetector 54D can be a photodiode array, a large area photosensor, a linear photosensor, a collection or array of photodiodes, a CMOS detector, a CCD camera, and the like. An example proximity sensor head 54H is an OSRAM proximity sensor head, type SFH7773, which uses a 850 nm light source 54L and a high linear photosensor for the photodetector 54D. In one example, the proximity sensor 54 need not have a light source 54L and a photodetector 54D, and in some embodiments these components are separated from each other and can still perform the intended function.

  FIG. 3A also shows an example light deflection element 86 above the light source 54L and the photodetector 54D. Recall that the light deflection element 86 is disposed on the lower surface 84 of the transparent cover sheet 80 (not shown in FIG. 3A). In one example, the light source 54L emits the light 55 toward the light deflection element 86, and the light deflection element 86 returns the light to the photodetector 54D as the deflection light 55R. The proximity sensor head 54H and the light deflecting element 86 allow the deflected light 55R to cover the first area a1 of the photodetector 54D when the light deflecting element is in the first separation and the first orientation (FIG. 3B). Configured. Furthermore, when the light deflection element 86 is in the second separation (and / or second orientation), the deflected light covers the second area a2 of the photodetector (FIG. 3C). This means that the detector (force) signal SF changes depending on the position and / or orientation of the light deflection element 86.

  4A and 4B are enlarged side views of the edge region of the display system 11 showing the transparent cover sheet 80 and the adjacent capacitive touch screen 70 with one of the proximity sensors 54. In FIG. 4A, there is no touch event, and no force by the user 100 is applied to the display system 11. In this case, the light from the light source 54L is deflected by the light deflection element 86 to cover a certain region (area) of the photodetector 54D. This is shown as 55R, represented by a thick black line, which covers the entire detector area as an example.

FIG. 4B shows an example embodiment in which the touch tool (finger) 20 is pressed onto the transparent cover sheet 80 at the touch position TL to generate the touch event TE. Force _{F T} accompanying the touch event TE is to bend the transparent cover sheet 80. This bending acts to move the light deflection element 86, in particular, the light deflection element is brought closer to the proximity sensor 54 and in some cases slightly rotated. This subsequently changes the optical path of the deflected light 55R relative to the photodetector so that different amounts of deflected light impinge on the light receiving surface of the photodetector 54D. This is simply indicated by a thick black line representing the spread of the deflected light 55R displaced with respect to the photodetector 54D. A change in the amount of the deflected light 55R detected by the photodetector 54D is represented by a change in the detector (force) signal SF.

It should also be noted that the deflection of the transparent cover sheet 80 changes the distance between the light source 54L and the photodetector 54D, and a change in this distance can cause a change in irradiance detected at the photodetector. In one example, the photodetector 54 </ b> D can also detect the irradiance distribution and changes to the irradiance distribution as caused by the displacement of the transparent cover sheet 80. Irradiance distribution, for example, the detector can be a relatively small light spot moves toward the area, the position of the light spot is the size of the displacement is thus correlated with the magnitude of the touch force F _T. In another example, the irradiance distribution has a pattern like light scattering, and the scattering pattern changes as the transparent cover sheet is displaced.

  In another embodiment shown in FIGS. 4C and 4D, the proximity detector head 54H is on the lower surface 84 of the transparent cover sheet 80 and the light deflection element is on the upper surface 52 of the flex circuit 50, for example. Also in this other embodiment, the electrical wiring 56 of the flex circuit 50 is connected to the proximity sensor head 54H.

  In another example embodiment, the transparent cover sheet 80, the capacitive touch screen 70, and the display 60 are joined. In this case, the proximity sensor 54 can be arranged in an operable manner with respect to the display 60, and the proximity sensor head 54H or the light deflection element 86 can be arranged in an operable manner on the upper surface 62 of the display.

  Although the optical force sensing system 14 of the touch screen system 12 has been described above with respect to many different examples of the proximity sensor 54, other optical sensing means can be used by modifying the proximity sensor 54. For example, the proximity sensor 54 in which the diffracted light is detected by the photodetector 54D using the reflecting member 86 having a diffraction grating that diffracts the light instead of reflecting the light can be configured.

Furthermore, light may detect various wavelengths of light in the spectral band, the size of the given transparent cover sheet 80 displacement (hence, the magnitude of the applied to the transparent cover sheet 80 touch force F _T Can have a spectral bandwidth that can be related to The light source 54 </ b> L can also inject light into the waveguide on the lower surface 84 of the transparent cover sheet 80. The light deflection element 86 can be a waveguide diffraction grating configured to extract guided light, and the output light travels to the photodetector 54D and is light in different amounts or positions depending on the displacement of the transparent cover sheet. The light enters the detector 54D.

In another embodiment, proximity detector 54 can be configured as a microinterferometer by inserting a beam splitter into the optical path that provides a reference waveform to the photodetector. If the coherent light source 54L is used, the reference waveform and the reflected waveform from the light deflection element 86 can interfere with each other in the photodetector 54D. Then, in order to establish the displacement of the transparent cover sheet by touch force F _T, it is possible to use a change in the interference fringe pattern (irradiance distribution).

In one example, the proximity sensor 54 Fabry - can be configured to define a Perot cavity, the displacement of the transparent cover sheet 80, correlated to the magnitude of the applied touch force F _T that was used to produce a displacement It can change the finesse of the Fabry-Perot cavity. This can be achieved, for example, by adding a second partially reflecting window (not shown) arranged in an operable manner with respect to the reflecting member 86.

Proximity sensor head 54H and corresponding reflection member 86 as a result of the magnitude of the change of the touch force F _T, so that the change in the force signal SF by the displacement of the transparent cover sheet 80 occurs, constructed. On the other hand, capacitive touch screen 70 is associated with a touch force F _T as detected by the known capacitive sensing means, the touch event TE (x, y) position signal indicating a touch position TL SL is sent to the microcontroller 16. Therefore, a force signal SF represents the magnitude of the touch force F _T microcontroller 16 is provided at the touch position TL, also the touch position TL (x, y) be the position signal SL representing the position, receive. In one example, multiple force signals SF from different proximity sensors 54 are received and processed by the microcontroller 16.

In one example, the microcontroller 16 is calibrated such that the value (eg, voltage) provided for the force signal SF corresponds to the magnitude of the force. The calibration of the microcontroller measures the change in the force signal (due to a change in intensity or irradiance incident on the photodetector 54D), and the measured value is a known touch force F applied at one or more touch locations TL. It can be implemented by relating to the size of _T. That is, the relationship between the applied touch force _FT and the force signal can be established empirically as part of the calibration process of the display system or touch screen process.

  In one example, the occurrence of a touch event TE can be used to zero the proximity sensor 54. This can be done to compensate the sensor for any temperature difference that would make different proximity sensors 54 work differently.

  The microcontroller 16 is described below to control the operation of the touch screen system 10 and to generate display functions (eg, for event objects having incidental actions on the display 11). It is also configured to process the force signal SF and the position signal SL. In some embodiments, the microcontroller 16 is all operative, eg, disposed on a motherboard or integrated into a single integrated circuit chip or structure (not shown). It includes a processor 19a, a memory 19b, a device driver 19c, and an interface circuit 19d (see FIGS. 4A and 4B).

  In one example, the microcontroller 16 is configured or otherwise adapted to execute instructions stored in firmware and / or software (not shown). In one example, the microcontroller 16 is required to measure the operation of the touch screen system 10 and the relative magnitude of pressure or force and / or displacement of the transparent cover sheet 80 and also the touch position of the touch event TE. It can be programmed to perform the functions described herein, including any signal processing. As used herein, the term “microcontroller” is not limited to integrated circuits as technically referred to as computers, but broadly includes computers, processors, microcomputers, programmable logic controllers, application specific integrated circuits and Other programmable circuits, and combinations thereof, are also referred to, and these terms can be used interchangeably.

  In one example, the microcontroller 16 includes software configured to perform or assist in performing the functions and operations of the touch screen system 10 disclosed herein. The software can be installed in the microcontroller 16 in an operable manner, including its internals (eg, in the processor 19a). Software functions can include programming, including executable code, and such functions can be used to implement the methods disclosed herein.

  Such software code can be executed by a microprocessor. In operation, code and possibly associated data records are stored in a general purpose computer platform, in a processor unit, or in local memory. However, in some cases, the software can be stored elsewhere and / or transferred for loading into a suitable general purpose computer system. Accordingly, the embodiments discussed herein include one or more software products in the form of one or more code modules carried by at least one machine-readable medium. Execution of such code by a processor of a computer system or by a processor unit is essentially a platform discussed in this specification and implemented in the illustrated embodiment, in a catalog and / or software download function platform. Enables implementation by

  Referring again to FIG. 3A, the microcontroller 16 controls the light source 54L with the light source signal S1 and receives and processes the detector signal SF from the photodetector 54D. The detector signal SF is the same as the force signal described above and is therefore referred to hereinafter as the force signal. The plurality of proximity sensors 54 and the microcontroller 16 can be connected in a manner operable by the plurality of electrical wirings 56 described above and can be considered as part of the optical force sensing system 14. That is, both the capacitive touch screen 12 and the one or more proximity sensors 54 are electrically connected to the microcontroller 16 to provide the position signal SL and the force signal SF to the microcontroller.

In one example embodiment of the touch screen system 10, each force signal SF has a count value over a selected range, for example, 0-255. In one example, a count value of 0 represents the proximity sensor head 54H in contact with the transparent cover sheet 80 (or the light deflection element 86 thereon), and a count value of 255 is sufficient for the light deflection element to be sufficiently away from the proximity sensor head. Represents the situation. During calibration, readings from the proximity sensor 54 when the force on the touch screen system 10 is not applied α is recorded with the sensor readings β relative to the touch force F _T of the given large value.

The following equation also shows how the data represented by the force signal SF is normalized for a given proximity sensor and how it applies to other proximity sensors. Normalization factor N is:
N = [α _A −A] / [α _A −β _A ] · 100
Given in. Here, A is the proximity sensor data to a force signal SF, alpha proximity sensor readings when no force is applied F _T, beta is the force F _T is the proximity sensor reading at the maximum.

The data for all normalized proximity sensors 54 is then averaged. In addition, a rolling average process by taking the three most recent average values is used to smooth the data. Table 1 below serves to illustrate this concept: 'Ac # n' represents "array column #n" and 'AVG _R ' represents "moving average" for different times T. At the time of disclosure, a blank three-column array is initialized by the microcontroller 16 and contains no values. During the first time point, the first column (AC # 1) contains the average of all normalized sensors (denoted P1). At the next time point, data for P1 is moved to the second column (AC # 2), where AC # 1 is replaced with the average of all normalized sensors at the second time point (denoted P2).

This process continues for each point in time. The final value is averaged over three columns of data and accessed by software for various uses. The rolling average from the array is ignored until all columns are filled. Parameter P is:
P = standardized sensor data = standardized A + standardized B
+ Standardized C + standardized D
Given in. Value for AVG _R, in order to modify the width of the line in the form of a display image during a swipe, is used in the custom drawing program in the microcontroller 16. During a swipe, the line width increases when a certain amount of force _FT is applied. If the applied force is small, the line width is reduced.

In the exemplary embodiment of the present disclosure, a certain amount of touch pressure or touch force (pressure = F _T / area) is applied at the touch position TL in accordance with the touch event TE. Aspects of the present disclosure includes the relative magnitude of the applied force F _T as a function of the displacement of the transparent cover sheet 80 is directed to the sensing of the occurrence of a touch event TE. Time course of the displacement (or displacement over the course of time) and, therefore, can also be determined the time course of the touch force F _T.

That is, even the size change over time of the touch force F _T is also quantified by the proximity sensor and the microcontroller 16 based on the magnitude of the deflection of the transparent cover sheet 80. Software algorithms within the microcontroller 16 are used to smooth (eg, filter) the force signal SF, remove noise, and normalize force data. Thus, the applied force F _T, in combination with the position information to manipulate the properties of the graphical objects on a graphical user interface system 10 (GUI) to be able to use, also be used for control applications it can. Both single finger events and multi-finger events can be monitored. In order to cause the system 10 to perform various operations, such as selection, highlighting, scrolling, zooming, rotation and panning, among others, force information embedded in the force signal SF, taps, knobs Can be used instead of or in conjunction with other gesture-based controls, such as rotation, swipe, pan and long press operations.

For example, referring to FIGS. 5A and 5B, for a zoom-based event, a single finger event TE that includes pressure (ie, force F _T ) is used to zoom in on an image, such as the house image 200 shown. be able to. FIG. 5B shows the zoomed-in (high magnification) image 200. Graph plugged in FIG 5A shows an example of how to obtain changes to how the image magnification as the applied force F _T. To zoom out, a separate two finger event can be used, zooming out when the pressure is reduced. The combination of touch and force here is useful because force reduction can be used to reset the zoom. In this case, the user presses with a force by a single finger to zoom in, then changing the magnitude of the force F _T against the touch surface to another finger if you want to zoom out.

In another example, the system 10 replaces delay-based controls, such as long press touches, to allow fast response to equivalent functions. To alter the characteristics of the display image 200, for example, in drawing application, to modify the width of the line, or brush size during use (i.e., a paint brush size, eraser size) using a touch force F _T in order to change the it can. For image-based applications, force information from the force signal SF can be used to lighten / darken a photo or adjust contrast. As discussed above, in an image application or map program, the force data can provide a moving speed of the image during panning or an image scaling speed during the zoom function.

Touch-based data can be used with another user gesture (ie pinch and zoom) to perform some action (ie lock, pin, crop). To invert the display image (eg, graphic object) (from front to back) or rotate it by a selected amount, eg, 90 °, a strong press on the touch screen (ie, a relatively large touch force F _T ) Can be used. Referring to FIGS. 6A and 6B, for turning a plurality of pages of books image 200 at a time, can be used a touch event TE by considerable touch force F _T with swipe gesture SW. Force data can be used for speed control during a scroll event. A game application will find utility in setting the level of action or speed for a given graphic object or action (eg, corfu swing, bat swing, race acceleration, etc.). As shown in FIG. 7, force data can be used to open a submenu of menu list 210 or scroll through the list.

Figure 8A shows a scroll bar 220, the application of touch force F _T that becomes larger at the touch position corresponding to the scroll position, non-touch scroll bar (1), first lightly touched the scroll bar (2) and strong Increase the scroll speed as indicated by the pressed scroll bar (3). The arrows in FIG. 8A indicate increased mobility (speed). FIG. 8B is a velocity versus pressure or force graph that can be used to manipulate the speed at which the graphic object moves.

9A and 9B are similar to FIGS. 8A and 8B, it is possible to discretize the applied touch force F _T as a direct movement to another from one position of the object can be made as a function of the scroll position, 1 illustrates an example embodiment.

  FIGS. 10A and 10B show an example embodiment where a graphic image in the form of a line is swiped (SW) by a touch force FT at a touch position TL at one end of the line to increase the line width.

  FIG. 11 shows how the display system 11 shows how the graphic object 200 can be panned across the field of view (FOV) by wise application of touch force at one or more touch locations (TL) on the touch screen system 10. It is another example of a function. An area away from the FOV center within the FOV of the electronic document (ie, map, image, etc.) can be pushed to move the image in that direction. The stronger the pressing force, the faster the image moves in that direction. The main direction will be up, down, left or right as indicated by the arrows.

  FIG. 12 shows that the user can touch the selected touch position TL to determine the direction and apply pressure to increase the rotational speed of the various graphic objects forming the rotating rack of objects. Shows carousel application.

  In some cases there will be a maximum touch force F that can be used. Rather than exceeding the maximum touch force, in one example, a pumping action or a pulsing action can be used in which the touch device 20 pushes force multiple times within a given time. This option may be useful for applications such as games or in satellite images where the user will want to zoom in / out much faster than when applying maximum force.

  FIG. 13 briefly illustrates the use of a pumping or pulsing operation at the touch location TL to traverse large amounts of data of unknown size without the limitations of pressure sensing techniques. If pressure is required directly for motion translation (rather than speed), the user can alternately increase or decrease the pressure using a pulsing or pumping action. In this way, the decreasing pressure is ignored and the user can end the interaction simply by not applying the pressure. In this example, the user can increase the resizing magnification, and therefore can give a larger magnification without losing direct pressure accuracy.

  Although embodiments have been described herein with reference to specific aspects and features, it is to be understood that these embodiments are merely illustrative of the desired principles and applications. Thus, it will be appreciated that numerous modifications may be made to the exemplary embodiments and other arrangements may be devised without departing from the spirit and scope of the appended claims.

DESCRIPTION OF SYMBOLS 10 Touch screen system 11 Display system 12 Capacitive touch screen system 13 Display assembly 14 Optical force sensing system 15,16 Microcontroller 18 Battery 19a Processor 19b Memory 19c Device driver 19d Interface circuit 20 Touch tool 30 Frame 32 Frame side wall 33 Upper frame side wall 34 Frame bottom wall 36 Open inner chamber 50 Flex circuit 52 Flex circuit upper surface 53 Flex circuit edge 54 Proximity sensor 54D Photo detector 54H Proximity sensor head 54L Light source 55, 68 Light 55R Deflection light 56, 76 Electrical wiring (connection) )
60 Display 61 Display controller 62 Display upper surface 64 Display lower surface 65 Display outer edge 66 Space holding element (spacer)
67 Gap 70 Capacitive touch screen 72 Capacitance touch screen upper surface 74 Capacitance touch screen lower surface 80 Transparent cover sheet 82 Transparent cover sheet upper surface 84 Transparent cover sheet lower surface 85 Transparent cover sheet outer edge 86 Light deflection element 88 Opaque cover (bezel)
100 viewer 200 display image 210 menu list 220 scroll bar _FT touch force S1 light source signal SF force sensing signal (force signal)
SL Position sensing signal (position signal)
TE touch event TL touch position

Claims (9)

In a touch screen system for displaying a display image and detecting a touch event,
A capacitive touch system configured to detect a touch position and generate a position signal representing the touch position with respect to the touch event;
A force signal that is arranged in an operable manner with respect to the capacitive touch system and optically detects a touch force applied at the touch position and represents the touch force applied at the touch position. An optical force sensing system configured to generate, and a microcontroller electrically connected to the capacitive touch system and the optical force sensing system, wherein the microcontroller is the force signal and the position signal. And changing the characteristics of the display image based on the force signal and the position signal,
A touch screen system comprising: The optical force sensing system is at least one optical proximity sensor arranged in an operable manner with respect to the transparent cover sheet, the displacement of the transparent cover sheet due to the touch force applied at the touch position At least one optical proximity sensor configured to optically detect and generate the force signal in response thereto,
Each of the optical proximity sensors is
A light deflection element disposed on the lower surface of the transparent cover sheet;
A light source and a light detector that communicate with the light deflection element; and
The light source emits light that is deflected by the light deflection element to form deflected light that is detected by the photodetector, and the displacement of the transparent cover sheet is detected by the photodetector. Changing the amount of the deflected light,
The touch screen system according to claim 1. In a display system for viewing display images,
The touch screen system according to claim 1 or 2, and a display unit arranged in an operable manner relative to the touch screen system so that the display image is visible through the touch screen system.
A display system comprising:   The display unit of claim 3, wherein the display unit is configured to change characteristics of the display image based on the force signal and the position signal received from the touch screen system. In a method for changing at least one characteristic of an image displayed on a display system,
Detecting the position of the touch event at the touch position by a capacitive method, and in response to generating a touch signal representing the touch event position;
Detecting a touch force associated with the touch event by an optical method, and generating at least one force signal representing the touch force in response thereto;
Processing the touch signal and the at least one force signal to change the at least one feature of the displayed image;
Having
The step of detecting by the optical method includes the step of measuring the displacement of the transparent cover sheet of the display system by an optical method.
A method characterized by that.   The step of measuring the displacement of the transparent cover sheet by an optical method includes a step of detecting a change in the amount of detected light as a result of a change in an optical path along which the deflected light caused by the displacement travels. The method of claim 5.   7. A method according to claim 5 or claim 6, wherein the features of the displayed image include features selected from the group of features including size, shape, magnification, position, motion, color and orientation. . The displayed image is
An electronic document image having a plurality of pages, wherein the change of the at least one characteristic of the displayed image includes a change of the page, or a scroll bar having a scrolling speed, wherein the display of the displayed image The change of the at least one feature includes a change of the scroll speed, or a line having a width, wherein the change of the at least one feature of the displayed image includes a change of the width of the line;
The method according to claim 5, comprising one of the following: The step of detecting the touch force by an optical method includes the step of measuring the displacement of the transparent cover sheet using a plurality of optical sensors, and the method includes:
Normalizing a plurality of force signals from the plurality of photosensors; and taking a rolling average of the photosensors over two or more time frames;
The method according to claim 5, further comprising:

Images