TWI719034B - User identification through an external device on a per touch basis on touch sensitive devices - Google Patents

Abstract

Techniques for communicating particular information from a user, using a sensing device, to a touch device by way of a touch event is provided. Sensors that are operatively coupled to a sensing device sense an input from the user which conveys particular information. This input is then converted by the sensing device into another signal that is then transmitted from the sensing device to the touch device. When the user uses the sensing device to interact with the touch device, the interaction causes a touch event on the touch device. A touch event occurs when the sensors within the touch device receive a touch-signal from the sensing device. The touch-signal is based, at least in part, on detected biometric information of the user. The touch device then decodes the touch-signal to both (a) determine the location of the touch event on the touch device and (b) obtain the sensing device signal embedded in the touch signal to extract the particular information related to the user.

Description

User identification based on each touch of the touch-sensitive device on the external device

The content disclosed in the present invention is about conveying specific user-related information to touch-sensitive devices through touch events. More specifically, an external device is used to convey specific user-related information to the touch-sensitive device through touch events.

Touch screen technology has revolutionized digital devices such as smartphones and tablets, making them more and more capable of handling daily operations. At present, consumers can perform tasks such as e-mail, web browsing, word processing, and online banking through touch screen devices without the need for traditional computers.

The current touch screen device transmits the two-dimensional coordinates of the touch event to the operating system of the device. A touch event occurs when the touch screen senses the proximity of a conductive object (for example, a finger or a stylus) within a certain distance of one or more node sensors attached to the touch screen. The capacitance value of the finger interferes with the capacitance value measured at the node sensor. Similarly, a device (such as a stylus) is used to interfere with the capacitance value at a given node sensor. The node sensor picks up the change in the measured capacitance value, which is manifested as a bump in the capacitance value on the specific node sensor. The touch screen node sensor is designed to deliver the capacitance value to the touch screen controller. The touch screen controller is configured to deliver the touch event when the protruding spike exceeds a predetermined threshold. Equal two-dimensional coordinates.

Specifically, once the predetermined threshold is exceeded, the touch screen controller will interpolate the touch position of the touch event, and then transmit the two-dimensional coordinates (such as X and Y coordinates) to the device on the device working system. The operating system then maps the two-dimensional coordinates from the touch screen controller to the two-dimensional coordinate space of the operating system to calculate whether the touch event falls on the display control in order to trigger a certain action.

The current touch screen technology requires a two-step process for authentication and user touch input. The first step is to authenticate a user to unlock the device or authorize the action. The second step is the subsequent user touch input to perform the required action on the device. For example, the traditional method of accessing a security device involves the input of a user identification code (ID) and password. This authentication method assumes that after the user ID and password have been successfully entered, the authorized user performs the subsequent actions. However, this is vulnerable to unauthorized access because the hijacker can simply steal or obtain the user ID and password, and then enter the stolen user ID and password to gain access to the device. The device will still assume that the person performing the subsequent actions is the authorized user. Alternatively, the hijacker can simply steal the device after the authorized user has entered the password.

Current authentication methods against the stolen user ID and password scenario include fingerprint scanning and other techniques. The fingerprint scanner is embedded in the device, such as the current iPhone ^™ home button, so the user must use his unique fingerprint to unlock the device before performing subsequent touch events. However, this technique is still an indirect authentication method. Hijackers may not be able to steal your fingerprints, but they can obtain them after the device has been authenticated by an authorized user. Once authenticated and unlocked, the device still assumes that these subsequent touch events are performed by authorized users, not by hijackers.

Other techniques can use technologies such as Bluetooth or near field communication to continuously authenticate that the user is in the vicinity. However, this implements the same two-step method to authenticate where the communication point between the Bluetooth device and the touch device is vulnerable to hijacking. Once hijacked, the touch device will still assume that the authorized user is performing a subsequent touch event.

The methods described in this paragraph are methods that can be explored, but not necessarily methods that have been previously conceived or explored. Therefore, unless otherwise indicated, it should not be assumed that any of these methods described in this paragraph have the prior technical conditions merely by virtue of being included in this paragraph.

Interactions refer to related applications; claims of interests

This application is a partial continuation of US Patent Application No. 14/303,493 filed on June 12, 2014, the entire content of which is hereby incorporated by reference for all purposes as fully described in the text. The applicant hereby revokes any waiver of the scope of claims in the original case or its prosecution history, and reminds the United States Patent and Trademark Office (USPTO) that the scope of patent applications in this application may be broader than any claims in the original case.

General overview

In order to overcome these vulnerabilities in the two-step method of authentication and subsequent user input, a technology in which authentication is performed within the touch event itself is provided. Specifically, the method, non-transitory computer readable medium, and system are provided for communicating specific information from the user to the touch device through touch events.

According to a specific embodiment, a sensing device separate from the touch device detects an input conveying specific information. Based on the sensor type used by the sensing device, the nature of the input detected by the sensing device can vary with different implementations. In a specific embodiment, the sensor detects The specific information obtained includes the unique biological information of the user.

Then, the input detected from the user is converted by the sensing device into another signal, which is then transmitted from the sensing device to the touch device. When the user interacts with the touch device using the sensing device, the interaction causes a touch event on the touch device. A touch event occurs when the sensors in the touch device receive touch signals from the sensor device. The touch signal is based at least in part on the biological information detected by the user. Then, the touch device decodes the touch signal to simultaneously (a) determine the location of the touch event on the touch device, and (b) obtain the signal of the sensing device embedded in the touch signal to extract information about the user Specific information.

This method of directly transmitting the user's unique specific data will not require the two-step process of authentication and subsequent touch events. By embedding user-specific data into the touch signal itself, the user can convey his identity together with the coordinates of the touch event.

101‧‧‧Sensing device; sensing unit; specific sensing

102‧‧‧Sensor; sensing device sensing 器; Sensing unit sensor; Device sensor

103‧‧‧Processing unit

104‧‧‧Transmission unit; touch-control transmission unit

105‧‧‧User's finger; wireless transmission unit

106‧‧‧User's skin

200‧‧‧Program

201-203‧‧‧Block

301‧‧‧Touch Device; Capacitive Touch Device

302‧‧‧Drive line

303‧‧‧Sensing line

304‧‧‧Target node

305‧‧‧Identification code signal

306‧‧‧Target sensing line

307‧‧‧Touch signal

308‧‧‧Sensor Controller

315, 316, 317, 318‧‧‧Capacitance value

320‧‧‧Set threshold

325‧‧‧Set Binary Threshold; Binary Threshold

400‧‧‧Program

401‧‧‧Step

402‧‧‧step; decoding step

403, 404‧‧‧ output

500‧‧‧Computer system

502‧‧‧Bus

504‧‧‧hardware processor; processor

506‧‧‧Main memory

508‧‧‧Read only memory

510‧‧‧Storage device

512‧‧‧Display

514‧‧‧Input device

516‧‧‧Cursor Control

518‧‧‧Communication interface

520‧‧‧Network link

522‧‧‧Local Area Network

524‧‧‧Host computer

526‧‧‧Internet Service Provider

528‧‧‧Internet

530‧‧‧Server

The first figure illustrates the system diagram of the sensing device.

The second figure illustrates the flow chart of the method implemented by the sensing device.

The third diagram A illustrates a diagram of a touch device receiving a signal from the sensing device.

The third B to third E diagrams illustrate graphs depicting the measured capacitance values at nodes on a specific sensing line.

The fourth figure illustrates a flowchart of the method implemented by the touch device to receive and decode signals from touch events.

The fifth figure illustrates an example computer system that can be specially configured to perform the various techniques described in this article.

In the following description, for the purpose of understanding, numerous specific details are described in order to provide a complete understanding of the present invention. However, it should be obvious that the present invention can be implemented without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessary obscuring of the present invention.

Sensing device

The first figure is a diagram illustrating a sensing device 101 that can be used to perform the real-time authentication technique described in the text. The sensing device 101 may have the form of a stylus, and the transmission unit 104 may be disposed on the tip of the stylus that is usually in contact with the touch device.

Other specific embodiments of the sensing device 101 include, but are not limited to, handheld devices or wearable devices such as gloves, loops, or thimble. The transmission unit 104 will be strategically placed so that it starts to directly contact the touch device during the touch event. For example, if the sensing device 101 is a ring, the transmission unit 104 is placed on the head of the ring. If the sensing device 101 is a glove, the transmission unit 104 is arranged or configured to transmit through the fingertips of the glove.

As will be explained in more detail below, the sensing device includes a sensor 102. The nature of the sensor 102 used by the sensing unit 101 will vary based on the type of information communicated to the touch device. For example, if the information is bio-impedance information, the sensing device sensor 102 in the sensing device 101 will be a specialized sensor designed to detect bio-impedance based on contact with the user’s skin Device. However, if the conveyed information is the retina scan of the user, the sensing device sensor 102 will be a dedicated camera configured to detect and read the retina of the user.

Other specific embodiments of the sensing unit 101 may include, but are not limited to, input devices such as a keyboard, a joystick, a trackball, a mouse, and a touchpad. These devices can be connected to touch devices Or other computers, and can use the sensor device sensor 102 to detect specific information from the user, and transmit the information to the touch device along with the device input requested by the user. For example, if the sensing unit 101 is a dedicated keyboard, in a specific embodiment, the sensing device sensor 102 will be placed on the keys of the keyboard to detect the user's biological information. After collecting the biological information, the processing unit 103 converts the biological information into a binary code, which is then transmitted to the touch device along with the key input pressed by the user. Similarly, if the sensing device 101 is a mouse or a trackball device, the sensing unit sensor 102 will be placed on the mouse button of the mouse shell.

Sense specific user data

The first figure illustrates a specific embodiment of the sensing device 101 for detecting information from the user. For the purpose of communicating the information to the processing unit 103 included in the sensing device 101, the sensing device 101 includes a sensor 102 that detects specific information about the user.

The information detected by the sensor 102 may include, but is not limited to, biological information about the user. Biometrics are measurable biological characteristics to identify individuals. The biological information that helps to determine the identity of the user must be easily measurable in all individuals, unique to each individual, and a permanent feature that does not change over time. The biological information used to determine user identification includes, but is not limited to, bioimpedance in fingers, hands or wrists, electrocardiogram (ECG), retinal scan, iris scan, DNA comparison, and veins in fingers or palm Distribute the image.

In a specific embodiment, the information detected by the sensing device 101 from the user is the bioimpedance of the user. Bioimpedance refers to the electrical properties of biological tissues when current flows through it. Depending on the specific density of the individual's bones, blood vessels, and flesh, the bio-impedance varies from individual to individual.

In a specific embodiment of detecting bio-impedance, the sensing device 101 uses a sensing device sensor 102 coupled to the sensing device 101, and detects the bio-impedance when the user touches the sensing device 101. In a specific embodiment, the sensing device sensor 102 is strategically placed on the sensing device 101, so the sensing device sensor 102 starts to interact with the user's skin 106 (for example, the fingertip of the user's finger 105) contact. Once in contact with the user's skin 106, the sensing device sensor 102 will detect the bio-impedance from the user. In a specific embodiment, the sensing device sensor 102 continuously detects biological information.

In another embodiment, the sensing device sensor 102 coupled to the sensing device 101 may include a camera that scans the retina and/or iris of the user in order to scan uniquely identifiable biological information. Other specific embodiments may also include the sensing device sensor 102, which detects the vein distribution image of the user's finger or palm.

Generate identification code signal

The processing unit 103 is operatively coupled to the sensing device 101. In a specific embodiment, the processing unit 103 is configured to receive the input detected by the sensor 102 of the sensing device, and convert the input (for example: biometric information from the user) into the text referred to as "identification code". Signal". The identification code signal is a signal that will encode specific information detected from the user. As will be explained in more detail below, the identification code signal is generated in a form that can be transmitted from the sensing device 101 to another device.

Referring to FIG. 3A, which illustrates the sensing device 101, in which the processing unit 103 converts the input from the device sensor 102 into an identification code signal 305. Specific examples of the identification code signal 305 include, but are not limited to, a modulated alternating current (AC) at high frequency. For example, the identification code signal 305 is converted biological information. The biological information is at a frequency of 2000 Hz. The rate is modulated to AC signal. Other embodiments of the identification code signal 305 can encode the detected information in other ways, for example, by changing the frequency of the AC signal.

In other specific embodiments, the processing unit 103 receives the sensed biological information sensed by the sensor 102 of the sensing device, converts the biological information into a digital bit stream, and adds information about the user or the sensor. The additional information of the device is added to the digital bit stream, and then the digital bit stream is converted into an identification code signal 305. The additional information added to the biological information by the processing unit 103 may include, for example, personal data about the user, location information about the user, or personal preferences set by the user.

Regarding information about the sensing device 101, such information may include, but is not limited to, the type of the sensing device 101, for example: whether the device is a stylus, glove, ring, or some other special sensing device 101. In addition, the additional information may also include whether the sensing device 101 is configured to display a specified color. For example, if the sensing device 101 is a stylus and is used on the touch device 301 (for example, a large multi-touch device), it is helpful to have multiple sensing devices, each of which is 101 all specify different colors on the multi-touch device. In this situation, the touch signal 307 will convey the biological information of the user using the sensing device 101, the type of the sensing device 101, and the color of the specific sensing 101 configured and displayed on the multi-touch device. Alternatively, a single sensing device may have controls for selecting colors. The currently selected color is reflected in the digital bit stream. By using the control to select different colors, the user causes the information conveyed in the digital bit stream to change to reflect the newly selected color.

Transmit the identification code signal

According to a specific embodiment, the sensing device 101 includes a touch-type transmission unit 104, which transmits the identification code signal 305 generated by the processing unit 103 to the touch-control event during a touch event. Device. In a specific embodiment, the transmission unit 104 transmits the identification code signal 305 in the form of a modulated AC signal containing the user's biological information encoded as a binary code. In a specific embodiment, the transmission unit 104 is disposed on the sensing device 101, so the transmission unit 104 enters a position close enough to the touch device to transmit the identification code signal 305 to the touch device during the touch event. For example, if the sensing device 101 is a stylus, the transmission unit 104 is arranged at or near the tip of the stylus.

In a specific embodiment, the sensing device 101 is configured such that the sensing device sensor 102 continuously detects the user input, the processing unit 103 continuously converts the detected input into the identification code signal 305, and The transmission unit 104 continuously transmits the identification code signal 305. In a specific embodiment, the sensing device 101 is configured not to store any biological data of the user, so when the user is not in contact with the sensing device 101, the sensing device 101 will stop transmitting specific information about the user . Since this function can prevent unauthorized users from stealing the sensing device 101 and deceive the touch device to allow the unauthorized user to access the same functions and information that the user can use, it is advantageous as a security measure .

In addition, this function allows multiple users to use the same sensing device 101 to convey their specific information to the touch device. For example, if two users work together on the same touch device, each user can use the same sensing device 101. Since each time a user uses the sensing device 101 to interact with the touch device, the sensing device 101 conveys specific information unique to the user holding the sensing device at the time, so the touch device can distinguish User input. Therefore, the sensing device 101 is used as an agent that transmits specific information of each user to the touch device when each user interacts with the touch device.

Example real-time authentication operation

The second picture is a flow chart, which depicts the operation of the user on the touch device A program 200 for instant authentication at a time. In block 201, the input conveying the specific information unique to the user is detected using the sensing device sensor 102. As discussed, the specific information may include biological information of the user. In a specific embodiment, the sensing device sensor 102 can be configured to continuously search for the biological information of a specific user, regardless of whether the user is currently interacting with the sensing device. In another specific embodiment, the block 201 can be triggered when the user holds the sensing device 101. For example, if the sensing device is placed on a table, the sensing device 101 does not actively search for the biological information of a specific user. However, once the user picks up the sensing device 101, the sensing device 101 will detect the human hand and the sensing device sensor 102 will actively search for the biological information of the specific user. In a specific embodiment, the sensing device sensor 102 is configured to have an active mode and an inactive mode, where the active mode represents that the sensing device sensors are actively detecting biological information of a specific user (block 201) status.

In block 202, the input is converted into an identification code signal 305 by the processing unit 103. In a specific embodiment, the processing unit is configured to immediately process the input signals from the sensors of the sensing devices without storing the input signals. This ensures that the step of converting the input signal into the identification code signal 305 (block 202) is related to the previous step (block 201). The security advantage of not storing the previously detected biological information of a specific user is to ensure that the identification code signal 305 created by the sensing device 101 is connected to the user currently interacting with the sensing device 101.

For example, if the user A is interacting with the sensing device, then in block 201, the sensing device sensor 102 will detect the biological information of the user A and transmit it to the processing unit 103. In block 202, the processing unit 103 will immediately receive a signal from the sensor 102 of the sensing device to convey biological information to the user A. If user A puts down the sensing device 101, the sensing device sensor 102 will stop detecting biological information for user A, and the processing unit 103 will stop the biological information. The information signal is converted into an identification code signal 305. If an unauthorized user (user B) tries to pick up the sensing device 101 quickly, hoping to use the identification code signal 305 of user A to gain unauthorized access, because once user B and the sensing device 101 interacts, the sensing device sensor 102 (block 201) will detect the user B, and transmit the biological signal associated with the user B to the processing unit 103, so the user B will be rejected.

In block 203, the transmission unit 104 transmits the identification code signal 305 from the sensing device 101 to the touch device when the user interacts with the sensing device 101. For example, when the user A uses the sensing device 101 to interact with the touch device, the transmission unit 104 transmits the identification code signal 305 created by the processing unit 103 to the touch device, just like transmitting from the tip of the sensing device 101 AC signal to the screen of the touch device. In a specific embodiment, the transmission unit 104 is configured to immediately transmit the identification code signal 305 from the processing unit 103 to the tip of the sensing device 101, wherein the tip of the sensing device 101 is the direct interaction between the sensing device 101 and the touch device. section. In other specific embodiments of the sensing device 101, the transmission unit 104 may transmit the identification code signal 305 to the part where the sensing device 101 interacts with the touch device. For example, if the sensing device 101 is a glove, the transmission unit 104 is configured to transmit the identification code signal 305 to the cloth on the fingertips of the glove. In another example, if the sensing device 101 is a ring, the transmission unit 104 is configured to transmit the identification code signal 305 to the crown of the ring, wherein the crown of the ring is interacting with the touch device section.

The signal is received by the touch screen

Referring again to FIG. 3A, in addition to the sensing device 101, a capacitive touch device 301 having a touch sensor is illustrated. The touch device 301 includes a driving line 302 along a specific axis (X axis), which is used to carry current across the specific axis; and a sensing line 303 along another axis (Y axis), It is used to detect the voltage at each specific sensor node such as the target node 304. The capacitance value across a particular sensor node is calculated as the current divided by the rate of change of the voltage across the sensor node. When no external object is close to the sensor node, the capacitance value provides a relative baseline for the capacitance value level when no touch event occurs.

When the user places the sensing device close to the sensor nodes on the capacitive touch device 301, the user uses the sensing device 101 to transmit a touch signal 307 to the capacitive touch device 301. A part of the sensing device 101 is designed with conductive materials, so when the sensing device 101 approaches the capacitive touch device 301, it causes interference with the capacitive field crossing the target node 304. The target node 304 reads the voltage change rate to determine the capacitance value. When the voltage change rate decreases, the target node 304 interprets this reduced voltage change rate as a spike in the capacitance value 315. All the sensor nodes including the target node 304 report their capacitance value data to the sensor controller 308. The sensor controller 308 then determines whether the sharp wave in the capacitance value 315 at the specific node meets the condition of the touch event.

The third B to third E diagrams illustrate spikes in the capacitance value 315 from the touch signal 307 that exceed the set threshold 320 of the touch event. The capacitance threshold 320 is set by the manufacturer to ensure that only the user’s real touch event is interpreted as a touch event, and a slight change in the capacitance value due to noise generated by external factors will not be interpreted as a touch event. Once the sensor controller 308 recognizes the touch event, the sensor controller 308 will interpolate the two-dimensional (2D) coordinates of the touch event.

The transmission unit 104 transmits the identification code signal 305 to the conductive area of the sensing unit 101. Therefore, when the user transmits the touch signal 307 to the touch device 301, the identification code signal 305 is transmitted as a part of the touch signal 307. This is because the identification code signal 305 (which is a modulated AC signal) The touch signal 307 is transmitted to the capacitive field during the transmission period. When the user transmits the touch signal 307 to the target node 304, the modulated AC signal is also received at the target node 304 of the target sensing line 306. The voltage level emitted by the identification code signal 305 is sufficient to clearly distinguish the capacitance value measured when the identification code signal 305 is applied and the capacitance value measured when the identification code signal 305 is not applied. The effect of the identification code signal 305 on the target node 304 is illustrated in the third B to third E diagrams.

The third graphs B to E show the capacitance values measured at the target node 304 during the continuous period of a single touch event. During this period of time, the different bits of the binary code constituting the identification code signal 305 are transmitted to the touch device 301 as part of the touch signal 307. Each graph in the third B to third E represents a single period of the entire touch event, in which the single bit of the binary code is transmitted to the touch device 301. In each graph, the X axis represents the measured nodes on the target sensing line 306, and the Y axis represents the measured capacitance values at each measured node. The third graph in Figure B shows the sharp wave in the capacitance 315 at the target node 304 at the specified time 0, where the time 0 represents the beginning of the touch event. The sharp wave in the capacitance value 315 depicts the capacitance value change when the conductive area of the sensing device 101 approaches the target node 304 of the touch device 301.

Since the identification code signal 305 is a modulated AC signal modulated at a specified frequency, the sensor controller 308 can interpret the different bits of the capacitance value data transmitted as the binary sequence of the identification code signal 305 from the node sensor. As shown in FIG. 3B, the spike in the capacitance value 315 exceeds the set threshold 320 of the touch event and also exceeds the set binary threshold 325. In a specific embodiment, the sensor controller 308 uses the binary threshold 325 to determine the binary code of the input. If the input spike in the capacitance value is a recorded touch event and exceeds the binary threshold 325, the sensor controller 308 assigns a value of 1. If the input spike in the capacitance value is a recorded touch event and does not exceed If the binary threshold 325 is exceeded, the sensor controller 308 assigns a value of 0. The third diagram B depicts the time event 0, where the spike in the capacitance value 315 exceeds the binary critical value 325; therefore, the value 1 is assigned to the time event 0. The third diagram C depicts time event 1, in which the spike in the capacitance value 316 does not exceed the binary critical value 325 and the time event 1 is assigned a value of 0. The third diagram D depicts time event 2 in which the spike in the capacitance value 317 does not exceed the binary critical value 325 and the time event 2 is assigned a value of 0. The third E diagram depicts time event 3, where the spike in capacitance value 318 does not exceed the binary critical value 325 and time event 3 is assigned a value of 1. The sensor controller 308 then compiles the values from consecutive periods into a binary sequence. This decoded binary sequence represents the input originally detected by the sensing device 101, which is then converted into an identification code signal 305, and then transmitted together with the touch signal 307.

Other specific embodiments include, but are not limited to, encoding multiple signals in the identification code signal 305 based on different frequencies. The sensor controller 308 can be programmed to decode the identification code signal 305 and various frequencies in order to compile multiple sets of binary sequences during the time interval of the touch event.

In another specific embodiment, the touch device 301 can distinguish simultaneous touch signals. The sensor controller 308 can then decode the separate and distinct touch signals into separate binary sequences. The sensor controller 308 can then associate those separate binary sequences with their corresponding position signals.

Once the sensor controller 308 decodes the binary sequence, the sensor controller 308 transmits: the interpolated 2D coordinates of the touch event associated with the touch signal 307 and the decoded binary sequence to the job system. In a specific embodiment, the operating system can map the interpolated 2D coordinates of the touch event to the 2D coordinate space of the operating system to determine whether the button has been pressed. The operating system can then use the decoded binary sequence from the user to It is determined whether the user is authorized to perform the action associated with the corresponding pressed button.

The fourth figure is a flowchart, which depicts a method for receiving a touch signal 307, decoding the touch signal 307 to extract and determine the 2D coordinates associated with the touch event, and decode the identification code in the touch signal 307 The signal 305 is used to determine the process 400 representing a binary sequence of specific information from the user. In step 401, the touch device 301 receives a touch signal 307 from the user. The touch signal 307 can trigger a touch event at a specific target node 304 and includes an identification code signal 305. In step 402, the touch signal 307 is then decoded by the sensor controller 308. The decoding step 402 includes determining the 2D coordinates of the touch signal 307 on the touch device 301, where the sharp wave in the capacitance value 315 exceeds the set threshold 320 that meets the condition of the touch event. The output 403 represents the touch event and its interpolated 2D coordinates. In addition, step 402 also decodes the identification code signal 305 from the touch signal 307 to determine the binary sequence representing the specific information from the user. The output 404 represents the specific information communicated from the user in binary format.

Devices other than touch devices

The specific embodiment has previously been discussed with regard to the touch screen as the input device. Other specific embodiments of the input device may include a computing device in addition to the touch device. Specific embodiments of the computing device may include, but are not limited to, a computer with a connected keyboard, mouse, or other accessories designed to convey user interaction.

In the touch screen user interaction description as the background of the touch event, the touch screen receives input from the user to perform a specified action. The sensing device is a keyboard and the input device is a user interaction of the connected computer, which can be defined as a single key on the keyboard. In a specific embodiment, if the sensing device is a keyboard, the sensors will be attached to each key on the keyboard to sense the user’s biological data when the user presses the key. News. Since each keystroke is defined as a user interaction, the keyboard will transmit the biological information sensed by the user to the computer along with the signal of the key pressed. In this way, the computer can decode the specific information from the signal (the user's biological information), and at the same time decode which specific key was pressed. This allows the computer to have the ability to authenticate each key input to determine whether the user has the authority to input the key at that time. For example, if user A uses the keyboard of the sensing device to type an email and user A walks away for a while, unauthorized user B will not be able to type on the same email because Each keystroke input by user B must be authenticated first, and only when user B enters and displays on user A's email on the computer is allowed.

In another example, the sensing device may be a mouse, wherein the input device is a computer connected to the mouse. The sensors on the mouse can be strategically placed to capture the user's biological information when the user manipulates the mouse or presses the mouse buttons. The mouse will then send a position signal plus the user’s sensed biological information when the mouse button is pressed, or the pressed mouse button signal plus the user’s sensed biological information Bio-information. As discussed in the previous example, the input computer device will then decode the user's biometric information along with the mouse position or mouse button information, and determine whether the user has the authority to perform the requested user action.

Other examples include the sensing device as an accessory, such as a stylus and track pad, a head-mounted device with a camera, a joystick, or a game controller. In each of these examples, the accessory is equipped with a biological sensor to sense the user's biological information when the user interaction occurs. The accessory then sends the biometric information to the computer together with the user interaction input for the computer to decode and determine whether the user has the authority to perform the requested action.

Wireless transmission of the identification code signal

According to a specific embodiment, in addition to the touch-sensitive transmission unit 104, the sensing device 101 also includes a wireless transmission unit 105 for the purpose of transmitting additional information to the touch-sensitive device. In a specific embodiment, the wireless transmission unit 105 uses a wireless protocol such as Bluetooth to directly transmit data to the touch device. The wireless transmission unit 105 matched with the transmission unit 104 is used to transmit specific information about the user in the form of encrypted data. Encrypted data provides a higher level of security for the transmitted data, in which only authorized recipients with the decryption key can decrypt and use the encrypted data. In a specific embodiment, the wireless transmission unit 105 transmits encrypted biological information to the touch device. The transmission unit 104 is used to transmit the decryption key to the touch device, so the touch device can read the encrypted data transmitted through the wireless transmission unit 105. In another embodiment, the transmission unit 104 is used to transmit the encrypted biological information to the touch device, and the wireless transmission unit 105 is used to transmit the decryption key to the touch device.

In another specific embodiment, the wireless transmission unit 105 can be used to transmit other information specific to the user, such as user preferences, to the touch device. In a specific embodiment, the user can configure the sensing device 101 to wirelessly transmit information about the user experience on the configurable touch device 301 via the wireless transmission unit 105. For example, a user who uses a brand new touch device can use the sensing device 101 to wirelessly transmit the user's preferences to the brand new touch device when the user interacts with the touch device for the first time.

In another specific embodiment, the wireless transmission unit 105 may be used to transmit designated information of the sensing device 101 to the touch device. For example, if the sensing device 101 is configured to represent a designated color such as red, when the sensing device 101 interacts with the touch device, the sensing device 101 uses the wireless transmission unit 105 to transmit the configured color to the Touch device. In this user As the final result when interacting with the touch device, the sensing device 101 transmits the biometric information of the specific user to the touch device via the transmission unit 104, and at the same time transmits the configured "red" color to the touch device via the wireless transmission unit 105 Touch device, which causes the user to display the input on the touch device in red.

Simultaneous initiation and certification

Using the techniques described in the text, the same input for the specified operation is used to determine whether the operation is being performed by an authorized user. For example, suppose that user X is authorized to perform operations A and B, user Y is authorized to perform operation C, and user Z is not authorized to perform any operations. In response to detecting a touch event, the touch device determines what operation the touch event is configured to trigger (the touch event triggers different operations based on the location of the touch event and the background in which the touch event occurs). In addition to determining the operation, the touch device determines the identity of the user by mapping the biological information conveyed in the touch event to the user ID. The user ID is then checked against the list of user IDs authorized to perform the operation. If the user ID corresponding to the biological information matches the user ID authorized to perform the operation, the operation is executed. For example, if the touch event triggers operation C and the biological information belongs to user Y, operation C is executed. On the other hand, if the biometric information does not comply with the authorized user for the operation, the operation is not performed. Instead, an error message can be generated. For example, if operation C is attempted by user X through a touch event, the touch device may generate an error message instead of performing operation C.

Since the touch event signal provides the dual purpose of specifying the operation and identifying the user who is trying to perform the operation, users who are authorized to perform different actions can only pass the touch device between them instead of a user Log out to allow another user to log in. For example, user X can perform operation A (for user Y is not authorized to perform), and then transfer the device To user Y. Without any login or logout, user Y can then perform operation C (because user X is not authorized to perform it). When sharing the touch device with user Y, user X does not have to worry that user Y will perform operation A, because any attempt by user Y to do so will be rejected.

Simultaneous users

The techniques described in the article can be used to detect which touch events simultaneously input on the same touch device are performed by which users. For example, in a game environment, user X and user Y can simultaneously perform touch events on the same touch screen. For each touch event, the touch device can determine the user who performed the touch event by decoding the biological information from the touch event signal. Depending on who makes which touches, the game (or any other software) can be played differently.

Another example of identifying which touch events belong to which user is in a collaborative environment, where multiple users can interact with a large multi-touch device at the same time. For example, suppose that a large multi-touch device is used as a whiteboard during a meeting. Multiple sensing devices can be provided to the participants in the conference, and each sensing device is a stylus that is programmed to be used as a different colored pen when applied to the whiteboard application. If the team leader user X picks up the "black" sensor device to draw the system layout of the project, from these touch events, the multi-touch device determines that user X is drawing with the "black" stylus. The whiteboard application can be edited programmatically, so input from user X and other team leaders is not allowed to be edited by others. When the junior member user Y picks up the same "black" sensing device and tries to modify the drawing of user X, the whiteboard application will determine that the modification comes from an unauthorized user and will reject the modification. Another example of using multiple sensing devices on the whiteboard application allows certain users to select certain colors Sexual editing (different sensing devices). In addition, the whiteboard application (or any other software) can be programmed to selectively display the input of some users while removing the input of other users. This implementation provides numerous advantages in collaborative meetings and whiteboard assemblies.

Specific user operation

It does not only determine whether the operation is authorized, the biological information conveyed in the touch event signal may affect how the requested operation is performed. For example, if a touch event triggers the "read email" operation, the email messages displayed to the user may be based on the biological information conveyed by the signal that triggered the touch event. For example, if the biometric information matches the user X, the touch event causes the display of an email message from the user X's inbox. On the other hand, if the biometric information matches the user Y, the touch event causes the display of the email message from the user Y's inbox.

User-specific operations in the application can be customized and enhanced based on the user associated with the biological information conveyed in the touch event signal. As an example, a web browser application does not require any user-specific information to operate, but can be customized and enhanced based on the user's browsing history, bookmarks, and other personalizations. For example, when user X triggers a touch event for starting a web browser, the operating system can activate or resume the web browsing state in the background of user X. This background may include, but is not limited to, loading respective small text files (cookies), restoring the previous browsing state to the remote site, and loading the user's bookmark folder. If user Y triggers a touch event to trigger a new web page, the web browser can activate a new web page with user Y's cookies, previous browsing status history, and bookmark folder.

As another example, the user can use the touch screen to select controls to purchase items. The touch event signal can be decoded to determine the identity of the user who selected the button. If it should be If the user has registered and has sufficient funds available, the purchase can be executed immediately. If the user has registered but does not have enough funds, a prompt indicating the payment method can be presented to the user. If the user is not registered or cannot be identified at all, a prompt to create an account can be presented to the user.

In yet another example, one user can conduct transactions on behalf of another user. For example, if user X is using a bookstore application, the bookstore can display recommended products based on user X's previous purchase records. User X can then touch the "buy" button of a specific book, which will charge user X's account. If user X does not have enough funds, the bookstore application will not be able to complete the purchase. However, during the same browsing state, the user Y can touch the "purchase" button of the user X. The touch event performed by the user Y will notify the bookstore application, and the user Y purchases the book during the browsing state of the user X. The bookstore application can prompt user Y in a dialog box, asking whether user Y wants to purchase the book for user X. If user Y agrees, the bookstore application will deliver the book to the shared library, and the purchase will be completed by charging user Y's account.

In the example of sharing data between users, one user temporarily allows another user to access it. For example, if user X uses a photo library application to access the photo album, the photo library application will only allow user X to access the photo album belonging to user X. If user X wants to select an album belonging to user Y, the photo application will refuse to allow user X. However, if user Y wants to share one of his photo albums with user X, user Y can do so temporarily. Sharing can be implemented, where both user X and user Y touch the desired album at the same time. The photo app will then recognize the touch events from both user X and user Y, and since the two users touched the album they want to share, the photo app will recognize user Y ( Authorized user) is the “sharer” and user X (unauthorized user) is the “sharer” Sharee". The photo application will then temporarily allow access by the sharer, in this case user X. Alternative embodiments may implement sharing based on different combinations of simultaneous or continuous input from the user. Sharing access can be revoked by exiting the application or based on some other time-out function.

Options to protect the displayed content

The user-restricted access content based on triggering touch events can also be implemented on non-user-induced output, such as input notifications displayed on touch devices. The sensitivity of input events is closely related to the classification of access permissions. The touch device displays the input notification from the application (not the foreground application) in the form of a pop-up notification. A touch device that uses biological touch event decoding for program editing can be configured to display limited information in a pop-up notification based on the access privileges from the application that caused the pop-up notification.

For example, if the application that triggers the pop-up notification does not have any user-based restrictions or personalization, the pop-up notification will not be restricted based on who the target user of the notification is. If the notification originates from an application that does have user-based restrictions, the message can convey the limited presentation of the notification's existence. Access to certain pop-up notifications can be based on which user is triggering the touch event in response to the notification. For example, notifications can pop up on the touch screen indicating "Yahoo! New messages in Yahoo Messenger". This type of notification can be programmed to not display the annotations themselves, because the annotations can be privileged for the intended recipient. Only the authorized user can trigger touch events to reveal the Yahoo! Such detailed information notified by Qimo in real time. In addition, if multiple notifications exist for multiple users, when the user X triggers a touch event to reveal the detailed information of the notifications, only the notification for the user X is expected to be revealed. The detailed information of the notification for user Y can only be accessed by user Y.

After the notified detailed information has been revealed by the authorized user, the detailed information needs to be removed from the touch screen, so unauthorized users may not see it. In a specific embodiment, the detailed information of the display notification is only displayed during the duration of the touch event. For example, if a notification for a message of user X pops up, user X can display the detailed information of the notification by triggering a touch event, and those detailed information can only be sensed by user X It can be seen when the device touches the touch device. Once the user removes the sensing device, thereby ending the touch event, the detailed information of the notification will disappear from the touch device.

Another example of protecting the displayed content is to prompt the user with a highly confidential notification. Highly confidential notification is information that requires the user to take special actions to avoid others around to ensure safety. For example, the user may need to cover the touch device with his other hand, so he is the only person who can see the screen. In a specific embodiment, the notification system may require an authorized user to trigger a touch event with a sensing device to convey his biological information, and trigger a second touch event with the base of the other hand, where the touch device uses The shape analysis of the second touch event is analyzed to determine that the hand of the authorized user is covering the display of the touch device to prevent others from seeing the screen.

Explicit login for custom parts

The touch device interface can be customized based on the biometric information of the user. For example, the touch device initially displays a start screen or a lock screen that must be unlocked before the user can use the device. Unlocking the device may require a touch event from the user. During the touch event, the touch device can be customized to have a custom background and a home screen of the application about the user who unlocked the touch device. After unlocking the device, the second user can have other Different home screens with custom applications and backgrounds.

Similarly, logging in or opening a highly secure document such as a commercial transaction may require the presence of two authorized users. In this case, if the touch device simultaneously receives biometric information during the touch events from both the user X and the user Y, the touch device can only display the high security document. This can be advantageous for securely accessing information that requires authorization from multiple users.

Operating system-based authorization

According to a specific embodiment, the techniques described in the text can be used in a manner related to the operating system in the authentication procedure. For example, the operating system can convey the event type, the screen location, and the identity of the user performing the event to the application, rather than simply conveying that the touch event occurred at a specific location on the touch screen. Based on this additional information, the application can authenticate and/or trigger specific user actions.

Hardware overview

According to a specific embodiment, the techniques described in the text are implemented by one or more dedicated computing devices. The dedicated computing devices may be hard-wired to perform these techniques, or may include digital electronic devices, such as one or more application-specific integrated circuits (ASICs) that are continuously programmed to perform these techniques. ) Or Field programmable gate arrays (FPGAs), or may include one or more general-purpose hardware processors, which are programmed to perform the These skills of program instructions. Such dedicated computing devices can also combine custom hard-wired logic, ASICs, FPGAs, and custom program editing to achieve these skills. These dedicated computing devices can be desktop computer systems, portable computer systems, handheld devices, network devices, or any other devices that incorporate hard-wired and/or program logic to implement these techniques.

For example, the fifth figure is a block diagram illustrating the computer system 500 on which the specific embodiment of the present invention can be implemented. The computer system 500 includes a bus 502 or other communication mechanism for communicating information; and a hardware processor 504 coupled to the bus 502 for processing information. The hardware processor 504 may be, for example, a general-purpose microprocessor.

In addition, the computer system 500 also includes a main memory 506, such as a random access memory (RAM) or others coupled to the bus 502 for storing information and instructions to be executed by the processor 504 Dynamic storage device. The main memory 506 can also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processor 504. When such instructions are stored in a non-transitory storage medium accessible to the processor 504, the computer system 500 becomes a dedicated machine customized to perform the operations specified in the instructions.

The computer system 500 further includes a read only memory (ROM) 508 or other static storage device coupled to the bus 502 for storing static information and instructions for the processor 504. The storage device 510 (for example, a magnetic disk or an optical disk) is provided and coupled to the bus 502 for storing information and commands.

The computer system 500 may be coupled to the display 512 via the bus 502, such as a cathode ray tube (CRT) for displaying information to a computer user. An input device 514 including alphanumeric characters and other keys is coupled to the bus 502 for communicating information and command selection to the processor 504. Another type of user input device is a cursor control 516, such as a mouse, trackball, or cursor direction key for communicating direction information and command selection to the processor 504 and for controlling the movement of the cursor on the display 512. Such input devices usually have two degrees of freedom on two axes that allow the device to specify a position on a plane: a first axis (for example: x) and a second axis (for example: y).

The computer system 500 can use custom hard-wired logic, one or more ASICs or FPGAs, firmware, and/or program logic that are combined with the computer system to cause or program the computer system 500 to become a dedicated machine. skill. According to a specific embodiment, the techniques described herein are performed by the computer system 500 in response to the processor 504 executing one or more sequences of one or more instructions contained in the main memory 506. Such instructions can be read into the main memory 506 from another storage medium such as the storage device 510. The execution of the sequences of instructions contained in the main memory 506 causes the processor 504 to perform the program steps described in the text. In alternative embodiments, hard-wired circuits can be used in place of or in combination with software instructions.

The term "storage media" as used in the text refers to any non-transitory media that stores data and/or instructions that cause the machine to operate in a specified manner. Such storage media may include non-volatile media and/or volatile media. Non-volatile media includes, for example, optical disks or magnetic disks such as the storage device 510. Volatile media include dynamic memory such as main memory 506. Common forms of storage media include, for example, a floppy disk, a flexible disk, a hard disk, a solid state drive, a magnetic tape, or any other magnetic data storage medium, a CD-ROM, Any other optical data storage media, any physical media with a hole pattern, a RAM, a programmable read-only memory (PROM), and an erasable programmable read-only memory (EPROM), a flash-erasable Except programmable read-only memory (FLASH-EPROM), non-volatile RAM (NVRAM), any other memory chips or storage cartridges.

The storage medium is different from the transmission medium but can be used in combination with it. Transmission media participates in transferring information between storage media. For example, the transmission medium includes coaxial cables, copper wires, and optical fibers, which include these wires including the bus bar 502. Transmission media can also take the form of sound waves or light waves, such as those generated during radio wave and infrared data communication.

Various forms of media may involve carrying one or more sequences of one or more instructions to the processor 504 for execution. For example, these commands can initially be carried on the disk or solid-state drive of the remote computer. The remote computer can load the commands into its dynamic memory, and use a modem to send the commands through the telephone line. The regional modem of the computer system 500 can receive the data on the telephone line, and use an infrared transmitter to convert the data into an infrared signal. The infrared detector can receive the data carried in the infrared signal, and an appropriate circuit can put the data on the bus 502. The bus 502 carries the data to the main memory 506, and then the processor 504 retrieves and executes the instructions from the main memory 506. The commands received by the main memory 506 can be stored on the storage device 510 as needed before or after being executed by the processor 504.

In addition, the computer system 500 also includes a communication interface 518, and the communication interface 518 is coupled to the bus 502. The communication interface 518 provides two-way data communication coupled to the network link 520 connected to the local network 522. For example, the communication interface 518 may be an integrated services digital network (ISDN) card, a cable modem, a satellite modem, or a modem that provides data communication connection to a corresponding type of telephone line. As another example, the communication interface 518 may be a LAN card that provides data communication connection to a compatible local area network (LAN). In addition, wireless links can also be implemented. In any such implementation, the communication interface 518 transmits and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information.

The network link 520 usually provides data communication to other data devices through one or more networks. For example, the network link 520 can provide a data device connected to the host computer 524 or to an Internet Service Provider (ISP) 526 through the local area network 522. ISP 526 then provides data communication services through a global packet data communication network now commonly referred to as the "Internet" 528. Both the local area network 522 and the Internet 528 use bearer data The electrical, electromagnetic, or optical signal of the bit data stream. The signals carrying the digital data to and from the computer system 500, through the various networks, and the signals on the network link 520 through the communication interface 518 are exemplary forms of transmission media.

The computer system 500 can transmit messages and receive data, including program codes, through the network(s), network link 520, and communication interface 518. In this Internet example, the server 530 can transmit a request code through the Internet 528, the ISP 526, the local area network 522, and the communication interface 518 to provide an application program.

The received code may be executed by the processor 504 when it is received, and/or stored in the storage device 510 or other non-volatile storage for later execution.

Benefits of some specific embodiments

In a specific embodiment, the solution described in the text can improve the security on a device that uses a touch screen for input. Current touch devices usually adopt security measures to selectively unlock the device or perform designated actions that require authentication. Touch input will not be directly authenticated. By incorporating the authentication step into the touch input itself, the information security risk of unauthorized touch event signals after the device has been unlocked or the user has been authenticated can be eliminated. Furthermore, since each touch input is now authenticated against a list of authorized users, this solution can enhance the security level of each touch input.

In the foregoing specification, specific embodiments of the present invention have been described with reference to numerous specific details that may vary due to different implementations. Accordingly, this specification and the accompanying drawings should be viewed in an illustrative rather than restrictive sense. The unique and exclusive indication of the scope of the invention, as well as the scope of the invention anticipated by the applicant, is to issue a designated form including any subsequent corrections in the scope of such patent applications, a collection of the scope of patent applications issued from this application Zigzag and equivalent category.

200: program

201-203: block

Claims (21)

A method for transmitting information about a user to an input device, the method comprising: when the user interacts with the input device using a sensing device to perform a predetermined operation, the following steps are performed: using the sensing device One or more sensors in the device detect the user’s biological information; a processing unit in the sensing device converts the user’s biological information into a first signal; The first signal encodes specific information about the user; and a transmitter in the sensing device transmits the first signal encoding specific information about the user to the input device through direct contact, so that the The specific information can be used to authenticate the predetermined operation of the user on the input device, where the specific information includes a user-specific display information sent to the input device before authenticating the user, and an attribute of one of the touch device sensors Is greater than a critical value, and the attributes of the other one or more touch device sensors are less than the critical value, wherein when the sensing device is in direct contact with the input device, the attribute reflects that the first signal is being transmitted . Such as the method of the first item in the scope of patent application, wherein the input device is a touch device. For example, the method of claim 2 further includes: when a second user uses the sensing device to interact with the touch device, the following steps are performed: The one or more sensors are used to detect the biological information of the second user; the processing unit converts the biological information of the second user into a first conveying specific information about the second user Three signals; and through the transmitter, transmitting the third signal to the touch device. Such as the method of the first item in the scope of patent application, wherein the sensing device is a handheld device. Such as the method of claim 1, wherein the step of transmitting the first signal includes transmitting authentication information about the user to the input device. For example, the method of claim 1 further includes wirelessly transmitting a second signal to the input device by a second transmitter in the sensing device, wherein the second signal is the first signal transmitted to the input device using A coded signal decoded by the signal. Such as the method of claim 1, wherein the step of converting the biological information of the user into a first signal further includes embedding the designated information of the sensing device into the first signal. For example, the method according to item 7 of the scope of patent application, wherein the designated information of the sensing device is configured as information related to how the first signal is interpreted. A method for transmitting information about a user to an input device includes: receiving, by a set of one or more sensors operably coupled to an input device, a signal transmitted through direct contact with a sensor device, The sensor device is being used by a user to create an interaction event on the input device to perform a predetermined operation, wherein the signal: encodes specific information about the user so that the specific information can be used to authenticate the The predetermined operation of the user on the input device, wherein the specific information includes a user-specific display information sent to the input device before authenticating the user, and wherein The processing unit encodes the specific information in the signal of the input device, and obtains the specific information by sensing the biological information of the user by the sensing device; and decodes the signal on the input device to determine the interaction A position of an action event and specific information about the user, where an attribute of the touch device sensor corresponding to the position is greater than a threshold, and the other one or more touch device sensors The attribute of is less than the critical value, wherein when the sensing device is in direct contact with the input device, the attribute reflects that the signal is being transmitted. Such as the method of item 9 of the scope of patent application, the input device further includes determining the identity of the user based on the specific information. For example, the method of claim 10 further includes: displaying user-specific information on a display associated with the input device according to the user's identity determined based on the specific information. Such as the method of claim 11, wherein the user-specific information is displayed only within a period of time based on the duration of the touch event. Such as the method of claim 9 in which the input device is a touch device. Such as the method of claim 13 in which: the touch event recognizes a specific operation on the touch device; and based on the identity of the user, it is determined whether to perform the specific operation recognized by the touch event. For example, in the method of item 9 of the scope of patent application, the specific information includes a time stamp, and the time stamp is related to how long the specific information is valid to determine the identity of the user. Such as the method of item 9 of the scope of patent application, where the specific information includes sensor device designation Information about a specific label associated with the touch event is displayed together with the specific label on a display associated with the input device. For example, the method of claim 9 further includes: receiving from the set of one or more sensors from being used by a second user to establish a first connection between the sensor device and the input device. A second signal transmitted by the sensor device of two touch events; wherein the second signal conveys specific information about the second user based on the biological information sensed by the sensor device from the second user ; Decode the second signal on the input device to determine the following two: a position of the second touch event, and specific information about the second user; and based on the signal from the interaction events and Specific information decoded by the second signal to track interaction events associated with the user and the second user. A sensing device for transmitting information about a user to an input device includes: when the user uses a sensing device to interact with the input device to perform a predetermined operation, one or more of the sensing device A sensor configured to detect the biological information of the user; a processing unit in the sensing device is configured to convert the biological information of the user into a first signal, and in the first signal Encoding specific information about the user; and a transmitter in the sensing device configured to transmit a first signal encoding specific information about the user to the input device through direct contact, so that the specific information can be used recognize To verify the predetermined operation of the user on the input device, wherein the specific information includes a user-specific display information sent to the input device before authenticating the user, and an attribute of one of the touch device sensors is greater than a threshold , And the attribute of the other one or more touch device sensors is less than the threshold, wherein when the sensing device is in direct contact with the input device, the attribute reflects that the first signal is being transmitted. Such as the 18th sensing device in the scope of patent application, wherein the input device is a touch device. For example, the sensing device of item 18 in the scope of patent application, wherein the sensing device is a handheld device. For example, the sensing device of claim 18 further includes a second transmitter configured to wirelessly transmit a second signal, wherein the second signal is a decoded using the first signal transmitted to the input device Encoded signal.

Images