BR102022002419A2 - DEVICES AND ONE OR MORE COMPUTER READABLE NON-TRANSITORY MEDIA - Google Patents

Abstract

A device includes a display and one or more processors that cause a speaker associated with the device to serially recite a plurality of auditory cues. Each of the plurality of auditory cues corresponds to one of a set of characters, and each recitation of the plurality of auditory cues occurs at a predetermined time period. The one or more processors also receive display touch input from a user during recitation of the plurality of auditory cues, determine a character from the set of characters that corresponds to the touch input based on the predetermined amount of time, and store the character as a value in a string of user-specific information.

Description

DEVICES AND ONE OR MORE COMPUTER READABLE NON-TRANSITORY MEDIA

[001] Personal Identification Numbers (PIN) are widely used for a variety of applications that require identity verification. As the use of touch surfaces has increased, it has become commonplace to provide interfaces for PIN entry that make use of touch technology. Touch surfaces, such as touchscreen devices, often rely on a user's visual sensing ability to interact with a touchscreen device's display. Problems arise for visually impaired users using a touch screen device as they cannot visually feel the display.

SUMMARY

[002] In accordance with some aspects of the present disclosure, a device is disclosed. The device includes a display and one or more non-transient computer-readable media comprising computer-readable instructions stored therein, which when executed by one or more processors of the device, cause the one or more processors to cause a loudspeaker. associated with the device serially recite a plurality of auditory cues, such that each of the plurality of auditory cues corresponds to one of a set of characters, and such that each recitation of the plurality of auditory cues occurs in a predetermined period of time, receive a display touch input from a user during recitation of the plurality of auditory cues, determine a character from the set of characters that corresponds to the touch input based on the predetermined amount of time, and store the character as a value into a string of user-specific information.

[003] In accordance with some aspects of the present disclosure, one or more non-transient computer-readable media having computer-readable instructions stored thereon are disclosed. The computer-readable instructions when executed by one or more processors of a device cause the one or more processors to cause a speaker associated with the device to serially recite a plurality of auditory cues such that each of the plurality of auditory cues matches one of a set of characters, and such that each recitation of the plurality of auditory cues occurs within a predetermined period of time, receives touch input on the display from a user during recitation of the plurality of auditory cues, determines a character from the character set that corresponds to touch input based on the predetermined amount of time and store the character as a value in a string of user-specific information.

[004] In accordance with some aspects of the present disclosure, a device is disclosed. The device includes a touch screen display, a speaker, and one or more computer readable non-transient media comprising computer readable instructions stored therein, which when executed by one or more processors of the touch device, cause that the one or more processors cause a speaker associated with the device to serially recite a plurality of auditory cues, such that each of the plurality of auditory cues corresponds to one of a set of characters, and wherein each recitation of the plurality of auditory cues corresponds to one of a set of characters. auditory cues occur within a predetermined amount of time, receive touch input on the display from a user during recitation of the plurality of auditory cues, determine a character from the set of characters that corresponds to the touch input based on the time period of predetermined time and store the character as a value in a string of user-specific information.

[005] The foregoing summary is illustrative only and is not intended to be limiting in any way. In addition to the illustrative aspects, modalities and resources described above, other aspects, modalities and resources will become apparent by reference to the following drawings and the detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

[006] FIG. 1 is an exemplary block diagram of a point of sale device, in accordance with some embodiments of the present disclosure.

[007] FIG. 2 is an exemplary block diagram showing further details of the point of sale device of FIG. 1, in accordance with some embodiments of the present disclosure.

[008] FIG. 3 is an exemplary flowchart outlining operations for a sonic tapping mechanism to receive data input to the point of sale device by a visually impaired user, in accordance with some embodiments of the present disclosure.

[009] FIG. 4 is another exemplary flowchart outlining certain operations of FIG. 3 in more detail, in accordance with some embodiments of the present disclosure.

[010] FIG. 5 is an exemplary sequence diagram showing interactions between an application processor and a secure processor of the point of sale device of FIGS. 1 and 2 in implementing the operations of FIGS. 3 and 4, in accordance with some embodiments of the present disclosure.

[011] FIG. 6 is an exemplary sequence diagram showing additional interactions between the application processor and the secure processor, in accordance with some embodiments of the present disclosure.

[012] FIG. 7 is an exemplary display of the point of sale device of FIGS. 1 and 2, in accordance with some embodiments of the present disclosure.

[013] Other and foregoing features of the present disclosure will become apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. Understanding that these drawings only represent various embodiments according to the disclosure and, therefore, should not be considered limiting its scope, the disclosure will be described with specificity and additional details through the use of the attached drawings.

DETAILED DESCRIPTION

[014] In the following detailed description, reference is made to the attached drawings, which form a part of the same. In drawings, like symbols normally identify like components unless the context indicates otherwise. The illustrative embodiments described in the detailed description, drawings and claims are not meant to be limiting. Other modalities may be used, and other alterations may be made, without departing from the spirit or scope of the material presented here. It will be readily understood that aspects of the present disclosure, as generally described in the present invention, and illustrated in the figures, can be arranged, substituted, combined and designed in a wide variety of different configurations, all of which are explicitly contemplated and form part of this disclosure. .

[015] The present disclosure is generally directed to touch screen devices for use by users who are visually impaired (eg, blind or have low vision). Visually impaired users often rely on tactile and auditory sensory skills to discern the information being presented. For example, a visually impaired user can identify the locations of keys on a keyboard and interpret the characters or commands on those keys if the keyboard layout is taught, or previously known. This is not possible with a touch screen device that has a smooth contact display surface. In some cases, tactile guides can be added to a touchscreen device's display surface. However, this approach does not solve the multitude of problems that visually impaired users face. For example, inadvertent and unwanted touch inputs can register on the display surface as tactile guides are used to find touch points of interest. Also, visually impaired users may not be able to verify their touch inputs using just the touch point signifiers. In some cases, an audio device can be used with the touchscreen display to repeat user input aloud. This approach may inappropriately share private information with others nearby if in a public setting.

[016] A point of sale (POS) device is an example of a touch screen device where a user may be required to interact with a POS device display surface without assistance from another party to protect privacy and user's private information. For example, in some cases, a user may be required to enter a PIN using the display surface (for example, to provide payment information, access a user account, or for other purposes). Increasingly, POS devices have smooth display surfaces (eg glass, plastic, resin surfaces, etc.) that are not very conducive to use by visually impaired users. A visually impaired user (or visually impaired user) may not have difficulty using such a POS device to enter their PIN. However, a visually impaired user may struggle to enter their PIN on the same POS device without any cues or tactile surfaces to help the visually impaired user. In these situations, the visually impaired user may need to provide their PIN to an employee/manager at the POS device, which undesirably provides a disadvantageous security situation, as both the employee/manager and potential spies may be able to obtain the PIN.

[017] Thus, the present disclosure provides a robust and secure mechanism to allow visually impaired users to enter data (e.g., PIN) on a touch screen device, such as a POS device, while maintaining the advantages offered to users with visual impairment, thus providing a mechanism that can be used by visually impaired users and visually impaired users. In addition, this disclosure protects the privacy of visually impaired users, as well as protecting users' confidential information from eavesdroppers.

[018] Specifically, the present disclosure provides a touchscreen device (e.g., a POS device) that uses auditory cues or cues to allow a visually impaired user to enter their PIN. In some embodiments, the auditory cues may be recited using a speaker associated with the touchscreen device (eg, POS device). Auditory cues can be configured according to the PIN configuration. For example, if the PIN includes a series of numbers, the auditory cues may include numbers (eg, 0-9). Auditory cues can be recited in one or more cycles depending on the number of values in the PIN. For example, if the PIN is 4 digits, the auditory cues can be recited in 4 cycles, with each cycle reciting the numbers 0-9. On each cycle, when the visually impaired user hears the auditory cue corresponding to the PIN value, the visually impaired user can provide touch input (eg tapping) on the touchscreen device to enter the PIN.

[019] For example, if the PIN is 1234, the first cycle can recite the numbers 0-9. When the visually impaired user hears the number 1 being recited, the visually impaired user can tap the touchscreen device to input the number 1. In a second cycle, the numbers 0-9 can be recited again. When the visually impaired user hears the number 2 being recited, the visually impaired user can tap the touchscreen device to enter the number 2. The third and fourth cycles can be repeated similarly to enter the numbers 3 and 4 of PIN. In some embodiments, within each cycle, auditory cues may continue to be recited after the visually impaired user has provided touch input. For example, after the user taps 1 on the touchscreen device, the numbers 2-9 can still be recited to avoid a situation where someone hears the audio signals and detects the PIN that way.

[020] Thus, this sonic tapping mechanism allows a visually impaired user to easily and conveniently enter user-specific confidential data (eg PIN) without needing assistance or worrying about privacy or eavesdropping.

[021] It should be understood that, although this disclosure is discussed in the context of a POS device, this disclosure may be applicable to any type of touch screen device or human-machine interface that allows visually impaired users to enter data . Furthermore, although the present disclosure has been discussed in the context of entering a PIN, in other embodiments, the present disclosure may be used to enter other types of data, such as payment information, address, zip code, etc. In general, the present disclosure can be used to enter any numerical data, alphabetic data, special characters and/or alphanumeric characters.

[022] Referring now to FIG. 1, there is shown an exemplary block diagram of a POS device 100, in accordance with some embodiments of the present disclosure. In some embodiments, the POS device 100 can be used to facilitate the purchase of goods and/or services (eg, at a retail store, at a restaurant, at a gas station, etc.). The POS 100 device can be configured to receive information from users and present information to users. In some embodiments, the POS device 100 can be configured to present information to users in a variety of ways. For example, in some embodiments, a speaker 105 associated with the POS device can provide auditory cues, voice commands, or cues to users (e.g., telling a user how to enter a PIN, facilitating PIN entry, guiding the user through the payment process, etc.). In some embodiments, a display 110 can be a graphical user interface configured to provide visual information (eg, to provide prompts to enter data, etc.) to a user. The POS 100 device can also be configured to present outputs/information to entities other than users. For example, the POS 100 device can be configured to present information to external systems such as printers, speakers, etc. Therefore, although not shown, the POS device 100 can be associated with a variety of hardware, software, firmware or combinations thereof, which allow a variety of information to be output from the POS device.

[023] Likewise, the POS device 100 can be configured to receive information from users in a variety of ways. For example, in some embodiments, the POS device 100 may be associated with a microphone 115 to allow a user to provide auditory cues, commands or indicative sounds to the POS device (e.g., to speak into the microphone to enter information such as a PIN). In some embodiments, display 110 may include a touch sensor 120 to receive information from the user (eg, by tapping a designated area of display 110 to enter information such as a PIN). By virtue of having the touch sensor 120, the display 110 can be considered a touch screen display. Thus, the POS device 100 can provide a touch screen interface and the display 110 can be configured to display information to users, as well as serve as a tool for users to enter information into the POS device 100. The POS device 100 can include other mechanisms for receiving input from users. For example, the POS device 100 can be configured to receive user input using input technologies including, but not limited to, a keyboard, stylus, mouse, trackball, keypad, other types of voice recognition , motion recognition, remote controllers, gesture detectors such as gesture sensors or cameras, input ports, one or more buttons, dials, joysticks, etc. that allow an outside source, such as the user, to enter information into the POS device.

[024] In general, the POS device 100 can be associated with any type of hardware, software and/or firmware component that allows the POS device to perform the functions described in the present invention. Thus, the POS device 100, and particularly the display 110, can be configured in a variety of ways for use by visually impaired and visually impaired users. An example of a display is shown in FIG. 7 below. In some embodiments, the POS device 100 can be associated with other devices or systems. For example, in some embodiments, the POS device 100 can include or be associated with a business device that can be operated by a business employee or administrator. In some embodiments, the POS device 100 can receive instructions from such a commercial device. Based on instructions received, the POS device 100 can issue information to a user or request information from the user. In some embodiments, based on instructions received, the POS device 100 can change an operating mode. For example, in some embodiments, the POS device 100 can switch between operating in a first mode configured for visually impaired users and operating in a second mode configured for visually impaired users. In some embodiments, changing the mode of operation may be facilitated in a variety of ways. For example, in some embodiments, changing the mode of operation may be facilitated by selecting the appropriate option from the commercial device and/or the POS 100 device. In other embodiments, changing the mode of operation may be facilitated by plugging into a headset on the POS device 100 and/or the commercial device to activate the second mode and allow a visually impaired user to enter the PIN using the display 110. Upon unplugging the headset, the operating mode may switch back to the first mode for visually impaired users. In other embodiments, the change in mode of operation can be facilitated in other ways.

[025] The display 110 of the POS device 100 can form the front end of the POS device. A secure processor 125, an application processor 130 and a touch controller 135 can form a backend of the POS device 100. The secure processor 125, the application processor 130 and the touch controller 135 can be configured to process data received from of a user, present data to the user, and otherwise a transaction. Although the secure processor 125, application processor 130, and touch controller 135 are shown as separate components, in some embodiments, one or more of these components may be integrated into a single component, and the single component may perform the operations of the components. individual that are combined. In some embodiments, one or more of the 125 secure processor, the 130 application processor, and the 135 touch controller may be located on one or more printed circuit boards of the POS device 100. In some embodiments, each of the 135 secure processor 125, application processor 130, and touch controller 135 may be configured as software, hardware, firmware, or a combination thereof.

[026] Although not shown, each of the secure processor 125, the application processor 130 and the touch controller 135 can be associated with a processor (for example, a central processing unit). The processor can be implemented in hardware, firmware, software or any combination thereof. The processor can be configured to execute instructions to implement the various functions of the respective component (for example, the secure processor, the application processor and the touch controller). “Execute an instruction” means that the processor can perform the operations called by that instruction. The processor can retrieve a set of instructions from memory for execution. For example, in some embodiments, the processor may retrieve instructions from a permanent memory device such as a read-only memory (ROM) device and copy the instructions in an executable form to a temporary memory device that is generally some form of random access memory (RAM). The ROM and/or RAM may be part of POS device memory 100 and/or individually allocated memory for the respective one of the secure processor 125, the application processor 130, and the touch controller 135. In some embodiments, the processor can be configured to execute instructions without first copying those instructions to RAM. The processor may be a special purpose computer and include logic circuits, hardware circuits, etc. to carry out the instructions. The processor may include a single stand-alone processor, or a plurality of processors using the same or different processing technology. Instructions can be written using one or more programming languages, scripting language, assembly language, etc.

[027] One or more of the secure processor 125, the application processor 130 and the touch controller 135 may also include a memory. For example, secure processor 125 is shown to include secure memory 140. Although application processor 130 and touch controller 135 are not shown to include memory, one or both of these components may include memory. The memory may be any of a variety of volatile and/or non-volatile memories that may be deemed suitable for use with the POS device 100. In some embodiments, the memory may be configured to store the instructions that are used by the component's processor. underlying (e.g., secure processor 125, application processor 130, and touch controller 135). Furthermore, although not shown, each of the secure processor 125, application processor 130, and touch controller 135 may include, or be associated with, additional or different elements (e.g., timers) to allow these components to perform. the functions described in the present invention.

[028] The secure processor 125 can be configured to perform secure operations on the POS device 100, such as handling payment information and identifying the PIN entered on the display 110. The secure processor 125 can be in operative communication with an application processor 130 and touch controller 135. For example, in some embodiments, secure processor 125 may receive a communication/command from application processor 130 that the PIN entry process on display 110 is starting. Upon receiving this communication/command, the secure processor 125 can begin the process to recognize the PIN characters. In some embodiments, upon receiving the PIN characters, the secure processor 125 can encrypt the PIN characters and send the encrypted PIN to the application processor 130 to perform further actions. Thus, in some embodiments, the user's PIN may be known only to the secure processor 125, thereby increasing the security of the POS device 100 and limiting which components are aware of confidential information. Similarly, in some embodiments, secure processor 125 may receive a communication/command from application processor 130 to begin receiving payment information. Upon receiving the communication/command, the secure processor 125 can start to recognize the payment data. Secure processor 125 may also encrypt the payment data before sending the encrypted payment data to application processor 130. Secure processor 125 and application processor 130 may similarly be configured to handle other sensitive data.

[029] In some embodiments, the secure processor 125 can be a chip that is separate from the application processor 130. The secure processor 125 can handle confidential information in an isolated environment to mitigate hardware and software attacks, which can compromise data from confidential card or customer PIN. In some embodiments, the separate chip may be required to meet the Payment Card Industry (PCI) PIN Transaction Security (PTS) Point of Interaction (POI) device requirements that define a security standard that devices payment gateways that optionally accept PINs and process sensitive cardholder account data.

[030] The application processor 130 can be considered a master processor for the POS device 100. In some embodiments, the application processor 130 can be responsible for instantiating an operating system for the POS device 100. In some embodiments, the application processor applications 130 may be responsible for handling the interaction of the secure processor 125 with the different mechanisms for issuing and inserting information, and isolating the secure processor from these mechanisms, thus improving the security of the POS device. For example, audio outputs (eg, speaker 105) may be managed by application processor 130 to recite the auditory cues. To "recite" the auditory cues, the application processor 130 can generate one or more signals to cause the speaker 105 to generate the auditory cues and output the auditory cues. The content to be displayed on display 110 may also be similarly controlled by application processor 130. Communications between secure processor 125 and application processor 130 are discussed in more detail below.

[031] The touch controller 135 can be configured to detect touch signals or inputs from the display 110. Thus, in some embodiments, the touch controller 135 can be associated with the touch sensor 120. In some embodiments, the controller touch point 135 may be configured to detect touch coordinates (e.g., X coordinates and Y coordinates) of display 110 corresponding to the location on the display where a touch input is received. Depending on the type of data being entered into display 110, touch controller 135 can selectively pass that data (e.g., the touch coordinates of the data) to application processor 130 or secure processor 125. In some embodiments, touch controller 135 can be in operative communication with secure processor 125 and application processor 130. In some embodiments, touch controller 135 can be configured so that touch coordinates for non-secure operations, such as adjusting display settings 110 can be routed directly to application processor 130, and touch coordinates for secure operations, such as entering a PIN, can be routed directly to secure processor 125.

[032] In some embodiments, the touch controller 135 can be configured so that the communicative connection between the secure processor 125 and the touch controller does not pass through the application processor 130. For example, in some embodiments, the touch controller 135 may be configured so that all touch data (e.g., touch coordinates) from the touch controller is routed first to secure processor 125 and is routed forward to application processor 130 only if it has determined that the touch controller is not being used to provide payment information. As another example, touch controller 135 can be configured to change a routing state for touch data between being routed to application processor 130 and secure processor 125 based on a sensed mode that POS device 100 is in. operating. In some embodiments, data may be routed directly to application processor 130 in an unsecure mode (e.g., where unsecured data is being received) and to secure processor 125 in a secure mode (e.g., where secure data is being received). are being received). The application processor 130, or an application running thereon, may be configured to transmit a command, such as an accessibility sequence entry start command, to the secure processor 125 to indicate which sensitive data, such as a PIN, is to be entered. on display 110 now. Touch sensor data 120 may be routed to secure processor 125 after the command is received by the secure processor. In some embodiments, secure processor 125 may send a command acknowledgment to indicate that POS device 100 is now in a confidential data entry mode. Interactions between secure processor 125, application processor 130, and touch controller 135 are discussed in more detail in FIG. two.

[033] Returning now to FIG. 2, a POS device 200 is shown, in accordance with some embodiments of the present disclosure. The POS Device 200 has the same components as the POS Device 100 and therefore these components are not described again. Thus, the POS device 200 has a secure processor 205 similar to the secure processor 125, an application processor 210 similar to the application processor 130, and a touch controller 215 similar to the touch controller 135. The POS device 200 also includes a display 220 similar to display 110. Although not shown, display 220 may include a touch sensor (e.g., touch sensor 120). POS device 200 may also include output device(s), such as an audio output device or speaker 225 (eg, speaker 105). Although not shown, POS device 200 may also include input device(s) such as a microphone similar to microphone 115.

[034] The POS device 200 shows the interactions between the secure processor 205, the application processor 210 and the touch controller 215 in more detail. In order for touch inputs (e.g., touch coordinates) to be routed to one or both of application processor 210 or secure processor 205, in some embodiments, touch controller 215 may be associated with a multiplexer 230. In these embodiments, multiplexer 230 may be implemented by touch controller 215 or it may be a dedicated element within POS device 200. Multiplexer 230 may be configured in software, hardware, firmware, or a combination thereof. Touch controller 215 may receive touch inputs from a touch sensor (e.g., touch sensor 120) of display 220. Touch controller 215 may send the touch inputs to multiplexer 230, which may then route the touch inputs to the respective one of the secure processor 205 and/or the application processor 210 depending on the type (e.g., secure data or non-secure data) of the touch inputs, the mode (e.g., secure mode, non-secure mode) in which the POS 200 device is operating, etc.

[035] In some embodiments, the secure processor 205 can not only receive touch inputs from the touch controller 215 through the multiplexer 230, but also send data such as commands to change the status of the multiplexer. For example, in some embodiments, secure processor 205 can control the flow of ring inputs from multiplexer 230 so that secure data is not sent to application processor 210. For example, when application processor 210, or an application running thereon, sends a command such as a reachability sequence entry start command (e.g., indicating the start of the PIN entry process) to the secure processor 205, the secure processor may send a command to multiplexer 230 to toggle the status of the multiplexer so that ring inputs are now routed to the secure processor. Thus, in some embodiments, transmission of ring inputs from multiplexer 230 may be controlled by secure processor 205. In other embodiments, application processor 210 may be configured to control multiplexer 230 directly.

[036] FIG. 2 also illustrates examples of interface types that can be used for communication between the different components of the POS 200 device, such as I2C (Inter-Integrated Circuit Protocol), GPIO (General Purpose Input/Output), USB (Universal Serial Bus) , SPI (Serial Peripheral Interface) and DSI (Serial Display Interface). In other embodiments, additional or different interfaces may be used. Furthermore, in some embodiments, one type of interface shown in FIG. 2 can be replaced with viable alternatives depending on, for example, design constraints, supply chain requirements, component availability, etc. For example, SPI can be used instead of USB and vice versa. Various components of the POS 200 device can be involved so that touch inputs are processed within the POS device in different ways (e.g. when a PIN entry routine is running on the POS 200 device as opposed to when the display is being used to input/output non-confidential information). For example, the application processor 210, the secure processor 205, the touch controller 215, the multiplexer 230, the display 220 and the speaker 225 may be involved in receiving and processing touch data.

[037] During a payment process, for example, the application processor 210 may request that the secure processor 205 enter a PIN entry mode. This may be done via a specific command, such as the reachability sequence entry start command sent from the application processor 210 to the secure processor 205. In embodiments in which the application processor 210 and the secure processor 205 are connected via a USB interface, as in the example in FIG. 2, the application processor can communicate with the secure processor, for example via RPC (Remote Procedure Call) messages over USB. Communication can also be performed via any other viable alternative technology. In some embodiments, secure processor 205 may use a GPIO hardwired to multiplexer 230 to control where the touch data flows.

[038] Since touch inputs can be sensitive data such as a PIN, it can be beneficial to keep touch inputs secure. In some embodiments, and as discussed above, touch inputs may flow to secure processor 205 or application processor 210, and only the secure processor may switch the stream through multiplexer 230. In some embodiments, a mechanism used to securing the touch inputs may include the secure processor 205 switching the multiplexer 230 such that the touch inputs are fed into the secure processor itself, where the application processor 210 does not receive touch inputs until the PIN entry is complete. In some embodiments, the secure processor 205 does not send any unencrypted PIN digits (e.g., touch inputs) to the application processor 210. In some embodiments, the format of the touch inputs may be vendor-specific and may depend on the touch controller vendor 215. In some embodiments, RPC messages may be encoded in tag length value (TLV) format, however, the TLV may be replaced with any other message encoding such as XML, JSON, etc.

[039] When the application processor 210 sends a command to the secure processor 205 indicating that PIN entry is to begin, the secure processor may trigger the multiplexer 230 to switch so that the secure processor now receives all ring data from from touch controller 215. Application processor 210 may periodically send messages to secure processor 205, such as RPC messages, requesting information about the status of the PIN entry process. Secure processor 205 may return relevant events (e.g. also via RPC messages or other signals) as they occur, such as "first digit entered", "second digit entered", "canceled" or "PIN entry completed". In response to these messages, application processor 210 may then provide some feedback to users or otherwise output information via the means for outputting information. For example, application processor 210 can manage display 220 so that an asterisk is displayed on the display (e.g., for visually impaired or low vision users) whenever a signal is received from secure processor 205 that a digit has been entered. As another example, the application processor 210 can manage the loudspeaker 225 so that an auditory message is played every time a signal is received from the secure processor 205 that a digit has been entered. In this way, the application processor 210 can manage the components of the POS device 200 during the PIN entry process while keeping them isolated from the secure processor 205 where the confidential data is being processed.

[040] The 205 secure processor can use touch inputs to determine the PIN digits and encrypt the PIN once it is fully entered. The secure processor 205 can receive touch inputs during PIN entry and convert the touch inputs into PIN digits (e.g., numbers). The secure processor 205 can send encrypted information to the application processor 210, which the application processor can then send, for example, to a payment gateway and eventually to a card issuer where it can be decrypted and verified. In some embodiments, the application processor 210 may not need to access unencrypted confidential data and therefore management of the other components of the POS device 200 can be performed securely and isolated from the confidential data itself.

[041] The components shown and explained with reference to the POS 100 device and the POS 200 device are not limiting. The POS 100 device and the POS 200 device may include additional components and be connected to peripherals that are useful for the overall intended performance of these terminals. For example, in some embodiments, the POS device 100 and POS device 200 may also include various payment interfaces, such as NFC or ICC interfaces or a magnetic stripe reader (MSR) and various peripherals, such as a bar code scanner or external payment interface device. Furthermore, although the components of the POS device 100 and the POS device 200 are shown as part of a unitary device, they may be distributed across a plurality of discrete devices. For example, the processing core can be the main component of the POS device 100 and the POS device 200 and all mechanisms for issuing or inputting information can be peripherals connected to the main core. As another example, POS device 100 and POS device 200 can be a discrete device that can be coupled to an existing system so that the system can be augmented with the disclosed functionalities. The display 110 and the display 220 can be a display of a personal user device being used in combination with the POS device 100 and the POS device 200. The loudspeaker 105 or the loudspeaker 225 can be a personal loudspeaker or headphones connected to the POS 100 device and the POS 200 device.

[042] Referring now to FIG. 3, there is shown an exemplary flowchart outlining operations of a process 300, in accordance with some embodiments of the present disclosure. Process 300 is discussed in conjunction with POS device 200. Process 300 may be used to receive a PIN from a visually impaired user of POS device 200. Process 300 begins at operation 305 with POS device 200 receiving a indication to operate in a second mode to allow a visually impaired user to interact with the POS 200 device. As discussed above, the second mode can be entered in a variety of ways, such as plugging in a headset, by personnel selecting the second mode ( e.g. on a commercial device) etc. In some embodiments, the visually impaired user may also be prompted to begin interacting with the POS device 200. For example, the speaker 225, at the direction of the application processor 210, of the headset may provide an auditory cue to the user begins to interact with the POS 200 device to enter the PIN. In other embodiments, personnel who selected the second mode can notify the user to start interacting with the POS device 200 and/or a message can be displayed on display 220 (eg, for low vision users). In other embodiments, other mechanisms can be used to notify the user to start interacting with the POS 200 device.

[043] The PIN may be associated (eg, unique to) the user and may be required for a variety of purposes. For example, in some embodiments, the PIN may be required to identify the user, user account, payment information, etc. In some embodiments, the PIN can be a predetermined set of values (eg a set of 4 numbers). In other embodiments, the PIN can be a predetermined set of alphabets, combination of numbers and alphabets, etc. The explanation below is in regards to the PIN being numbers. However, in other modalities, the PIN can assume other configurations. The auditory cues that are provided to the user can be based on the PIN configuration. For example, if the PIN is a series of numbers, the auditory cues might include numbers. If the PIN is a series of alphabets, the auditory cues might include alphabets, and so on.

[044] In addition, the user can interact with the POS device 200, and particularly the POS device display 220, in a variety of ways to enter the PIN. For example, in some embodiments, the POS device 200 can be programmed to detect user interactions such as taps, taps for predetermined periods of time, swiping in different directions, gestures, voice commands or other types of input. For example, the user may be able to enter one digit of a PIN per tap (eg, short tap) in a designated area of display 220. In some embodiments, the user may be able to enter additional commands during the entry process. PIN codes, such as cancel, end and/or delete commands. In modalities where the user enters a PIN digit via tapping, these additional commands can be given, for example, by swiping in different directions opposite the tapping, or holding down a finger for a predetermined period of time (for example, a long beat). As another example, commands can be given by voice commands through microphone 115. In other embodiments, the user can enter the PIN and other commands into the POS device 200 in other ways.

[045] User actions or gestures can be mapped to commands to be executed by the POS 200 device. A non-exhaustive list of possible user actions or gestures can include taps, swipes in different directions, such as swiping right, swiping left, swipe up or swipe down, long presses as opposed to taps, such as holding down a finger until the device determines the command has been entered, or voice commands, such as saying “delete” to delete, “cancel” to cancel, “finalize”, “completed” or “enter” to finalize, “yes” to confirm, “no” to reject, etc. A non-exhaustive list of possible commands that can be mapped to user actions or gestures include entering a PIN digit, deleting, canceling, ending, prompting for instructions, skipping instructions, repeating instructions, confirming, rejecting, or moving back and forth through of the PIN entry process.

[046] Based on the detected touch input, the POS 200 device can perform actions. For example, in some embodiments, POS device 200 (e.g., application processor 210 and/or secure processor 205) can recognize, and differentiate, between a short tap and a long tap (e.g., a press and hold ). Upon detecting a press and hold action, the POS 200 device can detect that the user wants to cancel the PIN entry process and restart. In such cases, the application processor 210 and/or the secure processor 205 may restart the PIN entry process. Likewise, application processor 210 and/or secure processor 205 can be configured to map user inputs on display 220 to specific actions.

[047] In addition to activating the second mode to allow the visually impaired user to enter the PIN and prompt the user to begin interacting with the POS device 200, the application processor 210 may receive an indication, as discussed above, that the PIN entry process should begin. Upon receiving such an indication, the application processor 210 may send a command to the secure processor 205 that the PIN entry process should begin. Upon receiving such a command from the 210 application processor, the 205 secure processor may send a command to the 230 multiplexer to begin sending any detected touch inputs to the secure processor instead of the application processor. Furthermore, each of the application processor 210 and the secure processor 205 can reset a respective timer.

[048] Specifically, in some embodiments, each of the application processor 210 and the secure processor 205 may have a timer associated therewith. For example, application processor 210 may have a first timer and secure processor 205 may have a second timer. Each of the first timer and second timer may be a local timer on application processor 210 and secure processor 205, respectively. In some embodiments, one or both of the first timer and the second timer may be a hardware timer built into application processor 210 and secure processor 205, respectively, and controlled by the respective processor. In other embodiments, one or both of the first timer and the second timer may be a software module instantiated in application processor 210 or secure processor 205, respectively.

[049] During each cycle, the application processor 210 and the secure processor 205 can reset their respective timers. More specifically, the PIN entry process may include a plurality of cycles depending on the number of values in the PIN. For example, if the PIN is a 4-digit number, the PIN entry process may include 4 cycles. At the start of each cycle, application processor 210 and secure processor 205 may reset the first timer and second timer, respectively. Application processor 210 may recite the auditory cues to the user at a predetermined rate. For a PIN that includes numbers, the auditory cues may include numbers that are recited in a specific order. For example, on each cycle, application processor 210 may recite numbers 0-9 in a designated order. Furthermore, each number can be recited in a predetermined amount of time. For example, in some embodiments, application processor 210 may recite the first number (e.g., 0) at time, t0. Application processor 210 can then recite the second number (eg, 1) in time, t(0+X). X can be thought of as a waiting period or break between reciting two digits. The wait period can provide the user with enough time to enter an entry before the next digits are recited. In some embodiments, X may be 3 or 4 seconds. In other embodiments, X may be another number of seconds. Thus, in some embodiments, the waiting period may be a predetermined period of time (t) fixed for all users and for all instances of the process where the user must enter a digit of a PIN. Furthermore, in some embodiments, the waiting period between two consecutive recitations may be the same. In other words, the same waiting period can be used throughout a cycle. In other embodiments, the waiting period between one or more consecutive recitations may vary. In other words, multivalued wait periods can be used in each cycle.

[050] The application processor 210 can recite each number in the predetermined time period based on the running counter in the first timer. Furthermore, in some embodiments, the predetermined time period may be known by the secure processor 205. For example, the secure processor 205 may know that the application processor 210 recites the first number (e.g., 0) at the time, t0, recite the second number (e.g. 1) at time, t(0+X), recite the third number (e.g. 2) at time, t(0+X), after the second number, recite the fourth number ( for example, 3) in time, t(0+X), after the third number, and so on. In some embodiments, X between the recitation of two numbers may be the same in some embodiments. Thus, the application processor 210 and the secure processor 205 can agree on the time period/speed at which the numbers are recited. Therefore, at the start of each cycle, the application processor 210 and the secure processor 205 may reset their respective timers. In some embodiments, the secure processor 205 may also know the order in which the numbers are being recited by the application processor 210.

[051] At operation 310, the application processor 210 begins reciting the auditory cues. For example, application processor 210 begins reciting the numbers in the predetermined order at the predetermined rate. After reciting each number, application processor 210 waits for a waiting period before reciting the next number. Thus, the application processor 210 recites a number, waits for the wait period, then recites another number, waits for the wait period, then recites yet another number, and so on, until all numbers (e.g., 0-9 ) have been recited. After each number is recited, the user can enter touch input (eg tap on display 220) if the PIN number matches the number just recited. For example, if the PIN is 1234, on the first cycle, application processor 210 might recite the numbers 0-9. After the application processor 210 recites number 1 and before the application processor 210 recites number 2 (e.g., during the wait period between numbers 1 and 2), the user may tap (or provide other designated touch input) on display 220 to enter the number 1. Likewise, on the second cycle, application processor 210 may recite the numbers 0-9. After the application processor 210 recites the number 2 and before the application processor recite the number 3, the user can tap on the display 220 to enter the number 2 and so on.

[052] Thus, in operation 315, the touch controller 215 receives a touch input during the recitation of the auditory cues of operation 310. The touch controller 215 can determine the touch coordinates of the display where the user has tapped (or entered the touch input). Touch controller 215 may send touch coordinates to multiplexer 230, which may then send touch coordinates to secure processor 205.

[053] In operation 320, secure processor 205 converts the touch coordinates to a digit. As noted above, secure processor 205 also has a timer (eg, second timer) that resets at the start of each cycle. Secure processor 205 is aware of the rate at which numbers are recited. For example, the secure processor 205 may be aware of the wait period, the predetermined time period in which each number is recited, PIN configuration (e.g., number of digits in the PIN), and/or the order in which the security processor 210 applications is reciting the numbers. For example, the secure processor 205 may know that the application processor 210 is reciting the numbers in order 0-9, with a wait period of X between two numbers.

[054] In some embodiments, the secure processor 205 may observe the time (e.g., from the second timer) when the touch coordinates are received from the multiplexer 230. Based on the time that the touch coordinates are received at with respect to resetting the second timer in the current cycle, secure processor 205 may convert this time to a digit. For example, in some embodiments, touch coordinates can be received within 5 seconds from when the second timer was reset in the current cycle. Since the secure processor 205 is aware of the delay period (e.g., 4 seconds) and reciting order, the secure processor can determine that the touch coordinates have been received after the application processor 210 recites two digits (e.g., one digit at 0 seconds and another digit at 4 seconds) and before reciting the third digit (e.g. third digit at 8 seconds). Since the secure processor 205 knows the order (eg, 0-9) in which the digits are recited, in the example above, the secure processor can know that the second digit that was recited is the number 1. touch coordinates are received between the numbers 1 and 2, the secure processor 205 can determine that the touch coordinates correspond to the number 1.

[055] In other embodiments, instead of calculating the period of time elapsed between resetting the second timer and receiving the ring coordinates, the secure processor 205 may determine the digit based on the last auditory cue recited before the ring beat. user is received. For example, in some embodiments, the secure processor 205 can determine (e.g., based on the wait period and the runtime in the second timer) which digit is currently recited in the auditory cue. For example, the secure processor 205 may know that the application processor 210 recites the number 0 in 0 seconds, number 1 in 4 seconds, number 3 in 8 seconds, and so on. Thus, based on the time in the second timer, the secure processor 205 can know which digit is currently being recited. For example, 5 seconds from resetting the second timer in the current cycle, the secure processor 205 knows that the application processor 210 has finished reciting the number 1 and is in the waiting period before reciting the number 2 in 8 seconds. Thus, every second in a cycle, the secure processor 205 can know which digit in the sequence of recitations was recited last and which digit should be recited next. When the secure processor 205 receives indication of the touch coordinates during a delay period, by virtue of tracking the current (or last) digit recited in the cycle, the secure processor can know which digit was recited immediately before the touch input is received. Thus, secure processor 205 can associate the touch coordinates with the last recited auditory cue in the sequence. For example, if touch coordinates are received within 5 seconds, secure processor 205 can determine that touch coordinates were received after recitation of number 1 and before recitation of number 2. Thus, when touch coordinates are received , secure processor 205 can know the last number that was recited and determines that the last number is the digit intended for the user to enter.

[056] Thus, in some embodiments, the secure processor 205 can determine the digit in two ways: (1) calculate the amount of time elapsed between resetting the second timer and the time of receiving the touch coordinates, and (2) calculate the digit based on the last digit that was recited before the touch coordinates were received. At operation 325, upon converting the touch coordinates to a digit, the secure processor 205 stores the identified digit in a secure memory (eg, secure memory 140).

[057] It should be understood that, in some embodiments, the application processor 210 may not know when the user has tapped the display to maintain security. In other embodiments, application processor 210 may know that user input has been received, but may not know the touch coordinates. In some embodiments, and regardless of whether the application processor 210 knows whether the user has entered the entry or not, the application processor may continue reciting the numbers. For example, if the user enters an entry after reciting the number 1, the application processor 210 can still continue to recite the numbers 2-9 at the same rate (e.g., same waiting period), thus avoiding the possibility that spies can listen to the auditory cues and steal the PIN. After the application processor 210 finishes reciting the numbers 0-9, the application processor resets the first timer and begins reciting the numbers 0-9 again. For an N digit PIN, application processor 210 may recite the numbers 0-9 N times. Thus, in operation 330, application processor 210 determines whether all N PIN values have been received. If not, process 300 loops back to operation 310 where application processor 210 begins reciting the numbers again. Thus, operations 310-325 can be repeated N times for an N-digit PIN.

[058] Furthermore, when determining in operation 330 that additional PIN digits are to be received, secure processor 205 also resets its timer (eg, second timer). In some embodiments, application processor 210 resets its timer (eg, first timer) by reciting the numbers 0-9. Since the secure processor 205 does not receive any further communication from the application processor after the initial command indicating that the PIN entry process is starting, the secure processor relies on its timer (e.g., second timer) to determine when that second timer must be reset. For example, if the application processor 210 is reciting numbers with a 4 second wait period, the secure processor can know that in 36 seconds, the application processor would have recited all the numbers 0-9 in one cycle. Thus, after 40 seconds, the secure processor 205 can automatically reset the second timer without waiting for any communication from the application processor 210. In other embodiments, the application processor 210 can send a command to the secure processor 205 indicating that the input of the next digit is starting. Upon receipt of this command, secure processor 205 may reset the second timer.

[059] Upon receipt of all PIN digits (for example, when the application processor 210 determines at operation 330 that no further digits are to be received), the process 300 terminates at operation 335. Upon receipt of all digits, the secure processor 205 encrypts the PIN and sends the encrypted PIN to the application processor 210 for further processing and forwarding. Although the secure processor 205 has been described as identifying the digit in operation 320 on each cycle, in some embodiments, the secure processor may receive the touch coordinates of all cycles, record the time in each cycle when input is received, and, actually identify the digits at the end when all the digits have been received.

[060] In addition, in some modes, the user can provide two entries in a single cycle. For example, the user may tap display 220 after the digit 1 and again after the digit 2. In some embodiments, the secure processor 205 may be configured to record only the first entry. In other embodiments, secure processor 205 can be configured to record the last entry on each cycle.

[061] Returning to FIG. 4, there is shown an exemplary flowchart outlining a process 400, in accordance with some embodiments of the present disclosure. Process 400 may be performed by secure processor 205 on each cycle. Upon starting at operation 405, secure processor 205 resets its timer. At the start of the first cycle, secure processor 205 may receive a command from application processor 210. In response to that command, secure processor 205 may reset the timer. On subsequent cycles, secure processor 205 may reset its timer based on the amount of time that has elapsed since the timer was reset in the current cycle, as discussed above, or upon receiving a command from application processor 210.

[062] In operation 415, the secure processor 205 receives touch coordinates from the multiplexer 230 and converts the touch coordinates to a character (e.g., digit) in operations 420 and 425 based on the amount of time that has elapsed since the reset of the timer in the current cycle, as also discussed above, or based on the last character recited in the current cycle, as also discussed above. At operation 430, secure processor 205 stores the identified character and process 400 ends at operation 435.

[063] With reference to FIG. 5, there is shown an exemplary sequence diagram outlining a process showing the interactions between the application processor 210 and the secure processor 205 during the PIN entry process, in accordance with some embodiments of the present disclosure. The process shows application processor 210 on the left side and secure processor 205 on the right side. In operation 500, application processor 210 sends a command to secure processor 205 to begin PIN capture. The application processor 210 may send this command at the start of the PIN entry process when the application processor receives an indication that the PIN capture process should begin. In operation 505, secure processor 205 sends a command to multiplexer 230 to begin sending ring inputs to the secure processor. Secure processor 205 may also send a message back, at operation 510, to application processor 210 acknowledging the command at operation 500.

[064] Upon receiving the acknowledgment from the secure processor 205, the application processor 210 may send a message at operation 515 to the secure processor indicating that recitation of numbers should begin. Upon receiving the message, secure processor 205 may reset the second timer and send a message back to application processor 210 at operation 525. Application processor 210 may also reset the first timer and begin reciting the digits one by one. For example, application processor 210 may recite a first digit (e.g., 0) in operation 530A, a second digit in operation 530B, a third digit in operation 530C, and so on until all digits 0-9 have been recited. After finishing reciting all digits, application processor 210 begins reciting the second cycle in operation 535. The second cycle and each subsequent cycle then continues similar to the first cycle.

[065] In addition, each digit is recited after a waiting period 540A, 540B, 540C, and so on. On each wait period, secure processor 205 waits to receive touch input (eg, touch coordinates). When touch input is received, secure processor 205 identifies and records the character. Thus, application processor 210 sends a message to secure processor 205 requesting that the secure processor capture a digit of a PIN. Secure processor 205 acknowledges and loops back through each digit choice from 0 to 9 with a fixed time between choices, calling back to application processor 210 to let the application processor know when the secure processor has started the process. range for a specific digit choice. This allows the application processor 210 to speak the digit, and inform the user to knock if the spoken digit matches the digit of their PIN. This process is repeated for each digit of the PIN until all digits are captured, or until the user fails to select a digit and the process ends in failure.

[066] Furthermore, regardless of when or if the user taps on display 220, application processor 210 continues to speak the digits zero through nine. This prevents application processor 210 from knowing which digit has been entered. The PIN is never exposed to application processor 210, and the selected PIN digits are never displayed or spoken back to the user. The application processor 210 may be allowed to know if a digit has been selected after the application processor has gone through all the digit choices. Once a digit is captured at the 205 secure processor, all further touch events can be ignored until it is time to capture the next digit of the PIN. This process is repeated for each digit of the PIN until all digits are captured, or until the user fails to select a digit and the process ends in failure.

[067] The sample pseudocode implemented by the secure processor 205 may be as follows:
/**
* Wait millisToWait milseconds. When time expires, return true if a touch event
* occurred while waiting, otherwise false.
*/
boolean waitForTouch(int millisToWait);
/**
* Switch touch controller mux to "AP" or "SP". The data from
touch will only flow to the
* processor selected.
*/
void setTouchMux(String selection);
/**
* Return current ringtone selection "AP" or "SP"
*/
String getTouchMux();
/**
* Store current PIN. This is never exposed outside of SP.
*/
int[] pinDigits = new int[4] ;
/**
* Invoke a callback to the AP.
*/
void invokeCallback(String methodName, Object... args);
void startCapturePin() {
setTouchMux("SP");
}
void stopCapturePin() {
setTouchMux("AP");
}
/**
* Get a single digit of a PIN. If this method returns true then the value
* pinDigits[digitNumber] is filled with the selected digit. If false,
* then pinDigits[digitNumber] is not the selected digit and should not be
* considered.
*/
boolean captureDigit(int digitNumber, long digitPeriodMs) {
if (getTouchMux() != "SP") {
// If we are invoked when SP touch controller is not selected
// returns immediately on failure.
returns false;
}
int selected = -1;
for (int choice = 0; choice <9; choice++) {
invokeCallback("onStartDigitChoice", digitNumber);
if (selected != -1) {
// The user has already hit, continues to wait, but ignores
other beats
wait(digitPeriodMs);
} else {
// Wait for the user to hit
if (waitForTouch(digitPeriodMs)) {
selected = choice
}
}
}
if (selected == -1) {
// No touch events received for any choice return false;
} else {
// Register the typed type
pinDigits[digitNumber] = selected
return true
}
}

[068] In the code above and the code below, SP is the secure processor 205 and AP is the application processor 210. Exemplary pseudocode for the application processor 210 is as follows:
String[] ORDINALS = {"first", "second", "third", "fourth"};
long DIGIT_CHOICE_PERIOD_MS = 3000; // 3 seconds
int DIGIT_COUNT = 4;
/**
* This is a wrapper for invoking RPC methods on the Secure Processor.
*/
class SecureProcessor {
...
}
/**
* Callback interface between SP and AP
*/
interface SecureProcessorCallback {
void onStartDigitChoice(int digitNumber);
}
void capturePin(long digitPerdiodMs) {
SecureProcessor.setListener(new SecureProcessorCallback() {
void onStartCaptureDigit(int digitChoice) {
// Speak the current digit choice. If the user hits
// after this is spoken, but before the next digit
// choice is spoken, SP will register.
//
// The SP is a simplified runtime and
// does not have the ability to play text for
// speak then must call back on the AP.
speak(digitChoice);
}
});
SecureProcessor.startCapturePin(digitPerdiodMs);
try {
for (int i = 0; i <DIGIT_COUNT; i++) {
speak("Knock when you hear your" + ORDINALS[i] + "digit.")
if (!SecureProcessor.captureDigit(i, DIGIT_CHOICE_PERIOD_MS)) {
speak("You failed to hit when your digit was spoken.");
return;
}
}
speak("PIN entered successfully.");
} finally {
SecureProcessor.stopCapturePin();
}
}

[069] With reference to FIG. 6, an exemplary diagram describing a process 600 for communications between a secure processor application or secure payment application (SPA) 605, a connection provider 610, and a secure processor 615 is shown, in accordance with some embodiments of the present disclosure. Secure processor 615 is analogous to secure processor 205. In some embodiments, secure processor 615 (also referred to as secure board or secure microcontroller) may be a chip in the POS device (e.g., POS device 200) that is separate from the secure processor. applications, as discussed above. The application processor (e.g. application processor 210) may be a main application processor running an operating system such as Android. Secure processor 615 can handle sensitive information in an isolated environment to mitigate hardware and software attacks that can compromise sensitive data such as card data or customer PIN in the case of devices that are used to process payment information. Communication between secure processor 615 and application processor (e.g., application processor 210) can occur in a variety of ways. For example, communication between an operating system (such as Android), or the application processor running such an operating system, and the 615 secure processor can occur via remote procedure calls (RPCs). As another example, communication between an operating system (such as Android), or the application processor running such an operating system, and the secure processor 615 can occur via asynchronous signals.

[070] RPCs can provide most of the functionality and can be served, for example, over USB. The operating system, or the application processor running such an operating system, may be the USB host and the secure processor 615 may be a USB device providing a bulk input and bulk output interface. The operating system can "wake up" the secure processor 615 if the secure processor is "asleep" using a signal. The operating system can invoke a bulk USB output method to send the RPC command and associated parameter data. The operating system can invoke the bulk USB input method and block until the 615 secure processor responds to the RPC with response data. In some embodiments, only one RPC can be invoked at a time. Signals can be implemented via GPIOs to provide a mechanism for sending notifications between the two processors. In some embodiments, a STOP signal may be provided to notify secure processor 615 to stop what the secure processor is doing in certain situations. In some embodiments, an NFC signal may be provided, for example, to Mini/Mobile, to signal that the operating system can play an NFC tap success tone. Such a signal may be necessary due to stringent performance requirements imposed on contactless payments, which the secure micro may beep itself. In some embodiments, a secure micro-event signal may be provided to notify the operating system that something interesting/relevant has just happened at secure processor 615, such as a card insert or slip or a chip card inserted.

[071] In some embodiments, the operating system may serve as a gatekeeper for the secure processor 615 by managing communication with the secure processor. For example, a privileged application included in the Android platform can serve as a gatekeeper for the secure processor 615 by providing a ContentProvider that can manage all communications with the secure processor. In some embodiments, the ContentProvider may be implemented by a SecureBoardConnectionProvider (for example, the 610 connection provider). The SecureBoardConnectionProvider can provide an interface to open and close a session with the 615 secure processor to ensure that multi-step RPC operations, such as receiving a payment, are not interrupted by other operations, such as updating a real-time clock or injecting keys. .

[072] In some embodiments, once a session is obtained, a token can be returned and the UID of the caller can be recorded. Thereafter, the caller holding the token can invoke an RPC until the session is closed. There may be time limits on RPCs and sessions. Attempts to open a session when an existing session is already open can be blocked in a fair queue for a given amount of time until the session is available or expires. In some embodiments, POS devices may include more than one secure processor. On such devices where there are multiple secure processors, when a session is opened all secure processors can be reserved for the session owner. In this situation, the RPC call can include the intended secure processor, and RPCs can be invoked on secure processors simultaneously, if desired.

[073] In some embodiments, the secure payment application (SPA) 605 may be one of the users of the secure processor 615. In the process 600, the exchange of calls to open and close session, wake and sleep secure processor, incoming transfers in bulk and bulk output, etc., are provided as examples of ways in which the SPA 605 and secure processor 615 can initiate a session and exchange information. In other modalities, other forms of communication are contemplated. As illustrated, SPA 605 may send a call open session function call to connection provider 610 which may then send a GPIO wakeup signal to secure processor 615. The GPIO signal may be received via a GPIO connection between the secure processor 615 and the application processor (e.g., application processor 210). Connection provider 610 may then send a response to the call open session function call to SPA 605. SPA 605 may then send a call transfer RPC to connection provider 610 which may in turn initiate a bulk output USB transfer of data to the 615 secure processor using another function call to the connection provider. The bulk output USB transfer of data can pass to the secure processor 615 all the information necessary to conduct the session (e.g., the amount being charged on the transaction that the SPA 605 is processing, a transaction identifier for the transaction, etc.) . To complete the session, secure processor 615 may invoke a remote procedure call which leads to an incoming bulk USB transfer of data. The incoming USB bulk or data transfer may involve the 615 secure processor transferring the data necessary for the process to continue (eg, encrypted payment information, transaction authorization confirmation, etc.). Connection provider 610 may use other means to pass the information back and forth with secure processor 615. Regardless, connection provider 610 serves to return the information requested by the initial transfer RPC. As such, process 600 can then continue with connection provider 610 returning the data from the session in a call transfer RPC return signal. At this point, the SPA 605 can then close the session by sending a connection provider 610 closed session function call. Just as the connection provider 610 opened the session, the connection provider can then return a call response closing session signal to the SPA 605 to confirm that the session is complete and sending a GPIO or remote procedure call signal to the secure processor 615 to place the secure signal processor in a sleep state.

[074] Returning now to FIG. 7, an exemplary display 700 is shown, in accordance with some embodiments of the present disclosure. Display 700 is analogous to display 110 and 220. Display 700 may be part of a POS device (eg POS device 200). Display 700 provides a user interface that facilitates human-computer interaction between users and application/secure processors. Display 700 is configured to receive user input from users via a graphical user interface ("GUI") and transmit that user input to application/security processors. The display 700 is also configured to receive information/outputs from the application/secure processors and present that output/information to users through the GUI of the display. The GUI can feature a variety of graphical icons, windows, visual indicators, menus, visual widgets, and other cues to facilitate user interaction. In other embodiments, display 700 can be configured with other types of user interfaces, including, for example, text-based user interfaces and other human-machine interfaces. Thus, the display 700 can be configured in a variety of ways.

[075] The display 700 may include an active area 705 and buttons such as a delete button 710 and a cancel button 715. In some embodiments, the delete button 710 and the cancel button 715 may be configured for use by users with visual impairment or users with low vision. In some embodiments, additional or different buttons may be provided. Also, the location, shape and size of each button may vary from what is shown. Active area 705 may be the area where touch sensor 120 is located. Thus, in some embodiments, the active area 705 may be the area of the display 700 where visually impaired users can enter their touch input (e.g., taps).

[076] In some embodiments, the active area 705 of the display 700 may include one or more tactile features surrounding the active area to allow a visually impaired user to distinguish the active area from the remaining surface of the display 700. For example, the active area 705 may be surrounded by a border 720. In some embodiments, the border 720 may be a raised or chamfered border to assist a visually impaired person in locating the active area. In other embodiments, edge 720 may be configured in other ways to allow a visually impaired user to locate the active area. Active area 705 may be located within rim 720. In other embodiments, other types of features may be provided to allow a visually impaired user to easily identify the active area.

[077] Additionally, when the 700 display is in a PIN entry mode, touch buttons and other touch indications that relate touches to specific locations on the display screen to specific actions to be taken by the POS device can be disabled. For example, a regular user interface during a PIN entry process might provide several action buttons, which could cause the POS device to perform specific actions when the user taps that specific touch coordinate, such as the numbers on a on-screen PIN pad and buttons for additional actions such as back, cancel or enter. These action buttons may not be useful for a visually impaired user and may interfere when that user is interacting with the device, unintentionally taking the process in an unwanted direction. In this way, in some embodiments, a different interface can be provided, where a visually impaired user can interact with a large portion of the display 700 without risk of mistakenly touching the action buttons and unintentionally interfering with the overall process. This different interface can be different in that any action buttons on the screen can be disabled so that the screen is only responsive to user gestures such as taps or swipes, regardless of the exact position where the gesture occurred on the screen.

[078] In some modalities, specific action buttons may still be required on the 700 display while the visually impaired user is interacting with the display. These action buttons may be needed, for example, for a device administrator to exit PIN entry mode or perform any other action to move the process forward. In embodiments where such action buttons exist, these can be provided in a non-trivial pattern so that a visually impaired user does not accidentally trigger an unwanted action when trying to enter the PIN via tapping. For example, these action buttons can be activated by pressing two specific marked locations at the same time. As another example, additional confirmation might be requested when a user attempts to press an on-screen action button, and the confirmation button might be located in a random, unconventional position on the screen. In this way, a visually impaired user can be prevented from performing unwanted actions when interacting with the 700 display while the device administrator can still have on-screen options to assist in the process. Similar functionality can be provided without presenting on-screen buttons, such as requiring a device administrator to enter a known special gesture or tap pattern to exit.

[079] In specific embodiments of the invention, while a visually impaired PIN holder is interacting with the display 700, the content on the screen can be useful for other purposes. For example, messages can be displayed throughout the process so that a third party, such as a device administrator, can assist the PIN holder and monitor progress. These cues can be projected onto the 700 display for assistance, individually or in combination with auditory cues. Instructions relevant to a device administrator, such as, for example, how to exit PIN entry mode, may also be displayed on display 700 while the user is interacting. As another example, the display 700 can show a progress indication for the PIN entry process, for example, by incrementing a star or filling a status bar every time the user successfully enters a PIN digit, as shown by the reference number 725. This way, a device administrator or a user with low vision may be able to monitor the progress of the PIN entry process. Alternatively or in combination, progress indication can be provided on a different device manager display.

[080] Thus, the present disclosure provides an audible tapping mechanism to allow visually impaired users to securely enter personal identification numbers (PINs) using a touch-capable surface (such as a touchscreen display). The engine provides a secure computing environment (eg, secure processor 205) that is configured to connect to the touch-capable surface and identify user taps on the touch-capable surface. An application processor 210 may be configured to output speech and speech queues and send and receive messages to and from the secure processor.

[081] The object described in the present invention sometimes illustrates different components contained within, or connected to, different other components. It should be understood that such represented architectures are merely exemplary and that, in fact, many other architectures can be implemented achieving the same functionality. In a conceptual sense, any arrangement of components to achieve the same functionality is effectively "joined" so that the desired functionality is achieved. Thus, any two components combined in the present invention to achieve a particular functionality can be seen as "associated with" each other, so that the desired functionality is achieved, regardless of architectures or intermediate components. Likewise, any two components so associated can also be seen as being "operationally connected" or "operationally coupled" to one another to achieve the desired functionality, and any two components capable of being associated can also be seen as being "operationally coupling " with each other to achieve the desired functionality. Specific examples of operably attachable include, but are not limited to, physically attachable and/or physically interacting components and/or wirelessly interacting components and/or wirelessly interacting and/or logically interacting and/or logically interacting components.

[082] With regard to the use of substantially any plural and/or singular terms in this document, those skilled in the art may translate from plural to singular and/or from singular to plural as appropriate to the context and/or application. The various singular/plural permutations may be expressly set forth in the present invention for purposes of clarity.

[083] It will be understood by those within the art that, in general, the terms used in the present invention and especially in the appended claims (e.g. bodies of the appended claims) are generally understood to be "open" terms (e.g. the term " including" shall be construed as "including but not limited to", the term "having" shall be construed as "having at least", the term "includes" shall be construed as "includes but is not limited to", etc. ). It will further be understood by those skilled in the art that if a specific number of an introduced claim recitation is intended, such intent will be explicitly recited in the claim and, in the absence of such recitation, such intent is not present. For example, as an aid to understanding, the appended claims below may contain the use of the introductory phrases "at least one" and "one or more" to introduce recitations of claims. However, the use of such phrases should not be interpreted as implying that the introduction of a claim recitation by the indefinite articles "a" or "an" limits any particular claim containing such introduced claim recitation to disclosures containing only such a recitation, even when the same claim includes the introductory phrases "one or more" or "at least one" and indefinite articles such as "a" or "a" (e.g., "a" and/or "a" would typically be interpreted as " at least one" or "one or more"); the same applies to the use of definite articles used to introduce claim recitations. Furthermore, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation is typically to be interpreted as meaning at least the recited number (e.g., the simple recitation of "two recitations", without other modifiers, usually means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to "at least one of A, B and C etc." is used, in general, such a construction is intended in the sense that one skilled in the art would understand the convention (e.g., "a system having at least one of A, B, and C" would include, but not be limited to, systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B and C together, etc.). Furthermore, in those instances where a convention analogous to "at least one of A, B or C etc." is used, in general, such a construction is intended in the sense that one skilled in the art would understand the convention (e.g., "a system having at least one of A, B, or C" would include, but not be limited to, systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B and C together, etc.). It will be further understood by those skilled in the art that virtually any disjunctive word and/or phrase featuring two or more alternative terms, whether in the description, claims or drawings, should be understood to contemplate the possibilities of including one of the terms, any of the terms, or both terms. For example, the phrase "A or B" will be understood to include the possibilities of "A" or "B" or "A and B". Also, unless otherwise noted, the use of the words “approximate”, “about”, “around”, “substantially”, etc., means plus or minus ten percent.

[084] The foregoing description of illustrative embodiments has been presented for purposes of illustration and description. It is not intended to be exhaustive or limiting as to the precise form disclosed, and modifications and variations are possible in light of the above teachings or may be acquired by practicing the disclosed embodiments. It is intended that the scope of the disclosure be defined by the claims appended thereto and their equivalents.

Claims (25)

Device characterized by the fact that it comprises:
a display; It is
one or more non-transient computer-readable media comprising computer-readable instructions stored thereon which, when executed by one or more processors of the device, do the one or more processors:
causing a speaker associated with the device to serially recite a plurality of auditory cues, wherein each of the plurality of auditory cues corresponds to one of a set of characters, and wherein each recitation of the plurality of auditory cues occurs at a predetermined period of time;
receiving a display touch input from a user during recitation of the plurality of auditory cues;
determining a character from the set of characters that corresponds to the touch input based on the predetermined time period; It is
store the character as a value in a user-specific information string. Device, according to claim 1, characterized in that the user-specific information is a personal identification number (PIN). Device according to claim 1, characterized in that the one or more processors additionally execute computer-readable instructions to cause the recitation of each one of the plurality of auditory cues to be separated by a waiting period, and in that touch input is received within the waiting period. Device according to claim 1, characterized in that the one or more processors additionally execute computer-readable instructions to determine the character from the character set based on an amount of time elapsed between an input receipt time of ringing and the beginning of the recitation of the plurality of auditory cues. Device according to claim 1, characterized in that the one or more processors additionally execute computer-readable instructions to determine the character from the character set based on:
tracking which character from the character set is associated with each recitation of the plurality of auditory cues based on the predetermined time period; It is
identify the character that corresponds to touch input based on a last character recited before touch input is received. Device according to claim 1, characterized in that the one or more processors comprise a secure processor and an application processor, wherein the application processor executes computer-readable instructions to cause the loudspeaker to recite the a plurality of auditory cues, and wherein the secure processor executes computer readable instructions to receive touch input and determine the character from the character set. Device according to claim 6, characterized in that the sequence of user-specific information comprises one or more values, and in which each of the one or more values is received in each cycle of one or more recitation cycles in series of the plurality of auditory cues. Device according to claim 7, characterized in that, in a first cycle among the one or more cycles, the application processor executes computer-readable instructions to send a first command to the secure processor indicating a start of the sequence, and wherein, upon receiving the first command, the secure processor executes computer-readable instructions to send a second command to a multiplexer associated with a touch controller of the POS device to send the touch input to the secure processor, wherein the controller touch detects touch input on the display. Device according to claim 7, characterized in that each of the application processor and the secure processor comprises a timer, and in which each of the application processor and the secure processor resets the timer at the beginning of each one of the one or more cycles. Device according to claim 7, characterized in that each of the one or more values is a number, and in which the plurality of auditory cues comprises numbers from 0 to 9. Device according to claim 1, characterized in that the device is a point of sale (POS) device. One or more non-transient computer-readable media, characterized in that it comprises computer-readable instructions stored thereon that, when executed by one or more processors of a device, do the one or more processors:
causing a speaker associated with the device to serially recite a plurality of auditory cues, wherein each of the plurality of auditory cues corresponds to one of a set of characters, and wherein each recitation of the plurality of auditory cues occurs at a predetermined period of time;
receiving a display touch input from a user during recitation of the plurality of auditory cues;
determining a character from the set of characters that corresponds to the touch input based on the predetermined time period; It is
store the character as a value in a user-specific information string. One or more non-transient computer-readable means according to claim 12, characterized in that the user-specific information is a personal identification number (PIN). One or more non-transient computer-readable media according to claim 12, characterized in that the one or more processors additionally execute computer-readable instructions to cause the recitation of each of the plurality of auditory cues to be separated for a waiting period, and where touch input is received within the waiting period. One or more computer readable non-transient media according to claim 12, wherein the one or more processors further execute computer readable instructions to determine the character from the character set based on an amount of time elapsed between a time of receiving the touch input and the beginning of the recitation of the plurality of auditory cues. One or more non-transient computer-readable media according to claim 12, characterized in that the one or more processors additionally execute computer-readable instructions to determine the character from the character set based on:
tracking which character from the character set is associated with each recitation of the plurality of auditory cues based on the predetermined time period; It is
identify the character that corresponds to touch input based on a last character recited before touch input is received. The one or more non-transient computer-readable media according to claim 12, wherein the one or more processors comprise a secure processor and an application processor, wherein the application processor executes computer-readable instructions to do having the loudspeaker recite the plurality of auditory cues, and wherein the secure processor executes computer-readable instructions to receive the touch input and determine the character from the character set. One or more non-transient computer-readable media according to claim 17, characterized in that the sequence of user-specific information comprises one or more values, and each of the one or more values is received at each cycle one or more cycles of serial recitation of the plurality of auditory cues. One or more non-transient computer-readable media according to claim 18, characterized in that, in a first cycle of the one or more cycles, the application processor executes computer-readable instructions to send a first command to the secure processor indicating a start of the sequence, and wherein, upon receiving the first command, the secure processor executes computer readable instructions to send a second command to a multiplexer associated with a touch controller of the POS device to send the touch input to the secure processor, where the touch controller detects touch input on the display. One or more non-transient computer-readable media according to claim 18, wherein each of the application processor and the secure processor comprises a timer, and wherein each of the application processor and the secure processor secure resets the timer at the start of each of the one or more cycles. One or more non-transient computer-readable media according to claim 12, characterized in that each of the one or more values is a number, and wherein the plurality of auditory cues comprises numbers from 0 to 9. One or more non-transient computer-readable media according to claim 12, characterized in that the device is a point-of-sale (POS) device. Device characterized by the fact that it comprises:
a touchscreen display;
a loudspeaker; It is
one or more non-transient computer-readable media comprising computer-readable instructions stored thereon which, when executed by one or more processors of the touch screen device, do the one or more processors:
causing a speaker associated with the device to serially recite a plurality of auditory cues, wherein each of the plurality of auditory cues corresponds to one of a set of characters, and wherein each recitation of the plurality of auditory cues occurs at a predetermined period of time;
receiving a display touch input from a user during recitation of the plurality of auditory cues;
determining a character from the set of characters that corresponds to the touch input based on the predetermined time period; It is
store the character as a value in a user-specific information string. Device according to claim 23, characterized in that the sequence of user-specific information comprises one or more values, and in which each of the one or more values is received in each cycle of one or more recitation cycles in series of the plurality of auditory cues. Invention of product, process, system, kit, means or use characterized by the fact that it comprises one or more elements described in the present patent application.

Images