WO2009053860A1

WO2009053860A1 - Aural compensation for achieving uniform accessibility for computing resources

Info

Publication number: WO2009053860A1
Application number: PCT/IB2008/052086
Authority: WO
Inventors: Shyam Kedare; Sangita Kedare; Siddharth Kedare
Original assignee: Shyam Kedare; Sangita Kedare; Siddharth Kedare
Priority date: 2007-10-26
Filing date: 2008-05-28
Publication date: 2009-04-30

Abstract

A method and a device allowing an aural Interaction as a compensation for visual interaction with a local/remote computing resource by mapping user inputs from computer peripheral devices and/or electronic sensors to Dual Tone Multi Frequency (DTMF) signal capable codes for Expert Interaction and Navigation called as Navigation and Expert Interaction Logic (NEIL) is disclosed. The invention describes a method to interact with a computing resource without actually accessing the computer or viewing the monitor of the computer and a device to achieve the said method. The invention enables visually impaired people to work on a computer. The invention is initially aiming improved communication capability, computer literacy and mainstream employment opportunities for approximately ten million visually impaired people in India and throughout the world.

Description

Aural Compensation for Achieving Uniform Accessibility for

Computing Resources

[I] Claim of Priority

[2] This application claims the priority benefit of Indian Provisional Patent Application

No. 2431/CHE/2007, filed with the Government of India, Patent Office Mumbai on October 26, 2007, and entitled 'Aural Compensation for achieving uniform accessibility of computer resources'.

[3] DESCRIPTION

[4] Background of the Invention

[5]

[6] Communication has been a key factor in a developed Society. Many older ways of communication are enhanced by new means like Telecommunication, Internet etc. in the present developed and developing countries.

[7] Internet needs devices with Graphical User Interface (GUI) to operate, where as telecommunication devices at low end do away with GUI, operate on voice and 12-key number keypad. GUI richness reduces as we move away from personal computer towards PSTN Telephones with 12-Key Number Keypad. As one move away from personal computer, Graphical or Visual Interface reduces and the user starts relying more on the audio and keypad to interact. The cost almost follows the GUI Richness.

[8] The invention is initially aiming improved communication capability, computer literacy, and mainstream employment opportunities for approximately ten million visually impaired people in India and throughout the world.

[9] Local government and non-profit organizations are consistently putting efforts in making villagers computer literate (communicate through email, etc). Even if the government is contributing computers free of cost, the system is not fully and effectively used in rural areas due to ineffective power system prevailing in India and many developing parts of the world.

[10] Internet cafes have sprung up quickly, even in small Indian cities and rural areas.

Where most people live, one of the main problems is that there is a lack of uninterrupted power supply with consistent operable voltage for sensitive Personal Computers. It is therefore the aim of the invention to overcome the above said lacunae coming in the prevailing systems.

[I I] Summary of the invention

[12] The present invention endeavors to find a solution for such problems and provide a better way for communication at a lesser cost and lesser power consumption. [13] The present invention also endeavors a method to effectively communicate with a computing resource from a remote location without actually accessing or seeing monitor of the system. The method helps visually impaired people to work on computer since the method eliminates the need for visually communicating with the computer.

[14] The methodology used to achieve the invention is named as N avigation and E xpert

I nteraction L ogic (NEIL) by the inventor. The hardware device fabricated by the inventor to accomplish the invention is named as NEIL device and the server device, which is a normal computer holding the NEIL Software is called a NEIL server.

[15] An example of the embodiment of the invention is illustrated with reference to Figure

1. NEIL Device (101) is connected to NEIL Server (105) through telecom network (104). Input Devices (102) and Output devices (103) are also connected to the NEIL device.

[16] Input devices include telephone devices, Keyboard, Mouse, Joy-Stick, Bio-Electric like Electro-Cardiogram (ECG) or Electronic Sensor and all other input devices/ sensors that can be used with a normal computer and output devices include telephone devices, LCD display, VDU, printer and all other output devices/sensors that can be used with a normal computer.

[17] In this embodiment of the invention as in figure 1, NEIL device (101) generates

DTMF (Dual Tone Multi Frequency) sine waves according to the data entered by the user to communicate the user's interaction. This signal is send to the NEIL server (102) through the telecom network (104). The server on the other end decodes the DTMF signal to audio signal and delivers to the telephone device at the user end as voice signal or any other output device as required by the user.

[18] Brief Description of Drawings

[19] 1. 1. Figure 1 shows the arrangement of NEIL Server, NEIL client, telecom

Network, input devices and output devices.

[20] 1. 2. Figure 2 shows a graph of GUI Richness, Cost and Power Consumption for various access devices including a NEIL Device.

[21] 1. 3. Figure 3 shows simplest configuration of the invention.

[22] 1. 4. Figure 4 shows preferred configuration of the invention.

[23] 1. 5. Figure 5 shows the configuration of the invention with Electro Cardiogram as input device.

[24] 1. 6. Figure 6 shows the configuration of the invention with TCP/IP connection simultaneously with Telecom Network Connection.

[25] 1. 7. Figure 7 shows the configuration of the invention with server software resting in the user device

[26] 1. 8. Figure 8 shows the elaborate configuration of NEIL device. [27] 1. 9. Figure 9 shows NEIL Code Map for Keyboard as a peripheral.

[28] Detailed Description of the invention

[29] A method and a device allowing an aural Interaction as a compensation for visual interaction with a local/remote computing resource by mapping user inputs from computer peripheral devices and/or electronic sensors to Dual Tone Multi Frequency (DTMF) signal capable codes for Expert Interaction and Navigation called as N avigation and E xpert I nteraction L ogic (NEIL) is disclosed.

[30] The words 'client device', 'NEIL device' and 'NEIL Client Software' are used to denote the access device or software as used by the end user. The words 'server device', 'NEIL Server', 'NEIL Server Software' are used to denote the software technology to communicate with the complete spectrum of access devices from Personal Computers to PSTN Telephones. The words 'NEIL Code' and 'code(s)' denote unique communication protocol between client device and NEIL Server, where NEIL acronym stands for Navigation and Expert Interaction Logic as called by the inventor. The method used to achieve this process is called as NEIL Technology.

[31] Although the present system will be discussed with reference to various illustrated examples, these examples should not be read to limit the broader spirit and scope of the present invention.

[32] Figure 2 gives a pictorial comparison of different features like GUI richness, cost and power consumption of different means of communication including telephone device, NEIL device, low end mobile, high end mobile and personal computer. As demonstrated in the graph, the GUI richness, cost and power consumption of NEIL device is very low as compared to computer and mobiles and slightly higher than an ordinary PSTN telephone device.

[33] The simplest configuration of the invention employs a telephone device, telecom network and a server device as illustrated in Figure 3. NEIL server (301) is accessed by the Telephone device (304) connected to RJ-11 (303) through Telecom Network (302). The user dials to the NEIL server with a conventional telephone device and gets connected to his computing resource through the server. Using the telephone device the user presses NEIL codes as input data. DTMF signals for the codes will be produced by the telephone device which will be transmitted to the NEIL server through the telecom network. The server decodes the DTMF signals back to NEIL code and perform functions accordingly.

[34] The actions taken by the server will be transmitted to the user either as voice to reach either user's telephone device or any other output device as required by the user.

[35] For example, if a user wants to shut down his computer when he is away, he dials into the server using a conventional telephone device to connect to his computer through the server. He then enters NEIL code 97 (NEIL Code for LeftAlt Break) and 96 (NEIL Code for F4). The DTMF signals for the numerals will be automatically generated by the telephone and send across the telecom network to the server device. At the server device this DTMF signals will be converted back to NEIL Codes and server take actions over LAN or WAN to bring up 'Shut Down' Dialog box on the user's computer. The action taken by the server will be informed to the user as voice. The user hears the dialog 'Shut Down' and understands that the 'Shut Down' Dialog box is open in his computer. He can then enter 60 (NEIL code for Enter) to actually shut down the system. If he wants to cancel the action he can press 70 (NEIL Code for Esc). Similar to 'Shut Down' Dialog box, which depicts a simple form of GUI, user can operate GUI remotely for other operations.

[36] The above said embodiment of the invention offers the remote computing using a low cost standard PSTN Telephone device. Low End Mobile Phone can replace Telephone Device (304) when the user prefers to access Server by wireless connection.

[37] A modification of this embodiment employs a computer keyboard with NEIL device and a telephone device. The NEIL device would be connected to the telecom network to enable sending of the data to the server. The user enters data using the keyboard, the NEIL device connected to the keyboard converts the input data to NEIL Code and produces corresponding DTMF signals. These signals would be sent to the server upon which the server decides the function to be performed. This embodiment eliminates the need for the user to remember the codes as said in previous embodiment.

[38] A preferred embodiment of the invention is illustrated in Figure 4. In this embodiment the user directly accesses his computer. The NEIL device (405) is connected to both input device of the computer and telephone device using RJ- 12 (404) and the telephone device (407) is connected to Telecom Network Line using RJ-11 jack (403). The user delivers data using the keyboard (406), the NEIL device produces DTMF signals corresponding to the user input and these are sent across to the server (401) through telecommunication network (402). The server on the other end produces voice signals / sound waves after processing the decoded DTMF signals, which will be transmitted on the standard telephone device at the user end as voice signals. The user thus gets the feedback for the input data after processing the same in the form of voice signals / sound waves.

[39] In another embodiment of the invention as illustrated with reference to Figure 5, the patient is hooked up to ECG (Electro-Cardiogram) Sensors (507). The NEIL device (505) produces DTMF signals corresponding to the patient's ECG Sensor Signals and are sent across to the server (501) through Telecom Network (502). The server decodes the DTMF signals to NEIL code and either waits for more signals or takes actions according to the input received. The server produces ECG chart according to the input received which can be monitored by the doctor at remote location. The telephone device (506) is connected to Telecom Network Line using RJ-11 jack (503) and NEIL device is connected to Telephone device using RJ- 12 (504). The patient thus gets the feedback for the ECG after processing the same in the form of voice signals / sound waves.

[40] In yet another embodiment of the invention as illustrated in figure 6, where NEIL code is either be generated by NEIL Device (603) or manually by any telephone device (604). NEIL Code is transmitted either using DTMF signals through telecom network (602) or over TCP/IP connection (606) to NEIL Server (601) from NEIL Client Software (605).

[41] In yet another embodiment of the invention as illustrated with reference to Figure 7,

NEIL Server Software is installed on a Pen Drive (702) and attached to a standard Personal Computer or High End Mobile Phone on its USB port. The Visual Interaction is customized to Audio Interaction in the NEIL Server Software (703). The Voice/ Audio signals/waves are received by the user on Speakers (704) installed on Personal Computer or High End Mobile Phone. The basic input device for the Audio Interaction is Keyboard (701) installed or attached to Personal Computer or High End Mobile Phone.

[42] Although the above embodiments have been explained with telephone device as output device, the NEIL device can convert the output generated by the NEIL server to such form to reflect on the output device as desired by the user. In certain cases the server generates output as DTMF signals to be sending to the user and the NEIL device at the user end decodes the DTMF signals to voice.

[43] The NEIL Device and NEIL Server Software defining the NEIL Technology are developed by the inventor for achieving this invention. The NEIL device is a portable device which can be carried along with user and can be connected to the required input and output devices/sensors to accomplish the desired work and can remove the peripheral devices or sensors after accomplishing the job.

[44] Figure 8 shows the elaborate configuration of NEIL device. This device essentially contains translator (807), Micro coded Integrated Circuit (808), ports for input/output devices including PS/2 ports (801), USB ports (804) and (805), RS232 port (802), Sensor Port (803), VDU Port (806), ports for connecting telecom devices RJ- 12 (811), DTMF Encoder (809) and DTMF Decoder (810). The user can give input through input devices like keyboard, mouse, light pen, joystick, tablet, touch screen or any other input device that can be used with a normal computer. The client device also contains a power module (813) and a solar panel interface (814) for providing power for the working of the client device. The device can also be connected to an external power supply (812). The solar panel (814) used in the NEIL device, enables the working of the device in natural light. Power module (813) regulates the power supply for the components in the NEIL device. It can receive an input either from External Power (812) or Solar Panel Interface (814).

[45] Translator (807) is a component that accepts device specific state machine signal/data for the various input and/or output peripheral devices connected at the ports (801) to (806). Micro coded Integrated Circuit (808) is programmed to convert the signals into the NEIL codes or map as understands by the NEIL server. Using NEIL Technology, any computer peripheral device and/or any electronic sensors can be mapped into a unique code for communicating with NEIL server using 12 key characters '0 to 9, hash (#) and asterisk (*)'. DTMF encoder (809) in the NEIL device converts NEIL Code to DTMF sine waves.

[46] The client device thereby converts input data and generates tones corresponding to the input from the user and sends the tones over the telecom network. The tones would be received by the server and decoded to NEIL code to perform functions.

[47] If server sends DTMF signals along with voice as a feedback, DTMF sine waves is converted into digits by DTMF decoder (810) in the NEIL device and then to NEIL Code using Micro coded Integrated Circuit (808) and then to peripheral device's Escape Code Sequence using Translator (807) in case the peripheral device is an output device like VDU or a printer.

[48] Another accomplishment of the invention is to enable working of the device very effectively in a very low power consumption of the range of 0. Iw to 2w depending on the peripheral devices and/or sensors.

[49] The NEIL server as described in this description can be a regular computer server, which can recognize, receive and decode the tones (DTMF) as send by the NEIL device. The server hosts in-house software technology called NEIL technology enabling the server in:

[50] 1. Decoding DTMF Signals as received from NEIL device.

2. Encoding DTMF Signals to be sent to NEIL device.

3. Initiation of computing action(s) or process(es) as per the decoded DTMF Signal.

4. Preparing voice feedback corresponding to the action(s) or process(es).

5. Error handling for the decoded DTMF Signals.

[51] The NEIL device is programmed in its Micro-coded Chip with the mapping as shown in Figure 9. A similar mapping is made available to NEIL Server Software to allow NEIL Code sent by NEIL Device or NEIL Client Software or manually from a PSTN Telephone Device. After decoding the NEIL Code, NEIL Server Software takes a decision to either wait for more user input from Computer Peripheral Devices or Electronic Sensors or dispatches a Voice Signal/Sound wave to the available channel like Telecom Network or a TCP/IP connection. [52] The NEIL server is programmed to accept the NEIL code and make the decisions based on Voice User Interface (VUI) configuration in XML form. This includes various navigation parameters like field name/type, database connection table/field, hyperlink, access mode, database update details, database default details, format, validation condition/range, field entry/exit sequences, hot key definition. This VUI configuration can be static or dynamic (generated on the fly) depending on the dynamic nature of the GUI interface to be handled. This configuration also allows a complex GUI interface to be abstracted or simplified to suite user's requirement ranging from novice to expert interface definition. This VUI configuration allows Just- In-Time validation for an error free checks or syntax and semantics of the data.

Claims

[1] I claim,

[I] A system for aural interaction as a compensation for visual interaction with a local/remote computing resource, the system comprising: an input device for entering user data or corresponding codes; a device converting input data to codes and producing DTMF tones for the codes; telecom network to transfer DTMF signal to server device; server device receiving input signal, performing functions according to the input data and producing output data; output device to reflect the output data send by server device.

[2] A system for aural interaction as a compensation for visual interaction with a local/remote computing resource as said in claim 1 where the input device includes telephone device, keyboard, mouse, joy-Stick and Bio-Electric like

Electro-Cardiogram (ECG) or Electronic Sensor.

[3] A system for aural interaction as a compensation for visual interaction with a local/remote computing resource as said in claim 1 where output device includes telephone device, LCD display, VDU, and printer.

[4] A system for aural interaction as a compensation for visual interaction with a local/remote computing resource as said in claim 1 where a device converts the output signal send by the server device into peripheral device's escape code sequence to generate output in a form desired by the user.

[5] A system for aural interaction as a compensation for visual interaction with a local/remote computing resource as said in claim 1 where the server software resides in a storage medium that can be run on user device.

[6] A system for aural interaction as a compensation for visual interaction with a local/remote computing resource as said in claim 1 where the output data is send over TCP/IP connection.

[7] A device as said in claim 1 converting input data to multi character codes for communicating with server device and producing DTMF signals when computer peripheral devices and electronic sensors are not capable of converting input data and producing DTMF signals.

[8] A device as said in claim 1 converting the output data send by the server device to peripheral device's escape code sequence to generate output in a form as desired by the user.

[9] A device as said in claim 1 to work on a computer very effectively in a very low power consumption of the range of 0.1 w to 2w depending on the peripheral devices and/or sensors.

[10] A server device capable of receiving the DTMF signals, decoding the signal, taking decisions, performing functions according to the signals received, preparing output data of the received codes and delivering output data.

[H] A method for aural interaction as a compensation for visual interaction with a local/remote computing resource, the method comprising: a device receiving input data from computer peripheral devices; mapping the data to codes; sending codes as DTMF signal over telecom network to the server device; server device decoding DTMF signal to codes, performing functions according to input data and sending output to the output device at the user end.

[12] A method to convert the output signal send from the server device in a form as desired by the user, preferably as voice or DTMF signals.

[13] A method to define codes based on digits 0 to 9 and characters asterisk and hash/pound to transmit user interaction from computer peripheral devices.

[14] A method to convert user interaction from computer peripheral into Dual

Tone Multi Frequency (DTMF) signals capable codes as defined in Claim 11.

[15] A method to receive codes at the server device and decoding the codes as said in claim 11.

[16] A method for aural interaction as a compensation for visual interaction with a local/remote computing resource, the method comprising; inputting data as DTMF signal capable codes using a telephone device including a mobile phone; sending DTMF signal over telecom network to the server device; server device decoding DTMF signal to codes, performing functions according to the input data and sending output to the output device at the user end.

[17] A method for aural interaction as a compensation for visual interaction with a local/remote computing resource, the method comprising: client device receiving input data from computer peripheral devices; mapping the data to codes; sending codes as DTMF signal to the server software installed in a pen-drive attached to the user PC; server device decoding DTMF signal to codes; performing functions according to input data and sending output to the output device at the user end.

[18] A method to convert user interaction from computer peripheral or signals from electronic sensors into Dual Tone Multi Frequency (DTMF) signals capable codes as defined in Claim 17.

[19] A method for visually impaired people and computer illiterate people to work on a computer.

[20] A method to define Voice User Interface (VUI) configuration in XML format to abstract or simplify GUI Interface in Static or Dynamic form allowing user definable navigation and to perform syntax/semantic checks for the data entered by user.