Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
  • G09B21/00—Teaching, or communicating with, the blind, deaf or mute
  • G09B21/001—Teaching or communicating with blind persons
  • G09B21/008—Teaching or communicating with blind persons using visual presentation of the information for the partially sighted
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T1/00—General purpose image data processing
  • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining

Definitions

• Assistive technologies are software or hardware products that make software applications or operating systems accessible to individuals with a range of disabilities, such as impaired mobility, sight, hearing, etc.
• Examples of assistive technologies include magnifiers, screen readers, and Braille displays. These products use a variety of data interception techniques throughout the operating system in order to operate.
• assistive technologies intercept graphics primitive function calls at the display driver interface (DDI) level and use the operating system kernel state to build off-screen models. Such techniques often cause system instability and crashes.
• DPI display driver interface
• a graphics pipe is provided that can be called in user mode from multiple accessibility programs simultaneously and/or separately.
• a request is received from an accessibility application to access the graphics pipe, and a connection is established.
• the accessibility application listens to the graphics pipe for particular content of interest and builds a model based on that content.
• the model is used to deliver content in an accessibility application to an end user appropriately. Screen captures can be performed on at least part of the content and then rendered onto another surface.
• FIG. 1 is a diagrammatic view of parts of a graphics pipe system.
• FIG. 2 is a diagrammatic view of a computer system of one implementation of the system of FIG. 1.
• FIG. 3 is a diagrammatic view of an accessibility graphics pipe application operating on the computer system of FIG. 2.
• FIG. 4 is a high-level process flow diagram for one implementation of the system of FIGS. 1 and 2.
• FIG. 5 is a process flow diagram for one implementation of the system of FIGS. 1 and 2 illustrating the stages involved in performing screen captures on the graphics pipe and drawing the screen captures to another surface.
• FIG. 6 is a process flow diagram for one implementation of the system of FIGS. 1 and 2 illustrating the stages involved in screen readers or Braille displays accessing the graphics pipe and building a content model.
• FIG. 7 is a process flow diagram for one implementation of the system of FIGS. 1 and 2 illustrating the stages involved in magnifiers accessing the graphics pipe and building a content model.
• the system may be described in the general context as an application that improves the operation of accessibility applications and their related assistive technologies, such as screen readers, screen magnifiers, and Braille displays.
• One or more of the techniques described herein can be implemented as features within a graphics pipe application, or from any other type of program or service that facilitates accessibility scenarios.
• a graphics pipe is provided that can be called in user mode from multiple accessibility programs simultaneously.
• the accessibility application listens to the graphics pipe for particular content of interest and builds a model based on that content. The model is used to deliver content in an accessibility application to an end user appropriately.
• graphics pipe system 20 includes graphics pipe 21 and accessibility applications (22, 24, and 26, respectively).
• Graphics pipe 21 allows accessibility applications 22, 24, and/or 26 to intercept graphics primitives (e.g. geometry calls, text calls) 18, and/or information related to custom owner drawn controls 19.
• Graphics pipe 21 serves as a central location for accessibility applications to retrieve graphic display information that can be modeled and used in rendering content (and modifying content, if appropriate) in accessibility scenarios.
• connections through graphics pipe 21 are in user mode, instead of kernel mode, thereby providing a more reliable operating environment.
• accessibility application 22 is coupled to graphics pipe 21 in read-only user mode over communication pathway 28, and serves as screen reader 34.
• Accessibility application 24 is coupled to graphics pipe 21 in read and/or update user mode over communication pathway 30, and serves as a screen magnifier 36.
• accessibility application 26 is coupled to graphics pipe 21 in read-only user mode over communication pathway 32, and serves as a Braille display.
• screen readers and Braille displays do not need to alter the content of graphics pipe 21, so their respective connections to graphics pipe 21 are read-only. Numerous other accessibility applications and assistive technologies could be used instead of or in addition to those shown in Figure 1.
• an exemplary computer system to use for implementing one or more parts of the system 20 includes a computing device, such as computing device 100.
• computing device 100 typically includes at least one processing unit 102 and memory 104.
• memory 104 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.) or some combination of the two.
• This most basic configuration is illustrated in Figure 2 by dashed line 106.
• device 100 may also have additional features/ functionality.
• device 100 may also include additional storage (removable and/or non-removable) including, but not limited to, magnetic or optical disks or tape.
• Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
• Memory 104, removable storage 108 and non-removable storage 110 are all examples of computer storage media.
• Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by device 100. Any such computer storage media may be part of device 100.
• Computing device 100 contains one or more communications interface (s) 114 that allow the device to communicate with other devices.
• communications interface(s) 114 allows computing device 100 to communicate with one or more other computers and/or applications 115, where applicable.
• Examples of communications interfaces are serial ports, Universal Serial Bus (USB) ports, parallel ports, wireless communication adapters, network adapters, etc.
• Communications interface(s) 114 are used by computer 100 to exchange information such as communication media with external devices.
• Some examples of communication media are computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and include any information delivery media.
• modulated data signal means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
• communication media includes wired media such as a wired network or direct- wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.
• computer readable media includes both storage media and communication media.
• Device 100 may also have input device(s) 112 such as keyboard, mouse, pen, voice input device, touch input device, etc.
• Output device(s) 111 such as a display, screen reader, Braille display, magnifier, speakers, printer, etc. may also be included. These devices are well known in the art and need not be discussed at length here.
• accessibility graphics pipe application 200 operating on computing device 100 is illustrated.
• accessibility graphics pipe application 200 is included as part of the resident operating system on system memory 104, such as MICROSOFT® WINDOWS® or Linux.
• accessibility graphics pipe application 200 is one of the application programs that reside on computing device 100.
• one or more parts of accessibility graphics pipe application can be part computers and/or applications 115, or other such variations as would occur to one in the computer software art.
• Accessibility graphics pipe application 200 includes business logic 204, which is responsible for carrying out some or all of the techniques described herein.
• Business logic may include logic 206 for allowing read and/or updates to the graphics pipe by accessibility applications, logic 208 for supporting legacy content primitives, logic 210 for tagging content in the pipe with a handle to the control window, logic 212 for making off- screen content available as a bitmap, logic 214 for providing an indication that one or more assistive technologies are connected to the graphics pipe, logic 216 for allowing multiple clients to access the pipe concurrently and/or asynchronously, logic 218 for allowing owner drawn controls to be accessed through the graphics pipe, logic 220 for forcing applications to repaint upon new client connection to the graphics pipe, and other logic 222 for operating accessibility graphics pipe application 200.
• accessibility graphics pipe application 200 resides on computing device 100. It will be understood that business logic 204 of graphics pipe application 200 can alternatively or additionally be embodied as computer-executable instructions on one or more computers and/or in different variations than shown on Figures 2 and 3. As one non-limiting example, one or more parts of business logic 204 could alternatively or additionally be implemented as a service that resides on an external computer that is called when needed. [022] Turning now to Figures 4-7 with continued reference to Figures 1-3, the stages for implementing one or more implementations of accessibility graphics pipe application 200 are described in further detail.
• Figure 4 is a high level process flow diagram of one implementation of accessibility graphics pipe 200. In one form, the process of Figure 4 is at least partially implemented in the operating logic of computing device 100 and is executed as part of business logic 204.
• the process begins at start point 240 with accessibility application opening a connection to graphics pipe in user mode (stage 242).
• the graphics pipe tells all applications to repaint (stage 244) so they will have the most current content.
• the graphics pipe provides content (stage 246), and the accessibility application listens to the pipe for that content (stage 248).
• the content can include a client status flag indicating whether or not assistive technology is connected (stage 246), and/or the content can include off-screen content rendered in bitmaps (stage 246).
• the accessibility application builds a model to use at least part of the pipe (stage 250).
• the accessibility application closes the connection to the graphics pipe when finished (stage 252).
• the stages are repeated for each accessibility application (one or more of 22, 24, and/or 26) that accesses the graphics pipe, which can be simultaneously and/or separately (stage 254).
• stage 254 ends at end point 256.
• FIG. 5 a process flow diagram for one implementation of the system of Figure 1 illustrates the stages involved in performing screen captures on the graphics pipe and drawing the screen captures to another surface.
• the process of Figure 5 is at least partially implemented in the operating logic of computing device 100.
• the process begins at start point 260 with accessibility application opening a connection to the graphics pipe in user mode (stage 262).
• Accessibility application listens to the graphics pipe and performs screen captures on at least a portion of the content (stage 264).
• Accessibility application draws at least some of the screen captures to another surface, such as to a file or video for vision assistance and/or training (stage 266).
• the accessibility application then closes the connection to the graphics pipe (stage 268).
• the process ends at end point 269.
• FIG. 6 a process flow diagram for one implementation of the system of Figure 1 illustrates the stages involved in screen readers or Braille displays accessing the graphics pipe and building a content model.
• the process of Figure 6 is at least partially implemented in the operating logic of computing device 100.
• the process begins at start point 270 with a screen reader or Braille display client application opening a connection to the graphics pipe, such as in a read-only fashion (stage 272).
• the screen reader or Braille display client application listens to the graphics pipe for relevant information (stage 274).
• the screen reader or Braille display builds off-screen models and uses these models to output spoken voice or tactile feedback (stage 276).
• the screen reader or Braille display client application closes the connection to the graphics pipe (stage 278).
• the process then ends at end point 280.
• FIG. 7 a process flow diagram for one implementation of the system of Figure 1 illustrates the stages involved in magnifiers accessing the graphics pipe and building a content model.
• the process of Figure 7 is at least partially implemented in the operating logic of computing device 100.
• the process begins at start point 300 with the magnification application opening a connection to the graphics pipe such as in read-only and/or update mode (stage 302).
• the magnification application listens to the graphics pipe for relevant information (stage 304).
• the magnification application removes the client window from magnified content, if applicable (stage 306).
• magnification application rescales content that it obtains from the graphics pipe, such as primitives and/or surfaces (stage 308).
• magnification application composes visuals and renders the data that have been magnified (stage 312).
• Post-composition filtering is performed by magnification application, if applicable (stage 314).
• magnification application closes the connection to the graphics pipe (stage 316). The process then ends at end point 318.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• General Health & Medical Sciences (AREA)
• Educational Technology (AREA)
• Educational Administration (AREA)
• Business, Economics & Management (AREA)
• General Engineering & Computer Science (AREA)
• Health & Medical Sciences (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Software Systems (AREA)
• Mathematical Physics (AREA)
• Databases & Information Systems (AREA)
• Data Mining & Analysis (AREA)
• Human Computer Interaction (AREA)
• User Interface Of Digital Computer (AREA)
• Image Generation (AREA)
• Digital Computer Display Output (AREA)
• Processing Or Creating Images (AREA)

Priority Applications (5)

Applications Claiming Priority (2)

Publications (1)

Family

ID=38138816

Family Applications (1)

Country Status (8)

Families Citing this family (8)

Citations (3)

Family Cites Families (46)

• 2005
  • 2005-12-12 US US11/299,535 patent/US7773096B2/en not_active Expired - Fee Related
• 2006
  • 2006-11-17 BR BRPI0618551-7A patent/BRPI0618551A2/pt not_active IP Right Cessation
  • 2006-11-17 JP JP2008545614A patent/JP4928558B2/ja not_active Expired - Fee Related
  • 2006-11-17 CN CN2006800463346A patent/CN101326513B/zh not_active Expired - Fee Related
  • 2006-11-17 KR KR1020087014169A patent/KR101331337B1/ko not_active Expired - Fee Related
  • 2006-11-17 WO PCT/US2006/044927 patent/WO2007070225A1/en not_active Ceased
  • 2006-11-17 RU RU2008123838/08A patent/RU2433462C2/ru not_active IP Right Cessation
  • 2006-11-17 EP EP06838086A patent/EP1960900A4/en not_active Withdrawn

Patent Citations (4)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events