Editing Screen reader (section)

=== Graphical ===

==== Off-screen models ====

With the arrival of [[graphical user interface]]s (<abbr>GUI</abbr>s), the situation became more complicated. A <abbr>GUI</abbr> has characters and graphics drawn on the screen at particular positions, and therefore there is no purely textual representation of the graphical contents of the display. Screen readers were therefore forced to employ new low-level techniques, gathering messages from the [[operating system]] and using these to build up an "off-screen model", a representation of the display in which the required text content is stored.<ref>According to "[ftp://service.boulder.ibm.com/sns/sr-os2/sr2doc/guitalk.txt Making the <abbr>GUI</abbr> Talk]{{dead link|date=May 2025|bot=medic}}{{cbignore|bot=medic}}" (by Richard Schwerdtfeger, ''BYTE'' December 1991, p. 118-128), the first screen reader to build an off-screen model was outSPOKEN.</ref>

For example, the operating system might send messages to draw a command button and its caption. These messages are intercepted and used to construct the off-screen model. The user can switch between controls (such as buttons) available on the screen and the captions and control contents will be read aloud and/or shown on a [[refreshable braille display]].

Screen readers can also communicate information on menus, controls, and other visual constructs to permit blind users to interact with these constructs. However, maintaining an off-screen model is a significant technical challenge; hooking the low-level messages and maintaining an accurate model are both difficult tasks.{{citation needed|date=January 2015}}

==== Accessibility APIs ====

Operating system and application designers have attempted to address these problems by providing ways for screen readers to access the display contents without having to maintain an off-screen model. These involve the provision of alternative and accessible representations of what is being displayed on the screen accessed through an <abbr>[[Application programming interface|API]]</abbr>. Existing <abbr>API</abbr>s include:
* Android Accessibility Framework<ref>[https://developer.android.com/training/accessibility/index.html Implementing Accessibility on Android].</ref>
* Apple Accessibility <abbr>API</abbr><ref>[https://developer.apple.com/documentation/Accessibility/Reference/AccessibilityLowlevel/index.html Apple Accessibility <abbr>API</abbr>].</ref>
* [[AT-SPI]]
* [[IAccessible2]]
* [[Microsoft Active Accessibility]] (<abbr>MSAA</abbr>)
* [[Microsoft UI Automation]]
* [[Java Access Bridge]]<ref>{{cite web|url=http://java.sun.com/products/accessbridge/|title=Oracle Technology Network for Java Developers – Oracle Technology Network – Oracle}}</ref>

Screen readers can query the operating system or application for what is currently being displayed and receive updates when the display changes. For example, a screen reader can be told that the current focus is on a button and the button caption to be communicated to the user. This approach is considerably easier for the developers of screen readers, but fails when applications do not comply with the accessibility <abbr>API</abbr>: for example, [[Microsoft Word]] does not comply with the <abbr>MSAA</abbr> <abbr>API</abbr>, so screen readers must still maintain an off-screen model for Word or find another way to access its contents.{{citation needed|date=January 2015}} One approach is to use available operating system messages and application object models to supplement accessibility <abbr>API</abbr>s.

Screen readers can be assumed to be able to access all display content that is not intrinsically inaccessible. Web browsers, word processors, icons and windows and email programs are just some of the applications used successfully by screen reader users. However, according to some users,{{Who|date=January 2015}} using a screen reader is considerably more difficult than using a GUI, and many applications have specific problems resulting from the nature of the application (e.g. animations) or failure to comply with accessibility standards for the platform (e.g. Microsoft Word and Active Accessibility).{{citation needed|date=January 2015}}