[xgl-mag] XGL/compiz screen magnifier planning

[Henrik Nilsen Omma]

Hello everyone,

The application deadline for SoC students has just ended. In the same 
way that I posted some new thoughts about SOK I will also post some 
musings on the XGL magnifier. See: 
https://lists.ubuntu.com/archives/ubuntu-accessibility/2006-May/000371.html

Again I've copied in the applicants on Bcc. Feel free to jump in with 
comments and ideas or just hang back and read the suggestions. To sign 
up: https://lists.ubuntu.com/mailman/listinfo/ubuntu-accessibility

The advantage in this open approach is that there are many people 
reading this list who know a great deal about accessibility and others 
who know much about ubuntu and development. The spec ideas still need a 
fair bit of fleshing out and input from experienced people. The sooner 
we start that process the better positioned we will be when it is time 
to start coding.


XGL/compiz screen magnifier
--------------------------------------------------

I'm afraid most student applicants have underestimated this project, 
thinking we are simply looking for a magnifier window to follow the 
mouse cursor around. That is trivial to implement, but not very helpful. 
This misunderstanding is largely my fault for not writing up the project 
in greater detail before the SoC application process. So let me try to 
explain better here.

The best way to get a feel for what we are looking for is to play with 
gnopernicus in magnifier mode. It has the basic features we need, but 
performance is not great and the usability could also do with some love. 
You'll notice the vital cursor tracking feature. If you open Gedit with 
the magnifier on, you can see that the zoom window will stay focused on 
the input cursor as you type along. If you try the same thing with kmag 
you will see that this feature is missing.


The must-have features:
 
 * Fixed zoom factors - This could be pre-set by the user, but it should 
also be possible to switch zoom levels on the fly with a hotkey
 * Ability to follow the text-entry cursor - This requires AT-SPI magic, 
but is a vital feature.
 * Configurability - As many settings as possible should be configurable 
by the user (but it can be a simple config file to start with).


Some desirable features:

 * Variable zoom factor - Fixed factors may not be exactly want you 
want. It would be cool to be able to adjust the zoom level precisely 
with the mouse wheel, say.

 * Scriptability - Again Orca has shown that being able to tailor the 
behaviour of the AT app to your working application can be very useful. 
This might well apply to the magnifier as well.

 * Fit window to zoom area - If you are working in Gedit and you have 
zoomed in to a comfortable level, you now want to make sure that the 
whole window fits into the viewable area. You then press a hot key and 
Gedit's window is resized so that it fits.

 * Jumping between window sections - If you zoom to a fixed level of 3x 
you then have in a sense 9 sections on the screen. You could float 
around the desktop with the mouse, but there might be cases where you 
want to know exactly where you are. On Gnome you can switch between 
desktops with the Hotkeys Ctr-Alt-Right/Left/Up/Down. We could do the 
same with zoomed sections.

 * Interaction with screen readers and braille systems - This adds 
another level of complexity but could potentially be extremely powerful. 
You could browse at a moderate zoom level and then have sections of text 
read out to you.


Let's look at some use cases to get an idea of the kind of features we 
need to provide:

Use case - Reading a news website:

Take this news page: http://news.bbc.co.uk/1/hi/uk/4752819.stm
It has some text and pictures and a great deal of non-essential 
navigation items (most other newspapers are worse). For the sake of 
example, let's consider a user who might use 2x zoom to get a general 
overview of a desktop of webpage layout but would need 8x zoom on normal 
text to read it.

It would be useful to be able to zoom in on the different elements 
separately. Imagine that you could select the picture first. Press some 
hotkey to skip to the picture, and have the picture by itself displayed 
at 8x zoom where you can pan around. Another keystroke would give you 
the caption overlayed at the bottom of the picture in a high contrast 
colour or it could be read out via the screen reader.

You then want to move on to the main body text. You return to the 2x 
overview page. You press another hot key to highlight some text. The 
first block of text appears with a heavy bright red (customisable of 
course) frame around it or with a full rectangle at 50% opacity so you 
can pick out the block you want. the next hotkey or mouse command would 
then zoom that text up to 8x, filling most of the screen. It would be 
wrapped to fit nicely and could be scrolled if needed. It would be as if 
you selected the text, copied and pasted it into a special app that 
would magnify it to your liking or read it out if you preferred. You 
could also select your own fonts.


Use case - Document composition:

In using a word processor like OpenOffice you can set the font size very 
large or the zoom to a high factor in OpenOffice itself, which might be 
sufficient to read the text. However there are several problems:

 * The text entry cursor is narrow and difficult to spot
 * The toolbar buttons, slider widgets, etc. are small and in low 
contrast grey tones
 * The menubar menus and context menus are small

You could go for a large high-contrast theme, but in fact you might 
prefer to keep the buttons and widgets small so they don't take up too 
much space and distract from the text you are working on.

Solution: You should be able to hover over the toolbar and have the 
buttons zoomed by 8x and the contrast enhanced. You move the mouse over 
to the right and the scrolling widget grows a large handle in high 
contrast that you can easily grab and scroll. The context menus also 
appear zoomed.


Implementation:

Both of these cases need a fair bit of information from the application 
via AT-SPI and there needs to be custom script knowledge about the 
application and individual user preferences. In the case of the BBC 
website the system would also need to know where the useful information 
usually is located. But if you have 10-20 websites you visit often you 
could teach the magnifier to behave a certain way on those domains, and 
you could share those custom scripts with others.

It soon becomes apparent that this is a great deal more complicated than 
simply providing a moveable magnification window. But then simple 
problems are boring really :)  With the combination of the new XGL 
technology, an experienced user group in our community and fresh 
programming talent we have an opportunity to do some great stuff!

- Henrik