user interface animations Re: Next Meeting **Revised Date**: May 6 2010 10:00 UTC

[Eric S. Johansson]

On 5/3/2010 2:23 PM, Phillip Whiteside wrote:
> Hi,
>
> not sure I understand, do you want something like a slide-presentations
> (power-point style) as done on these
> https://wiki.ubuntu.com/UbuntuDeveloperWeek or a screen cast such as
> http://lubuntu.net/node/32?
>
> If it is the latter, I will ask leszek if he could spare some time to
> help you. If it is the former, I will make some further enquiries

The end result I want is an animated sequence intermixing audio and graphics 
representing a user interface and its operation. F take a simple example of 
changing a directory. For the most part, the is an open loop system where you 
have the grammar of "cd <dir>" And dir is just a list of names. Usually static 
or, at best customized by the developer on each machine.

This sucks.

open a terminal window
Try "push to "
observe: sidebar with last 10 reference directories on-screen.

"Push to 10"

The 10th item on the list is used to execute a pushd command and some marker is 
left on the screen somewhere indicating it's been pushed. Maybe a [dir] in the 
title bar.

But what if you didn't want one of the 10 on the first list. You want something 
from the most frequently used list

"Go frequently used"

and the sidebar indication would change to list of most frequently used directories.

One could also search so with something like

"Name starts with albc"

a search box would open up on top of the sidebar and you could edit by voice or 
by hand search box.

However, at no point the user interface give you any capability whatsoever to do 
anything by hand other than the simple editing feature. Everything must be 
driven by voice

As you can see, this is a moderately complex user interaction that should be 
simpler when presented visually and by request for a helpful animator.

Things get more complicated why start talking about the enhanced dictation box 
I've been pushing for a couple of years. It's probably the best solution 
available for bridging between NaturallySpeaking and Linux.

I advocate for the solution because I do not believe it is possible for the 
open-source community to come up with a workable speech recognition solution by 
the time I die. I would like to spend the next 15 years of my life being 
effective, working with speech recognition to make money, so I can write, and so 
I can have a comfortable life. I want to make programming by voice work far 
better than we've done so far. Enhanced dictation box is core to my ideas on 
that topic. But, enhanced dictation box is a simple complicated idea that needs 
animation to make the concept accessible to many people.

now that I'm already wound up on the topic. :-)

Enhanced dictation box is the same as the regular dictation box with four major 
differences.

1) user definable cut and paste sequences (mouse control, keystroke sequences) etc.

2) persistence. Does not go away until you tell it to. Implies one copy per 
active application but active dictation box matches window focus

3) internal log/journaling system. Can't lose data, cannot figure out what you said

4) input and output transformations. After you grab data, and transform it into 
some speech recognition friendly form. When you paste it, the data returns to 
the speech recognition hostile form


If one is truly clever, the application we split into two parts. The first part 
is the user interface. Light, fast, doesn't degrade recognition. The second part 
talks to the first part by the net which means that do not need to be on the 
same machine. Do it right and you can dictate on Windows or wine and have your 
results show up in Linux.

Not that difficult to build if you have hands but potentially extremely useful.

>
> Regards,
>
> Phill.
>
>
> On Mon, May 3, 2010 at 6:06 PM, Eric S. Johansson <esj at harvee.org
> <mailto:esj at harvee.org>> wrote:
>
>     On 5/3/2010 12:59 PM, Penelope Stowe wrote:
>      > The meeting will be in #ubuntu-accessibility.
>      >
>      > And for the person who asked, yes, that's 6AM EDT in the US. I know
>      > it's really early, but last time most of the people at the meeting
>      > were in the UK/Europe and also due to my work schedule this week, I
>      > needed to hold an early meeting.
>      >
>      > Please let me know if you have any further questions!
>
>     I could use a graphics person with some animation knowledge.  I've been
>     advocating a tool and a new form of user interface to make speech
>     user interface
>     is far more practical and discoverable. Obviously my mouse hand is
>     on the fritz
>     and, well speech just as the wrong tool for creating graphics.
>
>     I could really use the help with this because I believe that these user
>     interface models or something derived from them would be the next
>     step in a
>     better direction for us. Everyone I've given the whiteboard talk to
>     really loves
>     it but fuzzy whiteboard and me waving my hands around just doesn't
>     translate to
>     the Internet. :-)
>
>     --- eric
>
>     --
>     Ubuntu-accessibility mailing list
>     Ubuntu-accessibility at lists.ubuntu.com
>     <mailto:Ubuntu-accessibility at lists.ubuntu.com>
>     https://lists.ubuntu.com/mailman/listinfo/ubuntu-accessibility
>
>