The Braille Writing Tutor

Week 5.5-6: 9/07/06 - 9/15/06

2006-09-13T21:46:00.000+05:30

I'm back in Bangalore and things have really picked up. I can't believe its my last week already. Lots has happened.

[ New Breakthrough! ]
While I was away, the teachers switched out the three students from each group that were already on the tutor with those that weren't. I returned to a major success. Mangala*, a third standard student, has been taking Braille class for the past four years or so (I think she's been held back). According to Muktha and the teachers, in those four years she has never written a single word of Braille. After one week on the tutor, Mangala started writing Braille.

* Not the student's real name.

It seems that Mangala half-understood the concept of Braille. Apparently, whe knew there were six dots and that (1=a)(1,2=b), etc. but when told to write the alphabet on paper, she would write every letter in the same cell. The result: at the end of 10 minutes, every single dot in just one cell was punched out; the other cells were empty (this is what I saw when I tested her when I arrived). To the teachers, this meant Mangala did not know Braille at all.

However, the Braille Tutor allowed the teachers to "see" what Mangala was doing. Because the tutor does allow you to write the same cell in every letter (if you want) and speaks those letters, the blind teachers could hear her write "a b c d ... "! She was still writing in the same cell, but now we had insight into why she apparently couldn't write on a regular paper. In this case, the tutor acted as a diagnostic tool which demonstrated her problem to the teachers; now they can focus on teaching her how to switch cells!

You might wonder: couldn't somebody have caught this problem just by watching her? This is one of the challenges blind teachers face: they cannot see what the student is doing and have to assess what is happening from the end result (the paper).

A student writes with the tutor.

[ Visitors ]
I was visited at Mathru by Shiv and Ashish, two engineers working in local companies. Shiv is working on a device to help autisic children communicate with others. Its a really interesting project that he's working in independently in his spare time. Also, he gave me some great suggestions about the reducing the cost of the tutor and insight into how much it might cost to manufacture in india (maybe only $20!).

Left to right: Shiv, me, and Ashish

Shiv's device to help the autistic communicate.

[ Presentations ]

I gave a presentation of the Braille Tutor at Microsoft Research here in Bangalore. I was very excited to be there and share my ideas -- the response was very positive and I believe there is scope and interest for future collaboration. Hopefully I'll submit and then have accepted a paper into their IJCAI workshop on ICT for Development.

I visit Kentaro at MSR.

[ Press Club ]
I gave a press conference at the Bangalore Press Club to about 20-25 reporters and journalists from both local and national media. The conference was all about the collaboration between Carnegie Mellon and Mathru to use innovative technology (i.e. the Braille Tutor) to improve literacy among the blind. I'm very excited about that and am looking forward to seeing the articles. I'll post the news organization/date/pages etc. as I become aware of them. Also, I gave a live demonstration but I forgot my Braille half way through! I eek-ed by.

I talk to a group of reporters and journalists.

The entrance to the press club.

[ So Long, Farewell... ]
And, of course, I'm leaving tomorrow. Although I miss Pittsburgh , I am very sad to leave Mathru. The experience has been one of a kind. I have learned so much from the teachers and teh students, about blindness, about charity work in India, about technology, about teaching, and much more. I hope to come back in about a year and a half with a better Braille Tutor. But until then, the collaboration will continue.

Here's some photos of my last couple of days...

Manoj and I have a picnic on his roof
after returning from the Microsoft visit.

Manoj takes me around on his motorbike.
Who's that in the mirror?

Week 4 - 5.5: 8/26/06 - 9/6/06

2006-09-13T21:23:00.000+05:30

I spent the past 10 days in Delhi visiting my family members, many of whom I haven't seen in a few years. We shopped, ate out, and enjoyed my cousin's birthday. I caught a cold too (can you imagine? In Delhi, where its regularly 100 degrees.)

In Mathru-relevant news, I visited the National Association for the Blind (NAB) in India and showed them photos and videos of the tutor. They were very interested and eager to help out if we needed to remotely do user testing and the like. That will be huge for anyone who continues doing the Braille Tutor but who perhaps cannot come to India.

Week 3: 8/19/06 - 8/25/06

2006-09-06T22:24:00.000+05:30

Sorry for the delay in posting. I've been on dial-up for the last wee while and it hasn't been easy to do much online. Also, because the computer I'm using has an ancient version of Internet Explorer, I can't upload pictures yet. -- (9/10) Okay, I'm back on broadband. Here are photos.

[ Clearing your mental buffer ]
This week I made the 4th+ standard Braille tutor which includes math symbols and punctuation. Although I'm not doing testing with the 4th standard students, I plan on getting qualitative feedback from them. In designing the advanced version of the tutor, I discovered something interesting about how the students interpret and internalize their use of the tutor. The fourth standard students asked me (via the teachers) if I could make a "carriage return" button on the stylus so they could start writing on a fresh line as they would on a real slate and stylus. Now I explained that you can already start on a fresh line whenever you want just by going to the head of the line -- you don't have to explicitly create a new line since its all digital anyway. Still they insisted. I then explained that I could just as well make a dummy button that says "newline!" but does absolutely nothing else. They said that would be fine, but nevertheless, could I please create it. I was reluctant to just go ahead and incorporate something that seemed so pointless to me; my confusion suggested that I was misunderstanding something fundamental about how the students used the tutor.

Long story short, I'm learning that the students really want to use the Tutor as they would a regular slate in which they "start fresh" by leaving a blank line between sentences; in a sense, this allows them to flush their mental buffer of the sentence they were previously writing. It seems that the students were unable to achieve this "fresh start" in particular because the tutor does let you start wherever you want. That is, because they know that starting at the beginning of a line does not have any effect on the tutor, they are unable to clear their mind as they otherwise would. So, I've simply created a dummy carriage return that says "Newline!" and then clears the tutor's history buffer. They love it.

[ Visitors to Mathru ]
We also had quite a few visitors at Mathru this week to see the Braille Tutor. First I meet with Rana Nanjappa, the Project Steward at Asha for Mathru, and two software engineers from Bangalore who work at Dell. I gave them an informal presentation of the Braille Tutor and we had a very lively technical discussion. Prasanth and his brother have agreed to help out with the project after I leave. I've shown them how them the code, hardware schematics, and installation procedures in case anything should break or need to be reinstalled.

They brought some toys for the children.
The blow-up dolphin was a real hit!

Also, Ranaji brought his family and friends to see the Braille Tutor. Shalini is twelve years old and blind, and she studies at the main blind school in Bangalore. I was particularly interested to have her try out the tutor. Although she was not used to using a slate and stylus since she transitioned several years ago to a Perkins Brailler, she quickly learned how to use the tutor. Her response was enthusiastic and she felt that the tutor would have been very useful and entertaining when she was just learning to write. Shalini is very talented and sang and played her recorder for the whole school during prayer time. (On a side note, some of the kids are cutely naughty. I noticed at least one fast asleep during the 45 minute prayer period, but nobody else could tell.) We also had a lot of new ideas for improvements such as having a single large braille cell for the young children to learn six dots and using different voices for the text-to-speech synthesizer that are more appealing to children.

Shalini and company.

Shalini's teacher, Maya, claps Nagesree's hands to the rhythm of the prayer.

Shalini's mother brought a tactile map of India
which she had made when Shalini was young.

Satish, front and right,
catches up on his beauty sleep during prayers.

[ Extra stuff ]
These are random photos from around the school.

A brightly colored lizard basks on the wall surrounding the school.

Food is served on a banana leaf at a local politician's open house party
to which the students were invited.

The school uses solar energy to heat the water for bathing.
They are right outside my bedroom window,
but it took me a week to figure out what they heck they were.

Week 2: 8/12/06-8/18/06

2006-08-18T17:04:00.000+05:30

We made a lot of progress this week in incorporating the Braille Tutor into the students' regular curriculum. Firstly, I got the tutor to work on the school's laptops with relative ease. Its a simple two step process: copy the Braille Tutor executables and wav files and install Cepstral's Callie voice for text to speech synthesis. I was very excited to get it working so quickly. We also have all three tutors working in three dedicated stations.

The Braille Tutor Station

I held a session last Friday evening with the teachers to ensure they could run the tutor by themselves. This meant more exercises with the tutor to familiarize them with the interface (e.g. the button functions). I also showed Vanaja the computer instructor (who has very low vision) how to hook up the power supply, serial cable, and stylus to the E-Slate. We didn't wrap up the session until almost 11 pm.

[Trip out of town]
My aunt came to visit me on Saturday and I spent the next three days with her and two of her friends traveling around Bangalore. We spent two days at Sringeri, a large Hindu temple complex with 1000 year old history. One of her friends spends a lot of time there and knows the students/priests at the temple; she gave us a VIP-only look at the research center. Much work is going on to preserve centuries old texts on paper and palm leaves (average age, 400 years). Turns out, the Sringeri research center is part of the Digital Texts project that CMU's Raj Reddy started! They have a very impressive high-res scanner and will begin scanning and translation work soon. On our way back from Sringeri, we also stopped at Belur and Halibedu, two ancient temples with very intricate sculptures and carvings.

The temple at Halebidu.

A statue of a cow at Halebidu.

[Worrisome developments]
When I returned to Mathru, I was pleased to find that the teachers had practiced using the tutor and also introduced it to more of the students. Wednesday was a long day. I set up the tutor on a second laptop and we created the Braille Tutor stations. We started out having all of the second and third grades students give it a try. The first hour was surprising and worrisome. Many of third grade children are very proficient at Braille using a regular slate and stylus but they were writing very slowly with the E-Slate. I was concerned that there is some fundamental flaw with the tutor system. I tried to have a dialogue with the students individually to see what was going on but there were additional challenges there.

Firstly, there is an incredible language barrier that I hadn't anticipated. The younger students can read English but their grasp of conversational English is limited. Moreover, my american accent is impossible for them to understand so really there's no conversation between us. Second, the students are very eager to please. They don't want to suggest that they don't like the tutor so they hesitate to say anything remotely negative and it becomes a real challenge to get their honest opinion. Third, they are not often asked their opinion about things and usually try to figure out the right, factual answer to things (which is useful in math but a problem for me).

So, I had to carefully explain to the teachers so that they could in turn explain to the students that I would like their honest, thought-out answer and there is no right and wrong answer, that I won't be angry, and that "yes miss" is not an acceptable answer to a "why" question. So, back to the original issue of why they were slow. It turns out that they were *afraid* of the tutor! I would never have guessed this! They were concerned that they would get an electric shock if they touched the wires on the slate so they were handling it very gingerly. Additionally, they were afraid they would break the tutor if the used it as they use a regular slate. So, i first made it clear to them that the slate could not hurt them. I took the slate in my hand and put their hand on the back of mine. Then I pressed the stylus to my face and on my neck and forehead and showed them that there was nothing to be afraid of. I also made it clear that they could not break the stylus. At most, the cells might pop off but I would glue them back on. So, after a solid hour of discussion about the threat the E-Slate posed to them and vice versa, we were on a roll and the kids really started enjoying themselves.

[The quantitative study]
With this success, we decided it was time to start the quantitative portion of the pilot study. We have the 2nd and 3rd graders two simple test of proficiency. The first was to see how well they could punch all six dots in the cells in one minute (this allows us to test students who don't know the alphabet yet). And then the more proficient students also took an alphabet test (how far along can you get in writing the alphabet in 60 seconds and how many mistakes were made). Finally, we randomly chose three of the six students in each of 2nd and 3rd grade to work on the tutor for 2 weeks. After that time, we will test again and switch the students using the tutor. Finally, we will test before I leave. Because of the small sample size and wide range in students' abilities, I'm not expecting statistically conclusive results, but its good to have the data and demonstrate that we can measure skill.

Ms. Muktha's niece Krithika is visiting
from California and helps the children with the tutor.

[The emotional impact of technology]
We also showed the tutor to the older students who know Braille but can benefit by using the tutor for practice. Preethi picked up the concept immediately and within five minutes had spelled out "c-o-m-p-u-t-e-r" using the tutor and gotten the text-to-speech software to say "computer." She was so overcome with shock and happiness that she almost started crying! It was a very moving moment for me (and clearly for her); it reminded me how little it takes to make these children glow and showed me that there are so many beneficial aspects of the tutor that I hadn't anticipated.

[The tutor as a beginner's device]
Unfortunately, not all the children are as skilled as Preethi. Aditi, a young student, is finding Braille extremely challenging. She doesn't yet understand that there are six positions in a Braille cell and is lagging behind her peers. When Aditi uses a regular slate and stylus, she punches away but, because of the feedback delay, doesn't appear to correlate the presses she makes to the raised dots that appear on the paper. This is where I hope the Braille tutor can help. Right now, we are trying to show Aditi that the cause and effect relationship by having her repeat aloud the number she hears when she touches the stylus to the E-Slate. Hopefully the idea of dots and positions and letters will follow.

* I've changed the names to protect the students' identities.

[Independence day]
August 15th is independence day and the school had prepared a small program. Turns out I was the guest of honor (and walked to the front of the school to the sound of drumming!) and had to give an impromptu speech about the meaning of independence and the importance of education. The children sang some songs and Satish gave a speech about the history of India's independence.

I give a short speech on the importance of education.

The kids sing patriotic songs for independence day.

[Teaching dance]
I was watching Vanaja teach the young students how to dance and am intrigued by her technique. To teach them a series of steps, she places their hands on the tops of her feet and performs the steps herself. They pick it up after a few repetitions and are able to do it themselves. I'm impressed by the creativity.

[The cockroach incident]
I was up in my room on Friday night, getting ready for bed when I was thrown into panic at the sight of a cockroach the size of a small gila monster perched high on the wall near the door. I then saw a baby gila monster cockroach a few feet away from it. I tried to calm my self with the thought that at least they were far from my bed and couldn't fly and that if I just moved my bed from the wall, there's no way they could get me. I watched alertly as they wandered about and was about to go brush my teeth when -- ba bum -- the mommy cockroach started *FLYING* around my room! I nearly had a coronary and threw my blanket over myself so it wouldn't "get me" and took off to find Ms. Muktha or the night watchman or somebody to get them out of my room.

The school was dark and I only saw Vanaja wandering around. I told her what was up but she said everybody was asleep. After pausing a minute she shouted for Bhagya to get a broom and told me they would take care of it. I was relieved as they confidently marched up the stairs, until it dawned on me that they were *BLIND* and that there was NO WAY they were going to find, kill, and throw out the cockroaches. I insensitively called after them, "Hey, you're BLIND! How are you going to find this thing?" Bhagya called back to me, "Nevermind, miss. Its okay."

Once upstairs, they went into my room and asked me to show them where the cockroach was, but just as I was about to enter, the mommy cockroach started whirring round and I leapt back outside and peered in through the window. Inside, Vanaja (with 5 inches of vision) thwacked away at something small and black on my bed. I was relieved until we heard it fall to the floor with a decidedly plastic "clink". It was the cap of my USB drive. Doh.

So, they clearly needed my eyes and I gathered my courage and shuffled to a corner of the room. I could see the baby cockroach behind the chair but no sign of the mommy. I pointed it out to Vanaja who stuck her face right up to it to see (then I really started shouting) and began her attack again. This time they got it and threw the carcass out of the room. Whew. We then basically turned my room upside down looking for its mommy but found nothing. They tried to assuage my fears by telling me "Its just a small thing. It won't hurt you" but it didn't help. After all, I'm the only one who actually saw the thing. I went to bed that night imagining all types of creepy crawlies but the momma hasn't turned up since. Whew.

In addition to being grateful for their help, I am incredibly impressed by their self-confidence (two blind women trying to find a bug in my room!) and fearlessness. I obviously could not do what they did.

A picture of the "electronics street" where
I bought power supplies for the tutors.

Week 1: 8/5/06 - 8/11/06

2006-08-11T10:29:00.000+05:30

I arrived in Bangalore in the middle of the night on Sunday (1:30 am Monday); Ms. Muktha picked me up from the airport and we arrived at Mathru a half hour later. I spent the first 2-3 days getting oriented and making sure the tutor was working. I got a real scare on Wednesday when NONE of the E-Slates would communicate with ANY of the laptops. Fortunately, it was an issue with flaky power and they started working again that evening.

My immediate goal is to have a dialog with the teachers about what I've brought, whether they think its suitable, and how we can make it better. Yesterday we worked for about two hours in the afternoon and another hour in the evening just having them familiarize themselves with the hardware and writing. This evening my goal is for them to be able to set up the tutor on their own (power, serial cable, software, etc.) so I have to do quite a bit myself to make that happen (label the E-Slate and cables, write up an instruction manual, etc.) However, we've already had real success. Because all the teachers are blind, its a unique experience for me to try and explain what we've made. Nevertheless, they are very quick to pick up on the idea and make recommendations.

We've outlined several software requirements for each of the grades/standards: a simple position-letter feedback system for the 2nd standard/grade children, a position-letter-word system for the 3rd standard/grade children, and letter-word-essay system for the 4th, 5th, and 6th standard kids.

The teachers listen while I explain the features of the tutor.
Left to right: Ms. Muktha, Gauri, Jayashri, Bhagya, Manjula

They inspect the E-Slate by touch and
enthusiastically discuss its design.
Left to right: Jayashri, Bhagya, Manjula, and Vanaja

The teachers take turns using the tutor to write Braille.
Pictured: Gauri

This morning we also showed the E-Slate to a few of the students, just to get an idea of what we are up against. Their enthusiasm was incredible! We just as four students to come to the computer lab to play with it but we quickly had several students hover by the door once they heard the commotion. Also, the students caught on immediately and were jostling for turns to write "Mathru" on the E-Slate. Even the young kids had a go.

Vanaja shows the kids how to use the tutor.
Left to right: Manjula, Gauri, Srinivas, Gana, and Vanaja

Ms. Muktha explains it to Srinivas and Gana.

I peer over their shoulders as they write.

Finally, here are some other photos that you might find interesting.

The kids have breakfast together, sitting in rows according to grade/standard.

Two of the little ones Nagasri and Kanna
are wrapped up in towels so they don't spill.

Bhagya leads the 2nd grade/standard children to their classroom.
They walk "follow-the-leader" style.

Lastly, here is my cute studio apartment on the
top floor of the school.

Introduction

2006-08-06T13:52:00.000+05:30

I'm off to Bangalore to the Mathru School for the Blind for the six week pilot study of the Braille Writing Tutor. I will try to put weekly updates of my trip and the progress of the study here. Right now, I'm in Frankfurt, enjoying the broadband and the air conditioning. See you in Mathru!