I went to a seminar today titled "Conducting User-Centered Design Tests in 5 Minutes or Less." The speaker was Matthew Winkel, Communications Officer for Web and New Media at The College of New Jersey. The presentation took about 75 minutes, including questions.
The gist was that most people don't have a big budget for design testing, or alot of time. To top it off, most users have an attention span that tops out at 15 minutes. This led to the idea of quick and dirty user design tests, which I summarize below. The first three can be accomplished with a print out of your interface and a pencil or highlighter, the last two are a little more involved.
The short version of the different types of tests:
1.) Label test: Hand out a printed screen shot and have the user circle or highlight the labels they find confusing. This shows which labels aren't adequately descriptive or are ambiguous.
2.) Visual Affordance Tests: On a printed screen shot, have the users highlight which areas of the page are selectable. This helps you pinpoint which parts the site the users don't realize are clickable, and which parts fool the user into thinking they are.
3.) Brand Tests: Hand the user a sheet of paper that has a list of attributes (ex: trustworthy, cluttered, simple, contrived, etc), and have them circle the ones they think applies to your site. Of course, this test only works with people who have actually seen/used the site. You could also provide a printed screenshot if your site lends itself to that.
4.) Single Question Web Poll: A popup dialog that asks a single question. Figure out what one thing you most want to know. Given it's only one question, it's more likely people will respond than if there were more.
5.) One or Two Task Tests: Have user sit down and assign them a single task (or two, max). Record what they say/do. Users are more likely to participate if they know it will only take 5 minutes or so.
What to ask about? Identify your key task or most desired task. Look at the search logs to see what people are most interested in. Look at Google Analytics (or other tracker/logger) to see what questions people navigate to in the FAQ. That tells you what parts of your site are the most confusing.
How to find people for tests? Approach people in common areas, get student help to recruit their friends and friends of friends. Do not use same students repeatedly.
Thursday, March 27, 2008
Thursday, March 20, 2008
Status Report: 3/17-3/20
What I said/did:
- Assess and start training the new guy
<snip>
- Meet w/ ASUH representative to get student-oriented feedback.
Had the meeting and wrote it up on the blog: http://uhmis-ecafe.blogspot.com/2008/03/asuh.html.
The short version is that they pretty much approved the system, liked what they saw, and are very happy we talked to them. They also said that it appeared that privacy concerns were addressed and they had no issues there. We spent the majority of the meeting talking about possible methods of increasing response rates.
- Implementing staff features.
Writing code? What's that?
- Reading: MySQL and ?
The ? turned out to be Ruby. My eyes are tired. Florescent lights suck. Let's declare the Manoa Starbucks a "Remote Office," they have windows.
In other news:
- I mentioned how there needs to be a small barrier for requesting features. I'm trying out this. We'll see how it goes.
- The BSAR specs were posted to the blog and I sent an email to D on Wednesday.
- Some advisory board members were saying that they like the blog, but want notification of when posts are made. So, I started a Google Group which users can join and opt for email notification. When a post is made, the blog automatically sends notice to the group, the notice is then disseminated by email to all members of the group.
Next week:
- Reading (will it EVER end?)
- Working on eCAFE Staff features
- Going to that "User Centered Design Tests" lecture
- Looking into how we can "hide" grades when students don't do their surveys.
- Assess and start training the new guy
<snip>
- Meet w/ ASUH representative to get student-oriented feedback.
Had the meeting and wrote it up on the blog: http://uhmis-ecafe.blogspot.com/2008/03/asuh.html.
The short version is that they pretty much approved the system, liked what they saw, and are very happy we talked to them. They also said that it appeared that privacy concerns were addressed and they had no issues there. We spent the majority of the meeting talking about possible methods of increasing response rates.
- Implementing staff features.
Writing code? What's that?
- Reading: MySQL and ?
The ? turned out to be Ruby. My eyes are tired. Florescent lights suck. Let's declare the Manoa Starbucks a "Remote Office," they have windows.
In other news:
- I mentioned how there needs to be a small barrier for requesting features. I'm trying out this. We'll see how it goes.
- The BSAR specs were posted to the blog and I sent an email to D on Wednesday.
- Some advisory board members were saying that they like the blog, but want notification of when posts are made. So, I started a Google Group which users can join and opt for email notification. When a post is made, the blog automatically sends notice to the group, the notice is then disseminated by email to all members of the group.
Next week:
- Reading (will it EVER end?)
- Working on eCAFE Staff features
- Going to that "User Centered Design Tests" lecture
- Looking into how we can "hide" grades when students don't do their surveys.
Friday, March 14, 2008
Status Report: 3/10-3/14
What I said/did:
- Factor D's most recent comments into the BSAR specs.
Done. I'll be posting it to the site and asking for his review on Monday.
- Implementing staff features.
Didn't really do much here. Ended up doing some edits to the database layout and looking into how/if I can accommodate a feature request. In contemplating how the feature would work, I generated a number of questions I needed them to answer. When I forwarded the questions, the request got dropped.
At one of the sessions I went to at the conference, there was a discussion on users asking for features and how it quickly bloats software. The determination was that users should have to go over some sort of hurdle when making a feature request. If it's too easy, then they'll toss up any old idea, even if hardly anyone would use it. This was proven true in the above example, and now I'm debating how to incorporate a small "barrier" for future feature requests to make sure it's something that people have really thought about and need, rather than just a whim.
- Reading up on MySQL and maybe some other technologies for BSAR.
I'm working my way through some of the MySQL books. Will be ongoing for a while.
Next week:
- Assess and start training the new guy
- Meet w/ ASUH representative to get student-oriented feedback.
- Implementing staff features.
- Reading: MySQL and ?
While I wrote the above as a guide, it's going to be pretty fluid depending on what happens with new guy. I'll be playing things by ear next week.
- Factor D's most recent comments into the BSAR specs.
Done. I'll be posting it to the site and asking for his review on Monday.
- Implementing staff features.
Didn't really do much here. Ended up doing some edits to the database layout and looking into how/if I can accommodate a feature request. In contemplating how the feature would work, I generated a number of questions I needed them to answer. When I forwarded the questions, the request got dropped.
At one of the sessions I went to at the conference, there was a discussion on users asking for features and how it quickly bloats software. The determination was that users should have to go over some sort of hurdle when making a feature request. If it's too easy, then they'll toss up any old idea, even if hardly anyone would use it. This was proven true in the above example, and now I'm debating how to incorporate a small "barrier" for future feature requests to make sure it's something that people have really thought about and need, rather than just a whim.
- Reading up on MySQL and maybe some other technologies for BSAR.
I'm working my way through some of the MySQL books. Will be ongoing for a while.
Next week:
- Assess and start training the new guy
- Meet w/ ASUH representative to get student-oriented feedback.
- Implementing staff features.
- Reading: MySQL and ?
While I wrote the above as a guide, it's going to be pretty fluid depending on what happens with new guy. I'll be playing things by ear next week.
Friday, March 7, 2008
Status Report: 3/3-3/7
What I said/did:
- Open the original eCAFE up to instructors.
Done. Been answering the inevitable questions that result from it. On a side note, S and I agreed on an arrangement for answering emails. We'll alternate weeks for now, adjusting/pitching-in as needed. We had a few missteps where we both answered the same email, hence the need for the arrangement. I just wanted to let you know this since you are on the mailing list now and might think that she was shirking the job. This was my week, next week it's all her.
- Factor D's most recent comments into the BSAR specs.
Didn't happen. Pushing this to next week.
- Rework the specs to account for the Women's Studies changes (once approved).
Done and posted to the blog.
- Work on implementing the staff features (this will be a repeating item on my list for the next month or so).
Ongoing.
In other news: I did the summary of other schools' experiences with going to an online evaluation system, and posted it to the blog. I haven't yet heard back any feedback on it, or on any of the ideas that came out of it.
I've also been trying to re-work part of the database that just seems... well, messy. It works, but I'm sure there's a better way to arrange things so it's more intuitive, yet I've been unable to see another way to do it. These few tables are a hold out from the last version and I think I'm so used to seeing them, that my brain won't drop them for another way.
Next week:
- Factor D's most recent comments into the BSAR specs.
- Implementing staff features.
- Reading up on MySQL and maybe some other technologies for BSAR.
- Open the original eCAFE up to instructors.
Done. Been answering the inevitable questions that result from it. On a side note, S and I agreed on an arrangement for answering emails. We'll alternate weeks for now, adjusting/pitching-in as needed. We had a few missteps where we both answered the same email, hence the need for the arrangement. I just wanted to let you know this since you are on the mailing list now and might think that she was shirking the job. This was my week, next week it's all her.
- Factor D's most recent comments into the BSAR specs.
Didn't happen. Pushing this to next week.
- Rework the specs to account for the Women's Studies changes (once approved).
Done and posted to the blog.
- Work on implementing the staff features (this will be a repeating item on my list for the next month or so).
Ongoing.
In other news: I did the summary of other schools' experiences with going to an online evaluation system, and posted it to the blog. I haven't yet heard back any feedback on it, or on any of the ideas that came out of it.
I've also been trying to re-work part of the database that just seems... well, messy. It works, but I'm sure there's a better way to arrange things so it's more intuitive, yet I've been unable to see another way to do it. These few tables are a hold out from the last version and I think I'm so used to seeing them, that my brain won't drop them for another way.
Next week:
- Factor D's most recent comments into the BSAR specs.
- Implementing staff features.
- Reading up on MySQL and maybe some other technologies for BSAR.
Status Report: 2/25-2/29
What I said/did:
> - Hook up the new eCAFE to MySQL and start adding parts back in (copy
> code from the old eCAFE and edit it for the new schema).
Done. I have the stripped-down eCAFE working off my local MySQL server, and I've started adding basic functionality back in and editing it all for the new schema.
> - Once I have the changes to the schema finished, I'll add more tables in
> the MySQL database as necessary.
Made many changes, posted the new schema to the blog along with a listing of what changed between each revision and why. I'm creating tables in the db as they become necessary for implementing code/functionality.
> - Try to get some meetings with users.
I had a meeting with the chair of the Women's Studies department today (finally), and it was a very productive conversation. We came around to a different way of handling some things. I'm working on a post and an email to send to all interested parties to put the changes out there and get feedback. Assuming the changes are ok with everyone, then I'll do the necessary revisions to the schema.
Next week:
- Open the original eCAFE up to instructors.
- Factor Darrell's most recent comments into the BSAR specs.
- Rework the specs to account for the Women's Studies changes (once approved).
- Work on implementing the staff features (this will be a repeating item on my list for the next month or so).
> - Hook up the new eCAFE to MySQL and start adding parts back in (copy
> code from the old eCAFE and edit it for the new schema).
Done. I have the stripped-down eCAFE working off my local MySQL server, and I've started adding basic functionality back in and editing it all for the new schema.
> - Once I have the changes to the schema finished, I'll add more tables in
> the MySQL database as necessary.
Made many changes, posted the new schema to the blog along with a listing of what changed between each revision and why. I'm creating tables in the db as they become necessary for implementing code/functionality.
> - Try to get some meetings with users.
I had a meeting with the chair of the Women's Studies department today (finally), and it was a very productive conversation. We came around to a different way of handling some things. I'm working on a post and an email to send to all interested parties to put the changes out there and get feedback. Assuming the changes are ok with everyone, then I'll do the necessary revisions to the schema.
Next week:
- Open the original eCAFE up to instructors.
- Factor Darrell's most recent comments into the BSAR specs.
- Rework the specs to account for the Women's Studies changes (once approved).
- Work on implementing the staff features (this will be a repeating item on my list for the next month or so).
Monday, February 25, 2008
Status report 2/4-2/22
The last few weeks were in upheaval from the mainland trip and getting
sick, so the last weekly status report I filed was on 2/1.
What I said/did:
> - Setting up a meeting w/ the official advisory board.
Had the meeting. Got approval on the specs so we can move forward with
development. There are a few issues still outstanding, but we have time
before those need to be resolved, and people are assigned to get them
resolved.
> - Setting up interviews w/ various people who do things "differently."
No one wants to talk. It seems that everyone wants to
complain, but when I try to set up meetings, they're busy. At
least some actually read the doc and provided feedback. I haven't
given up on meetings, or at least phone conversations, yet.
> - Working on db schema (where I plan to spend the majority of my time)
Some changes came out of the meeting which still need to be incorporated.
I plan on finishing this today.
> - HCC stuff.
Never touched it.
In other news:
I made a stripped down version of eCAFE which logs in all the different
user types and takes them to the appropriate page. It doesn't do anything
else. This is necessary due to the huge changes going on in the database
schema. We had to strip all but the most basic of code out so we can run
the system on the new schema. This new eCAFE is checked in as a
completely separate new project.
I spent some time looking into possible methods of transferring data from
ods to eCAFE, I know this is tasked to someone else, but since it's going
to affect S and I in a big way, I think I should at least have a
passing familiarity with the options.
Spending time working on my MySQL knowledge. Apparently, I've forgotten a
lot. No worries, it's coming back.
Next week:
- Hook up the new eCAFE to MySQL and start adding parts back in (copy
code from the old eCAFE and edit it for the new schema).
- Once I have the changes to the schema finished, I'll add more tables in
the MySQL database as necessary.
- Try to get some meetings with users.
sick, so the last weekly status report I filed was on 2/1.
What I said/did:
> - Setting up a meeting w/ the official advisory board.
Had the meeting. Got approval on the specs so we can move forward with
development. There are a few issues still outstanding, but we have time
before those need to be resolved, and people are assigned to get them
resolved.
> - Setting up interviews w/ various people who do things "differently."
complain, but when I try to set up meetings, they're busy. At
least some actually read the doc and provided feedback. I haven't
given up on meetings, or at least phone conversations, yet.
> - Working on db schema (where I plan to spend the majority of my time)
Some changes came out of the meeting which still need to be incorporated.
I plan on finishing this today.
> - HCC stuff.
Never touched it.
In other news:
I made a stripped down version of eCAFE which logs in all the different
user types and takes them to the appropriate page. It doesn't do anything
else. This is necessary due to the huge changes going on in the database
schema. We had to strip all but the most basic of code out so we can run
the system on the new schema. This new eCAFE is checked in as a
completely separate new project.
I spent some time looking into possible methods of transferring data from
ods to eCAFE, I know this is tasked to someone else, but since it's going
to affect S and I in a big way, I think I should at least have a
passing familiarity with the options.
Spending time working on my MySQL knowledge. Apparently, I've forgotten a
lot. No worries, it's coming back.
Next week:
- Hook up the new eCAFE to MySQL and start adding parts back in (copy
code from the old eCAFE and edit it for the new schema).
- Once I have the changes to the schema finished, I'll add more tables in
the MySQL database as necessary.
- Try to get some meetings with users.
Friday, January 25, 2008
Status Report 1/24-1/28
What I said/did:
- Release to staff users. Set up HCC campus-wide.
- Volunteering at TIP conference for most of the week, and there's a holiday on Monday, so I'll work in eCAFE/BSAR specs as I can, but I'm not expecting much.
I did send out the notice to the staff users. HCC needs a bit more work as I'm trying something new with setting them up which needs more testing but hopefully will save me effort later in the semester.
I'm still holding on the BSAR specs as I've only gotten three responses so far, and not all of my questions have been addressed. Darrel sent an email to all probable users yesterday, asking them to provide feedback. I'll give that a week and then put out a final plea if I don't get much back.
The eCAFE specs are coming along nicely.
Had a meeting with Hae regarding eCAFE. She's going to talk to some faculty organization about a plan we discussed regarding who gets to see results. She opines that any faculty who is mandatory likely has to share their results, so we could allow the dept to see those without having the instructor send it. She'll be looking into it, meanwhile, I'll write it into the specs and see if instructors flip about it or what.
We also discussed the advisory board. She told me about hers, and I told her about mine. We've agreed that I'll publish the new specs to the blog, gather feedback from both boards, and then we'll set up a meeting with her official board to discuss and make any needed decisions.
Next week:
- Finish eCAFE specs rewrite and post to blog
- Test new HCC setup
- Work on setting up the schema in eCAFE MySQL db.
- Release to staff users. Set up HCC campus-wide.
- Volunteering at TIP conference for most of the week, and there's a holiday on Monday, so I'll work in eCAFE/BSAR specs as I can, but I'm not expecting much.
I did send out the notice to the staff users. HCC needs a bit more work as I'm trying something new with setting them up which needs more testing but hopefully will save me effort later in the semester.
I'm still holding on the BSAR specs as I've only gotten three responses so far, and not all of my questions have been addressed. Darrel sent an email to all probable users yesterday, asking them to provide feedback. I'll give that a week and then put out a final plea if I don't get much back.
The eCAFE specs are coming along nicely.
Had a meeting with Hae regarding eCAFE. She's going to talk to some faculty organization about a plan we discussed regarding who gets to see results. She opines that any faculty who is mandatory likely has to share their results, so we could allow the dept to see those without having the instructor send it. She'll be looking into it, meanwhile, I'll write it into the specs and see if instructors flip about it or what.
We also discussed the advisory board. She told me about hers, and I told her about mine. We've agreed that I'll publish the new specs to the blog, gather feedback from both boards, and then we'll set up a meeting with her official board to discuss and make any needed decisions.
Next week:
- Finish eCAFE specs rewrite and post to blog
- Test new HCC setup
- Work on setting up the schema in eCAFE MySQL db.
Subscribe to:
Posts (Atom)