Usability Testing the WF Designer vNext (or, Yelling at Customers)

One of the things that my team is working on is the next version of the workflow designer.  In order to help us get real feedback, we engaged with our usability teams to design and execute a usability study. 

For details on what the test looks like (when we did them 3 years ago for the first version of the WF designer, see this great channel9 video).  The setup is still the same (one way glass mirror, cameras tracking the face, screen, posture of the subject), the only difference is the software, we're busy testing out some new concepts to make workflow development much more productive.  At this stage of the lifecycle, we're really experimenting with some different designer metaphors, and a usability test is a great way to get real feedback.

One thing I've always tried to do since I came to Microsoft is being sucked into the Redmond bubble.  The symptoms of placement inside said bubble are a gradual removal from the reality that everyday developers face.  When I came to the company two years ago, I was chock full of great thoughts and ideas from the outside, and much less tolerant of the "well, that's just how it works" defense. 

Slowly, though, as you start to get deep into thinking about a problem, and tightly focusing on that problem, those concerns start to fade away, as you look to optimize the experience you are providing.  Sitting in on the usability labs yesterday was a great reminder to me of how easily one can slip into the bubble.  Our test subject was working with a workflow in the designer and had a peculiar style of working with the property window in VS.  Now, when I use VS, I use the property grid in one way.  I have it docked, and I have the dock set to automatically hide.  I have known some developers who prefer the Apple / photoshop style where the property pane floats.  The customer's way of working with the property grid was that he had it floating, but he would close it after every interaction.  This required him to do one of two things in order to display the grid again, either go to the View menu, or (and what his style of work was) right clicking on an element and selecting properties.

The prototype we were doing the usability testing with, however, does not have that feature wired up, in fact, it currently doesn't display the properties item in the context menu at all.  Not because we have an evil, nefarious plan to remove the properties item inconsistently throughout our designer, but rather because no one gave it any thought when we put the prototype together as we had other UI elements we wanted to focus on. 

This became a serious problem for our customer, as the way he expected to work was completely interrupted.  At one point, we asked him to dock the property window so we could continue with the test.  This is the most fascinating part of the study to me, and that was watching him work to dock the property grid in the left panel.  I've become so used to the docking behavior in VS (see screenshot below), that it didn't even occur to me that this might present a problem for the user.  Instead, we watched for 3 minutes or so as he attempted to figure out how to move the window, and then try to process the feedback that the UX elements give.  About 60 seconds in or so, the property grid was about at a similar location to the screenshot, with just a centimeter or two's distance away from being in "the right place".  Watching his face, we saw him look slightly confused and then move it elsewhere.  Two more times he came back to that same spot, just far enough away to not get the feedback that might help him in the right direction.  It was at this point, the spontaneous yelling started among the observers in the room.  Something that has become so obvious to us, something we have internalized and accepted as "just the way the world," was becoming crystal clear to us how much difficulty this was causing.  The yelling was things like "Move up, move up" "no, wait, over, over" "oh, you almost, almost, no...." trying to will the customer through the soundproof wall what we wanted him to do.

This situation repeated itself time and time again with different UI elements, and it was very, very educational to see the way different users manage their workspace and interact with a tool that I've become so familiar with that I forget to see the forest for the trees.  I also realized, that although I had worked with a lot of customers and other developers, very rarely had I paid attention to how they work, rather than simply their work. 

Now, here's where I open up the real can of worms.   We're looking to make usability improvements in the WF designer.  Are there any that really bother you?  What can we do to make you a more productive WF developer?

Comments

• Anonymous
  May 16, 2008
  Hello Matt, It's true that docking a window in VS is not easy..the first time you do it. Once you get it, it's really not a problem anymore. In a previous post you asked some feedback about refactoring. In addition to the one you offered, a "convert to XAML workflow" refactoring could be interesting. To improve the usability of the WF designer, it would be nice to be able to resize activities. Sometimes, the name of an activity is too long and isn't shown entirely. It is not really a problem when developing a workflow but it is if we want to print the view of a workflow or use it in a report. A better understanding of XAML only workflows in VS would be nice. I have read a lot about the "VS becoming slow" issue when workflows contains too many activities or a project contains too many workflows. But it seems there will be some improvements in this field in VS 2008 SP1. (About that SP1, anything else concerning WF?) To improve productivity, I currently have to close VS 2-3 times a day, empty the ProjectAssemblies directory and restart VS (I am using VS 2008 on Vista). It would be nice to get this fixed as well as the "VS forgets the layout of state machine workflow" issue. It would be cool to have some GUI tool to generate activity validators, activity designer and activity designer themes. In a previous post, you asked us to take a survey about rehosting the workflow designer. Too bad for me, I missed it :(. So here is my opinion about it: I use it to let non programmers create XAML only workflows. I think it is a really nice feature but it would be even better with an easier programming model. A WPF wrapper? If you don't mind, I would also like to give you some improvement idea of WF in general:

• Generic activities support (or at least generic base classes).
• Not having to write WorkflowMarkupSerializer anymore.
• Provide access to TimerSubscription through a runtime service instead of (read "as well as") through the workflow.
• All my custom UITypeEditor are in Winform. I remember trying without success to make them in WPF. Maybe it's just me and then ignore this idea.
• Merging WPF and WF dependency properties I know the two first suggestion are difficult to implement because of XAML. The third is not really needed. It just goes better with the WF programming model. Could you tell us what will be released in .NET 3.5 SP1 concerning WF? Sorry for this long post which is not always related to your post but I thought it would be good place to write it here. I really enjoy working with WF and didn't want to criticize it too much but it's how things move on, isn't it? Feel free to contact me if you need more feedback :)

• Anonymous
  May 16, 2008
  Sorry but I forgot to ask one last thing in my previous post: Pageflow and BPEL for WF have been in CTP for months. Are you planning on updating these? If the answer is "No" (which would seem logical to me), don't you think it would be a good idea to release BPEL for WF in a codeplex project?

• Anonymous
  May 16, 2008
  Benoît Dion, Pageflow is not a CTP, it is a sample, so there is no official roadmap for it (although the patterns and practices group has been looking at it).  With regards to BPEL, I'm not sure what the status on that is, I'll send an email around internally. In .NET 3.5 SP1, there are no features introduced, but there have been a number of internal improvements made, including some substantial perf gains in the WF designer (for certain scenarios).  I'm not sure if those are in the beta or not.   Thanks for all the suggestions!

• Anonymous
  May 16, 2008
  Thanks Matt, A few more questions and I'll give you a break for a week or two :). Is it possible to have a mid/long term roadmap concerning WF? I heard a rumor about a new WF community web site. Can you confirm it? Is there any plan on a deeper integration between WF and UI technologies (WF as a controller)? Do you think it could simplify developement of complex UI (wizards, etc) or just make everything more complicated?

• Anonymous
  May 17, 2008
  I would like to see some timeout-feature for activities, at least for EventDrivenActivity, including an alternative view/handler (just like the fault- and cancel-handlers). It's already possible to implement timeout-logic, but I'd prefer to simply configure the duration and implement the Timeout-handler. Perhaps even support multiple timeout-handlers to enable sending reminders, etc. Furthermore, VS is pretty slow when used in a virtual machine. Especially expanding the toolbox is pretty slow, which is why I have it expanded all the time. Maybe loose (or simplify) the animations, cache icons, whatever, but speed it up please!

• Anonymous
  May 18, 2008
  From my experience the WF designer is very  unpredictable. Let say I work on a project and there are days when WF designer works fine and there are days when it crashes VS 2005 a few times a day. What is more it always crashes VS 2005 when I want to autogenerate a handler of an event using Properties windows.

• Anonymous
  May 20, 2008
  @kwaazaar, do you mean that VS in general is slow in virtual machines, or that the workflow designer is noticably slower than the rest of VS when in a virtual machine?

• Anonymous
  May 26, 2008
  Something I forgot, which is maybe more important than everything else: When an activity is renamed, bindings should be updated in consequence.

• Anonymous
  May 27, 2008
  The comment has been removed

• Anonymous
  June 11, 2008
  In my use I think it would be nice to be able to split the views up a little bit.  It would almost be nice to be able to do two types of drill downs; drag and drop; and like a CTL-DBL-CLICK which would open a new window without leaving the main design surface. It seems like most developers these days have 2-3 monitors.  At least I do.  Sometimes I find myself looking at one event-driven activity and realizing I also need to a) copy and paste some activities (I know refactor into a custom activity) or b) just trying to refresh myself on what will be happening in the next/previous event-driven activity in another state.  Being able to open multiple windows of the same workflow would be AWESOME.

• Anonymous
  June 30, 2008
  The comment has been removed

• Anonymous
  July 16, 2008
  A really useful feature, would be to provide > go to definition in the context menu when you have selected a custom activity in a workflow.

• Anonymous
  October 13, 2008
  The comment has been removed