Agile Documentation - Tested to Destruction

Agile development is an important technique here at p&p; and throughout much of Microsoft. However, I'm yet to be convinced that it's a good way of creating user guidance and documentation. It seems to me that the process often gets in the way more than it helps to produce a great final product.

I've rambled on many times in the past about agile documentation. Most specifically in Can Writers Dance The Agile? and other posts here. Yet I keep thinking it needs deeper investigation - especially as the group I'm officially assigned to, CSI, insists we keep prodding it to see if it works.

Note: Here at Microsoft, CSI is "Content Services & International". It's probably a bit less exciting than doing clever stuff with fingerprints and DNA, but we do have fancy computers that nearly come close to those you see in the TV versions of CSI. Maybe we should call ourselves "CSI Microsoft," wear white coats, and walk around with a flashlight held over our head.

Anyway, coming back to the point, we've had another go at agile docs recently. Instead of a solid and agreed structure plan and detailed implementation notes of what we wanted to achieve, we started off with a "vision" (but see this post from a couple of weeks ago) and "brainstorming" to get a list of topics. Then we "asked the audience" with developer and advisory board surveys to see which of the topics they liked most. Next, we threw together some rough notes about each topic and then produced a first draft of each topic document.

The next stage was multiple reviews. After each review we restructured the content, and often the whole document, to get it closer to the ideal. But, because it's agile, we often changed our mind about what the doc was trying to achieve (remember, there's no detailed implementation notes to guide us here) and completely rewrote it. And then did this again after the next review pointed out the holes left by (or introduced by) the previous reworking. Some documents went through four or more complete restructurings, and several were rewritten twice.

The agile process also resulted in some of the "envisioned" topics being abandoned, often after they'd been rewritten several times, because - after investigation - they were too hard to define accurately. Or because they turned out to be not really relevant or practical. And as there is no overall structure plan, it was hard to see which topic should contain what section of the content, and how it related to other sections. It also meant that the focus could only ever be at the individual document level rather than the entire guide level, because that isn't defined yet.

But what we could be sure of was that each individual topic was precise, accurate to the nth degree, and compact with no irrelevant content. This is, of course, extremely important; especially if the content will be used as reference information. But is it still "guidance?" I guess that's the core of the problem. What exactly should "guidance" look like?

Typical equivalents of the word "guidance" include the obvious ones such as "help" and "advice." However, there are broadly two categories of meaning: "leadership" and "assistance." These almost seem like opposites - one leading from the front and the other pushing from the back. Yet the sub-meanings according to my thesaurus include "direction", "support", "management", and "control." Some of these seem more like they are related to aiding through understanding, whereas others are more related to despotic regulation. I'm going to take a guess that we're aiming for the first of these.

So did we end up with what we wanted, and does it aid through understanding? It's not completely finished yet and it will be a while before we see any user feedback. And there's no doubt that the content will be extremely useful for the specific users and use cases it addresses. But it still seems like we missed opportunities. The agile process narrowed the focus and transformed the content based on individual reviews of segments, and forced additional depth of detail. It also removed a lot of the general "understanding" content because it already was familiar to the experienced reviewers. Most of all, it resulted in huge amounts of extra work writing and repeatedly updating (and then sometimes discarding) the content.

Without a predefined (if flexible) structure and an overall feel for how it will all fit together and appear to the readers, there is nothing to prevent this wandering. As a writer, I'm lost if I can't see the finished thing in the back of my mind. It's like driving through a city in your car while blindfolded, and navigating by reversing and choosing a new direction at random every time you hit something.

Maybe agile is good thing that can help to focus guidance more accurately. Or maybe it just allows the guidance to wander away from the original vision and risk irrelevance. If the original plan was to provide guidance around X and you end up with fabulous documentation of Y instead, did you do a good job?

Comments

• Anonymous
  December 20, 2013
  What if you provide guidance about X and no one wanted guidance about X, they wanted guidance about Y, did you do a good job?To label rewrite as huge amount of work is one thing, but if it better fits the value it is not wasted to work, polishing something no one cares about is wasted work.The is the same as top-down vs bottom-up design dilemma, also the "work in my bubble with focus" or "be exposed to feedback early".The trick is that the feedback loop should be quick, the rework should be many small changes, not complete rewrites.If you get to complete rewrites you must have had the wrong vision to start with, or the wrong target audience, or the wrong audience reviewing.One way to look at it is that you should investigate why you needed to completely rewrite and why it was a lot of work. Perhaps you did too much up front work before getting the first feedback, wrote too much, so it required too much work to rewrite the "wrong" things you wrote.If you required a lot of rewrite, I would actually wages a guess that you should be happy you did it the agile way, but perhaps you used the wrong review audience too, not matching your target audience.Whatever problem you end up having, I don't think Agile caused it, it usually exposes problems, it doesn't solve them.
• Anonymous
  December 21, 2013
  Thanks for the insightful comments Niclas. My point is not that agile doesn't work - I've seen it be very successful in building code. My concern is using the same techniques when creating guidance documentation. OK, if you are documenting a software product, then the docs need to evolve with the product. But when we create non-product related guidance (usually architectural or design guidance) we typically have a specific area and scope in mind that is determined from user questions, feedback, and research over a problem area. However, starting with a vision but with no firm plan, and allowing it to evolve along agile lines, sometimes seems to mean the focus shifts and the result does not end up meeting the original vision or requirements.My take is that the problem arises when the focus and detailed reviews are on the actual content at a low level, and there is insufficient attention paid to staying on course towards the original vision because the plan is not developed first to manage the direction and evolution. Yes, if the evolution reveals you actually asked the wrong question at the start then reworking to a new plan is fine. But if the question is still proven to be valid, but evolution steers you somewhere else, it seems like you lost control and ended up answering the wrong question.