Tribal Knowledge and Engineering Don't Mix

In many engineering teams, having tribal knowledge is normal and accepted. This is when information about a product, process, or technology owned by the team is described verbally from person to person, release after release, without writing it down. Or if it is written down, only a select group of people know where the documentation is. Tribal knowledge causes a ton of problems for engineering teams, yet it still continues to exist. Know why? Job security. People seem to think that if they know something that few people know, then they have job security. But what happens when that knowledge is needed by a person who didn't know that anyone had that information? Here are lots of reasons why tribal knowledge does not belong in an Engineering team:

Information gets lost. Over time, people with tribal knowledge will forget what they specifically know vs. what is common knowledge to everyone on the team. Then it becomes difficult to write it down or teach it. How do you document all the pertinent information from a person who may have had that same job for years? It's nearly impossible.

No vacations. I find people with the most tribal knowledge are the least likely to be able to take time off from work. If they aren't there, things just can't get done since they are the only ones that know how to do it.

Bad decisions get made. If a person with the tribal knowledge designs features using that knowledge, the design may not withstand the test of time. As time goes on, new people come to the team and add other features and dependencies into the code that turns the design into a mess. If the design was documented and reviewed so more people saw and understood it, fewer problems would occur later on. Instead, poor decisions are made because information just isn’t available to the new people who need it. And the subtleties of the design are now added to that tribal knowledge.

Information cannot be tracked. When people share their knowledge verbally, they put the responsibility on the person listening to now remember all that was discussed. Typically, engineering teams have a lot going on (which means a lot of information is flowing) so that is too high of an expectation to put on anyone. If the person giving the information isn’t documenting it, the person receiving the information should be.

Teams become dependent on reverse engineering. If the people that are in-the-know leave, then reverse engineering is the only way to understand what was done. Code becomes the only form of documentation and that leads to a lot of misinterpretations and lack of understanding the creator’s intentions.

Poor workload balancing. Job responsibilities need to be transferrable in order for teams to have the flexibility to meet varied project scopes and schedules. With tribal knowledge, the team becomes very limited on who they can assign to their different tasks because only certain people have certain information.

Management becomes falsely dependent on individuals so instead of business requirements driving project deliverables, management has to factor the tribal knowledge of the team into those deliverables. If the business requirements involve using a product technology that only one person on the team knows about, then the deliverable becomes solely based on the availability of that one person.

I believe all engineering teams deal with tribal knowledge to some degree and it is a burden. It hinders the efficiency and flexibility of the team. Although I don't expect engineers to be tech writers, I do expect enough documentation to allow an engineering team to function properly through information tracking and discovery, and not by word-of-mouth. How well does your team handle tribal knowledge?

Comments

• Anonymous
  July 03, 2010
  The corollary is that engineering team managers, project leaders, and management must budget time / cost to write down --  in a structured, accessible manner -- the engineering details of the system.  Beginning from the documentation side by writing requirements and then writing tests to requirements, using test driven development helps.  Another method is to insist engineers document at their attention level using an appropriate framework (PHPdoc, Javadoc, etc), then have the requirements managers and architects police up an evolving documentation suite for the system.
• Anonymous
  July 04, 2010
  Excellent write up...so true... thanks for taking the time out to write this up! I wish more of the team leads put their awareness caps on and tried to reduce this...
• Anonymous
  July 04, 2010
  @K. Geiger - I agree that managers need to budget time.  Another method to help improve this is to add an accountability around documentation to engineers' review goals and commitments.  If they are going to be evaluated based on their goals, you can drive many good behaviors (beyond just documentation) by setting those expectations clearly.
• Anonymous
  July 04, 2010
  @rpatel - thanks!