Abstractions: You can have too much of a good thing

Architects love abstracting things. And why wouldn't they - it allows you to hide those pesky implementation details out of the way so they don't trouble your callers, and lets you completely change the implementation at a later stage, provided the interface isn't changed. But like most of the good things in life, there comes a point where more is no longer better.

All of the Enterprise Library application blocks include great examples of abstraction, generally through the use of the Factory and Plug-In patterns. An application using an Enterprise Library application block need only code against an interface or abstract base class, such as Database, AuthorizationProvider or Validator (or in some cases this is abstracted even further via a façade class such as Logger or ExceptionPolicy). The concrete implementation of each class is determined at runtime by a factory class, which will do things like inspect configuration files and attributes to figure out which one is appropriate. This pattern provides a lot of great benefits, as you can write code at an abstract level, such as "validate this!", "log this!" or "call this stored procedure!", while the specifics of how each of these is done can be encapsulated into a different class and changed without impacting your code.

Over the years I've spoken to a lot of people who wanted to (or actually did) take this a step further by completely abstracting away the use of application blocks by hiding them behind their own interfaces and factories. The main arguments I've heard for doing this are as follows:

1. It provides an insurance policy just in case you ever need to get rid of Enterprise Library, by letting you build your own alternative implementation without requiring you to change your application code
2. It lets you build reusable assets that can be used by different users who may or may not want to use Enterprise Library

I appreciate that these goals can be important for at least some people, however I'm far from convinced that abstracting away an Enterprise Library block is a good solution. David Hayden started a vibrant discussion on this topic around achieving this second goal in the Repository Factory, and continued it with a number of blog posts (here, here and here). Persistent though David is, I don't buy his arguments. This is not because I think the blocks are perfect and that nobody would ever want to use anything different. However since the blocks already expose an abstracted interface, any further abstractions would need to expose an almost identical interface (or potentially a much watered-down subset). David didn't share his example, but I'm assuming he built something like this for the DAAB:

public interface IDataAccessor{    int ExecuteNonQuery(string command, params object[] parameters);    IDataReader ExecuteReader(string command, params object[] parameters);    DataSet ExecuteDataSet(string command, params object[] parameters);    // Add as many more DAAB-like members as you want}public static class DataAccessorFactory{    public IDataAccessor Create(string instanceName)    {        // Look up configuration and figure out what to return    }}public class EnterpriseLibraryDataAccessor : IDataAccessor{    // Wraps an EntLib Database instance and defers interface calls to it}

This solution meets David's goals of breaking the dependency between the client code (such as the Repository Factory) and the Data Access Application Block. But let's think a little more about what we've achieved here. We've replaced the DAAB dependency with a dependency on a new interface which is almost identical to the DAAB. Any arguments around why the DAAB dependency may not be desirable should apply equally to this new interface (plus we've added more complexity and more moving parts without adding any new functionality).

Even putting this argument aside, another problem I have with this approach is that I can't really see anyone being able to build any other implementations of this interface that are different enough to be interesting. I'm not saying there aren't other interesting ways of accessing a database - solutions such as NHibernate and LINQ to SQL are great examples. But these are philosophically so different to DAAB that they couldn't possibly implement the same interface that was built primarily as a DAAB Abstractor.

The only way I can see to avoid this problem is to build an interface so abstract that it provides almost no functionality. Using Logging as as simpler example, one could build an interface like this:

public interface ILogger{    void Log(string message, string category)}

...and build different implementations that could talk to the Logging Application Block or Log4Net. But we were only able to do this because we dumbed down the interface to the lowest common denominator. Both logging solutions use quite different approaches for things like routing, filtering, formatting, and these decisions influenced the design of their native interfaces. In our desire to provide a single interface that is abstract enough to work with both solutions, we're going to lose a lot of functionality. I'll wager that not many developers are willing to give this up for the theoretical goodness that this additional layer of abstraction provides.

To finish off this discussion, I wanted to add support to Chris Tavares's observation that David's problem is really a design-time one, not a runtime one, meaning that once you've decided what classes you want to use to talk to your database you're unlikely to want to change your mind after deployment. While the abstraction patterns we've been talking about do allow for both design-time and runtime flexibility, Software Factories provide some additional options for design-time variability, such as code generation, that are probably more appropriate in this case. The nice thing about going this way is that at runtime you only have the code you need - and you have just the right amount of abstraction to provide a great flavour, and not so much to cause a stomach ache.

Comments

• Anonymous
  August 25, 2007
  I think the actual problem arised here is about when people adding one more abstraction layer or hiding app-blocks behind the interfaces, they usually makes not well-thought interfaces. Most of time when coder want to hide something behind interface (with "books says to do that" motd banner), he just copy current methods signatures into the interface and continue working with it. It's not adding anything to breaking dependencies because there is nothing about abstraction for current class to more abstract one. Its just another copy-pasted layer. Imo this is one of the differences between developer and coder - developer can imagine consequences of the code he writting.

• Anonymous
  August 26, 2007
  Tom, most of your arguments are quite valid, but there's at least one good reason for abstraction away Enterprise Library: Until now, p&p have been releasing software faster than most enterprise organizations can keep up, and they don't guarantee backwards compatibility (or at least: That's the perception). Many development organizations have significant investments in software that uses a particular version of EntLib. As soon as this software has been released, it enters maintainance mode, which means that it will typically require great effort to change the version of EntLib (or so the organization thinks). Abstracting away such implementation details as EntLib (or Log4Net, or anything similar) will make it easier to move code in maintainance mode forward and keep it current. Yes, there's a price to pay, but there are also benefits. In a single software project, it may not make sense, but it might when you look at the entire application lifecycle ecology of a larger organization.

• Anonymous
  August 26, 2007
  Tom, I agree with Ploeh here. In our organization we deal with clients who have their own application blocks/foundation services, few clients who use EL (multiple versions) and few clients who use other blocks like Log4Net. It makes sense for us as a huge organization to standardize on the interfaces for basic foundation services (Data access, Logging, Exception Handling, Authorization etc). Then create adapters for each of the current blocks. Eg Adapter for Log4Net as well as EL logging etc... Currently we have adapters for EL 2.0, EL 3.1 and few of our client frameworks. We have our own foundation services as well. This abstraction helps our huge developer community in our organization a single common interface. Our group works on providing the adapters which ensures to use the latest features of the changing application blocks ( EL 2.0 to EL 3.1 ). We do not break the interface. This would ensure that we have a pure plug and play option here. So, I think it does make sense in our situation. I do agree that we end up not supporting some fucntionalities at times but its worth the flexibility if offers for a huge organization like ours. Thanks, Vyas

• Anonymous
  August 26, 2007
  Good post, Tom. If I always wanted to use Enterprise Library and being dependent on it wasn't a big deal, I agree with you 100%. There is no reason to add another layer of abstraction in that case. Unfortunately, this is not my world. I don't or cannot always use Enterprise Library and therefore I need more of a pluggable model around the data access in the Repository Factory. This pluggable model also insulates me from version changes in Enterprise Library, which has been a real issue for me when dealing with the software factories. This is why I chose to define a data access contract in my version and use the Adapter Pattern to isolate Enterprise Library as just a provider, not a dependency. My situation is very much similar to what Ploeh and Vyas are describing above. In my case, I am not adding an unnecessary layer of abstraction, but doing it very much based on the needs of my clients and their applications. I hope this at least helps clarify my position. As to whether this is important to do as part of the Repository Factory, perhaps not. I was only making a suggestion based on my needs and am not necessarily saying that these needs ring true for all the customers of the Repository Factory. Thanks, Dave

• Anonymous
  August 26, 2007
  Good post Tom, strong argument Ploeh. My 2 euro cents: Adding an abstraction can be justified in 2 scenario's.

1. I solve a problem, which I would like to make easier to consume. E.g.: "The people in my company find this library too complex to figure out, I'll make this easier by adding another abstraction level". To me this is valid, even with EL. Remember, Enterprise Library is build for 80% of the scenarios, with the remaining 20%, there might be scenario's or team's in which EL is perceived as complicated to consume.
2. I use a library someone else made, which I don’t want to take a binary dependency on and hopefully localize the change in case I find a library that does this job better (or upgrade to a new version for the matter of argument). David, this is where you are, right? I think this makes sense; thought a clean abstraction is very hard to come by. Following this argumentation, you would typically have to start with the consumer of the library to define the contract. In many cases (not all, say 80% :P), this type of abstraction is not worth the effort.. Since it not only implies creating a new contract for each consumer (or standardizing within an org.). This also implies you'd better think it through rather well, or else, after switching libraries you’d have something that smells of A, but actually behaves like B. Creating yet another "1-solution-fits-most-scenario's" - abstraction seems like a big waste of time, to me. (assuming you are not re-targeting this to another audience).

• Anonymous
  August 26, 2007
  To that I'll add: I would not consider "Software Factories" re-targeting to another audience. with "re-targeting" i was thinking more along the lines of DLR, CopmactFx, maybe an IoC application host, whatever. I also do not think it is possible to standardize the logging needs of Software Factories in general either.

• Anonymous
  August 26, 2007
  The comment has been removed

• Anonymous
  August 26, 2007
  I started using EntLib at 1.x and have migrated many projects from one version to the next.  The only time that I had problems was from a version of 1.x customized in-house to work better with .NET 2.0 to 2.0 of EntLib. Even then, the dozens of projects migrated in a few weeks, not months.  Every version since has been an easy re-compile, re-test, release. I don't buy that there is a need to abstract away EntLib in an organization to ease migration or enable use of multiple versions. I can think of only one case where it'd make good sense to put an abstraction in front of EntLib:  to replace the implementation of an interface that's used pervasively in a existing code-base and only in the case that the code-base is sufficiently large or brittle.

• Anonymous
  August 26, 2007
  The comment has been removed

• Anonymous
  August 26, 2007
  But David, you haven't eliminated the dependency on Enterprise Library - you've just moved it down a level. In order to make the factory work with something that's not EntLib, you still need to build and test new code. Sure, you won't need to recompile the data access layer to do this, but so what? I appreciate that there are users that can't or don't want to use EntLib, but I don't understand why you're dismissing the design-time solution (replacing or augmenting the code gen templates) that Chris suggested. Tom

• Anonymous
  August 26, 2007
  I think David is spot on.  Having a Repository pattern in your application without dependencies on Ent Lib is not a bad thing. Seems to be an attachment to EntLib from it's makers whereas the user community would see value in this. I know I would.  That is all that is needed.  +1 to David. To me, the same argument can be said about IoC - I would rather use Windsor in my applications.  I should be able to plug that into the EntLib or detach it without concern. Loose coupling is a positive thing  :)

• Anonymous
  August 27, 2007
  I think you are right to be concerned, but I don't think that the answer is "dont' do that" Take a look at Castle Project's Castle.Core.Logging namespace and notice their version of ILogger is an abstraction for their own logger. Then take a look at their log4net and Nlog Services. It is really great to abstract away both log4net & NLog and use only Castle Core's ILogger and simple log implementations. Then you can pull in log4net and/or NLog if you need them. IMO, it is a great example of abstraction done right.

• Anonymous
  August 28, 2007
  The comment has been removed

• Anonymous
  September 02, 2007
  One new subscriber from Anothr Alerts

• Anonymous
  September 11, 2007
  The only things I tend to hide behind a custom interface are static facades. It helps with mocking/unit testing.

• Anonymous
  September 12, 2007
  The comment has been removed

• Anonymous
  September 13, 2007
  What David Hayden has done with Repositories is fantastic. I've listened to his podcast, seen his examples.  I might actually start using ETLib now. I agree with Steve above, allow substituting Windsor in place of ObjectBuilder along with David's work on Repositories and we'll actually have software I'm interested in using.

• Anonymous
  September 13, 2007
  By the way, this is a pattern that you can read about from Martin Fowler.  EntLib allowing Domain driven designs patterns is fantastic. (I'd like to see his Repository factory sit on top of NHibernate)

• Anonymous
  September 22, 2007
  I couldn't agree with you more Tom, in fact I think some of p&p stuff is too abstract itself.  I've been buried in bugs caused by abstraction upon abstraction whose sole reasoning was 'what if' scenerio's.  And you get down to it and there's no quantitative purpose behind all the extra layers that were introduced; it just cost the company a few extra hundred thousand dollars because a few developers thought it was cool.  But don't get me wrong; abstraction is needed, coupling is bad, but like spices in a dish a little is good, a lot ruin the meal. So often I run across software that is so fricken bloated because of all these added interfaces because someday someone may up and decided to change the core architecture of the system without consulting the business or stakeholders who sponsor the project.  Bah.  If you're worried about that level of change in you're architecture then someone obviously failed to analyze, document, or enforce the strategic enterprise architecture requirements.  I've found most of these 'problems' where stuff just happens to change with the wind is because of lack of or poor project management or the project is run by coders who are hip to all the new technologies and want to keep up with change rather than properly retiring the current version and putting a new version in place...again strategic enterprise level requirements.  But yeah...requirements...who needs those right??? I'm going to paraphrase something Martin Fowler said in his book 'Refactoring' - speculative generaliteis are when a coder adds something or some architecture because 'we may need it someday' - if it's not needed it's just adding extra complexity and maintnance costs.  On the side, it's interesting that so many people who add so many layers of abstraction are big fans of Martin Fowlers Enterprise Patterns (or similiar books) and this is why they do it (because the book said so, which was said earlier), yet so few of them has read a book that is so much more important.