Unforeseen Costs of an encapsulated approach: We need to abstract, encapsulate and hide complexity in order to build complex behaviour This is why we have OO, SOAP, web services and other SOA approaches. But for all the benefits they give I see practical disadvantages emerging in the real world. As we sew more encapsulated objects together there is an associated management cost, which quickly runs out of check when ignored or put aside for other priorities.
I’ve termed the act of encapsulating, blackboxing or providing a service as an ‘Island’ (aka information island). I’m not fond of the term but its seems apt. Islanding occurs in reality and frequently within enterprise environments. Different ‘information islands’ are joined together with the expectation they will fit like a neat little jigsaw – only they don’t. Here are some of the reasons I see the pieces not joining neatly together as one would expect:
Replication: Repeat checking, Performance
Often there is duplication replication of functions within separate islands. A simple example of this is transferring information between islands through XML, also functions may be repeated to wrap up information into a format that is easier to pass around. Passing information between Islands means the Islands aren’t quite sure if its been syntactically correct or validated. This creates overhead, as multiple same checks for the passage of data through the system is performed.
Demand: Disregard for execute time, Performance
Example: If I require an array of objects from an Island, should I ask for one at a time or all at once? In most cases all at once, however in the real world this is down to the experience/skill of the requester. I’ve seen plenty of cases where a service has been provided and the external user to the island has placed unnecessary high demands on the island. When you black box its easy for the requesters to lose respect for efficiency, resulting in a higher than necessary overall execution time.
Abstraction: Loose of Understanding, Maintenance
I think some items can be abstracted too much. Too much abstraction can mean its difficult to understand what should be a simple function (e.g Date formatting). I’ve lost count of the times I’ve had to open 20+ screens to understand a very simple process or function. Anything with a complexity function depth of 5+ inherently starts to introduce code risk. There is a little irony here we need to encapsulate to build complexity, but the more we encapsulate (or abstract) the harder it is to understand and there is an associated risk of defects being introduced.
Deluge: Overhead, CPU cycles
If a service requires a small piece of information, but an Island returns that information with an associated large amount of ancillary information (which is swiftly disregarded), this is wasteful. I’ve seen this happen because internal teams simply haven’t communicated well. Services are built and then requests for small pieces of information result in a large XML being returned, much of which is redundant. This places a large amount of demand on run-time memory, the Island’s DB and unnecessary processing of the XML. Islands do not know exactly what is going to be required (or how frequency), so they overengineering of return data which then become redundant
If you open your island up to the external world, there is code overhead – you have to actively police the requests coming in. Throttling, authentication and ensuring the business as usual system isn’t being affected. If some form of policing and control isn’t put in place someone can misuse unintentionally.
Loss of Control: Complexity: Execution
If information is sent to an essential external Island but and no response is received what should happen? If I get a response back that is outside my SLA, how do I prove? Is it my service?, the other service? or something in-between? How do I replicate? Relying on external services by third parties (such as Credit Card) introduces a whole raft of difficult issues should it not perform as expected. Black box islands are fine when they work however they can be incredibly tricky to isolate as a source of issues if you have no control or visibility.
There are other costs that are overlooked … I’ll add as I think of more.
Wasted CPU cycles are BAD.
Some of the above items result in wasted or duplication of CPU cycles, these quickly add up to a large performance overhead and an associated large financial cost, either in lost business, developer re-engineering time or associated hardware costs.
With ‘islanding’ an unintentional loss of respect for CPU cycles naturally occurs, system behavior and architectural respect is naturally lost. High demands and a loss of performance are symptomatic of this. This gap is why APM solutions have grown and become such large businesses – customers need help managing the OO jigsaw.
Many of the above instances can be prevented with oversight, having someone responsible for a system wide view (a performance by design role) – working across the project and communicating within all teams. However I’m seeing a tendency for teams to increasingly work in isolation – either because they have high demands on their time, they are geographically dispersed and the effort to communicate becomes difficult.