I’m going to depart some from what I have been posting and touch on a topic that will get a little technical because it involves technical things.  Read on even if you are a non-technical kind of person; I promise there will not be any code.

A big part of what I specialize in is distributed systems design and getting computer systems to talk nice with one another.  As a whole the IT industry has been doing this for a very long time so nothing I do is inherently new or revolutionary.  But the way we do it has evolved over time and we’re now in a paradigm which, I think, captures the essence of what computers systems really should do: represent the ‘real’ world processes we are automating.

Because I’ve made a niche for myself in this arena my boss delegates a lot of data sharing projects to me, if at least to review and make recommendations on what approaches we could take.  In the Justice and law enforcement domain data sharing is the hot topic since 9/11 and believe me when I say a lot of data is finally being shared.  This year alone I think I’ve reviewed around 15 new interfaces and we are implementing about a dozen.

I am currently the lead on four interfaces that will be created to share data with an external vendor for a system migration that is happening for one of our clients.  Originally the interfaces were planned with some of our legacy integration patterns but we have since chosen to implement them in newer technologies that will have a longer sustainable life.  Our team in particular has a lot of systems tied to our mainframe which limits our methods of interfacing with other systems and long term will cause an issue because most schools aren’t teaching COBOL as a mainstay of their MIS or CS programs. 

I like to look at the future as well and keep those things in mind as we go forward sharing our data and enabling new interfaces.  As I said, nothing I suggest is inherently new or revolutionary, but hopefully it is sustainable and represents the real world processes in a meaningful way to all the partners in the project.

The four interfaces I have been assigned were four that we had identified as candidates for a Service Oriented Architecture approach using web services or another service such as a queue.  Due to resource constraints we were not going to work on these for some time; until this migration came up.  Because a client needs this immediately we will be able to shift resources to make it happen.  This is where the balancing act begins.

We are integrating with a vendor that we have not worked with yet.  Most of my experience with external vendors has either been with the Department of Justice (which is going really well) or with a few other local vendors, one who’s idea of a project plan consisted of "Just send us what you have and we’ll take care of it" (80 work hours later they finally sent us their data element requirements).  My organization has a lot of experience doing integration and as such we can be seem like a big contract driven beast, but that is from a lot of experiences where the plan resembles "send us what you got" and ends up in huge disputes and a round or two of the blame game.

In our final proposals to the client and vendor we changed our approach to use web services.  A decision that myself and a senior staff member (as in 44 years with the organization) agreed would be best long term, even though he doesn’t speak XML or web services, he understood why we would want to use that approach.  I believe this to be the best choice we can make given all the other variables.

What is beginning to happen now is the inevitable interface creep.  Because every computer system has certain idiosyncrasies in how it works with data or represents those processes things get really fun when you start connecting two systems, especially when the domains you work in already have a history of differences i.e., courts and law enforcement agencies. 

Many of the legacy interfaces I see were developed for specific purposes and for specific systems.  Which really is fine because they get the job done.  System A shares it data to System B and everyone is happy.  But then system C comes along and it has different idiosyncrasies from System B and so System C demands that System A accommodate for those.  The result: Someone copies the code from System A and modifies it to share data with System C thus resulting in two interfaces which share the same strengths and flaws.  And someone has to maintain those.  Indefinitely.

I want to prevent the situation above with Systems A,B, and C from starting at all.  How will I do that?  System A will expose a consistent abstraction of it’s data with a standard contract that is unchanging.  The data will be shared in a meaningful way that, hopefully, will accurately reflect the most common business processes that system performs.

That last sentence is key to this whole balance act.  It is your job as the architect to ensure that your systems interface accurately reflects the processes at a high enough level of abstraction that it is useful to any external parties even with all the little idiosyncrasies their system and your system has.

Undoubtedly as this project goes on requests will be made like "How about you always send us X with this response for this operation and Z for this one?"  This is where the architect has to mediate, protecting the integrity of the external interface while allowing the flexibility to support the needs of known and unknown third parties who need your data.  It will be a delicate act and one that I will be walking once again over the next weeks and months. 

I can’t prescribe a one size fits all approach because a lot of this is situational.  Is it an existing vendor or partner who has worked with you?  Then they most likely understand.  What if it is a new partner?  What if they have no way to speak to you using the same technologies (web services, what are those?). 

As we continue this paradigm shift from one model of systems to the next these situations will continue to put architects in the middle of balancing the integrity and consistency of their own interfaces, growing those systems into the futures in a sustainable way, and all the while meeting contractual obligations.

If you joined technology to get away from these kinds of problems it might be time to find a new job!