Architecture

Modularizing WCF with Unity

For a new project at work I thought it best to use dependency injection to deal with some complexities we were facing.  I also wanted to get in and do some mocking in our unit tests to get a good feel for that.  Since we use the Enterprise Library I decided it would be easiest to roll with using Unity.  I have previously blogged about using Unity but this would go even further, because we would be injecting a dependency into the constructor of the service as well as injecting dependencies into the service’s core library.

I did quite a bit of homework and reached the conclusion that in order to do this you would need to hook into WCF’s pipeline and override several key areas during the service host creation and service instance creation.  The best article I found to get me started was this one by Steve Moseley.  Steve broke it down into 6 steps:

  1. Creating a custom instance provider that resolves the service
  2. Creating a custom service behavior to plug in the new instance provider
  3. Creating a custom service host that will add the new behavior
  4. Creating a custom service host factory (which configures your Unity Container)
  5. Changing the service host factory in the .svc file
  6. Inject your objects (through configuration)

His instructions were fantastic for steps 1-5 but step 6 left me to think on my own a bit.  The constructor of the new service we were working on needed an instance of its “core” library which will be the real work horse for the service.  Typically our services only provide a gateway into some other functionality so the service layer handles messaging and translation then delegates the work to a core library which can do whatever it needs to. 

This service would also be a little different than most because we would be relying on both internal and external “suppliers” to provide data to our core library.  These suppliers could be in the form of remote web services, local services, or shared assemblies.  Our core library itself will also need it’s dependencies injected at runtime.

Step 6 involved a bit more detailed study of Unity in which I did the following:

  • Defined typeAliases for each of the dependencies I would be injecting
  • Defined type mappings for the simple dependencies which had only a default constructor
  • Defined typeConfigs for the complex dependencies which required other dependencies passed to their constructor

I chose the constructor injection route because the service should not be created without a core library and the core library should not be created without its supplier dependencies and data access layer.  I believe I could have accomplished something similar had I used default constructors and exposed read/write properties for the dependencies.  This would have removed the constructor requirements but potentially caused issues if someone were using this class without supplying the dependencies to the read/write properties.  So, constructor injection wins for now.

In the end the config looked something like this:

<configSections>
  <section name="unity" type="Microsoft.Practices.Unity.Configuration.UnityConfigurationSection, Microsoft.Practices.Unity.Configuration, Version=1.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
</configSections>
<unity>
  <typeAliases>
    <!-- Lifetime manager types -->
    <typeAlias alias="singleton" type="Microsoft.Practices.Unity.ContainerControlledLifetimeManager, Microsoft.Practices.Unity, Version=1.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
    <typeAlias alias="external" type="Microsoft.Practices.Unity.ExternallyControlledLifetimeManager, Microsoft.Practices.Unity, Version=1.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
    <!-- User-defined type aliases -->
    <typeAlias alias="ISupplier" type="MyService.Contracts.ISupplier, MyService.Contracts"/>
    <typeAlias alias="MyService" type="MyService.Service.MyService, MyService.Service"/>
    <typeAlias alias="ICore" type="MyService.Contracts.ICore, MyService.Contracts"/>
  </typeAliases>
  <containers>
    <container>
      <types>
        <type type="ISupplier" mapTo="MyService.Suppliers.ExternalSupplier, MyService.Suppliers" name="ExternalSupplier"/>
        <type type="ISupplier" mapTo="MyService.Suppliers.InternalSupplier, MyService.Suppliers" name="InternalSupplier"/>

        <type type="ICore" mapTo="MyService.Core.MyCore, MyService.Core" name="MyCore">
          <typeConfig extensionType="Microsoft.Practices.Unity.Configuration.TypeInjectionElement, Microsoft.Practices.Unity.Configuration, Version=1.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
            <constructor>
              <param name="externalSupplier" parameterType="ISupplier">
                <dependency name="ExternalSupplier"/>
              </param>
              <param name="internalSupplier" parameterType="ISupplier">
                <dependency name="InternalSupplier"/>
              </param>
            </constructor>
          </typeConfig>
        </type>

        <type type="MyService" mapTo="MyService.Service.MyService, MyService.Service">
          <typeConfig extensionType="Microsoft.Practices.Unity.Configuration.TypeInjectionElement, Microsoft.Practices.Unity.Configuration, Version=1.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
            <constructor>
              <param name="myCore" parameterType="ICore">
                <dependency name="MyCore"/>
              </param>
            </constructor>
          </typeConfig>
        </type>
      </types>
    </container>
  </containers>
</unity>  

Once this was working successfully I took a step back to the unit tests project to do some mocking.  I know someone out there is thinking “why would he try that before unit tests?   Couple reasons: 1) This was an entirely new endeavor and its easier for me to get the final concept working 2) The first 5 steps from Steve’s article required some low-level work to hook into WCF.  Rather than unit testing that I decided to flush it out by actually firing up a sample service to work out the kinks.  Next time the unit tests will be done first…

The Calm Before: New Role, New Challenge

I have been quiet the last few weeks though I have been meaning to post a few things.  Since the beginning of 2009 I have been working to get several people on my team up-to-speed on Windows Communication Foundation and service oriented design which has proven to be no easy task.  Condensing two years of study and experience into short lessons has been difficult because there is no cookie cutter mold that you can hand someone so they can bake up some services.  Services take thought and consideration, planning, and careful implementation.  Giving someone a template to start from is helpful, but without understanding why the template is set up the way it is much of that experience and knowledge is lost.  I think this is largely a fault of my own because of the rushed feeling I have to get this knowledge transfer going into 2009.

On Friday the general manager of REJIS announced something that I think most people saw coming; the beginning of our technology migration plan to move from a Mainframe environment into a Windows based environment.  As he outlined it at a very high level we see the picture of a multi-year company-wide effort to make this happen.  This is a major shift for REJIS because we are known for our ?green screens? and our core competencies lie in the processes that are running on our mainframe systems.  REJIS will be redefining itself as it moves in a new direction to offer new products and services to customers.

Just prior to the announcement I learned the role I would play in this new endeavor.  I will be working on the small team that will design the overall architecture as the lead architect.  The team is currently comprised of three people including myself.  One is a Senior.NET developer and the other comes from the mainframe side though she has spent her more recent time studying .NET and assisting with project designs and documentation.  We will be working directly with our IT Director and probably every division of the company as this affects REJIS all the way from the top down.

To say that the coming months and years will be easy would be a gross understatement.  We will be converting a collective hundreds of years of peoples work to a new platform.  Some of my closest colleagues have literally spent their entire careers building these systems.  To call it a conversion is a bit misleading as well.  What is being converted is that knowledge and experience in the industry we serve and in that conversion will be a redefining of REJIS into a more agile company ready to handle the changes the future will bring.

For me this is the best opportunity I could be called for.  Over the last few years I have been one of several voices calling for better attention to architecture and design on our new systems and for constant attention to how new technologies will affect what we can do and how quickly we can do it.  Now will be my opportunity to make REJIS a little bit better and to prepare our IT systems for a future that will allow them to grow, scale, and change in the demanding environment we work in.

I press forward humbled that I will play such a big role in this change and looking to peers and colleagues for their support and advice.  People like David who never cease to amaze with their forward thinking, Clint and Denny who make their mark evangelizing the need for architecture and providing sound advice and wisdom from their experience, and my other (blogless) colleagues at REJIS who go along with my crazy ideas and add a little crazy of their own. 

The next few weeks will involve a lot of groundwork, some travel, and time to reflect on the road ahead.  It will be long and trying and for those of you who subscribe to my feed you will have a front row seat to the transformation that will happen.

Dependency Injection Part 2: Microsoft’s Unity

In my previous post on dependency injection I discussed a simple way to enable dependency injection using configuration and System.Activator.  The method described would be sufficient for many cases but may not offer enough power for every situation.  This time we will explore the use of an Inversion of Control container, namely Microsoft’s Unity Application Block.

Taking the "another tool in the tool box" approach let’s explore unity using our previous example of a system which requires multiple authentication methods.

I downloaded and installed Microsoft Enterprise Library 4.1 (October 2008) which you can download here.  If you do not need the entire Enterprise Library (EL) then you can download the standalone Unity Application Block here.  I highly recommend trying out the EL because it has a lot of very useful application blocks for common needs of enterprise developers.

The benefits of using a more powerful framework such as Unity are that you get additional features such as simplified object creation and lifetime management (can register objects as singletons) in addition to the other benefits of looser coupling and more flexibility.  You can also use caching to store containers and then retrieve them when you need to access it again.

I will be using the same solution and project as my previous post and we will start by adding a reference to the Unity assembly.  The first method I will use is to manually register types with the unity container.  The second will be using configuration to specify the container’s configuration and types to use.  The ultimate bench mark for me will be to determine which method uses the least amount of code and leaves us with the most flexibility to make changes.

Manually Building and Registering Objects with the Container

Following the guidance in "Setting Up the Unity Container" I came up with this code:

Module Module3

    Sub Main()
        Dim container As Microsoft.Practices.Unity.IUnityContainer = _
        New Microsoft.Practices.Unity.UnityContainer

        container.RegisterType(Of Authenticator.IAuthenticator,  _
        Authenticator.DatabaseAuthenticator)()

        Dim authenticator As Authenticator.IAuthenticator = _
         container.Resolve(Of Authenticator.IAuthenticator)()
        Dim userName As String
        Dim password As String
        Dim user As Authenticator.User

        Console.WriteLine("Username: ")
        userName = Console.ReadLine

        Console.WriteLine("Password: ")
        password = Console.ReadLine

        Console.Clear()

        user = authenticator.Authenticate(userName, password)

        If user.IsAuthenticated Then
            Console.WriteLine(String.Format("User authenticated via {0}!", _
              user.AuthenticationType))
        Else
            Console.WriteLine("User authentication failed!")
        End If

        Console.ReadLine()
    End Sub

End Module

This was a pretty basic setup and the learning curve was almost zero.  I was already using a similar method to set up my own dependency injection and this was a very natural seeming approach.  However, I am not entirely satisfied with this approach as it has constrained us to making code changes and recompiling to switch to a new implementation of IAuthenticator. 

Now this was a very basic and simple example so I wouldn’t say this is a fault of Unity but more a fault of me just doing the bare minimum to create a container and add an object to it.

Configuring a Container Using Configuration Files

Let’s look at using Unity’s configuration schema.

The big disappointment I had so far is that there does not appear to be a visual configuration tool.  I know, how sad, but I do like having those tools available as I think they can help reduce little mistakes you can make when editing XML.  The broader EL configuration tool is very handy and now that I am used to the structures it creates for things like logging and validation it is very easy to use.

After reading "Entering Configuration Information" and some trial and error I developed Module4 which will use a configuration to define the container.

Module Module4

    Sub Main()
        Dim container As Microsoft.Practices.Unity.IUnityContainer = _
        New Microsoft.Practices.Unity.UnityContainer

        Dim section As _          Microsoft.Practices.Unity.Configuration.UnityConfigurationSection _
          = CType(Configuration.ConfigurationManager.GetSection("unity"),  _
          Microsoft.Practices.Unity.Configuration.UnityConfigurationSection)
        section.Containers.Default.Configure(container)

        Dim authenticator As Authenticator.IAuthenticator = _
         container.Resolve(Of Authenticator.IAuthenticator)()
        Dim userName As String
        Dim password As String
        Dim user As Authenticator.User

        Console.WriteLine("Username: ")
        userName = Console.ReadLine

        Console.WriteLine("Password: ")
        password = Console.ReadLine

        Console.Clear()

        user = authenticator.Authenticate(userName, password)

        If user.IsAuthenticated Then
            Console.WriteLine(String.Format("User authenticated via {0}!", _
              user.AuthenticationType))
        Else
            Console.WriteLine("User authentication failed!")
        End If

        Console.ReadLine()
    End Sub

End Module

After adding the required configSection the unity configuration block looks like so:

<unity>
    <typeAliases>
        <!-- User-defined type aliases -->
        <typeAlias alias="IAuthenticator"
             type="Authenticator.IAuthenticator, Authenticator" />
    </typeAliases>
    <containers>
        <container>
            <types>
                <!-- Type mapping using aliases defined above -->
                <type type="IAuthenticator"
                      mapTo="CustomAuthenticator.XmlAuthenticator,
                      CustomAuthenticator" />
            </types>
        </container>
    </containers>
</unity>

I used the alias feature to create an alias name for the IAuthenticator.  This lets you reference it by alias rather than including the full type information every time you are mapping an instance of the type.

In this example I rewrote the same program I was using for my simple dependency injection to use the Unity Application Block.  The code changes amounted to about 5 lines of code for the simple use of Unity and 6 to use configuration.  The code to use unity was very plain and had no real branching logic in it so it could easily be abstracted to a helper class that would assist in creating the container and returning the instance to the caller.

The flexibility that you can gain by moving to a IOC framework such as Unity is very impressive.  Overall I think this will be beneficial to use as a more robust way to enable dependency injection. 

Next time I will cover a more real world example using Unity in the adapter pattern we use for our WCF service clients.  The basic model is that our service client calls the service and then initializes an instance of a "IMessageAdapter" and calls its Transform method.  The purpose of the adapter is to take the data in the services format and transform or adapt it to something else like a database, fixed length file, etc.  We use this model because it allows us to fully reuse the client and only requires that we create a new implementation of the Interface.

Dependency Injection: Modularize Your Services and Applications

There are many different patterns for objected oriented development that have emerged over the years.  More often than not I will stumble across one that we are inadvertently using in our services or applications at REJIS.  Most recently I wrote about an approach we use when we need a service client that will take data from a service and transform or adapt it to another format (database, xml model, fixed length file, etc).  At the core of that pattern is dependency injection.

Martin Fowler has an excellent (and lengthy) discussion on Inversion of Control and Dependency Injection which is well worth the read.  Wikipedia defines dependency injection as "…the process of supplying an external dependency to a software component. It is a specific form of inversion of control where the concern being inverted is the process of obtaining the needed dependency." 

For example lets say that you have an interface, IMyInterface, which Class A depends on for some functionality.  Class B and C both implement IMyInterface making them substitutable for Class A’s dependency.  At runtime, through configuration or programmatically, you can decide which concrete class (B or C) for Class A to use.  Rather than having Class A directly depend on Class B or C you reduce coupling by relying on the interface and then you inject this dependency at runtime.

Interfaces represent one of the best possible abstractions available to an object oriented programmer.  The benefits to using interfaces and dependency injection are numerous:

  • By using an interface you have defined a contract of expected behavior for implementers.
  • Interfaces reduce coupling, allowing for independent unit testing, and multiple implementations of the interface.
  • Code becomes simplified and focused by reducing branching logic and overall lines of code.
  • Changes to the application can potentially be applied at runtime (if this is a web app, windows apps would require closing and reopening) and would not require a recompile of the entire application (if the new implementing class is in another assembly only that assembly needs to be compiled).
  • The application will be more "plug and play" and allow for adding or removing functionality as needed.

MSDN has an article that discusses Dependency Injection in depth and how it relates to a few other Creational patterns such as factory.  There are a number of Inversion of Control (IOC) frameworks that assist with dependency injection that use the notion of containers which help manage the lifecycle of objects and apply configuration settings to modify behaviors at runtime.  For a list of open source IOC frameworks check out this post by Scott Hanselman.

Dependency Injection Example – Supporting Multiple Authentication Methods

I have created a sample project which I will use to illustrate the flexibility of this approach.  The sample covers a very common scenario where we must support multiple authentication methods to allow flexibility for our customers.  We’ll look at a traditional approach and then an updated approach that will use Dependency Injection.

At the core of this example is the IAuthenticator Interface and the User Class.  Both are very simple for examples sake.

Public Interface IAuthenticator
    Function Authenticate(ByVal userName As String, _       ByVal password As String) As User
End Interface

Public Class User
    Public Name As String
    Public AuthenticationType As String
    Public IsAuthenticated As Boolean
End Class

The customer requested that the application first be built to support Database Authentication.  Because our architect was thinking ahead and at least forced us to use an interface we will implement that interface in our DatabaseAuthenticator class.

Public Class DatabaseAuthenticator
    Implements IAuthenticator

    Public Function Authenticate(ByVal userName As String, _
      ByVal password As String) As User Implements IAuthenticator.Authenticate
        Dim user As New User
        user.AuthenticationType = "DatabaseAuthentication"
        'Simulate DB authentication
        If userName = "Test" AndAlso password = "Testing!" Then
            user.IsAuthenticated = True
            user.Name = userName
            'else
            'could throw security exception or any other action here.
        End If

        Return user
    End Function
End Class

To test our authentication we’ll go "old school" with a little console program action.  The console program will display a username prompt and password prompt then it will invoke our IAuthenticatorInstance to perform the authentication. 

Let’s first look at life without dependency injection:

Module Module2

    Sub Main()
        Dim authenticator As Authenticator.IAuthenticator

        Select Case My.Settings.AuthenticationType
            Case "Database"
                authenticator = New Authenticator.DatabaseAuthenticator
            Case Else
                Throw New ArgumentException("Invalid AuthenticationType!")
        End Select

        Dim userName As String
        Dim password As String
        Dim user As Authenticator.User

        Console.WriteLine("Username: ")
        userName = Console.ReadLine

        Console.WriteLine("Password: ")
        password = Console.ReadLine

        Console.Clear()

        user = authenticator.Authenticate(userName, password)

        If user.IsAuthenticated Then
            Console.WriteLine(String.Format("User authenticated via {0}!", _
                       user.AuthenticationType))
        Else
            Console.WriteLine("User authentication failed!")
        End If

        Console.ReadLine()
    End Sub

End Module

In order for Module2 to work we’ve had to hard-code a string value in our case statement to help us decide the type we want to create.  From there we instantiate the appropriate class manually.  This works fine; it will run and there are no problems with it…until you need a new authentication method.  Then you must go back and add a new class that implements IAuthenticator, add a branch to the Case statement and then instantiate your new class.  Of course you could do this without interfaces as well…but I digress. 

On to life with dependency injection:

Module Module1

    Sub Main()
        Dim authenticatorType As System.Type = _
         Type.GetType(My.Settings.AuthenticatorType)
        Dim authenticator As Authenticator.IAuthenticator = _
         Activator.CreateInstance(authenticatorType)
        Dim userName As String
        Dim password As String
        Dim user As Authenticator.User

        Console.WriteLine("Username: ")
        userName = Console.ReadLine

        Console.WriteLine("Password: ")
        password = Console.ReadLine

        Console.Clear()

        user = authenticator.Authenticate(userName, password)

        If user.IsAuthenticated Then
            Console.WriteLine(String.Format("User authenticated via {0}!", _
              user.AuthenticationType))
        Else
            Console.WriteLine("User authentication failed!")
        End If

        Console.ReadLine()
    End Sub

End Module

You probably noticed the first line where we dim up the authenticator variable by calling Activator.CreateInstance and passing it a type variable which I resolved using the Type.GetType method.  Type.GetType(string) returns the System.Type which we can then pass to activator to get our instance created.

Our settings file contains this section:

<applicationSettings>
    <DepdencyInjectionConsole.My.MySettings>
        <setting name="AuthenticatorType" serializeAs="String">
            <value>Authenticator.DatabaseAuthenticator, Authenticator</value>
        </setting>
    </DepdencyInjectionConsole.My.MySettings>
</applicationSettings>

The notation I used for the type information is the standard "Class, Assembly".  Note that I am not using strong names so it is not necessary to include version and public key information.  Also note that none of the code in Module 1 references the concrete class DatabaseAuthenticator.

You may have also noticed that, including whitespace, Module1 has 29 lines versus Module2’s 36 lines of code (excluding line continuations).  In our simple example 7 lines isn’t much but in complex systems that factor could be easily multiplied by 10 or more.

The code that was written for Module1 has no dependencies other than it has to know about Authenticator.IAuthenticator.  The true dependency is injected at runtime.  If we try to use a type in the AuthenticatorType setting that the system cannot resolve then our program will blow up. 

Which leads us to the paradigm shift; you have to be aware of your operating environment and ensure that the assemblies you need are available in the GAC or in your applications executing or bin folder (for web apps and IIS hosted services).  This does not negate managing your dependent assemblies but it shifts it from something necessary throughout development to something that could potentially only happen during deployment.  Because the code only depends on the Interface IAuthenticator two completely different teams could be coding the main module and an IAuthenticator implementation.  Development for Module2 is more susceptible to breaking changes because of direct dependencies on the classes that implement IAuthenticator.

Adding Another Authenticator Implementation

The customer you wrote this system for now wants to add authentication that will read an XML document for user accounts.  We create a new assembly called CustomAuthenticator and our new class looks like this:

Public Class XmlAuthenticator
    Implements Authenticator.IAuthenticator
    Private Shared authenticationStore As Xml.Linq.XDocument

    Sub New()
        authenticationStore = _
        Xml.Linq.XDocument.Load("C:\Temp\AuthenticationStore.xml")
    End Sub

    Public Function Authenticate(ByVal userName As String, _
      ByVal password As String) As Authenticator.User _
      Implements Authenticator.IAuthenticator.Authenticate

        Dim user As New Authenticator.User
        Dim userQuery = From u In authenticationStore...<users>...<user> _
         Where u...<name>.Value = userName _
         And u...<password>.Value = password _
         Select u

        user.AuthenticationType = "XmlAuthentication"
        user.IsAuthenticated = userQuery.Count = 1
        If user.IsAuthenticated Then
            user.Name = userName
        End If

        Return user
    End Function
End Class

The XmlAuthenticator uses an XML file to store user accounts and loads in its constructor.  Using LINQ we query the XML document to find if our user exists.  This class resides in a completely separate assembly. 

If you are adept at reading LINQ XML Queries you can guess the documents structure:

<?xml version="1.0" encoding="utf-8" ?>
<users>
    <user>
        <name>Test</name>
        <password>Testing!</password>
    </user>
</users>

Adding XmlAuthenticator to Module2 (Anti-Dependency Injection Module)

In order to implement this in Module2 we need to add a branch to our case statement:

Case "XML"
    authenticator = New CustomAuthenticator.XmlAuthenticator

Total cost – two lines of code, another hard-coded string, someone has to update the config file and deploy the new assembly.

Adding XmlAuthenticator to Module1 (Pro-Dependency Injection Module)

In order to implement this in Module1 we need to…do nothing.  Not programmatically anyhow, but we do need to update our AuthenticatorTypeValue in the app.config.

Here’s the new config:

<applicationSettings>
    <DepdencyInjectionConsole.My.MySettings>
        <setting name="AuthenticatorType" serializeAs="String">
            <value>CustomAuthenticator.XmlAuthenticator, CustomAuthenticator</value>
        </setting>
    </DepdencyInjectionConsole.My.MySettings>
</applicationSettings>

Total cost – zero lines of code and someone has to update a config file and deploy the new assembly.  Less work and less code to maintain.  Ladies and gentlemen I think we have a winner here!

This is a very simple example that hopefully illustrated what you can achieve using dependency injection.  The code is simply for illustration and I do not recommend or condone anyone using for their authentication components (nor will I be liable for anything you do with this)!

What are some real world ways we use this?

For service clients where we are responsible for retrieving data from a service and doing something with it we use dependency injection for what we our "adapters".  The adapters take a known type from a service and do whatever is needed; store data in a database, flat file, or another XML format.

For a service we are developing on a statewide data sharing project dependency injection is being used to allow for a custom implementation of activities within that service so that the service can be deployed to another geographical region in the state and customized via configuration.  The major functionality points were identified and abstracted with interfaces so the functionality could be implemented in different ways for the two regions.  This also allows a separate code base that the other region can maintain and update as needed.  The core functionality is completely similar between each region and should require no modification except any bugs that may surface.

We will also use this approach on another service to allow a custom "path" for a document that we will receive from an external source.  We have cases where, depending on what external partner sends us information, we need to do multiple things with it and this can vary from partner to partner.  This last example may require a more sophisticated method so I am leaning towards the Unity Framework from Microsoft as we already use Enterprise Library extensively in our services.

Another Tool in the Toolbox

The best way to look at all of these object-oriented approaches is that they are tools in your toolbox.  I will likely not use dependency injection everywhere.  But in the work I do with services we need this pattern more often than not.  Hopefully this gives you enough insight to decide if you may be able to benefit from using it in your own applications.

The Power of Naming

Today I was walking down to the lunchroom to get some Christmas cake with one of my teammates and he was telling me about these services he wants to create for the application he is now project lead for.  On the way up he said "Ok so you know how important names are so how about these…" and proceeded to name his services.  The names sounded good but I starting thinking about one of them.  He had called it the "ParcelUploadService".  After a minute I said "You know, upload is real ‘computery’, how about ParcelStorageService?"  His eyes got wide and he had that same look he gets when he talks about Silverlight and then he said "Oh I like that.  I’m changing the name."

Later on we were talking and I said, "If you think about it, I don’t upload files to my file cabinet, I store them.  So I think storage service is pretty good." 

"That sounds like a blog post to me."

Here it is Dave.  Now go start your blog I can link to it.

Exposing Services: Some Principles Revisited

In my opinion the principles driving service oriented architecture are a great evolution of software design.  I have been working to apply many of those principles in work we are doing to update our core offerings as services.  While the principles are widely published I wanted to add some of my own value to them from recent experience and reflection.

Services should enable business processes. If process can’t be completed "on paper" or no one understands it exposing it as a service may not be successful.  Services should expose that business process in a meaningful way that replicates the paper process as closely as possible.

Services aid in loose coupling between systems.  By exposing services with standard contracts that represent their messages in a canonical format you reduce coupling between systems.  The schemas and WSDL become the contract by which clients and the service communicate.  As long as that contract is not violated the service can do whatever is necessary to fill requests.  Changes to data stores or processes should have no impact on those contracts.

Services exchange messages. I have seen several services which were passing encrypted byte arrays that were strings which were cast into a byte array, encrypted, and serialized  by the service then deserialized, decrypted, and cast into a string and transformed to an XML document on the receiving end.  While this offers security through obscurity it violates other principles including meta data exchange and contract first design which uses that meta data so all clients know what messages to expect.  Services should use messages that are exposed in a canonical format via schemas and WSDL’s (for web services).

Messages should contain all the elements required to complete the business process.  This is a great principle I found via Nicholas Allen’s Indigo Blog in a white paper published by Ythos.  This will allow you the most flexibility in situations where that message will be routed to several services to fulfill the business process.  In a real world context I can think of few, if any, processes where you fill out multiple parts of a form and send it to different entities for processing.  The form contains all the data needed and you send to one place and are not concerned if it needs to be routed to several locations to complete your request.

If systems need data from each other they should ask for it.  This is one of my favorite principles gleaned from Clint Edmonson.  The simplicity of this principle is also its source of elegance.  A good service offers meaningful ways for clients to request the data that service knows about.  The requests should require minimal amounts of "state" or "context" and be flexible to allow for data to be requested multiple times if one attempt fails.

Exposing processes as services lets you focus on the process and build several different UI’s to consume those services.  I have not yet reached a point where this is a reality at my employer but we are very close to getting there.  As you build more services and more processes are enabled in standard ways you can begin to design different types of UI’s to interact with those services.  Do you have a core service exposed as a web service using SOAP but are concerned that is too much lifting for your AJAX clients?  No problem, expose a wrapper service that uses REST and a lighter message format and have that translate to your internal web service.  This keeps your functionality in a central location but has the flexibility to meet special needs for clients.

These are some practical things I and others have been working to apply with great success.  A key to building an SOA from the ground up is to start with simple services and keep them simple.  The trick is to keep an eye to the future and be sure your services will be governable and promote reuse of data models, etc.  While a ground up approach is not always the best in some situations that is where the call to services needs to begin.  Good architects will be looking out for what will better align IT with the business, add value to the processes, and save on costs due to a more maintainable infrastructure.

Building With Lego’s: Using What’s in the Box

My wife and I recently decided to dust off the old Lego sets and start building.  We had collected quite a few Star Wars Lego sets over the years but they were all dismantled with the pieces mixed together in a big Tupperware container.  What I always found fun about Lego’s is that for the basic sets you see a lot of the same pieces over and over and they get used in very clever ways.  Javelins from the Knight set become stabilizers for the wings of the Ewok Attack set.  Revolvers from the cowboy set are the same pistols used by space assassin Boba Fett.  There are very few custom pieces in any of the variety of branded Lego sets.

Lego has obviously made a conscious decision that it is just not sustainable to make all custom bricks for every set produced.

At the August LLF event I had the fortune to speak with a data architect with a very large financial company.  One of the first questions he asked me was if we had developed a canonical data model.  He also asked if our domain had a model we could use, which the justice domain does – NIEM.  This was a very interesting conversation that really clicked while snapping together Lego bricks.

Today I spoke with one of the architects who has done a lot of work on the NIEM data model, in particular LEXS which is a law enforcement specific subset.  I am currently working on an exchange with a victim notification service and have chose to use NIEM for the exchange.  This is a conscious effort on my part to use NIEM for external data sharing and a smaller model for our internal systems.  I also made a decision that rather than piecing together my own custom schema from NIEM that I would reuse an existing one and ended up choosing LEXS.  With some help from the aforementioned NIEM architect I was provided with a schema that expresses about 90% of what I need and left some open ends for me to fill in.

I now have all the pieces I need for the exchange and I just have to fill in a few pieces which I can build myself and still keep within the spirit of LEXS and save some time on schema development.

Now to help my wife find the last piece she needs.  Some times it can be hard to find the color you need but I’m sure we’ll find something in that box that will fit just right.

Technorati Tags: ,,

Push Versus Pull Interfaces

The best analogies for technical things most often come from the completely non-technical sources.  This post on the Four Hour Work Week blog really resonated with me, especially given some of my current projects.

Push versus pull describes the fundamental choice that must often be made when integrating two (or more) connected systems.  I have struggled with this many times and have found myself over complicating things by coming up with elegant strategies to push data from system A to B and C and so on.  While doing so was leveraging appropriate technologies a simpler answer was staring at me the whole time.

Early on I had developed two interfaces which were pull based.  You want this data, come get it and I’ll give you the data you asked and nothing more.  Simple and effective though it put the burden on consumers to come get it.

Neither of those interfaces has failed to date (we have had problems with how they were integrated in a few systems).

In the course of some new projects I was tempted into over complicating them because I had modeled the whole scenario as pushing a message through our systems.  Thanks to some insightful tutelage from Clint Edmonson he pointed me right back to my pull model and reminded me of a quote I am very fond of.

"Doing old things in new ways"

A fundamental principle Clint teaches is that if systems need data from each other, they should ask for it.  This is some what of a reversal of the legacy approach where systems would often send data to another system that it knew needed it.

Take an example where you have a sales system that needs to get information from n number of vendors.  You could have an FTP site or web service where everyone can send their data.  That should work fine, until you really try it.  Then you end up with n number of file formats and n number of incoming FTP connections, user accounts, etc.  It’s a mess to maintain; ask anyone who has to.

What if you reversed that?  What if your company came up with a simple web service definition and each vendor put an instance of that service up and provided their data in the format specified by the web service?  Then your sales system simply has to go down its list of vendors and call them up to get the data?

Perfect, no problems ever.  Right?  Not quite.  Every solution will, hopefully, present you with new or different problems which hopefully have simpler solutions than problems created by other choices.

I mentioned all of this without bringing up publish/subscribe but that is the core of what we’re talking about here.  You have publishers of data and subscribers to that data.  The publisher provides a consistent, well-defined method of getting that data and subscribers come and get it.  It solves a lot of problems.  Service down, no problem, subscribers can catch up later.  Something out of sync, not a problem, the publisher can resend something if needed.

Push versus pull?  Right now I think things should lean towards pull.  It was a good enough model that the Department of Justice began developing WSDLs and asking partners to put up web services that met the specifications of that WSDL so they could come and get from those partners.  To date it has been very successful, at least in our implementation.