Architecture

“Icky” Decisions

@KeithDahlby posted a good question for thought the other day on “the” Twitter:

When implementing the Observer pattern, how do you handle exceptions from observers?

This seemingly harmless question should spark at least a good 15 minute debate.  An obvious question (and the one I asked):

Do you care if an observer throws an exception?

If you consider the classic observer pattern you’ll note that the observed only knows about it’s observers through an interface.  Using an interface for observers to reduce coupling is a good practice but by doing that you’ll want to be sure to retain that reduction of coupling by not allowing the observed to know about any of the concrete observers. 

Deciding to allow the observed to know about it’s observers and allow it to handle exceptions would introduce coupling that could lead to bugs down the road. 

Keith is a super smart guy and knew all along what to do, but the “what” is icky:

That’s what we’re doing, but agreed that it’s icky – wish there were a better option that catch(Exception)

This is icky because in the event an observer is very broken you won’t have a first-hand way to find out (and ReSharper will let you know about the dangers of your empty catch block).  Each observer is now responsible for logging or notifying someone about exceptions which adds some additional complexity (possibly through configuration or another mechanism).

You’ll run into these icky decisions and each one certainly needs healthy debate and some documentation to cover the bases later on when someone might question why the icky option was chosen. 

What was your most recent “icky” situation?

Lessons Learned About Being a Software Architect By Trying to Be One

For most of my software development career I cared about how software was organized, how I had to work with it; how I had change it.  I never had any formal training or courses on being an architect and, until recently, it seemed like such courses were few and far between.

Over the last few years I learned a lot about being a software architect through trials by fire.  When I left the Library world and transitioned into the Justice world I had to step up my game quite a bit.  During that time I learned what it took to build systems that had to work round-the-clock.  I learned what it took to make disparate systems talk to each other to perform work and remain autonomous.  I even tried my best to do things the way I was learning.  But often times I was learning more than I could apply and at the pace work was coming at me I couldn’t rewrite systems just to apply new things I learned.

The first thing I learned was that you can’t do it alone.  I knew that there was a better way to do things and I couldn’t settle for the status quo.  I also didn’t agree with all my coworkers, so I went outside for help.  Getting outside perspective was huge and hooked me into people who share industry best practices.  Blending that knowledge with inside perspective helped me see where best practices fit into the environment.

The next thing I learned is that people probably won’t like you.  Something about architects being perceived as telling people how to code leaves architects out of popularity contests.  Personality also works against you too.  Learning to control the personality traits that work against you (unwillingness to be flexible, “my way, no highway option”) and helping people discover best practices in the way they need to learn will help.  Both of those take time to do and you can’t reach those goals all at once.  The architects who learned this lesson are some of the best and I am fortunate to know a few of those architects.

You have to be willing to constantly question yourself. Software architects have to make a lot of assumptions.  Part of the key to success is to get good at making good assumptions.  Another part is to know when to question yourself and when to be OK with something.  These two attitudes can work for you and against you.  You’ll know when something is as right as its going to be.

Software architecture is personal and you have to be open to exposing how you think to others.  This is hard.  Especially when someone who you think is not capable of objectively reviewing a solution reviews one and gives you negative feedback.  Dealing with that requires more interpersonal skills than anything.  Be prepared to make mistakes and see lesson #2.

The final lesson I learned is that you will never be done learning and it takes small leaps to make big ones.  I was introduced to OO patterns by a Java programmer.  I was clueless.  As I studied those patterns and the concepts taught, I struggled to apply them in an OO world.  But in the SOA world things started to make sense.  Encapsulation.  Abstraction. Loose-coupling.  Suddenly concepts and patterns made sense at the OO level.  For me to learn I had to see the problems in context.  My context was SOA and the bigger picture.  When it all clicked, that “a-ha” moment helped me make leaps that I never imagined possible.

For a few years now I have considered myself an architect though I have never had that title from an employer.  A good friend and mentor told me that the way I worked and what I did made me an architect.  He had that title once, so I felt ok taking that advice from him.

The truth is, every developer is a little bit of an architect.  Some of us just care a little more than the next guy.

Using Dependency Injection Containers Effectively

Out of all the SOLID principles the one I feel the strongest about is Dependency Injection (DI).  The motivational poster Derick Bailey created sums it up the best.  Why are we hard-coding our systems together, making them harder to test, and harder to change?

Containers evolved to support DI at runtime so we could apply inversion of control.  Containers also give us some other things we can leverage besides just keeping track of all those dependencies.  We have to be careful not to go container crazy though!

Containers can manage object lifetimes.  Ever had to write a singleton?  With the availability of containers you almost don’t have to anymore.  Before you write your next singleton, think about letting a container manage that lifetime for you.

Use containers to enforce interface/abstract dependencies. Refer to the motivational poster referenced above.  Do you really want to solder the light to the socket?

Make sure you initialize and maintain the container at the right level.  Ideally your application should have one container to serve all.  In WCF you want this to be at the Service Host level.  In ASP.NET it will be something initialized in global.asax.

Encapsulate your container initialization in a class.  I refer to these classes as “Providers”, ex StructureMapProvider or ContainerProvider (Manager might also work too).  This class should be responsible for initializing the container and maintaining the reference to it.  When you need to change your dependencies there’s one place to make that change.

Use containers to handle abstractions built around third-party dependencies or types you don’t own.  Be careful not to follow this anti-pattern pointed out by Jeffrey Palermo.

Favor code based configuration over .config files. Say what?  Configuration files are great for a lot of things.  By setting up a container in one you may think, “Great, now I can just change the .config to add new types, etc”.  Realistically you are probably never going to do that.  Why do I say this? Because I used to believe that I would use .config that way. And it never happened.

Currently my favorite container is StructureMap.  Your mileage may very and fortunately, there are a lot of choices out there.

The Power of Patterns

Design Patterns have been around for some time now but many of us are still learning how and when to leverage these powerful concepts for engineering software.  Design Patterns have also been the source of backlash for those who have suffered under design pattern crazed architects.  As with any useful tool or concept the balance lies somewhere in the middle.

Patterns exist even when you don’t see them.  If I had to choose a way to describe my early programming career (classic ASP & MS Access/MySql) it would be “DRY” (don’t repeat yourself).  I spent a lot of time doing things to avoid duplicate work.  In ASP this meant lots of include files and shared routines.  This led to some common patterns that I used to tackle certain problems; by no means were these actual design patterns, it was just a way I found to best solve the problems I encountered.  Looking back on that early experience I see how the design patterns could have helped.

When framing the problem if you see a pattern try to leverage it to solve the problem.  Before the holidays the project architect (We’ll call him “Brad”) and I were trying to tackle a problem.  As he was looking at what we were doing he suggested the strategy pattern.  We weren’t sure it was a good fit but it gave us something to look at to help us with the problem.

One of the barriers to understanding patterns is a lack of real-world examples.  If you study the strategy pattern I referred to above you’ll see most of the examples discuss simple calculations, A+B type of stuff.  That is helpful, but what if I have 4 factors that are needed to calculate the price and there is other data we must include with the price, like if there was a discount, etc?  Does that mean I can’t use the strategy pattern?

Patterns require you to generalize and forget about your specific domain.  This is probably the hardest part.  A lot of people think that their situation is totally unique; for the domain it may be, but in reality a lot of people have the same problems.  This is where patterns really help.

One of my goals for 2010 will be to continue studying and applying patterns where they fit.  Some great resources I’m using are:

Service-Oriented, Event-Driven Part 2: Autonomous Services

Last time we looked at events, notable changes in state, that can be used to notify other systems who may be interested in the event.  We ended with some pseudo code showing examples of how it might look and finally with some thoughts on coupling, which is where we slide into part two.

What is an autonomous service?  Firstly, let’s define autonomous – via Bing:

- self-governing: politically independent and self-governing
- able to choose: able to make decisions and act on them as a free and independent moral agent
- self-sufficient: existing, reacting, or developing as an independent, self-regulating organism

When we talk about autonomous services, what we mean are services that are self-governing, self-sufficient, and able to choose.  Autonomous services are the gate-keepers to systems of record and exist on their own.  Back to our pizza example we could feasibly have a kitchen service that exists on it’s own, independent of any specific order service.  To put it Vegas style, “What happens in the Kitchen Management Service, stays in the Kitchen Management Service” (except when it’s an event we want people to know about…more on that later).

Autonomous services will be focused to a particular set of related tasks such as our Kitchen Management Service which monitors orders in being cooked and stock levels of products.  I lean towards looking at a service through a single responsibility lens.  The more focused you keep that service, the cleaner the abstraction remains and the easier it will be to integrate in an environment of collaborating services.

Autonomous services are self-contained and therefore internal implementations can change as needed.  By keeping services focused and isolated we can reduce the coupling between other collaborating services or systems.  The only thing an autonomous service must do is fulfill it’s contract.  The contract is the definition of the abstraction the service is representing.  Typically this is a WSDL in the web service world along with the accompanying schemas.  But what we’re talking about here aren’t request-response services, so how does the contract apply in an event driven world?

This question opens the door to what we will explore in Part 3 of Service Oriented, Event Driven: Collaborating Services.  Stay tuned!

Service-Oriented, Event-Driven Part 1: Events

The real-world does not follow a linear, step-by-step process in order for things to occur.  We may rationalize and tell ourselves that things happen in sequence and that ‘c’ won’t happen before ‘b’.  We model our systems this way and we code this way because we view the world through this lens.  I am beginning to fully believe that we must shatter this lens, because it colors the world with a perspective that doesn’t fit.

The real-world is disorderly, it is chaotic, and it is made up events – notable changes in state – that we must react to.  The traffic light turns green, so you go, except the car in front of us isn’t going so now you can’t go. You order coffee and wait, and then the guy behind you gets his coffee first with yours ready a moment later.

Integrating systems using the lens of events is key to success, especially when the systems you are integrating are not yours or they belong to an entire other organization.  This is where a firm understanding of the information being exchanged as well as when it will be exchanged becomes key.

Udi Dahan posted an article titled “Don’t Delete – Just Don’t” a few weeks back.  In that article Udi explores soft deletes versus hard deletes, but he also raises another interesting thought that I’d like to build on.  Our systems maintain the state of records over time.  That state can be things like “entered”, “active”, “inactive”, “accepted”, “denied”, and whatever the business case may be.  As I have already described events as notable changes in state, so what if we start looking at our systems records and the notable changes in state, what would we see?

Dru Sellers frames this in “Events Are Awesome” where he writes:

“I have also started to think about what it would mean to develop a system ‘event first’, and then you can look at what needs to be done when these events happen. It builds completely different systems, based on some initial drawings.”

Over the last few weeks I have been thinking about the same thing.  So let’s hop on the example bus and see where it takes us.

Suppose we run a pizza delivery business and we’re going to automate what has been a manual process.  At a high-level, we could see a few processes we need: Customers should be able to place orders, the kitchen needs to be able to be notified of orders so they can cook them, the delivery supervisor needs to know where orders are going so drivers can be scheduled, and the store manager needs to know about the volume of orders so that the financials can be reviewed.  Skipping through the normal business analysis that needs to take place, we’ll say that we end up with four systems, each with a distinct job. 

  • The order system processes, validates, and accepts pizza orders from customers.
  • The kitchen management system accepts orders, verifies stock levels, and notifies the cooks so the order can be prepared.
  • The delivery scheduling system accepts orders, determines the route, and schedules a driver to deliver the order.
  • The financial system accepts orders and records details about the underlying transaction amounts.

If we examine the order system using the events lens we can come up with some notable events: “Customer Submitted Order”, “Order Validated”, “Order Accepted”.  With each one of these events a notable change in state has occurred and the information about that order should be communicated.  For example, the delivery scheduling system might want to know about orders when they are validated because it takes longer to schedule a driver, whereas the kitchen management system might not want to know about orders until they are accepted so that ingredients are not used too soon in the process.

You may argue that this is not much different than procedural thinking.  We could just code something to the effect of:

if (order.status == validated) {SendOrderToDeliverySystem(order);}

While that one line of code may be the simplest thing that could possibly work, what have we done?  In essence, we have now coupled our order system to the delivery system.  And this is where procedural thought begins to break down because that line of code is so specific that when we need to change it, the change has an effect that we may not fully understand especially because over time code becomes more complex as systems are expected to do more.

Consider the case where now the Kitchen System wants to know about orders when they are validated instead of accepted.  We now have to refactor our one liner to something like:

if (order.status == validated)
{
SendOrderToDeliverySystem(order);
SendOrderToKitchenSystem(order);
}

Now consider if we used the .NET Framework’s notion of events:

public delegate void OrderValidatedEventHandler(PizzaOrder order);
public event OrderValidatedEventHandler OrderValidated;

…{ //more code and curly braces }

if (order.status == validated && OrderValidated != null) {OrderValidated(order);}

If all of our systems were implemented as components with references to each other we could hook onto these events and reduce our codebase.  But even then we still have a problem – coupling.  Using direct references is a form of coupling that ties those components together.  We can use Dependency Injection to reduce that, but then we still have the question to answer – what if each of our 4 systems comes from a different vendor?

In Part 2, we will discuss autonomous services and why we want to reduce coupling.

Complexity, Simplicity, and Elegance

A friend and I were recently talking shop (over Xbox Live of course) and began discussing patterns we see when people get comfortable with what they know.  There are a lot of programmers who are good and reach a certain point where they are continually writing what is, essentially, the same program for every problem they are solving.  While this may seem to be an optimal solution, what it leads to is a loss of creative thinking and solutions that don’t quite fit the problem.

During the conversation I likened this to people from the 80s or 90s who never wanted to update their style and the analogy kind of stuck.  This friend is updating his style in a big way; by taking a leap to a job where I have no doubt he will be successful.  He said one key thing that stays with me every day.  Even when you reach a place where you are comfortable with your style, if you are not constantly questioning your style or trying to figure what it is that you don’t know, there will be no true growth and you may never find the next level of solution to your problem.

A recent link that came across one of my RSS feeds pointed to this article, part five of a five piece interview with Ward Cunningham that took place in 2003.  The article is titled “The Simplest Thing that Could Possibly Work” and explores simplicity in software design drawing from Einstein’s quote “As simple as possible, but no simpler.”  This quote is very easy for people to use against you, especially when it is in the context of a solution they do not fully understand.  The key is that the complexity of the solution is relative to the problem it is solving.

If you take your average web developer who is performing front-end work on an E-commerce site and stick them on the back-end of an enterprise service bus that is using an event-driven architecture, that developer will most likely think it is complicated.  That perceived complexity is misplaced because a service bus is an elegant solution that was crafted by breaking through the complexity of messaging, loosely coupling back-end systems, and orchestrating it all in a way that meets the business process needs and solves the problem. 

In his book “Pursuit of Elegance: Why the Best Ideas Have Something Missing”, author Matthew May broaches the search for elegance by defining it as the simplicity that comes after you break through the complexity.  In order to reach that solution, you must understand the complexity, get tangled in it, and ultimately break through it.

By no means does this imply it impossible to build an overly complex solution.  If you are trying to build a  blog you may not have a strong case for using multiple services and a service bus.  There are simpler solutions for blogs out there.  However, if you are trying to connect autonomous services in a scalable way that allows those services to operate independent of the state of other services, then a service bus is probably a good choice.  The complexity of the solution is relative to the complexity of the problem.  When both the complexity of the problem and relative complexity of the solution are understood, the solution can then be held to the standard of “Simple as possible, but no simpler.”

This is where staying current, questioning yourself, and “updating your style” come into play.  If you are not constantly seeking a better way to do something, trying solutions, questioning the solution, and trying it a different way, you will ultimately get stuck writing the same program to solve every problem.  In some cases the solution may fit, but when it does not the solution becomes more complicated and does not fit the mantra: “Simple as possibleNo simpler.”

Wednesday Links

Via Architect’s Rule

Forrester: Inquiry Spotlight: Developing Architects, Q2 2009

Whether you’re an architect yourself or you have architects reporting to you, you’ve probably wondered: What does it take to be an architect?

Read the full article and link to Forrester: http://blogs.msdn.com/architectsrule/archive/2009/07/01/the-architect-profession-according-to-forrester.aspx

Secure Coding Resources

From our WAN Security Manager at REJIS:

Top 10 Secure Coding Practices: https://www.securecoding.cert.org/confluence/display/seccode/Top+10+Secure+Coding+Practices

Build Security In: https://buildsecurityin.us-cert.gov/daisy/bsi/home.html

What is Build Security In?

Build Security In (BSI) is a collaborative effort that provides practices, tools, guidelines, rules, principles, and other resources that software developers, architects, and security practitioners can use to build security into software in every phase of its development. BSI content is based on the principle that software security is fundamentally a software engineering problem and must be addressed in a systematic way throughout the software development life cycle.

Enjoy!

Software Architect’s Creed

Here are the nine points I presented tonight at the St. Louis .NET User Group.  The slides and a printable PDF will be available in the next day or two.  Thanks to all who attended!

Accepting my responsibility as an architect I will strive daily to learn and perfect my trade.

Readily will I defend my organizations IT investments from complexity, our greatest enemy.

Changing requirements will not break my design for it will be flexible and able to change with the requirements.

Helping developers to understand the reasons for the architecture and seeking their input is of great importance to our success.

I will never use acronyms or suggest technologies that are not pertinent to the problem at hand.

Technologies evolve and I know that solid architectures should accommodate and enable these evolutions.

Every day I will strive to help our software achieve ideals that will make it flexible and easy to maintain.

Concern for the success of my organizations IT investments will drive me to make appropriate decisions.

Teaching others about my trade will be an overarching responsibility that I accept as part of my duties.

*Update*

Posted slides to slidshare.net here.

Saint Louis Dot Net Users Group – Monday, March 30th

Monday, March 30th I will be presenting during the second half of the .NET Users Group meeting and Microsoft’s office in Creve Coeur.  Come out for both Stan Brooks presentation on "Users are Losers!" and mine on "The Software Architects Creed".  This will be the first meeting where we are trying the new format as well as introducing the architecture track.  I’m very excited to be a part of this and look forward to delivering my talk.

Abstract of "The Software Architects Creed" from The St. Louis .NET User Group Homepage:

Solid software architecture is emerging as one of many key factors to successful software projects. While there is great debate on which patterns are best or what tools to use, one factor remains the same: Creating solid software architectures requires a technical leader with the IT and business know how. The Software Architects Creed distills nine key points that when applied will help take your software to the next level. If you are a seasoned architect or a developer wanting to better understand architecture, the creed will provide you with action statements that you can take with you and apply daily to your job.

Hope to see you there!