Friday, October 30, 2009

Book Review: Enterprise Service Bus

ESB About a year ago, I was lucky enough to attend the Kaizenconf in Austin. When I joined the discussions on ESB Patterns, Dru Sellers and Chris Patterson (also known as the MassTransit guys) were talking about this book called Enterprise Service Bus from David Chappell. I finally took the time to read it and for the most part it was a real eye opener. As you might have guessed, the book provides an architectural overview of the ESB concept. Although it does provide some amount of detail, there are no in depth discussions on any particular technologies. But after reading the book it is quite clear that the author comes more from a Java background. However this is not that important for a book like this as these concepts are technology agnostic anyway.

I must admit that getting through the first chapter, which tries to introduce the ESB, was not that easy. Too abstract and high level for my taste.

The second chapter is about the State of Integration and discusses how both business and technical drivers contributed to the ESB approach as opposed to Enterprise Application Integration (EAI). One of the best parts in the book is the discussion about "Accidental Architecture" which is an accurate and far too familiar description of the current architecture in most companies. Although the book is approximately 5 years old, it is still very relevant as the adoption rate of ESB's is still not that common. But then again, I could be wrong about this :-).   

The third chapter examines the key concepts of an ESB were the author tries to prove that these are born out of necessity, based on real requirements and problems that couldn't be solved with typical EAI broker architectures (like Biztalk for example).

Chapter 4 states that XML is the exchange format of choice for passing data structures between applications and services. Nothing new here.

Chapter 5 till 8 provide in dept information about each of the key concepts described in chapter 4, like Message Oriented Middleware (MOM), Service Containers and Endpoints, Routing, Transformation, Messaging, etc. ...

Chapter 9 goes back to the real world by exploring the most common form of integration that is practiced today: bulk data transfer using ETL and an endless amount of small batch applications. Sounds familiar? This chapter also provides the necessary steps in order to migrate away from latency and reliability issues towards a real-time integration and how this affects your business.

Chapter 10 is a bit more technology focused as it talks about Java Components in an ESB. However, this could equally be .NET or any other platform.

For me, chapter 11 is a real masterpiece especially the part on Portal Server Integration patterns like Forward Cache and Federated Query. Highly recommended reading.

The final chapter discusses the WS-DeathStar (WS-*) specifications and what they could mean for an ESB.

In the end, I have a lot to think about after reading this book. It challenged a lot of my earlier assumptions on distributed computing and it certainly helped me understand a couple of things while I was exploring NServiceBus.

Till next time

Thursday, October 29, 2009

Next European VAN on 18 November 2009

Mark Nijhof is going to enlighten us all with his DDD/CQRS sample application. You can read all about it at the E-VAN blog.

Friday, October 16, 2009

Mapping From IDataReader/IDataRecord with AutoMapper

A while ago, I submitted a patch to AutoMapper that added basic support for mapping data from an IDataReader/IDataRecord to an object. For those of us who don't have the luxury to use NHibernate in their projects, this feature can save you from writing lots of repetitive and tedious code.

Its usage is pretty much the same as with regular object-to-object mapping using AutoMapper. Lets show a very simple example.

Suppose we have a view object like the one shown below:

public class SomeView
{
    public Int32 SomeNumber { get; private set; }
    public Guid AnId { get; private set; }
    public Double OrNothing { get; private set; }
}

Now when we can execute a query like this,

SELECT ColumnA AS SomeNumber,
       ColumnB AS AnId,
       ColumnC AS OrNothing
FROM SomeTable
WHERE ...

and read the results using a data reader. Now we can use AutoMapper to map the results to instances of our view class:

var dataReader = ... // Execute a data reader
var views = Mapper.Map<IDataReader, IEnumerable<SomeView>>(_dataReader);

This results in a collection of one of or more view objects. When our query is guaranteed to always return one record, we can use the following syntax:

var dataRecord = ... // Execute data reader and read first record
var = Mapper.Map<IDataRecord, SomeView>(_dataRecord);

This approach expects that a convention is followed whereby the name of a field returned by the query matches the name of a property on the target class. Its also possible to use projection as already provided for regular object-to-object mapping.

Suppose we add a new property to our view,

public class SomeView
{
    ...
    public DateTime SomeDate { get; private set; }
}

and we modify the query so that we retrieve the corresponding date value from the database:

SELECT ...
       ColumnD AS BirthDay
FROM SomeTable

Notice that we've broken the convention here and we need to use projection to ensure that the retrieved date value is mapped to the correct property.

Mapper.CreateMap<IDataRecord, SomeView>()
    .ForMember(dest => dest.SomeDateAndTime, 
               options => options.MapFrom(
               src => src.GetDateTime(src.GetOrdinal("BirthDay"))));

var dataRecord = ...    // Execute data reader and read first record
var = Mapper.Map<IDataRecord, SomeView>(_dataRecord);

Using projection we're able to manually map from a data reader or data record. In some sense,  we're back to square one if we have to do this for all fields. Trying to follow the convention is of course the most useful. 

I know it's not much, but I think it can be helpful for those cases where you actually need to map from a data reader or a data record to an object.

Saturday, October 10, 2009

Exploring NServiceBus

I've been learning a bit more about Service-Oriented Architecture (SOA) and Event-Drive Architecture (EDA) over the last couple of months. Something that kept coming back in the articles and books I've read so far is the concept of an Enterprise Service Bus (ESB). I've heard numerous times about NServiceBus, Mass Transit and Rhino Service Bus in the past, but I've never fully realized what types of problems these technologies try to solve. Besides trying to learn about the scenarios and business needs that drive these technologies, I also decided to take a closer look at NServiceBus by creating the simplest example I could think of: Starbucks!

You've probably found out by now that yours truly isn't capable of making up innovative sample applications, but since we actually have two (!) Starbucks shops already here in Belgium, I couldn't let this go unnoticed either.

The scenario is actually quite simple:

  1. The customer places his order at the cashier.
  2. The cashier passes the order to a barista who starts preparing the drinks.
  3. The cashier asks the customer to pay for his order.
  4. When the customer paid for his order at the cashier, the barista is informed that the payment has been completed.
  5. When the barista finishes its order, he checks whether payment has been completed and delivers the order to the customer.

I've used NSB 2.0 for this exercise, which is the trunk version at the moment. The thing that amazed me the most was how easy it is to get started. I pretty much expected to be muddling around for a week before I could actually send my first message, but this was certainly not the case. Let me show a couple of basic code snippets that illustrate how NSB is used.

Here's an example of the fluent API for bootstrapping NSB:

Configure.With()
    .StructureMapBuilder(ObjectFactory.Container)
    .MsmqSubscriptionStorage()
    .XmlSerializer()
    .Sagas()
    .NHibernateSagaPersisterWithSQLiteAndAutomaticSchemaGeneration()
    .MsmqTransport()
        .IsTransactional(true)
        .PurgeOnStartup(false)
    .UnicastBus()
        .ImpersonateSender(false)
        .LoadMessageHandlers()
    .CreateBus()
    .Start();

Notice that I'm able to reuse my IoC container of choice that is also used elsewhere in the application. After bootstrapping NSB, the IoC container contains an instance of IBus which we can use to send and publish messages, etc. ...

A message that can be sent with NSB must implement the IMessage interface.

[Serializable]
public class NewOrderMessage : IMessage
{
    ...
}

Notice that NSB doesn't require a message to be a class. You can just as well use an interface:

public interface INewOrderMessage : IMessage
{
    ...
}

A message can be send using an instance of the bus:

_bus.Send(new NewOrderMessage(...));

Handling a message is accomplished using a message handler that implements the IHandleMessages interface:

public class CashierMessageHandler 
    : IHandleMessages<NewOrderMessage>
{
    public void Handle(NewOrderMessage message)
    {
        ...
    }    
}

The destination of a message must be configured in the configuration file of the sending application.

<UnicastBusConfig>
    <MessageEndpointMappings>
        <add Messages="CashierContracts" Endpoint="cashier"/>
    </MessageEndpointMappings>
</UnicastBusConfig>

When publishing a particular message, this kind of configuration goes in the configuration file of the subscribing application instead of the publisher.

Bus.Publish(new PaymentCompleteMessage(...));
<UnicastBusConfig>
    <MessageEndpointMappings>
        <add Messages="CashierContracts.PaymentCompleteMessage, 
                       CashierContracts" 
             Endpoint="cashier"/>
    </MessageEndpointMappings>
</UnicastBusConfig>

NSB also supports sagas. Here's an example of how to create a saga.

public class CashierMessageHandler 
    : Saga<CashierSagaData>,
      IAmStartedByMessages<NewOrderMessage>,
      IHandleMessages<PaymentMessage>
{
    public void Handle(NewOrderMessage message)
    {
        ...
    }
    
    public void Handle(PaymentMessage message)
    {
        ...
    }
}

public class CashierSagaData : IContainSagaData
{
    public virtual Guid Id { get; set; }
    public virtual String Originator { get; set; }
    public virtual String OriginalMessageId { get; set; }

    ...
}

The saga data class contains data that should be persisted during the lifetime of a saga so that it can be used when different messages arrive. So make sure that the public members of saga data classes are virtual!

You can find the code of this sample application using NServiceBus at the Elegant Code repository (turns out we actually have one of those at Google Code). I've probably did a whole bunch of things wrong with it, but I definitely learned a thing or two in the process :-).

There's a lot more that I want to learn about NSB like testing sagas, the generic host, etc. ...

Let me round of this post by providing a couple of resources that were very helpful:

I just want to say thanks to Andreas for answering my stupid questions on the NServiceBus user group.

Till next time,

Monday, September 07, 2009

Next European VAN on 05 October 2009

Read all about it at the E-VAN blog.

Wednesday, August 19, 2009

Book Review: Enterprise Integration Patterns

26636021 I just finished reading Enterprise Integration Patterns written by Gregor Hohpe and Bobby Woolf. This book follows the same concept used by Martin Fowler and his marvelous PoEAA book, but obviously focused on integration patterns. Although the title is somewhat misleading, this book fully concentrates on message based architectures as opposed to other integration styles.

I'm somewhat new to message based architectures. By reading this book, I learned a lot of the concepts around messaging and showed me how to make the mind shift that it requires. It also made me realize some important things about SOA and DDD as well.

The introduction and the first chapter discuss the various needs of integration solutions. The first chapter also provides a nice example of a message based integration architecture. Don't miss out on these opening chapters as they clearly opened my eyes about some of the aspects of messaging.

Chapter two describes the different approaches for integrating applications, like File Transfer, Shared Database, Remote Procedure Invocation and of course Messaging, with all their advantages and disadvantages. This is the only chapter that discuss other integration styles, after which messaging is discussed throughout the rest of the book.

The third chapter introduces the six root messaging patterns (Message Channel, Message, Pipes and Filters, Message Router, Message Translator and Message Endpoint). The subsequent chapters take a more detailed view on each of these six root patterns. Except for Pipes and Filters, every fundamental messaging pattern leads to its own chapter that provides a concise reference of the respective supporting patterns.

As with most pattern books like these, don't expect to see lots and lots of code although this book does contain some technical stuff. Code snippets are shown at the end of each pattern, mostly on JMS and MSMQ (occasionally TIBCO and WebSphere MQ). There are also three separate interlude chapters that provide one or more examples about some of the patterns discussed in earlier (or sometimes upcoming) chapters. So it could be worse.

The book can be read from cover to cover, but I'd certainly not recommend it. Although concisely written, this piece of work is not something you want to read bedtime stories from (although it has occasionally put me to sleep). As I already mentioned, I'm fairly new to message based  architectures. I read the first six chapters, skimmed through the chapter on Message Routing and picked up again by reading the chapters on Message Transformation and Messaging Endpoints. The latter is probably of the most practical use as this chapter discusses how to integrate an application with a messaging system.

I have a feel that I'm going to be referencing this book a lot in the near future. ;-) This book, like the PoEAA book, definitely falls into the category of reading it more than once, which is certainly worth the effort. I really enjoyed reading this book and I welcome it to my growing list of favorite books.

Till next time

Saturday, August 01, 2009

Fluent Domain Methods

In my last post, I talked about a Generic Expression Builder class for easily creating domain entities through expression builders. This generic base class takes away some of the burden while developing these expression builder classes.  Expression builders are generally useful inside the domain when you need to create and set up a complete aggregate root from scratch, not to say that expression builders aren't useful in other parts of an application (e.g. creating message objects, DTO's, etc.).

Another place where a fluent interface can add value is for implementing behavior on a domain entity. Taking the same example from my previous post, where we have a domain class named Document with the name of the author, a title and one or more associated tags. Suppose we have a command that needs to add a new Tag to the existing list of tags for a particular Document. We usually end up with code like this:

public class Document
{
    var IList<Tag> _tags = new List<Tag>();

    ...

    public void AddTag(String name)    
    {
        var newTag = new Tag(name);
        _tags.Add(newTag);
    }
}

The usual mistake I see a lot is to let the AddTag method directly take a Tag object. However, in this case the Document class is the aggregate root where Tag is merely a value object that lives within the boundaries of its aggregate root. This means that the aggregate root is responsible for creating instances of a Tag (also check out this post from Udi Dahan).

Lets make this code a bit more fluent. In my previous post, I provided a separate expression builder for creating new Tag objects. This TagBuilder is already used by the expression builder that creates Document objects.

public interface ITagBuilder
{
    void Named(String name);
}

public class TagBuilder : ITagBuilder
{
    private readonly Action<Tag> _afterBuildAction;

    public TagBuilder(Action<Tag> afterBuildAction)
    {
        _afterBuildAction = afterBuildAction;
    }

    public void Named(String name)
    {
        var tag = new Tag(name);
        _afterBuildAction(tag);
    }
}
Now lets reuse this builder class for our domain method that adds a new Tag for a Document. This is how the AddTag method is implemented:
public ITagBuilder AddTag()
{
    return new TagBuilder(newTag => _tags.Add(newTag));
}

And this is how the calling code looks:

document.AddTag().Named("Science"));

I have to admit that it does involve some overkill for this simple example. However, the code becomes a lot more clear when you have to provide a couple of more arguments to a domain method. Again, not everything is a nail for this shiny hammer. But it does bring some nice, readable code when needed. Here's another example that attaches the data of a file to a Document:

document.Attach()
    .FileWithName("The Universe in a Nutshell.pdf")
    .AndData(new Byte[] { ... });

The fact that we can just reuse an existing expression builder is also a clear advantage.

Till next time

Saturday, July 25, 2009

Generic Expression Builder

I blogged about fluent interfaces and expression builders a couple of times before. For this post, I want to share a base class that I've been using for taking away some of the burden when creating expression builders for domain classes.

Here's the expression I'm after:

var document = DocumentBuilder.BuildDocument()
    .AuthoredBy("Stephen Hawking")
    .Titled("The Universe in a Nutshell")
    .TaggedWith(tag => tag.Named("Physics"))
    .Build();

This creates an instance of a domain class named Document with the name of the author, its title and with an associated tag. Let me first show the code of the expression builders that make this happen.

public interface IDocumentAuthorBuilder
{
    IDocumentTitleBuilder AuthoredBy(String author);
}

public interface IDocumentTitleBuilder
{
    IDocumentTagBuilder Titled(String title);
}

public interface IDocumentTagBuilder : IBuilder<Document>
{
    IDocumentTagBuilder TaggedWith(Action<ITagBuilder> 
                                                buildUsing);
}

public class DocumentBuilder : Builder<Document>,
                               IDocumentAuthorBuilder,
                               IDocumentTitleBuilder,
                               IDocumentTagBuilder
{
    private DocumentBuilder()
    {}

    public static IDocumentAuthorBuilder BuildDocument()
    {
        return new DocumentBuilder();
    }

    IDocumentTitleBuilder AuthoredBy(String author)
    {
        ProvideValueFor(document => document.Author, author);
        return this;
    }

    IDocumentTagBuilder Titled(String title)
    {
        ProvideValueFor(document => document.Title, title);
        return this;
    }

    public IDocumentTagBuilder TaggedWith    (Action<ITagBuilder> buildUsing)
    {
        var tagBuilder = new TagBuilder(tag => 
            ProvideValueFor(document => document.Tags, tag));

        buildUsing(tagBuilder);
        return this;
    }
}



public interface ITagBuilder
{
    void Named(String name);
}

public class TagBuilder : ITagBuilder
{
    private readonly Action<Tag> _afterBuildAction;

    public TagBuilder(Action<Tag> afterBuildAction)
    {
        _afterBuildAction = afterBuildAction;
    }

    public void Named(String name)
    {
        var tag = new Tag(name);
        _afterBuildAction(tag);
    }
}

The expression builders all provide progressive interfaces. Also notice that the DocumentBuilder derives from a base class named Builder. This class provides a method ProvideValueFor that is used to feed the base class with the name of a property and a corresponding value. Collections are also supported. Here's the code for the Builder class.

public interface IBuilder<T>
{
    T Build();
}

public abstract class Builder<T> : IBuilder<T>
{
    private Dictionary<PropertyInfo, Object> PropertiesAndValues 
    { get; set; }

    protected Builder()
    {
        PropertiesAndValues = 
            new Dictionary<PropertyInfo, Object>();
    }

    public static implicit operator T(Builder<T> builder)
    {
        return builder.Build();
    }

    protected void ProvideValueFor(Expression<Func<T, Object>> expression, 
                                   Object value)
    {
        var property = ReflectionHelper.GetProperty(expression);
        
        if(false == PropertiesAndValues.ContainsKey(property))
            RegisterPropertyAndValue(property, value);
        else
            SetPropertyAndValue(property, value);
    }

    private void SetPropertyAndValue(PropertyInfo property, 
                                     Object value)
    {
        if(IsCollection(property))
        {
            var values = (List<Object>) PropertiesAndValues[property];
            values.Add(value);
        }
        else
        {
            PropertiesAndValues[property] = value;   
        }
    }
    
    private void RegisterPropertyAndValue(PropertyInfo property, 
                                          Object value)
    {
        if(IsCollection(property))
            PropertiesAndValues.Add(property, 
                                    new List<Object>() { value });          
        else
            PropertiesAndValues.Add(property, value);   
    }

    private static Boolean IsCollection(PropertyInfo property)
    {
        if(property.PropertyType == typeof(String))
            return false;

        var collectionType = typeof(IEnumerable<>);
        return IsCollectionOfType(collectionType, 
                                  property.PropertyType);
    }

    private static Boolean IsCollection(FieldInfo field)
    {
        var collectionType = typeof(ICollection<>);
        return IsCollectionOfType(collectionType, field.FieldType);
    }
    
    private static Boolean IsCollectionOfType(Type collectionType, 
                                              Type type)
    {
        if(collectionType.Name == type.Name)
            return true;

        var interfaces = type.GetInterfaces();
        return interfaces.Has(@interface => 
            @interface.Name == collectionType.Name);    
    }

    public T Build()
    {
        var typeToBuild = typeof(T);
        if(false == HasParameterlessConstructor(typeToBuild))
            throw new InvalidOperationException(
                "No parameterless constructor.");

        var instance = (T)Activator.CreateInstance(typeToBuild, true);
        foreach(var entry in PropertiesAndValues)
        {
            var property = entry.Key;
            if(IsCollection(property))
                SetCollectionValuesFor(property, instance, 
                                       (List<Object>) entry.Value);
            else
                SetValueFor(property, instance, entry.Value);
        }

        return instance;
    }

    private static Boolean HasParameterlessConstructor(Type type)
    {
        const BindingFlags bindingFlags = 
            BindingFlags.Public | 
            BindingFlags.NonPublic | 
            BindingFlags.Instance;
                                          
        var defaultConstructor = 
            type.GetConstructor(bindingFlags, null, 
                                new Type[0], null);
        return null != defaultConstructor;
    }

    private static void SetValueFor(PropertyInfo property, T instance, 
                                    Object value)
    {
        property.SetValue(instance, value, null);    
    }

    private static void SetCollectionValuesFor(PropertyInfo property, 
                                               T instance, 
                                               List<Object> values)
    {
        var backingField = BackingFieldResolver.GetBackingField(property);
        if(false == IsCollection(backingField))
        {
            var message = String.Format(
                ResourceLoader<Builder<T>>
                    .GetString("InvalidCollectionType"), property.Name);
                
            throw new InvalidOperationException(message);    
        }

        var collection = property.GetValue(instance, null);
        foreach(var value in values)
        {
            const BindingFlags bindingFlags = 
                BindingFlags.Public | 
                BindingFlags.Instance | 
                BindingFlags.InvokeMethod;
                                              
            backingField.FieldType
                .InvokeMember("Add", bindingFlags, null, 
                              collection, new[] { value });
        }
    }
}

Using this approach, its no longer necessary to make any compromise of exposing property setters or a dedicated constructor just for serving the expression builders. The Builder class uses refection to set the value of a property or to fill a collection. The BackingFieldResolver is a class I picked up from this post. Very cool stuff!

I have only used this approach in a couple of side projects, but let me know you think.

Till next time.

Monday, July 20, 2009

NHibernate 2.1 and Collection Event Listeners

In a previous post, I talked about cascading deletes being a new feature introduced by NHibernate 2.0. If you haven't heard about this before, then you'd probably be interested to read about it first.

Cascading deletes are all great if your database of choice supports CASCADE DELETE foreign key constraints. But what if it doesn't provide this feature or, as in my case, the database in question does support this feature but the DBA's don't want anything to do with it? In case of a parent domain object having a collection of many child objects, you still might want to have a one-shot delete capability instead of having separate DELETE statements for each child record.

The newly released NHibernate 2.1 (congratulations to the entire team for their efforts and hard work) comes to the rescue, which introduces a couple of new event listeners that deal with collections.

First we need an example. Suppose we are building an auction web site and the domain has a class called Item which in turn has a collection of Bids.

public class Item
{
    private ISet<Bid> _bids;    
    public Int64 Id { get; private set; }
    
    ...
}

public class Bid
{
    public Double Amount { get; private set; }
    public Sting Code { get; private set; }

    ...
}

The mapping for these classes looks something like this:

<class name="MyAuction.Item, MyAuction" 
       table="Item">
    <id name="Id" type="Int64" unsaved-value="-1">
        <column name="Id" sql-type="integer"/>
        <generator class="native"/>
    </id>
    
    ...
    
    <set name="Bids" 
         access="field.camelcase-underscore" 
         lazy="false" 
         cascade="all-delete-orphan" 
         inverse="true" 
         optimistic-lock="false">
        <key>
            <column name="ItemId"/>
        </key>
        <one-to-many class="MyAuction.Bid, MyAuction"/>
    </set>
</class>

<class name="MyAuction.Bid, MyAuction" 
       table="Bid">
    <composite-id>
        <key-property name="ItemId" 
                      column="ItemId" 
                      type="Int64"/>
        <key-property name="Code" 
                      column="Code" 
                      type="String"/>
    </composite-id>
    
    ...
    
</class>

Just to give you a general idea of the situation here. Now suppose we want to delete a quite popular Item object  which has a numerous amount of Bids. Because the collection of Bids is mapped as inverse, NHibernate will remove every record for a Bid with a separate DELETE statement for each row.

DELETE FROM Bid WHERE ItemId=@p0 AND Code=@p1 ;@p0 = 2, @p1 = 'F1001'
DELETE FROM Bid WHERE ItemId=@p0 AND Code=@p1 ;@p0 = 2, @p1 = 'F1002'
DELETE FROM Bid WHERE ItemId=@p0 AND Code=@p1 ;@p0 = 2, @p1 = 'F1003'
...
DELETE FROM Item WHERE Id = @p0; @p0 = 2

We could solve this by creating a collection event listener. The first thing we have to do is figure out how to issue a one-shot delete instead of those separate DELETE statements.

public interface IOneShotDeleteHandler
{   
    Type ForEntity();
    Type[] ForChildEntities();
    void GiveItAShot(ISession session, Object entity);
}

public class OneShotDeleteHandlerForItem : IOneShotDeleteHandler
{
    public Type ForEntity()
    {
        return typeof(Item);
    }

    public Type[] ForChildEntities()
    {
        return new[] { typeof(Bid) };
    }

    public void GiveItAShot(ISession session, Object entity)
    {
        var item = (Item)entity;

        session.CreateQuery("delete Bid where ItemId = :itemId")
            .SetInt64("itemId", item.Id)
            .ExecuteUpdate();
    }
}

We created an IOneShotDeleteHandler interface with one implementation for the Item class.  The most notable aspect of this implementation is the use of the HQL delete statement that removes all Bids for a particular Item.

Next step is to create a collection event listener that implements the IPreCollectionRemoveEventListener interface.

public interface IEventListener
{
    void ConfigureFor(Configuration configuration);
}

public class CollectionRemoveEventListener 
    : IPreCollectionRemoveEventListener,
      IEventListener
{
    private readonly IEnumerable<IOneShotDeleteHandler> 
        _oneShotDeleteHandlers;
    
    public CollectionRemoveEventListener(
        IEnumerable<IOneShotDeleteHandler> oneShotDeleteHandlers)
    {
        _oneShotDeleteHandlers = oneShotDeleteHandlers;
    }

    public void OnPreRemoveCollection(
        PreCollectionRemoveEvent @event)
    {
        var affectedOwner = @event.AffectedOwnerOrNull;
        if(null == affectedOwner)
            return;

        var oneShotDeleteHandler = 
            _oneShotDeleteHandlers.SingleOrDefault(handler =>
            handler.ForEntity() == affectedOwner.GetType());

        if(null == oneShotDeleteHandler)
            return;

        oneShotDeleteHandler
            .GiveItAShot(@event.Session, affectedOwner);
    }

    public void ConfigureFor(Configuration configuration)
    {
        configuration
            .SetListener(ListenerType.PreCollectionRemove, this);
    }
}

Don't worry about the IEventListener interface. Its just there for registering all NHibernate event listeners in an IoC container.By doing so, it enables us to inject a collection of IOneShotDeleteHandler objects into the constructor of our event listener. When the OnPreRemoveCollection method is called, we simply lookup whether there's a handler available for the type of entity that's going to be deleted and give it a shot at removing its child collection in one sweep.

Now we only have to register this event listener:

var eventListeners = _dependencyContainer
    .ResolveAll<IEventListener>();
eventListeners.ForEach(eventListener => eventListener
    .ConfigureFor(configuration));

Now, if we would use this 'as is', NHibernate will give us the following error:

Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect)

This is due to the fact that the collection event listener does its work and deletes all Bids by executing the HQL statement, but NHibernate still tries to issue a DELETE statement for each Bid. This is the result of the all-delete-orphan cascading rule we imposed in the mapping. We could reduce it to save-update, but then no individual DELETE statements are executed when a singe Bid is removed from the collection. Now what?

Well, we could provide a regular delete event listener that allows individual DELETE statements for Bid entities as long as their parent Item is not removed.

public class DeleteEventListener : 
    DefaultDeleteEventListener,
    IEventListener
{
    private readonly IEnumerable<IOneShotDeleteHandler> 
        _oneShotDeleteHandlers;

    public DeleteEventListener(
        IEnumerable<IOneShotDeleteHandler> oneShotDeleteHandlers)
    {
        _oneShotDeleteHandlers = oneShotDeleteHandlers;
    }

    protected override void DeleteEntity(
        IEventSource session, object entity, 
        EntityEntry entityEntry, Boolean isCascadeDeleteEnabled, 
        IEntityPersister persister, ISet transientEntities)
    {
        var oneShotDeleteHandler = _oneShotDeleteHandlers
            .SingleOrDefault(handler =>
                handler.ForChildEntities()
                    .Contains(entity.GetType()));

        if(null == oneShotDeleteHandler ||
           !IsParentAlsoDeletedIn(
                session.PersistenceContext, 
                oneShotDeleteHandler.ForEntity())
            )
        {
            base.DeleteEntity(session, entity, entityEntry, 
                              isCascadeDeleteEnabled, persister, 
                              transientEntities);
            return;
        }

        CascadeBeforeDelete(session, persister, entity, 
                            entityEntry, transientEntities);
        CascadeAfterDelete(session, persister, entity, 
                           transientEntities);
    }

    public void ConfigureFor(Configuration configuration)
    {
        configuration.SetListener(ListenerType.Delete, this);
    }

    private static Boolean IsParentAlsoDeletedIn(
        IPersistenceContext persistenceContext, Type typeOfParent)
    {
        foreach(DictionaryEntry entry in 
                                    persistenceContext.EntityEntries)
        {
            if(typeOfParent != entry.Key.GetType())
                continue;
            
            var entityEntry = (EntityEntry)entry.Value;
            if(Status.Deleted == entityEntry.Status)
                return true;
        }

        return false;
    }
}

With both event listeners registered, deleting a single Bid results in a single DELETE statement as one would expect:

DELETE FROM Bid WHERE ItemId=@p0 AND Code=@p1 ;@p0 = 2, @p1 = 'F1002'

and  removing an entire Item now results in a one-shot delete for all Bids:

DELETE FROM Bid WHERE ItemId=@p0; @p0 = 2
DELETE FROM Item WHERE Id = @p0; @p0 = 2

Make sure you use this solution for one-shot deletes wisely and only if you have to. If you can use the CASCADE DELETE foreign key constraints, then by all means, this is the preferred option. If not, only resort to this kind of solution only if you must and that you can prove that its going to give you a tremendous performance benefit. Also take a look at the batching support that NHibernate provides (at the moment only SQL Server and Oracle are supported).

Till next time

Sunday, July 19, 2009

Design Documents in CouchDB and Validation

To start off, here are the links to my previous posts about CouchDB:

  1. Relaxing on the Couch(DB)
  2. Installing the Couch(DB)
  3. PUTting the Couch(DB) in Your Living Room
  4. GETting Documents From CouchDB
  5. DELETE Documents From CouchDB
  6. Adding Attachments to a Document in CouchDB
  7. Views into CouchDB

I briefly mentioned 'design documents' in my previous post as the way views are being stored in CouchDB. This is probably the most common use of design documents, but there's more. One thing that's interesting is the ability of performing validation. Now before anyone starts raving like a mad man, I'm not implying that all validation or, to make matters even worse, that business rules should now all be handled by CouchDB, let along they should all be written in JavaScript from now on. In fact, quite the contrary.

But one concern that comes to mind that might be interesting for its use is validating the structure of a document.

As you might have noticed from my previous posts, we always dealt with one type of object. But a hello-real-world application typically has a domain where multiple types of objects need to be persisted. With an RDBMS, we usually have separate tables to store these different kinds of objects. CouchDB on the other hand only has the notion of storing documents. Now suppose, we need to store both Customer objects and Order objects (every application needs those, right?). How would we deal with that in CouchDB?

Well, very simple. We'll just add a Type field to every document.

{
    "Type" : "Customer"
    "FirstName" : "Homer"
    "LastName" : "Simpson"
}

{
    "Type" : "Order"
    "Supplier" : "Duff"
    "Subject" : "Sweet, sweet beer" 
}

Just common sense. In order to get a list of all orders, we could provide the following map function:

function(doc) 
{
  if(doc.Type != "Order") 
    return true

  emit(null, doc);
}

Piece of cake. Now we can use the validation capabilities of CouchDB to ensure that every document that is stored contains a Type attribute. They way to handle this is to create a design document that contains an attribute named 'validate_doc_update' that specifies a validation function of the following signature:

function(newDoc, oldDoc, userCtx)
{}

Now in order to keep clear of documents that specify no Type attribute, we could write the following function:

function(newDoc, oldDoc, userCtx)
{
    if(!newDoc.Type)
    {
        throw(
        {
            "Error" : "Documents need a type around here."
        });
    }
}

Every time a document is saved or updated, CouchDB will call every validation function that is stored in a design document with the key 'validate_doc_update'. When every function passes, the document will be stored. Otherwise, CouchDB will return HTTP status 403 (Forbidden) with a JSON response that contains the error we specified in the validation function.

As you an see, CouchDB provides you with a nice and easy to use validation mechanism that can be useful in a couple of scenarios.

Till next time

Monday, July 13, 2009

Using NHibernate for Legacy Databases

One of the downsides of being confronted with a shared legacy database day in and day out is that you have to map your domain objects to database tables that are also used by other applications. A typical scenario in this case is that those database tables contain more columns than those that are required for your application. These extra columns are specifically there to serve those other legacy applications. Heck, to make matters even worse, there are probably some new columns added specifically for your application as well. This is the fairy tale of shared legacy databases.

Using NHibernate in these scenarios can be challenging sometimes but its built in flexibility and extensibility really helps you to deal with those cases. The problem I ran into last week was that we needed to store a domain object into a table that had a lot more columns than were actually required for our application. If it would be possible to store null values in these columns or if they had default values configured for them in the schema, then this would not be a problem. Instead, these unnecessary columns could not store null values and had no default values associated with them.

First option would be to make some changes to the schema of the table. Alas, no luck there because the other legacy applications that are using the same table would break. Now what?

We needed to insert the default values ourselves, but those columns are not known by NHibernate because they are not mapped to any members of the domain object. One way to solve this, is to pollute the domain object by adding private fields that are initialized to the required default values.

public class SomeDomainEntity
{
    // Legacy fields with no purpose for the domain but required 
    // by the database.
    private Int32 _legacyField1 = 2;
    private Boolean _legacyField2 = false;
    private String _legacyField3 = "";
}

This is probably the simplest option, but imposes a broken window as infrastructure concerns are bleeding into the domain this way. In other words, this is not a viable solution. Keeping the legacy stuff isolated as much as possible, NHibernate provides some ways to deal with this by providing an extensive extensibility model.

After some snooping around in the source code of NHibernate, the solution we chose for dealing with this issue is by creating a custom access strategy. The built in property access strategies are probably already well known, but its also possible to write your own access strategy by implementing the IPropertyAccessor interface.

public class SomeDomainObjectAccessor : IPropertyAccessor
{
    private IEnumerable<IGetter> _defaultValueGetters;
    
    public SomeDomainObjectAccessor()
    {
        _defaultValueGetters = new List<IGetter>()
        {
            { new DefaultValueGetter<Int32>("LegacyColumn1", 2) }
            { new DefaultValueGetter<Boolean>("LegacyColumn2", false) }
            { new DefaultValueGetter<String>("LegacyColumn3", 2) }
        }
    }

    public IGetter GetGetter(Type type, String propertyName)
    {
        return _defaultValueGetters        
            .Where(getter => getter.PropertyName == propertyName)        
            .SingleOrDefault();
    }
    
    public ISetter GetSetter(Type type, String propertyName)
    {
        return new NoopSetter();
    }

    public Boolean CanAccessTroughReflectionOptimizer
    {
        get { return true; }
    }
}

private class DefaultValueGetter<T> : IGetter
{
    private readonly String _propertyName;
    private T Value { get; set; }

    public DefaultValueGetter(String propertyName, T value)
    {
        _propertyName = propertyName;
        Value = value;
    }

    public Object Get(Object target)
    {
        return Value;
    }

    public Type ReturnType
    {
        get { return typeof(T); }
    }

    public String PropertyName
    {
        get { return _propertyName; }
    }

    public MethodInfo Method
    {
        get
        {
            var method = typeof(BasicPropertyAccessor)              
                .GetMethod("GetGetterOrNull",
                           BindingFlags.Static | BindingFlags.NonPublic);

            var result = (BasicPropertyAccessor.BasicGetter)method              
                .Invoke(null, new Object[] { GetType(), "Value" });

            return result.Method;
        }
    }

    public object GetForInsert(Object owner, IDictionary mergeMap,
                               ISessionImplementor session)
    {
        return Get(owner);
    }
}

private sealed class NoopSetter : ISetter
{
    public void Set(Object target, Object value)
    {}

    public String PropertyName
    {
        get { return null; }
    }

    public MethodInfo Method
    {
        get { return null; }
    }
}

This simply involves a getter for providing default values and a dummy setter as we're not interested in setting any values on the domain objects. The DefaultValueGetter class uses a trick so that we can keep using the reflection optimizer of NHibernate. This also seems to be necessary when using NHibernate Profiler.

Now we only have to provide some properties in the mapping of the domain object using our custom access strategy:

<property 
    name="LegacyColumn1"
    column="LegacyColumn1"
    not-null="true"
    type="Int32"
    access="SomeNamespace.SomeDomainObjectAccessor, SomeAssembly"/>
          
<property 
    name="LegacyColumn2"
    column="LegacyColumn2"
    not-null="true"
    type="Boolean"
    access="SomeNamespace.SomeDomainObjectAccessor, SomeAssembly"/>
          
<property 
    name="LegacyColumn3"
    column="LegacyColumn3"
    not-null="true"
    type="String"
    access="SomeNamespace.SomeDomainObjectAccessor, SomeAssembly"/>

This is probably not the best solution, but it does the job and prevents polluting the domain objects as a result of database quirks like these. I'm interested in hearing feedback or any better approaches.

Anyway, the easy extensibility of NHibernate makes it the best data access solution around. This way, one can deal with all edge case scenarios that weren't anticipated by the framework builders.

Till next time  

Friday, July 10, 2009

Views into CouchDB

To start off, here are the links to my previous posts about CouchDB:

  1. Relaxing on the Couch(DB)
  2. Installing the Couch(DB)
  3. PUTting the Couch(DB) in Your Living Room
  4. GETting Documents From CouchDB
  5. DELETE Documents From CouchDB
  6. Adding Attachments to a Document in CouchDB

So far, we've mostly talked about managing documents in CouchDB. Now I want to discuss another important concept of CouchDB, namely views.

Views are the primary means for querying and searching documents that are stored by CouchDB. As mentioned in one of my previous posts, CouchDB doesn't support SQL for querying documents. Consequently, views are to CouchDB as SQL is to an RDBMS. They are defined as MapReduce functions using JavaScript. When you've never heard about MapReduce, then take a look at the Introduction to MapReduce for .NET Developers for a quick dip into the concepts behind MapReduce.

CouchDB supports two kinds of views, permanent views and temporary views. A permanent view is stored as a special kind of document between the other regular documents. These special kind of documents are called 'design documents'. A permanent view can be executed by performing a GET operation with the name of the view.  A temporary view, as its name implies, is not stored by CouchDB. Instead, the code for the view is posted to CouchDB where it is executed once.

For me, permanent views are the most interesting so we will use this option for the example in this post. Recall that in the examples used in previous posts we've had documents like the following:

{
"_id":"96f49e5a-6b5b-47ed-9234-9a98d600013e",
"_rev":"2-1534297415",
"Author":"Stephen Hawking",
"Title":"The Universe in a Nutshell",
"Tags":[{"Name":"Physics"},{"Name":"Universe"}]
}

Lets create a view that we can use for retrieving the title of all documents for a particular  tag. For creating a permanent view, there are again two options: using Futon (the web-based user interface of CouchDB) or through the HTTP View API. For keeping things simple, let's use Futon for creating our view in CouchDB.

 image

When creating a new view, CouchDB provides a map function with a default implementation and an optional reduce function. The purpose of a map function is to perform a number of computations using arbitrary JavaScript and to emit key/value pairs into the view.  If the view also has a reduce function, then its used for aggregating the results. We'll ignore the reduce function for now and focus our attention to the map function. 

The basic anatomy of the map function as provided by Futon looks like this:

function(doc) {
  emit(null, doc);
}

As already mentioned, the emit function takes care of inserting a key/value pair of your own choosing into the view. For our example, we'll emit the name of a tag as the key and the title of a document as the value.

function(doc) 
{
  for each(var tag in doc.Tags)
  {
    emit(tag.Name, doc.Title);
  }
}

image

Using Futon it's possible to code up a map and reduce function and try it out on the documents that are stored in CouchDB. Executing the map function as is using an HTTP GET operation yields the following results:

GET /documentstore/_design/documents_by_tag/
_view/documents_by_tag HTTP/1.1

{"total_rows":5,"offset":0,"rows":[
{
 "id":"0afc1fc2-7b39-461f-87cf-1ed1e21d2f34",
 "key":"Universe","value":"The Universe in a Nutshell"
},
{
 "id":"7b287e5d-c467-46ad-a10e-66d3c0696743",
 "key":"Universe","value":"The Theory of Everything"
},
{
 "id":"0afc1fc2-7b39-461f-87cf-1ed1e21d2f34",
 "key":"Physics","value":"The Universe in a Nutshell"},
{
 "id":"7b287e5d-c467-46ad-a10e-66d3c0696743",
 "key":"Physics","value":"The Theory of Everything"},
{
 "id":"0afc1fc2-7b39-461f-87cf-1ed1e21d2f34",
 "key":"Space","value":"The Universe in a Nutshell"}
]}

We call the view by using its name in the URL. The result is a key for every tag we encounter and the title of the document as value. CouchDB also provides the document identifier for each key/value pair. This way we can see that we have five tags for two documents.

Now in order to get all distinct document titles for a particular tag, we have to add the tag name as an extra query parameter called 'key':

GET /documentstore/_design/documents_by_tag/
_view/documents_by_tag?key=%22Universe%22 HTTP/1.1

This yields the following result from our view:

{"total_rows":5,"offset":0,"rows":[
{
 "id":"0afc1fc2-7b39-461f-87cf-1ed1e21d2f34",
 "key":"Universe",
 "value":"The Universe in a Nutshell"},
{
 "id":"7b287e5d-c467-46ad-a10e-66d3c0696743",
 "key":"Universe",
 "value":"The Theory of Everything"}
]}

This way we can also use different tag names and reuse our view. Notice that the total_rows attribute indicates that the original result set from our view contains five objects.

Doesn't seem to be very hard now doesn't it? Please do mind that I've barely scratched the surface here. For more information, you can take a look at the CouchDB wiki or check out these forthcoming books:

If you're interested in distributed, non-relational databases in general you might want to check out the recordings from the NOSQL meetup that also hosted a talk on CouchDB as well.

Till next time

Adding Attachments to a Document in CouchDB

To start off, here are the links to my previous posts about CouchDB:

  1. Relaxing on the Couch(DB)
  2. Installing the Couch(DB)
  3. PUTting the Couch(DB) in Your Living Room
  4. GETting Documents From CouchDB
  5. DELETE Documents From CouchDB

Today, I want to talk about how to create attachments for a document. Documents in CouchDB can have attachments just like an email. CouchDB has two ways for dealing with attachments:

  1. Inline Attachments
  2. Standalone Attachments

Inline Attachments

Inline attachments can be added to a document by using the dedicated _attachments attribute while PUTting the document into CouchDB.

PUT /documentstore/4f754d4b-540d-4b77-8507-6b7243ef8325 HTTP/1.1

{
"Author":"Stephen Hawking",
"Title":"The Universe in a Nutshell",
"Tags":[{"Name":"Physics"},{"Name":"Universe"}],
"_attachments":
{
  "The Universe in a Nutshell.pdf": 
  {
    "content_type": "application/pdf",
    "data": "JVBERi0xLjUNCiW1tbW1DQox ... "
  }
}}

For creating an attachment, we need to provide a file name, the MIME type and the base64 encoded binary data. Its even possible to have multiple attachments for a single document.

Standalone Attachments

Standalone attachments are a fairly recent feature of CouchDB that has been added to version 0.9. As it name implies, it involves adding, updating and removing attachments without the document itself being involved. 

PUT documentstore/4f754d4b-540d-4b77-8507-6b7243ef8325/
The%20Universe%20in%20a%20Nutshell.pdf?rev=1-1437623276 HTTP/1.0
Content-Length: 911240
Content-Type: application/pdf

JVBERi0xLjUNCiW1tbW1DQox ...

The major difference and advantage of this approach is that the binary data is sent directly to CouchDB without the need for a base64 conversion on both the client and server. This implies a significant performance improvement when storing attachments in CouchDB. Notice that you still need to provide a MIME type using the Content-Type header.

GETting Documents with Attachments

When retrieving a document with either an inline or standalone attachment, the actual binary data is not returned. Instead, CouchDB returns a stub to inform that the requested document has an attachment associated with it. 

GET /documentstore/4f754d4b-540d-4b77-8507-6b7243ef8325 HTTP/1.1

{
"_id":"4f754d4b-540d-4b77-8507-6b7243ef8325",
"_rev":"1-1969924333",
"Author":"Stephen Hawking",
"Title":"The Universe in a Nutshell",
"Tags":[{"Name":"Physics"},{"Name":"Universe"}],
"_attachments":
{
    "The Universe in a Nutshell.pdf":
    {
        "stub":true,
        "content_type":"application/pdf",
        "length":911240}
    }
}}

In order to retrieve the binary data of the attachment itself (yes please!), we have to issue a second GET but now using both the document identifier and the file name of the attachment.

GET /documentstore/4f754d4b-540d-4b77-8507-6b7243ef8325/
The%20Universe%20in%20a%20Nutshell.pdf HTTP/1.1

CouchDB responds to this request by returning the binary data of the attachment. When an inline attachment is used, the binary data automatically gets decoded.   

For my next post, I will talk about MapReduce functions providing a simple example of a Map function in particular. 

Till next time.

Wednesday, July 08, 2009

The Europe Virtual ALT.NET Blog

Colin has set up a dedicated blog for the Europe Virtual ALT.NET gatherings. We'll be posting all announcements, details of recordings and any related stuff to this blog.

If you're disgusted from your RSS reader, then maybe you'll want to follow up on the announcements and other stuff via Twitter? We've created an account there as well, so if you'd like you can follow here.

Saturday, July 04, 2009

DELETE Documents from CouchDB

To start off, here are the links to my previous posts about CouchDB:

  1. Relaxing on the Couch(DB)
  2. Installing the Couch(DB)
  3. PUTting the Couch(DB) in Your Living Room
  4. GETting Documents From CouchDB

Today, I want to talk about how to delete a document from CouchDB. In order to do that, we have to use the HTTP DELETE operation (how convenient). Removing a document from CouchDB can be done using the following request:

DELETE /documentstore/13ce4780-62c8-4074-9955-8c99966b84bb
?rev=1-2901013762

This makes CouchDB return the following response:

{
"ok":true,
"id":"13ce4780-62c8-4074-9955-8c99966b84bb",
"rev":"2-3500205450"
}

This confirms that the document has been removed by CouchDB. But is it really gone? Not exactly. We might be able to retrieve it using its revision number. Sending the following request:

GET /documentstore/13ce4780-62c8-4074-9955-8c99966b84bb
?rev=2-3500205450

still indicates that the document we just deleted still exists:

{
"_id":"13ce4780-62c8-4074-9955-8c99966b84bb",
"_rev":"2-3500205450",
"_deleted":true
}

Notice that the _deleted attribute indicates that the document has indeed been deleted. It's even possible to resurrect the document at hand by performing an update, bringing the document back amongst the living documents.

However, I don't consider that a good idea in all scenarios. In fact, as pointed out by this comment on my previous post, one should not blindly rely on the MVCC tokens for version control of documents, something that I neglected to point out in that post. When CouchDB is configured for compaction or replication with other instances of CouchDB, then this approach is not recommended. In case of compaction , CouchDB will actively purge old versions of documents and deleted documents. In case of replication, a particular node in a clustered environment doesn't necessarily have the complete version history of a document. Bottom line, use this feature very carefully.

For my next post, I will talk about how to create attachment for a document.

Till next time