2008-10-24

How I would like to write code

Aspect Oriented Programming looks great. It's something I have always wanted to use, but I avoid it because it doesn't work exactly how I would like it to. The first thing I want to avoid is lots of reflection at runtime (compile time is fine), it is for this reason I have mainly been interested in PostSharp.

PostSharp looks great! You decorate your class with attributes that you create yourself. After compiling your assembly PostSharp inspects the result for Attributes that are descended from one of its own special PostSharp AOP classes. When it sees these attributes it modifys your code in a specific way.

To use the same example as everyone else in the world (yawn) you could create an attribute from the method-boundard attribute. Override the methods declared on that attribute for entry/exit of the method, and write some code in there to write to the IDE's output window using System.Diagnostics.Debug.WriteLine(). Now when I add that attribute to a method on one of my classes I get that code injected into my method automatically

[Logging]
public void DoSomething()
{
}

would be the same as

public void DoSomething()
{
System.Diagnostics.Debug.WriteLine("Enter");
try
{
}
finally
{
System.Diagnostics.Debug.WriteLine("Exit");
}
}

So, this looks great, but what's so good / bad about it?

Good


On the good side it lets me describe code functionality rather than writing it. Instead of looking at loads of lines of code to see what something does we can look at a .NET attribute which simply tells us what generic task the class/method does. Take Assets as an example. An Asset is worth money so its location and movement history must always be tracked. To achieve this I would have assets always linked to an AssetHolder class...

Asset holder


Now, what qualifies as an asset holder? You could say that a person has an asset, a department could be using it, it could be in a stock room, or it could be out on hire to a customer.

Digressing slightly...Don't descend these classes from AssetHolder! Holding assets is something a person DOES, not something a person IS!

Person with asset holder


Now my Person class can hold assets. I can obtain its AssetHolder instance via its IAssetHolder interface, from there I can create AssetTransfer instances and so on. Now I have to implement the same thing for Department, Room, and Customer! Wouldn't it be nice if I could just do this?

[AssetHolder]
public class Person { }

[AssetHolder]
public class Department { }

[AssetHolder]
public class Room { }

[AssetHolder]
public class Customer { }

That's the good side of PostSharp. I could create an AOP attribute called "AssetHolderAttribute" and then build my business classes with that attribute adorning them. After a successful compilation PostSharp would discover these attributes and do the following to each of my classes

01: Add a member AssetHolder to the class.
02: Ensure that member is created within the constructor.
03: Implement IAssetHolder, returning this new member from the GetAssetHolder method

What we have is a very small amount of code (two words stuck together, wrapped in a pair of square brackets) which not only clearly identifies a feature of these classes, but even implements it

Bad


This is the only thing that is stopping me from using this approach to write applications. PostSharp cannot add the code until after the assembly has finished compiling. This means that an assembly consuming the business classes can easily write code such as

IAssetHolder holder = (IAssetHolder)person;

but code within the same assembly as the Person class cannot, because Person does not implement IAssetHolder until after the assembly has successfully compiled. A catch-22 situation!

How I would like to write code


I would love to see pre-compile time support for AOP, so that it feels like part of the development experience rather than like something that has been bolted onto my binary afterwards. Instead of a UML diagramming tool to help me to design my complex models, what I would like to do is to use the diagramming tool to design patterns that occur frequently in applications (archetypes), and then in simple code decorate my classes to identify what they are capable of...

[BusinessObject] //Adds persistence capability using your favourite framework
[AssetHolder] //Responsible for holding assets
[StockHolder] //Responsible for holding stock
[RoleHolder] //Can be assigned roles, such as Customer, Supplier, etc
public class Person
{
[DataMember] //This property is persistent
public string FirstName { get; set; }
}

//I could now write code like this
Person p = new Person(EcoSpace);
decimal heldValue = p.AssetHolder.GetHeldAssetValue();
heldValue += p.StockHolder.GetHeldStockValue();

Or for defining the model itself using simple code...

[BusinessObject]
public class PurchaseOrder
{
//Persistent property that is an AutoIncrementing column in the DB
[DataMember(SaveAction = SaveActionKind.DbAssigned)]
public int OrderNumber { get; private set; }
}

[BusinessObject]
public PurchaseOrderLine
{
//Also adds a property to PurchaseOrder called "Lines"
[Association(Aggregation = AggregationKind.Composite, OtherEndName="Lines")]
public PurchaseOrder Order {get; set; }
}

//I could now write code like this
var order = new PurchaseOrder(EcoSpace);
var line = new PurchaseOrderLine(EcoSpace);
line.Order = order;

//AOP created property "PurchaseOrder.Lines"!
if (order.Lines.Count != 1 || order.Lines[0] != line)
throw new ........

The future


I really hope to see pre-compiler support for AOP in the future in C#. I've been working on one of the guys at RemObjects recently who writes the Oxygene .NET language compiler (formerly known to me as "Chrome"). I have always thought their language has some really nice features in it, but never started using it because of two reasons

01: No ECO support.
02: I wanted to stay in C# to keep my skills more "main-stream".

Oxygene support is being worked on in ECO 5, which only leaves the second point. I really like some of their features as I said, but not quite enough to make me switch. However, if you are going to gain massively from working in a less popular language then switching to that language is the right thing to do, and this would be that feature for me! If in the future I need a C# job I believe my skills will remain current enough to be able to not exclude me from this part of the job market.

In the meantime I will continue to use C# and hope that some relentless annoying nagging will get me what I want on the Oxygene front, and that my dream of writing solid reliable code with very little effort takes that big leap towards becoming a reality! :-)

2008-10-17

Sufficient architecture

I have a friend who once decided to teach himself VB. We had a mutual friend who was the manager of a video shop so he decided to write some new software for him. He started off by creating an MS Access database to hold the data for his application. Each week I'd pop around to his house and he'd have MS Access open, reworking his tables etc. After a few months I asked "Have you started to write the app yet?".

"No, not yet." He replied, "I am trying to get the DB right first, then I will get onto writing the application". I asked what he meant by getting the DB right. He explained that he wanted to make this the best video-hire software ever, and that I should hear about some of the features that are going into it as a result of the DB structure he has made.

  • Record the cast of the film.
  • Record the director, producer etc.
  • Record the genre; horror, comedy, etc.
  • Record film information for films due to be released in the future.

All sorts of stuff. His idea was that he could analyse rental history and match up upcoming releases with people who might be interested in them. Then the computer could do a mailshot to the relevant people, enticing them to come back and rent out a film or two.

"Great ideas!" I said, "but.....Can members rent out films?"

"No, " he replied, "I haven't done that bit yet!"

This is the message I always try to get across when I am helping to design a new system. I either meet people who want their app to be a Swiss army knife application (who inevitably deliver late, and deliver rubbish) or those who want to write any old rubbish to get the job done and get it out of the door (which inevitably falls apart whenever a change is required). Someone once actually said to me "The problem with doing things properly is that it takes too long!".

Anyway, I read someone else's post on this subject this morning and just wanted to say

HEAR HEAR!

Don't go designing apps with features nobody has asked for, if there is something the customer wants in the future they are quick enough to ask for it, and if they really want it they will pay for it too!

2008-10-12

Accommodation manager - runtime error

I've just spotted something I omitted before zipping up my AccommodationManager app and making it available.

When you run the app in Release mode you will experience SQL errors. These errors are intermittent, and a query that worked only seconds ago might not always work. The reason for this is that ECO executes all queries within a transaction; SQLite creates a journal file for every transaction and then deletes it when done; and my anti-virus decides it wants to take a look at this new journal file to see what's inside it; resulting in SQLite not being able to open its own journal file exclusively.

This was something I noticed a while ago in another ECO+SQLite app of mine and the guy who writes the library spent a couple of hours with me on MSN trying to work out what the problem was (he had a fix to me by the next morning!). Anyway, I have updated the project so that the connection string tells SQLite not to delete the journal file when it has finished with it, as a consequence there is no conflict with Avast! anti-virus.

In case you don't want to download the example project again, here is the change you need to make inside AccommodationManager\AccommodationManagerPMP.cs

  private void ConfigureConnectionString()
  {
#if !DEBUG
    sqLiteConnection1.ConnectionString = string.Format(
      "Data Source={0};Version=3;Fail If Missing=True;Journal Mode=Persist", Settings.DataBaseFileName);
#endif
  }

2008-10-08

ECO, Winforms, ASP.NET, and WCF

The technologies I used in an app I wrote for friends recently. The app manages properties at different locations, bookings, and tariffs. In addition to this the application (which uses SQLite) connects to their website using WCF and updates their database so that people can check prices and availability.

I need to get them using it now so that there is data available by the time I put up the website, but I thought I'd share the app source anyway. Don't be afraid to run it and click Publish, it will only publish to your local machine, I'm not that stupid :-)

There source is available here.

Implementing complex unit testing with IoC

I have a method that looks something like this

public void DoSomething(SomeClass someInstance, User user)
{
  var persistence = someInstance.AsIObject.GetEcoService<IPersistenceService>();
  persistence.Unload(someInstance.AsIObject());
  
  if (someInstance.CurrentUser != null)
    throw new ..........;
  someInstance.CurrentUser = user;

  persistence.UpdateDatabase(someInstance);
}


This unloads the local cache before ensuring someInstance.CurrentUser == null, it then sets someInstance.CurrentUser and updates the DB. The unit test I wanted would check what happens when two users try to perform this at the same time. What I wanted was

User A: Unload
User B: Unload
User A: Check == null, it is
User B: Check == null, it is
User A: Change + update DB
User B: Change + update DB + experience a lock exception

What I didn't want was

User A: Unload
User B: Unload
User A: Check == null, it is
User A: Change + update DB
User B: Check == null, it isn't

To achieve two things running at once I need to either

A: Execute a line of code at a time for each user within the unit test instead of executing the method.
B: Execute the same method from two different threads.

Option A is easy to follow but is rubbish because it involves copying the method source out into the test method, no thanks! Option B is good but harder because I need to ensure that the two threads execute the lines of code in sync. What I really could do with is sync code inside the method, but there is no way I want to add additional sync logic because it is only needed for testing! However, there is already an opportunity to inject some code into the method; take a look here

  var persistence = someInstance.AsIObject.GetEcoService<IPersistenceService>();
  ...
  persistence.UpdateDatabase(someInstance);


Because we are using Inversion of Control it means that the method will call UpdateDatabase() on the reference we pass rather than a hard-coded reference. This means that we could quite easily replace the IPersistenceService with our own implementor and intercept the call to UpdateDatabase.

  public class PersistenceServiceWithEvents : IPersistenceService
  {
    //An event to call back
    public event EventHandler BeforeUpdateDatabase;

    //A reference to the original persistence service
    private readonly IPersistenceService PersistenceService;

    //Constructor
    public PersistenceServiceWithEvents(IEcoServiceProvider serviceProvider)
    {
      PersistenceService =
        serviceProvider.GetEcoService<IPersistenceService>();
    }

    //Interface implementation
    public void UpdateDatabase(IObjectProvider obj)
    {
      EventHandler handler = BeforeUpdateDatabase;
      if (handler != null)
        handler(this, EventArgs.Empty);
      
    }

    //Other methods omitted
  }


Now I have the opportunity to call back some code just before the real UpdateDatabase is executed, which gives me the chance to insert some thread sync' code:

  [TestMethod]
  public void HandlesConcurrency()
  {
    //Create a shared object to work with
    var ecoSpace = TestHelper.EcoSpace.Create();
    var someInstance = new SomeClass(ecoSpace);
    ecoSpace.UpdateDatabase();
    //Get it's external ID
    string someInstanceID = ecoSpace.ExternalIds.IdForObject(someInstance);

    //Create a thread sync object which both threads will wait for before
    //passing on their UpdateDatabase call to the original persistence servce
    var startUpdateDB = new ManualResetEvent(false);

    //Create a thread sync object which tells the test when this thread has called
    //PersistenceService.UpdateDatabase
    var user1ReadyToUpdateDB = new AutoResetEvent(false);

    bool user1Conflict = false;
    var user1ThreadStart = new ThreadStart(
        delegate()
        {
          HandlesConcurrency_UserEmulation(
            ecoSpace.PersistenceMapper,
            someInstanceID,
            user1ReadyToUpdateDB,
            startUpdateDB,
            out user1Conflict);
        }
      );


    //Create a thread sync object which tells the test when this thread has called
    //PersistenceService.UpdateDatabase
    var user2ReadyToUpdateDB = new AutoResetEvent(false);

    bool user2Conflict = false;
    var user2StartUpdateDB = new AutoResetEvent(false);
    var user2ThreadStart = new ThreadStart(
        delegate()
        {
          HandlesConcurrency_UserEmulation(
            ecoSpace.PersistenceMapper,
            someInstanceID,
            user2ReadyToUpdateDB,
            startUpdateDB,
            out user2Conflict);
        }
      );

    //Create and start both threads
    var user1Thread = new Thread(user1ThreadStart);
    var user2Thread = new Thread(user2ThreadStart);
    user1Thread.Start();
    user2Thread.Start();

    //Wait until they have both signalled that the call to UpdateDatabase has been reached
    user1ReadyToUpdateDB.WaitOne();
    user2ReadyToUpdateDB.WaitOne();

    //Both threads have now executed DoSomething() as far as the call to
    //persistenceService.UpdateDatabase and are waiting for me to tell them
    //to continue.
    startUpdateDB.Set();

    //Both threads will now wake up and call the original PersistenceService.UpdateDatabase.
    //Wait for both threads to finish so that the test does not end prematurely.
    user1Thread.Join();
    user2Thread.Join();

    //Check at least one experienced a conflict
    int lockCount = 0;
    if (user1Conflict)
      lockCount++;
    if (user2Conflict)
      lockCount++;
    Assert.AreEqual(1, lockCount, "One should experience a lock conflict");
  }

  private void HandlesConcurrency_UserActionEmulation(
    PersistenceMapper persistenceMapper,
    string someInstanceID,
    AutoResetEvent readyToUpdateDB,
    WaitHandle startUpdateDB,
    out bool lockConflict)
  {
    //SETUP

    //Create secondary ecospaces with the same persistence mapper as the
    //original. This is because my test EcoSpace uses a memory persistence mapper
    var ecoSpace = TestHelper.EcoSpace.CreateWithSharedMapper(persistenceMapper);

    //Create the persistence service with the BeforeUpdateDatabase event
    var callbackPersistenceService = new TestHelper.PersistenceServiceWithEvents(ecoSpace);
    //Register it
    ecoSpace.RegisterEcoService<IPersistenceService>(callbackPersistenceService);

    //Ensure update happens in sync, this occurs when we try to lock the object
    callbackPersistenceService.BeforeDatabaseUpdate += (sender, args) =>
      {
        //Notify we are ready to update
        readyToUpdateDB.Set();
        //Now wait to be told to complete the update
        startUpdateDB.WaitOne();
      };

    //ACT

    //Now test our code
    var someServiceToTest = new SomeService(ecoSpace);
    var someInstance = ecoSpace.ExternalIds.ObjectForId(someInstanceID).GetValue<SomeClass>();
    try
    {
      lockConflict = false;
      someServiceToTest.DoSomething(someInstance, new User(ecoSpace));
    }
    catch (OptimisticLockException)
    {
      lockConflict = true;
    }
  }


So there you have it. An example of how Inversion of Control and the Service Provider pattern can enable you to redirect calls in parts of your code to enable complex testing scenarios without having to change your implementation!

2008-10-03

Modeling ValueTypes

There are (apparently) two types of classes you can model. These are Entities and ValueTypes. An Entity would be something like a Company or a Person, where even two instances with the same data are not considered the same; whereas a ValueType would be something like a colour which would be considered the same no matter how many instances you had representing the value Red.

ValueType classes are usually modeled for things like an address. You might decide that whenever someone enters an address that already exists you should reference an existing instance rather than create a new instance with the same data, this could be used for example to see all deliveries made to a single address regardless of to whom the package was addressed.

The problem with ValueTypes is that they are shared across so many different classes. An address for example could be attached to a Person, a Delivery, a PurchaseOrder, all sorts. This is really only a problem because

A: How do you know when the ValueType is no longer referenced and can be deleted.
B: If you don't delete them then your DB will become full of data that is no longer needed.

The first point (A) was that we don't know when this ValueType is no longer referenced. The solution to this is very simple.

  [Address] (Address) 1 <>----0..* (References)[AddressReference]


The Address class has an aggregated association to the class AddressReference, meaning that if Address.References is not empty the Address cannot be deleted. The rule to obey is that we should never have any persistent associations to the Address class, the only associations to Address should always be instances of AddressReference, this our scheduled app can find all addresses to delete

  Address.allInstances->select(references->isEmpty)


Person (for example) would instead of having a reference to Address would have a privately owned AddressReference which it creates in its constructor

  public Person(IEcoServiceProvider serviceProvider)
  {
    ...
    this.CurrentAddressReference = new AddressReference(serviceProvider);
    ...
  }


The problem is that this code is not "very nice" to write:

  Person.CurrentAddressReference.Address = someAddress;


when what we really want is

  Person.Addres = someAddress;


So here's a comprimise. You make the association Person.AddressReference private, so it is not accessible via code at all. Then you create a transient association from Person to Address and mark it HasUserCode

  [Person] 0..* ------> 1 (CurrentAddress) [Address]


  public Address CurrentAddress
  {
    get { return CurrentAddressReference.Address; }
    set { CurrentAddressReference.Address = value; }
  }


So now we have a transient association with user code so ECO will always get/set via our property accessor methods, this propery is just an indirection for our private CurrentAddressReference. As a consequence we have the ability to use code and in-memory OCL such as

  somePerson.CurrentAddress = someAddress;


but the persistent association is actually made via an AddressReference class, so Address still only ever has one association end and is therefore easy to check if it is being associated from any class in the model.

Note: To do an OclPs evaluation you would need to use person.CurrentAddressReference.Address, as Person.Address is transient.

So, that's how we achieve the ability to reference a class and know its current reference count, but how do we go about deleting non-referenced values? The first thing to accept is that an individual client should not attempt to delete value types. Two clients could attempt to delete the ValueType at the same time, fail, and then no instance would ever try again, so your DB would still fill up. What you need is a server application scheduled to run regularly, maybe once per day (at a time when nobody is using it preferably, but this is not essential).

The final challenge is to determine which classes are ValueTypes, and what the name of their single association is. At this point I could suggest you use a tagged value on the class to identify it as a ValueType, and then use the runtime model information to find its only association, but I feel this is not expressive enough and the data is a bit hidden away. What I prefer is a .NET attribute identifying the class as a ValueType and also identifying the association member.

  [AttributeUsage(AttributeTargets.Class, AllowMultiple=false)]
  public class ValueTypeAttribute : Attribute
  {
    public readonly int MemberLoopbackIndex;
    public ValueTypeAttribute(int memberLoopbackIndex)
    {
      MemberLoopbackIndex = memberLoopbackIndex;
    }
  }


The member is identified using its member index, this way if you rename the association end your app will not compile until you fix your ValueType declarations.

  [ValueType(Address.Eco_LoopbackIndices.References_MemberIndex)]
  public class Address
  {
    ...
  }


Your application can now use reflection to find all classes which define the ValueType attribute:

  var valueTypeClasses =
    from c in typeof(MyPackage).Assembly.GetTypes()
    where c.GetCustomAttributes(typeof(ValueTypeAttribute), true).Length > 0
    select c;


Now you can loop through each class and delete non-referenced instances

  //Find all ValueType classes
  var valueTypeClasses =
    from c in typeof(MyPackage).Assembly.GetTypes()
    where c.GetCustomAttributes(typeof(ValueTypeAttribute), true).Length > 0
    select c;

  //Process each class in turn
  foreach (Type currentClass in valueTypeClasses)
  {
    //Get the ValueTypeAttribute for this class
    var attribute = (ValueTypeAttribute)
      currentClass.GetCustomAttributes(typeof(ValueTypeAttribute), true)[0];

    //Get a list of non-referenced instances
    using (var ecoSpace = new MyEcoSpace())
    {
      ecoSpace.Active = true;

      //Get the model information for this type
      IClass ecoClass = ecoSpace.TypeSystem.GetClassByType(currentClass);

      //Convert the loopback index to a feature index
      int featureIndex =
        ecoClass.GetStructuralFeatureIndexByLoopbackIndex(attribute.MemberLoopbackIndex);
      
      //Now get the name of the association
      string associationName = ecoClass.AllStructuralFeatures[featureIndex].Name;

      //Create our OCL expression
      string ocl =
        string.Format("{0}.allInstances->select({1}->isEmpty)",
          ecoClass.Name,
          associationName);

      //Get the non-referenced ValueType instances
      IObjectList objectsToDelete = ecoSpace.OclPs.Execute();

      
      foreach (IObject currentLocator in objectsToDelete)
      try
      {
        currentLocator.Delete();
      }
      catch (OptimisticLockException)
      {
        //After selecting someone has referenced the object, so ignore this exception
        //without deleting the instance
      }
    }//using ecoSpace
  }//foreach type