Showing posts with label ECO. Show all posts
Showing posts with label ECO. Show all posts

2008-12-30

Domain driven design, Test driven design

I was just reading through the VBUG events list when I came across an event entitled "Domain driven design approach, using unit testing".

"Sounds interesting!" I thought to myself, I hope I can make it! As I started to read it I thought it looked familiar. At that point I realised it was me doing the talk! So hopefully I will be able to make it :-)

The posting is here. If you come along make sure you say "Hello".

2008-10-12

Accommodation manager - runtime error

I've just spotted something I omitted before zipping up my AccommodationManager app and making it available.

When you run the app in Release mode you will experience SQL errors. These errors are intermittent, and a query that worked only seconds ago might not always work. The reason for this is that ECO executes all queries within a transaction; SQLite creates a journal file for every transaction and then deletes it when done; and my anti-virus decides it wants to take a look at this new journal file to see what's inside it; resulting in SQLite not being able to open its own journal file exclusively.

This was something I noticed a while ago in another ECO+SQLite app of mine and the guy who writes the library spent a couple of hours with me on MSN trying to work out what the problem was (he had a fix to me by the next morning!). Anyway, I have updated the project so that the connection string tells SQLite not to delete the journal file when it has finished with it, as a consequence there is no conflict with Avast! anti-virus.

In case you don't want to download the example project again, here is the change you need to make inside AccommodationManager\AccommodationManagerPMP.cs

  private void ConfigureConnectionString()
  {
#if !DEBUG
    sqLiteConnection1.ConnectionString = string.Format(
      "Data Source={0};Version=3;Fail If Missing=True;Journal Mode=Persist", Settings.DataBaseFileName);
#endif
  }

2008-10-08

ECO, Winforms, ASP.NET, and WCF

The technologies I used in an app I wrote for friends recently. The app manages properties at different locations, bookings, and tariffs. In addition to this the application (which uses SQLite) connects to their website using WCF and updates their database so that people can check prices and availability.

I need to get them using it now so that there is data available by the time I put up the website, but I thought I'd share the app source anyway. Don't be afraid to run it and click Publish, it will only publish to your local machine, I'm not that stupid :-)

There source is available here.

2008-10-03

Modeling ValueTypes

There are (apparently) two types of classes you can model. These are Entities and ValueTypes. An Entity would be something like a Company or a Person, where even two instances with the same data are not considered the same; whereas a ValueType would be something like a colour which would be considered the same no matter how many instances you had representing the value Red.

ValueType classes are usually modeled for things like an address. You might decide that whenever someone enters an address that already exists you should reference an existing instance rather than create a new instance with the same data, this could be used for example to see all deliveries made to a single address regardless of to whom the package was addressed.

The problem with ValueTypes is that they are shared across so many different classes. An address for example could be attached to a Person, a Delivery, a PurchaseOrder, all sorts. This is really only a problem because

A: How do you know when the ValueType is no longer referenced and can be deleted.
B: If you don't delete them then your DB will become full of data that is no longer needed.

The first point (A) was that we don't know when this ValueType is no longer referenced. The solution to this is very simple.

  [Address] (Address) 1 <>----0..* (References)[AddressReference]


The Address class has an aggregated association to the class AddressReference, meaning that if Address.References is not empty the Address cannot be deleted. The rule to obey is that we should never have any persistent associations to the Address class, the only associations to Address should always be instances of AddressReference, this our scheduled app can find all addresses to delete

  Address.allInstances->select(references->isEmpty)


Person (for example) would instead of having a reference to Address would have a privately owned AddressReference which it creates in its constructor

  public Person(IEcoServiceProvider serviceProvider)
  {
    ...
    this.CurrentAddressReference = new AddressReference(serviceProvider);
    ...
  }


The problem is that this code is not "very nice" to write:

  Person.CurrentAddressReference.Address = someAddress;


when what we really want is

  Person.Addres = someAddress;


So here's a comprimise. You make the association Person.AddressReference private, so it is not accessible via code at all. Then you create a transient association from Person to Address and mark it HasUserCode

  [Person] 0..* ------> 1 (CurrentAddress) [Address]


  public Address CurrentAddress
  {
    get { return CurrentAddressReference.Address; }
    set { CurrentAddressReference.Address = value; }
  }


So now we have a transient association with user code so ECO will always get/set via our property accessor methods, this propery is just an indirection for our private CurrentAddressReference. As a consequence we have the ability to use code and in-memory OCL such as

  somePerson.CurrentAddress = someAddress;


but the persistent association is actually made via an AddressReference class, so Address still only ever has one association end and is therefore easy to check if it is being associated from any class in the model.

Note: To do an OclPs evaluation you would need to use person.CurrentAddressReference.Address, as Person.Address is transient.

So, that's how we achieve the ability to reference a class and know its current reference count, but how do we go about deleting non-referenced values? The first thing to accept is that an individual client should not attempt to delete value types. Two clients could attempt to delete the ValueType at the same time, fail, and then no instance would ever try again, so your DB would still fill up. What you need is a server application scheduled to run regularly, maybe once per day (at a time when nobody is using it preferably, but this is not essential).

The final challenge is to determine which classes are ValueTypes, and what the name of their single association is. At this point I could suggest you use a tagged value on the class to identify it as a ValueType, and then use the runtime model information to find its only association, but I feel this is not expressive enough and the data is a bit hidden away. What I prefer is a .NET attribute identifying the class as a ValueType and also identifying the association member.

  [AttributeUsage(AttributeTargets.Class, AllowMultiple=false)]
  public class ValueTypeAttribute : Attribute
  {
    public readonly int MemberLoopbackIndex;
    public ValueTypeAttribute(int memberLoopbackIndex)
    {
      MemberLoopbackIndex = memberLoopbackIndex;
    }
  }


The member is identified using its member index, this way if you rename the association end your app will not compile until you fix your ValueType declarations.

  [ValueType(Address.Eco_LoopbackIndices.References_MemberIndex)]
  public class Address
  {
    ...
  }


Your application can now use reflection to find all classes which define the ValueType attribute:

  var valueTypeClasses =
    from c in typeof(MyPackage).Assembly.GetTypes()
    where c.GetCustomAttributes(typeof(ValueTypeAttribute), true).Length > 0
    select c;


Now you can loop through each class and delete non-referenced instances

  //Find all ValueType classes
  var valueTypeClasses =
    from c in typeof(MyPackage).Assembly.GetTypes()
    where c.GetCustomAttributes(typeof(ValueTypeAttribute), true).Length > 0
    select c;

  //Process each class in turn
  foreach (Type currentClass in valueTypeClasses)
  {
    //Get the ValueTypeAttribute for this class
    var attribute = (ValueTypeAttribute)
      currentClass.GetCustomAttributes(typeof(ValueTypeAttribute), true)[0];

    //Get a list of non-referenced instances
    using (var ecoSpace = new MyEcoSpace())
    {
      ecoSpace.Active = true;

      //Get the model information for this type
      IClass ecoClass = ecoSpace.TypeSystem.GetClassByType(currentClass);

      //Convert the loopback index to a feature index
      int featureIndex =
        ecoClass.GetStructuralFeatureIndexByLoopbackIndex(attribute.MemberLoopbackIndex);
      
      //Now get the name of the association
      string associationName = ecoClass.AllStructuralFeatures[featureIndex].Name;

      //Create our OCL expression
      string ocl =
        string.Format("{0}.allInstances->select({1}->isEmpty)",
          ecoClass.Name,
          associationName);

      //Get the non-referenced ValueType instances
      IObjectList objectsToDelete = ecoSpace.OclPs.Execute();

      
      foreach (IObject currentLocator in objectsToDelete)
      try
      {
        currentLocator.Delete();
      }
      catch (OptimisticLockException)
      {
        //After selecting someone has referenced the object, so ignore this exception
        //without deleting the instance
      }
    }//using ecoSpace
  }//foreach type

2008-09-17

Parameterised queries in ECO

Whenever I generate OCL queries in code I find myself having to escape user input in order to avoid making the query invalid, or allowing malicious input.

I've decided instead to use the ECO equivalent of parameterised queries (variables in ECO) and here is the result.

public static string CreateParameterisedQuery(
  this IEcoServiceProvider serviceProvider,
  string query,
  out IModifiableVariableList vars,
  params object[] args)
{
  vars = serviceProvider.GetEcoService<IVariableFactoryService>().CreateVariableList();
  for (int varIndex = 0; varIndex < args.Length; varIndex++)
  {
    string variableName = "autoVar_" + varIndex.ToString();
    query = query.Replace("{" + varIndex.ToString() + "}", variableName);
    vars.AddConstant(variableName, args[varIndex]);
  }
  return query;
}



To use this code you would do something like this

//1: Create the OCL with string.format style parameters
string query = "Person.allInstances" +
  "->select(name.sqlLikeCaseInsensitive({0}))" +
  "->select(gender = {1}";

//2: Parse the query and build the variable list
IModifyableVariableList vars;
query = self.AsIObject().ServiceProvider.CreateParameterisedQuery(query,
  out vars,
  "Peter Morris",
  Gender.Male);


Now you can use IOclPsService or IOclService to execute the new query passing the variables.

2008-07-15

AccommodationManager

I've decided to upload a project I have been working on over the weekend. So far it's taken about 5 hours, this includes me deciding exactly what it was I was going to write plus hunting for pretty graphics :-)

The requirement came about because I was asked if I could display booking availability on www.trentinney.co.uk - in addition they wanted to show availability for a number of other sites + the cost to stay during certain periods. I decided to write a single-user ECO app using SQLite as the database. This app will track customer information, bookings, properties, and prices; whenever the user is happy they will click a Publish button and it will communicate with a webservice to publish the data (this part is not yet written).

The app uses some practises I would recommend, such as
  1. Having one EcoSpace per form where possible in order to ensure your memory is freed when no longer needed.
  2. Using OCL constraints to validate user input - using my DroopyEyes extensions.
  3. Replacing the IPersistenceService to perform double-check no constraints are broken, and to allow classes to define PreSave validation for more complicated validation (such as doing in-DB searches for clashing bookings so that you don't have to load in all bookings into memory).
  4. Adding a non-ECO DBStructure table to the DB so that you can detect the current version of the DB and apply relevant SQL upgrade instructions when needed (no upgrade code in there yet).

To run the project you will need to download SQLite. You will also need to remove the AirSoftware.Common and Eco.Persistence.SQLite projects and re-add them (I have copied them into the zip file).

The file is available here. The file uses only a few classes so it will work in the trial too. The project is a Visual Studio 2008 one.

2008-07-08

More leak fixes

I have changed the DirtyObjectCatcher so that it initially only hooks

Form.Disposed - Automatically disposes the DirtyObjectCatcher
Form.Closed - Unhooks all additional form events (below)
Form.Shown - To hook additional form events (below)

==Additional form events==
Form.Activated
Form.MdiParent.Activated
Form.MdiChildActivate

The additional events are to ensure that the DirtyObjectCatcher's undo block is moved to the top. The reason that these events are now unhooked is so that there is no strong event references from the application's main form (Form.MdiParent) to this component, keeping it alive. Now we only have long-term event references from the owning form itself. Really though this is just an added precaution against something I may not have thought of :-)

The true memory saver comes from only holding a WeakReference to the owning form. Otherwise in an MDI application we have the following

MainForm.MdiChildActivate->DirtyObjectCatcher->Form

In such a case closing the MDI child form will not be collected it because it is referenced by the DirtyObjectCatcher, which cannot be collected because it is referenced by the application's main form. Unhooking these events and holding only a WeakReference prevents the leakage.

In addition I have hooked Form.Disposed so that this component is disposed along with its owner. I have also hooked DirtyObjectCatcher.Disposed from ObjectValidator so that it may also auto-dispose itself and knows not to place any new subscriptions.

Again the files are available here.

In addition I was able to track down and reproduce a couple of leaks in ECO 4 which are now fixed internally.

  1. Deactivating / Reactivating an EcoSpace would leak an object reference each time. This is not the case if you allow the EcoSpace to be collected.
  2. Using OclVariables with OclPsHandle and calling EcoSpace.Persistence.Refresh would leak a single object reference, also if OclPsHandle.Execute was executed.

I monitored my app with over 4,000 tasks today (normally there are about 200) for a few hours and the memory usage didn't budge!

2008-06-27

Memory leaks

As a follow up to yesterday's post here is a list of problems I found...

01: The following code for some reason causes the form holding the DirtyObjectCatcher to remain referenced.


if (Owner.MdiParent != null)
{
Owner.MdiParent.Activated += new EventHandler(MdiParent_Activated);
Owner.MdiParent.MdiChildActivate += new EventHandler(MdiParent_MdiChildActivate);
}
The odd thing about this is that it really is the events that matter! I subscribe Shown, Disposed, Activated on the Owner (which is a form) and that doesn't cause the same behaviour. The reason is that the Owner normally gets disposed and takes out the DirtyObjectCatcher with it, however, if I have Owner.MdiParent events referencing DirtyObjectCatcher then the form will never get disposed because DirtyObjectCatcher has a strong reference to Owner. Maybe I should change it, but for now the Shown and Activated events seem to be doing the trick.

02: This one was a real pain! DirtyObjectCatcher creates its own UndoBlock and then subscribes to it, so that whenever the user modifies an object it goes into the UndoBlock and the catcher is notified. At this point other components (such as ObjectValidator) can get a list of modified objects and do something (such as evaluate their OCL constraints).

Unfortunately this event fires at some point between the user making the change and the ECO cache being ready for the new value to be read. For this reason in the past I had to set

Application.Idle += CheckForModifiedObjects;

CheckForModifiedObjects would then trigger the relevant events in Application's Idle event so that we know the changes to the ECO cache are ready to be read. Unfortunately I made a silly mistake, I forgot to do this as the first line of CheckForModifiedObjects

Application.Idle -= CheckForModifiedObjects;

As a result the static Application class had a reference to my DirtyObjectCatcher via an event. DirtyObjectCatcher holds a reference to the form (Owner) and the EcoSpace (ServiceProvider). So the form and the EcoSpace would remain in memory.

03: In the past you could not subscribe to UndoBlock so I had to subscribe to DirtyListService. Unfortunately this would not get triggered if you activated a DirtyObjectCatcher on an object that was already dirty, or transient. To overcome this I added a CheckForUpdates method which would get every instance of DirtyObjectCatcher to check its UndoBlock for changes.

The idea was that you plug into the ECO cache and call the static DirtyObjectCatcher.CheckForUpdates whenever something changed. A pain in the ass. I requested UndoBlock.Subscribe and it was implemented within days (thanks Jonas!) but I left that code in just in case anyone was using it. Unfortunately this meant I had a static List sitting around holding references to my DirtyObjectCatchers. Again this referenced the EcoSpace and the Form, so the Form was never disposed in order to dispose of my DirtyObjectCatcher.

This wasn't noticed for some time because I use the AutoFormService to handle the lifetime of my forms, but my current app has quite a few forms where I manage the lifetime manually so the form wasn't automatically being disposed as I had expected. In addition the Form class does not call Dispose on any of its components when you call its Dispose method, which I find very strange!

Anyway, that's the lot. What a hellish week! I have uploaded the lastest version of EcoExtensions here:

www.peterlesliemorris.com/blogfiles/ecoextensions.zip

DirtyObjectCatcher

Oh boy, what a nightmare! After days of messing around I finally found where the memory leak is in my app, it was in DirtyObjectCatcher!

The DirtyObjectCatcher used to subscribe to the DirtyListService, so that it was notified whenever an object was made dirty. I experienced this problem...

01: User creates a "Call" to a customer site.
02: User edits a purchase order.
03: Save purchase order (merges the undo block to the Call undo block and closes the form)
04: Edit the purchase order again from the Call form

The PurchaseOrder is already dirty so it wont get triggered again, this used to result in no constraints being checked etc and the possibility of entering dodgy data. The solution at the time was to have a static list in DirtyObjectCatcher

private List Instances;

whenever a new instance was created it would be added, whenever Dispose was called it would be removed. I then hooked into the cache chain and whenever a value changed I would call a static method on DirtyObjectCatcher which iterated over Instances and told each to check if anything had changed. It worked fine enough, and I put in a request to add a Subscribe method to IUndoBlock.

My request for IUndoBlock.Subscribe was soon added so I changed the code. Now it worked lovely! Unfortunately there was a problem! I had left the static code in the component just in case anyone was using it, I should really have just removed it as it is now obsolete. The really big problem however was that I had never added a Finalizer to the DirtyObjectCatcher class to call Dispose(). I had assumed the following in the Windows.Form default code template would call Dispose.



private System.ComponentModel.IContainer components = null;
protected override void Dispose(bool disposing)
{
if (disposing && (components != null))
{
components.Dispose();
}
base.Dispose(disposing);
}



However I was wrong. The components on the form are never added to this "components" member. As a result the Instances list grew and grew, and was never emptied. DirtyObjectCatcher holds a reference to the EcoSpace, so this is why the memory usage got bigger and bigger! I was going to add the Finalizer and remove the instance from Instances, but I have decided to remove Instances completely!

Update: There were some other odd issues too which I will blog about tomorrow after I have released an update!

2008-06-16

TimeBasedSyncHandler

I recently had to tweak my remote persistence server settings. I noticed that I had leaft the SyncHandler.HistoryLength at the default value of 10,000 items. This was overkill because my clients sync every 3 seconds. I thought maybe I should drop this down to about 100, that should be okay? Each client only does about one update every few minutes so I it should, right?

Problem is not all of my users are people. One user is a messenger service which looks for unsent messages, sends them one at a time, marking each in turn as sent and updating the database. This gets run every five minutes so probably sends about fifty messages at the most, but what if someone decides to change it to ten minutes, or thirty? So maybe I should increase the HistoryLength to about 200?

Then there is the other client, this one syncs with an external database. This could perform hundreds of updates every minute. If the messenger is set to run every thirty minutes...I'm not sure what a good HistoryLength would be! If I set it too low I fill it up quickly and my clients will have to deactivate/reactivate their EcoSpace instances. If I set it too high I might use up too much memory, especially if the messenger runs infrequently and the history gets filled up with "human" client updates which are much larger.

So, I wrote my own SyncHandler. This one accepts a TimeSpan property, which by default is five minutes but on my server I dropped it to two minutes (forty times longer than my clients' sync intervals.) The idea is that I keep changes for the specified period of time no matter how irregularly the updates occur. This way the server uses more memory during heavy usage but

A: Usage drops during lower activity, in fact it goes right down if there are no updates for the specified TimeSpan.
B: It wont run out of history length too quickly for the client.

If you'd like to try it for yourself take a look here.

2008-06-11

Making a generic type from a Type

List<Person> p = new List<Person>();

This works fine, but what about this?

Type someType = typeof(Person);
List<someType> p = new List<someType>();

Nope!

But you can do this...

Type genericType = typeof(List<>).MakeGenericType(someType);

Why would I want to? Because I am creating a tool that generates plain old .NET objects from an EcoSpace's model, and I wanted to implement multi-role association ends as

public List<Building> RoleName;

rather than just

public Building[] RoleName;

2008-05-28

Hooking into ECO multi-association events

Although I have not (yet) needed this myself I can see myself needing it in the future and the question has been asked before.

"Setting HasUserCode=True on a Child.Parent single role does what I want, but how do I handle the scenario where Parent.Children.Add(item) is called on a multirole?"

By default you can’t, but with the addition of a single class and a small amount of tweaking you can get it to do what you want! Here is how to do it:


01: Mark Parent.Children’s association end with HasUserCode=True in the modeler and then generate code.
02: In the source code of your class (not within an ECO region) add the following


  private EcoMultiAssociation<Child> m_Children;


This is a class that does not yet exist, I will show the source code for it later.


02: In the source code locate the "Children" property and change it like so

  public IEcoList<Child> Children
  {
    get
    {
      if (m_Children == null)
      {
        m_Children= new EcoMultiAssociation<Child>((IList)(this.eco_Content.get_MemberByIndex(Eco_LoopbackIndices.Children_MemberIndex)));
        m_Children+= new AssociationItemChangedEventHandler<Child>(result_ItemChanged);
        m_Children+= new AssociationChangedEventHandler<Child>(result_ItemInserted);
        m_Children+= new AssociationChangedEventHandler<Child>(result_ItemRemoved);
      }
      return m_Children

#if NeverDoThis
      #region MM_ECO_Generated
      return new ObjectListAdapter<Child>((IList) (this.eco_Content.get_MemberByIndex(Eco_LoopbackIndices.Children_MemberIndex)));
      #endregion
#endif
    }
  }


Note that I have put an #if around the original code that will never be true. You cannot remove the MM_ECO_Generated section due to the source code generator expecting to find it, but you can make sure it is never even compiled! The parameters for the EcoMultiAssociation<T> constructor were just copied directly from the ObjectListAdapter constructor below.

(Just a small note, I don’t usually name my private members m_Name, I just did it in this case to make it easier to spot the difference between the Children property and the m_Children private member).


03: The event handlers are implemented like so

    void result_ItemRemoved(object sender, AssociationChangedEventArgs<Child> args)
    {
      System.Diagnostics.Debug.WriteLine(string.Format("Removed index {0}", args.Index));
    }

    void result_ItemInserted(object sender, AssociationChangedEventArgs<Child> args)
    {
      System.Diagnostics.Debug.WriteLine(string.Format("Inserted at index {0}", args.Index));
    }

    void result_ItemChanged(object sender, AssociationItemChangedEventArgs<Child> args)
    {
      System.Diagnostics.Debug.WriteLine(string.Format("Changed object at index {0}", args.Index));
    }


The args parameter has a reference to the old object and also the new object in the case of ItemChanged which is executed when you do this...

Association[x] = y;



Finally here is the source code for the EcoMultiAssociation<T> class. There’s quite a bit here, but that’s really because I have to implement so many interfaces, the actual code is very small.

  public delegate void AssociationChangedEventHandler<T>(object sender, AssociationChangedEventArgs<T> args);
  public class AssociationChangedEventArgs<T> : EventArgs
  {
    public readonly T Item;
    public readonly int Index;

    public AssociationChangedEventArgs(int index, T item)
    {
      Item = item;
      Index = index;
    }
  }

  public delegate void AssociationItemChangedEventHandler<T>(object sender, AssociationItemChangedEventArgs<T> args);
  public class AssociationItemChangedEventArgs<T> : AssociationChangedEventArgs<T>
  {
    public readonly T OriginalItem;

    public AssociationItemChangedEventArgs(int index, T newItem, T originalItem)
      : base(index, newItem)
    {
      OriginalItem = originalItem;
    }
  }


  public class EcoMultiAssociation<T> : IEcoList<T>, IList
  {
    private readonly IList Adaptee;

    public EcoMultiAssociation(IList adaptee)
    {
      if (adaptee == null)
        throw new ArgumentNullException("Adaptee");

      Adaptee = adaptee;
    }

    public void Add(T item)
    {
      if (Adaptee.IndexOf(item) == -1)
      {
        Adaptee.Add(item);
        OnItemInserted(Count - 1, item);
      }
    }

    public void Clear()
    {
      List<T> originals = new List<T>(this);
      Adaptee.Clear();
      for (int index = 0; index < originals.Count; index++)
        OnItemRemoved(index, originals[index]);
    }

    public bool Contains(T item)
    {
      return Adaptee.Contains(item);
    }

    public void CopyTo(T[] array, int arrayIndex)
    {
      Adaptee.CopyTo(array, arrayIndex);
    }

    public int Count
    {
      get { return Adaptee.Count; }
    }

    public IEnumerator<T> GetEnumerator()
    {
      return new ObjectEnumeratorAdapter<T>(Adaptee.GetEnumerator());
    }

    public int IndexOf(T item)
    {
      return Adaptee.IndexOf(item);
    }

    public void Insert(int index, T item)
    {
      if (Adaptee.IndexOf(item) == -1)
      {
        Adaptee.Insert(index, item);
        OnItemInserted(index, item);
      }
    }

    public bool IsReadOnly
    {
      get { return Adaptee.IsReadOnly; }
    }

    public bool Remove(T item)
    {
      int index = Adaptee.IndexOf(item);
      if (index == -1)
        return false;

      Adaptee.RemoveAt(index);
      return true;
    }

    public void RemoveAt(int index)
    {
      T item = this[index];
      Adaptee.RemoveAt(index);
      OnItemRemoved(index, item);
    }

    public T this[int index]
    {
      get
      {
        return (T)Adaptee[index];
      }
      set
      {
        T originalItem = (T)Adaptee[index];
        Adaptee[index] = value;
        OnItemChanged(index, originalItem, value);
      }
    }

    public event AssociationChangedEventHandler<T> ItemInserted;
    protected void OnItemInserted(int index, T item)
    {
      AssociationChangedEventHandler<T> handler = ItemInserted;
      if (handler != null)
      {
        AssociationChangedEventArgs<T> args = new AssociationChangedEventArgs<T>(index, item);
        handler(this, args);
      }
    }

    public event AssociationChangedEventHandler<T> ItemRemoved;
    protected void OnItemRemoved(int index, T item)
    {
      AssociationChangedEventHandler<T> handler = ItemRemoved;
      if (handler != null)
      {
        AssociationChangedEventArgs<T> args = new AssociationChangedEventArgs<T>(index, item);
        handler(this, args);
      }
    }

    public event AssociationItemChangedEventHandler<T> ItemChanged;
    protected void OnItemChanged(int index, T originalItem, T newItem)
    {
      AssociationItemChangedEventHandler<T> handler = ItemChanged;
      if (handler != null)
      {
        AssociationItemChangedEventArgs<T> args = new AssociationItemChangedEventArgs<T>(index, newItem, originalItem);
        handler(this, args);
      }
    }


    #region IEnumerable Members

    IEnumerator IEnumerable.GetEnumerator()
    {
      return GetEnumerator();
    }

    #endregion

    #region IList Members

    int IList.Add(object value)
    {
      Add((T)value);
      return IndexOf((T)value);
    }

    bool IList.Contains(object value)
    {
      return Contains((T)value);
    }

    int IList.IndexOf(object value)
    {
      return IndexOf((T)value);
    }

    void IList.Insert(int index, object value)
    {
      Insert(index, (T)value);
    }

    bool IList.IsFixedSize
    {
      get { return Adaptee.IsFixedSize; }
    }

    void IList.Remove(object value)
    {
      Remove((T)value);
    }

    object IList.this[int index]
    {
      get
      {
        return this[index];
      }
      set
      {
        this[index] = (T)value;
      }
    }

    #endregion

    #region ICollection Members

    void ICollection.CopyTo(Array array, int index)
    {
      Adaptee.CopyTo(array, index);
    }

    bool ICollection.IsSynchronized
    {
      get { return Adaptee.IsSynchronized; }
    }

    object ICollection.SyncRoot
    {
      get { return Adaptee.SyncRoot; }
    }

    #endregion

  }



Note that these events will not be executed if you do Child.Parent = p; For this case you have to set HasUserCode=True on Child.Parent as normal and react accordingly.

2008-04-07

New look

www.capableobjects.com has a new look. Nice!

2008-03-12

Small sample website

I'm playing with a little website called "Not A Sausage". I will reveal what it is for later :-)

I've decided to make it available for download for those of you who might want to take a look. It uses the MS Web Extensions so you will need to install those first from here. You will also need Rhino Mocks from here. Ofcourse you will also need ECO!

When you download the project (VS 2008, C# 3.5) you will need to add the following references to the NotASausageWebsite project
  • System.Web.Abstractions
  • System.Web.Mvc
  • System.Web.Routing
and the following to the Tests project
  • System.Web.Abstractions
  • System.Web.Mvc
  • System.Web.Routing
  • Rhino.Mocks
The System.Web.* files will be in {Program Files}\Microsoft ASP.NET MVC Preview 2\Assemblies

All it does so far is
  • Allow you to create an account.
  • Show errors based on modeled constraints if the user input is invalid.
  • Checks for duplicate email addresses by using an InPS evaluation so that not all users are loaded into memory.
  • Sends an email to the user with a temporary login URL (to validate their email address).
In addition to this the site (so far) is fully unit tested. The tests were written before the actions were implemented. It uses mock objects (Rhino Mocks) to mock the whole HTTP request, and even mocks out other services so that I can check an email is sent to the user without actually having to send the email.

Anyway, take a look and see what you think. You can download the project here.

2008-03-10

Test Driven MVC and ECO

I have decided that mocking IEcoServiceProvider is not the way to go. Your controller will use the mocked provider during testing but


     
  1. You don’t want to have to mock every service the provider may return, it’s a lot of work!

  2.  
  3. You don’t want your controller using a mocked service, and then the EcoSpace using the real one!



At first I was mocking every possible service request. IUndoService, IPersistenceService, IOclService, etc. I get bored typing them out in this blog, so doing it in tests was really annoying me. I decided I would instead only mock the service in question. So if I were ensuring that an action wont save an object with broken constraints I would mock GetEcoService<IConstraintProvider> and ensure that I always got a broken constraint.

The problem was that the test to ensure I can save a valid object would then invoke the IPersistenceService.UpdateDatabaseWithList method. In my EcoSpace I have decorated my persistence service so that it checks every object in the update list to ensure it has no invalid constraints. At the point it asks for IConstraintProvider it is using the real IEcoServiceProvider and as a result it gets the real IConstraintProvider. In short the object would only save if it were really valid, and not if my mocked constraint provider pretended it was.

Ultimately I found it much easier just to register the mock service on the EcoSpace. To do this all I had to do was to expose a public method on the EcoSpace like so

#if DEBUG
 public void RegisterMockService(type serviceType, object serviceInstance)
 {
  RegisterEcoService(serviceType, serviceInstance);
 }
#endif


Now I can replace any service on the EcoSpace, so even the EcoSpace itself will get mocked services during testing. This is a lot less work!


Talking of a lot less work, although I was initially annoyed that the new field test for the ASP .NET web extensions had switched from using interfaces back over to using objects it turns out that testing is actually much easier than it was previously. Scott Hanselman posted some testing classes on his blog recently, it came out all screwy so I will repost the corrected version below (I hope he doesn’t mind). Testing is now as easy as this...

[TestClass]
public class AccountTests
{
 //This is a descendant of my real EcoSpace,
 //but I replace the persistence mapper with
 //PersistenceMapperMemory after construction so that
 //no DB access is needed.
 MemoryEcoSpace EcoSpace;
 AccountController Controller;
 MockRepository Mocks;
 FakeViewEngine FakeViewEngine;

 [TestInitialize]
 public void SetUp()
 {
  EcoSpace = new MemoryEcoSpace();
  EcoSpace.Active = true;

  Mocks = new MockRepository();
  FakeViewEngine = new FakeViewEngine();
  Controller = new AccountController();
  Controller.ViewEngine = FakeViewEngine;
  using (Mocks.Record())
  {
   Mocks.SetFakeControllerContext(Controller);
  }
 }

 [TestMethod]
 public void Create()
 {
  using (Mocks.Playback())
  {
   Controller.Create();
   Assert.AreEqual("Create", FakeViewEngine.ViewContext.ViewName);
  }
 }

 [TestMethod]
 public void CreateUpdate_ConfirmationEmailDoesNotMatch()
 {
  using (Mocks.Playback())
  {
   Controller.CreateUpdate("Mr", "Peter", "Morris", "me@home.com", "");
   Assert.IsTrue(ValidationHelper.ErrorExists(Controller.ViewData, NotASausageWebsite.Constants.ErrorMessages.ConfirmationEmailAddressDoesNotMatchEmailAddress));
  }
 }
}


public static class ValidationHelper
{
 public static bool ErrorExists(IDictionary<string, object> viewData, string errorMessage)
 {
  if (!viewData.ContainsKey(NotASausageWebsite.Constants.ViewDataKeys.Global.ErrorMessages))
   return false;
  List<string> errors = (List<string>)viewData[NotASausageWebsite.Constants.ViewDataKeys.Global.ErrorMessages];
  return errors.IndexOf(errorMessage) >= 0;
 }
}



Nice and easy! If you want the testing code from Scott here it is:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Web;
using Rhino.Mocks;
using System.Web.Mvc;
using System.Web.Routing;
using System.Collections.Specialized;

namespace Tests.Helpers
{
public static class MvcMockHelpers
{
public static HttpContextBase FakeHttpContext(this MockRepository mocks)
{
HttpContextBase context = mocks.PartialMock<HttpContextBase>();
HttpRequestBase request = mocks.PartialMock<HttpRequestBase>();
HttpResponseBase response = mocks.PartialMock<HttpResponseBase>();
HttpSessionStateBase session = mocks.PartialMock<HttpSessionStateBase>();
HttpServerUtilityBase server = mocks.PartialMock<HttpServerUtilityBase>();

SetupResult.For(context.Request).Return(request);
SetupResult.For(context.Response).Return(response);
SetupResult.For(context.Session).Return(session);
SetupResult.For(context.Server).Return(server);

mocks.Replay(context);
return context;
}

public static HttpContextBase FakeHttpContext(this MockRepository mocks, string url)
{
HttpContextBase context = FakeHttpContext(mocks);
context.Request.SetupRequestUrl(url);
return context;
}

public static void SetFakeControllerContext(this MockRepository mocks, Controller controller)
{
var httpContext = mocks.FakeHttpContext();
ControllerContext context = new ControllerContext(new RequestContext(httpContext, new RouteData()), controller);
controller.ControllerContext = context;
}

static string GetUrlFileName(string url)
{
if (url.Contains("?"))
return url.Substring(0, url.IndexOf("?"));
else
return url;
}

static NameValueCollection GetQueryStringParameters(string url)
{
if (url.Contains("?"))
{
NameValueCollection parameters = new NameValueCollection();

string[] parts = url.Split("?".ToCharArray());
string[] keys = parts[1].Split("&amp;".ToCharArray());

foreach (string key in keys)
{
string[] part = key.Split("=".ToCharArray());
parameters.Add(part[0], part[1]);
}

return parameters;
}
else
{
return null;
}
}

public static void SetHttpMethodResult(this HttpRequestBase request, string httpMethod)
{
SetupResult.For(request.HttpMethod).Return(httpMethod);
}

public static void SetupRequestUrl(this HttpRequestBase request, string url)
{
if (url == null)
throw new ArgumentNullException("url");

if (!url.StartsWith("~/"))
throw new ArgumentException("Sorry, we expect a virtual url starting with \"~/\".");

SetupResult.For(request.QueryString).Return(GetQueryStringParameters(url));
SetupResult.For(request.AppRelativeCurrentExecutionFilePath).Return(GetUrlFileName(url));
SetupResult.For(request.PathInfo).Return(string.Empty);
}

}
}



 public class FakeViewEngine : IViewEngine
 {
  public void RenderView(ViewContext viewContext)
  {
   ViewContext = viewContext;
  }

  public ViewContext ViewContext { get; private set; }

 }

2008-03-07

ECO, LINQ, Anonymous types, and Web Extensions

I’ve been finding LINQ + Anonymous types really compliment ECO and the new ASP web extensions approach to writing websites. I may have mentioned recently that I don’t like the idea of passing instances of my business objects to the presentation layer. The reason is that someone else will be writing the views for this site and I want to be able to control what they are capable of displaying. It’s not just that though, the fact is that your view might need to look completely different to how your business classes are structured, one layer should not dictate the structure of another.

The example I am about to show does in fact have similar structures for the view and model. Having said that there is a slight difference in that the MinorVersion class has its own "int VersionNumber" property, and gets the major part of the version number from self.MajorVersion.VersionNumber. Anyway, now to get on with it.

My requirement was to show all major versions, within each major version show each minor version, and within each minor version show a list of what’s new. In addition, a minor version should only be displayed if its status is #Released, and a major version should not be displayed if it has no minor versions which meet this criteria.


The following code generates a structure like so

MajorVersion (VersionNumber)
 1..* MinorVersion (MajorVersionNumber, MinorVersionNumber)
  1..* WhatsNew (ID, Headline)



and stores the resulting anonymous type into the ViewData for my view to render.

ViewData[GlobalViewDataKeys.WhatsNewKeys.WhatsNewList] = 
 from majorVersion in software.MajorVersions
 where
  (from minorVersionCheck in majorVersion.MinorVersions
   where minorVersionCheck.Status == MinorVersionStatus.Released select minorVersionCheck ).Count() > 0
 select
  new
  {
   VersionNumber = majorVersion.VersionNumber,
   MinorVersions =
    from minorVersion in majorVersion.MinorVersions
    select
     new
     {
      MajorVersionNumber = majorVersion.VersionNumber,
      VersionNumber = minorVersion.VersionNumber,
      WhatsNew =
       from whatsNew in minorVersion.WhatsNew
       select
        new
        {
         ID = whatsNew.ID,
         Headline = whatsNew.Headline
        }
     }
  };
RenderView("AllHistory");




The code behind of my view reads like this:

protected void Page_Load(object sender, EventArgs e)
{
 MajorVersionRepeater.DataSource = ViewData[GlobalViewDataKeys.WhatsNewKeys.WhatsNewList];
 MajorVersionRepeater.DataBind();
}



And finally I use nested ASP:Repeater tags to render the nested HTML.


<ul class="AllHistoryMajorVersionList">
 <asp:Repeater id="MajorVersionRepeater" runat="server">
  <ItemTemplate>
   <li>
    Major version
     <%# DataBinder.Eval(Container.DataItem, "VersionNumber") %>
    <ul class="AllHistoryMinorVersionList">
     <asp:Repeater
      id="MinorVersionRepeater"
      DataSource=’<%# DataBinder.Eval(Container.DataItem, "MinorVersions") %>’
      runat="server">
      <ItemTemplate>
       <li>
        Minor version
        <%# DataBinder.Eval(Container.DataItem, "MajorVersionNumber") %>.
        <%# DataBinder.Eval(Container.DataItem, "VersionNumber") %>
        <ul class="AllWhatsNewList">
         <asp:Repeater
          id="WhatsNewRepeater"
          DataSource=’<%# DataBinder.Eval(Container.DataItem, "WhatsNew") %>’
          runat="server">
          <ItemTemplate>
           <li>
            <a href="/WhatsNew/View/<%# DataBinder.Eval(Container.DataItem, "ID") %>">
             <%# DataBinder.Eval(Container.DataItem, "Headline") %>
            </a>
           </li>
          </ItemTemplate>
         </asp:Repeater>
        </ul>
       </li>
      </ItemTemplate>
     </asp:Repeater>
    </ul>
   </li>
  </ItemTemplate>
 </asp:Repeater>
</ul>



I think the point is that there is just no need to pass your business class instances through to the UI layer. In fact if you later changed the structure of your business classes this LINQ would no longer compile, whereas the markup in the view is evaluated at runtime so you wouldn’t spot an error here until you tried to view the page.

2008-02-20

Embedded Firebird, error trying to write to file

This error has been really annoying me tonight!

I have an app that uses Embedded Firebird for its DB so that I don't need to
install a DB server. On Vista my app throws an exception "Error trying to
write to file (the correct path here)".

I recreated the DB on my development machine (XP) and tried running it, it should work, it has for months, but it didn't! The same error too!

For the life of me I couldn't work out why it would suddenly stop working on both machines, what did they have in common? I uninstalled stuff, reinstalled it, etc, no joy.

The answer on my XP box was simple. I used the local server to create the GDB file + generate my DB structure using ECO. What I hadn't thought of was the fact that the firebird server then holds a file handle open on that GDB file in case I want to use it again. Embedded firebird needs an exclusive lock on the file so this was the problem on my XP box. I wish the error had read something like "Error trying to write to file, unable to obtain an exclusive lock", would have saved me some time!

However, I don't have the firebird server installed on my Vista test machine so what was causing the problem there? It seems that embedded firebird cannot access the GDB on vista if it is in the CommonApplicationData folder. This is a real pain because

A: I need the database in a common place so that any user using the software will see the same data.
B: This is where it is supposed to go!

So doesn't the current user have sufficient privileges to write to this folder? The following snippet of test code says they do.

static void Main(string[] args)
{
  string fileName = Environment.GetFolderPath(System.Environment.SpecialFolder.CommonApplicationData);
  fileName = Path.Combine(fileName, "MyTest.txt");
  Console.WriteLine("Writing to " + fileName);
  StreamWriter sw = new StreamWriter(fileName);
  using (sw)
    sw.WriteLine("Hello");
  Console.WriteLine("Done");
  Console.ReadLine();
}



I have checked and the database file is in CommonApplicationData on the Vista machine, so it's not as though I am installing into the wrong folder or something either.

The only thing I can think of is that the UAC rules are different between .NET assemblies and native DLLs. It's the only thing I can think of, I really could do with a solution to this!



Pete

2008-02-17

Test driven ECO

Here are my latest revelations :-)

01
Instead of having to mock IEcoServiceProvider and IOclPsService in order to avoid DB access simply use PersistenceMapperMemory. This way I can create the objects I want, UpdateDatabase, and then run my tests. It’s much easier to read, and more importantly less typing.

02
My page controllers no longer use an EcoSpace. Instead the code always uses a ServiceProvider property of type IEcoServiceProvider. When I want to test my controller I create an instance and set its ServiceProvider property. Now whenever the controller needs to do anything it will go through the ServiceProvider I specified.

This is beneficial for a number of reasons. Firstly it means that I can create an EcoSpace in my test and set its PersistenceMapper to PersistenceMapperMemory before activating it. Secondly I can also opt to pass a mocked IEcoServiceProvider which either returns the real service requested or returns a mocked one. An example of this is that I validate my page by using a registered IConstraintProvider interface (defined in DroopyEyes.Eco.Validation). I can check that a controller action wont save a modified object it if is invalid. Instead of having to know how to make the object invalid I just mock the IConstraintProvider and have it always return a single constraint with the expression "false" so that it always fails. In addition, because I know the name of the constraint that is broken, I can then check ViewData["Errors"] and ensure that the controller action has displayed the error messages.

Sure I can just write the action in a minute and know it works, but having these test cases ensures that if someone else modifies my project’s source code without fully understanding what they are doing I will know what they broke. Or, they will know what they broke and can fix it themself!

So there you are. Same end result, less code.

2008-02-14

ECO, should we mock it?

I watched a video on Rhino Mocks yesterday. What a great framework! Obviously I wanted to know if I could use this with ECO so I thought I'd give it a try.

In my website's AccountController there is a method like so

public void AttemptLogin(string emailAddress, string password, string redirectUrl)
{
}


Now I could just go ahead and write some OCL to find the user, but instead of doing this I really want to separate the code a bit. So I created a class

public class UserRepository
{
  private readonly IEcoServiceProvider ServiceProvider;

  public UserRepository(IEcoServiceProvider serviceProvider)
  {
    ServiceProvider = serviceProvider;
  }

  public User GetByEmailAddressAndPassword(string emailAddress, string password)
  {
    string searchEmail = BusinessClassesHelper.EscapeOcl(emailAddress);
    string criteria = string.Format("->select(emailAddress.sqlLikeCaseInsensitive('{0}'))", searchEmail);
    return BusinessClassesHelper.SelectFirstObject<User>(ServiceProvider, "User", criteria);
  }
}


Now I can get my user like so....

public void AttemptLogin(string emailAddress, string password, string redirectUrl)
{
  MyWebsiteEcoSpace ecoSpace = new MyWebsiteEcoSpace();
  ecoSpace.Active = true;
  try
  {
    UserRepository repository = new UserRepository(ecoSpace);
    MyWebsite.Model.User user = repository.GetByEmailAddressAndPassword(emailAddress, password);
  }
  finally
  {
    ecoSpace.Active = false;
  }
}


So what's the benefit? The important thing to note is that I pass in an instance of IEcoServiceProvider to the UserRepository object. So if I want to test the UserRepository class on its own I can pass a dummy object for the serviceProvider. This means that I don't have to access the DB which would slow things down (especially if I have to keep clearing the DB down), in fact I don't even need to connect to the DB at all!

If you remember UserRepository.GetByEmailAddressAndPassword() looks like this
public User GetByEmailAddressAndPassword(string emailAddress, string password)
{
  string searchEmail = BusinessClassesHelper.EscapeOcl(emailAddress);
  string criteria = string.Format("->select(emailAddress.sqlLikeCaseInsensitive('{0}'))", searchEmail);
 return BusinessClassesHelper.SelectFirstObject(ServiceProvider, "User", criteria);
}


and BusinessClassesHelper uses the IOclPsService and IOclService in combination to get to the result. Surely this is all too complicated to mock? Not with Rhino, no!

[TestFixture]
public class UserRepositoryTests
{
  MockRepository Mocks;
  IEcoServiceProvider MockServiceProvider;
  IOclPsService MockOclPsService;
  MyWebsiteEcoSpace EcoSpace;

  [SetUp]
  public void SetUp()
  {
    //Create a mock repository
    Mocks = new MockRepository();

    //Create the mock IEcoServiceProvider
    MockServiceProvider = Mocks.CreateMock<IEcoServiceProvider>();

    //I also need a mock IOclPsService to avoid DB access
    MockOclPsService = Mocks.CreateMock<IOclPsService>();

    //Create a transient version of my EcoSpace
    EcoSpace = new MyWebsiteEcoSpace ();
    EcoSpace.PersistenceMapper = null; //No persistence!
    EcoSpace.Active = true;
  }

  [TearDown]
  public void TearDown()
  {
    EcoSpace.Active = false;
    Mocks.ReplayAll(); //Just in case we forgot, calling twice has no effect!
    Mocks.VerifyAll(); //Ensure everything expected was called
  }

  [Test]
  public void GetUserByEmailAddressAndPassword()
  {
    //Create a list of users to return from the mock IOclPsService
    IObjectList userList = EcoSpace.VariableFactory.CreateTypedObjectList(typeof(User), false);
    
    //Add a single user to that list
    User expectedUser = new User(EcoSpace);
    expectedUser.EmailAddress = "me@home.com";
    expectedUser.SetPassword("1234567890");
    userList.Add(expectedUser.AsIObject());

    //Start specifying what we expect to be called, and what we should do as a result
    Mocks.Record();

    //When GetEcoService<IOclPsService> is called return our MockOclPsService
    Expect.Call(MockServiceProvider.GetEcoService<IOclPsService>()).Return(MockOclPsService);
  
    //Same for GetEcoService(typeof(IOclPsService))
    Expect.Call(MockServiceProvider.GetEcoService(typeof(IOclPsService))).Return(MockOclPsService);

    //When asked for the IOclService (not PS service) return the real one
    Expect.Call(MockServiceProvider.GetEcoService<IOclService>()).Return(EcoSpace.Ocl);
    Expect.Call(MockServiceProvider.GetEcoService(typeof(IOclService))).Return(EcoSpace.Ocl);

    //When MockOclPsService.Execute is executed return our userList
    Expect.Call(MockOclPsService.Execute(null)).Return(userList);
    //This means we don't care what the exact parameter is, any OCL will do
    LastCall.IgnoreArguments();

    //Now go into play back mode
    Mocks.ReplayAll();

    //Create the UserRepository using our mock services
    UserRepository repository = new UserRepository(MockServiceProvider);

    //Ask for the user
    User foundUser = repository.GetByEmailAddressAndPassword(expectedUser.EmailAddress, "1234567890");

    //Ensure that we got the same user back
    Assert.AreEqual(expectedUser, foundUser, "Found the wrong user");
  }
}



Nice eh :-)

2008-02-13

Unit testing MonoRail controllers

I spent yesterday finishing off (mostly) my business model, then the end of yesterday + today writing test cases for those classes. Everything was going great, I found at least 3 errors in my code that I hadn’t realised was there and also realised there were a few more things I needed.

Then it was time to start testing the controllers in my MonoRail site. What a disaster!

Attempt 1:
[Test]
public void AdminOnly_Home()
{
  AdminController controller = new AdminController();
  controller.Home();
  Assert.IsTrue(Controller.Response.WasRedirected, "Should have been redirected");
}


The problem with this was pretty obvious, Controller doesn’t have a Response etc set up. So along came attempt 2:

[Test]
public void AdminOnly_Home()
{
  AdminController controller = new AdminController();
  PrepareController(controller);
  controller.Home();
  Assert.IsTrue(Controller.Response.WasRedirected, "Should have been redirected");
}


Now the controller is set up with mock objects and will run! Unfortunately the BeforeAction filter on my action was not being executed. Aha! Pretty obvious problem! If I call the method directly how can the framework possible find all of the reflection attributes and process them etc? *slaps head*

Attempt 3
[Test]
public void AdminOnly_Home()
{
  AdminController controller = new AdminController();
  PrepareController(controller, "Admin", "Home");
  controller.Process(Controller.Context, Controller.ControllerContext);
  Assert.IsTrue(Controller.Response.WasRedirected, "Should have been redirected");
}


Still no joy! The filters just aren’t being executed. Someone on the user groups said that this is expected behaviour and that the filter should be tested in isolation. Whereas I agree for the most part unfortunately it doesn’t apply in this case. My ECO extensions to MonoRail allow the developer to specify pooling, session, default EcoSpace types, and so on. If these reflection attributes aren’t processed then the action just isn’t going to act in the same way it will at runtime!

At the moment I am sorely disappointed! I was really looking forward to writing a test driven website but unless this guy was wrong it doesn’t look like it is going to be possible!

It’s at times like these I wonder how difficult it really is to write your own MVC framework? Maybe I will take another look at the MS offering. If I had enough free time I'd make my own :-)