2008-03-30

Postal codes within a radius

My hobby MVC website allows people to place adverts. When searching for adverts I would like the user to be able to specify a UK postal code and radius to filter the adverts down to ones within travelling distance. The trick to this was to record a list of UK postal codes and their latitude/longitude.

The first step is to write a routine which will give a straight line distance between to coordinates:

public static class MathExtender
{
  public static double GetDistanceBetweenPoints(double sourceLatitude, double sourceLongitude, double destLatitude, double destLongitude)
  {
    double theta = sourceLongitude - destLongitude;
    double distance =
      Math.Sin(DegToRad(sourceLatitude))
      * Math.Sin(DegToRad(destLatitude))
      + Math.Cos(DegToRad(sourceLatitude))
      * Math.Cos(DegToRad(destLatitude))
      * Math.Cos(DegToRad(theta));
    distance = Math.Acos(distance);
    distance = RadToDeg(distance);
    distance = distance * 60 * 1.1515;
    return (distance);
  }

  public static double DegToRad(double degrees)
  {
    return (degrees * Math.PI / 180.0);
  }

  public static double RadToDeg(double radians)
  {
    return (radians / Math.PI * 180.0);
  }
}


Additionally, on my PostalCode class I have a helper method like so:

public double GetDistanceTo(PostalCode destination)
{
  return MathExtender.GetDistanceBetweenPoints(Latitude, Longitude, destination.Latitude, destination.Longitude);
}


Now the next problem is that these routines are not selectable via SQL and therefore to find all postal codes within a radius of another I would have to load all postal codes into memory and then evaluate them, which I don’t want to do! The DB might be able to use SIN/COS/ACOS etc but if it does then it would be DB specific and I don’t want that either. My decision was to first look for postal codes within a square area, the square being just big enough to encompass the circle, and then to use the in-memory PostalCode.GetDistanceTo() method to whittle the postcodes down to the exact list of matching objects.
The problem remained that I would still need to establish whether Longitude/Latitude coordinates will be within this square. To overcome this I decided that if the world were divided up into a grid of squares each 1 mile in size I could then easily select postal codes within any size square by checking which 1 mile square it is within. So I decided to establish Longitude=0 Latitude=0 as 0,0 on my grid and work out every postal code’s distance from that point. I added two persistent "Double" properties to my postal code class named GridReferenceX and GridReferenceY which I set whenever the Longitude or Latitude is set:

public double GridReferenceX { get; private set; }
public double GridReferenceY { get; private set; }

public double Longitude
{
  get { ..... };
  set
  {
    ......;
    CalculateGridReference();
  }
}

public double Latitude
{
  get { ..... };
  set
  {
    ......;
    CalculateGridReference();
  }
}

private void CalculateGridReference()
{
  //Latitude distance only (X)
  GridReferenceX = MathExtender.GetDistanceBetweenPoints(0, 0, Latitude, 0);
  //Longitude distance only (Y)
  GridReferenceY = MathExtender.GetDistanceBetweenPoints(0, 0, 0, Longitude);
}


Now I have a grid reference for each postal code which really means nothing, but the advantage is that they can be compared to each other like so...

public IList<PostalCode> GetAllWithinRadius(PostalCode postalCode, double radiusInMiles)
{
  List<PostalCode> result = new List<PostalCode>();
  string criteria = string.Format("->select(gridReferenceX >= {0} and gridReferenceX <= {1} and gridReferenceY >= {2} and gridReferenceY <= {3})",
    postalCode.GridReferenceX - radiusInMiles,
    postalCode.GridReferenceX + radiusInMiles,
    postalCode.GridReferenceY - radiusInMiles,
    postalCode.GridReferenceY + radiusInMiles);

  IList<PostalCode> allPostalCodes = BusinessClassesHelper.SelectObjects<PostalCode>(ServiceProvider, "PostalCode", criteria);
  return
    (
      from selectedPostalCode in allPostalCodes
      where postalCode.GetDistanceTo(selectedPostalCode) <= radiusInMiles
      select selectedPostalCode
    ).ToList<PostalCode>();
}



This first uses SQL to exclude all postal codes which cannot possibly be within the radius, then a simple LINQ query returns only the postal codes that are actually within the radius of the specific postal code. Another benefit is that this will also work for postal codes anywhere in the world, I could quite easily check how far (in a straight line) it is from my UK postal code to a US zip code!

2008-03-13

Today's nuggets

Today presented two slightly odd problems, so here they are in case they help someone else out in the future.



Problem 1: <form action="/Account/AttemptSignIn"> going to wrong controller action.



First the user visits MySite.com/Account/SignIn - the view that is rendered has two forms in it



<form action="/Account/AttemptSignIn" method="post">

...

</form>

<form action="/Account/ForgottenPassword" method="post">

...

</form>



The problem was that although clicking the "ForgottenPassword" submit button would correctly invoke AccountController.ForgottenPassword, when clicking the submit button for "AttemptSignIn" AccountController.SignIn would be invoked again instead of AccountController.AttemptSignIn.



At some point I must have edited the site’s master page in Design mode. Something I did must have made VS think "Ah, Pete needs a form with runat=server to do this!". As a result VS added



<form name="form1" runat="server"> to my master page. Obviously it thought I wanted a standard ASP .NET setup which always posts back to the same URL. Removing the <form> in the master page fixed the problem.





Problem 2: Exception not being caught



The following code



if (!AuthorDataSource.IsValidNewValues(e.Keys, e.NewValues))

e.Cancel = true;



would show a FormatException in the HTML if I entered "abc" when a decimal value was expected. Obviously this is expected behaviour is expected so the code was changed like so



try

{

...

}

catch (FormatException)

{

//Show the error

}



but still it didn’t catch the exception! After investigating for a while I observed the following



A: The following code would throw a FormatException



value = descriptor.Converter.ConvertFrom(value);



B: Immediately afterwards it would then throw a System.Exception (who chose to use that type?)



C: The new System.Exception would have the FormatException as its InnerException



D: When ASP .NET came to show the exception it would show only the InnerException (the FormatException) and not the actual exception encountered (System.Exception).



So it appeared that FormatException was not caught, but also that "catch (FormatException)" could not catch it.





Some things are just plain silly :-)

2008-03-12

Small sample website

I'm playing with a little website called "Not A Sausage". I will reveal what it is for later :-)

I've decided to make it available for download for those of you who might want to take a look. It uses the MS Web Extensions so you will need to install those first from here. You will also need Rhino Mocks from here. Ofcourse you will also need ECO!

When you download the project (VS 2008, C# 3.5) you will need to add the following references to the NotASausageWebsite project
  • System.Web.Abstractions
  • System.Web.Mvc
  • System.Web.Routing
and the following to the Tests project
  • System.Web.Abstractions
  • System.Web.Mvc
  • System.Web.Routing
  • Rhino.Mocks
The System.Web.* files will be in {Program Files}\Microsoft ASP.NET MVC Preview 2\Assemblies

All it does so far is
  • Allow you to create an account.
  • Show errors based on modeled constraints if the user input is invalid.
  • Checks for duplicate email addresses by using an InPS evaluation so that not all users are loaded into memory.
  • Sends an email to the user with a temporary login URL (to validate their email address).
In addition to this the site (so far) is fully unit tested. The tests were written before the actions were implemented. It uses mock objects (Rhino Mocks) to mock the whole HTTP request, and even mocks out other services so that I can check an email is sent to the user without actually having to send the email.

Anyway, take a look and see what you think. You can download the project here.

2008-03-10

Test Driven MVC and ECO

I have decided that mocking IEcoServiceProvider is not the way to go. Your controller will use the mocked provider during testing but


     
  1. You don’t want to have to mock every service the provider may return, it’s a lot of work!

  2.  
  3. You don’t want your controller using a mocked service, and then the EcoSpace using the real one!



At first I was mocking every possible service request. IUndoService, IPersistenceService, IOclService, etc. I get bored typing them out in this blog, so doing it in tests was really annoying me. I decided I would instead only mock the service in question. So if I were ensuring that an action wont save an object with broken constraints I would mock GetEcoService<IConstraintProvider> and ensure that I always got a broken constraint.

The problem was that the test to ensure I can save a valid object would then invoke the IPersistenceService.UpdateDatabaseWithList method. In my EcoSpace I have decorated my persistence service so that it checks every object in the update list to ensure it has no invalid constraints. At the point it asks for IConstraintProvider it is using the real IEcoServiceProvider and as a result it gets the real IConstraintProvider. In short the object would only save if it were really valid, and not if my mocked constraint provider pretended it was.

Ultimately I found it much easier just to register the mock service on the EcoSpace. To do this all I had to do was to expose a public method on the EcoSpace like so

#if DEBUG
 public void RegisterMockService(type serviceType, object serviceInstance)
 {
  RegisterEcoService(serviceType, serviceInstance);
 }
#endif


Now I can replace any service on the EcoSpace, so even the EcoSpace itself will get mocked services during testing. This is a lot less work!


Talking of a lot less work, although I was initially annoyed that the new field test for the ASP .NET web extensions had switched from using interfaces back over to using objects it turns out that testing is actually much easier than it was previously. Scott Hanselman posted some testing classes on his blog recently, it came out all screwy so I will repost the corrected version below (I hope he doesn’t mind). Testing is now as easy as this...

[TestClass]
public class AccountTests
{
 //This is a descendant of my real EcoSpace,
 //but I replace the persistence mapper with
 //PersistenceMapperMemory after construction so that
 //no DB access is needed.
 MemoryEcoSpace EcoSpace;
 AccountController Controller;
 MockRepository Mocks;
 FakeViewEngine FakeViewEngine;

 [TestInitialize]
 public void SetUp()
 {
  EcoSpace = new MemoryEcoSpace();
  EcoSpace.Active = true;

  Mocks = new MockRepository();
  FakeViewEngine = new FakeViewEngine();
  Controller = new AccountController();
  Controller.ViewEngine = FakeViewEngine;
  using (Mocks.Record())
  {
   Mocks.SetFakeControllerContext(Controller);
  }
 }

 [TestMethod]
 public void Create()
 {
  using (Mocks.Playback())
  {
   Controller.Create();
   Assert.AreEqual("Create", FakeViewEngine.ViewContext.ViewName);
  }
 }

 [TestMethod]
 public void CreateUpdate_ConfirmationEmailDoesNotMatch()
 {
  using (Mocks.Playback())
  {
   Controller.CreateUpdate("Mr", "Peter", "Morris", "me@home.com", "");
   Assert.IsTrue(ValidationHelper.ErrorExists(Controller.ViewData, NotASausageWebsite.Constants.ErrorMessages.ConfirmationEmailAddressDoesNotMatchEmailAddress));
  }
 }
}


public static class ValidationHelper
{
 public static bool ErrorExists(IDictionary<string, object> viewData, string errorMessage)
 {
  if (!viewData.ContainsKey(NotASausageWebsite.Constants.ViewDataKeys.Global.ErrorMessages))
   return false;
  List<string> errors = (List<string>)viewData[NotASausageWebsite.Constants.ViewDataKeys.Global.ErrorMessages];
  return errors.IndexOf(errorMessage) >= 0;
 }
}



Nice and easy! If you want the testing code from Scott here it is:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Web;
using Rhino.Mocks;
using System.Web.Mvc;
using System.Web.Routing;
using System.Collections.Specialized;

namespace Tests.Helpers
{
public static class MvcMockHelpers
{
public static HttpContextBase FakeHttpContext(this MockRepository mocks)
{
HttpContextBase context = mocks.PartialMock<HttpContextBase>();
HttpRequestBase request = mocks.PartialMock<HttpRequestBase>();
HttpResponseBase response = mocks.PartialMock<HttpResponseBase>();
HttpSessionStateBase session = mocks.PartialMock<HttpSessionStateBase>();
HttpServerUtilityBase server = mocks.PartialMock<HttpServerUtilityBase>();

SetupResult.For(context.Request).Return(request);
SetupResult.For(context.Response).Return(response);
SetupResult.For(context.Session).Return(session);
SetupResult.For(context.Server).Return(server);

mocks.Replay(context);
return context;
}

public static HttpContextBase FakeHttpContext(this MockRepository mocks, string url)
{
HttpContextBase context = FakeHttpContext(mocks);
context.Request.SetupRequestUrl(url);
return context;
}

public static void SetFakeControllerContext(this MockRepository mocks, Controller controller)
{
var httpContext = mocks.FakeHttpContext();
ControllerContext context = new ControllerContext(new RequestContext(httpContext, new RouteData()), controller);
controller.ControllerContext = context;
}

static string GetUrlFileName(string url)
{
if (url.Contains("?"))
return url.Substring(0, url.IndexOf("?"));
else
return url;
}

static NameValueCollection GetQueryStringParameters(string url)
{
if (url.Contains("?"))
{
NameValueCollection parameters = new NameValueCollection();

string[] parts = url.Split("?".ToCharArray());
string[] keys = parts[1].Split("&amp;".ToCharArray());

foreach (string key in keys)
{
string[] part = key.Split("=".ToCharArray());
parameters.Add(part[0], part[1]);
}

return parameters;
}
else
{
return null;
}
}

public static void SetHttpMethodResult(this HttpRequestBase request, string httpMethod)
{
SetupResult.For(request.HttpMethod).Return(httpMethod);
}

public static void SetupRequestUrl(this HttpRequestBase request, string url)
{
if (url == null)
throw new ArgumentNullException("url");

if (!url.StartsWith("~/"))
throw new ArgumentException("Sorry, we expect a virtual url starting with \"~/\".");

SetupResult.For(request.QueryString).Return(GetQueryStringParameters(url));
SetupResult.For(request.AppRelativeCurrentExecutionFilePath).Return(GetUrlFileName(url));
SetupResult.For(request.PathInfo).Return(string.Empty);
}

}
}



 public class FakeViewEngine : IViewEngine
 {
  public void RenderView(ViewContext viewContext)
  {
   ViewContext = viewContext;
  }

  public ViewContext ViewContext { get; private set; }

 }

2008-03-07

PaySpam

Spam really annoys me, and spam filters are not 100% accurate. There's only one sure way to ensure you get all your emails and that is to accept spam. I remember a suggestion (by Microsoft?) to make people pay for emails, and everyone hated it. I think that idea was wrong too, but not 100% wrong.

Here's a copy of an idea I just sent to Google.

Get rid of spam, and get paid to do it!

01: I sign up to your new "payspam" email service (it's free).

02: I add a list of people I want to be able to send me emails.

03: Someone on my list sends me an email, it comes straight through.

If someone I don't know sends me an email....

01: Their email client using a new email protocol has to send a user/pass for their google account. If they don't do this then their email will not be accepted by the server.

02: $1 is transferred from their account into a holding area.

03: I now get the email, but at the top it has 3 links

A: Not spam
B: Add to white list
C: Spam

"Not spam" will cancel the money transfer from the sender's account.

"Add to white list" will do "A" and also white list the person.

"Spam" will transfer X% of the $1 to my account.

If I get spam now I will get paid to receive it. Beneficially most spammers wont bother sending spam to "payspam" email addresses.

It is important that the server accepts/rejects the email immediately. Sending a confirmation email to a legitimate sender is likely to get caught in *their* spam filter.

ECO, LINQ, Anonymous types, and Web Extensions

I’ve been finding LINQ + Anonymous types really compliment ECO and the new ASP web extensions approach to writing websites. I may have mentioned recently that I don’t like the idea of passing instances of my business objects to the presentation layer. The reason is that someone else will be writing the views for this site and I want to be able to control what they are capable of displaying. It’s not just that though, the fact is that your view might need to look completely different to how your business classes are structured, one layer should not dictate the structure of another.

The example I am about to show does in fact have similar structures for the view and model. Having said that there is a slight difference in that the MinorVersion class has its own "int VersionNumber" property, and gets the major part of the version number from self.MajorVersion.VersionNumber. Anyway, now to get on with it.

My requirement was to show all major versions, within each major version show each minor version, and within each minor version show a list of what’s new. In addition, a minor version should only be displayed if its status is #Released, and a major version should not be displayed if it has no minor versions which meet this criteria.


The following code generates a structure like so

MajorVersion (VersionNumber)
 1..* MinorVersion (MajorVersionNumber, MinorVersionNumber)
  1..* WhatsNew (ID, Headline)



and stores the resulting anonymous type into the ViewData for my view to render.

ViewData[GlobalViewDataKeys.WhatsNewKeys.WhatsNewList] = 
 from majorVersion in software.MajorVersions
 where
  (from minorVersionCheck in majorVersion.MinorVersions
   where minorVersionCheck.Status == MinorVersionStatus.Released select minorVersionCheck ).Count() > 0
 select
  new
  {
   VersionNumber = majorVersion.VersionNumber,
   MinorVersions =
    from minorVersion in majorVersion.MinorVersions
    select
     new
     {
      MajorVersionNumber = majorVersion.VersionNumber,
      VersionNumber = minorVersion.VersionNumber,
      WhatsNew =
       from whatsNew in minorVersion.WhatsNew
       select
        new
        {
         ID = whatsNew.ID,
         Headline = whatsNew.Headline
        }
     }
  };
RenderView("AllHistory");




The code behind of my view reads like this:

protected void Page_Load(object sender, EventArgs e)
{
 MajorVersionRepeater.DataSource = ViewData[GlobalViewDataKeys.WhatsNewKeys.WhatsNewList];
 MajorVersionRepeater.DataBind();
}



And finally I use nested ASP:Repeater tags to render the nested HTML.


<ul class="AllHistoryMajorVersionList">
 <asp:Repeater id="MajorVersionRepeater" runat="server">
  <ItemTemplate>
   <li>
    Major version
     <%# DataBinder.Eval(Container.DataItem, "VersionNumber") %>
    <ul class="AllHistoryMinorVersionList">
     <asp:Repeater
      id="MinorVersionRepeater"
      DataSource=’<%# DataBinder.Eval(Container.DataItem, "MinorVersions") %>’
      runat="server">
      <ItemTemplate>
       <li>
        Minor version
        <%# DataBinder.Eval(Container.DataItem, "MajorVersionNumber") %>.
        <%# DataBinder.Eval(Container.DataItem, "VersionNumber") %>
        <ul class="AllWhatsNewList">
         <asp:Repeater
          id="WhatsNewRepeater"
          DataSource=’<%# DataBinder.Eval(Container.DataItem, "WhatsNew") %>’
          runat="server">
          <ItemTemplate>
           <li>
            <a href="/WhatsNew/View/<%# DataBinder.Eval(Container.DataItem, "ID") %>">
             <%# DataBinder.Eval(Container.DataItem, "Headline") %>
            </a>
           </li>
          </ItemTemplate>
         </asp:Repeater>
        </ul>
       </li>
      </ItemTemplate>
     </asp:Repeater>
    </ul>
   </li>
  </ItemTemplate>
 </asp:Repeater>
</ul>



I think the point is that there is just no need to pass your business class instances through to the UI layer. In fact if you later changed the structure of your business classes this LINQ would no longer compile, whereas the markup in the view is evaluated at runtime so you wouldn’t spot an error here until you tried to view the page.