C# scripting in .NET

Based on a couple of days of experimenting it seems that I now have C# scripting working in my application to a level that I am satisfied.  Because I couldn’t have achieved this without looking at other people’s examples I thought it was only fair that I share what I now have.  Firstly I want to say that my purpose was to give the user the ability to write procedural code rather than object code.  Although it is still possible for the user to write OOP source code my requirement was to let the user define a function which returned a specific return type.

public interface ICompiledFunction<T>
{
  T Execute(Dictionary<string, object> variables);
}

A typical script might look something like this

public decimal string Main()
{
  return "Bob Monkhouse";
}

To create an instance of the ICompiledFunction<T> I use ICompilerService, which is defined like so

public interface ICompilerService
{
  bool Compile<T>(
    string[] scripts,
    Dictionary<string, Type> variableDefinitions,
    IEnumerable<Assembly> referencedAssemblies,
    out ICompiledFunction<T> function,
    out string sourceCode,
    out IEnumerable<string> compilerErrors);
}

1. Scripts: An array of methods which should be combined to make the full script.  Each may include its own “using” statements.



2. VariableDefinitions: A dictionary of variable names with their types.  These will appear to be global variables available to the combined scripts, but in fact they will be properties defined as part of the wrapper class which is generated automatically.



3. ReferencedAssemblies: A collection of Assembly, used to ensure that the script has access to any types declared.



4. Function: The compiled function



5. SourceCode: The complete source code which is compiled after it has been merged; useful for working out what is wrong with your source code.



6. CompilerErrors: A collection of compiler errors.

The importance of CompilerService is that the implementation will create a new AppDomain each time you call Compile.  When the CompilerService is dispose it will unload all assemblies.  I will post the code with comments inline

using System;
using System.CodeDom.Compiler;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.Reflection;

namespace PeterMorris.Scripting
{
  public class CompilerService : ICompilerService, IDisposable
  {
    Stack<AppDomain> SandboxAppDomains = new Stack<AppDomain>();
    bool Disposed;

    public bool Compile<T>(
      string[] scripts,
      Dictionary<string, Type> variableDefinitions,
      IEnumerable<Assembly> referencedAssemblies,
      out ICompiledFunction<T> function,
      out string sourceCode,
      out IEnumerable<string> compilerErrors)
    {
      if (scripts == null)
        throw new ArgumentNullException("Scripts");

      //Define app domain settings such as the code base
      var appDomainSetup = new AppDomainSetup();
      string appBase = typeof(CompilerService).Assembly.CodeBase;
      appBase = new Uri(appBase).LocalPath;
      appDomainSetup.ApplicationBase = Path.GetDirectoryName(appBase);
      var evidence = AppDomain.CurrentDomain.Evidence;

      //Create a uniquely named app domain and push it onto a stack for disposing later
      string sandboxAppDomainName = "Sandbox" + Guid.NewGuid().ToString().Replace("-", "");
      var sandboxAppDomain = AppDomain.CreateDomain(sandboxAppDomainName, evidence, appDomainSetup);
      SandboxAppDomains.Push(sandboxAppDomain);

      //Clone the original scripts, we don't want to alter the originals
      scripts = scripts.ToArray();

      //Strip out the using clauses from each script so that they can be combined
      var usingClauses = StripOutUsingClauses(scripts).OrderBy(x => x).ToList();

      //Build the complete source code to be compiled
      sourceCode = BuildSourceCode(usingClauses, scripts, variableDefinitions);

      CompiledFunction<T> result;
      try
      {
        //Create an instance of CompiledFunction<T> in the new app domain
        result = (CompiledFunction<T>)sandboxAppDomain.CreateInstanceAndUnwrap(
          assemblyName: typeof(CompiledFunction<T>).Assembly.GetName().FullName,
          typeName: typeof(CompiledFunction<T>).FullName);
      }
      catch (Exception unexpectedException)
      {
        function = null;
        compilerErrors = new string[] { unexpectedException.Message };
        return false;
      }

      //Compile the function
      //Variable definitions and source code are passed so that we can translate line numbers
      //in the error message back to the line number of the snippet of source the user
      //has typed in
      function = (ICompiledFunction<T>)result;
      return result.Compile(
        sourceCode: sourceCode,
        variableDefinitions: variableDefinitions,
        referencedAssemblies: referencedAssemblies,
        compilerErrors: out compilerErrors);
    }

    public static HashSet<string> StripOutUsingClauses(string[] scripts)
    {
      //Make a hashset of using clauses, to ensure we have no duplicates
      var usingClauses = new HashSet<string>();
      //Ensure that the standard using clauses are present
      usingClauses.Add("System");
      usingClauses.Add("System.Linq");
      usingClauses.Add("System.Collections.Generic");

      //Create a regex which matches a using statement
      var regex = new Regex(@"^[\s]*using[\s]+(\S*?)[\s]*;[\s]*$", RegexOptions.IgnoreCase);

      //Loop through each script
      for (int scriptIndex = 0; scriptIndex < scripts.Length; scriptIndex++)
      {
        string script = scripts[scriptIndex];
        if (string.IsNullOrEmpty(script))
          continue;

        var scriptReader = new StringReader(script);
        var scriptBuilder = new StringBuilder();
        while (true)
        {
          //Read the next line of the current script, null == end of script
          string line = scriptReader.ReadLine();
          if (line == null)
            break;

          //Ignore blank lines
          if (line.Trim() == "")
            continue;

          var match = regex.Match(line);
          if (match.Captures.Count > 0)
          {
            //If there is a match then add the name to the using clause hash set
            string nameSpace = match.Groups[1].Value;
            usingClauses.Add(nameSpace);
          }
          else
          {
            //If no match then the using clauses are finished, add the rest
            //of the text from the reader into the writer
            scriptBuilder.AppendLine(line);
            scriptBuilder.Append(scriptReader.ReadToEnd());
            break;
          }
        }//Loop through each line of script

        //Replace the script with text without the using clauses
        scripts[scriptIndex] = scriptBuilder.ToString();
      }//foreach script
      return usingClauses;
    }

    //Builds the full source code with wrapping class
    string BuildSourceCode(List<string> usingClauses, string[] scripts, IEnumerable<KeyValuePair<string, Type>> variableDefinitions)
    {
      if (usingClauses == null)
        throw new ArgumentNullException("usingClauses");

      var scriptBuilder = new StringBuilder();
      //Add using clauses
      foreach (string nameSpace in usingClauses.OrderBy(x => x))
        scriptBuilder.AppendFormat("using {0};\r\n", nameSpace);
      if (usingClauses.Any())
        scriptBuilder.AppendLine();

      //namespace
      scriptBuilder.AppendLine("namespace Sandbox");
      scriptBuilder.AppendLine("{");
      {
        //Wrapper class
        scriptBuilder.AppendLine("public class ScriptHolder");
        scriptBuilder.AppendLine("{");
        {
          //Add variable definitions as properties within the class
          BuildProperties(scriptBuilder, variableDefinitions);

          //Combine all scripts as a single string
          foreach (string script in scripts)
          {
            scriptBuilder.AppendLine(script);
            scriptBuilder.AppendLine();
          }
        }
        scriptBuilder.AppendLine("}"); //class
      }
      scriptBuilder.AppendLine("}"); //namespace
      return scriptBuilder.ToString();
    }

    //Add variable definitions as properties within the script
    void BuildProperties(StringBuilder scriptBuilder, IEnumerable<KeyValuePair<string, Type>> variableDefinitions)
    {
      if (variableDefinitions == null)
        return;

      foreach (var kvp in variableDefinitions.OrderBy(x => x.Key))
      {
        //ToScriptableString is required to expand generic types back into
        //proper looking source code
        string scriptableTypeName = kvp.Value.ToScriptableString();
        string variableName = kvp.Key;
        scriptBuilder.AppendFormat("\tpublic {0} {1};\r\n", scriptableTypeName, variableName);
      }
      if (variableDefinitions.Any())
        scriptBuilder.AppendLine();
    }

    void Dispose(bool isDisposing)
    {
      if (Disposed)
        return;
      Disposed = true;
      if (isDisposing)
        //Unload all sandbox app domains,
        //in reverse order - just because I have OCD :-)
        while (SandboxAppDomains.Count > 0)
          AppDomain.Unload(SandboxAppDomains.Pop());
    }

    public void Dispose()
    {
      Dispose(true);
    }
  }
}

One thing to note here is the use of Type.ToScriptableString() – this is implemented via a helper class.  This is because when you use GetType().FullName on a generic type you get a mangled name which would not compile in a script, so it must be unmangled.

using System;
using System.Text;

namespace PeterMorris.Scripting
{
  public static class TypeHelper
  {
    public static string ToScriptableString(this Type type)
    {
      if (type == null)
        return "Void";

      //If not a generic type then just return the full name
      if (!type.IsGenericType)
        return type.FullName;

      //Convert a generic type back to a compilable representation
      var builder = new StringBuilder();
      Type genericType = type.GetGenericTypeDefinition();
      //Unmangle the name
      builder.Append(genericType.Name.Remove(genericType.Name.IndexOf("`")) + "<");
      string separator = "";
      //Include the inner types of the generic type
      foreach (var genericArgument in type.GetGenericArguments())
      {
        //Recursively call ToScriptableString for the typename, as the inner type
        //itself may also be a generic type
        builder.AppendFormat("{0}{1}", separator, genericArgument.ToScriptableString());
        separator = ",";
      }
      builder.Append(">");
      return builder.ToString();
    }
  }
}

And now the implementing code for CompiledFunction<T>

using System;
using System.CodeDom.Compiler;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using Microsoft.CSharp;

namespace PeterMorris.Scripting
{
  public class CompiledFunction<T> : MarshalByRefObject, ICompiledFunction<T>
  {
    bool IsCompiled;
    Dictionary<string, Type> VariableDefinitions;
    Dictionary<string, FieldInfo> FieldReferences;
    object Instance;
    MethodInfo Method;

    public CompiledFunction()
    {
      VariableDefinitions = new Dictionary<string, Type>();
      FieldReferences = new Dictionary<string, FieldInfo>();
    }

    public bool Compile(
      string sourceCode,
      Dictionary<string, Type> variableDefinitions,
      IEnumerable<Assembly> referencedAssemblies,
      out IEnumerable<string> compilerErrors
      )
    {
      if (IsCompiled)
        throw new InvalidOperationException("Already compiled");
      IsCompiled = true;

      if (variableDefinitions != null)
        this.VariableDefinitions = variableDefinitions;

      //Get the full path to the executing binary, this is used to find assembly
      //references which are not in the GAC.  Use Uri to convert to a local path.
      string binaryPath = new Uri(AppDomain.CurrentDomain.BaseDirectory).LocalPath;
      var fullReferencedAssemblyList = new List<Assembly>(referencedAssemblies);
      //System.dll
      fullReferencedAssemblyList.Add(typeof(string).Assembly);
      //System.Core.dll
      fullReferencedAssemblyList.Add(typeof(IQueryable).Assembly);
      //Get assembly names
      string[] referencedAssemblyNames =
          fullReferencedAssemblyList
          .Select(x => x.GetName().Name + ".dll")
          .Select
          (
            //If the file exists in the binary folder then add it by full path,
            //otherwise add it without a path so that it is assumed to be in the GAC
            assemblyName => 
              File.Exists(Path.Combine(binaryPath, assemblyName)) ?
              Path.Combine(binaryPath, assemblyName) :
              assemblyName
          )
          .Distinct()
          .ToArray();

      //Set the compiler parameters to compile in memory
      var compilerparameters = new CompilerParameters
      {
        GenerateExecutable = false,
        GenerateInMemory = true,
        IncludeDebugInformation = true,
        TreatWarningsAsErrors = true
      };
      //Add the list of referenced assemblies
      compilerparameters.ReferencedAssemblies.AddRange(referencedAssemblyNames);

      //Use C# 3.5 so that we have access to nice features such as LINQ
      var options = new Dictionary<string, string>
      {
        { "CompilerVersion", "v3.5" }
      };

      //Create the compiler
      var compiler = new CSharpCodeProvider(options);
      //Compile the source code
      CompilerResults compilerResults;
      try
      {
        compilerResults = compiler.CompileAssemblyFromSource(
          options: compilerparameters,
          sources: new string[] { sourceCode });
      }
      catch (Exception unexpectedException)
      {
        compilerErrors = new string[] { unexpectedException.Message };
        return false;
      }

      var errors = new List<string>();
      if (compilerResults.Errors.Count == 0)
      {
        //Create an instance of the new class
        Instance = compilerResults.CompiledAssembly.CreateInstance("Sandbox.ScriptHolder");
        //Find the Main() method
        Method = Instance.GetType().GetMethod("Main");
        //Report an error if it is not decalred
        if (Method == null)
          errors.Add(string.Format("public {0} Main() has not been defined", typeof(T).Name));
        else if (!typeof(T).IsAssignableFrom(Method.ReturnType))
          //If the Main() method's return type is not assignable to the expected return type
          //then report an error
          errors.Add(string.Format("Expected return type {0} but found {1}", 
            typeof(T).FullName, Method.ReturnType.FullName));
      }
      //Add any compiler errors.  FormatError uses the source code and variable definitions
      //in order to work out the line number relative to the script entered by the user
      //rather than the position in the composite source code
      if (compilerResults.Errors != null)
        compilerResults.Errors.Cast<CompilerError>().ToList()
          .ForEach(x => errors.Add(x.FormatError(sourceCode, variableDefinitions)));

      //Create field references (i.e. global variables)
      if (!errors.Any())
        CreateFieldReferences();
      compilerErrors = errors.ToArray();
      return !errors.Any();
    }

    //Execute the compiled function
    public T Execute(Dictionary<string, object> variables)
    {
      if (!IsCompiled)
        throw new InvalidOperationException("Function has not been compiled");

      //If we have any variable definitions, set the field references to their default values
      if (VariableDefinitions != null)
        foreach (var kvp in VariableDefinitions)
        {
          string fieldName = kvp.Key;
          Type fieldType = kvp.Value;
          object defaultValue = fieldType.IsValueType ? Activator.CreateInstance(fieldType) : null;
          SetVariable(fieldName, defaultValue);
        }

      //Set the variables passed by the caller
      if (variables != null)
        foreach (var kvp in variables)
          SetVariable(kvp.Key, kvp.Value);
      //Invoke the compiled scripted method
      return (T)Method.Invoke(Instance, null);
    }

    //Populate the FieldReferences list so that it can be built into the composite source code
    void CreateFieldReferences()
    {
      if (VariableDefinitions == null)
        return;

      foreach (var kvp in VariableDefinitions)
      {
        string fieldName = kvp.Key;
        Type fieldType = kvp.Value;
        FieldInfo fieldInfo = Instance.GetType().GetField(fieldName);
        FieldReferences.Add(fieldName, fieldInfo);
      }
    }

    //Set a field (global variable) value
    void SetVariable(string variableName, object value)
    {
      FieldInfo fieldInfo;
      if (!FieldReferences.TryGetValue(variableName, out fieldInfo))
        throw new ArgumentException(string.Format("Variable {0} has not been defined", variableName));
      fieldInfo.SetValue(Instance, value);
    }
  }
}

And finally the helper class to reverse the line number from the composite script back to the individual script.

using (var compilerService = new CompilerService())
{
  string script = @"
    public string Main()
    {
      return ""Hello "" + Name;
    }";

  var scripts = new string[] { script };
  var variableDefinitions = new Dictionary<string, Type>
  {
    { "Name" typeof(string) }
  };

  string fullSourceCode;
  IFunction<string> function;
  IEnumerable<string> compilerErrors;

  if (!compilerService.Compile(
    scripts: scripts,
    variableDefinitions: variableDefinitions,
    referencedAssemblies : null,
    function: out function,
    sourceCode: out fullSourceCode,
    compilerErrors: out compilerErrors))
  {
    compilerErrors.ToList()
      .ForEach(x => Console.WriteLine(x));
    return;
  }

  //Now execute it as many times as you like
  var variableValues = new Dictionary<string, object>
  {
    { "Name", "Bob Monkhouse" }
  };
  Console.WriteLine(function(variableValues));

  variableValues["Name"] = "Peter Poppov";
  Console.WriteLine(function(variableValues));
}//Calls CompilerService.Dispose, which unloads all temporary app domains