2017-07-08

Redux sub-reducer pattern for complex / nested data structures

I love the idea of the Redux pattern, the way all reducers are pure and predictable. I love the way they are effectively listeners that act on event notifications received from a central dispatcher, and I really like the way the whole approach simplifies the application and makes code in my Angular app's components only concerned with view related concerns rather than business logic. However, what I am not a big fan of is the normalised data approach. I've seen it recommended so many times for use with the Redux approach. You essentially have any object (read "row from table on server") in memory once and then everywhere else you hold a reference to the locally stored record/object by its unique ID. So, a view state you receive that looks like this

[
  { "id": "Emp1024", "name": "Peter Morris", "address": { "id": "123", "street": "xxx" } },
  { "id": "Emp4096", "name": "Bob Smith", "address": { "id": "254", "street": "yyy" } }
]

Might be flattened out to something like this

{
  "people": {
    "Emp1024": { "name": "Peter Morris", "address": "123" },
    "Emp4096": { "name": "Bob Smith", "address": "254" },
    "_ids": [ "Emp1024", "Emp4096" ]
  },
  "addresses": [
    "123": { "street": "xxx" },
    "254": { "street": "yyy" }
  ]
}

What I do like about this is approach

  1. When your reducer has to locate an object to update it, it can find it immediately without having to iterate over the whole array.
  2. Only objects (rows) that have their state affected by the dispatched notification will be recreated, the rest will be copied as-is into the new array of objects 

What I don't like about this is

  1. API servers don't usually return data in a normalised format, so the client has to normalise data from the server (especially if you don't own the server code and cannot change it).
  2. Because the keys of an object can be stored in an indeterminate order we have to maintain an additional property containing an array of the Ids so we know in which order to display them.
  3. A local copy of the remote data is cached locally by the client and isn't unloaded.
My preferred approach is to have state per task, like so

  {
    "task1": { .............. },
    "task2": { .............. },
    etc
  }

The state that each task (read "route") works with can be exactly what the server sent to the client. We can clear out unwanted state in the core reducer by only keeping cross-task data whenever the client's route changes (such as user-name, session token etc).


function coreReducer(state: any, action: Action) {
  switch (action.type) {
    case 'ChangeRoute': 
      return { loginInfo: state.loginInfo };
         
    default:
      return state;
  }
}

Of course, normalised data can be cleared down in this way too, when switching between tasks. However, imagine a task such as "Dashboard" onto which users can drop any number of widgets that present server data of all kinds. As these widgets update their data they are unable to relinquish any data they previously loaded, because without reference counting on everything they refer to there is no way to know which rows of data are not being used by any of the other widgets currently on screen. But if our data is nested then each widget contains all of the data it needs at any moment. Some of the same data (addresses, employees, etc) may exist in memory on the client more than once (one per widget) but that data is unloaded when finished with. The use of additional memory means that our memory usage doesn't creep up throughout the day.

To reduce nested (complex) data is quite easy. Using an approach I learned on a Martin Odersky course it is possible to change an object at the left of a complex data tree without having to replace the entire tree.



export const subReduce = (state: any, action: any, reducers: any) => {
  // If we have an original state then copy it, otherwise keep a null state
  let newState = state ? {...state} : null;
  for (const key of Object.keys(reducers)) {
    // Grab the previous member state. If passed state is null the member state is null
    const previousMemberState = state !== null ? state[key] : null;
    const reducer = reducers[key];
    const newMemberState = reducer(previousMemberState, action);

    // Scenarios are
    // 1: No initial state object and new member state is null = keep null state
    // 2: No initial state object and has new member state = new state + set member
    // 3: Has initial state object and member state set = keep state + set member
    // 4: Has initial state object and member state is null = keep state + unset member
    // If we need to set a member to a non null value then we need an instance
    if (newState === null && newMemberState) {
      newState = {};
    }
    if (newState) {
      newState[key] = newMemberState;
    }
  }
  return newState;
}

There is still a reducer per object type, but these reducers are now nested too. They used the subReduce() function to declared their complex child properties.


export const addressReducer = (state: Address = null, action: any) => {
  switch (action.type) {
    case 'address.update':
      return Object.assign({}, state, action.payload);
  }
  return state;
}

export const addressArrayReducer = (state: Address[] = null, action: any) => {
  if (!state) {
    return state;
  }

  return state.map(x => {
    if (x.id === action.payload.id) {
      return addressReducer(x, action);
    } else {
      return x;
    }
  });
}

export const personReducer = (state: Person = null, action: any) => {
  state = subReduce(state, action, {
    addresses: addressArrayReducer
  });
  switch (action.type) {
    case 'person.setName':
      return {
        ...state,
        name: action.payload
      };
  }
  return state;
}

export const rootReducer = {
  person: personReducer
}

Of course, you can use this approach and still use normalised data to benefit from the redundant data-unloading whilst keeping your normalised data.


{
  task1: null,
  task2: null,
  dashboard: {
    widget1: {
      "people": {
        "Emp1024": { "name": "Peter Morris", "address": "123" },
        "Emp4096": { "name": "Bob Smith", "address": "254" },
        "_ids": [ "Emp1024", "Emp4096" ]
      },
      "addresses": [
        "123": { "street": "xxx" },
        "254": { "street": "yyy" }
      ]
    },
    widget2: { /* whatever */ },
    widget3: {
      "people": {
        "Emp1024": { "name": "Peter Morris"},
        "Emp4096": { "name": "Bob Smith"},
        "Emp8192": { "name": "Aled Jones"},
        "_ids": [ "Emp4096", "Emp8192", "Emp1024" ]
      }
    }
  },
}

2017-06-14

Loading an assembly from a specific path, including all sub dependencies

    public static class AssemblyLoader
    {
        private static readonly ConcurrentDictionary<string, bool> AssemblyDirectories = new ConcurrentDictionary<string, bool>();

        static AssemblyLoader()
        {
            AssemblyDirectories[GetExecutingAssemblyDirectory()] = true;
            AppDomain.CurrentDomain.AssemblyResolve += ResolveAssembly;

        }

        public static Assembly LoadWithDependencies(string assemblyPath)
        {
            AssemblyDirectories[Path.GetDirectoryName(assemblyPath)] = true;
            return Assembly.LoadFile(assemblyPath);
        }

        private static Assembly ResolveAssembly(object sender, ResolveEventArgs args)
        {
            string dependentAssemblyName = args.Name.Split(’,’)[0] + ".dll";
            List<string> directoriesToScan = AssemblyDirectories.Keys.ToList();

            foreach (string directoryToScan in directoriesToScan)
            {
                string dependentAssemblyPath = Path.Combine(directoryToScan, dependentAssemblyName);
                if (File.Exists(dependentAssemblyPath))
                    return LoadWithDependencies(dependentAssemblyPath);
            }
            return null;
        }

        private static string GetExecutingAssemblyDirectory()
        {
            string codeBase = Assembly.GetExecutingAssembly().CodeBase;
            var uri = new UriBuilder(codeBase);
            string path = Uri.UnescapeDataString(uri.Path);
            return Path.GetDirectoryName(path);
        }
    }

2017-06-08

Running dotnet core xUnit tests on Visual Studio Team Services (VSTS)


1: Run dotnet restore to restore package dependencies.

2: Run dotnet build to build the binaries.

3: Run dotnet test to run the tests. Note the additional parameters --no-build to prevent a rebuild and --logger "trx;LogFileName=tests-log.trx" to ensure the test results are written to disk,

5: Use  a Publish Test Results tasks to output the results of the tests. Make sure you set the following properties
  • Test Result Format = VSTest
  • Test Results Files = **/tests-log.trx
And under the Advanced section make sure you set the Run This Task option so that it will run even if the previous task failed.


2017-05-04

Get list of object keys in Angular

import { PipeTransform, Pipe } from "@angular/core";
@Pipe({ name: 'keys' })
export class KeysPipe implements PipeTransform {
  transform(value, args:string[]) : any {
    return Object.keys(value);
  }
}

Then to get a list of errors for a form element you can do this

<ul *ngIf="form.get('userName').invalid" class="help-block with-errors">
   <li *ngFor="let error of form.get('userName').errors | keys">{{ error.key }}</li>
</ul>

2016-12-21

Preventing Unity3D IL2CPP from stripping your code

I was trying to get a list of a type's constructors at runtime using reflection, so that I could create an instance of the class using dependency injection.

All worked just fine until we tried to build the app for iOS. At first we were using Mono as the scripting back-end, but it seems that new versions of iOS pop up a dialog telling the user the app is 32 bit and may run slowly (i.e. "Your app is crap"). When switching the backend scripting to IL2CPP (in File->Builder->Player Settings) the app suddenly wasn't working. It turns out that SomeType.GetConstructors().Count was returning zero, which was a problem because obviously I wanted to invoke those constructors with dependencies.

The problem was that because these constructors weren't being calling explicitly from anywhere in my app IL2CPP decided I didn't need them, and stripped them out.

The solution is to create a file in your Assets folder called link.xml and fill it in like so....

<linker>
  <assembly fullname="Assembly-CSharp">
    <type fullname="Holovis.*" preserve="all"></type>
  </assembly>
</linker>

Assembly-CSharp is the default name created for all of your scripts by Unity. If you are using any others you need to prevent stripping on then you can add additional <assembly> nodes. As you can see from the example you can list types individually or use * as a wild card.  You can have multiple <type> nodes per <assembly> node.

Read this page from the manual for more information.

2016-12-09

Forcing a device-orientation per scene in Unity3D

Unity3D has a Screen class with an orientation property that allows you to force orientation in code, which lets you have different scenes with different orientations (useful in mini-games). this works fine for Android but crashes on iOS.
The problem is the file UnityViewControllerBaseiOS.mm that gets generated during the build for iOS has an assert in it which inadvertently prevents this property from being used. It is possible to create a post-build class that runs after the iOS build files have been generated that can alter the generated code before you compile it in XCode.
Just create a C# script named iOSScreenOrientationFix.cs and paste in the following code - adapted from this Unity3D forum post.
using UnityEngine;
using UnityEditor;
using UnityEditor.Callbacks;
using System.IO;

namespace Holovis
{
    public class iOSScreenOrientationFix : MonoBehaviour
    {
#if UNITY_CLOUD_BUILD
    // This method is added in the Advanced Features Settings on UCB
    // PostBuildProcessor.OnPostprocessBuildiOS
    public static void OnPostprocessBuildiOS (string exportPath)
    {
        Debug.Log("OnPostprocessBuildiOS");
        ProcessPostBuild(BuildTarget.iPhone,exportPath);
    }
#endif

        [PostProcessBuild]
        public static void OnPostprocessBuild(BuildTarget buildTarget, string path)
        {
#if !UNITY_CLOUD_BUILD
            ProcessPostBuild(buildTarget, path);
#endif
        }

        private static void ProcessPostBuild(BuildTarget buildTarget, string path)
        {
            if (buildTarget == BuildTarget.iOS)
            {
#if !UNITY_CLOUD_BUILD
                Debug.Log("Patching iOS to allow setting orientation");
#endif
                string filePath = Path.Combine(path, "Classes");
                filePath = Path.Combine(filePath, "UI");
                filePath = Path.Combine(filePath, "UnityViewControllerBaseiOS.mm");

                Debug.Log("File Path for View Controller Class: " + filePath);

                string classFile = File.ReadAllText(filePath);

                string newClassFile = classFile.Replace("NSAssert(UnityShouldAutorotate()", "//NSAssert(UnityShouldAutorotate()");

                File.WriteAllText(filePath, newClassFile);
            }
        }
    }
}
You can set it in a scene by attaching the following MonoBehaviour to a game object
using UnityEngine;

namespace Holovis
{
    public class SetDeviceOrientation : MonoBehaviour
    {
        public ScreenOrientation orientation = ScreenOrientation.AutoRotation;

        void Awake()
        {
            Screen.orientation = orientation;
        }
    }
}

2016-11-29

A UI thread dispatcher for Unity3D

I've recently been working on implementing an IHttpService that can work outside of a MonoBehaviour and call out to web services asynchronously.

Rather than providing my IHttpService.JsonPost method with callbacks I decided to have it return a Promise, the code for which I took from this Real Serious Games GitHub repository.

The problem is that when you use WebClient's async methods they call back the Complete events on the worker thread, so code like this won't work because you cannot manipulate UI from any thread other than the main one.

httpService.JsonPost<MyInfo>(url)
  .Then(myInfo => someTextUiObject.text = myInfo.Name);
And there seems to be no kind of thread Dispatcher in Unity3D for UI updates as there is in Windows.Forms - so I wrote my own.

using System.Collections;
using System;
using System.Threading;
using System.Collections.Generic;
using UnityEngine;
public class UiThreadDispatcher : Singleton<MonoBehaviour>
{
    static volatile int lockValue = 0;
    static Queue<Action> actionQueue = new Queue<Action>();
    void Awake()
    {
        StartCoroutine(CheckForDispatchedActions());
    }
    public static void Dispatch(Action action)
    {
        Lock();
        actionQueue.Enqueue(action);
        Unlock();
    }
    static void Lock()
    {
        while (Interlocked.Exchange(ref lockValue, 1) != 0) { }
    }
    static void Unlock()
    {
        lockValue = 0;
    }
    private IEnumerator CheckForDispatchedActions()
    {
        while (true)
        {
            Action action = null;
            Lock();
            try
            {
                while (actionQueue.Count > 0)
                {
                    action = actionQueue.Dequeue();
                    action();
                }
            }
            catch (Exception unexpectedException)
            {
                Debug.LogException(unexpectedException);
            }
            finally
            {
                Unlock();
            }
            yield return null;
        }
    }
}

The idea is that any piece of code can call the static Dispatch method, for example

//Instead of this
httpService.JsonPost<MyInfo>(url)
  .Then(myInfo => someTextUiObject.text = myInfo.Name);
//You would do this
httpService.JsonPost<MyInfo>(url)
  .Then(myInfo => UiThreadDispatcher.Dispatch(() => > someTextUiObject.text = myInfo.Name));
The Queue object needs to be locked to Enqueue/Dequeue items, but I use a simple SpinLock pattern because clashes will be very rare. Then I just have the MonoBehaviour instance execute the queued actions.

*Note to self: Must test on iOS.