David Develops (Software)

As browsing becomes more commonplace on phones, sub-notebooks and (within the next year or so) Tablet PCs, there’s an increased appetite to tailor your user experience for people using these “non-desktop” devices.  You can leverage your existing application infrastructure without having to create costly or outsource applications for specific (*cough* iPhone) platforms that have lots of market share, to the exclusion of others (*cough* iPhone).

There are some excellent examples of this in the wild, Facebook and the BBC do a great job of this already.

So what about us ASP.NET MVC types?  Well thankfully, there’s a lot of stuff built in to the framework to allow us to do this with relative ease.  First you’re going to need a few things..

1) Go grab the latest Mobile Device Browser File from http://mdbf.codeplex.com.  If you’re familiar with the old “Browser Caps”, it’s the same sort of thing.  A regularly updated collection of device data.  It’s pretty interesting on it’s own, breaking down incoming devices by user agent / headers and offering you various stats (touch enabled, screen resolution etc).  You’ll probably want to keep this up to date periodically.

2) A new ViewEngine!  Thankfully, ASP.NET MVC has a pretty flexible pipeline when it comes to slotting in new ViewEngines.  The most transparent way to switch out which view you’re serving for any given request is to override some of the logic in the default ViewEngine to look elsewhere when a View is requested by the MVC framework.

3) A way to test this stuff.  I normally break out the “User Agent Switcher” FireFox plugin which you can grab here: https://addons.mozilla.org/en-US/firefox/addon/59

The View Engine

In order to keep this as close to vanilla MVC as possible, I’m going to extend the regular WebFormsViewEngine to add mobile device detection and view overriding.  What I’m going to do is check for a mobile browser when a request reaches the view engine, and if one is found, add some extra paths to look for the view files in at the top of the list of locations searched.  By doing this, we can prioritise the mobile edition of a website if the user is visiting from a phone, while degrading gracefully, allowing the regular version of the website to be served if a mobile version of a given page isn’t available.  This actually allows us to support mobile views piece by piece rather than forcing us to support the entire site out of the box.

Unfortunately, a few of the methods surrounding view resolution are marked as private in the WebFormsViewEngine and as such are inaccessible.  I’ve had to reflect in and copy a couple of methods to get around this.  Ideally the access modifier on these could be changed in later versions of the framework.

The key method we have to work with is

public override ViewEngineResult FindView(ControllerContext controllerContext, string viewName, string masterName, bool useCache)
{ … }

What we’re going to do is add a few extra locations based on the browser type.  With the mobile device browser file installed, this is really simple:

public override ViewEngineResult FindView(ControllerContext controllerContext, string viewName, string masterName, bool useCache)
{
    if (controllerContext == null)
    {
        throw new ArgumentNullException("controllerContext");
    }
    if (String.IsNullOrEmpty(viewName))
    {
        throw new ArgumentException("viewName");
    }

    List<string> viewLocationsSearched;
    List<string> masterLocationsSearched;

    string[] viewLocationsToSearch = ViewLocationFormats;
    string[] masterLocationsToSearch = MasterLocationFormats;

    viewLocationsToSearch = AddMobileViewLocations(controllerContext, viewLocationsToSearch, MobileViewLocationFormats);
    masterLocationsToSearch = AddMobileViewLocations(controllerContext, masterLocationsToSearch, MobileMasterLocationFormats);

    string controllerName = controllerContext.RouteData.GetRequiredString("controller");
    string viewPath = GetPath(controllerContext, viewLocationsToSearch, viewName, controllerName, CacheKeyPrefixView, useCache, out viewLocationsSearched);
    string masterPath = GetPath(controllerContext, masterLocationsToSearch, masterName, controllerName, CacheKeyPrefixMaster, useCache, out masterLocationsSearched);

    if (String.IsNullOrEmpty(viewPath)
        || (String.IsNullOrEmpty(masterPath)
        && !String.IsNullOrEmpty(masterName)))
    {
        return new ViewEngineResult(viewLocationsSearched.Union(masterLocationsSearched));
    }

    return new ViewEngineResult(CreateView(controllerContext, viewPath, masterPath), this);
}

You’ll notice there’s a few methods that get called in there.  The most important of which is “AddMobileViewLocations”.  This really is where all the legwork is done, and looks like this

public class SwitchingViewEngine : WebFormViewEngine
{
    private const string CacheKeyFormat = ":ViewCacheEntry:{0}:{1}:{2}:{3}:";
    private const string CacheKeyPrefixMaster = "Master";
    private const string CacheKeyPrefixView = "View";
    private static readonly List<string> EmptyLocations = new List<string>();

    protected string[] MobileViewLocationFormats { get; private set; }
    protected string[] MobileMasterLocationFormats { get; private set; }

    public SwitchingViewEngine()
    {
        ViewLocationFormats = new[]
                                  {
                                      "~/Views/{1}/{0}.aspx", "~/Views/{1}/{0}.ascx", "~/Views/Shared/{0}.aspx",
                                      "~/Views/Shared/{0}.ascx"
                                  };

        MobileViewLocationFormats = new[]
                                        {
                                            "~/Views/{1}/{0}.mobile.aspx", "~/Views/{1}/{0}.mobile.ascx",
                                            "~/Views/Shared/{0}.mobile.aspx",
                                            "~/Views/Shared/{0}.mobile.ascx"
                                        };

        MasterLocationFormats = new[] {"~/Views/{1}/{0}.master", "~/Views/Shared/{0}.master"};
        MobileMasterLocationFormats = new[] {"~/Views/{1}/{0}.mobile.master", "~/Views/Shared/{0}.mobile.master"};
    }

    public override ViewEngineResult FindView(ControllerContext controllerContext, string viewName, string masterName, bool useCache)
    {   …    }

    private static string[] AddMobileViewLocations(ControllerContext controllerContext,
                                                   string[] viewLocationsToSearch,
                                                   IEnumerable<string> mobileViewLocations)
    {
        if (controllerContext == null
            || controllerContext.HttpContext == null
            || controllerContext.HttpContext.Request == null
            || controllerContext.HttpContext.Request.Browser == null
            || viewLocationsToSearch == null
            || viewLocationsToSearch.Length == 0
            || mobileViewLocations == null
            || mobileViewLocations.ToList().Count == 0
            || !controllerContext.HttpContext.Request.Browser.IsMobileDevice)
        {
            return viewLocationsToSearch;
        }

        var mobileViews = viewLocationsToSearch.ToList();
        foreach (var view in mobileViewLocations.Reverse())
        {
            mobileViews.Insert(0, view);
        }

        viewLocationsToSearch = mobileViews.ToArray();

        return viewLocationsToSearch;
    }

This method takes the current ViewLocations that are defined at the top of the class, does a bunch of guard checks (to prevent mobile switching crashing your request.. call me paranoid), then verifies that controllerContext.HttpContext.Request.Browser.IsMobileDevice is true.  This check makes use of the browser device file.  If all the checks pass, it inserts the new “mobile view paths” at the very top of the list of paths to be searched when resolving the location of a view file.  The calling method then subsequently calls the method “GetPath” (which is one of the private methods I’ve had to reflect out of the framework source code).  GetPath searches the supplied list of potential view locations, and returns as soon as it finds a match.  In our case, if the browser is mobile, and a view file with the extension mobile.aspx is found, this will be the first view resolved and returned.

The full code listing for the view engine (don’t worry, there’s a link at the bottom to grab all of this in one archive):

using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Web.Mvc;

namespace MultipleViewMvcExample.DemoCode
{
    public class SwitchingViewEngine : WebFormViewEngine
    {
        private const string CacheKeyFormat = ":ViewCacheEntry:{0}:{1}:{2}:{3}:";
        private const string CacheKeyPrefixMaster = "Master";
        private const string CacheKeyPrefixView = "View";
        private static readonly List<string> EmptyLocations = new List<string>();

        protected string[] MobileViewLocationFormats { get; private set; }
        protected string[] MobileMasterLocationFormats { get; private set; }

        public SwitchingViewEngine()
        {
            ViewLocationFormats = new[]
                                      {
                                          "~/Views/{1}/{0}.aspx", "~/Views/{1}/{0}.ascx", "~/Views/Shared/{0}.aspx",
                                          "~/Views/Shared/{0}.ascx"
                                      };

            MobileViewLocationFormats = new[]
                                            {
                                                "~/Views/{1}/{0}.mobile.aspx", "~/Views/{1}/{0}.mobile.ascx",
                                                "~/Views/Shared/{0}.mobile.aspx",
                                                "~/Views/Shared/{0}.mobile.ascx"
                                            };

            MasterLocationFormats = new[] {"~/Views/{1}/{0}.master", "~/Views/Shared/{0}.master"};
            MobileMasterLocationFormats = new[] {"~/Views/{1}/{0}.mobile.master", "~/Views/Shared/{0}.mobile.master"};
        }

        public override ViewEngineResult FindView(ControllerContext controllerContext, string viewName, string masterName, bool useCache)
        {
            if (controllerContext == null)
            {
                throw new ArgumentNullException("controllerContext");
            }
            if (String.IsNullOrEmpty(viewName))
            {
                throw new ArgumentException("viewName");
            }

            List<string> viewLocationsSearched;
            List<string> masterLocationsSearched;

            string[] viewLocationsToSearch = ViewLocationFormats;
            string[] masterLocationsToSearch = MasterLocationFormats;

            viewLocationsToSearch = AddMobileViewLocations(controllerContext, viewLocationsToSearch, MobileViewLocationFormats);
            masterLocationsToSearch = AddMobileViewLocations(controllerContext, masterLocationsToSearch, MobileMasterLocationFormats);

            string controllerName = controllerContext.RouteData.GetRequiredString("controller");
            string viewPath = GetPath(controllerContext, viewLocationsToSearch, viewName, controllerName, CacheKeyPrefixView, useCache, out viewLocationsSearched);
            string masterPath = GetPath(controllerContext, masterLocationsToSearch, masterName, controllerName, CacheKeyPrefixMaster, useCache, out masterLocationsSearched);

            if (String.IsNullOrEmpty(viewPath)
                || (String.IsNullOrEmpty(masterPath)
                && !String.IsNullOrEmpty(masterName)))
            {
                return new ViewEngineResult(viewLocationsSearched.Union(masterLocationsSearched));
            }

            return new ViewEngineResult(CreateView(controllerContext, viewPath, masterPath), this);
        }

        private static string[] AddMobileViewLocations(ControllerContext controllerContext,
                                                       string[] viewLocationsToSearch,
                                                       IEnumerable<string> mobileViewLocations)
        {
            if (controllerContext == null
                || controllerContext.HttpContext == null
                || controllerContext.HttpContext.Request == null
                || controllerContext.HttpContext.Request.Browser == null
                || viewLocationsToSearch == null
                || viewLocationsToSearch.Length == 0
                || mobileViewLocations == null
                || mobileViewLocations.ToList().Count == 0
                || !controllerContext.HttpContext.Request.Browser.IsMobileDevice)
            {
                return viewLocationsToSearch;
            }

            var mobileViews = viewLocationsToSearch.ToList();
            foreach (var view in mobileViewLocations.Reverse())
            {
                mobileViews.Insert(0, view);
            }

            viewLocationsToSearch = mobileViews.ToArray();

            return viewLocationsToSearch;
        }

        private string GetPath(ControllerContext controllerContext, string[] locations, string name,
                               string controllerName, string cacheKeyPrefix, bool useCache,
                               out List<string> searchedLocations)
        {
            searchedLocations = EmptyLocations;
            if (string.IsNullOrEmpty(name))
            {
                return string.Empty;
            }
            if ((locations == null) || (locations.Length == 0))
            {
                throw new InvalidOperationException("Property cannot be null or empty.");
            }
            bool flag = IsSpecificPath(name);
            string key = CreateCacheKey(cacheKeyPrefix, name, flag ? string.Empty : controllerName);
            if (useCache)
            {
                string viewLocation = ViewLocationCache.GetViewLocation(controllerContext.HttpContext, key);
                if (viewLocation != null)
                {
                    return viewLocation;
                }
            }
            if (!flag)
            {
                return GetPathFromGeneralName(controllerContext, locations, name, controllerName, key,ref searchedLocations);
            }
            return GetPathFromSpecificName(controllerContext, name, key, ref searchedLocations);
        }
        private static bool IsSpecificPath(string name)
        {
            char ch = name[0];
            if (ch != ‘~’)
            {
                return (ch == ‘/’);
            }
            return true;
        }

        private string GetPathFromSpecificName(ControllerContext controllerContext, string name, string cacheKey,
                                               ref List<string> searchedLocations)
        {
            string virtualPath = name;
            if (!FileExists(controllerContext, name))
            {
                virtualPath = string.Empty;
                searchedLocations = new List<string> {name};
            }
            ViewLocationCache.InsertViewLocation(controllerContext.HttpContext, cacheKey, virtualPath);
            return virtualPath;
        }

        private string GetPathFromGeneralName(ControllerContext controllerContext, string[] locations, string name,
                                              string controllerName, string cacheKey, ref List<string> searchedLocations)
        {
            string virtualPath = string.Empty;
            searchedLocations = new List<string>();
            for (int i = 0; i < locations.Length; i++)
            {
                string str2 = string.Format(CultureInfo.InvariantCulture, locations[i],
                                            new object[] {name, controllerName});
                if (FileExists(controllerContext, str2))
                {
                    searchedLocations = EmptyLocations;
                    virtualPath = str2;
                    ViewLocationCache.InsertViewLocation(controllerContext.HttpContext, cacheKey, virtualPath);
                    return virtualPath;
                }
                searchedLocations[i] = str2;
            }
            return virtualPath;
        }

        private string CreateCacheKey(string prefix, string name, string controllerName)
        {
            return String.Format(CultureInfo.InvariantCulture, CacheKeyFormat,
                                 GetType().AssemblyQualifiedName, prefix, name, controllerName);
        }
    }
}

 

The Wiring

Now you have your view engine, you need to register it as the default view engine in your MVC application.  Easy!  Open up your Global.aspx.cs file and add the following to ApplicationStart

ViewEngines.Engines.Clear();
ViewEngines.Engines.Add(new SwitchingViewEngine());

Done!

Now you need to actually add the browser detection file to your application.  Presuming you downloaded the latest archive from the codeplex url at the top of this article, all you need to do is copy the supplied mobile.browser file to App_Browsers/Device/* in your MVC application.

That’s all the wiring you need to get everything up and running.

If you’ve done it right, a default “New MVC Template” project with these additions might look something like this:

For the sake of this demo, I put the view engine in a “DemoCode” sub-namespace. You don’t want to do that, put it somewhere sensible!

The more astute reader might now notice that my Views/Home directory in the above screenshot has an extra file, not supplied by the template, called “Index.mobile.aspx”.  Likewise, my /Views/Shared directory has Site.mobile.Master.  These are files I want the view engine to resolve if a user hits the http://localhost/Home default route from a mobile device.

Index.mobile.aspx looks like this:

<%@ Page Language="C#" MasterPageFile="~/Views/Shared/Site.mobile.Master" Inherits="System.Web.Mvc.ViewPage" %>

<asp:Content ID="indexTitle" ContentPlaceHolderID="TitleContent" runat="server">
    Mobile Home Page
</asp:Content>

<asp:Content ID="indexContent" ContentPlaceHolderID="MainContent" runat="server">
    <h2><%= Html.Encode(ViewData["Message"]) %></h2>
    This is my mobile index page.
</asp:Content>

and Site.mobile.Master looks like this:

<%@ Master Language="C#" Inherits="System.Web.Mvc.ViewMasterPage" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title><asp:ContentPlaceHolder ID="TitleContent" runat="server" /></title>
</head>

<body>
    <div class="page">

        <div id="header">
            <div id="title">
                <h1>My MVC Application - mobile master page</h1>
            </div>
            <div id="logindisplay">
                <% Html.RenderPartial("LogOnUserControl"); %>
            </div>
            <div id="menucontainer">
                <ul id="menu">             
                    <li><%= Html.ActionLink("Home", "Index", "Home")%></li>
                    <li><%= Html.ActionLink("About", "About", "Home")%></li>
                </ul>
            </div>
        </div>

        <div id="main">
            <asp:ContentPlaceHolder ID="MainContent" runat="server" />

            <div id="footer">
            </div>
        </div>
    </div>
</body>
</html>

Nothing especially revolutionary.

Testing

First, install that FireFox plugin I mentioned at the top of the article, create a new MVC project (just use the template), and wire up the view engine and browser file.  Add some extra views with the .mobile.aspx prefix.  Or download the sample attached to the bottom of this post!

Start the site up in Cassini (Visual Studios default web server) and hit /Home.  You should see this:

Now, lets use the new FireFox plugin…

Select iPhone 3.0 from that menu and refresh the page…

Bang!  The more eagle eye reader might have notice that all I did in my “mobile” master page, was delete the CSS reference from the default MVC template, thus the above screenshot.

And that’s it.

Further Thoughts

This solution goes quite a way, but here are a few other ideas:

• Don’t just do browser type detection, detect and switch on subdomain, so any visitors hitting http://m.mysite.com get a different view.
• Allow the user to opt-out of the reduced view with a session cookie.
• Switch views to a low-fi version to victimise IE6 users!
• Target tablet PCs based on resolution to build a “touch UI”

It’s all pretty simple.  The really nice thing about this solution is that your designers can just add these mobile views as and when they see fit, as the same action methods that execute for the “full fat” website are run, and the same strongly typed view models (which you’re using, right? get out of here with your ViewData..) are delivered.  Designers can implement portable websites, piece by piece, just with a little view engine change.

As with all internet code, your mileage may vary, but this technique works for me.

You can download a working VS2008 solution (so long as you have ASP.NET MVC1 installed on your system) containing all the code used above from: http://github.com/davidwhitney/MultipleViewMvcExample

What are APIs anyway?

Everyone’s heard of APIs these days.  Facebook has them, Twitter has them, Hotmail has them, Microsoft Office has them, Windows has them, Mac OS has them, pretty much everything has them.  I’m going to try and explain in simple terms, but also, almost by contradiction, in detail, what an API really is.  I’ll try and explain why SOAP isn’t how you clean yourself, and how being restful isn’t the same as being lazy.

APIs aren’t new, in fact, APIs are really really old.

Let’s start with a really simple definition.  API is an acronym and it stands for “Application Programming Interface”

To steal the current definition from Wikipedia “An application programming interface (API) is an interface implemented by a software program to enable interaction with other software, much in the same way that a user interface facilitates interaction between humans and computers. APIs are implemented by applications, libraries and operating systems to determine the vocabulary and calling conventions the programmer should employ to use their services. It may include specifications for routines, data structures, object classes and protocols used to communicate between the consumer and implementer of the API.”

That’s a mouthful.  So to translate that into English, an API is a pre-defined set of “stuff” that allows one program to “talk” to another program.  This “stuff” is normally a set of “functions” that another program can call which makes the program being called do something.  Sometimes that other program is really just another part of the same program, and sometimes it’s a different system on a different machine in a different country.  That something is normally described in some kind of documentation, written somewhere, by someone.

If this sounds really vague it’s because in the real world, it really is.  Some of the first “APIs” involved dropped text files into directories on a computer that another program would watch for a read from, and subsequently do “something”.  Arguably the most widely used API is the one that we use to write software for Windows (the Win32 API) and by contrast, that’s a C++ library of code that you can only call if you’re a programmer.  In the real world, we’ve tried to solve some of this ambiguity by standardizing the way APIs work around a few common bits of technology; both for our own sanity and to hopefully help software all just kind of work together.

A lot of APIs are code libraries called by other code libraries to do a specific task.  Microsoft released DirectX to deal with 3d graphics, OpenGL offered an open alternative.  Microsoft released the Windows API for Windows development; Apple released Carbon and Cocoa to program Mac OS.  That said when you hear about or discuss APIs casually, what you’re probably thinking about are actually “Web APIs”.

Web APIs

A Web API is an API like anything else, except it’s designed to work over the web.  As a result of this, since about 1994, there have been a number of efforts by a number of people (yes, that vague again!) to standardize the way systems communicate over the internet.

At first, everyone kind of invented their own way of communicating over the internet, people would connect to a server on some port (a port is just a pre-defined “way in”) and send some random pre-defined data in there, and that would make the computer at the other end do something.  Every API was different, and every time that you needed to talk to a different system, you had a really steep learning curve to work out how to talk to every application you wanted to integrate with.  It was a bit rubbish really.

So in 1998 a bunch of really smart people (Dave Winer, Don Box, Bob Atkinson, and Mohsen Al-Ghosein) got together and came up with SOAP (“Simple Object Access Protocol”).  SOAP is a big bunch of XML that was designed (and ratified) as a standard language that every different system could understand. SOAP is often also referred to as “web services”.

Basically it was designed to reduce the learning curve of learning the details of each system.  So now, if Timmy wanted to talk to Peters shiny new web application, he could point his developer tools at a standard location (something called a WSDL (web service description) document) and he could just ask Peters web application what it did, and how he could use it.

SOAP was actually pretty good and solved a lot of problems and is still widely used today.  It really made waves by helping Microsoft software work pretty well with Java software and everyone was happy.

Almost.

See, because SOAP was solving a lot of big problems in freaking huge enterprise systems, it had a lot to be concerned about; lots of security, lots of encryption, lots of authentication.   That meant that the SOAP “language” was pretty big.  As an example, here (stolen from Wikipedia again!) is the SOAP message that would be sent across the internet to get the current stock price of IBM, from some cool stock price giving web service.

POST /InStock HTTP/1.1

Host: www.example.org

Content-Type: application/soap+xml; charset=utf-8

<?xml version=”1.0″?>

<soap:Envelope

xmlns:soap=”http://www.w3.org/2001/12/soap-envelope”

soap:encodingStyle=”http://www.w3.org/2001/12/soap-encoding”>

<soap:Body xmlns:m=”http://www.example.org/stock”>

<m:GetStockPrice>

<m:StockName>IBM</m:StockName>

</m:GetStockPrice>

</soap:Body>

</soap:Envelope>

People soon started thinking “Look at all that crap! What’s it all for! Why do I need it?! There’s lots of overhead here!  It slows me down!” and everyone thought “oh actually, well said” and RESTful web services were born.

REST is yet another stupid acronym that actually doesn’t count as an acronym because it uses random letters but developers think is either cool clever or funny.  What it claims to mean is “REpresentational State Transfer” and was largely both a reaction to the complexity and overheads of SOAP, but also being designed by Roy Fielding (one of the authors of the “Hypertext transfer protocol”, the silly http:// bit you type in a browser) it was conceived as an API model that was “closer to the nature of the web”.

What this means to us lay folk, is that while SOAP can technically be used over other protocols (it’s not bound to http), and as such has to bake in security and authentication models into its protocol, REST is designed specifically for the web, it doesn’t work without the web, and it wouldn’t make any sense without the web.  Fielding basically decided that “the web does all of that stuff anyway”, we have security via https / SSL certificates, we have semantics that describe getting and pushing data in the HTTP headers (HTTP headers explain what you’re trying to do to the server you’re connecting to, for example, you use GET for getting stuff, POST and PUT for doing stuff), so let’s just use that and be done with it.

As a result of this way of thinking, REST is a simplified and less general Web API pattern, not concerned with infrastructure like SOAP is.  I’ll convert the above stock price example into a REST example below.

GET /Stocks/Price/IBM HTTP/1.1

Host: www.example.org

Content-Type: application/xml; charset=utf-8

Done.

Basically, REST takes out all the fluff, an decides that if you want to get a stock price for IBM, just make a GET request to the URL http://www.example.org/Stocks/Price/IBM and let the web server at the other end work out what to do from the URL.

The above example would probably return an XML document that looks like this.

<stocks><stock name=”IBM” price=”3.45”/></stocks>

People have started to gravitate towards REST due to its simplicity.  There’s lots of other stuff a RESTful web service should do, it should provided you with all the information that a computer would need to go through a process, just like a user interface gives the user all the information they need to click through a process, but fundamentally it’s simple and leverages the existing semantics of the web to its advantage.

Ok, Give Me One Of Those!

So now we know what an API is there to do (let two systems talk to each other) and how they do it (as a general rule, either via SOAP or RESTful services) and we know that everyone else has one, let’s get one of our own.

There are plenty of tools available in pretty much any programming language to make making creating APIs pretty easy.  There are plenty of design concerns to take into account when building an API but in my opinion, the way to approach the problem is to work out what your users really want to do with your application, website or platform, and let them do it.

A good API lets another programmer talk to your system using terms that he understands, so take the time building up a glossary of your business terms vs. what the public understands those concepts to be.  Don’t build API methods that look like:

/PaymentResolutionProcess/PaymentResolveTable3/Resolve/EntityId/123

because it means nothing to anyone, instead, build a bunch of methods that make sense for people to use, the above example could look like this instead:

/Payment/MakePayment/123

Watch your language, and build sensible interactions with your system.  Don’t make APIs for stuff that isn’t going to be used, and where possible, just have the APIs call the same code that your website does.

Get this right, and you’re on a gradual but successful road to calling your “website” a “platform”.

With the popularity of service oriented architectures and other buzz phrases related to software as service, good API design has become a significant selling point for any software platform in the past 5-10 years.  People make purchasing decisions based on how easy it is to interoperate with your applications and code and as such the number of client / public facing APIs attached to software has skyrocketed.  I’d like to believe the days of dropping strategic text files in directories to trigger some action or another in an application are behind us.

In this article I’m going to talk about the following things

• Why you should choose your method names carefully, and what to call
• Why pretending to be a data access layer is a terrible thing for an API to do
• Talk about the dangers of leaky abstractions in an API
• Explain the benefits of creating a data contract between you and the calling code
• Explain why it’s vital to support standards
• Make sure that your users can retrieve values they’re going to want to modify
• Suggest supplying compiled libraries alongside your API documentation
• Explain why it’s important to keep your API implementation clean
• Talk about the benefits of dogfooding your API
• Consider supporting atomic operations including rollbacks on failure
• Discuss bulk operations
• Try and convince you that both logging and security should be first class citizens
• Beg you to maintain integration tests and most of all to keep it simple!

Your API Sucks

You’ve probably used an API and there’s a good chance you’ve had to write one.  This probably won’t surprise you; most APIs suck.  They’re horrible to use and built around illogical leaky abstractions that leave you flicking through huge wads of documentation just to make the most rudimentary feature work.

About 18 months ago, after a year of struggling with a broken third party API that almost brought a business to it’s knees by placing significant roadblocks in front of in house development, I was part of a team tasked with designing our own client facing API.  With no desire to expose other developers to the cruel and unusual punishments of software design we’d had to endure, we came to the conclusion that it was really important that we god this piece of the system right.  People say first impressions are everything, and your API design can make or break the faith other developers have in your ability to produce software.  Show somebody a shitty API and they’ll perhaps correctly assume the rest of your code sucks too.

The Best Man For The Job

There’s a bit of a trend that I’ve noticed with some of the worst APIs I’ve worked with: they seem to be designed by the wrong people.  The wrong people to design an API are 1) the guy that wrote the internal code to do the job the API is providing access to and 2) the consumer of the API. 

The guy that wrote the code that the API is calling under the hood will be inherently slanted to implement an API which exposes this functionality and will have a predisposition to creating a leaky abstraction.  This is especially bad for the consumer APIs designed by the internal implementer tend to assume the consumer knows far more than he really does, or has access to internal data that in reality, he doesn’t.

Conversely, an API designed by the consumer of the API will have a tendency towards solving problems that are not the concern of the API itself.  The consumer will, either accidently or by intention, attempt to offload some of the work that should be the responsibility of the calling code into the API.

Ideally, the person that’s writing the API will have knowledge of the system internals, but not be the guy that wrote them. A fellow team member with passive experience to the code would be a good person, or ideally, a pair design exercise between the person that originally wrote the code and an API designer, with the consumer as a consultant.

Speaking The Same Language

Like a lot of software development, you make good progress when you get your terminology right and understand exactly what you’re trying to produce.  I’ve consistently found that the best way to think of a client facing API is as a orchestrating thin wrapper that summarises, in code, a set of business processes that you wish to expose to the public.

In order to get your API design right, you need to clearly define and agree on the boundaries of the system with both your internal team, and your consumers.  It’s important that you have a clear understanding of the following:

• The responsibility of the calling code
• The responsibility of the API layer
• The responsibility of the internal code the API makes calls to

This might sound like a really simple suggestion but I’ve taken part in countless discussions where people on both sides of the API just “presumed” that either the calling code or the API would perform specific functions (data cleansing, logging etc) when in fact, this confusion had lead to none of the implementers bothering to write the required functionality.  Make sure you know for certain what your API is responsible for doing.

Defining Your API – Tips and Tricks

Defining your API methods (or the “contract” of the API) is the most important thing to get right and there are several vital things to remember.

• Choose Your Names Wisely Using the language of the business
  
  It’s vital that your API methods speak in terms that the caller is going to understand.  Your API should be readable.  If your users go hunting for the documentation every time they want to use a method, then you’re probably doing it wrong.

  Clarity in naming is exceptionally important.  The names of your API methods should succinctly state what action that method call is going to perform.  Don’t fear using long method names, embrace them for clarity.  As a general rule, your pmethods should probably always be in the form DoSomething(object withThis);

  Ensure that when naming methods you reflect business operations in the method names, not the underlying implementation.

  Bad example:     void InsertToTblCustomer(string[] custDataValues);
  Good example:   void AddCustomer(Customer customer);
  Good example:   void DisableAccount(string accountId); 

• Don’t pretend to be a data access layer

  APIs should summarise business operations in a logical and meaningful fashion.  You are not a public facing data access layer and you should never pretend to be.  If your users want raw database access give them read only permissions on some tables and a copy of SQL Management Studio.  So don’t write methods for CRUD operations in your API (unless you’re writing some kind of online file management utility).

  Bad example:     void InsertToTblCustomer(string[] custDataValues);
  Bad example:     void UpdateTblCustomer(string[] custDataValues);
  There are no good examples!

• Avoid leaky abstractions

  This is a fundamental and simple rule – don’t expose your callers to anything that they’re not interested in or won’t understand.  If it’s not important, don’t show it.  Don’t code for things nobody will ever need and don’t require your callers to have intimate knowledge of data types or internal categories in your system.

• Create a data contract between you and the calling code
  
  I’m going to borrow some of the terminology from WCF here because I’ve found it an appropriate label.  Create a Data Contract library for use in your API.  This library should summarise the business process and the outward facing view of your software.  It might contain terminology that doesn’t actually exist in the software itself, but in the business processes that the software models.  Either way, this, and only this, should be the language that the API talks to your callers.

  Where possible, create this data contract in a separate assembly that’s entirely decoupled from your core system and distribute it to people that want to use your API.  This is especially beneficial when using WCF as your clients can generate a service proxy and deal in the same data types that you are in your API code.

  It should be the responsibility of the API layer to marshal the data from your data contract into the domain model of your internal components.

  You data contract should contain every type used to communicate with your API and the object model should be named in a way which is meaningful for the consumers.

  Because your data contract is NOT the object model of your internal components, you’re able to add properties and objects that don’t logically exist in your internal components.  This means that you can perform an operation using some internal component, gather the output in your API layer and then compose the output data in a meaningful way using classes written specifically for the data contract.  This way, by the time the user has access to the output data, it’s in a format and language which they understand.

• Support standards!  Don’t reinvent the wheel!
  
  Here’s a true story; while working with an API, my team was faced with the following API method:
  object Run(string request);

  It was the only method on the API, and “covered up” for around 30 methods all made available through one giant black hole in the side of the system.  Underneath that there was an XML format that the request had to be in in order to call the appropriate method.

  If you’re writing an API, stick to some kind of standards.  Ideally, expose a web service endpoint with an accurate WSDL that people can call or use a simple and obvious REST endpoint.  

  Please, please, please, do not make other developer suffer by rolling your own delivery mechanism.  We have enough of them, don’t confuse people by adding some more.  If you’re going to use a raw sockets connection, supply a calling library and stick to some standard middleware like WCF rather than rolling your own. 

  Thousands of people have spent thousands of man years writing code based on existing techniques, you’re not better than all of them combined.

  Reinventing the wheel is never good for anyone.

• Ensure that the user can retrieve values they’re going to want to modify

  The number of times I’ve used APIs that let me set or update objects without returning the current state of the object is mind blowing.  So always remember: If you’re going to let them set it, let them get it.  Providing an update or “upsert” method without allowing your consumers to query the current state of data in the system is a complete waste of time.

• Supply compiled libraries to work with your API documentation

  This may not always be possible, but it’s a really good idea to supply a sample implementation and compiled binaries with your API that covers the most common scenarios of usage.  Not only does this prevent the consumer from struggling to get to grips with you API, but it allows you to outline and illustrate a set of best practices for usage.  In an ideal world, the user could just use your sample code in their application, so ensure you license the code appropriately.

  This is an exceptionally good way to deal with any authentication your API may require as you have the ability to provide additional helper classes to perform some common tasks (authenticate –> perform action –> logout, for example).

• Keep your implementation clean

  Delegate the API logic to your middleware components / reusable libraries.  Do your best to ensure the API layer doesn’t actually contain the logic required to perform operations, just the logic required to marshal the data from the API format into your internal data structures.  The API should simply orchestrate calls to one or more internal methods because your API should simply be exposing existing functionality.

  If the API is exposing some new, API specific functionality, consider splitting this behaviour into a separate assembly or binary to aid testability.

• Consider dogfooding

  Dogfooding is the act of using the software you’re creating.  I worked on a project where we were developing an order placement system in ASP.net MVC, and as part of the design process we decided that we wanted to have an API that was a first class citizen.  It then dawned on us that in order to produce the API in a way that accurately mirrored the functionality of the website, we should have the website consume the API like any other client would.  The website had it’s own concept of user authentication, and when a user logged in, the web application logged in to the API as the current user.

  Doing this not only ensured that our security model was watertight, but that any additional web functionality would immediately be available to API users because they were actually the same thing.  On top of that, you gain confidence in your own API because you know that it’s called often by your own code, reducing the likelihood of users discovering bugs in your API because it’s not a product you actually use.

• Support atomic operations including rollbacks on failure

  When implementing your API methods, ensure that if an exception occurs or an operation doesn’t complete, the all the changes made by your API call are reversed.  Consider explicitly supporting transaction scopes in your API to let your consumers compose their own “set” of operations.

• Support bulk operations where appropriate

  Building support for bulk operations into your API can often prevent performance issues occurring later when a user tries to, for example, insert 10,000 customers sequentially.  Consider pluralising your methods, so instead of providing an AddCustomer(Customer customer) method, provide only a AddCustomers(List<Customer> customers); method.  Doing this prevents callers from overloading your system by bulking data through your API in unintended ways, allowing you to properly cache required data and cater for these bulk operations.

  This isn’t always appropriate, however I’d always strongly suggest offering pluralised versions of methods that you suspect may be used in bulk, in order to help optimise your API calls and reduce the amount of data being transferred over the wire.

• Logging as a first class citizen

  Don’t wait until somebody asks about API usage to decide to log it.  Build logging into your API wrapper, from the start, at the point of every method call.  It doesn’t need to be fancy, and you can use a number of freely available components to handle these logs and log rotation (consider using log4net or log4j for simple log rotation).

  Log each method call and some summary or identifiable element of the data passed to it.  This’ll help you profile API usage, and identify how data changed in your previously closed system.

• Security as a first class citizen

  Consider the security of your API from the start of the project.  Understand who will have access to your API, which organisations and which individuals.  Do you require roll based security?  Do you need a way to disable API support for specific customers?  Are you transferring data that needs to be encrypted over the wire?

  Beware of over baking your security.  WS-* offers some very robust packet level security features, but if your API doesn’t need them, or is restricted to an internal network, then don’t bog down your implementation in unneeded security.  Beware of making security choices that tie you down to a specific protocol or technology stack – you want to keep your API usable for the consumers.  Do the simplest thing that works.

• Have integration tests!

  Make sure you have integration tests with mocking at the business logic layer. These tests are for your API wrappers, NOT your logic. The logic should be tested independently, you’re just ensuring your API layer, marshalling and method calls operate correctly at this level.

  With any luck, your business logic should already be tested as part of your existing test suite (which you have right?) but if not, ensure the business logic is tested separate from the API code.

  Consider using a TDD or BDD approach to designing your API calls, designing the specification first in the form of some calling code, then write the code required to make your usage examples compile.  This will help you understand exactly what calls the client will have to make for to your API to achieve specific functionality.  These tests can happily double up as regression tests when you make changes to the API.

• Keep it simple!  If all else fails, do what the big boys do.

  Always strive to keep your API simple.  Pretend your the consumer at all times.  If you’re unsure of how to proceed, I’ve always found inspiration, both for what to do and what not to do, from reading the API documentation of some large companies that have widely used APIs.  It’s safe to say that the likes of Amazon, Google and Microsoft have had to put some thought into their API designs.  Beware of trusting their decisions blindly, but liberally borrow anything you, as a consumer, would find pleasing in your API.

I’m not going to try and convince you that by following my advice that your API design will be flawless.  I’m really hoping for a little discussion on this topic as it seems like something that is rarely covered and often “felt out” by the people left to implement APIs for the public.  These are just some lessons I’ve learnt on the way while implementing several public facing APIs.

Want to talk about APIs?  Send me an email!

So the sticky issue of banning gamers based on sexual preference has come up yet again on Kotaku today.

The comments on this thread make for both a good and disheartening read.

From the article:

Perhaps more alarming than someone being banned for their expression of sexual preference was how this particular gamer was treated by the gaming public.

Teresa tells the Consumerist “I was harassed by several players, ‘chased’ to different maps/games to get away from their harassment. They followed me into the games and told all the other players to turn me in because they didn’t want to see that crap or their kids to see that crap.

“Five minutes worth of interaction with the general Xbox Live population will get the point across—they don’t care for “the gays.” But does the policy go too far in one sense, not far enough in another?

There’s a lot of interesting questions in there about her pride deliberately putting her in harms way in a known hostile environment (anyone that’s played any of the AAA mutliplayer games on XBL knows all about the terrible undercurrent of racism and sexism).

Microsoft, however, are taking flak (and rightfully so) for banning the account.  Though I have a certain amount of empathy for their point.  I guess my question to you guys is “is it prudent to be less “proud to be different” in scenarios where you know it would distinctly be to your disadvantage?  What do you think about removing sexuality and politics from gaming in order to “keep everything running smoothly”.

Certainly it’s very hard to police- XBL is largely based on a Peer-to-Peer architecture resulting in a lack of server logs.  As a result the only way to police an incident of this kind is to look for evidence before banning.  Unfortunately the only evidence in the case of this horrible persecuted XBL member is a terms of service violating gamer profile.

Whilst it’s a sorry state of affairs that it comes down to this, if the gamer in questions recount of the griefing she suffered is an example of general XBL behavior (and from experience I wouldn’t doubt it) then actually, keeping a “just keep any mention of sexuality off XBLA profiles for your own good” policy is probably prudent.

It’s not Microsoft’s fault that the world is full of hostile intolerant fucks, and if not stating your sexual preference diffuses a load of awful griefing then it’s probably a good idea.

That said, banning people over it, rather than just notifying and removing the perceived “offensive” content is really off the mark.

Ultimately the question is if it should be left to user choice to make themselves obviously a target, given the known nature of the audience. I think it’s terrible that stating a sexual preference can be the same as putting yourself in harms way (virtual harm, at that), but I wonder if the same person would go into a known homophobic bar and loudly proclaim they were gay/lesbian? Would their natural better judgment take over in that case.

At the end of the day it’s just representative of how you can’t be totally open around the unknown quantity of strangers without unpredictable results. It’s a shame because I think games should be allowed to deal with politics and sexuality and difficult topics, but unless the audience that consumes that material is on the same intellectual level as the material itself, things like this will *always* happen.

The solution probably is that when people run across any kind of intolerance on services like XBL, they should report the abuse.  There are far too many terrible instances on griefing that go unreported, allowing the perpetrators to “escape” while the victims get punished.  Microsoft have spoken about this in the past.

Or maybe it’s just indicative that people who play COD4 online really are drooling intolerant fucks, who knows.

So what do you think?

It’s a tough issue when you think about it and there’s no obvious answer or solution.  Microsoft are getting shot at because they’re the messenger, but it certainly raises some debate on human nature.

Recently I’ve been trying to pay a debt of sorts.  I use the software provided on the fantastic XDA-Developers frequently on my Windows Mobile devices and have been doing for almost three years now.

So currently, if I have a little bit of free time, I’ll nip past their development section and try and fulfil a random request.

Today’s request might come in useful to a little bit of a wider audience so I’ll post it here.  The initial problem is outlined in this thread and the upshot was that someone needed a very simple Http server that could run on Windows Mobile, written in C#’, and was programmable.

What the guy really seemed to be looking for was something that used the Http protocol to return random computed data, so after a tiny bit of googling and a simple MSDN example, I’ve built a really simple web server for Windows Mobile.

Amusingly, my sample doesn’t behave much like a web server at all.  It just tells you what you requested, but it should be enough to get you going in the right direction.  It’s derived from the MSDN example with some additional sugar, and I certainly wouldn’t suggest that it’s got a threading model that’d stand up in a production environment (at a glance it looks like it’d process requests in sequence…) but hopefully it’s useful to somebody looking to produce a simple server, or who is just interested in how Http works.

I’m not really sure if I can accurately call it a webserver, seeing as it doesn’t even support a full set of Http Verbs, but you get the idea.

I’ve not actually bothered compiling this on a mobile device yet (lack of inclination) however seeing as it was explicitly based on a socket programming for Windows Mobile MSDN example, I suspect it’ll work just fine.  No warranty, do what you will with it.

Download Simple C# Http Server for Windows Mobile (Source Code Only) (7kb)

http://www.intellectualhedonism.com/2008/02/13/RunAsRadio44ScottKvetonSharesHisOpenID.aspx
Richard and Greg talk to Scott Kveton about OpenID. OpenID is a single sign-on solution that could very well make the classic username and password obsolete. This is a fast half hour - you’ll find yourself wanting to listen again!

Coincidently so soon after writing my last post, something I’ve been listening to whilst coding today, really interesting half hour show on openId and unification of functionality.

I’ve spent a little time in the past evening writing a .NET/ASP OpenId provider based on some open source code build on top of the JanRain librarys. Their sample was simple and forms authentication based, and I’ve been expanding it to work in a multiuser environment using flat files to store XML user identities behind the scenes. When it’s done I’ll submit the changes back to the source / open it, but it’s more of a fork of the orignial project (which itself appears to be based on just about everything ever).

This post was originally going to be a response to my friend Stevens post on the same topic “Isn’t it odd..” (Steven Westwell on social networking) so I’d read that first in case I’ve missed something in the overwrite to a post here.  His response turned in to an essay, and so has my own.

I’m eternally divided on the topic of social and collaborative software and the workplace, and even in leisure time.

The experience people currently have with social networking is really zeitgeist and faddish behaviour, people check their facebook over personal email at the moment because it’s new and exciting, it’s something they’ve not had previously, though month after month since about November 2007 facebook and to an even greater extent myspace have seen a steady reduction in both traffic and user activity.  I’m convinced that today’s social networking, from the walled garden of the MSN network, sorry, Facebook ( ;) ), to the focusless indirection of twitter aren’t actually setting the mould for community communication, they’re a “web two point oh-ey” version of phpBB and usenet more than anything.

I’d actually say spam currently poses a greater threat to traditional email than social networking could possibly manage.

That said, it’s correct to identify user driven communities as the direction things are and should be moving, but I see the success in this is directly linked to the maturity of unified online identities, rather than scattered ecosystems.  It’ll take a killer app in the form of Windows Live (even based on the success of XboxLive), or OpenId (I prey for the latter but I’m reasonably certain it’ll be the former) to act as the overall identity broker before people start to establish solid content driven identities on the Internet.

The biggest barrier obstructing the path of social networking becoming as ubiquitous as email is the walled gardens it builds around itself.

The Mecca of the “web 2.0″ concept really is distributed social networking, driven by network events to publish stories, events, “friend requests” and other unspecified interactions (RRS and Atom feeds serving as the first implementation of this “half-push” technology).  A user should control their own identity, have the right to elect the authority who controls their identity (see OpenId), or control their own.  Until the social networking sites stop trying to wall in their users (like AOL, ICQ, MSN, Yahoo, local BBS’, The Microsoft Network, Facebook, Myspace, etc before them) we’ll spend a few years in the wilderness of disparate online identities and large communities achieving moderate success before vanishing.

Social networking platforms need to concentrate on adding value to the user experience, rather than trying to consistently reinvent the wheel.  If they were software project propositions this kind of behaviour alone would doom them to failure.  Innovation should be celebrated, and developers certainly shouldn’t have to produce “yet another user management system” with all the features of their direct predecessors, just a little better, or a little more shiny.  The bread and butter of social interaction should be standardised protocols for communication between unrelated innovative “value added” sites.

The kind of interactions suggested, distributed file storage, notifications and calendars exist in products like SharePoint, and mediawiki, iCal and traditional email, but until they’re tied together correctly they’re doomed to be noise.  I really believe that until collaboration technology matures to be platform independent that it’ll struggle to permanently compete and shape networking on a broader scale.  I work entirely in the Microsoft ecosystem and I still feel it’s very important that Microsoft should not tell you who you are online, nor should Facebook, Blogger nor Apple.

Interestingly there’s a different set of challenges facing the adoption of technologies like instant messaging in the workplace, and they’re all entirely human.  Staff training and trust with communication mediums that can be painted as time wasting will always struggle to garner acceptance.  That said, I’ve used instant messaging for years in the workplace, but I’d imagine most technical companies will be ahead of the curve in this respect.  It’s interesting if you look at studies of how wasteful people make checking email on a daily basis, after you account for disruption in flow and response times.  It’s actually (at least in my experience) far more efficient to have email delivered on a pull basis, in intervals measured in hours.  People get more done when they only check their mail four times a day.

I’m really really excited about the future of social networking and its place as the main content driver of “web 3.0″, but I like to imagine it more in the vein of the Ainsible networks of Orson Scott Cards “Enders Game” than the cheap message board hacks of myspace and their application ecosystem.  I’m striving to unify my online identity and I’d implore anyone else to do the same.