Archive

Archive for the ‘Programming’ Category

Automatic Html Encoding in ASP.NET 4.0

April 13th, 2010

I spent today at Microsofts Techdays Visual Studio 2010 launch event in London.  Lots of interesting stuff, covered in depth all over the internet (just Google, or bing, if you’re a sadist “changes from .NET 3.5 to .NET 4.0″).  A tiny thing that caught my attention for it’s pure utility to the masses is a new operator in ASP.NET 4.0 that deals with Html Encoding of data implicity.

You’ll likely be familiar (especially if you’re working with ASP.NET MVC) with the <%= notation for referencing properties in the context of the ASP page.  In ASP.NET 4.0 this has been joined by <%: This addition automagically HtmlEncodes any content between the opening and closing tags to prevent repetitive tag soup of <%=HttpUtility.HtmlEncode(myProperty)%> all over your views, and removes the temptation to push HtmlEncoding into your Controller or Model, two places where encoding really shouldn’t be a concern.

Just struck me as a nice little change that’ll make everyday life that little bit easier for a majority of web developers. Sure it’s not quite the new Xaml designer (which is a work of beauty) or Intellitrace (which looked like a total game changer for retrospective debugging), but you know, it’s the little things that count.

ASP.NET MVC View Engine That Supports View Path Inheritance

January 19th, 2010

I was working on a small MVC project where we were dealing with Inherited controllers (SomeController was inherited by SomeMoreSpecificController) and we decided that it’d be nice to have a similar hierarchy of sharing and inheritance at the View level.

Unfortunately, out of the box, ASP.net MVC looks in two default locations for your views and partials by convention.  The first is /Views/ControllerName/ViewName.aspx, the second is /Views/Shared/ViewName.aspx.  We wanted to allow SomeController to have it’s own set of more generic views, that could later be overridden in special cases by the views provided by SomeMoreSpecificController.

In order to do this in ASP.net MVC, you need to override the default view engine to change the location that the runtime looks for your views.

“Here’s one I made earlier”.

Using this ViewEngine, if you call an action method on SomeMoreSpecificController it’ll first check /Views/SomeMoreSpecificController/…, then /Views/SomeController/… (the base class), then finally /Views/Shared, allowing you a little more control over the organisation of your views.

using System.Collections.Generic;
using System.Web.Mvc;
using System;
using System.Globalization;
using System.Linq;

namespace MyMvc.Mvc
{

    public class InheranceViewEngine : WebFormViewEngine
    {
        private const string CacheKeyFormat = ":ViewCacheEntry:{0}:{1}:{2}:{3}:";
        private const string CacheKeyPrefixMaster = "Master";
        private const string CacheKeyPrefixView = "View";
        private static readonly List<string> EmptyLocations= new List<string>();

        public InheranceViewEngine()
        {
            ViewLocationFormats = new[]
                                  {
                                      "~/Views/{1}/{0}.aspx", "~/Views/{1}/{0}.ascx", "~/Views/Shared/{0}.aspx",
                                      "~/Views/Shared/{0}.ascx"
                                  };
        }

        public override ViewEngineResult FindView(ControllerContext controllerContext, string viewName, string masterName, bool useCache)
        {
            if (controllerContext == null)
            {
                throw new ArgumentNullException("controllerContext");
            }
            if (String.IsNullOrEmpty(viewName))
            {
                throw new ArgumentException("viewName");
            }

            List<string> viewLocationsSearched;
            List<string> masterLocationsSearched;

            string controllerName = controllerContext.RouteData.GetRequiredString("controller");
            string viewPath = GetPath(controllerContext, ViewLocationFormats, viewName, controllerName, CacheKeyPrefixView, useCache, out viewLocationsSearched);
            string masterPath = GetPath(controllerContext, MasterLocationFormats, masterName, controllerName, CacheKeyPrefixMaster, useCache, out masterLocationsSearched);

            if (String.IsNullOrEmpty(viewPath) || (String.IsNullOrEmpty(masterPath) && !String.IsNullOrEmpty(masterName)))
            {
                return new ViewEngineResult(viewLocationsSearched.Union(masterLocationsSearched));
            }

            return new ViewEngineResult(CreateView(controllerContext, viewPath, masterPath), this);
        }

        public override ViewEngineResult FindPartialView(ControllerContext controllerContext, string partialViewName, bool useCache)
        {
            List<string> strArray;
            if (controllerContext == null) { throw new ArgumentNullException("controllerContext"); }
            if (string.IsNullOrEmpty(partialViewName)) { throw new ArgumentException("Partial View Name is null or empty.", "partialViewName"); }
            string requiredString = controllerContext.RouteData.GetRequiredString("controller");
            string str2 = GetPath(controllerContext, PartialViewLocationFormats, partialViewName, requiredString, "Partial", useCache, out strArray);
            if (string.IsNullOrEmpty(str2))
            {
                return new ViewEngineResult(strArray);
            }

            return new ViewEngineResult(CreatePartialView(controllerContext, str2), this);
        }

        private string GetPath(ControllerContext controllerContext, string[] locations, string name, string controllerName, string cacheKeyPrefix, bool useCache, out List<string> searchedLocations)
        {
            searchedLocations = EmptyLocations;

            if (String.IsNullOrEmpty(name))
            {
                return String.Empty;
            }

            if (locations == null || locations.Length == 0)
            {
                throw new InvalidOperationException();
            }

            bool nameRepresentsPath = IsSpecificPath(name);
            string cacheKey = CreateCacheKey(cacheKeyPrefix, name, (nameRepresentsPath) ? String.Empty : controllerName);

            if (useCache)
            {
                string result = ViewLocationCache.GetViewLocation(controllerContext.HttpContext, cacheKey);

                if (result != null)
                {
                    return result;
                }
            }

            if (nameRepresentsPath)
            {
                return GetPathFromSpecificName(controllerContext, name, cacheKey, ref searchedLocations);
            }
            return GetPathFromGeneralName(controllerContext, locations, name, controllerName, cacheKey, ref searchedLocations);
        }

        private string GetPathFromGeneralName(ControllerContext controllerContext, string[] locations, string name, string controllerName, string cacheKey, ref List<string> searchedLocations)
        {
            string result = String.Empty;
            searchedLocations = new List<string>();

            for (int i = 0; i < locations.Length; i++)
            {
                string virtualPath = String.Format(CultureInfo.InvariantCulture, locations[i], name, controllerName);
                if (FileExists(controllerContext, virtualPath))
                {
                    searchedLocations = EmptyLocations;
                    result = virtualPath;
                    ViewLocationCache.InsertViewLocation(controllerContext.HttpContext, cacheKey, result);
                    return result;
                }
                searchedLocations.Add(virtualPath);
            }

            return GetPathFromGeneralNameOfBaseTypes(controllerContext.Controller.GetType(), locations, name, controllerContext, cacheKey, result, ref searchedLocations);
        }

        private string GetPathFromGeneralNameOfBaseTypes(Type descendantType, string[] locations, string name, ControllerContext controllerContext, string cacheKey, string result, ref List<string> searchedLocations)
        {
            Type baseControllerType = descendantType;
            if (baseControllerType == null
                || !baseControllerType.Name.Contains("Controller")
                || baseControllerType.Name == "Controller")
            {
                return result;
            }

            for (int i = 0;i < locations.Length;i++)
            {
                string baseControllerName = baseControllerType.Name.Replace("Controller", "");
                string virtualPath = String.Format(CultureInfo.InvariantCulture, locations[i], name, baseControllerName);

                if (!string.IsNullOrEmpty(virtualPath) &&
                    FileExists(controllerContext, virtualPath))
                {
                    searchedLocations = EmptyLocations;
                    result = virtualPath;
                    ViewLocationCache.InsertViewLocation(controllerContext.HttpContext, cacheKey, result);
                    return result;
                }

                searchedLocations.Add(virtualPath);
            }

            return GetPathFromGeneralNameOfBaseTypes(baseControllerType.BaseType, locations, name, controllerContext,
                                                     cacheKey, result, ref searchedLocations);
        }

        private string CreateCacheKey(string prefix, string name, string controllerName)
        {
            return String.Format(CultureInfo.InvariantCulture, CacheKeyFormat,
                                 GetType().AssemblyQualifiedName, prefix, name, controllerName);
        }

        private string GetPathFromSpecificName(ControllerContext controllerContext, string name, string cacheKey, ref List<string> searchedLocations)
        {
            string result = name;

            if (!FileExists(controllerContext, name))
            {
                result = String.Empty;
                searchedLocations = new List<string>{ name };
            }

            ViewLocationCache.InsertViewLocation(controllerContext.HttpContext, cacheKey, result);
            return result;
        }

        private static bool IsSpecificPath(string name)
        {
            char c = name[0];
            return (c == ‘~’ || c == ‘/’);
        }

    }
}

Reusable Editable Fields for ASP.net MVC Using jQuery

October 8th, 2009

A friend recently asked me about editing items inline using ASP.net MVC, the kind of thing that was auto magically wired up with post backs in “old fashioned” asp.net so I’ve whipped up a small example showing how you can use jQuery to declaratively set up interactive field editing with a sprinkling of Ajax and JSON.

I’m basing this example on the default ASP.net MVC starter project for brevity (download attached) but here’s an overview:

First you need to set up an Action method (or multiple action methods) on your controller to accept the modification of data.  In my example I’ve added an unimaginative method called “SetField” to the HomeController that looks like this:

public ActionResult SetField(string fieldName, string fieldValue)
{
    var response = Json(fieldValue);
    return response;
}

As you can see, it doesn’t do very much (useful implementation left to the reader) but it accepts the parameters of a field name, and a field value.  You’ll need to roll your own validation and sanity checking here.  It then returns the fieldValue using the MVC Json helper object, as a Json object.  In a real world example, you’d want to call and update in this method.

Now, in the view, jQuery does most of the hard work.

First I added a few CSS classes to the header on the master page (for the sake of example):

<style type="text/css">
    .editableItem { display: block; }
    .fieldViewer { display: block; }
    .fieldEditor { display: none; }
    .editableItemCancel { display: block; }
    .editableItemBox { display: block; }
</style>

I then added an example to the view that looked like this:

<div class="editableItem" id="editable_FieldName">
    <div class="fieldViewer">Click me to edit me!</div>
    <div class="fieldEditor"><input class="editableItemBox" type="text"/><span class="editableItemCancel">cancel</span></div>
</div>

With this HTML I set up some conventions that I’ll rely on when using jQuery.  Firstly, every editable item should use the class “editableItem” and have the id “editable_FieldName”.  I use the class in a jQuery selector and the Id to establish which field is being edited.  Inside the editableItem should be a fieldViewer, containing the current data, and a fieldEditor, which is hidden by default, and contains some kind of editable controller and a cancel button.  You could insert these elements at runtime if you wished, but in order to keep the example simple I’ve declared them in the HTML.

Next I added some jQuery… The jQuery defines some Javascript behaviour associated with the classes used in the HTML, this way, the mark-up can be reused to edit multiple fields rather than being keyed to the Id of a specific field.

<script src="/Scripts/jquery-1.3.2.js" type="text/javascript"></script>
<script type="text/javascript">
    jQuery(document).ready(function() {

        $(".editableItem .fieldViewer").click(function() {
            var parentId = $(this).parent().attr("id");
            $(’#’ + parentId + " .fieldEditor .editableItemBox").val($(’#’ + parentId + " .fieldViewer").text());
            $(’#’ + parentId + " .fieldViewer").toggle();
            $(’#’ + parentId + " .fieldEditor").toggle();
        });

        $(".editableItem .fieldEditor .editableItemCancel").click(function() {
            var parentId = $(this).parent().parent().attr("id");
            $(’#’ + parentId + " .fieldViewer").toggle();
            $(’#’ + parentId + " .fieldEditor").toggle();
        });

        $(’.editableItem .editableItemBox’).keypress(function(e) {
            if (e.which == 13) {
                var parentId = $(this).parent().parent().attr("id");
                var fieldName = parentId.replace(/editable_/, "");

                $.post(’/Home/SetField’,
                {
                    fieldName: fieldName, fieldValue: $(’#’ + parentId + " .editableItemBox").val()
                },
                    function(data) {
                        $(’#’ + parentId + " .fieldViewer").text(eval(’(’ + data + ‘)’));
                        $(’#’ + parentId + " .fieldViewer").toggle();
                        $(’#’ + parentId + " .fieldEditor").toggle();
                    })
            }
        });

    });
</script>

Quite simply, if you click the editable field, it toggles into a textbox.  If you hit enter on the textbox, the value is posted to the previously defined Action on the Controller.  If you hit cancel, the display is toggled back.

I’d not recommend copy and pasting this exact example into a production system, but hopefully it’ll guide you through a simple scenario.  You can use a similar technique to add all sorts of little Ajax tricks (auto-suggest, lookups, dynamic menus) to your ASP.net MVC site using jQuery and Json (both of which are included in the core asp.net MVC framework).

Download the example solution here

Creating a WCF Proxy to talk to Magento

October 5th, 2009

I got a message from a friend who was struggling to do an integration piece with the Magento eCommerce Platform using the SOAP endpoint available at http://yourserver.co.uk/api/v2_soap?wsdl.

He brought an interesting problem to me, namely that the WCF svcutil executable (and built in Visual Studio 2008) was failing to generate any proxy code when supplied with a seemingly valid wsdl.

I did a quick test and managed to instantly reproduce the error.

Weirder still, when using the “old” .Net 2.0 add web reference method rather than the .Net 3.0+ “Add Service Reference”, the framework managed to create a non-WCF reference just fine.

Dropping down to the command line I saw some unusual messages being displayed by svcutil.exe:

c:\Program Files\Microsoft SDKs\Windows\v6.0A\Bin>SvcUtil.exe http://yourserver.co.uk/api/v2_soap?wsdl
Attempting to download metadata from ‘http://yourserver.co.uk/api/v2_soap?wsdl’ using WS-Metadata Exchange or DISCO.

(Lots of error messages here…)

Error: Cannot import wsdl:portType
Detail: An exception was thrown while running a WSDL import extension: System.Se
rviceModel.Description.XmlSerializerMessageContractImporter
Error: The ‘ ‘ character, hexadecimal value 0×20, cannot be included in a name.
Parameter name: name
XPath to Error Source: //wsdl:definitions[@targetNamespace='urn:Magento']/wsdl:p
ortType[@name='Mage_Api_Model_Server_V2_HandlerPortType']

Error: Cannot import wsdl:binding
Detail: There was an error importing a wsdl:portType that the wsdl:binding is de
pendent on.
XPath to wsdl:portType: //wsdl:definitions[@targetNamespace='urn:Magento']/wsdl:
portType[@name='Mage_Api_Model_Server_V2_HandlerPortType']
XPath to Error Source: //wsdl:definitions[@targetNamespace='urn:Magento']/wsdl:b
inding[@name='Mage_Api_Model_Server_V2_HandlerBinding']

Error: Cannot import wsdl:port
Detail: There was an error importing a wsdl:binding that the wsdl:port is depend
ent on.
XPath to wsdl:binding: //wsdl:definitions[@targetNamespace='urn:Magento']/wsdl:b
inding[@name='Mage_Api_Model_Server_V2_HandlerBinding']
XPath to Error Source: //wsdl:definitions[@targetNamespace='urn:Magento']/wsdl:s
ervice[@name='MagentoService']/wsdl:port[@name='Mage_Api_Model_Server_V2_Handler
Port']

Generating files…
Warning: No code was generated.
If you were trying to generate a client, this could be because the metadata docu
ments did not contain any valid contracts or services
or because all contracts/services were discovered to exist in /reference assembl
ies. Verify that you passed all the metadata documents to the tool.

Warning: If you would like to generate data contracts from schemas make sure to
use the /dataContractOnly option.

c:\Program Files\Microsoft SDKs\Windows\v6.0A\Bin>

So I did a little digging through the WSDL and found an undocumented bug in Magneto’s schema.

First I saved a local copy of the WSDL and used visual studio to reformat the document into some sort of readable state, then I had to make a few corrections to the WSDL to allow SvcUtil to correctly parse the malformed document.

Change 1:  Replace a badly encoded apostrophe – I removed the “’s” from the following operation definition…

<operation name="customerGroupList">
<documentation>Retrieve customer’s groups</documentation>
<input message="typens:customerGroupListRequest"/>
<output message="typens:customerGroupListResponse"/>
</operation>

Change 2: Replace a trailing space in an operation name

<message name="catalogProductGetSpecialPriceRequest">
  <part name="sessionId" type="xsd:string"></part>
  <part name="product" type="xsd:string"></part>
  <part name="storeView " type="xsd:string"></part>
</message>

If you look carefully at the above message definition, you’ll notice that name=”storeView “ contains a space, making the wsdl invalid.  Remove the space so it reads “storeView”.

With these two errors corrected, SvcUtil had no problem generating an appropriate WCF proxy from the corrected wsdl file.

Magneto will hopefully fix this error in the WSDL, but until this time, it’s probably quite safe to follow these steps to generate your own proxy.

To reproduce:

  • Go to http://yourserver.co.uk/api/v2_soap?wsdl and save the contents of your file to the local disk (c:\test\main.wsdl)
  • Open the file in visual studio, and reformat the document for readability (CTRL+K, CTRL+D).
  • Remove the apostrophe from the documentation tag for the customerGroupList operation.
  • Remove the space after the name=”storeView “ in the catalogProductGetSpecialPriceRequest message definition.
  • Open a command prompt and enter
  • c:\test>c:\Program Files\Microsoft SDKs\Windows\v6.0A\Bin\svcutil.exe main.wsdl
  • SvcUtil will produce two files, Magento.cs (your WCF proxy) and output.config, your endpoint configuration.

Designing Client Facing APIs – Best Practices

September 30th, 2009

With the popularity of service oriented architectures and other buzz phrases related to software as service, good API design has become a significant selling point for any software platform in the past 5-10 years.  People make purchasing decisions based on how easy it is to interoperate with your applications and code and as such the number of client / public facing APIs attached to software has skyrocketed.  I’d like to believe the days of dropping strategic text files in directories to trigger some action or another in an application are behind us.

In this article I’m going to talk about the following things

  • Why you should choose your method names carefully, and what to call
  • Why pretending to be a data access layer is a terrible thing for an API to do
  • Talk about the dangers of leaky abstractions in an API
  • Explain the benefits of creating a data contract between you and the calling code
  • Explain why it’s vital to support standards
  • Make sure that your users can retrieve values they’re going to want to modify
  • Suggest supplying compiled libraries alongside your API documentation
  • Explain why it’s important to keep your API implementation clean
  • Talk about the benefits of dogfooding your API
  • Consider supporting atomic operations including rollbacks on failure
  • Discuss bulk operations
  • Try and convince you that both logging and security should be first class citizens
  • Beg you to maintain integration tests and most of all to keep it simple!

Your API Sucks

You’ve probably used an API and there’s a good chance you’ve had to write one.  This probably won’t surprise you; most APIs suck.  They’re horrible to use and built around illogical leaky abstractions that leave you flicking through huge wads of documentation just to make the most rudimentary feature work.

About 18 months ago, after a year of struggling with a broken third party API that almost brought a business to it’s knees by placing significant roadblocks in front of in house development, I was part of a team tasked with designing our own client facing API.  With no desire to expose other developers to the cruel and unusual punishments of software design we’d had to endure, we came to the conclusion that it was really important that we god this piece of the system right.  People say first impressions are everything, and your API design can make or break the faith other developers have in your ability to produce software.  Show somebody a shitty API and they’ll perhaps correctly assume the rest of your code sucks too.

The Best Man For The Job

There’s a bit of a trend that I’ve noticed with some of the worst APIs I’ve worked with: they seem to be designed by the wrong people.  The wrong people to design an API are 1) the guy that wrote the internal code to do the job the API is providing access to and 2) the consumer of the API. 

The guy that wrote the code that the API is calling under the hood will be inherently slanted to implement an API which exposes this functionality and will have a predisposition to creating a leaky abstraction.  This is especially bad for the consumer APIs designed by the internal implementer tend to assume the consumer knows far more than he really does, or has access to internal data that in reality, he doesn’t.

Conversely, an API designed by the consumer of the API will have a tendency towards solving problems that are not the concern of the API itself.  The consumer will, either accidently or by intention, attempt to offload some of the work that should be the responsibility of the calling code into the API.

Ideally, the person that’s writing the API will have knowledge of the system internals, but not be the guy that wrote them. A fellow team member with passive experience to the code would be a good person, or ideally, a pair design exercise between the person that originally wrote the code and an API designer, with the consumer as a consultant.

Speaking The Same Language

Like a lot of software development, you make good progress when you get your terminology right and understand exactly what you’re trying to produce.  I’ve consistently found that the best way to think of a client facing API is as a orchestrating thin wrapper that summarises, in code, a set of business processes that you wish to expose to the public.

In order to get your API design right, you need to clearly define and agree on the boundaries of the system with both your internal team, and your consumers.  It’s important that you have a clear understanding of the following:

  • The responsibility of the calling code
  • The responsibility of the API layer
  • The responsibility of the internal code the API makes calls to

This might sound like a really simple suggestion but I’ve taken part in countless discussions where people on both sides of the API just “presumed” that either the calling code or the API would perform specific functions (data cleansing, logging etc) when in fact, this confusion had lead to none of the implementers bothering to write the required functionality.  Make sure you know for certain what your API is responsible for doing.

Defining Your API – Tips and Tricks

Defining your API methods (or the “contract” of the API) is the most important thing to get right and there are several vital things to remember.

  • Choose Your Names Wisely Using the language of the business

    It’s vital that your API methods speak in terms that the caller is going to understand.  Your API should be readable.  If your users go hunting for the documentation every time they want to use a method, then you’re probably doing it wrong.

    Clarity in naming is exceptionally important.  The names of your API methods should succinctly state what action that method call is going to perform.  Don’t fear using long method names, embrace them for clarity.  As a general rule, your pmethods should probably always be in the form DoSomething(object withThis);

    Ensure that when naming methods you reflect business operations in the method names, not the underlying implementation.

    Bad example:     void InsertToTblCustomer(string[] custDataValues);
    Good example:   void AddCustomer(Customer customer);
    Good example:   void DisableAccount(string accountId); 

  • Don’t pretend to be a data access layer

    APIs should summarise business operations in a logical and meaningful fashion.  You are not a public facing data access layer and you should never pretend to be.  If your users want raw database access give them read only permissions on some tables and a copy of SQL Management Studio.  So don’t write methods for CRUD operations in your API (unless you’re writing some kind of online file management utility).

    Bad example:     void InsertToTblCustomer(string[] custDataValues);
    Bad example:     void UpdateTblCustomer(string[] custDataValues);
    There are no good examples!

  • Avoid leaky abstractions

    This is a fundamental and simple rule – don’t expose your callers to anything that they’re not interested in or won’t understand.  If it’s not important, don’t show it.  Don’t code for things nobody will ever need and don’t require your callers to have intimate knowledge of data types or internal categories in your system.

  • Create a data contract between you and the calling code

    I’m going to borrow some of the terminology from WCF here because I’ve found it an appropriate label.  Create a Data Contract library for use in your API.  This library should summarise the business process and the outward facing view of your software.  It might contain terminology that doesn’t actually exist in the software itself, but in the business processes that the software models.  Either way, this, and only this, should be the language that the API talks to your callers.

    Where possible, create this data contract in a separate assembly that’s entirely decoupled from your core system and distribute it to people that want to use your API.  This is especially beneficial when using WCF as your clients can generate a service proxy and deal in the same data types that you are in your API code.

    It should be the responsibility of the API layer to marshal the data from your data contract into the domain model of your internal components.

    You data contract should contain every type used to communicate with your API and the object model should be named in a way which is meaningful for the consumers.

    Because your data contract is NOT the object model of your internal components, you’re able to add properties and objects that don’t logically exist in your internal components.  This means that you can perform an operation using some internal component, gather the output in your API layer and then compose the output data in a meaningful way using classes written specifically for the data contract.  This way, by the time the user has access to the output data, it’s in a format and language which they understand.

  • Support standards!  Don’t reinvent the wheel!

    Here’s a true story; while working with an API, my team was faced with the following API method:
    object Run(string request);

    It was the only method on the API, and “covered up” for around 30 methods all made available through one giant black hole in the side of the system.  Underneath that there was an XML format that the request had to be in in order to call the appropriate method.

    If you’re writing an API, stick to some kind of standards.  Ideally, expose a web service endpoint with an accurate WSDL that people can call or use a simple and obvious REST endpoint.  

    Please, please, please, do not make other developer suffer by rolling your own delivery mechanism.  We have enough of them, don’t confuse people by adding some more.  If you’re going to use a raw sockets connection, supply a calling library and stick to some standard middleware like WCF rather than rolling your own. 

    Thousands of people have spent thousands of man years writing code based on existing techniques, you’re not better than all of them combined.

    Reinventing the wheel is never good for anyone.

  • Ensure that the user can retrieve values they’re going to want to modify

    The number of times I’ve used APIs that let me set or update objects without returning the current state of the object is mind blowing.  So always remember: If you’re going to let them set it, let them get it.  Providing an update or “upsert” method without allowing your consumers to query the current state of data in the system is a complete waste of time.

  • Supply compiled libraries to work with your API documentation

    This may not always be possible, but it’s a really good idea to supply a sample implementation and compiled binaries with your API that covers the most common scenarios of usage.  Not only does this prevent the consumer from struggling to get to grips with you API, but it allows you to outline and illustrate a set of best practices for usage.  In an ideal world, the user could just use your sample code in their application, so ensure you license the code appropriately.

    This is an exceptionally good way to deal with any authentication your API may require as you have the ability to provide additional helper classes to perform some common tasks (authenticate –> perform action –> logout, for example).

  • Keep your implementation clean

    Delegate the API logic to your middleware components / reusable libraries.  Do your best to ensure the API layer doesn’t actually contain the logic required to perform operations, just the logic required to marshal the data from the API format into your internal data structures.  The API should simply orchestrate calls to one or more internal methods because your API should simply be exposing existing functionality.

    If the API is exposing some new, API specific functionality, consider splitting this behaviour into a separate assembly or binary to aid testability.

  • Consider dogfooding

    Dogfooding is the act of using the software you’re creating.  I worked on a project where we were developing an order placement system in ASP.net MVC, and as part of the design process we decided that we wanted to have an API that was a first class citizen.  It then dawned on us that in order to produce the API in a way that accurately mirrored the functionality of the website, we should have the website consume the API like any other client would.  The website had it’s own concept of user authentication, and when a user logged in, the web application logged in to the API as the current user.

    Doing this not only ensured that our security model was watertight, but that any additional web functionality would immediately be available to API users because they were actually the same thing.  On top of that, you gain confidence in your own API because you know that it’s called often by your own code, reducing the likelihood of users discovering bugs in your API because it’s not a product you actually use.

  • Support atomic operations including rollbacks on failure

    When implementing your API methods, ensure that if an exception occurs or an operation doesn’t complete, the all the changes made by your API call are reversed.  Consider explicitly supporting transaction scopes in your API to let your consumers compose their own “set” of operations.

  • Support bulk operations where appropriate

    Building support for bulk operations into your API can often prevent performance issues occurring later when a user tries to, for example, insert 10,000 customers sequentially.  Consider pluralising your methods, so instead of providing an AddCustomer(Customer customer) method, provide only a AddCustomers(List<Customer> customers); method.  Doing this prevents callers from overloading your system by bulking data through your API in unintended ways, allowing you to properly cache required data and cater for these bulk operations.

    This isn’t always appropriate, however I’d always strongly suggest offering pluralised versions of methods that you suspect may be used in bulk, in order to help optimise your API calls and reduce the amount of data being transferred over the wire.

  • Logging as a first class citizen

    Don’t wait until somebody asks about API usage to decide to log it.  Build logging into your API wrapper, from the start, at the point of every method call.  It doesn’t need to be fancy, and you can use a number of freely available components to handle these logs and log rotation (consider using log4net or log4j for simple log rotation).

    Log each method call and some summary or identifiable element of the data passed to it.  This’ll help you profile API usage, and identify how data changed in your previously closed system.

  • Security as a first class citizen

    Consider the security of your API from the start of the project.  Understand who will have access to your API, which organisations and which individuals.  Do you require roll based security?  Do you need a way to disable API support for specific customers?  Are you transferring data that needs to be encrypted over the wire?

    Beware of over baking your security.  WS-* offers some very robust packet level security features, but if your API doesn’t need them, or is restricted to an internal network, then don’t bog down your implementation in unneeded security.  Beware of making security choices that tie you down to a specific protocol or technology stack – you want to keep your API usable for the consumers.  Do the simplest thing that works.

  • Have integration tests!

    Make sure you have integration tests with mocking at the business logic layer. These tests are for your API wrappers, NOT your logic. The logic should be tested independently, you’re just ensuring your API layer, marshalling and method calls operate correctly at this level.

    With any luck, your business logic should already be tested as part of your existing test suite (which you have right?) but if not, ensure the business logic is tested separate from the API code.

    Consider using a TDD or BDD approach to designing your API calls, designing the specification first in the form of some calling code, then write the code required to make your usage examples compile.  This will help you understand exactly what calls the client will have to make for to your API to achieve specific functionality.  These tests can happily double up as regression tests when you make changes to the API.

  • Keep it simple!  If all else fails, do what the big boys do.

    Always strive to keep your API simple.  Pretend your the consumer at all times.  If you’re unsure of how to proceed, I’ve always found inspiration, both for what to do and what not to do, from reading the API documentation of some large companies that have widely used APIs.  It’s safe to say that the likes of Amazon, Google and Microsoft have had to put some thought into their API designs.  Beware of trusting their decisions blindly, but liberally borrow anything you, as a consumer, would find pleasing in your API.

I’m not going to try and convince you that by following my advice that your API design will be flawless.  I’m really hoping for a little discussion on this topic as it seems like something that is rarely covered and often “felt out” by the people left to implement APIs for the public.  These are just some lessons I’ve learnt on the way while implementing several public facing APIs.

Want to talk about APIs?  Send me an email!

Localizing ASP.Net MVC Pages without the need to RunAt=”server”

August 20th, 2009

A common complaint when faced with localizing ASP.net MVC pages is that littering your code with tonnes of runat=”server” tags breaks the “purity” of the MVC model.  Regardless of how meaningful that debate is, there is a way to achieve globalisation without server controls.

I’m going to walk you through a sample implementation which should hopefully make this clearer.  As a standard disclaimer, none of this code has been tested in a production environment and I wouldn’t advise implementing it blindly.

I’m going to attempt to play this out mostly in screenshots…

The Idea

  • The required language is stored in a database / session / extrapolated from the Url route information
  • Your view pages derive from the type TranslatableViewPage
  • This page adds support for a LanguageCode property that you can make use of inside the view along with adding a class implementing ITranslator, a class that provides hooks into a translation database.
  • Your controller derives from the type TranslatableController.
    • This controller add a method called ViewInLanguage(string languageCode) that you use instead of View() or View(model) to return your Asp.net MVC view.
  • At application start up in the Global.asmx file, you register a default language code for failure conditions, and specify the implementation of ITranslator you wish to use to acquire localized strings.
  • If your TranslatableViewPage when rendering your output, you simple need to use the embedded <%=Translator.Translate(propertyName) %> method call to load a translated string while the page output is rendering.
  • Class Layout

    The following is built upon the standard Asp.net MVC starter project for the sake of illustration.

    image

    Implementation

    First, configure the translation settings and register your translator…

    image

    You need to configure a fail-safe default language code and a type that implements ITranslator.  This translator will be responsible for doing the heavy lifting.  Ideally we’d add some inversion of control here to allow you to define the translator implementation at configuration time, but that’s outside of the scope of this example.  I’ve implemented a very crude resource .resx translator for the example.

    Next, make your controllers inherit from TranslatableController

    image

    Once inherited, switch from using the View(); method to ViewInLanguage() passing in your desired language code (gathered from the user session / database / Url route).

    Then set your view type (or derive your view from) TranslatableViewPage

    image

    Once your view is inheriting from TranslatableViewPage, you’ll have access to an instance of your specified Translator and the LanguageCode inside the view which you can use in a manner similar to the built in HtmlHelper class to access your translated strings.

    How It Works

    The TranslatableController provides you with ViewInLanguage and when called generates the standard MVC ActionResult.  This ActionResult will be a ViewResult, which is then wrapped in a LocalizedViewResult wrapper class adding a LanguageCode.

    When ExecuteResult is called on the LocalizedViewResult to render the view, the LanguageCode is placed in the TempData array in the TranslatableViewPage.  Then, when the OnInit(EventArgs e) method is called on the TranslatableViewPage, this LanguageCode is extracted and placed in the LanguageCode property on the page.  In addition to this the page provides a constructed instance of ITranslator which you can use in your views to source translated data as the page is rendered.

    Source Code

    Download Here

    Xml Comment Hell – A Software anti-pattern

    July 6th, 2009

    One of the most valued practices in software development is brevity.  Writing code is bad.  When you write code you create bugs, and creating bugs is bad.  The solution?  Don’t write much code.  Commenting your code however, has traditionally been seen as a “good thing”.  So much so that most modern programming environments make some provision for document generation and offer style-guidelines for commenting your code.

    I believe, however, that excessive commenting is actually an anti-pattern and should treated with caution, and as far as possible, avoided.  I’m going to illustrate my point with a (somewhat contrived, but in no way unusual or outlandish) example.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;

    namespace XmlDocumentationAntiPattern
    {
        /// <summary>
        /// XmlCommentHell
        /// </summary>
        /// <example>var xmlCommentHell = new XmlCommentHell</example>

        public class XmlCommentHell
        {
            /// <summary>
            /// Description
            /// </summary>
            /// <remarks>String description of XmlCommentHell.</remarks>
            /// <example>instance.Description = "some description";</example>

            public string Description { get; set; }

            /// <summary>
            /// XmlCommentHell ctor
            /// </summary>
            /// <param name="description">Description string</param>

            public XmlCommentHell(string description)
            {
                // Assign description
                Description = description;
            }

            /// <summary>
            /// Gets the description property, but in reverse!
            /// </summary>
            /// <returns>Reversed description</returns>
            public string GetReversedDescription()
            {
                return Description.Reverse().ToString();
            }
        }
    }

    The context of this example is C# but similarly applies in other programming languages.

    If you find yourself engaging in XML comments like the above example, I really believe “you’re doing it wrong”.  Why?

    • Reduced code readability

      The Xml comment clutter in the code above actually reduces it’s usefulness.  The code now takes three times as much screen real estate to maintain and understand.

    • Low signal to noise ratio - too many characters that add no value

      When you’re maintaining systems however large, the act of actually writing code takes time and adds maintenance overhead.  Past the visual unpleasantness the amount of mark up required to document relatively simple methods actually reduces the codes utility.  Only ever type ANYTHING in your IDE if it adds value.

    • Incredibly obvious comments

      There’s a wonderful quote to the tune of “always pretend that the person that’s going to maintain your code is an axe wielding manic who knows where you live”.  I’d like to extend that by adding “so don’t insult their intelligence”.  There are few things I find as frustrating as reading comments that not only add no value but also make me feel like I’ve wasted a tiny piece of my life reading them.  Don’t waste your time or mine.

    • Violating the DRY principle

      The DRY principle is very well agreed upon in software development.  Don’t repeat yourself.  If your XML comments just repeat your method signatures, you’re not only repeating yourself (and thus wasting time) but you’re leaving yourself open to a maintained nightmare as the code changes and the comments are left unchanged leading to confusion and misinformation.

    • Possible misuse – the generation of utterly useless documentation

      When I see projects documented in this way I often joke that someone should run Sandcastle or javaDoc over the code to gain some tangible value in the form of MSDN-esq documentation.  I’m actually wrong.  This is a terrible idea.  Pop quiz time!  What’s worse than tedious documentation that adds no value?  Tonnes of hyperlinked documentation that adds no value!  Think of the poor developer wasting minutes to hours searching for the value in generated documentation before painfully realising that there’s none to be had.

    I occasionally get met with some resistance when I state my opinion on this for a couple of reasons.  People often argue that they do it for the sake of intellisense and IDE tooling, that they do it for generated documentation or just out of habit.  I feel like these are all dangerous reasons.

    Firstly, if you’re generating documentation for documentations sake, you’re either trying to please some form of middle management that doesn’t understand what documentation really exists for (hint: to help developers develop), or you’re wasting time.

    Secondly, if you’re documenting methods for intellisense you’re probably missing the point.  See, intellisense and other IDE self-help mechanisms are very good; they’ll display method signatures and method names with minimal fuss, any extra comments you glue on top of every method will either clutter your GUI, or become ignored white noise, potentially leading to that one important comment going ignored as your developers slowly desensitise themselves to reading the human generated “auto-doc” garbage.

    Finally, if you find that you need to cover your methods in comments because they legitimately don’t make too much sense at a glance then you’re falling foul of a different problem: you have code that’s not legible maintainable and clear.  In this case you’d be far better served by ensuring your methods are named well, take logical parameters and have single clearly defined purposes.  In my experience, whenever somebody says that they need to comment their methods because the method is unclear, they really should be refactoring, not documenting.

    Please DON’T bury your code in comments, just keep it all readable ok?

    How far has A.I. in games really come?

    June 1st, 2009

    There was a piece on Kotaku this weekend looking at the development at AI in games, It specifically set out to ask leading figures what they thought of the development of virtual “beings” in games,

    “Kotaku set out to ask experts in the fields of Hollywood movie magic, theme park creators, robotics experts and AI specialists to answer the question: Do the AI-controlled characters in games qualify as robots or some other form of artificial life. Are those creatures who are at the player’s mercy in Lionhead Studio’s Black & White games truly virtual beings?”

    Inside the article was a rather worrying quote from self declared futurist Thomas Frey, executive director of the DaVinci Institute,

    "In short, our games have indeed evolved into crude life forms," said Frey. "Innovations in the digital world are happening exponentially faster than in the material world, so the digital beings in games will soon become far more lifelike, and will eventually step out of the screens and exist as 3D avatars, interacting with us, much like other people."

    I always get a little upset and slightly concerned when people from places like the DaVinci Institute (from their "about us" page: "Launched in 1997 as a non-profit futurist think tank, the Institute has emerged as a centre of visionary thought, attracting both a national and international following of idea junkies and business leaders alike.") start to talk about things like programming and AI.  I always feel like these kind of think tanks highlight a bit of a problem with the development of new technology.

    All you learn is that "futurist thinkers" actually have no grasp on the practical implementation of the technology.  I’m all for creative thinking and speculation, but somehow implying that very simple game mechanics are going to develop rapidly and with little to no point into some Skynet like Ai is simply absurd. 

    Thankfully the Kotaku article actually asked someone with half a clue (Chris Darken, conference chair for Artificial Intelligence and Interactive Digital Entertainment and an associate professor of computer science at the Naval Postgraduate School) who equated videogame AI to an expert system.

    In all honesty it’s a stretch to even call them that.  The problem domain that your average game AI lives in is so rudimentary small that I’d rather use a phrase closer to "novice system".

    Game AI is nowhere near close to simulating human behaviour, near all "humans" are scripted, and the most simulated behaviour comes from games like Spore or The Sims where the actual behaviour is so generalised (eat/sleep/mate/die) that any nuance of actual behaviour is lost and what results is little more than a general abstraction to suite the purpose of game mechanics; which is utterly perfect for the task at hand, because honestly, developing something more sophisticated would be a huge undertaking with no practical benefit to the game project.

    I’m honestly perplexed as to why some gamers seem to think building “AI” is appropriate for a game.  The term AI is exceptionally misleading and would likely never be appropriate unless you were working on something that wasn’t as strictly defined as your average “regular” game environment.

    Imagine playing a game where a key NPC suddenly decides that they just weren’t into the plot anymore and just stopped playing, a “real AI” of this nature just isn’t for purpose.  I’m not suggesting that what we perceive as “AI” in games doesn’t have room for improvement, far from it, I am however suggesting that games will never lead to developing a “true” AI, simply because it’s not an appropriate avenue to approach game design from.

    People need to realise that good path finding is not the same thing as intelligence, and that they probably wouldn’t actually enjoy a game where the AI attempted to simulate sentience.  It’s work enough dealing with other players in a multiplayer game, let alone having to contend with a simulated intelligence in a single player oriented experience.

    Installing certificates using WiX / Voltive (A Code Sample)

    April 22nd, 2009

    I’ve previously provided a code snippet illustrating how to use WiX (the “new” Windows Installer framework) to install certificates onto the target machine.

    I’ve had some good feedback about the post, but also had a request for a working example (in order to illustrate how the code fragment is used in an Installer.wxs file).

    I’ve cooked up a sample visual studio solution that installs a test certificate (generated using the MSDN makecert certificate example “makecert -sk XYZ -n “CN=XYZ Company” testXYZ.cer “).

    You’ll probably need to re-path references to the WixIISExtension and WixUIExtension in relation to your development environment but hopefully this should help you.

    The previous post is the third most accessed post in my blog (which probably speaks as much for the number of people that read this as it does it’s utility) so this should be useful to someone.

    It’s worth noting that this example installs the junk certificate as a Root CA so you might want to change that or ensure you remove it once you’re done.

    Download the Sample Solution

    Controlling Your Software Development Environment And Release Cycle In An Agile Way

    March 22nd, 2009

    Fear Of Deployment

    When you start working on a new project there are four really important pieces of information you should be aware of regarding the deployment of your software

    1. Are you unafraid of deploying your code to production?
    2. Do you know how your code will react in your production environment with production configuration?
    3. Does your code react in the same way in your production environment as it does in development?
    4. Are you confident in the repeatability of your installation procedure?

    Fear and uncertainty in the deployment of software projects is a common theme I’ve seen across all the companies where I’ve worked and it’s never good for your project.  The fear of deployment is always a good indicator of the confidence your staff have in the project they are working on.

    A project with unknown qualities and suspicious (or known but unreported) bugs often causes an serious fear of deployment.  There’s nothing like the fear of deploying an application and reducing a business critical system to it’s knees.

    I’ve worked on a handful of projects where I’ve taken it upon myself to reign in this deployment fear and chaos, and more often than not, the fear has it’s roots in an uncontrolled development environment.

    Configuring A Development Environment

    I have very few suggestions regarding which tools you should use to develop your software.  I think that you gain the maximum benefit by allowing developers to pick and choose the absolute best tools for the job, be that technically or by preference.  Allowing developers to use their tools of choice makes happy developers.

    By contrast however, the one thing I’m really certain about is that every development environment should start with an absolute base configuration that should not be changed.  This is the first way to solve the eternal “works on my machine” developer problem.  In essence, if all the development environments start off the same, there’s no excuse for a “works on my machine” error.  Not only will this reduce friction between team members, but it’ll speed up the development process because you’re all working to one known configuration.

    Furthermore, the development environments alone shouldn’t share this configuration, they should share the base configuration with your production servers.  A huge percentage of software release issues are relating to the differences between production and test and development and the only way to solve what amounts to human error is to make this discrepancy go away.

    In the current IT landscape where virtualisation is painfully simple (and often free) there are no excuses not to have a development environment (or at the very least development staging virtual machine) that accurately and completely mirrors your production server.  If you’re struggling with licenses of third party software, produce mocks / proxies / approximations.  I can assure you that the time you spend building these glorified testing rigs will work healthily towards eradicating development-to-production errors.

    If It’s Worth Doing Once, It’s Worth Doing Three Times

    Another nasty pattern I see frequently is the development-to-production pattern.  If you’re compiling software on your development machines and releasing it to production, “you’re doing it wrong”.  What’s worse than a “works-on-my-machine” conversation with a colleague?  You got it, a “works-on-my-machine” conversation with a system administrator.  It’s not an excuse.  Stop it.

    You need to start dog-fooding your deployments, and this means environment cloning and real test servers.  I always follow a simple pattern; developer machine -> developer test box -> system-test box -> user acceptance test box -> production.

    In order to do this you’ll need a series of environments (physical or virtual)

    • Developer machine
    • Developer environment
    • Test environment
    • UAT environment
    • Production environment

    That might sound like a lot of hardware but it really isn’t.  Use virtualisation and use it well.  Clone your virtual environments if need be.  Use snapshots and rollbacks to test deployments.  It shouldn’t take any time to administer (and I’ll explain why in the next section) and by the time your code is running on production you’ll have moved it through three different environments.  This is absolutely key to having faith in your deployment procedures.  When you’ve deployed your code so many times that you’re sure it works, in an environment that’s a mirror image of production, production is a far less scary place to be deploying to.

    The key to this technique is that the environments all have very clearly defined purposes.  The developer produces his code changes to a system on his development box.  The code seems to work so it’s deployed to the developer environment and checked into your source control repository.  The development environment should be treated as a place where things could potentially get messed up, but it’s also a good place to host joint resources (a database, some middleware) that all the developers make use of.  If the code works in the developer environment it can be promoted to the test environment for the system tester (or a developer wearing a system testers hat) to test.  Once passed on the test box, the code changes can be promoted to the UAT environment for users to preview coming code releases.  Once these changes have been accepted, the code is released into production.

    Beware of release insanity.  It makes perfect sense to frequently deploy to the developer environment but only occasionally promote code up to test and UAT.  This should fit around your way of working, just be sure to keep the conceptual integrity of the environments.  If you find a bug in test, don’t fix it in the test environment, fix it in development. It’s up to you to ensure you know what code is running across your environments to avoid any nasty surprises.

    One Install To Rule Them All

    At this point you’re probably thinking that this all sounds like a huge amount of effort but it isn’t.  The reason it isn’t is that you should, for the most part, be able to deploy your entire system in about 5 clicks / commands.  Don’t let your deployment mechanism be the week link in your application.  There are tonnes of install technologies on the market, both free and costly, to match whichever platform and tool chain you’re working in.

    I wish I could remember where I read it, but one of the pieces of setup advice I read long ago that always stuck with me is that you should aim for a one-click install.  You might not achieve it, but you should aim to get very very close.

    You need to invest time in configuration management for your application.  Build tools to auto-generate configurations for your deployment environments, build tools to automate setup and deployment.  These tools are worthwhile, because from the moment they’re built, you never need to worry about deploying your software.  They remove risk, and anything that removes risk from software development should be embraced.

    Once you’ve build these tools and installers you’ll find your able to do things that you only thought were possible in large companies with seemingly limitless resources.  You’ll be able to configure nightly builds using a continuous integration package of your choice and have the nightly builds install themselves in your developer environment,  You’ll be confident that at any point you can provide a working copy of your software on request.

    No Fear Of The Unexpected

    The wonderful benefit you get at the end of this process is confidence in your software.

    So you’ve released your system to production, do you then feel the need to slavishly test the system in the live environment? 

    Imagine that urge going away, knowing that by the time the code hits production that any environmental glitches that could manifest have had the ability to fail three times already.  Any amount of cursory testing you could perform at this point should pale in comparison to the testing you did in the test environment and the user testing environment.  If your “at a glance” testing somehow discovers a bug, you’ve done something very very wrong in the lead up to release.  You should be confident in the fact that your code has been run exhaustively and often by the time it’s reached production.

    The world isn’t perfect however, and it’s obviously advisable to a quick audit of the software once installed, just don’t force yourself to repeat your system tests in production (an obvious anti-pattern) and you’ll gain no benefits from doing so.

    If you can devise an automated tool to verify your deployments after install, you’ll be able to round off your sense of deployment-security.

    A Zen-like State Of Satisfaction

    I opened this piece with a few simple questions.

    • Are you unafraid of deploying your code to production?
    • Do you know how your code will react in your production environment with production configuration?
    • Does your code react in the same way in your production environment as it does in development?
    • Are you confident in the repeatability of your installation procedure?

    I can answer yes to all three questions because I understand my environments and I hope you can too.