Dominic Baier, who gave a great presentation on the security pipeline in WebAPI, has published his slides here and his source code here. Not going to say much else, but this is going to be a great help for us in getting our API secured.
Tag Archives: WebAPI
DevWeek Day 2 summary
So, four sessions today;
Modern JavaScript — Scott Allen. A nice overview of JavaScript and the right sort of patterns to use to build larger, more maintainable systems. Covered basics like function scope and closures, and then delved into more advanced topics like IIFEs (immediately invoking function expressions) and the revealing module pattern.
So the nice outcome here for me is that I can go back to work and say that, yes, these things are good and worth doing — an expert at DevWeek told us. And I know that sounds kinda funny, but it means that it becomes easier to build a consensus amongst a team. Instead of saying ‘Steve’s an opinionated developer and has his way of doing things, but I’ve got my way,’ members of a team can feel more like they are following an independent best practice.
Temporal data in SQL Server — Dejan Sarka. This was a really interesting one. Dejan Sarka is a man who knows his stuff. Check out the book list. He wrote the Microsoft books on the internals of SQL Server 2005 and 2008.
Dejan Sarka’s presentation was about how to deal with timed data in SQL Server — a server which doesn’t include a timespan-like datatype. What he outlined was a system for building systems with really solid time-based support. The tl:dr; is this; identify a ‘time quantum’ like one day, and give each day an integer value. Then define a CLR datatype with a start-day and an end-day, forming a scalar value that can be used to represent a time range and which can be joined, unioned, etc — all the operations you need for querying. Now you can include ‘Interval’ columns in your database and use the interval as part of your key, for joins, etc.
Patterns and techniques for securing REST/HTTP-based services and (ASP.NET) Web APIs — Dominick Baier. The surprise big hit of the conference for me. This one was held in one of the big cinemas, and it got a lot of interest. It seems that the ASP.NET stack doesn’t have that strong a security story, and Dominick gave a nice presentation of how you might successfully secure a web service using OAuth2 and claims-based authentication. One to use, I think.
RESTful architectures on the Microsoft stack — Hadi Hariri. This one was interesting. Before I went, I thought that there was a danger that this would be a repeat of material from Christian Weyer’s talk yesterday and Dominick Baier’s talk today, but it took a different tack and I’m glad it did. The first half of the talk helped explain the difference between REST, which is a very particular design pattern for designing HTTP-based APIs, and non-REST HTTP APIs. The short thing here is that if you’re not sure if you’re doing REST, you aren’t. REST is very particular, has a fixed number of principles that need to be followed, and if you haven’t studied them and made sure you’re doing them, then you aren’t doing them.
The second half of the talk went into how you might go about implementing true REST on the parallel Microsoft stacks of ASP.NET MVC, and ASP.NET Web API — two products that are remarkably similar superficially, but which need their own different handling to become RESTful.
Now, the question is, do you want to be RESTful? Is it worth the effort, if you’re already doing an HTTP API? That question you can only answer for yourself, but it’s worth noting that it’s not free — it’s a re-write you need to do to your existing HTTP API.
Separation between Data Transfer Objects and Entities, and Automapper to the rescue
One question we had a little while ago. When you return objects from a controller, should they be the same objects you get out of your data layer?
That is, if your data layer has an object like this;
public class Project
{
public int ProjectId { get; set; }
public string Name { get; set; }
}
Should you return this direct from Controllers, like so;
public class ProjectController: ApiController
{
public Project Get(int id)
{
return this.db.GetProject(id);
}
}
And the answer Christian Weyer gave in his talk was no — use a Data Transfer Object instead. This is a pattern which has you translate from the database entity down into a very plain object designed exclusively for transferring across the wire. We’d started doing this and named them transmission objects — we would have a DB.Project class designed for inside the system, and a TX.Project class designed for transmitting over the wire, converting to JSON, etc. We had agonised a bit over the added complexity, but it’s good to see that it’s a recognised good pattern, and I’ll sleep better tonight knowing we’re doing the same as the thought-leaders.
Where his solution was much tighter than ours is that he’d used Automapper — a library which allows for very straightforward mappings between comparable types. So if DB.Project and TX.Project share similarly-named members, you end up being able to translate one to the other using calls like;
DB.Project dbProject = ...;
TX.Project txProject = dbProject.Map(); // auto-mapper kicks in here
As an adjunct to that — there is also a JsonValue class which is designed for converting to JSON. Might be worth investigating in more detail…
Enabling OData through WebAPI
Here’s a very quick bit of code which Christain Weyer’s talk leads me to believe gives you easy OData support. OData support is enabled by (1) calling .EnableQuerySupport() on the HttpConfiguration object you get in global.asax.cs, and (2) Returning IQueryable<T> from the .Get() method of your ApiController subclass, and (3) marking that method up as [Queryable]. Something like;
// in Global.asax.cs
config.EnableQuerySupport();
// in your Api Controller
class ToDoController: ApiController
{
[Queryable]
public IQueryable Get()
{
return this.repo.Todos;
}
}
Not sure about the details, or whether this is perfect code — it’s copied out of a desperate scribbling session during the talk — but this might be enough for someone to start searching down the right track.
Hosting options for WebAPI
So you’ll probably be using IIS to run your WebAPI apps, but you really don’t need to. WebAPI ships with a class which starts a web server handling WebAPI requests, in System.Web.Http.SelfHost..dll — HttpSelfHostServer.
Now, this is really interesting, for two reasons.
1) You can spin up a new server, run some tests against it, and shut it down. That means you can test all your HTTP traffic without having to set up a development server. I struggled with this all last week — how do you write a test to prove all your HTTP code is working, but not require all your developers to set up a web server like http://localhost:3000/myapp just so that you’ve got somewhere to act as an endpoint for calls? This short-lived server becomes your endpoint for the lifecycle of the tests.
2) It allows you to create a host outside IIS, on a non-server machine. For instance, imagine you were writing a document search application. It has a front end written in Windows Forms, and the backend is written in WebAPI. When you start the app, it starts an HTTP server at http://localhost:1234/mydocumentsearch. The front end loads its data by making GET requests to that server, and saves changes by POSTS and PUTS and DELETES. Now let’s say you want to search the corporate network using the same app. It would be simple to change the URI to http://corporateserver/documentsearch and now the same app, with no code changes, has become a corporate intranet-based application. Change it again to http://www.internetdocuments.com/search and it’s now an internet-enabled cloud-whatsit buzzword-filled bundle of awesome. And you don’t need to recompile to switch between these seriously different modes.
This approach says — code for HTTP first, and in fact only code for HTTP. You don’t need any other kind of public API. You don’t need an API that you compile into your app, but instead you really code in separate tiers communicating through HTTP. You can now code everything as if it were cloud-enabled, and then just switch to the cloud when you’re ready.
For me, it’s the testing scenario that’ll be most useful in the short term, but the other one is making me think about API design. After all, there are now even database engines (like CouchDb) which only offer HTTP/JSON interfaces. HTTP has become a universal communication mechanism. Will this become the widespread pattern of the future?
Prerelease Feature — Automatically-generated help
Search NuGet for the WebAPI test client (Search `testclient` and make sure you allow prerelease software) and you’ll add a really rather sexy feature to your site. It’s an automatic help generator for your site, so that if you go to `http://myapp/help` you get an HTML site allowing you to see all your URLs, their formats, what methods and parameters they take. It also allows you to execute the methods, giving you an in-browser form for building requests. In a recent project, I built something similar — I recognised the need — but this is just miles ahead of mine.
This is something I’ve only seen in a demo, but it looks like a very sweet addition to your site, at least in debug mode. It means you should be able to write a method then see it work in the browser without having to build a ‘proper’ client for it. Unlike the familiar testing pattern where you new up a controller and execute its methods, this tests the whole architecture of your site, including all the serious bits you need to test about model binding, error handling, etc.
Content Negotiation And Response Types
Type an address into the browser and the HTTP request from the browser contains a header like this;
HTTP GET /foo/bar
Accept: text/html,application/xhtml+xml,...
and this `Accept` header is a signal to the application of the kind of content that it’s expecting back. If the browser responds with a web page, it might have a response header like so;
Content-Type: text/html
<!DOCTYPE HTML>
<html lang="en">
<...
So there’s a conversation going on here; the client says ‘I’d like the HTML at /foo/bar’ and the server says ‘Here’s a document!’ That’s the basic pattern repeated over and over during normal web browsing. But this system is fairly flexible. What if /foo/bar represents a resource and you don’t want HTML, but want JSON instead? You can request this;
HTTP GET /foo/bar
Accept: application/json
And if the server can respond with JSON, it will. This is the sort of thing jQuery does when you write
$.getJSON(...);
This is really interesting, because it means that the same URL, with the same verb, can return two totally different representations. This process, this interaction between the `Accept` and `Content-Type` headers, is known as *content negotiation*.
So this little mechanism has some interesting applications. Imagine you are doing a project management app, and you decide to use a URL like this;
/api/Projects/1
This URL represents the resource of ‘Project #1’. But it doesn’t imply anything about the form of the response. If we request
GET /api/Projects/1 HTTP/1.0
Accept: application/json
Then we might expect the response body to contain
{
"ProjectId": 1,
"Name", "Adveriting Campaign"
}
But if we make this request;
GET /api/Projects/1 HTTP/1.0
Accept: application/xml
Then we might expect to recieve this body;
<Project>
<ProjectId>1</ProjectId>
<Name>Advertising Campaign</Name>
</Project>
Now, this is what you see in the OData specification — the same resource, at the same URI, with two different representations. And you’re not just limited to two. Why not return an HTML description if the request asks for `text/html`, or YAML if that’s what the client asks for?
WebAPI lets you add new representations to your system. In your Global.asax, you’ll have an `HttpConfiguration` instance floating around — it’s the object you add routes to. But it also has a member called `Formatters` which is a list of message formatters. The default list includes formatters for JSON, XML, and Form Url Encoding. The first two are for Web/HTTP/REST-style apps; the third is to interpret HTML Form posts. These objects do bi-directional translation; so that if your controller method looks like this;
public Project Get(int id)
{
... do work here...
}
It’s the formatter which determines how the `Project` instance returned by your method is converted into the JSON or XML returned in your HTTP response. Add a new formatter, and your system is now capable of returning a new representation without you having to change your controller method at all. Sweet!
And this also works the other way. The same formatter will take an HTTP request and turn it back into object instances, so that something like this;
public void Put(Project project)
{
... update your project here ...
}
Has the same process — the `Project` instance passed as a parameter has been constructed by the object in config.Formatters. This process is known as Model binding. if you want to look into it further. An example — binding using the Google protobuf format — exists on gitHub in the WebAPIContrib project — https://WebAPIContrib/WebAPIContrib.
First-class HTTP programming is available in WebAPI
Typically, a WebAPI method might look like this;
public Project Get(int id)
{
return this.repo.GetProject(id);
}
And what we’re seeing here is a method that returns a Project object — or rather, a Project object that is later converted into a JSON object…
So it may be that you need to do a whole lot more — say, in sniffing HTTP headers for a particular purpose, or returning more interesting things in the response. So here’s how you would alter the above method;
public HttpResponseMessage Get(HttpRequestMessage request)
{
if (request.Headers[...])
...
}
And now you’re able to do anything you like with the request — you’ve got full control if you need it. It’s the ‘escape hatch’ out of the sometimes-too-helpful set of conventions. Not something for everyday, but something to use when all else fails, I think.
Lightweight Architeture for everyone with ASP.NET Web API — Christian Weyer
This was by far the most pragmatically useful of the sessions so far. We’ve just started using WebAPI and it’s clear that there’s loads more to it than we’re using. Of vital importance to me was an understanding of just how the magic works.
It seems to me that when you get into a new technology, it’s always pretty fraught. Or it should be, because you should be always asking yourself, ‘Am I doing this the right way? Is this the way that a pro would do it, or is it the way an amateur hacker would do it?’ Christian Weyer helped point me down the right path. Luckily, much of the things we’ve been doing are right, and the rest are easy to address.
First up, then — what is WebAPI and why do we care?
Well, WebAPI is a way to build HTTP APIs for your services. The HTTP API is designed to live between the client app — be it web page, console app, Windows Forms app, or iOS app — and provide all the business logic and database logic.
There are a raft of tips, so rather than smashing them all into one big blog post, I’m going to post things one at a time. I’ll tag them all `WebAPI` and `DevWeek` so you can search them out more easily.
Steve-Driven Development 