Using the Xero OAuth 2.0 API from a .NET Core console application

[Edited on 5 August 2021 with some pseudocode at the end.]

This is not a post about SQL Server, but I need to write about something that I’ve been struggling with for a number of weeks so that other people don’t have to.

A few months ago Xero announced that they would be moving their API authentication mechanism to OAuth 2.0. This is good news for a number of reasons. For developers and maintainers of apps that connect to Xero, we are no longer restricted to two private apps, nor do we need to worry about certificate files that expire unexpectedly. From Xero’s side this makes their API management more centralized.

The basic principle of OAuth 2.0 is that you have to log in somewhere, at which point you are issued an access token. The access token will have an expiration date, but you can extend that expiration by refreshing the access token using a refresh token which is provided at the same time.

In some implementations of OAuth 2.0 you can log in automatically without human interaction (i.e. clicking a button), but in Xero’s case the first time you generate tokens it has to be done through a web browser, and you have to click on a button. I’ve seen this with other implementations as well, and for the record I am not a fan.

With Xero’s new OAuth 2.0 authentication procedure you get an access token with a 60-day validity and a 30-minute timeout between refreshes. That means if you can automatically refresh the token within 60 days, you don’t need to keep logging in with the browser. This is a reasonable compromise given that access tokens tend to have a 30-day expiration, but of course Xero is in the accounting business and some months have 31 days.

For one of my companies I maintain a console application — recently ported from .NET Framework to .NET Core — that generates invoices in Xero once a month with no human interaction. As long as I can get an initial set of valid tokens into the app when I first run it, it will be able to run every month as needed without any further interaction, provided that I request the “offline access” policy when I generate the tokens (which is part of the authorization “scope”).

Unfortunately there is no single piece of documentation on how to access and use Xero’s new API if you’re using a .NET Core console application. All of their .NET code samples are based on ASP.NET Core, which assumes a significant number of things that just don’t apply to the console environment.

Here’s a quick primer to explain how I got it to work (with a little help from my friend and colleague Werner):

Step 1: Download xoauth

You need to get xoauth from the Xero repository on GitHub. When I first tried running it on my MacBook it didn’t work and I kept getting TLS timeout errors. One of the things my colleague suggested was using the version from Homebrew instead, which ended up working a lot better.

Step 2: Create an App on the Xero Developer portal

Visit My Apps and add a new app. I followed the instructions to create a web app (as opposed to a mobile or desktop app) as my console application is only being run on one machine, so I am less worried about the security of the client secret. If you’re building a console app that your customers will use, you must choose the mobile or desktop app with PKCE instead.

For the callback URL I used http://localhost:8080/callback because that’s what xoauth is expecting. Remember that you will use xoauth to generate the initial tokens that are used by your application, and thereafter the application itself will refresh the tokens as needed.

Step 3: Configure xoauth to use the new app

The instructions from the xoauth page are fairly straightforward. I chose Authorisation code when configuring my app using xoauth setup. One of the things my colleague noted is that there is a bug with the scope entry. Instead of having a space-delimited list of scope requests, you need to type them one by one, followed by the letter d to indicate that you’re done. Remember here to ask for offline_access.

You can see the full documentation for the scopes you need from the developer documentation.

Step 4: Fetch tokens for the new app using xoauth

When I used xoauth connect my browser opened, I signed into my organization and clicked on the button to agree to the request. Once that happened both the browser and xoauth showed me the tokens I needed for the initial connection, which I could then copy and paste into a file.

Note: You can also refresh tokens from the command line using xoauth token <appname> --refresh

Step 5: Profit

This is where the process broke down for me in two separate ways, and in both cases by a lack of documentation.

For starters, if you use the sample application provided by Xero, it’s a web application. If you follow the process flow and run the app you get a web interface, and when you click through and get the tokens a file is persisted to your drive containing a token object in JSON format. On the other hand, if you don’t want to build a web app and just want to retrofit a console app that already exists, there is no token file to modify with the output from xoauth.

I couldn’t find a sample token file anywhere in the Xero documentation, so I had to mock up one of my own using information from the API in the Visual Studio object browser and the XeroToken entity. This is how it looks (I’ve redacted the GUIDs and tokens for obvious reasons):

{
	"Tenants": [
		{
			"id": "00000000-0000-0000-0000-000000000000",
			"authEventId": "00000000-0000-0000-0000-000000000000",
			"TenantId": "00000000-0000-0000-0000-000000000000",
			"TenantType": "ORGANISATION",
			"TenantName": "<redacted>",
			"CreatedDateUtc": "2020-08-15T02:44:51.380643",
			"UpdatedDateUtc": "2020-08-15T02:44:51.38312"
		}
	],
	"AccessToken": "<redacted>",
	"RefreshToken": "<redacted>",
	"IdToken": "<redacted>",
	"ExpiresAtUtc": "2020-08-15T21:24:16.679155Z"
}

If you visit https://api.xero.com/connections to get your list of tenants, the JSON is returned in traditional JSON format known as camel case, meaning that the first letter in the JSON object name is lower case (for example tenantId, tenantType, tenantName, and so on). However the API deserializer expects most of the names in Pascal case, meaning that the first letter is capitalized (see the sample above). That was fun to troubleshoot. A few JSON decorators in the API deserializer code wouldn’t go amiss is all I’m saying.

Once you have faked up a token file and placed it with your code, you now need an HttpClient to call the API to refresh tokens using the RefreshAccessTokenAsync() method.

This was the second major problem I faced, except now it was with .NET Core specifically. Microsoft really wants you to use IHttpClientFactory in your code, which is extreme overkill for doing a simple HTTP request, especially if you’re using a console application. The alternative is to use HttpClient, but that has a known issue with socket exhaustion because people don’t use it correctly. In the end I followed the instructions to use a SocketsHttpHandler with a PooledConnectionLifetime of 20 minutes (more here). This allows the use of a shared HttpClient across the lifetime of the app.

Finally

Once you have all those pieces together then it works as expected.

Getting to that point took me many hours of stress and swearing at my computer, on Twitter, in emails to the support team, and in a survey that I coincidentally received that week. Honestly it shouldn’t have. I’m still not happy, but hopefully this will help someone in the future avoid all of the pain and annoyance of poorly-conceived documentation that makes assumptions about the developer.

And now for some code

The big issue for me, once I used xoauth to generate the authorization code, was how to run what is effectively web stuff inside of the console app I have.

In Step 5 above, I created a dummy token and copied and pasted the values from xoauth into there, and saved that to disk (you have 30 minutes before it expires, so it took a couple of attempts to get it working through the debugger).

The TokenUtilities class is taken from their code sample on GitHub. The magic was really in getting the HttpClient working properly with the token and refreshing correctly, which you need to do at the start of the app.

Here’s pseudocode for the refresh:

public async Task<XeroOAuth2Token> RefreshToken()
{
     var config = new XeroConfiguration
     {
           AppName = "my-app", // the name you chose in xoauth
           ClientId = <ClientID from xoauth>,
           ClientSecret = <ClientSecret from xoauth>,
           Scope = <ScopeList from xoauth>,
           State = "someuniquestring"
     };

     var handler = new SocketsHttpHandler();
     var httpClient = new HttpClient(handler, false);

     var xeroToken = TokenUtilities.GetStoredToken();
     var utcTimeNow = DateTime.UtcNow;

     if (utcTimeNow > xeroToken.ExpiresAtUtc)
     {
           m_logger.Debug("Token is being refreshed...");
           var client = new XeroClient(m_config, httpClient);
           xeroToken = (XeroOAuth2Token)await client.RefreshAccessTokenAsync(xeroToken);
           TokenUtilities.StoreToken(xeroToken);
     }
     else
     {
           m_logger.Debug("Token does not require refresh...");
     }

     return xeroToken;
}

Here’s pseudocode for a sample use case of getting Xero customers and adding them to a generic list:

public async Task ListXeroCustomers()
{
     XeroOAuth2Token xeroToken;

     try
     {
           m_logger.Information("Connecting to Xero API ...");
           xeroToken = await m_connectionHelper.RefreshToken();
     }
     catch
     {
           m_logger.Error("Could not connect to the Xero API.");
           return;
     }

     var accessToken = xeroToken.AccessToken;
     var xeroTenantId = xeroToken.Tenants[0].TenantId;
     var tenantId = $"{xeroTenantId}";

     var instance = new AccountingApi();

     var org = (await instance.GetOrganisationsAsync(accessToken, tenantId))._Organisations.FirstOrDefault(x => x.Name.StartsWith("Company Name Goes Here", StringComparison.InvariantCultureIgnoreCase));

     m_logger.Information("Organisation Name: {0}", org.Name);

     var xeroCustomers = new List<XeroCustomerEntity>();
     XeroCustomerEntity xeroCustomer;
     IEnumerable<Contact> contacts;
     var pageId = 1;

     do
     {
           contacts = (await instance.GetContactsAsync(accessToken, tenantId, where: "AccountNumber!=null", order: "AccountNumber", page: pageId))._Contacts;

           foreach (var contact in contacts)
           {
                 m_logger.Debug($"Account Number: {contact.AccountNumber} -- Name: {contact.Name}");
                 xeroCustomer = new XeroCustomerEntity
                 {
                       AccountNumber = contact.AccountNumber,
                       ClientID = contact.ContactID.Value
                 };
                 xeroCustomers.Add(xeroCustomer);
           }

           pageId++;
     }
     while (contacts.Count() == 100);

     m_logger.Information($"Retrieved {xeroCustomers.Count} customer(s).");
}

Share your thoughts in the comments below.