API access learnings

We recently updated our API, and I wanted to share a couple of things in case you want to build a public API too.

A few years ago we decided Bugfender users could use an API to access the logs they capturing with our tool. We thought some people would build utilities to support their daily work, like integrations with other tools, data mining, or analytics. Some people could even make those tools public, as add-ons to Bugfender.

Building the API was fairly straightforward because we had made the decision to build Bugfender with the Single Page Application architecture early on, which meant there was already a working API, only it was being used exclusively by our frontend. We could reuse that.

However, when you make an API public there are some additional things to take into account:

• Authentication and authorization: you need to identify your users and applications, and decide which permissions each will have. It's likely your application already has a permission system, but now adding applications into the mix you might need to review it. We chose OAuth 2.0 because it fit well with our application model and the usage we were expecting of the API (but we learned something, see later!).
• Rate limits: opening the API will likely cause additional work for your backend. Are you prepared for that? Plan ahead and set limits to prevent accidental or intentional overload of your servers. We decided to use the draft-ietf-httpapi-ratelimit-headers RFC draft for that, because there are many options out there, all different, so we chose to go with a standard one (or at least we feel has the potential).
• Documentation: you need to describe your endpoints and provide examples for your users to know how to use your API. We chose OpenAPI (formerly known as Swagger) to describe the endpoints, which lets you automatically generate an API documentation portal, can be used to autogenerate code for many programming languages, and integrates with tools like Postman.

The theory

When we launched the API, we were expecting people would build their own tools on top of Bugfender for data mining, integrations with 3rd party platforms, and whatnot.

Therefore, we built an API access model based on "apps" that connect to Bugfender. Some apps would be for personal use, some for use within a specific organization or team, and some would be public for use by any Bugfender user.

This is also why we chose OAuth 2.0, because it's specifically built for this use case.

The reality

When we launched API access, we purposefully didn't build a UI for it. We asked people to contact us if they wanted to use it.

In hindsight, that was a great decision because we can learn how people want to use the API and we can provide them with pointers on how to get started. We also can find opportunities for improvement both in the API and the documentation.

So, the most common OAuth 2.0 use cases work like this, and this is what we built:

• A developer creates an "app" that works for any Bugfender user (OAuth 2.0 calls this a "client")
• A Bugfender user installs this "app" on their account and authorizes it
• Now the "app" has access to the user's data on Bugfender

However, most of the users requesting API access were not interested in building an "app". They just wanted to access their own logs and make queries or export them to another service. So they were expecting a flow like this:

• They request an access token
• They use that access token in their app
• FIN

After a bit of investigation, personal access tokens were not what they were looking for, either, because they might not want the application to have unlimited access to their account.

The learning

The piece in the puzzle we were missing were "service accounts". Service accounts represent an application in Bugfender, have their own credentials to access it, and can be given permissions like any other user.

Luckily, OAuth 2.0 client credentials grant is exactly for this use case! So we could easily adapt to this use case. Happy ending.

In summary

• Think how your users will want to use your application (theory)
• Once you build it, talk to your users and double-check that what you built is what they needed (reality)
• Learn and iterate