API Authentication

When building a publicly-available API on the Internet, we usually don't want to allow anyone in the whole wide world to use it. We may limit certain parts, or entire API, only to known users. This means our API needs to distinguish between known and anonymous users and allow or deny some actions based on who's asking.

This is where authentication and authorization come in. These two terms are closely related but are not the same thing. Authentication is the process of figuring out who the client is. Authorization is figuring out whether the client is allowed to do something. The difference is subtle but important. Here, we cover the basics of authentication.

Authentication

Authentication comes in may forms. A simplest example is password auth: a login page asking you for username and password. A more involved example is logging in with Google using something called OAuth2. In this case, the user is first directed to Google to log in there, and then Google tells the server who the user is.

In all cases, the goal is authentication is to determine who's calling. The server has a database of users it knows about (this database can be either local or hosted elsewhere). When the client request comes in, the client's credentials (such as username and password) are checked for a match in the database of users. If a valid user record is found, the client has been authenticated as that user. If there's no valid user record for the provided credentials, or none were provided at all, the authentication fails.

Credentials

The credentials used for authentication can be many things. Most often they are email and password, or username and password pair. They sometimes involve time-based one-time password (discussed later when we talk about multi-factor authentication) or additional information.

Another type of credential is a bearer token. This token is usually given after successful authentication with client credetials ("user login") and serves make it easier and faster to authenticate subsequent requests ("logged-in requests"), and to avoid having to send user credentials on each request.

Local auth

For standalone services, users are usually authenticated using a local database. Here "local" means the database of the service in question, not where the database physically resides (it may be a managed cloud database and still be considered a "local" database for the service). The database has a users table/collection, and authentication is done by matching the user credentials against the database.

Here's the typical flow when using local auth:

1. User supplies their credentials (for example, username and password)
2. The service looks up the user record in their local database, matching the provided username
3. The service hashes the provided password and compares it to the hashed password in the database
4. If user record is found and the hashed password matches, the service determines this is a valid user

For security reasons, the provided password is never stored in the database as-is (in "plaintext"). When creating the user or changing user's password, the service always hashes the password before storing it to the database. When checking the password during authentication, the service hashes the provided password and checks that the hashes are the same.

Hashing is somewhat similar to encrypting, but with one important difference: it's one way. This means once you hash something, you can never go back and calculate the original ("plaintext") password out of its hash. In contrast, with encryption, if you have the encryption key, you can get the original information ("decrypt"). Hashing has another useful property, in that the same original ("plaintext") is always hashed to the same output ("hash").

This means you can store a password hash in the database. If anyone ever breaks in your system and steals the user database, they won't get the passwords because hashes can't be reversed. But in normal situations, when your user tries to authenticate, you can easily create a hash of the password they provided right then, and compare it with what you have stored. If they match, you know the original passwords match as well.

An important practical note: although hashes can't be reversed, the attackers can (and do) try to go through all possible password combinations, hash them, and check against the stolen database to see if there's a match. For this reason, hash for each user is done in a slightly different way (called salting) so that attackers can't build a huge table with all possible hashes and just compare them. Instead, they must do this separately for each user.

For the same reason, password hashing algoritms (such as scrypt) are different from message digest algorithms (such as sha256). In both cases, they create a one-way "hash" of the data. But while message digest algorithms need to be as fast as possible, hashing algorithms must be as slow as possible, to make it harder for the attackers to check all possible combination.

And lastly, this is why it's important that passwords are complex, with upper- and lower-case letters, digits and symbols, and/or that they are of sufficient length. Rule of thumb is, the less varied in types of characters, the longer the password needs to be.

Using 3rd-party auth services

Most web frameworks have support for local authentication built-in, but there's still a risk of not implementing it correctly, or having a security breach. For that reason, many teams prefer using a 3rd-party cloud service for authentication. Additionally, 3rd-party services usually have additional user-management tools (user groups, policies, user management pages) that aren't available out-of-the-box if rolling your own auth solution.

Note that using a 3rd-party auth services, such as Auth0 or Okta is not the same as using a social login like Google or Facebook. With Auth0 and Okta, from the point of your users, they still create accounts on your service. They don't know if you're using Auth0, Okta, or a similar service.

A typical flow when using a 3rd-party auth service with password-based authentication:

1. User visits your login page
2. Visitor enters username and password, or gets redirected to the 3rd-party provider to enter username and password there
3. User credentials are verified:
   • If visitor entered username/password locally, the service uses the 3rd-party API to send the credentials, check for validity and get the user data
   • If visitor entered the credentials on the 3rd-party provider's site, they are redirected back with to the service with the information about the logged-in user

Although historically the cloud services supported the use where username+password is entered on your site, the best practice today is to enter the user credentials on the 3rd-party site and use OAuth2 protocol to get the user data. This completely avoids having to deal with username and password on your side and thus minimizes security risks.

Using online identity providers

At first glance, using 3rd-party auth services like Auth0 or Okta, and identity providers like Google, Facebook, Twitter or GitHub, seems similar. In both cases, users are redirected to the separate provider page, get authenticated there, then get back to your site and you get the info about them.

The big difference is that an account at an online identity provider like Google or GitHub is a real, separate, existing account that the user has. It's just that the user has decided to use that identity or account to authenticate with you as well. In contrast, with services like Auth0 or Okta, the user creates account - a separate identity - for your service, but it's just hosted in che cloud.

While this is an important difference, the underlying implementation might be very similar. In practice, all of these services use some form of OAuth for authentication.

A typical flow when using online identity provider:

1. User visits your login page, chooses an online identity provider to log in with and is redirected there
2. User logs in to their account on the online identity provider and explicitly gives your service the right to access their profile at the provider
3. User is redirected back to your service with a "grant" token
4. Your service uses the grant token to check the user identity using the online identity provider API

This is very similar to recommended flows when using a 3rd-party auth service, to avoid the user having to enter their credentials at your site. The difference is that in a 3rd-party auth service, the user doesn't need to grant your service the right to access their profile, since that profile is tied to your service. With Google or Facebook, the user must explictly allow you to access their profile info.

Usually this access is only for basic profile information like email, name, and user picture, as the only thing your service needs is to verify that the user is valid. Additionaly, that profile information is usually read-only, so you can't modify the user profile data at the online provider - you can just read it.

OAuth2, OpenID and OIDC

Using 3rd-party or online identity providers, you'll hear a lot about OAuth2, OpenID and OIDC. Sometimes, these terms are used imprecisela and almost interchangibly, causing confusion.

OAuth2 is a standard protocol for authorization on the web, and is now in its second version (ie. OAuth2 or OAuth 2.0). While we haven't discussed authorization in depth, note that OAuth2 by itself is not an authentication protocol. If you want to dig deep into kinds of problems arising if you tried it as an authentication protocol, here's a fun read. Instead, it's a building block used by most providers to implement the authentication.

OpenID is a standard for a decentralized authentication protocol. The standard went through several iterations that worked in slightly different ways, and is currently in the third major version, called OpenID Connect.

OpenID Connect or OIDC, is OpenID designed to work well with OAuth2. It builds on top of OAuth2 building blocks to provide secure and standard way of authenticating users. However, since many of the most popular online identity providers started before OIDC was standardized, they usually have slightly different implementations of something like OIDC on top of OAuth2. Some providers support multiple authentication protocols, including OIDC.

When creating a new service and using an online identity provider, ideally you'd want to aim to use OpenID connect since it should work the same way across multiple providers. But this will depend on whether the particular provider(s) you want to integrate with support that as an option.

SSO and SAML

Related terms used in more corporate and enterprise environments, are Single Sign-On (SSO) and SAML.

SSO describes authentication flows where a user signs on once to their account and then can use all available services without needing to sign on again. If this sounds very much like using an online identity provider, it's because it is, but in a different context and from a different point of view.

SSO is usually used in corporate context where a member of the organization (user) signs on once on their corporate account. It then accesses different web-based tools without having to log-in again in each of them. Here's an example to make it clearer:

User Jane Doe works in Acme Corp. Each morning she starts her work day by logging into Acme Corp. employee sign-on system. Then she logs into tools like Slack messaging platform, Miro online whiteboard or GitHub. Each of these services, noticing she's a member of the organization using SSO, checks Acme Corps' SSO system to authenticate her, and she's immediately "in" without having to log in on each of these services.

How does Slack know who to check with when authenticating Jane? That's what SAML is for. SAML is XML-based protocol in which each service (like Slack or Miro) knows about each of the organizations (like Acme Corp) they support. When Slack sees from Jane's e-mail address she's a member of Acme Corp, it uses its SAML configuration to figure out who needs to be asked about her. SAML uses X.509 certificates to make sure the communication between Slack and Acme Corp's SAML servers is secure.